February 23, 2006
Lightweight Programmatic Tweening
At my old studio, ioResearch, we were often coming up with custom Flash components to make our lives easier. Jon Williams, our engineer, even created a library of them and distributed them called ioLib. His library solved one very critical need for us, programmatic animation. His means were innovative and exhaustive and nearly every project we built used the library in some way. Today there are many infamous tweening libraries out there. There have even been competitions between them to prove which is best. I've used most of them and they can be a Godsend, but they all have one big problem - overhead. They are HUGE! Well, they are huge when size matters, and in mobile, it does.
Which brings me to my post. I have always longed for an easy way to do programmatic tweens without all the overhead of a huge included AS library. At the time, Jon's library clocked in just under 100k. The included MX tweening class, with all it's transitions and such is even more. In mobile, my whole app is often less than 100k. So to include a library like this is just a bad idea.
Now I'm not talking Eric Natzke like tweens here. We're not traveling along a crazy sine wave, we're just going from A to B. But that happens a lot. A lot a lot. And it's necessary to have a little help. To that end, I'd like to share with you a snippet of code I now use all the time. It's quite simple. So simple in fact you might think it's not worth mentioning. But there is a shred of cleverness in it, so I'll share.
Let's start with a few lines of code that will do a simple tween with easing for us.
mc.onEnterFrame = function() {
this._y += (newPos - this._y) * .5;
}
This could do the trick nicely. However, one of the big problems with programmatic tweening is knowing when the tween is complete. This enterFrame would keep going forever, even after the movement had stopped, and we can't have that. You would think this could be solved easily with something like..
mc.onEnterFrame = function() {
this._y += (newPos - this._y) * .5;
if (this._y == newPos) {
delete this.onEnterFrame;
}
}
But you can't. Why? Because when you add the easing bit, the values you start to get are fractions, and because of the floatng point math they may not be exactly equal in the end. Rounding using Math.round, Math.floor, or Math.ceil are all unreliable as well, since you may round a number up or down and miss the end value by rounding in the wrong direction. And while you could compensate with some conditionals, it was never as simple and elegant as I wanted.
This bugged me for a long time as I sought an easy way to identify when a tween was complete. I stared at my trace output seeing those values never exactly lining up and it killed me. Often the values would get close to their destination but never reach it. Shooting for zero, I might get something like this..
10 7 4 3 2 1 0.5 0.2 0.1111 0.1111 0.1111 ...
I finally saw something in those trace values that I thought could help. It didn't matter so much that I reached the end value as much as I knew when the tween was done. Well at the end of these tweens, you may notice that even if we don't reach THE end value we reach SOME end value, and when we get there, we stay there. The final value begins to repeat, and that is something we can rely on. With this mind, we can grab the value before we set it, then set it, and then compare it to the stored value. If the two are the same, we know the value isn't changing anymore.
mc.onEnterFrame = function() {
var t = this._y
this._y += (newPos - this._y) * .5;
if (t == this._y) {
delete this.onEnterFrame;
}
}
This way never fails. Our tweens always complete and we always clean up the event loop when we are done. Simple and sweet.
Given this basic recipe I may make a couple tweaks to solve some incidental issues. Since we are checking for repetition in lieu of completion, the destination of our clip may not be exactly what we want. We may have sent our clip to a location of 100, and it only made it to 99. So we put any cleanup code we need in the conditional. Also, in mobile specifically, when we set the position of movie clips to fraction amounts, it often looks bad. Rounding to whole pixel amounts for this purpose works quite well though.
mc.onEnterFrame = function() {
var t = this._y
this._y += Math.round((newPos - this._y) * .5);
if (t == this._y) {
this._y = newPos;
delete this.onEnterFrame;
}
}
The beauty of this technique is in its brevity. It's a tiny bit of code that always hits its mark. And in Flash, that's a wonderful thing.
Josh. XD.
Posted by julm at 10:55 AM | Comments (10)
November 14, 2005
Optimizing Flash Lite Content
Flash on mobile devices is a different breed. We have been talking at some length about the different aspects of designing for mobile lately, but authoring for mobile is unique as well. With that in mind, we have been putting together some documents on mobile authoring best practices - how to eek out every last drop of performance from Flash on your device. I'll share some of our insights with you now.
Of course there are some universal truths to optimizing Flash content: don't animate huge complex artwork; don't tween a zillion things simultaneously; go easy on the transparency. And while on the desktop Flash 8 (finally!) solves these performance issues, on devices we just don't have the oomph we need to kick out the crazy Hollywood effects, yet. Furthermore, some devices perform better than others, sometimes dramatically. And because mobile authoring often requires that we publish to many different devices, it may be the case that we have to author for the lowest common denominator. That being said, a byte here and there may make all the difference.
One last point before I get into the meat of this blog. As many of you know, we have announced Flash Lite 2.0 and are quite excited about it. The biggest difference with this version is the jump to the Flash Player 7 code base. This earns us a lot, but most importantly it gets us ActionScript 2.0. This alone makes things so much better for mobile authoring. I'll be talking about authoring for Flash Lite 2.0 in the near future, but in the meantime you are developing for Flash Lite 1.1 right now. So while many of these optimization tricks will apply to authoring for Flash Lite 2.0, Flash Lite 1.1 developers...this is for you.
Simplify
It depends on the specific device, but in general, phone CPUs are slow. So hands down, the best way to optimize your files is to keep things simple. What keeps Flash files small is the fact that the player does a fair amount of the rendering. The more the player has to 'think' about, the longer it will take to render a frame. The following suggestions will ensure your movie is as snappy as it can be.
Optimize vector shapes
The more dimension a shape has, the more Flash will have to do to render that shape. So optimize your vector shapes as much as possible. This can be especially helpful with small vector shapes such as icons. Your icon might be so small that lots of the detail is being lost, yet if the complexity of the shape is retained that's extra work for the player to render. In some cases using a bitmap instead of vectors shapes might be a better solution.
Corners are better than curves
Corners can be mathemtically simplier to render than curves. When possible, stick with flat edges, especially with very small vector shapes.
Avoid shape outlines
Whereas a fill has only an outside shape to render, outlines have an inside and an outside to render. This mean twice as much work to draw a line over a fill. Avoid them when possible.
Use text sparingly
Text is essentially just a very complex vector shape. This makes type one of the more complicated shapes for Flash to render. Of course text is often essential, so it can rarely be avoided entirely. When you do use text avoid placing it over animation and consider using text as a bitmap.
Avoid overlapping tweens
When Flash draws a region, it does so by defining a rectangular bounding box around the area. You can optimize this drawing by getting that rectangle as small as possible. This means that if you can, you should avoid overlapping tweens since Flash will see the merged area as a single rectangle resulting in a larger total region. Use Flash 8's 'show redraw region' feature within the player to optimize your tweens.
Look good
Although Flash Lite is built upon the desktop player, it has been customized for devices. So make sure you are thinking about the unique qualities of Flash Lite while authoring. Your content will look much better for it.
Gradients often show banding
Consider that the majority of devices currently on the market still only support thousands of colors, not millions. That means that gradients will often appear banded - stripes of solid colors instead of a smooth graduated transition. If banding becomes a problem, consider using a dithered GIF instead.
Avoid scaling or rotating bitmaps
The Flash Lite player does not support bitmap smoothing. This means that if you scale or rotate a bitmap in Flash, it will probably appear chunky. It is best to use bitmaps at their natural resolution. If you have a graphic that you need to scale or rotate, consider using vector shapes instead.
Plan and organize
I have always been an advocate of clean Flash authoring if for no other reason than it makes it so much easier for others to open your source and make sense of it. But on mobile it actually makes sense from a optimization standpoint. Consider the following:
Arrange bitmap layers and vector layers near each other
As the player renders the movie it needs to implement different renderers depending on the type of content. That switch between renderers takes time, not a lot, but if it happens a lot, it could add up. By ordering the layers that have vector content on them near each other and doing the same for bitmaps the player can render them faster by switching less.
Remove extraneous content and code
This should go without saying, but certainly you'll want to keep your movie as tight as possible. Make sure unused movie clips get removed, kill unecessary frame and code loops, and avoid too many frames. Believe it or not those empty frames can really add up!
Avoid first frame complexity
On the desktop, we've always said, preload all your content by putting it at the beginning of your movie. On mobile a heavy first frame can result in a movie that is slow to start. Space your content naturally throughout your movie.
Precalculate data tables instead of using math functions
If you are doing a lot of math, consider precalculating the values and storing them in a (pseudo) array of variables. Pulling those values from a data table can be much faster than having Flash do it on the fly.
We're excited that Flash Lite 2.0 will bring an even better player environment to mobile devices - it's faster and has better compression. As we get closer to its release I'll post some thoughts on 2.0 development. Until then, and with some clever authoring, Flash Lite 1.1 should continue to keep you quite busy.
Josh Ulm
Principal Designer/Developer, XD Team
Posted by julm at 03:19 PM | Comments (11)
October 27, 2005
Design for Devices
Our lives are full of devices... we have Desktops, Laptops, PDAs, Mobile Phones, Televisions, Cameras and more. The user experience of these devices is going to start to converge. We need to look at how the experience translates to each device.
Andrew and I recently created a cross-device prototype for a media management tool. We designed the experience to connect media across a Desktop, Television and Mobile Phone, allowing users to organize, buy and most importantly, play their media. Much smarter people than I have known for a long time now... A successful design for one device might not work on another. So we set out to create a great experience across these devices, with specific designs for each. What follows are a few major areas to consider when designing across devices. They might seem obvious, but it can be hard to shake the tendency to design everything for the desktop.
Form Factor and Environment:
A small device will not hold the same content as a large one. A mobile phone is not a desktop monitor. Simple, right? Not really. A television is large... larger than most desktop monitors. Yet given the viewing distance and general atmosphere, this large screen will only allow slightly more content than the mobile phone. And speaking of the phone; it's small. But not only is it small, it is also in an atmosphere full of motion and distractions. So it can safely portray even less content.
Form factor and relative content load
Function:
People don't carry mobile phones to check the weather or buy movie tickets. Sure, they could do these things, but the primary task is to make calls. Also, as I mentioned before, the user is distracted when using handheld devices, so we need to consider muscle memory. A desktop monitor demands your attention, so interactions can be more focused... but a phone must compete for it. As a result, we must design for mobile with very simple, intuitive interaction patterns. Up should mean up, left should mean left. The screen can map physically to the directional pad, allowing the user to navigate much like they would a video game. This allows the user to interact without thinking too much about it. It also allows them to memorize key combinations. They might learn that weather is Right, Right, Right, Enter. They can do this without even looking. The television space has many similarities (we think.) For example, the user isn't directly interfacing with the controls, but doing so via a proxy (usually a focus rectangle of some sort). This means that Enter is used over and over, contextually. Buttons will do different things based on where you are on the screen. This is much different than the desktop experience where common wisdom tells us to make each button have a specific action tied to it. And remember, the user wants to watch television, not stare at our interface.
On non-cursor devices, map content to the function of the controller
Context:
Probably most important of all... Users might not want the exact same app on all their devices. We need to think like people, not like software. Yeah, it's cool that my Flash app can resize to a mobile phone and have logic in it to show content based on format. But this is the software talking, trying to lure us into its web. If we step back and think like people, we soon realize that we want specialized apps for each device, because I use each device for a certain purpose. The desktop is (generally), focused, at home or work, and mostly online. The mobile phone is only truly powerful when the user is out and about. I don't need a full-blown shopping tool on my phone, but perhaps a way to buy movie tickets or map theatres would be useful. Similarly, when they are at home watching television, they aren't in the mode of shopping or managing anything, so we should serve a very basic, yet rich, experience that complements their main task... watching television.
I know that this sounds really basic... and it is. But I find that it's good to step back sometimes and restate the obvious. I find myself constantly tending to do what I know. I know the desktop pretty well, and we have tried-and-true patterns for it. So when we do mobile or other device design, it's just instinct to apply those same patterns.
Ty Lettau
Senior Interactive Designer, XD
Posted by tlettau at 10:47 AM | Comments (3)
September 23, 2005
Mobile UI Animation, Part 2
We have been building countless user interfaces for mobile devices using FlashLite these days. Flash can bring a lot to the mobile experience, but one of the most powerful areas Flash can change the mobile experience is with animation. In part one of this blog series I gave an example of poor mobile UI animation. In part two, I'll show some good examples and further discuss how animation can make the mobile experience useful and enjoyable instead of useless and distracting.
Now for the good stuff.
One of the best uses of animation in a Flash mobile UI is in reinforcing physical interactions. Recall Andrew's astute observation that mobile phones don't use a mouse. Seems obvious, but you might not fully realize how fundamental this is until you start building some mobile interfaces. The physical interaction on a phone is utterly different than on the desktop (in fact, the whole conceptual model is different, find more on that here). Phones typically make use of a five-way joystick or pad to move through an interface. This has a couple important implications to the user. Firstly, it means that all navigation is linear. You can't just click anywhere. If you want to get from point A to point C, you'll need to go through B to get there. Secondly, since there's no cursor, we need to be very clear to the user where they currently have focus. Which begs the question, if I try to move, where will I go from here? These are important questions in mobile UI design and animation can help considerably.
In this next example we use animation specifically to reinforce the connection between the physical interaction with the five-way and the content on the screen. Use the up and down arrows to move between screens. First try the example with no transitions.
Note: You will see many Flash movies in our blogs. To interact with them, click on them first to give the individual movie focus. Then you will be able to control them with the keyboard.
|
|
It's easy enough to get around, albeit a little disconcerting. Next try the interface with the sliding transition. As you move up and down here, the animation responds in the direction you move. It's simple, but effective. In this way, the user's actions are encouraged by the visual feedback from the interface, and an otherwise unfamilar interaction becomes immediately clear.
In this last example, let's look at another scrolling list. This one with a slightly different interaction. We call this UI a carousel. Essentially, it is a horizontal scrolling list. However, while the vertical list might be familiar to the average user, this horizontal scrolling can seem foreign - especially when they're combined.
|
|
Because there is no preview of the next or previous items, and left/right scrolling is fairly unfamiliar, the static version is downright confusing. Once again, by using animated transitions we reinforce the connection between the physical interaction (pushing left and right) with visual feedback to create a useful interface.
I'll talk more about creating device approriate UIs with FlashLite in future blogs.
Josh Ulm
Principal Designer/Developer, XD Team
Posted by julm at 10:52 AM | Comments (11)
September 19, 2005
Mobile UI Animation, Part 1
We have been building countless user interfaces for mobile devices using FlashLite these days. Furthermore, we have been looking at countless mobile UIs that do not use Flash and considering what role Flash should play in their evolution. Flash can bring a lot to the mobile experience; it is lightweight, dynamic, cross-platform, and has a mature authoring environment to mention just a few of its benefits. But one of the most powerful areas in which Flash can change the mobile experience is with motion. The majority of mobile experiences today are static and, well...boring. Flash can bring these experiences to life with animated content, effects and transitions.
In this two-part blog I'll discuss some of our ideas about animating the mobile UI with Flash and how animation can make the mobile experience useful and enjoyable instead of useless and distracting.
Let's start with a bad idea.
I'm sure the ideas are already popping into your head. Flash animation on mobile devices...good grief. Believe me, I still feel the sting of early Flash animation on the web. Hell, I'm responsible for some of it. At least now we have a little hindsight. Here are a few things to keep in mind. Often what would be bad Flash animation on the desktop is going to be bad on phones. Furthermore, devices typically have very little memory and run much more slowly than their desktop counterparts. So in general, anything large is going to be bad. However, instead of focusing on the obvious examples of poor mobile Flash animation lets look at some edge cases.
Pick up just about any mobile phone these days and you find yourself navigating through some sort of vertical list. It's just about the most common user interaction phones use these days. One of the first things we ask (or are asked) when designing an interface like this, is how can using Flash make this experience better? Certainly animation is going to come up early in the discussion and one might consider taking such a simple static list and animating it. Below are two examples of the same list. The first one is representative of a common static scrolling list, while the second one is subtly animated. Use the up and down arrows to move the selection.
Note: You will see many Flash movies in our blogs. To interact with them, click on them first to give the individual movie focus. Then you will be able to control them with the keyboard.
|
|
Which do you like better? It's easy for me. The static menu is much more responsive. Sure the little animation is nice, but is it a better experience?
Now it certainly could be argued that the animation in the second list reinforces the physical interaction of the user as they move through the items (in fact I'll be arguing that rationale for other uses of animation in part two of this blog entry). However, navigating a vertical list is so commonplace to phone UIs that here it is gratuitous and actually makes the menu feel sluggish. At the same time, it adds little to the usefulness of the UI - the change is purely aesthetic.
Furthermore, recall from Andrew's blog entry that on phones, space is precious. Since everything is so much smaller on the phone screen, animations need to be large to be noticed. On a phone, the animation of the focus area sliding up and down is so subtle that it goes largely unoticed.
If you haven't seen a PSP lately, then you should. Because their UI is gorgeous. And its animated, much like the example here. But keep in mind that there is a huge difference between the powerful PSP processor and its lucious screen and nearly every phone on the market. If space is precious, so is memory and performance. On a phone we average about 5-7 frames per second. Compared to what we're used to on the desktop, that's slow. Animations that take up the whole screen run even slower because so much more must be updated each frame. Which isn't to say that you should never do big animations, just have a very good reason and be efficient. Here again, the sliding list items offer very little to enhance the UI, while coming at major cost to playback performance.
The design moral of this little story is, "just because you can do it, doesn't mean you should". In part two of this blog entry I'll show some examples of more useful mobile UI animation using FlashLite.
Josh Ulm
Principal Designer/Developer, XD Team
Posted by julm at 01:22 PM | Comments (9)
September 15, 2005
MEX and The Mobile User Experience
I recently returned from MEX: The PMN Mobile User Experience conference. The conference took place in London and brought together a surprisingly eclectic group of players from every aspect of the mobile industry. The sessions covered everything from authoring, usability, hardware, services, and much more. However, the defining message from the conference was that in the mobile world, user experience is king. And indeed it is.
But the unfortunate reality is that the current user experience on mobile devices is painfully inadequate. The conference did a great job of creating an atmosphere of discussion among industry experts, despite the fact that very few good solutions to the problem were put forth. However, there was broad recognition that the user experience should, will and does drive the market. Everyone put a great user experience at the top of their mobile wish list. In my presentation to attendees, I outlined some of the major hurdles we have to overcome.
The biggest problem with mobile experiences, by far, is that they don't scale. Once an experience is built for a given device using current technologies, it is very difficult to change. Consider that some 800 million total devices will flood the global market this year alone. Each model with its own technical differences like processor speed, available memory, screen size and resolution, all of which require a customization to the user experience. Then for any given device, consider all the languages it may need to support and all the services it may or may not enable. And then there's personalization. Which actually is so difficult that the only things most phones let you change are superficial, like the background and ringtone.
While scaling the experience to accommodate different devices, services and users is a technical issue, it has the largest impact on the user experience. Due to the difficulty changing the user interface on a given device, most UIs are designed to suit the broadest range of users. Whether you are an expert user or a beginner, a professional or a college student, the experience is always the same.
And thats just on the device side. On the development side, the problem of scale is even uglier. While the mobile industry is taking off, there is hardly a platform available built with mobile in mind. Few technologies offer tools for creative development and then testing and deploying on countless devices.
Then there's the user experience paradigm itself. As more and more mobile devices support data services like WAP browsing, the paradigm implemented is taken from the web. The Internet/desktop is the wrong model for the mobile experience for many reasons, but let me highlight four of them.
Click and Wonder
The page based model of websites that WAP uses requires you to go out and find services, click and then wait for the new content to be delivered to you. This may be fine on the desktop where we expect to have an experience that lasts several minutes, but on a phone we expect results instantaneously. One solution is to push content onto the phone before a user requests it so that it is immediately accessible.
I Want My MTV
The largest share of mobile users is currently the high school/college student demographic. These users think about their phone very differently from their desktop computers. Studies show that the youth market sees their computer as a place for work and research, while the phone is a place for socialization and entertainment. They aren't looking for powerful data tools like they might from today's RIAs. They are looking for services that are fun, useful and entertaining.
Of Mice and Mobile
It's easy to overlook that mobile UIs are largely linear experiences. On the desktop we have a mouse and can click anywhere we want. Yet on devices, we typically rely on a 5-way joystick/button-pad for input. This means that in order to get from option A to C, you'll have to go through B to get there. Successful mobile experiences need to look at the form of the device they reside on.
Can You Hear Me Now?
Mobile experiences have a lot of competition from the outside world. We use our phones anywhere and everywhere. Whereas the desktop experience is largely relegated to, well, the desktop. Phones have to compete with noise from the likes of traffic and crowds and have the tiniest of screens to do it with. Mobile experiences need to anticipate this and work much harder to keep the attention of their audience.
In XD we think a lot about these problems and for good reason - the experience matters. The mobile experience is an exciting new platform we are dedicated to improving. Over the coming months you will see new solutions from Macromedia that aim to solve these problems. Studio 8, for example, includes a new emulator that simulates nearly 100 devices and exports directly to FlashLite 1.1, our latest mobile player. Our FlashCast platform has just gone live with a new service provided by NTT DoCoMo called iChannel and delivers rich data services OTA.
In coming blogs we'll explore some of our ideas and solutions in greater detail. I personally look forward to having a discussion about what is good and bad about the mobile experience, what users want and need, and how Macromedia can come to the rescue.
Josh Ulm
Principal Designer/Developer, XD Team
Posted by julm at 11:30 AM | Comments (2)
June 29, 2005
Our foray into mobile.
Writing the first entry in a blog is a challenge. In the case of what is essentially a corporate blog the challenge is compounded by various political considerations. What can I reveal about our current projects? What can I say about our past projects? Can I praise / trash our / other products? Do people even know who we are, or more importantly care about what we have to say?
I decided to post a note about mobile development, how our design evolved from the desktop to mobile, and some basic lessons we learned.
First a disclaimer about the team – we can sometimes appear overly skeptical or downright cynical. The truth is we strive to be pragmatic and utilitarian. We approach projects with caution, spend a lot of time discussing real frustrations with existing product implementations, and work to improve the experience in a coherent and determined way. We are more comfortable with “advancing� rather than “revolutionizing� the user experience.
It is an attitude Apple embraced only recently when they began using their design and engineering prowess to improve the experience of an existing product such as an MP3 player (and created the iPod) instead of inventing an entirely new one like the PowerMac G4 Cube. Their recent approach to product design is something I would like to blog about separately in the future.
My work in the last 6-9 months has almost entirely been in mobile. Flash is an impressive development platform on the desktop, but on mobile devices its unique capabilities make it an amazing platform. So amazing in fact, that we sometimes don’t even know what to do with it. This is why XD became involved.
It could have been our lack of prior mobile experience or baggage from our protracted involvement in planning and design of Flex and Breeze products, but our first mobile application looked roughly like this:
Figure 1.1 Early Designs
I think it was both. We spent nearly year developing neat, organized, and scalable container / UI component systems for Flex, and for the purposes of that product we were successful. We had a unique, rich, skinnable “Halo� set of components and behaviors with its hallmark green glow. We developed a relatively flexible “container� design with style-able properties. Armed with these assets it was easy to design (but very likely harder to build) an elegant application for virtually any task.
Figure 1.2 Flex Container Styles
After building several applications using this system (including Breeze Live), when faced with our first mobile project our inclination was to do a simple port. I made creating "Halo Lite" one of my first priorities, and designed a tiny set of UI components that we shipped with Flash Lite SDK.
Figure 1.3 Halo Lite for Mobile
We thought that mobile applications are basically just desktop applications with tiny buttons, and we were wrong. After tackling several “real� mobile projects we determined that visual design for mobile phones vastly differs from other platforms in two important ways:
Space is precious. Container chrome and borders waste screen real estate that could be content.
There is no cursor. Buttons and scrollbars are relatively useless and selection focus is the single most important design element (often dictating the design of the entire application).
This is a phone. The most important function of this device is making and receiving calls.
It may sound obvious but the full impact of these limitations is only realized when faced with actually having to build a mobile application. And it will take a lot of work to develop a system that overcomes these in a elegant manner. Below is a recent example of some of our thinking:
Figure 1.4 Recent Designs
We avoid chrome unless it could help or possibly substitute branding. Our scrollbars are minimal since their purpose is strictly orientation. Content is primary and visually engaging. Affordance is minimal with focus pronounced and consistent.
This is the first of many posts about mobile design and development. In the coming months we are likely to dissect specific projects in depth.
by Andrew Borovsky
Senior UI Designer, XD Team
Posted by aborovsky at 05:00 PM | Comments (16)