[Flashcoders] Automatic quality toggle
I have been thinking a lot about the problem of optimizing animation playback. I know that some people here are unfamiliar with working with real animations and have been doing applications instead. Please consider this aimed at real animations like in movies and games. While the best solution is to simply make the animation simpler so that it is easier to render, that is far from an easy, or even possible solution. Instead the idea is to dynamically adjust the rendering quality. But as I will explain, it is far from a simple task. Please excuse any too easy for me explanations, I aim for a post that any developer can read and understand. As most of you know the Flash player have three main rendering quality options, low, medium and high quality. There is also the option to go for best, but that is largely the same as high. What the rendering quality control mainly controls is the subpixel rendering. When flash detects that a shape has an edge on a non integer position it will do subpixel rendering for that pixel. The subpixel rendering is surprisingly simple, flash just scales up the area (factor 2 for medium, 4 for high) and renders the new pixels. It then takes the average value of the pixels and uses that as the value of the real pixel. Low quality skips subpixel rendering completely. Now, this does change the rendering cost quite a lot. As such, it is the stock option for controlling rendering cost in the player. What some people may not know is that the Flash renderer is smart enough to skip frames. What this means is that if it detects that it is too far behind it will simply skip the drawing step and move directly on to the next frame. This can happen multiple times in a row in severe cases. This is rather easy to detect. If the drawing skip is skipped, so is the Event.RENDERED event. Just by checking if there was one between Event.ENTER_FRAME events it is possible to see if the previous frame was skipped. Similarly you can get a good estimate of how long the frame took to draw by computing the time between the Event.RENDERED event and the Event.ENTER_FRAME event. It is not perfect since it includes the wait after the rendering, but that is not a problem in practice. Now we define the problem. We want to draw the graphics at the highest possible quality, but without forcing Flash to skip frames. To do this we have the option of changing the current rendering quality before each frame is rendered. In more specific situations we also have the option of changing various effects, but for easy discussion we will stick to just the rendering quality. So the question becomes, what is the condition for changing quality? A naïve option is to change the quality if the previous frame took too long to render. A slightly better option is if we can pre-render each frame and get the rendering cost on a reference machine. Then we can scale the reference value to the current machine based on a known reference point, a previously rendered frame. However, this approach has the issue of the rendering cost not scaling linearly between machines. The largest reason is that different machines might be displaying the animation at different sizes. As rendering in different sizes can change cost geometrically this makes it hard to scale properly. A different idea that I have been thinking about is to use the frameskip amount as a deciding factor. However, I think that it may lead to similar results as the rendering time. Furthermore complicating things is the fact that changing the rendering quality invalidates the dirty region caching that the player does. While the effect of this differs depending on the animation being suitable for the dirty region system, it is a possible cost. There is also the issue of it being jarring to the user when the quality does change. The solution to that seems to be hysterics, that we have a threshold before changing back. Overall, this is a complicated matter and I am requesting feedback from people with experience dealing with this issue. ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Automatic quality toggle
I don't have personal experience with this problem, but I did come up with some questions and ideas that might further your search. First off, it sounds like to me the requirement is to have a seamless playback of animations. No jitter, frameskips, or lag times. You are right, its quite the complex problem. The elastic racetrackhttp://www.craftymind.com/2008/04/18/updated-elastic-racetrack-for-flash-9-and-avm2/comes to mind when you explain your problem. The linked article has lots of information and great comments. Also, a link to Grant Skinner's look on optimization http://gskinner.com/talks/quick/ might help. Thus, I think the solution for you will have to include not only graphics rendering solutions, but logic and code execution solutions as well. My first thought was to consider the framerate you are aiming for. This will drastically effect the solution. A consideration of the human eye framerate may help you determine the optimal framerate for your program. With the framerate in mind, you could aim for a higher framerate, and drop a bit if frameskips happen, or not at all because if it's high enough you may not notice visually. However, this solution will only look fine if your animations are time based and not frame based. I think the framerate solution is directly related to the understanding of the elastic racetrack. Graphical enhancements could be turned off if noticing a problem with rendering like you said, removing filters or other 'nice to haves' for the graphics. Optimized algorithms could greatly benefit the visual rendering. Sometimes that means using 'hacks' to gain performance. Maybe multiple levels of hit detection for a game? If the machine is too slow, it could use a different type of hit detection. But to answer my own question, that would change the gameplay and create problems with high scoring or competition. So all of those may be things you can do to make performance better, they do not address the main issue which is, how do I know when to change things if they are not good enough. For this, there may be solution in writing a tool that would calculate how long each section of each slice of each frame is taking. Using the elastic racetrack article as a reference, one might be able to determine how to adjust certain aspects of the application to smooth out the time it takes for each section to finish. Maybe you can accomplish most of this task by listening to ENTER_FRAME, RENDER, EXIT_FRAME, FRAME_CONSTRUCTED events to calculate the timing of things. Then, when numbers pass specified thresholds, you could make changes to the appropriate sections. For example, you define a matrix of complexity for each 'section' of your program. These matrix define how much time each aspect of the racetrack can consume. So, during intense gameplay, maybe you want to allow more time for code execution and less on rendering. Then, if your thresholds are surpassed, an event could be fired and you could respond by, lowering the quality of the graphics, or turning off filters etc... To conclude, I wonder if the cost is worth the benefit. Ktu On Fri, Feb 25, 2011 at 6:19 AM, Henrik Andersson he...@henke37.cjb.netwrote: I have been thinking a lot about the problem of optimizing animation playback. I know that some people here are unfamiliar with working with real animations and have been doing applications instead. Please consider this aimed at real animations like in movies and games. While the best solution is to simply make the animation simpler so that it is easier to render, that is far from an easy, or even possible solution. Instead the idea is to dynamically adjust the rendering quality. But as I will explain, it is far from a simple task. Please excuse any too easy for me explanations, I aim for a post that any developer can read and understand. As most of you know the Flash player have three main rendering quality options, low, medium and high quality. There is also the option to go for best, but that is largely the same as high. What the rendering quality control mainly controls is the subpixel rendering. When flash detects that a shape has an edge on a non integer position it will do subpixel rendering for that pixel. The subpixel rendering is surprisingly simple, flash just scales up the area (factor 2 for medium, 4 for high) and renders the new pixels. It then takes the average value of the pixels and uses that as the value of the real pixel. Low quality skips subpixel rendering completely. Now, this does change the rendering cost quite a lot. As such, it is the stock option for controlling rendering cost in the player. What some people may not know is that the Flash renderer is smart enough to skip frames. What this means is that if it detects that it is too far behind it will simply skip the drawing step and move directly on to the next frame. This can happen multiple times in a row in severe cases. This is rather easy to detect. If
Re: [Flashcoders] Automatic quality toggle
You bring up a lot of valid points, but you fail to focus on the main concern here, how to deal with the graphics being the most heavy part. Please keep in mind that I am trying to focus on how to deal with animations that can't be simplified much. I am looking for a generic solution to the issue of the graphics taking too long to draw. Reducing complexity is of course a good idea. And yeah, filters are expensive and provide little benefit most of the time. But what can you do when you have exhausted your options to edit the content? You only got the quality setting left to use. With that said, you bring up some important points about how much frameskip can be tolerated. Also, thanks for reiterating my own points about how to capture performance statics. That part is easy. But it is what to do with the gathered data that I worry about. How do you decide on the thresholds? Can you successfully predict when it is worth to change the quality? ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Automatic quality toggle
But what can you do when you have exhausted your options to edit the content? You only got the quality setting left to use. - maybe then you've got to re-evaluate what you are doing or use the quality settings. How do you decide on the thresholds? Can you successfully predict when it is worth to change the quality? - Trial and error will probably be the only way,... ? I am looking for a generic solution to the issue of the graphics taking too long to draw. - Your solution will have to be based off of your requirements. A totally generic solution probably won't be actualized until more specific solutions are gathered, anaylized and merged. If your render phase of the elastic racetrack is consuming more than the entire frame/slice, then I think you will have a serious problem on your hands. If you share your requirements, maybe an appropriate solution could be determined. On Fri, Feb 25, 2011 at 12:19 PM, Henrik Andersson he...@henke37.cjb.netwrote: You bring up a lot of valid points, but you fail to focus on the main concern here, how to deal with the graphics being the most heavy part. Please keep in mind that I am trying to focus on how to deal with animations that can't be simplified much. I am looking for a generic solution to the issue of the graphics taking too long to draw. Reducing complexity is of course a good idea. And yeah, filters are expensive and provide little benefit most of the time. But what can you do when you have exhausted your options to edit the content? You only got the quality setting left to use. With that said, you bring up some important points about how much frameskip can be tolerated. Also, thanks for reiterating my own points about how to capture performance statics. That part is easy. But it is what to do with the gathered data that I worry about. How do you decide on the thresholds? Can you successfully predict when it is worth to change the quality? ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Ktu; The information contained in this message may be privileged and/or confidential. If you are NOT the intended recipient, please notify the sender immediately and destroy this message. ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Automatic quality toggle
Ktu skriver: But what can you do when you have exhausted your options to edit the content? You only got the quality setting left to use. - maybe then you've got to re-evaluate what you are doing or use the quality settings. How do you decide on the thresholds? Can you successfully predict when it is worth to change the quality? - Trial and error will probably be the only way,... ? I am looking for a generic solution to the issue of the graphics taking too long to draw. - Your solution will have to be based off of your requirements. A totally generic solution probably won't be actualized until more specific solutions are gathered, anaylized and merged. If your render phase of the elastic racetrack is consuming more than the entire frame/slice, then I think you will have a serious problem on your hands. If you share your requirements, maybe an appropriate solution could be determined. Well, I am aiming for a solution for animators. They don't have any code running. They are just animating. Often they aren't doing anything wrong. It is just too much. I got some rather stupid heavy animations if you want to see them, I can't say that they are doing anything wrong. The load just becomes unbearable in fullscreen. And when I say animations I mean stuff worthy of showing on TV. The original task that Flash dealt with. I want to find something to help animators who have some heavy scenes. They just love drawing detailed animations. And I just love watching the results, they are quite nice looking. I just wish that more people had things like vcams and as such, I aim to make the best tools. The remaining thing is a simple quality auto toggle. ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
