Re: Canvas blowing up (was Re: JavaFX Media issues)
This is a great idea. We should just enter a tweak and do it. If the area that is being cleared is larger than the current size of the canvas, we can throw away all pending draw commands. Steve On 09/08/2013 11:23 AM, Richard Bair wrote: I mean, it looks like it is working for a few seconds, but then as the memory fills with the Canvas backlog it can lead to the GC using a lot more CPU, thus reducing the ability for Canvas to process its command queue even further, well it just collapses in on itself and dies. Forking the thread. The problem with Canvas is that if you have a canvas and you scribble on it, and then scribble on it some more, and then scribble on it some more, then in order for us to get the right result in the end, we need to replay all those scribbles in order. If pulses are not happening, we still need to remember these scribbles so we can draw the right result. BUT, if you issue a command to the canvas which will cause it to clear all its contents, then we could throw away any previously buffered data. Right now the only way to do that would be a fillRect with a solid fill where the fillRect encompasses the entire canvas area, or a clearRect where the clearRect encompasses the entire canvas area. This seems like a very simple fix. GraphicsContext.clearRect and GraphicsContext.fillRect should both (under the right conditions) throw away the previously buffered commands. Then all you have to do is be sure to make one of these calls (likely just a clearRect) before each frame, and we'll never buffer more than a single frame's worth of data. We could also add a clear method which is clearRect(0, 0, w, h) to make this more foolproof, and then document it as a best practice to clear the canvas before each rendering if you intend to redraw the entire thing on each frame. If you're making use of manually operated dirty rects so that you only clear the damaged area to repaint, then we couldn't employ this technique and we'd have to buffer 'till kingdom come. So we still need a mechanism exposed in the scene graph of liveness and associated events so that when the scene is no longer live (for example, when minimized) you could stop your animation timer, but for your specific media use case this isn't as important. Richard
Re: Canvas blowing up (was Re: JavaFX Media issues)
That's okay for a quick hack. In the case of a video preview surface, I will be explicitly setting the value for every pixel from a ByteBuffer. You could save the extra step of doing a rectFill or clearRect if you knew that every pixel was about to be overwritten. It's a reasonable optimization.. but as a fix for this issue it's still only a half-fix hack. If pulses are not happening, we still need to remember these scribbles so we can draw the right result. No. If pulses are not happening you need to block or force a pulse somehow. Otherwise I don't see how having the unbounded queue is ever going to be 100% reliable. Since we are talking about painting to the Canvas surface as opposed to directly modifying the scene graph, why does the painting have to happen later when a pulse occurs? It's not like you have any other thread writing to the Canvas. Why can't the Platform thread actually *do* the scribbles,and the pulse just refreshes the portion of the Canvas that is visible on the screen? Is it some D3D/OpenGL multi-threading complication? Regards, Scott On Fri, Aug 9, 2013 at 11:43 AM, steve.x.northo...@oracle.com wrote: This is a great idea. We should just enter a tweak and do it. If the area that is being cleared is larger than the current size of the canvas, we can throw away all pending draw commands. Steve On 09/08/2013 11:23 AM, Richard Bair wrote: I mean, it looks like it is working for a few seconds, but then as the memory fills with the Canvas backlog it can lead to the GC using a lot more CPU, thus reducing the ability for Canvas to process its command queue even further, well it just collapses in on itself and dies. Forking the thread. The problem with Canvas is that if you have a canvas and you scribble on it, and then scribble on it some more, and then scribble on it some more, then in order for us to get the right result in the end, we need to replay all those scribbles in order. If pulses are not happening, we still need to remember these scribbles so we can draw the right result. BUT, if you issue a command to the canvas which will cause it to clear all its contents, then we could throw away any previously buffered data. Right now the only way to do that would be a fillRect with a solid fill where the fillRect encompasses the entire canvas area, or a clearRect where the clearRect encompasses the entire canvas area. This seems like a very simple fix. GraphicsContext.clearRect and GraphicsContext.fillRect should both (under the right conditions) throw away the previously buffered commands. Then all you have to do is be sure to make one of these calls (likely just a clearRect) before each frame, and we'll never buffer more than a single frame's worth of data. We could also add a clear method which is clearRect(0, 0, w, h) to make this more foolproof, and then document it as a best practice to clear the canvas before each rendering if you intend to redraw the entire thing on each frame. If you're making use of manually operated dirty rects so that you only clear the damaged area to repaint, then we couldn't employ this technique and we'd have to buffer 'till kingdom come. So we still need a mechanism exposed in the scene graph of liveness and associated events so that when the scene is no longer live (for example, when minimized) you could stop your animation timer, but for your specific media use case this isn't as important. Richard
Re: Canvas blowing up (was Re: JavaFX Media issues)
We would still do the fill but we could throw away any buffered commands that happened before the fill. Steve On 09/08/2013 12:16 PM, Dr. Michael Paus wrote: What would be the performance penalty for using this quick-fix? The clear/fill commands do not just clear the command buffer. They also fill the canvas area with a certain color. So in normal operation the canvas is always filled twice for each frame, isn't it? Am 09.08.13 17:23, schrieb Richard Bair: I mean, it looks like it is working for a few seconds, but then as the memory fills with the Canvas backlog it can lead to the GC using a lot more CPU, thus reducing the ability for Canvas to process its command queue even further, well it just collapses in on itself and dies. Forking the thread. The problem with Canvas is that if you have a canvas and you scribble on it, and then scribble on it some more, and then scribble on it some more, then in order for us to get the right result in the end, we need to replay all those scribbles in order. If pulses are not happening, we still need to remember these scribbles so we can draw the right result. BUT, if you issue a command to the canvas which will cause it to clear all its contents, then we could throw away any previously buffered data. Right now the only way to do that would be a fillRect with a solid fill where the fillRect encompasses the entire canvas area, or a clearRect where the clearRect encompasses the entire canvas area. This seems like a very simple fix. GraphicsContext.clearRect and GraphicsContext.fillRect should both (under the right conditions) throw away the previously buffered commands. Then all you have to do is be sure to make one of these calls (likely just a clearRect) before each frame, and we'll never buffer more than a single frame's worth of data. We could also add a clear method which is clearRect(0, 0, w, h) to make this more foolproof, and then document it as a best practice to clear the canvas before each rendering if you intend to redraw the entire thing on each frame. If you're making use of manually operated dirty rects so that you only clear the damaged area to repaint, then we couldn't employ this technique and we'd have to buffer 'till kingdom come. So we still need a mechanism exposed in the scene graph of liveness and associated events so that when the scene is no longer live (for example, when minimized) you could stop your animation timer, but for your specific media use case this isn't as important. Richard
RE: Canvas blowing up (was Re: JavaFX Media issues)
This question was recently asked on StackOverflow as well: http://stackoverflow.com/questions/18097404/how-can-i-free-canvas-memory How can I free Canvas memory? So others have been running into these kind of issues. Also the proposed clear() or empty() option only applies to Canvas correct? i.e. WritableImages don't suffer from these kind of issues and don't require such methods? Regards, John -Original Message- From: openjfx-dev-boun...@openjdk.java.net [mailto:openjfx-dev-boun...@openjdk.java.net] On Behalf Of Richard Bair Sent: Friday, August 09, 2013 9:43 AM To: Dr. Michael Paus Cc: openjfx-dev@openjdk.java.net Subject: Re: Canvas blowing up (was Re: JavaFX Media issues) What would be the performance penalty for using this quick-fix? The clear/fill commands do not just clear the command buffer. They also fill the canvas area with a certain color. So in normal operation the canvas is always filled twice for each frame, isn't it? That would be correct. Another option is to add, instead of clear() an explicit empty() method or something that would just blow away the buffer. Richard
Re: Canvas blowing up (was Re: JavaFX Media issues)
Also the proposed clear() or empty() option only applies to Canvas correct? i.e. WritableImages don't suffer from these kind of issues and don't require such methods? That is correct (WritableImage we don't provide a 2D API to use to fill the buffer, you just bash the pixels yourself however you like, so we don't have to buffer anything up). Richard
Re: Canvas blowing up (was Re: JavaFX Media issues)
On 9/08/2013 20:15, Richard Bair wrote: Also the proposed clear() or empty() option only applies to Canvas correct? i.e. WritableImages don't suffer from these kind of issues and don't require such methods? That is correct (WritableImage we don't provide a 2D API to use to fill the buffer, you just bash the pixels yourself however you like, so we don't have to buffer anything up). Hm, I didn't realize WritableImage had the same kind PixelWriter interface. For my usecase, where I just render full frames to a PixelWriter so JavaFX can display them in some fashion, why should I not use a WriteableImage instead? Looking at the docs, I see that one limitation is that the WritableImage cannot be resized after construction (something that I do use with Canvas), but I think I could work around that by just recreating the Image when its size changes... I could just wrap a class around it that does this transparently. Going to take a look if I can rip out the Canvas code and replace it with a resizable WritableImage and see what the results are for full screen video playback... --John
