RE: Image callback for each page?
Irv Salisbury III wrote: It seemed like most of the readers would read the image into memory. So, one thought on an enhancement might be to always read the bytes in from the URL and then just hand that off instead of opening the URL again. Of course, this doesn't allow for future enhancements where it doesn't need to read the whole thing into memory. Hi Irv: I spent the last couple of days refactoring my graphics package (not FOP, but derived from FOP) a bit. It needed to be done anyway, and your question gave me a good excuse to dive in. The upshot is this: 1. As you have suggested, I have changed most of the Readers (Factories in my package) to open the stream once in most cases, read what it needs, and reset. One of them (I think TIFF) still opens another stream later. That can probably be fixed also, but I don't have time to do it now. 2. More importantly, I replaced all of the class-loading stuff with pluggable factories, which means that you can create your own factory, register it, and, as the URL goes by, have it look at it to decide if it is one of your special cases. If so, create a Graphic instance that talks to your own classes, and you are on your way. The method that makes the Graphic instance (i.e. that manages the Factories) has a boolean that can be set to either cache the instance or not. If you decided to use any of this, you still have the non-trivial task of integrating my work back into FOP's code. It is very doable, but it will probably take some time. I'm not really suggesting that this is an efficient way to go -- I just wanted to let you know that it an option. You can see the code here: http://cvs.sourceforge.net/viewcvs.py/foray/foray/foray-graphic/src/java/org /foray/graphic/ Email me off-line if you need help finding out how to download the code. Victor Mote - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Image callback for each page?
Victor Mote wrote: Irv Salisbury III wrote: It seemed like most of the readers would read the image into memory. So, one thought on an "enhancement" might be to always read the bytes in from the URL and then just hand that off instead of opening the URL again. Of course, this doesn't allow for future enhancements where it doesn't need to read the whole thing into memory. Hi Irv: I spent the last couple of days refactoring my graphics package (not FOP, but derived from FOP) a bit. It needed to be done anyway, and your question gave me a good excuse to dive in. The upshot is this: 1. As you have suggested, I have changed most of the Readers (Factories in my package) to open the stream once in most cases, read what it needs, and reset. One of them (I think TIFF) still opens another stream later. That can probably be fixed also, but I don't have time to do it now. 2. More importantly, I replaced all of the class-loading stuff with pluggable factories, which means that you can create your own factory, register it, and, as the URL goes by, have it look at it to decide if it is one of your special cases. If so, create a Graphic instance that talks to your own classes, and you are on your way. The method that makes the Graphic instance (i.e. that manages the Factories) has a boolean that can be set to either cache the instance or not. If you decided to use any of this, you still have the non-trivial task of integrating my work back into FOP's code. It is very doable, but it will probably take some time. I'm not really suggesting that this is an efficient way to go -- I just wanted to let you know that it an option. You can see the code here: http://cvs.sourceforge.net/viewcvs.py/foray/foray/foray-graphic/src/java/org /foray/graphic/ Email me off-line if you need help finding out how to download the code. Victor Mote I appreciate that! Unfortunately, I have implemented what I needed by doing a custom URL. I had actually never done this with Java before, but it wasn't bad. So, I "register" my own URLConnection type with Java, and then when FOP hits that URL for the image, my code gets instantiated. I can then dynamically generate the image and return the InputStream to it. It works great and I don't have to mess with the FOP code. As a "side effect" I can use that same strategy with our library outside the FOP code. I hope to get some time at some point to give back to FOP. I have given back to most of the other open source projects I have used. We are on such a tight deadline with a few different projects that it won't be for awhile. However, I am hoping our current project will be a nice addition to the success stories of FOP. We are going to be printing about 3 million pages per month with a major company, right at the heart of its business. Once it is further along, I can discuss more, but I am hoping it will be a huge success for both FOP and open source. I am sure this won't be the last question I have! By the way, I found out the technology I had to integrate had a public URL. So, you can see it at: http://www.parc.com/research/istl/projects/dataglyphs/ So, we are using FOP to print the pages with the dataglyphs encoded within. Irv - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Image callback for each page?
Irv Salisbury III wrote: I apologize if I came across as accusational. No problem, I just thought I should supply some background. I was trying to understand the code so I knew the ramifications of creating a REST style interface for my dynamically generated images. You have answered my question. There's more, see below. It seemed like most of the readers would read the image into memory. So, one thought on an enhancement might be to always read the bytes in from the URL and then just hand that off instead of opening the URL again. A simple solution which may run into ressource problems for high resolution bitmaps, where you have to keep both the raw data as well as the processed image object in memory. Of course, this doesn't allow for future enhancements where it doesn't need to read the whole thing into memory. If I implement somethign like this, I would happily donate it back. Thanks, There are further considerations regarding image data retrival, which have been left to implementors by the FO spec. The same URL for retrieving an image may be used in several external-graphic FOs, and the same external-graphic FO may be rendered multiple times (as a child of static content, or if pulled into static content by a marker reference). This begs the question of how often the URL is accessed, with three obvious general strategies: - exactly once per unique URL in the whole FO document - exactly once per external-graphic FO - each time an external-graphic FO is rendered Independent image caching complicates the issue: image data may be discarded from the cache and retrieved again later. Technical issues like layout backtracking and out-of-order rendering of pages add further complexity. A special case may be an output format which doesn't require retrieving content at all (think of SVG where the FO URLs are copied through to the XLinks). Have fun J.Pietschmann - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Image callback for each page?
[EMAIL PROTECTED] wrote: We have an api that generates a sort of barcode image that needs to be put on the bottom of each page. However, each image is unique and needs to be generated with some information from the xml document. I know I can do this by using a REST api call and putting a URL in the fo:graphic that dynamically generates the image. However, I'd rather do it right in the Java code that sets up the Driver. So, is this possible? Some thoughts I have, but don't know if they exist: My guess is that markers with a URL would be the easiest and most standard way to do this. But I'll take your word for it that you want the callback. 1. Can I get a callback that calls my Java code when a certain fo:graphic element is hit and let me dynamically put the XSL:FO content in there? This would allow me to dynamically generate the images on the fly and put the appropriate image content in . I think you can do this, but again, it seems like a pain. IIRC correctly, the FONode class has a getParent() method which can be used to recursively go up the tree until you get to the Root (keep in mind that this is XML/FO inheritance, not Java inheritance). I would cache the object that will generate the image in Root, and, if it does not exist already, write an FONode method called getRoot() which will recursively go up the tree to Root and return Root. Then, you'll need to modify the code that creates the image in the AreaTree, and then use your (presumably) ExternalGraphic node to do something like the following: YourGraphicMaker ygm = externalGraphic.getRoot().getYourGraphicMaker(); Image image = ygm.makeGraphic(externalGraphic); // pass the image instance where it needs to go. I have used the above technique successfully to handle global items like loggers and subsystem servers. 2. Can I put custom xsl:fo elements in the document and then register my own renderers for dealing with those types? The renderers are more oriented toward a given output medium, so you are more likely going to be modifying or extending one to handle your custom elements, but yes, this general technique could work as well. My guess is that this is probably even more of a pain than option 1 above. Some information is here: http://xml.apache.org/fop/dev/extensions.html Victor Mote - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Image callback for each page?
Quoting Victor Mote [EMAIL PROTECTED]: [EMAIL PROTECTED] wrote: We have an api that generates a sort of barcode image that needs to be put on the bottom of each page. However, each image is unique and needs to be generated with some information from the xml document. I know I can do this by using a REST api call and putting a URL in the fo:graphic that dynamically generates the image. However, I'd rather do it right in the Java code that sets up the Driver. So, is this possible? Some thoughts I have, but don't know if they exist: My guess is that markers with a URL would be the easiest and most standard way to do this. But I'll take your word for it that you want the callback. 1. Can I get a callback that calls my Java code when a certain fo:graphic element is hit and let me dynamically put the XSL:FO content in there? This would allow me to dynamically generate the images on the fly and put the appropriate image content in . I think you can do this, but again, it seems like a pain. IIRC correctly, the FONode class has a getParent() method which can be used to recursively go up the tree until you get to the Root (keep in mind that this is XML/FO inheritance, not Java inheritance). I would cache the object that will generate the image in Root, and, if it does not exist already, write an FONode method called getRoot() which will recursively go up the tree to Root and return Root. Then, you'll need to modify the code that creates the image in the AreaTree, and then use your (presumably) ExternalGraphic node to do something like the following: YourGraphicMaker ygm = externalGraphic.getRoot().getYourGraphicMaker(); Image image = ygm.makeGraphic(externalGraphic); // pass the image instance where it needs to go. I have used the above technique successfully to handle global items like loggers and subsystem servers. 2. Can I put custom xsl:fo elements in the document and then register my own renderers for dealing with those types? The renderers are more oriented toward a given output medium, so you are more likely going to be modifying or extending one to handle your custom elements, but yes, this general technique could work as well. My guess is that this is probably even more of a pain than option 1 above. Some information is here: http://xml.apache.org/fop/dev/extensions.html Victor Mote You are correct that doing a custom servlet REST style api for generating the images would be the cleanest. However, we have some serious performance needs, and I am reluctant to throw another piece in this that introduces http with something that could be a local method call. Looking through the source code, it seems like the org.apache.fop.image.analyser .ImageReaderFactory class might be something I could use. It looks like if I write my own custom URL, and register my class as a reader, I will get called through the verifySignature method of ImageReader. As long as I am the only one that can deal with the image, I should be all set. The other thought I had with looking at the code is regsitering my own custom URL through the JDK mechanisms. Irv - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Image callback for each page?
Quoting [EMAIL PROTECTED]: Quoting Victor Mote [EMAIL PROTECTED]: [EMAIL PROTECTED] wrote: We have an api that generates a sort of barcode image that needs to be put on the bottom of each page. However, each image is unique and needs to be generated with some information from the xml document. I know I can do this by using a REST api call and putting a URL in the fo:graphic that dynamically generates the image. However, I'd rather do it right in the Java code that sets up the Driver. So, is this possible? Some thoughts I have, but don't know if they exist: My guess is that markers with a URL would be the easiest and most standard way to do this. But I'll take your word for it that you want the callback. 1. Can I get a callback that calls my Java code when a certain fo:graphic element is hit and let me dynamically put the XSL:FO content in there? This would allow me to dynamically generate the images on the fly and put the appropriate image content in . I think you can do this, but again, it seems like a pain. IIRC correctly, the FONode class has a getParent() method which can be used to recursively go up the tree until you get to the Root (keep in mind that this is XML/FO inheritance, not Java inheritance). I would cache the object that will generate the image in Root, and, if it does not exist already, write an FONode method called getRoot() which will recursively go up the tree to Root and return Root. Then, you'll need to modify the code that creates the image in the AreaTree, and then use your (presumably) ExternalGraphic node to do something like the following: YourGraphicMaker ygm = externalGraphic.getRoot().getYourGraphicMaker(); Image image = ygm.makeGraphic(externalGraphic); // pass the image instance where it needs to go. I have used the above technique successfully to handle global items like loggers and subsystem servers. 2. Can I put custom xsl:fo elements in the document and then register my own renderers for dealing with those types? The renderers are more oriented toward a given output medium, so you are more likely going to be modifying or extending one to handle your custom elements, but yes, this general technique could work as well. My guess is that this is probably even more of a pain than option 1 above. Some information is here: http://xml.apache.org/fop/dev/extensions.html Victor Mote You are correct that doing a custom servlet REST style api for generating the images would be the cleanest. However, we have some serious performance needs, and I am reluctant to throw another piece in this that introduces http with something that could be a local method call. Looking through the source code, it seems like the org.apache.fop.image.analyser .ImageReaderFactory class might be something I could use. It looks like if I write my own custom URL, and register my class as a reader, I will get called through the verifySignature method of ImageReader. As long as I am the only one that can deal with the image, I should be all set. The other thought I had with looking at the code is regsitering my own custom URL through the JDK mechanisms. Irv One more question: While looking into this code, it looks like the FopImageFactory opens up a stream to the image using the URL class, then the specific ImageReader implementation reopens the stream again. So, if I did a REST style interface to generate my images, it looks like it would be called twice. Am I missing something here? It looks like this is a little inefficient. Irv - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Image callback for each page?
[EMAIL PROTECTED] wrote: While looking into this code, it looks like the FopImageFactory opens up a stream to the image using the URL class, then the specific ImageReader implementation reopens the stream again. So, if I did a REST style interface to generate my images, it looks like it would be called twice. It depends on the image format, for some formats, the image reader tries to reuse the stream. Am I missing something here? It looks like this is a little inefficient. The problem is that some stock image readers only accept a stream with the pointer at the begin of the content, and if the stream pointer can't be reset (e.g. if the analyser needed so much info that the underlying implementation already discarded buffers), then the stream has to be reopened. If you want to donate an intelligent buffering system J.Pietschmann - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Image callback for each page?
J.Pietschmann wrote: [EMAIL PROTECTED] wrote: While looking into this code, it looks like the FopImageFactory opens up a stream to the image using the URL class, then the specific ImageReader implementation reopens the stream again. So, if I did a REST style interface to generate my images, it looks like it would be called twice. It depends on the image format, for some formats, the image reader tries to reuse the stream. Am I missing something here? It looks like this is a little inefficient. The problem is that some stock image readers only accept a stream with the pointer at the begin of the content, and if the stream pointer can't be reset (e.g. if the analyser needed so much info that the underlying implementation already discarded buffers), then the stream has to be reopened. If you want to donate an intelligent buffering system J.Pietschmann I apologize if I came across as accusational. I was trying to understand the code so I knew the ramifications of creating a REST style interface for my dynamically generated images. You have answered my question. It seemed like most of the readers would read the image into memory. So, one thought on an enhancement might be to always read the bytes in from the URL and then just hand that off instead of opening the URL again. Of course, this doesn't allow for future enhancements where it doesn't need to read the whole thing into memory. If I implement somethign like this, I would happily donate it back. Thanks, Irv - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Image callback for each page?
J.Pietschmann wrote: The problem is that some stock image readers only accept a stream with the pointer at the begin of the content, and if the stream pointer can't be reset (e.g. if the analyser needed so much info that the underlying implementation already discarded buffers), then the stream has to be reopened. If you want to donate an intelligent buffering system BufferedInputStream (and all InputStreams) have mark() and reset() functions which should work for this. I rather thought that the developer maybe didn't want to leave the stream open in between layout and rendering, but I don't know. WRT intelligent buffering system, I wrote one for my font work, based on java.io.RandomAccessFile with a wrapper of my own that allows you to read in bigger chunks at a time with a kind of window that slides back and forth over the file contents. I haven't looked at the ImageReader subclasses to know whether they really need a random-ish approach or whether resetting the Stream would be sufficient. If you are interested, the source is here (the inner class ByteSearcher is where the sliding window is managed): http://cvs.sourceforge.net/viewcvs.py/foray/foray/foray-common/src/java/org/ foray/common/RandomReader.java?view=markup I don't really have a good feel for how efficiently this works. It seems to work pretty well, but I haven't profiled it. Of course, FOP is always welcome to this code. Also, if Irv does donate some other intelligent buffering system, he needs to understand that the FOP developers won't use it unless it is written for the head/trunk. Victor Mote - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
