Re: How to use FOP in wysiswyg editor, or how to speed up
Hi, sorry for a bit late reply, Jeremias Maerki wrote: On 23.09.2006 23:12:06 Tomáa Studva wrote: Hi developers, We are using FOP as rendering engine for FO in wysiwyg xml editor http://sourceforge.net/projects/euromath2. When opening about 40 page fo document, the editation is ugly slow. We can track changes in source document(also in case using XSLT), but there is a principal error, because we don't know how to update the FOP produced trees, so new FOP is created to layout document after change and further render by Draw2d (we implemented some basic renderer composed of Draw2D Figures, which are organized into tree). I' ve also profiled our application on such big document, from typing to update of screen and found out, than FOP consumes 1/3 of processor time, that is too much if we optimize anything but not FOP- it takes on good PC about 10 seconds, so FOP 10/3 = 3sec. So, is there possibility to update the FOP produced trees according to change in XML (to use it in editor manner, not only rendering engine)? No, FOP has not been designed to be used as a backend of a WYSIWYG editor like EuroMath2. That's a new requirement/wish. We'd have to find out if and how FOP can be changed to fulfill it. At any rate it's an expansion of the project scope which could be subject to a project decision depending on how extensive the necessary changes can become. Yes, I know the design goals of FOP are * Conformance to the XSL-FO Specification * Process Files of Arbitrary Size * Minimize Memory Use and to use it in WYSIWYG editor is extend of FOP project scope. Co-developer of Euroamth2, has as diploma thesis theme, design of such rendering (layout, painting, ...) engine usable in WYSIWYG editor. The engine should be abstract enough to be usable for any graphical or textual view, and extendable to be usable for concrete document type, like FO. So he will work and maybe he will cooperate with FOP designers to design a good rendering engine. Or if not, is there possibility to recycle FOP PageSequence which haven't changed? Not at the moment, maybe with some hacking. It's not just the page-sequence that would have to be recycled. You'd have to find ways to keep the unchanged page-sequences in memory and that includes not only the FO tree but also the area tree. However, changing a page-sequence in the middle might invalidate later page-sequences. And last, is there option to speed up FOP generally, by lowering output quality or something? Not by lowering the output quality, no. You can help by profiling and optimizing FOP and you can make sure the editor only generates the minimal FO content to produce the right document. Many editor generate much too much (redundant) content not making use of property inheritance. That can have an influence on performance. Yes, I can make a profile of FOP if you don't have any yet, and can send it. I notice, but I think such optimization to produce minimal content would have only a small effect in document, where are plenty of blocks or inlines. You're welcome to help us improve FOP to better match the requirements of EuroMath2. I'm pretty sure we currently don't have the resources to help you much in this direction. We can give you pointers and we can help you figure out what needs to be changed. Help will be needed and as I said, but I must consult co-developers what to do and how. More, EM2 is academic project, so there is an opportunity to leave part of java coding as bachelor project. Please note this is my take on the situation and may not reflect the opinion of the whole community. I didn't know of EuroMath2 before your post. If I interpret this correctly it is a content editor rather than an editor to create XSLT stylesheets. Yes, you are right. I've also taken a peek into your wishlist. I don't think you can currently find any open source FO implementation that is better suited for EuroMath2. Concerning the partial FO rendering possibility: It might be possible to come up with a way to render, say, only a single fo:block-container with relatively little effort. This might have the possibly interesting side-effect that we could write a plug-in for Batik to render FO content within an SVG. :-) Interesting, now don't know if it is really needed. Jeremias Maerki We(EM2 people) are going to have a meeting, so I am sending emails, to collect as many info about BIG problems as possible. After meeting, more should be clearer. And thanks for answer. Tomas Studva
Re: How to use FOP in wysiswyg editor, or how to speed up
On 27.09.2006 13:41:01 Tomas Studva wrote: snip/ Yes, I know the design goals of FOP are * Conformance to the XSL-FO Specification * Process Files of Arbitrary Size * Minimize Memory Use and to use it in WYSIWYG editor is extend of FOP project scope. Co-developer of Euroamth2, has as diploma thesis theme, design of such rendering (layout, painting, ...) engine usable in WYSIWYG editor. The engine should be abstract enough to be usable for any graphical or textual view, and extendable to be usable for concrete document type, like FO. So he will work and maybe he will cooperate with FOP designers to design a good rendering engine. Something like that is HUGE! Even just supporting XSL-FO properly is half a nightmare. I've once investigated a commercial FO processor which was implemented on top of a general layout engine. They ended up writing a new engine just for XSL-FO because they simply couldn't get to the right quality level. So I'd be careful about the goals you set yourself. snip/ And last, is there option to speed up FOP generally, by lowering output quality or something? Not by lowering the output quality, no. You can help by profiling and optimizing FOP and you can make sure the editor only generates the minimal FO content to produce the right document. Many editor generate much too much (redundant) content not making use of property inheritance. That can have an influence on performance. Yes, I can make a profile of FOP if you don't have any yet, and can send it. Well, frankly, I can profile myself. More important would be to act on the results of the profiling. I notice, but I think such optimization to produce minimal content would have only a small effect in document, where are plenty of blocks or inlines. A colleague once got at least 30% more performance after manually optimizing a stylesheet generated by an FO editor. So I wouldn't call that a small effect. But granted, your editor is a content editor, not an FO editor that generates XSLT stylesheets, so the circumstances are a little different. snip/ We(EM2 people) are going to have a meeting, so I am sending emails, to collect as many info about BIG problems as possible. After meeting, more should be clearer. And thanks for answer. We look forward to hearing from you again and would be delighted if anyone from your project could assist us improving FOP. Jeremias Maerki
Re: How to use FOP in wysiswyg editor, or how to speed up
Hi Simon, Simon Pepping wrote: This is a very interesting and ambitious project. Yes, ambitious and hard enough, but much of work is done, now we are concerning on rendering engines, first to design rendering framework/engine and afterwards on implementation for popular document types. I would like to see such a project succeed. I too. But WYSIWYG editing is very hard to achieve. That was (and is) the case with TeX, where it was tried, and it is the case with FOP. As Jeremias already answered, FOP is not close to such a goal. We would need a whole lot more developers than we have now, and before that, a good design for this goal. In principle it should be possible to let FOP know which subtree of the FO file has changed this is partially done in EM2 , and one could let FOP update that part of the FO tree in memory. It is harder to determine which part of the layout has changed. In addition, FOP has a total fit approach to layout, which is not friendly to a partial update. Currently FOP tries to dispose of elements in memory as soon as possible, in the interest of minimal resource usage. A WYSIWYG approach would require that FOP keeps FO tree, layout elements and perhaps a number of finished pages in memory. I don't see so deep, but it shouldn't be hard to know what to update. All needed - all depending on change. FO layout depends on quite complex parameter model (inheritance, overriding, ...), but i thing that is clear parameter dependency mapping. Memory management is a problem. FOP architecture was designed as pipes and filters, so there are two trees (object tree, area tree) + DOM tree in EM. I think, it is not necessary to change this model, now 500MB is standard and that is enough for every document, even if memory consumption is three times larger. Do I remember correctly that EuroMath is developed in Academia? FOP could use a similar sponsorship. Without it, this is far too ambitious for us. Regards, Simon Yes, EuroMath is developed in Academia. This is the second version, Euromath 1 was scrapped. Euromath 2, is better developed, but main problem is time. There are problems, which needs time and that blocks development. Financing is OK, maybe it is possible to sponsor FOP and cooperatively develop, but I think not this time, maybe after next summer. I'll communicate to sponsorship manager. Tomas
Re: How to use FOP in wysiswyg editor, or how to speed up
Jeremias Maerki wrote: On 27.09.2006 13:41:01 Tomas Studva wrote: snip/ Yes, I know the design goals of FOP are * Conformance to the XSL-FO Specification * Process Files of Arbitrary Size * Minimize Memory Use and to use it in WYSIWYG editor is extend of FOP project scope. Co-developer of Euroamth2, has as diploma thesis theme, design of such rendering (layout, painting, ...) engine usable in WYSIWYG editor. The engine should be abstract enough to be usable for any graphical or textual view, and extendable to be usable for concrete document type, like FO. So he will work and maybe he will cooperate with FOP designers to design a good rendering engine. Something like that is HUGE! Even just supporting XSL-FO properly is half a nightmare. I've once investigated a commercial FO processor which was implemented on top of a general layout engine. They ended up writing a new engine just for XSL-FO because they simply couldn't get to the right quality level. So I'd be careful about the goals you set yourself. Implementation is really HUGE. But framework or concept can be investigated, and maybe some common needs will be found. Maybe really FOP is too hard to fit in any such abstract design, so FOP would be special. But simpler documents maybe fit. [One time, I've had an idea: layout engine configurable by context grammar - simple layout without constraints for math, I haven't thought more on it. ] We look forward to hearing from you again and would be delighted if anyone from your project could assist us improving FOP. Jeremias Maerki We have same needs, more people. If the bigger EM2 community exists, they would use EM2, also FOP in it, and that would be help for testing or more, but that is another story. Tomas
Re: How to use FOP in wysiswyg editor, or how to speed up
This is a very interesting and ambitious project. I would like to see such a project succeed. But WYSIWYG editing is very hard to achieve. That was (and is) the case with TeX, where it was tried, and it is the case with FOP. As Jeremias already answered, FOP is not close to such a goal. We would need a whole lot more developers than we have now, and before that, a good design for this goal. In principle it should be possible to let FOP know which subtree of the FO file has changed, and one could let FOP update that part of the FO tree in memory. It is harder to determine which part of the layout has changed. In addition, FOP has a total fit approach to layout, which is not friendly to a partial update. Currently FOP tries to dispose of elements in memory as soon as possible, in the interest of minimal resource usage. A WYSIWYG approach would require that FOP keeps FO tree, layout elements and perhaps a number of finished pages in memory. Do I remember correctly that EuroMath is developed in Academia? FOP could use a similar sponsorship. Without it, this is far too ambitious for us. Regards, Simon On Mon, Sep 25, 2006 at 09:08:40AM +0200, Jeremias Maerki wrote: On 23.09.2006 23:12:06 Tomá Studva wrote: Hi developers, We are using FOP as rendering engine for FO in wysiwyg xml editor http://sourceforge.net/projects/euromath2. When opening about 40 page fo document, the editation is ugly slow. We can track changes in source document(also in case using XSLT), but there is a principal error, because we don't know how to update the FOP produced trees, so new FOP is created to layout document after change and further render by Draw2d (we implemented some basic renderer composed of Draw2D Figures, which are organized into tree). I' ve also profiled our application on such big document, from typing to update of screen and found out, than FOP consumes 1/3 of processor time, that is too much if we optimize anything but not FOP- it takes on good PC about 10 seconds, so FOP 10/3 = 3sec. So, is there possibility to update the FOP produced trees according to change in XML (to use it in editor manner, not only rendering engine)? No, FOP has not been designed to be used as a backend of a WYSIWYG editor like EuroMath2. That's a new requirement/wish. We'd have to find out if and how FOP can be changed to fulfill it. At any rate it's an expansion of the project scope which could be subject to a project decision depending on how extensive the necessary changes can become. Or if not, is there possibility to recycle FOP PageSequence which haven't changed? Not at the moment, maybe with some hacking. It's not just the page-sequence that would have to be recycled. You'd have to find ways to keep the unchanged page-sequences in memory and that includes not only the FO tree but also the area tree. However, changing a page-sequence in the middle might invalidate later page-sequences. And last, is there option to speed up FOP generally, by lowering output quality or something? Not by lowering the output quality, no. You can help by profiling and optimizing FOP and you can make sure the editor only generates the minimal FO content to produce the right document. Many editor generate much too much (redundant) content not making use of property inheritance. That can have an influence on performance. You're welcome to help us improve FOP to better match the requirements of EuroMath2. I'm pretty sure we currently don't have the resources to help you much in this direction. We can give you pointers and we can help you figure out what needs to be changed. Please note this is my take on the situation and may not reflect the opinion of the whole community. I didn't know of EuroMath2 before your post. If I interpret this correctly it is a content editor rather than an editor to create XSLT stylesheets. I've also taken a peek into your wishlist. I don't think you can currently find any open source FO implementation that is better suited for EuroMath2. Concerning the partial FO rendering possibility: It might be possible to come up with a way to render, say, only a single fo:block-container with relatively little effort. This might have the possibly interesting side-effect that we could write a plug-in for Batik to render FO content within an SVG. :-) Jeremias Maerki -- Simon Pepping home page: http://www.leverkruid.eu
Re: How to use FOP in wysiswyg editor, or how to speed up
On 23.09.2006 23:12:06 Tomá Studva wrote: Hi developers, We are using FOP as rendering engine for FO in wysiwyg xml editor http://sourceforge.net/projects/euromath2. When opening about 40 page fo document, the editation is ugly slow. We can track changes in source document(also in case using XSLT), but there is a principal error, because we don't know how to update the FOP produced trees, so new FOP is created to layout document after change and further render by Draw2d (we implemented some basic renderer composed of Draw2D Figures, which are organized into tree). I' ve also profiled our application on such big document, from typing to update of screen and found out, than FOP consumes 1/3 of processor time, that is too much if we optimize anything but not FOP- it takes on good PC about 10 seconds, so FOP 10/3 = 3sec. So, is there possibility to update the FOP produced trees according to change in XML (to use it in editor manner, not only rendering engine)? No, FOP has not been designed to be used as a backend of a WYSIWYG editor like EuroMath2. That's a new requirement/wish. We'd have to find out if and how FOP can be changed to fulfill it. At any rate it's an expansion of the project scope which could be subject to a project decision depending on how extensive the necessary changes can become. Or if not, is there possibility to recycle FOP PageSequence which haven't changed? Not at the moment, maybe with some hacking. It's not just the page-sequence that would have to be recycled. You'd have to find ways to keep the unchanged page-sequences in memory and that includes not only the FO tree but also the area tree. However, changing a page-sequence in the middle might invalidate later page-sequences. And last, is there option to speed up FOP generally, by lowering output quality or something? Not by lowering the output quality, no. You can help by profiling and optimizing FOP and you can make sure the editor only generates the minimal FO content to produce the right document. Many editor generate much too much (redundant) content not making use of property inheritance. That can have an influence on performance. You're welcome to help us improve FOP to better match the requirements of EuroMath2. I'm pretty sure we currently don't have the resources to help you much in this direction. We can give you pointers and we can help you figure out what needs to be changed. Please note this is my take on the situation and may not reflect the opinion of the whole community. I didn't know of EuroMath2 before your post. If I interpret this correctly it is a content editor rather than an editor to create XSLT stylesheets. I've also taken a peek into your wishlist. I don't think you can currently find any open source FO implementation that is better suited for EuroMath2. Concerning the partial FO rendering possibility: It might be possible to come up with a way to render, say, only a single fo:block-container with relatively little effort. This might have the possibly interesting side-effect that we could write a plug-in for Batik to render FO content within an SVG. :-) Jeremias Maerki
How to use FOP in wysiswyg editor, or how to speed up
Hi developers, We are using FOP as rendering engine for FO in wysiwyg xml editor http://sourceforge.net/projects/euromath2. When opening about 40 page fo document, the editation is ugly slow. We can track changes in source document(also in case using XSLT), but there is a principal error, because we dont know how to update the FOP produced trees, so new FOP is created to layout document after change and further render by Draw2d (we implemented some basic renderer composed of Draw2D Figures, which are organized into tree). I ve also profiled our application on such big document, from typing to update of screen and found out, than FOP consumes 1/3 of processor time, that is too much if we optimize anything but not FOP- it takes on good PC about 10 seconds, so FOP 10/3 = 3sec. So, is there possibility to update the FOP produced trees according to change in XML (to use it in editor manner, not only rendering engine)? Or if not, is there possibility to recycle FOP PageSequence which havent changed? And last, is there option to speed up FOP generally, by lowering output quality or something? Thanx, Tomas Studva
