DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 [EMAIL PROTECTED] changed: What|Removed |Added CC||[EMAIL PROTECTED] -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-31 00:36 --- The TL info caching seems to work reasonably well, after review, so I'll commit it. The Options interface changes are not final. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 [EMAIL PROTECTED] changed: What|Removed |Added CC|[EMAIL PROTECTED] | -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-22 10:20 --- Ok, I've looked at the patch, and I think it looks good. Some of it could maybe be a bit more elegant (in particular, I think the instanceof should be replaced by the use the CacheTldXml flag). For additional optimizations, I think there's room seeing if includes could be cached as well (I'm fairly certain a static include happy webapp behaves as described by Stephane Bailliez in the original report), and of course optimizing hotspots (JspReader/Mark seems to be the biggest offender). -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 [EMAIL PROTECTED] changed: What|Removed |Added Attachment #16126|0 |1 is obsolete|| --- Additional Comments From [EMAIL PROTECTED] 2005-08-22 10:44 --- Created an attachment (id=16139) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16139action=view) Modified patch Small tweaks to make names more generic -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-22 10:45 --- I do have a problem: with the patch, precompilation of the admin webapp fails, so I cannot apply it for the time being. There must be something wrong somewhere. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-22 11:01 --- As the current code only caches the result of the xml parsing it should translates in roughly a 30% speedup for Liferay compilation. Liferay being static include happy, it is a good testcase as well. The xml parsing was the easy part :) As Remy mentions there is load of improvement in there as the JSP parsing is obviously not very efficient. As a side note, I'm nit-picking but I think it would be better to have a single method that returns the TreeNode. here I feel like they are 2 pieces of code related to 'is cache in use'. 1) getOptions().getCachedTldXmlMap() return the map or null 2) if (ctxt.getServletContext() instanceof org.apache.jasper.servlet.JspCServletContext) is used to figure out if it should look into the cache (why not checking if the map is null or not ?) I might be missing something, but this if is confusing me. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-22 11:20 --- (In reply to comment #30) As the current code only caches the result of the xml parsing it should translates in roughly a 30% speedup for Liferay compilation. There's weird stuff going on, though. Maybe I did something wrong when applying the patch. Before trying to debug, I'll try to see if caching the TaglibInfoImpl works better. Liferay being static include happy, it is a good testcase as well. The xml parsing was the easy part :) As Remy mentions there is load of improvement in there as the JSP parsing is obviously not very efficient. As a side note, I'm nit-picking but I think it would be better to have a single method that returns the TreeNode. here I feel like they are 2 pieces of code related to 'is cache in use'. 1) getOptions().getCachedTldXmlMap() return the map or null 2) if (ctxt.getServletContext() instanceof org.apache.jasper.servlet.JspCServletContext) is used to figure out if it should look into the cache (why not checking if the map is null or not ?) I might be missing something, but this if is confusing me. Yes, it could be better. I assume the cache map will eventually be used for more than TaglibInfoImpl. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 [EMAIL PROTECTED] changed: What|Removed |Added Attachment #16139|0 |1 is obsolete|| --- Additional Comments From [EMAIL PROTECTED] 2005-08-22 11:43 --- Created an attachment (id=16140) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16140action=view) Modified patch 2 This patch will cache TagLibraryInfoImpl objects rather than the XML objects. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-22 16:16 --- (In reply to comment #32) Created an attachment (id=16140) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16140action=view) [edit] Modified patch 2 This patch will cache TagLibraryInfoImpl objects rather than the XML objects. I looked at this patch, but I am afraid that the TagLibraryInfoImpl cannot be cached in Parser.java and JspDocumentParser.java in this simple way. In TagLibraryInfoImpl the individual CompilerContext is used for each jsp precompilation specifically. I guess that's why it didn't work on the admin app precompilation. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-22 16:33 --- (In reply to comment #30) As the current code only caches the result of the xml parsing it should translates in roughly a 30% speedup for Liferay compilation. In my testcase with liferay source package (more specifically, all jsp files under portal-web/docroot), I could get more than 30% speedup. I did a series of experiments and the results were: 1. Dell Dimension 4600, 2.8GHz, 768M: time reduced from ~130s to ~65s. 2. one cluster node, AMD Athlon(tm) Processor, 908MHz, 768M: time reduced from (12m + 50s) to (2m + 55s). It depends on the machines but my cases show significant improvement. Liferay being static include happy, it is a good testcase as well. The xml parsing was the easy part :) As Remy mentions there is load of improvement in there as the JSP parsing is obviously not very efficient. As a side note, I'm nit-picking but I think it would be better to have a single method that returns the TreeNode. here I feel like they are 2 pieces of code related to 'is cache in use'. 1) getOptions().getCachedTldXmlMap() return the map or null 2) if (ctxt.getServletContext() instanceof org.apache.jasper.servlet.JspCServletContext) is used to figure out if it should look into the cache (why not checking if the map is null or not ?) I might be missing something, but this if is confusing me. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-22 16:34 --- (In reply to comment #33) I looked at this patch, but I am afraid that the TagLibraryInfoImpl cannot be cached in Parser.java and JspDocumentParser.java in this simple way. In TagLibraryInfoImpl the individual CompilerContext is used for each jsp precompilation specifically. I guess that's why it didn't work on the admin app precompilation. No, the latest patch I posted doesn't fail, for some reason (the first patch I attached before did, but it might have been a build clean issue or something, as the problem disappeared magically). I know there's a compiler context field which might possibly cause issues. It could be investigated if it really does. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-21 17:27 --- (In reply to comment #25) Created an attachment (id=16128) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16128action=view) [edit] cpu profiling snapshot by class (call tree) after the change After the patch, the TagLibraryInfoImpl would take much less CPU time, less than 5%. As it's inconvenient to post the profiling snapshots here (one at a time), I would post the rest of them on my personal website to show the differences. I'll post the URL when it's done. The URL is http://www.cs.ucf.edu/~xbgao/jasper/jasper-33650.htm. It contains 12 screenshots in total. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-19 22:25 --- Created an attachment (id=16121) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16121action=view) modified Options targeted at jasper performance improvement -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-19 22:32 --- Created an attachment (id=16122) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16122action=view) modified JspC targeted at jasper performance improvement -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-19 22:37 --- Created an attachment (id=16123) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16123action=view) modified EmbeddedServletOptions targeted at jasper performance improvement -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-19 22:38 --- Created an attachment (id=16124) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16124action=view) modified TagLibraryInfoImpl targeted at jasper performance improvement -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-19 22:55 --- (In reply to comment #17) Created an attachment (id=16121) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16121action=view) [edit] modified Options targeted at jasper performance improvement You should submit patches in diff format (cvs diff). It's too difficult to evaluate the patch when you just post the entire modified file. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-20 04:12 --- (In reply to comment #21) (In reply to comment #17) Created an attachment (id=16121) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16121action=view) [edit] [edit] modified Options targeted at jasper performance improvement You should submit patches in diff format (cvs diff). It's too difficult to evaluate the patch when you just post the entire modified file. Thank you. I will replace the format. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 [EMAIL PROTECTED] changed: What|Removed |Added Attachment #16121|0 |1 is obsolete|| Attachment #16122|0 |1 is obsolete|| Attachment #16123|0 |1 is obsolete|| Attachment #16124|0 |1 is obsolete|| --- Additional Comments From [EMAIL PROTECTED] 2005-08-20 04:18 --- Created an attachment (id=16126) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16126action=view) proposed patches for Options.java, JspC.java, EmbeddedServletOptions.java and TagLibraryInfoImpl.java in cvs diff format This patch is about caching the TreeNode tld in TagLibraryInfoImpl. Because the profiling results show the repeating parsing of the same TLD has dominated the precompilation process, this change would make the jasper works much better on a large Ant build :-) (I would also submit the different profiling snapshots before and after the code changes). Here is what I basically did: 1. Options.java: add two public functions in Options interface. One is isCacheTldXml(), and the other is Map getCachedTldXmlMap(). The parsed TLD XML data is cached in a Map(String uri, TreeNode tld). 2. JspC.java: add switch -cacheTldXml and implement the two functions defined in Options. The cacheTldXml is defaulted to true but the users can set it. 3. EmbeddedServletOptions.java: add boolean cacheTldXml and implement the two functions. As EmbeddedServletOptions is called by run-time compilation, cacheTldXml is set to false by default. 4. TagLibraryInfoImpl.java: add an if-else clause in parseTld to distinguish command-line build and run-time build. If the CompilerContext is JspC, try to get the TreeNode from cache first; otherwise call parseXMLDocument to parse the TLD directly. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-20 04:40 --- Created an attachment (id=16127) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16127action=view) cpu profiling snapshot by class (call tree) before the change The testbed is the Liferay Portal Source (www.liferay.com). It is a quite big and 464 Java source files would be created. The build machine is a typical Dell 4600 desktop with Fedora Core 4, 2.8GHz, 768M. Before the patch, the TagLibraryInfoImpl class takes more than 50% CPU time. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-08-20 04:46 --- Created an attachment (id=16128) -- (http://issues.apache.org/bugzilla/attachment.cgi?id=16128action=view) cpu profiling snapshot by class (call tree) after the change After the patch, the TagLibraryInfoImpl would take much less CPU time, less than 5%. As it's inconvenient to post the profiling snapshots here (one at a time), I would post the rest of them on my personal website to show the differences. I'll post the URL when it's done. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-07-30 11:09 --- another aspect to this: Bug 31645 -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 [EMAIL PROTECTED] changed: What|Removed |Added CC||[EMAIL PROTECTED] Status|ASSIGNED|NEEDINFO --- Additional Comments From [EMAIL PROTECTED] 2005-06-07 10:44 --- (In reply to comment #11) I've made this item a candidate for the Summer of Code: http://wiki.apache.org/general/SummerOfCode2005. I followed this bugzilla from the SummerOfCode2005 link. This looks like an interesting optimization. Is there any whitepaper/architecture document that I can look at? I would surely enjoy taking up this challenge, contest or no contest. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-06-01 09:03 --- I was just looking at it. :) Good thing, Yoav, I unfortunately spend my time switching priorities due to the current nature of my work, so I hope someone will take over, there sure is a nice opportunity. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-06-02 06:27 --- (In reply to comment #11) I've made this item a candidate for the Summer of Code: http://wiki.apache.org/general/SummerOfCode2005. I have applied to participate in this project during google summer of code. If I am accepted or not I am still very interested in this issue. I am a student intern at a company and during my time there I have been involved in development of a very large web application involving several hundred jsps. I have encountered this issue during development / deployment and would enjoy learning more about this issue and finding a clean optimized solution. Any suggestions on where to start? Feel free to email me. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 [EMAIL PROTECTED] changed: What|Removed |Added Status|NEW |ASSIGNED --- Additional Comments From [EMAIL PROTECTED] 2005-06-01 04:26 --- I've made this item a candidate for the Summer of Code: http://wiki.apache.org/general/SummerOfCode2005. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] New: - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
http://issues.apache.org/bugzilla/show_bug.cgi?id=33650.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=33650
Summary: Jasper performance for multiple files processing
Product: Tomcat 5
Version: 5.0.30
Platform: PC
OS/Version: Windows XP
Status: NEW
Severity: enhancement
Priority: P2
Component: Jasper
AssignedTo: tomcat-dev@jakarta.apache.org
ReportedBy: [EMAIL PROTECTED]
Jasper performance is abysmal when using jspc to precompiles jsps, or more
especially to generate the java files.
I have a webapp with 450 jsps, jasper generates the java files in 12 minutes.
In comparison, javac compilation for those 450 generated java files is
performed in 1 minute.
Profiling and debugging the code, it appears clearly that most of the time is
spent doing the same parsing for static include directives over and over.
Typically a page includes an init page which itself includes 2 init pages.
page.jsp - init.jsp - { init-a.jsp, init-b.jsp, ... }
For each and every page, the parser always go down the tree without caching
includes and parse init.jsp, and so on. So basically the init.jsp is parsed
450 times.
The Mark object is also pretty inefficient as it generates bazillions of
objects which are not so lightweight (via Mark.mark()).
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the assignee for the bug, or are watching the assignee.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 [EMAIL PROTECTED] changed: What|Removed |Added Version|5.0.30 |5.5.7 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 11:53 --- Ok. Does this mean you have done some optimizations already (patches) ? Do you have test cases ? (JSPs which are particularly slow to compile) Note: This will looked at in 5.0.x (better to have a slow source generation than a *broken* source generation). -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 12:09 --- I'm currently hacking the jasper code to try to understand it as i'm not very familiar with the code and need to understand relations between objects and particulary how nodes are attached to each other, how dependant files are added and resolved, etc... it is a bit of obscure right now. I'm not sure yet what would be the best place to cache those include: ParserController, Parser, .. and I think I need to make those Node classes mutable to get control over the parent and reset/set it easily. I guess that for anyone familiar with the code, the view is pretty clear, mine is not at the moment. My current testcase is the liferay portal sources (www.liferay.com), it needs a bit of doc to setup so it's not convenient for an independant testcase. (NB: If I reported against 5.5.7 it is a mistake and is indeed 5.0.x) -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 13:47 --- I made the mistake. As I said, this will not be addressed (as in -1 = veto) in 5.0.x. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 14:00 --- Ah ok, did not really understood it that way since I consider it is 'better to have a slow source generation than a *broken* source generation' in any situation and not just in 5.0.x :). Thus the qualification as 'enhancement'. Nonetheless, I'm still trying to understand the code. I have made little progress as I realized I needed to delta the dependencies that are added through the Compiler instance if I want to cache the includeDirective fragment. I still need to find how are the taglibs evaluated to generate the java code. At this time it looks like I could dream sort of a 10x improvement if this sort of caching can be done. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 14:20 --- My comment was indeed missing two words. BTW, regardless of what your profiler says, I don't think you can get 10x. However, now that the runtime performance is good, I'm interested in optimizing compile times. One part has been done already by replacing the compiler by one which decreses I/O (although the main benefit was to get compilation on the JRE instead of the JDK). -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 14:47 --- The compilation process is an external compilation process with ant javac task. I'm not using the internal one as for compilation within jspc it is often inappropriate and it is not flexible enough for file selection, classpath, and jvm settings. I'm using this because many people tend to name jsp include with .jsp extensions and it will f* up blind attempt to compile from Jasper as they will be generated to java and it is invalid code by essence. Workaround process is usually. 1. jspc (.jsp - .java) 2. delete generated static includes based on text content selectors (looking for unevaluated taglibs syntax for example) 3. javac everything I'm myself always recommending people to name their includes as jspf (or .inc) extensions as it is: - visually more convenient - Jasper does not choke on it - JSP editors will still understand the syntax (except for .inc) I would really like to enhance this Ant jspc task but this is in my mile long todo list. As for the speed up, I cannot see any reason why it should be 10x slower than java compilation (2-3s for a single jsp-java is huge). but really, all I'm missing is the taglibs evaluation and the process apparently should be able to fly NB: For I/O it should not cost much to wrap BufferedOutputStream over the FileOutputStream. (I did not check if there was any change on this in cvs as I playing with 5.0.30 sources) but really it is peanuts compared to the zillions of I/O from constant loading and parsing of includes and tlds. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 17:01 --- FYI, putting a simple hashmap to cache the TLDs TreeNode to avoid the DOM parsing of TLDs in TagLibraryInfoImpl for each and every JSP brings the timing to 8 minutes instead of 12 on my 'testcase'. A 30% improvement. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 17:09 --- You still need to play nice with regular Jasper mode, when compilation occurs on access. I would complain if there are memory leaks, and/or abusive memory usage ;) If code generation times go down significantly, and since JDT is used to compile (fast and does very little I/O), I would assume it would encourage many people to not bother with precompilation (hence the need to avoid leaks and huge memory usage). -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 17:11 --- (In reply to comment #7) FYI, putting a simple hashmap to cache the TLDs TreeNode to avoid the DOM parsing of TLDs in TagLibraryInfoImpl for each and every JSP brings the timing to 8 minutes instead of 12 on my 'testcase'. A 30% improvement. Nice improvement. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 33650] - Jasper performance for multiple files processing
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=33650. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=33650 --- Additional Comments From [EMAIL PROTECTED] 2005-02-19 18:04 --- You still need to play nice with regular Jasper mode, when compilation occurs on access. I would complain if there are memory leaks, and/or abusive memory usage ;) I think the cache needs to be enabled only in command line mode (batch compile). Otherwise it does not make much sense and it complicate things when you are development mode. I would assume it would encourage many people to not bother with precompilation It is totally different. Pre-compiling your jsp is mostly a way to check that there is no syntax or configuration problems with your taglibs or jsp. You don't care much with a petstore webapp with 10 jsps that you can check manually at deployment time, but with hundred of thems, it is not something you can afford. Assuming you have some QA dept or some automated robot to go through all your webpages, that's also useless to deliver something that does not even compile. So it's mostly a development thing and must be part of your process, and it should be part of the daily build as well of course. So if we can cut down this check from 12 to 1min, believe me I think some people would love it. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Jasper performance query (circa 3.1 code)
I'm using Jasper in Weblogic from tomcat 3.1 (ish, we've patched a couple of things too). While profiling some of my JSP's, I see the JspRuntimeLibrary.introspectHelper consuming between 40-70% of the time, and a large %age of the object creation. Drilling down, most of the time is the line (which is still in Jasper34) java.beans.BeanInfo info = java.beans.Introspector.getBeanInfo(bean.getClass()); Which in turn is slow due to the construction of the BeanInfo (concrete class: GenericBeanInfo) So my question is: why isn't this cached? Or rather, if I were to cache this object, what are my problems? Those I see so far are: 1) If it's cached, where is the cache held? (a) If within JspRuntimeLibrary, I must synchronize the Map -- may cause a bottleneck between threads, since this is called hundreds of times per JSP. (b)If not here, then perhaps on the thread -- we're using our own subclass at this point (most of the time) so I could store it as a thread local there, or change the generation to pass in a storage object / cache. 2) Will this prevent class reloading from GC'ing the old classes, unless the map is released? For our app, reloading is only a dev feature so not too worried about this. 3) Am I reading my profiler wrong and this shouldn't be a bottleneck? I don't think we can wait long enough for the Jasper refactoring to happen, so am looking for smallish changes currently. Ken. This e-mail message is CONFIDENTIAL and may contain legally privileged information. If you are not the intended recipient you should not read, copy, distribute, disclose or otherwise use the information in this e-mail. Please also telephone or fax us immediately and delete the message from your system. E-mail may be susceptible to data corruption, interception and unauthorised amendment, and we do not accept liability for any such corruption, interception or amendment or the consequences thereof.
Re: Jasper performance/3.3 tag pooling
Hey Mel, I'll use this as a chance to explain some thoughts I've recently had on tag pooling. Maybe you and others have comments. Mel Martinez wrote: Hi folks! I'm still overwhelmed with other priorities (new job, house-hunting, moving, etc.) to be able to help again, but I managed over the last day or so to get caught up with the list (for the most part - skipped some major threads along the way). Hope things are working out well. I know where you're coming from. We just moved and are now in the process of putting in a yard. On tag-pooling: I am +1 on implementing a tag-cache on a per-page basis and tag-pooling on an application basis. If/when we move to the interfaces and toolkit metaphor I proposed for Jasper34, then page-based caching can best be done by enabling access to a tag-cache via the page-life-cycle handler, the JspPageHandler interface. I agree. I spent some time last week looking at possible optimizations. The general ideas were: - pool tag handler objects per application. This could still be turned on/off at runtime via module and is already available. - cache (re-use) handlers per page - i.e. only get the handler from the pool once (when it is needed). If application wide pooling is disabled then we'll just new a handler. To be clear: there won't be any data structures (stacks, etc.) for caching -- it will just be done via the rendering. - call most setters only once -- when the tag handler is first obtained. Attributes that are runtime expressions will need to be called every time. - in some situations we can pull out the tag handler initialization code from inside do loops. Currently, if a Tag is inside of a BodyTag, the initialization for the inner tag is done inside of the do-while(BodyTag.doEndBody() == EVAL_BODY_TAG). The inner tag initializations only need to be done once before the start of the do-while. Pulling the inner initializations/checks out of the do loop would be a nice thing, but it won't be as important once the tags and setter calls are re-used for the whole page. - maybe get rid of some redundant do-while(false) constructs and various other minor clean ups. So, to summarize: Each tag handler will be obtained once per page (either from pool or via new). Setters will be called once per handler per page. Both obtaining the handler and calling the setters will be done on an as needed basis. Yes, that will mean a synch call to get each handler when it's first used if application wide pooling is enabled. So far, I was thinking that there would still be one tag handler object per custom tag in the jsp page. For example, given: x:iterate count=100 x:someTag/ /x:iterate x:someTag/ there would still be 3 tag handler instances obtained. The spec says the someTag handler could be reused because tag scope allows it. But then we have the question of when to reuse handlers if scope allows it. For example, if the above was changed to: x:iterate count=100 x:someTag attrib=abc/ /x:iterate x:someTag attrib=xyz/ We could reuse the handler instance but would have to call the setters for attrib each time. What's more expensive, setter call or just using a different handler? Of course this depends on tag usage and what the setters are doing. So, for now, I was planning on not implementing tag handler reuse across multiple uses of a custom tag in the same jsp. This can, of course, be improved in the future. I am very much -1 on basing any sort of caching or pooling relative to the request thread using thread-local variables because it assumes the Thread will be pooled and reused - this may not be true in all containers and Jasper should be container-independent. I agree. After the above stuff gets done, I hope that we won't have to do any thread local variable stuff. I originally thought such optimizations would be nice, but am now hoping that they won't be needed. I'd like to make sure that all this stuff will (eventually) work with tomcat 4 and future tomcats as well. Basically, one should think in terms of what temporal and access scope different objects have: request, page, application, etc. The JspPageHandler persists between requests for a page and should be used to represent access to objects and services concerned with the life-cycle of a page beyond one request, such as modification checking, name mangling, recompilation/loading as needed and clearly, tag-object caching. Thus, it seems logical that tags could be retrieved via either directly from methods on the page handler or from a tag cache manager object retrieved from the page handler. Casey, do you see any problems following such a general design philosophy? You are the lord gawd of the tag-pooling code, so I will defer to your judgement. I agree with the general design philosophy. But don't defer to much :) (The more I learn about Tomcat the more I realize how much I don't know.) I haven't looked though all the new
Re: Jasper performance/3.3 tag pooling
I agree. I spent some time last week looking at possible optimizations. The general ideas were: - pool tag handler objects per application. This could still be turned on/off at runtime via module and is already available. - cache (re-use) handlers per page - i.e. only get the handler from the pool once (when it is needed). If application wide pooling is disabled then we'll just new a handler. To be clear: there won't be any data structures (stacks, etc.) for caching -- it will just be done via the rendering. - call most setters only once -- when the tag handler is first obtained. Attributes that are runtime expressions will need to be called every time. If you can get only these three in, and only that, it would help at least me/WebWork *immensely*. Heeding your advice regarding the examples I sent to you I've rewritten all tags to be reusable, and allow set-methods to be called only once, so I should see a huge improvement in performance. If it still performs bad, it's either my fault, or the rest of Tomcat :-) regards, Rickard
Re: Jasper performance/3.3 tag pooling
Thanks for sending the test application. I've been using monthlist.jsp while testing rendered then hand bashed code. After some preliminary bashing :) I think it will be possible to call all setters (except runtime expressions) only once per run of the page. Also, we can create / obtain tag handler instances only once per page run. That is an excellent start! Anyway... after tweaking the rendered code, I was disappointed that there wasn't much of a change in the performance when I hit the page directly. The optimized version was even a bit slower :( So, I hooked it up to a profiler. It said that most of the time was being spent in webwork.util.ValueStack.findValue which is called from some of the setters and some of the do[Start|End]Tag methods. Yeah, it's the by far most used method, and also where most of the magic happens (name-value conversions). I've been tweaking it some, but probably not enough. So, Is there any way you could send me a new webwork.jar (no need for the whole .war) that has tag implementations optimized for tag instance and setter call reuse?. If you can move some/many of the calls to ValueStack.findValue out of the doStart/doEnd methods, that should help. I will start working on this after the weekend. However I am also rewriting most of the tags for a new and simpler syntax, which is going to take some coding time too, since all the examples need to be modified, docs, etc. I will let you know as soon as possible though when the opts. are done. Yes, I know this doesn't solve the problem of why weblogic was faster for you, but any tag related optimizations we can make should help other tag users (me and my day job included). Yup, that was the basic idea when I started this thread :-) Glad to see things happen! :-) If you send a new jar, I'll test it out and let you know. Will do. /Rickard
Re: Jasper performance/3.3 tag pooling
[EMAIL PROTECTED] wrote: Jasper performance is a high priority thing for us (nr 1. on our system performance fixing list actually), so if possible, absolutely. I can't say I've delved that far into Jasper yet though. It was a bit hard to read. I'm working on a refactoring of jasper, and easy to read is a big priority. It's moving a bit slower than I expected - now I'm back on planning stage after hearing so much bad things about using XSLT :-) But I think there are ways to make ( almost ) everyone happy. I hope in few weeks ( after JavaOne ) we'll have something that works for both 1.1 and 1.2 ( the first part of refactoring ), and we can start the fun part - optimizing it. Sounds like a good plan :-) I'd be happy to help out with commenting on the code. Code contribution is tricky right now (lots of other OpenSource projects to manage :-). The main problem is to avoid one synchronized call for each tag ( new _is_also_ a synchronized call on most VMs - it synchronizes the heap !!! ). Whoa.. I didn't know that.. that's an important factor to consider then. Tomcat3.3 has one synchronized call to get the worker thread from the pool - and that doesn't seem to show up in any performance tests. As long as the synchs are few, and the block itself is short, you should be ok. We could keep a pool of pools ( probably for each context ). The local pools can be unsynchronized - so we'll have only one sync per page. Each context will have a set of pools ( with the size == the largest number of concurent requests for that context ). And we can of course be agressive in shrinking the pools that were not used recently. That could work. We can try to do some of this in 3.3 space - or in a branch of 3.3 ( jasper34 is going to take a while, and I have a feeling you need results fast ). The big date for me is 2002-01-01. Before that I need some kind of assurance that it'll improve greatly 'till that time, or we need to look at alternatives (*shiver*). So sooner than later, sure, but preferably righter than sooner. Get this stuff right, and I think you'll get lots of new supporters and keep old ones ;-) Regarding your application - I think we know the problem. There are few things you can do to reduce the memory use, and you could cache and recycle intermediary objects in each tag instance - you could benefit a lot from the reuse of the tags. I will start looking at optimizations on my side of things to, for sure. My code isn't extremely optimized, I know that, but OTOH I've tested the same code with WebLogic 5.0 (just to test) and it was wy faster, so there's definitely room on the JSP side to improve things. The memory savings of the thread pooling are not helping your application yet ( since they are very small compared with the rest ), yet you pay the synchronization price ( a lot - since you make heavy use of tags ). With few changes in your app and few changes in the tag pool we could reverse the situation - but if you disable the pool there's little chance you could increase the performance above the current level. Yup, that seems reasonable. You mentioned in your private email that there was a lot of MethodDescriptor's being created. Please look into where these are coming from. They could be associated with the tag setters. Pool/cache these and you'll probably see some immediate results. /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
RE: Jasper performance/3.3 tag pooling (XSLT)
Hi Costin, -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 23, 2001 10:20 AM ... I'm working on a refactoring of jasper, and easy to read is a big priority. It's moving a bit slower than I expected - now I'm back on planning stage after hearing so much bad things about using XSLT :-) But I think there are ways to make ( almost ) everyone happy. ... Considering your day job, I already had to change my diagnostic from you don't know what you are messing with to you know too much about XSLT and then you think it is easy for everybody. Still, I do not believe that using XSLT is a good idea. It goes a bit against your own motivation of making Jasper more readable to make the entry level lower for potential contributors. Besides learning about Jasper, many will have to learn about XSLT. And XSLT is not that easy to grasp. Worse, it is elusive - one tends to think to have it all figured out before time, and then fall easy prey of really big traps. I am not going to argue with you about the CPU cost issue, since your POV seems to be that everybody should manually compile JSPs for production and that developers have good enough machines anyway. We could start another holly war on this one, but I do not have the motivation. I think that many people prefer doing it without manual compilation, but we better just agree to disagree on that. Anyway, for me, the main issue is the complexity of XSLT being an obstacle for Jasper's code readability/maintenance. In case you are wondering, I do not intend to use JSP for presentation (I keep that Velocity is better at that) but I still think it as a lot of potential as a server side scripting mechanism, with taglibs providing an interesting way of defining simple dialects for common problems and embedded Java filling the gaps. For simple logic, it looks to have potential. Have fun, Paulo Gaspar
Re: Jasper performance/3.3 tag pooling
Rickard, Thanks for sending the test application. I've been using monthlist.jsp while testing rendered then hand bashed code. After some preliminary bashing :) I think it will be possible to call all setters (except runtime expressions) only once per run of the page. Also, we can create / obtain tag handler instances only once per page run. Anyway... after tweaking the rendered code, I was disappointed that there wasn't much of a change in the performance when I hit the page directly. The optimized version was even a bit slower :( So, I hooked it up to a profiler. It said that most of the time was being spent in webwork.util.ValueStack.findValue which is called from some of the setters and some of the do[Start|End]Tag methods. So even though the setters and tag handler news were optimized, there wasn't much change in page evaluation time. (A strange note: under the profiler, the optimized version ran in about 75% of the time it took the un-optimized version. Not sure why the big difference, but I have seen variations like that before when code is run under a profiler.) So, Is there any way you could send me a new webwork.jar (no need for the whole .war) that has tag implementations optimized for tag instance and setter call reuse?. If you can move some/many of the calls to ValueStack.findValue out of the doStart/doEnd methods, that should help. Yes, I know this doesn't solve the problem of why weblogic was faster for you, but any tag related optimizations we can make should help other tag users (me and my day job included). If you send a new jar, I'll test it out and let you know. -casey Rickard Öberg wrote: [EMAIL PROTECTED] wrote: Jasper performance is a high priority thing for us (nr 1. on our system performance fixing list actually), so if possible, absolutely. I can't say I've delved that far into Jasper yet though. It was a bit hard to read. I'm working on a refactoring of jasper, and easy to read is a big priority. It's moving a bit slower than I expected - now I'm back on planning stage after hearing so much bad things about using XSLT :-) But I think there are ways to make ( almost ) everyone happy. I hope in few weeks ( after JavaOne ) we'll have something that works for both 1.1 and 1.2 ( the first part of refactoring ), and we can start the fun part - optimizing it. Sounds like a good plan :-) I'd be happy to help out with commenting on the code. Code contribution is tricky right now (lots of other OpenSource projects to manage :-). The main problem is to avoid one synchronized call for each tag ( new _is_also_ a synchronized call on most VMs - it synchronizes the heap !!! ). Whoa.. I didn't know that.. that's an important factor to consider then. Tomcat3.3 has one synchronized call to get the worker thread from the pool - and that doesn't seem to show up in any performance tests. As long as the synchs are few, and the block itself is short, you should be ok. We could keep a pool of pools ( probably for each context ). The local pools can be unsynchronized - so we'll have only one sync per page. Each context will have a set of pools ( with the size == the largest number of concurent requests for that context ). And we can of course be agressive in shrinking the pools that were not used recently. That could work. We can try to do some of this in 3.3 space - or in a branch of 3.3 ( jasper34 is going to take a while, and I have a feeling you need results fast ). The big date for me is 2002-01-01. Before that I need some kind of assurance that it'll improve greatly 'till that time, or we need to look at alternatives (*shiver*). So sooner than later, sure, but preferably righter than sooner. Get this stuff right, and I think you'll get lots of new supporters and keep old ones ;-) Regarding your application - I think we know the problem. There are few things you can do to reduce the memory use, and you could cache and recycle intermediary objects in each tag instance - you could benefit a lot from the reuse of the tags. I will start looking at optimizations on my side of things to, for sure. My code isn't extremely optimized, I know that, but OTOH I've tested the same code with WebLogic 5.0 (just to test) and it was wy faster, so there's definitely room on the JSP side to improve things. The memory savings of the thread pooling are not helping your application yet ( since they are very small compared with the rest ), yet you pay the synchronization price ( a lot - since you make heavy use of tags ). With few changes in your app and few changes in the tag pool we could reverse the situation - but if you disable the pool there's little chance you could increase the performance above the current level. Yup, that seems reasonable. You mentioned in your private email that there was a lot of MethodDescriptor's being created. Please look into where
Re: Jasper performance/3.3 tag pooling
[EMAIL PROTECTED] wrote: We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That is one solution, but what do you do with the pool after page request? I'm not sure I understand. Each thread has a number of associated objects that are recycled and reused - a Request object will stick with a thread. Same can be done for the tag pools - except that this may use a lot of memory ( but less than allocating/freeing ). It is possible to use a middle ground, or tune this - but for maximum performance you can have a local pool in each Request. What I was considering is this: a pool is a managed resource, since it can shrink and grow. After a page request, you will want to reuse the pool and its contents, but you can't stick the contents into a global pool, because at next request how are you to know how many instances of a tag to put in the request-associated pool. I.e., thread-associated pools perform local optimizations per request/page, but for pools to be of any use they need to do global optimizations, i.e. pool the tags in a global pool. Otherwise you'll have lots of pools of tags and no way to manage them globally. Example: Page request. No pools created yet. Page uses tag foo 10 times. Pool, associated with request/thread, created containing one foo tag instance. Concurrent page request with the above. Page uses tag foo 5 times. Another pool, associated with request/thread, created containing one foo tag instance. Both requests end: what to do with the pools? 1 Keep separate, each containing one instance 2 Merge so that there is only one pool for foo tags If 1, then there's no way to do proper global optimizations, i.e. say don't create more than x tag instances. If 2, then at next request, what should the page associated foo pool contain? 1 tag instance, 2 tag instances..? The only way to really know is for the page to do its thing, and pull the tags from the global pool. And then you're in synch land again. Do you understand now? I realize the above is a bit fuzzy... I hope that Costin will be able to reproduce what I found. I hope not :-) Again - thanks for doing the tests and checking the code, and hope to see more contributions and maybe few patches :-) Jasper performance is a high priority thing for us (nr 1. on our system performance fixing list actually), so if possible, absolutely. I can't say I've delved that far into Jasper yet though. It was a bit hard to read. IMHO the right answer is depends. And it depends on the actual use case, on how many objects are created and where the synchronization occurs. For tomcat, where we expect hundred of concurent requests and each would create almost a hundred object ( that was the case in 3.0 ) - I doubt any VM could make a difference. Have you then considered the middle way I proposed, where each page creates the tags it needs, but only once. In my experience, the performance kills is in iterative and recursive use of tags, e.g.: iterate foo/ /iterate which will create one foo tag per iteration in 3.2. *That's* the biggest problem IMHO. Creating the iterate and foo tag once for that page is not, considering that the overhead of managing pools of these tag instances is more resource draining, and hard to do properly on a global scale. I might be wrong though 8-) Nah. :-) /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
[EMAIL PROTECTED] wrote: But in tomcat 3.3 we do a different trick - the thread pool is maintaining a local storage for each thread, and it's passing it to the worker. The only synchronization in tomcat is in getting a thread from the thread pool - besides that we shouldn't need anything else. Right now we keep the Request/Request pair - so we have a one-to-one relation between main request and thread ( in other words, the request is allways in the same thread ). Whatever attributes/notes you store in the request will be equivalent with thread-local data, without any sync. That's good. But you still have the global vs local issue outlined in my last post. Purely local, as you outline above, and proper pooling management becomes tricky (=how to know how many tag instances to use for the particular handled request). Sorry, I should have been more specific. Of course object reuse can be a good performance optimization. I'm just saying that you gotta balance it with regard to modern JVM's ability to manage memory and new objects. Optimize locally in code, and let the JVM do it globally. That's just IMHO though, so feel free to disagree :-) :-) I was thinking the reverse - optimize globally, let the VM optimize locally, but it depends on definitions... The problem is that global management is a tricky business. Say you keep global pools of tag instances. Not only do you have to manage one particular tag pool properly (not too big, not too small), you also have to consider how many other pools are active (too many active pools with too many tags, and you have a memory consumption problem). Then factor in the temporal issues, i.e. that some tags are used more often than others sometimes, and you have a situation that I dare you to find an optimal algorithm to solve the pool management. And when/if you have that algorithm, I can bet you quite a bunch of dolares that it won't be more efficient than letting the JVM manage the memory and making sparse new's per page anyway. Or (...standard disclaimer...) what am I missing? :-) /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
Rickard Öberg wrote: [EMAIL PROTECTED] wrote: We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That is one solution, but what do you do with the pool after page request? I'm not sure I understand. Each thread has a number of associated objects that are recycled and reused - a Request object will stick with a thread. Same can be done for the tag pools - except that this may use a lot of memory ( but less than allocating/freeing ). It is possible to use a middle ground, or tune this - but for maximum performance you can have a local pool in each Request. What I was considering is this: a pool is a managed resource, since it can shrink and grow. After a page request, you will want to reuse the pool and its contents, but you can't stick the contents into a global pool, because at next request how are you to know how many instances of a tag to put in the request-associated pool. I.e., thread-associated pools perform local optimizations per request/page, but for pools to be of any use they need to do global optimizations, i.e. pool the tags in a global pool. Otherwise you'll have lots of pools of tags and no way to manage them globally. Example: Page request. No pools created yet. Page uses tag foo 10 times. Pool, associated with request/thread, created containing one foo tag instance. Concurrent page request with the above. Page uses tag foo 5 times. Another pool, associated with request/thread, created containing one foo tag instance. Both requests end: what to do with the pools? 1 Keep separate, each containing one instance 2 Merge so that there is only one pool for foo tags If 1, then there's no way to do proper global optimizations, i.e. say don't create more than x tag instances. If 2, then at next request, what should the page associated foo pool contain? 1 tag instance, 2 tag instances..? The only way to really know is for the page to do its thing, and pull the tags from the global pool. And then you're in synch land again. Do you understand now? I realize the above is a bit fuzzy... I hope that Costin will be able to reproduce what I found. I hope not :-) Again - thanks for doing the tests and checking the code, and hope to see more contributions and maybe few patches :-) Jasper performance is a high priority thing for us (nr 1. on our system performance fixing list actually), so if possible, absolutely. I can't say I've delved that far into Jasper yet though. It was a bit hard to read. IMHO the right answer is depends. And it depends on the actual use case, on how many objects are created and where the synchronization occurs. For tomcat, where we expect hundred of concurent requests and each would create almost a hundred object ( that was the case in 3.0 ) - I doubt any VM could make a difference. Have you then considered the middle way I proposed, where each page creates the tags it needs, but only once. In my experience, the performance kills is in iterative and recursive use of tags, e.g.: iterate foo/ /iterate which will create one foo tag per iteration in 3.2. *That's* the biggest problem IMHO. Creating the iterate and foo tag once for that page is not, considering that the overhead of managing pools of these tag instances is more resource draining, and hard to do properly on a global scale. I just had an idea (dangerous things) regarding tag pooling optimizations. When Jasper translates a page it should be able to generate information about which tag handler classes it needs, and for each tag handler, the profile of the attribute/value pairs. At runtime Jasper could make one call to a tag pooling synchronized method to get all available instances of the tag handlers it needs. Any tag handlers/attribute profiles it couldn't obtain, it would have to create. Then it would manage its own local tag pool during the execution of the request. On termination of the request, after the page has been committed to the remote client, it would make one call to a tag pooling synchronized method to recycle/add the tag handlers it used. So there would only be 2 synchronized methods per page for implementing tag pooling regardless of how many tags are used. And you have the benifits of minimizing JVM memory usage by sharing a global JVM tag pool. Regards, Glenn -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance/3.3 tag pooling
Glenn Nielsen wrote: I just had an idea (dangerous things) regarding tag pooling optimizations. When Jasper translates a page it should be able to generate information about which tag handler classes it needs, and for each tag handler, the profile of the attribute/value pairs. At runtime Jasper could make one call to a tag pooling synchronized method to get all available instances of the tag handlers it needs. Any tag handlers/attribute profiles it couldn't obtain, it would have to create. Then it would manage its own local tag pool during the execution of the request. On termination of the request, after the page has been committed to the remote client, it would make one call to a tag pooling synchronized method to recycle/add the tag handlers it used. So there would only be 2 synchronized methods per page for implementing tag pooling regardless of how many tags are used. And you have the benifits of minimizing JVM memory usage by sharing a global JVM tag pool. I would be ok with that, as long as I can set the size of the pool to be zero, since I don't trust Java coders to outperform the optimizations done by the JVM as outlined in earlier posts. /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
Glenn Nielsen wrote: [snip] I just had an idea (dangerous things) regarding tag pooling optimizations. When Jasper translates a page it should be able to generate information about which tag handler classes it needs, and for each tag handler, the profile of the attribute/value pairs. At runtime Jasper could make one call to a tag pooling synchronized method to get all available instances of the tag handlers it needs. Any tag handlers/attribute profiles it couldn't obtain, it would have to create. Then it would manage its own local tag pool during the execution of the request. On termination of the request, after the page has been committed to the remote client, it would make one call to a tag pooling synchronized method to recycle/add the tag handlers it used. So there would only be 2 synchronized methods per page for implementing tag pooling regardless of how many tags are used. And you have the benifits of minimizing JVM memory usage by sharing a global JVM tag pool. that sparked something.. If we just bite the bullet and modify Jasper to correctly determine tag handler usage within a page (i.e. not get a new (from pool or new) tag each time) then we can probably get the best of both worlds -- local reuse of tag handler variables (I don't like to call that a pool) plus application wide tag pooling. If application wide tag pooling is too expensive, then we can chunk it. Regardless, the per page reuse should be a big gain. Next step might be to determine if, in certain situations, we can not re-call tag setters. The spec says that attributes set should be persistent. Rickard's test case would really benefit from setter optimization because most time was spent in body of setter methods. Any comments? btw, If i start tinkering, should I work with jasper34 or the 3.3 stuff? -casey Regards, Glenn -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance/3.3 tag pooling
On Wed, 23 May 2001, Casey Lucas wrote: btw, If i start tinkering, should I work with jasper34 or the 3.3 stuff? Difficult question... The problem with jasper34 is that it doesn't work yet ( the one in proposals/jasper34 - I still have to move it in the new repository ). Mea culpa - I tried to make big changes instead of the old slow evolution... I'll start importing the current jasper33 in the new repository and make sure it builds, and use it as the first step for 34. Then I'll stick with the step-by-step evolution. Merging with jasper40 and xslt can wait a bit. Costin -casey Regards, Glenn -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance/3.3 tag pooling
On Wed, 23 May 2001 [EMAIL PROTECTED] wrote:
On Wed, 23 May 2001, Casey Lucas wrote:
btw, If i start tinkering, should I work with jasper34 or the 3.3 stuff?
Difficult question...
The problem with jasper34 is that it doesn't work yet ( the one in
proposals/jasper34 - I still have to move it in the new repository ). Mea
culpa - I tried to make big changes instead of the old slow evolution...
I'll start importing the current jasper33 in the new repository and make
sure it builds, and use it as the first step for 34. Then I'll stick with
the step-by-step evolution. Merging with jasper40 and xslt can wait a bit.
I know Costin loves evolutionary change :-), and it's certainly a valid
approach to Jasper.
But there is also another approach we should consider - a green-field
recoding of at least some of the major components (conforming to an
agreed-upon overall architecture, of course).
What has struck me about the custom tags pooling question is that we're
trying to make a non-optimizing compiler into an optimizing compiler from
the ground up, rather than taking advantage of the decades of compiler
writing experience and designing one from the top down.
Just as a for instance about why we might want to consider this, let's
look at a mythical iterate tag with a nested tag inside:
x:iterate from=0 to=999
x:foo name=bar value=%= ... an expression %/
/x:iterate
Today, Jasper will do a new FooTag() plus all of the associated tag
setup, inside the loop, 1000 times. Tomorrow, using tag pooling can mean
that there will only be at most one create, but you've still got the
allocate/deallocate overhead plus the redundant setXxxx calls.
However, it's perfectly legal for the pseudo-code generated by this page
to look like this (see the examples in the JSP spec):
IterateTag iteratetag = new IterateTag(); // or allocate from a pool
iterateTag.setPageContext(...);
iterateTag.setParent(null);
iterateTag.setFrom(0);
itearteTag.setTo(999);
FooTag fooTag = new FooTag(); // Or allocate from a pool
fooTag.setPageContext(...); // Lifted out of the loop
fooTag.setParent(iterateTag); // Lifted out of the loop
fooTag.setName(bar);// Lifted out of the loop
... iterate-start implementation ... {
fooTag.setValue(expression);
fooTag.doStartTag();
fooTag.doEndTag();
} ... iterate-end implementation ...
fooTag.release();
fooTag = null; // or recycle to a pool
iterateTag.release();
iterateTag = null; // or recycle to a pool
In other words, you pay either *one* object creation or *one*
allocate/deallocate for the x:foo tag instance, and you don't waste your
time doing stuff inside the loop that only needs to be done once.
Optimizing compilers can be made smart enough to do things like this, as
long as you take the time to build in the appropriate knowledge of the
language (in this case, JSP syntax and semantics) that you are
translating. If you happened to take a compiler class along a degree
path, this kind of thing will probably look familiar.
I think we owe it to ourselves to consider whether a completely new
effort, at least for the compiler, might get us to better results
(probably with less overall development effort) than an evolutionary
approach based on the current code.
NOTE: For most of the rest of the overall problem (the PageContext
implementation, how Jasper fits in with the servlet container, and so
on) evolution is probably a very reasonable strategy. On the compiler,
though, I'm not so sure.
Costin
Craig
Re: Jasper performance/3.3 tag pooling
On Wed, 23 May 2001, Craig R. McClanahan wrote: I know Costin loves evolutionary change :-), and it's certainly a valid approach to Jasper. But there is also another approach we should consider - a green-field recoding of at least some of the major components (conforming to an agreed-upon overall architecture, of course). NOTE: For most of the rest of the overall problem (the PageContext implementation, how Jasper fits in with the servlet container, and so on) evolution is probably a very reasonable strategy. On the compiler, though, I'm not so sure. If we are talking about the compiler ( or code generator ): I partially agree, the current architecture will get a lot of pressure from more complex optimizations or tricks. But before we can even start to change the generator we need to do the initial refactoring and get the other components in order ( runtime, etc). We can also get some optimizations in, and use that to learn what's needed from a new generator architecture. I just don't think the new generator can happen in a 3.4 space - my goal is just to enable an effort to rewrite it, and gather as much information as possible about it's requirements. In any case - whatever happens in the current generator with regard to generated code will still be usefull for any new generator architecture. And if certain optimizations can't be done - that's even better, because it would help us understand what's needed. I had big hopes for an XSLT based generator, and I still think it may be a good way to implement the code generator - and I hope to hear other ideas. Costin
Re: Jasper performance/3.3 tag pooling
Casey, We hope to freeze 3.3 for a release in the next weeks. If you feel the changes are reasonably small and will not create problems - you can still do it. What about: 1. We copy the code from 3.3 as it is in jasper34 area ( except that we rename the package ). 2. We make sure it builds and runs in 3.3 - with a minimal set of changes: - copy jasper34.jar in lib/container - edit server.xml and replace JspInterceptor with JspInterceptor34 3. You can start making the optimizations, and I can start doing the refactoring, making sure we keep everything functional. 4. In time we can merge the changes from 4.0, add the new interfaces proposed by Mel, add a new generator and so on - while making sure the tests are passing and jasper is stable. This is unlikely to be finished before 3.3 is released, but we can make sure we keep passing all the test suites and we can release milestones of jasper34 for who needs the performance enhancements. When the code is ready we can make a 3.3.1 release and replace the old jasper ( and same for when ajp14 is ready ). What I tried in proposals/jasper34 is just wrong and against the basic ideas of evolution - and I realize that. Just give me 2 days, I need to finish with xalan ( we just had the 2.1.0 release and I need to finish my work on it ) Costin On Wed, 23 May 2001, Casey Lucas wrote: Costin Craig, I agree with both of you. The optimizations that Craig mentioned are exactly the ones that I was hoping to add. Yes, the least fun part will probably be adding to the existing generator code. But, until a new generating architecture is in place, I think it would be good to go ahead and try to add the code to the existing jasper. When the next gen jasper gets a lot of momentum, we can add the tag related optimizations to it. -casey [EMAIL PROTECTED] wrote: On Wed, 23 May 2001, Craig R. McClanahan wrote: I know Costin loves evolutionary change :-), and it's certainly a valid approach to Jasper. But there is also another approach we should consider - a green-field recoding of at least some of the major components (conforming to an agreed-upon overall architecture, of course). NOTE: For most of the rest of the overall problem (the PageContext implementation, how Jasper fits in with the servlet container, and so on) evolution is probably a very reasonable strategy. On the compiler, though, I'm not so sure. If we are talking about the compiler ( or code generator ): I partially agree, the current architecture will get a lot of pressure from more complex optimizations or tricks. But before we can even start to change the generator we need to do the initial refactoring and get the other components in order ( runtime, etc). We can also get some optimizations in, and use that to learn what's needed from a new generator architecture. I just don't think the new generator can happen in a 3.4 space - my goal is just to enable an effort to rewrite it, and gather as much information as possible about it's requirements. In any case - whatever happens in the current generator with regard to generated code will still be usefull for any new generator architecture. And if certain optimizations can't be done - that's even better, because it would help us understand what's needed. I had big hopes for an XSLT based generator, and I still think it may be a good way to implement the code generator - and I hope to hear other ideas. Costin
Re: Jasper performance/3.3 tag pooling
I think the changes will be too big to add before the 3.3 freeze. I like your alternative and will start bashing the jsp-java files to get something optimal before changing the jasper34 generator code. I'll let you know when I have something that looks interesting so that it can be discussed before taking the time to modify the generator code. -casey [EMAIL PROTECTED] wrote: Casey, We hope to freeze 3.3 for a release in the next weeks. If you feel the changes are reasonably small and will not create problems - you can still do it. What about: 1. We copy the code from 3.3 as it is in jasper34 area ( except that we rename the package ). 2. We make sure it builds and runs in 3.3 - with a minimal set of changes: - copy jasper34.jar in lib/container - edit server.xml and replace JspInterceptor with JspInterceptor34 3. You can start making the optimizations, and I can start doing the refactoring, making sure we keep everything functional. 4. In time we can merge the changes from 4.0, add the new interfaces proposed by Mel, add a new generator and so on - while making sure the tests are passing and jasper is stable. This is unlikely to be finished before 3.3 is released, but we can make sure we keep passing all the test suites and we can release milestones of jasper34 for who needs the performance enhancements. When the code is ready we can make a 3.3.1 release and replace the old jasper ( and same for when ajp14 is ready ). What I tried in proposals/jasper34 is just wrong and against the basic ideas of evolution - and I realize that. Just give me 2 days, I need to finish with xalan ( we just had the 2.1.0 release and I need to finish my work on it ) Costin On Wed, 23 May 2001, Casey Lucas wrote: Costin Craig, I agree with both of you. The optimizations that Craig mentioned are exactly the ones that I was hoping to add. Yes, the least fun part will probably be adding to the existing generator code. But, until a new generating architecture is in place, I think it would be good to go ahead and try to add the code to the existing jasper. When the next gen jasper gets a lot of momentum, we can add the tag related optimizations to it. -casey [EMAIL PROTECTED] wrote: On Wed, 23 May 2001, Craig R. McClanahan wrote: I know Costin loves evolutionary change :-), and it's certainly a valid approach to Jasper. But there is also another approach we should consider - a green-field recoding of at least some of the major components (conforming to an agreed-upon overall architecture, of course). NOTE: For most of the rest of the overall problem (the PageContext implementation, how Jasper fits in with the servlet container, and so on) evolution is probably a very reasonable strategy. On the compiler, though, I'm not so sure. If we are talking about the compiler ( or code generator ): I partially agree, the current architecture will get a lot of pressure from more complex optimizations or tricks. But before we can even start to change the generator we need to do the initial refactoring and get the other components in order ( runtime, etc). We can also get some optimizations in, and use that to learn what's needed from a new generator architecture. I just don't think the new generator can happen in a 3.4 space - my goal is just to enable an effort to rewrite it, and gather as much information as possible about it's requirements. In any case - whatever happens in the current generator with regard to generated code will still be usefull for any new generator architecture. And if certain optimizations can't be done - that's even better, because it would help us understand what's needed. I had big hopes for an XSLT based generator, and I still think it may be a good way to implement the code generator - and I hope to hear other ideas. Costin
Re: Jasper performance/3.3 tag pooling
Hi! [EMAIL PROTECTED] wrote: Could you send a small page ( and the taglibs ) and the test conditions ? I have sent taglib+page and test conditions to Costin. We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That is one solution, but what do you do with the pool after page request? That was planned for later, in jasper34 space - but if you send code that shows a significant degradation in performance in some reasonable situations we can still fix it in 3.3. I hope that Costin will be able to reproduce what I found. Regarding the default - I think it should be enabled, mostly because that will require people to make sure their taglibs are working corectly in a pooled environment. It should be possible to disable it on a per application basis, and we need to make sure we document how to do that. I agree. I don't think the design is bad - just the implementation of the pool, but this is the first attempt with the goal of making sure the pooling work, optimizations will come later. The current experience with tomcat performance evolution from 3.1 to 3.3 shows that recycling objects has a huge effect ( if implemented corectly ) - and I see no reason why jasper would be different. Agree, object reuse can have nice effects. But the hash+sync overhead must be balanced against the benefits of the actual object reuse. I'm looking forward to the VM where the allocation and GC will be free ( so far most do synchronize on object allocation ). I had a quick chat with the JRockit guys (the server JVM dudes) last J1, and they went Waaah! when I told them that JBoss pools EJB instances. The JVM would do a much better job with it in their opinion. If that's true or not I don't know, but I have seen other statements like that from other people. /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
Casey Lucas wrote: Also, did you run the tests with and without tag pooling enabled on the same version of tomcat? (By adding removing TagPoolManagerInterceptor.) Yes. The only thing changed between the runs was the TPMI flag. This is a bad design. Basically, any gains you get from reusing tags are lost due to the overhead and general performance decrease you get by using hashtables+synchronized. Hash lookup is done once per jsp page - when the jsp page is first run. After that, it's basically a synchronized push / pop pair on all subsequent runs of the page. In the future we can even get rid of the synch by using thread local storage... one step at a time though. :) Yup, that might improve things. Although then it depends on how ThreadLocal is implemented. I guess at some point deep down it has to do synchronized, although I haven't checked. I've looked at pages generated by other more efficient JSP compilers (e.g. Resin), and they generally use another approach: Only reuse a tag within a particular page, and don't use a synchronized pool to do it. Just pass the instance around within the generated code. What is important is not primarily the global optimization gained by using pools, but the local optimization gained by not creating tags for each iteration in a iterative custom tag. Yes reusing a tag handler within a single page will be more efficient (for that particular page) but I would guess that once we change to pool per thread there's no way newing the tags at each use will be faster (at least for a busy site with lots of tags.) And didn't say you should either. Right now 3.2 new's tags, and it's slow. I'm saying, do it the middle way, but new'ing it once per page, and then reuse within that page. No global optimization problems (i.e. when to grow/shrink the pool), no synch problems, no shared pool considerations, no hash lookup. All the good stuff and none of the bad. As I said, last time I checked Resin worked this way, and I think we can all agree that it's pretty snappy. This is wy more efficient, and also avoids the suboptimization of trying to reuse objects using Java code, something which is more efficiently handled by modern JVMs' memory management (i.e. creating objects using new is pretty snappy compared with Java-coded pools). I disagree. I've found that object reuse can be a good performance optimization. Sorry, I should have been more specific. Of course object reuse can be a good performance optimization. I'm just saying that you gotta balance it with regard to modern JVM's ability to manage memory and new objects. Optimize locally in code, and let the JVM do it globally. That's just IMHO though, so feel free to disagree :-) So, please remove the tag pooling, and do it right. If you don't believe me, do some benchmarking or something. I have done benchmarking and pooling enabled always wins. :) Maybe you can send some examples so that we can try and track down the problems. As above, I've sent code to Costin. But I'll forward it to you to, so you check it out. Lastly, I would be absolutely thrilled to find out that the fault was mine, i.e. some silly coding or config error. :-) regards, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
On Tue, 22 May 2001, Rickard Öberg wrote: Hi! [EMAIL PROTECTED] wrote: Could you send a small page ( and the taglibs ) and the test conditions ? I have sent taglib+page and test conditions to Costin. Thanks, I'll try to add it to the tests dir. We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That is one solution, but what do you do with the pool after page request? I'm not sure I understand. Each thread has a number of associated objects that are recycled and reused - a Request object will stick with a thread. Same can be done for the tag pools - except that this may use a lot of memory ( but less than allocating/freeing ). It is possible to use a middle ground, or tune this - but for maximum performance you can have a local pool in each Request. I hope that Costin will be able to reproduce what I found. I hope not :-) Again - thanks for doing the tests and checking the code, and hope to see more contributions and maybe few patches :-) I had a quick chat with the JRockit guys (the server JVM dudes) last J1, and they went Waaah! when I told them that JBoss pools EJB instances. The JVM would do a much better job with it in their opinion. If that's true or not I don't know, but I have seen other statements like that from other people. IMHO the right answer is depends. And it depends on the actual use case, on how many objects are created and where the synchronization occurs. For tomcat, where we expect hundred of concurent requests and each would create almost a hundred object ( that was the case in 3.0 ) - I doubt any VM could make a difference. Costin
Re: Jasper performance/3.3 tag pooling
On Tue, 22 May 2001, Rickard Öberg wrote: Hash lookup is done once per jsp page - when the jsp page is first run. After that, it's basically a synchronized push / pop pair on all subsequent runs of the page. In the future we can even get rid of the synch by using thread local storage... one step at a time though. :) Yup, that might improve things. Although then it depends on how ThreadLocal is implemented. I guess at some point deep down it has to do synchronized, although I haven't checked. It does ! :-) But in tomcat 3.3 we do a different trick - the thread pool is maintaining a local storage for each thread, and it's passing it to the worker. The only synchronization in tomcat is in getting a thread from the thread pool - besides that we shouldn't need anything else. Right now we keep the Request/Request pair - so we have a one-to-one relation between main request and thread ( in other words, the request is allways in the same thread ). Whatever attributes/notes you store in the request will be equivalent with thread-local data, without any sync. Sorry, I should have been more specific. Of course object reuse can be a good performance optimization. I'm just saying that you gotta balance it with regard to modern JVM's ability to manage memory and new objects. Optimize locally in code, and let the JVM do it globally. That's just IMHO though, so feel free to disagree :-) :-) I was thinking the reverse - optimize globally, let the VM optimize locally, but it depends on definitions... Costin
Jasper performance/3.3 tag pooling
Hi! Ok, so now I've tested the Jasper performance with the 3.3 tag pooling fix. The test was performed with a medium complexity page using lots of iterative custom tags in a hierarchical fashion (i.e. tags within tags). Results: The page ran slower, and above all the response time varied greatly (between 250ms and 460ms, whereas without tag pooling we got between 230ms and 340ms). Looking at the generated code, I'm not particularly surprised: it uses (it seems) a real shared tag pool, so using tags will execute code that needs to be synchronized. This is a bad design. Basically, any gains you get from reusing tags are lost due to the overhead and general performance decrease you get by using hashtables+synchronized. I've looked at pages generated by other more efficient JSP compilers (e.g. Resin), and they generally use another approach: Only reuse a tag within a particular page, and don't use a synchronized pool to do it. Just pass the instance around within the generated code. What is important is not primarily the global optimization gained by using pools, but the local optimization gained by not creating tags for each iteration in a iterative custom tag. This is wy more efficient, and also avoids the suboptimization of trying to reuse objects using Java code, something which is more efficiently handled by modern JVMs' memory management (i.e. creating objects using new is pretty snappy compared with Java-coded pools). So, please remove the tag pooling, and do it right. If you don't believe me, do some benchmarking or something. /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
Hi Rickard, Could you send a small page ( and the taglibs ) and the test conditions ? Most tests I run show a (significant) improvement in performance. We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That was planned for later, in jasper34 space - but if you send code that shows a significant degradation in performance in some reasonable situations we can still fix it in 3.3. Regarding the default - I think it should be enabled, mostly because that will require people to make sure their taglibs are working corectly in a pooled environment. It should be possible to disable it on a per application basis, and we need to make sure we document how to do that. I don't think the design is bad - just the implementation of the pool, but this is the first attempt with the goal of making sure the pooling work, optimizations will come later. The current experience with tomcat performance evolution from 3.1 to 3.3 shows that recycling objects has a huge effect ( if implemented corectly ) - and I see no reason why jasper would be different. I'm looking forward to the VM where the allocation and GC will be free ( so far most do synchronize on object allocation ). Costin On Mon, 21 May 2001, Rickard Öberg wrote: Hi! Ok, so now I've tested the Jasper performance with the 3.3 tag pooling fix. The test was performed with a medium complexity page using lots of iterative custom tags in a hierarchical fashion (i.e. tags within tags). Results: The page ran slower, and above all the response time varied greatly (between 250ms and 460ms, whereas without tag pooling we got between 230ms and 340ms). Looking at the generated code, I'm not particularly surprised: it uses (it seems) a real shared tag pool, so using tags will execute code that needs to be synchronized. This is a bad design. Basically, any gains you get from reusing tags are lost due to the overhead and general performance decrease you get by using hashtables+synchronized. I've looked at pages generated by other more efficient JSP compilers (e.g. Resin), and they generally use another approach: Only reuse a tag within a particular page, and don't use a synchronized pool to do it. Just pass the instance around within the generated code. What is important is not primarily the global optimization gained by using pools, but the local optimization gained by not creating tags for each iteration in a iterative custom tag. This is wy more efficient, and also avoids the suboptimization of trying to reuse objects using Java code, something which is more efficiently handled by modern JVMs' memory management (i.e. creating objects using new is pretty snappy compared with Java-coded pools). So, please remove the tag pooling, and do it right. If you don't believe me, do some benchmarking or something. /Rickard
Re: Jasper performance/3.3 tag pooling
Rickard, Can you please send in some complete examples? Also, did you run the tests with and without tag pooling enabled on the same version of tomcat? (By adding removing TagPoolManagerInterceptor.) My experience has been that if the jsp uses many tags, then pooling is a big performance gain. See comments below. Rickard Öberg wrote: Hi! Ok, so now I've tested the Jasper performance with the 3.3 tag pooling fix. The test was performed with a medium complexity page using lots of iterative custom tags in a hierarchical fashion (i.e. tags within tags). Results: The page ran slower, and above all the response time varied greatly (between 250ms and 460ms, whereas without tag pooling we got between 230ms and 340ms). Looking at the generated code, I'm not particularly surprised: it uses (it seems) a real shared tag pool, so using tags will execute code that needs to be synchronized. Yes, everytime you obtain a tag, there is a synch that occurs for the pool that exists for each uniquely named tag and specific set of attributes. There is actually a set of named pools per application context. This is a bad design. Basically, any gains you get from reusing tags are lost due to the overhead and general performance decrease you get by using hashtables+synchronized. Hash lookup is done once per jsp page - when the jsp page is first run. After that, it's basically a synchronized push / pop pair on all subsequent runs of the page. In the future we can even get rid of the synch by using thread local storage... one step at a time though. :) I've looked at pages generated by other more efficient JSP compilers (e.g. Resin), and they generally use another approach: Only reuse a tag within a particular page, and don't use a synchronized pool to do it. Just pass the instance around within the generated code. What is important is not primarily the global optimization gained by using pools, but the local optimization gained by not creating tags for each iteration in a iterative custom tag. Yes reusing a tag handler within a single page will be more efficient (for that particular page) but I would guess that once we change to pool per thread there's no way newing the tags at each use will be faster (at least for a busy site with lots of tags.) This is wy more efficient, and also avoids the suboptimization of trying to reuse objects using Java code, something which is more efficiently handled by modern JVMs' memory management (i.e. creating objects using new is pretty snappy compared with Java-coded pools). I disagree. I've found that object reuse can be a good performance optimization. So, please remove the tag pooling, and do it right. If you don't believe me, do some benchmarking or something. I have done benchmarking and pooling enabled always wins. :) Maybe you can send some examples so that we can try and track down the problems. Thanks for taking a look at tag pooling. I look forward to refining the implementation. -casey
RE: Jasper performance
-Original Message- From: Eduardo Pelegri-Llopart [mailto:[EMAIL PROTECTED]] Sent: Saturday, May 19, 2001 12:01 AM (sorry for the response lag, unfortunatly I don't read tomcat very frequently) ... * Using XSTL for templating... Like Jon and some others, I think that XSTL is a bit too complicated (and memory expensive) to be my favorite templating mechanism. But it is a widely used as a transformation mechanism. Some thoughts on the role of XML, JSP XSLT at http://java.sun.com/products/jsp/html/JSPXML.html This follown-the-bunch atitude with XSLT and other technologies, always reminds me of lemmings. =;o) ... Have fun, Paulo Gaspar
RE: Jasper performance - JMX
In the case of DoS, I don't believe a bit on trusted tags and such stuff. Why monitoring the tags at all if while(true) is so easy. I mean, the front door is wide open, why care about that little window? The only way to close everything is by monitoring the Servlets and allow setting some limitation on time per request. And that does not look so hard to do since Servlets are passive entities (meaning that they do NOT call Tomcat, Tomcat calls them) and their basic implementation (base class) belongs to Tomcat. The standard way to do it would be by instrumenting the Servlets, either directly or with a wrapper using JMX. JMX is not a J2EE exclusive and it is not that complex either. Additional advantages of implementing JMX in Tomcat: - Collecting other statistics; - Diagnostics; - Changing settings on the fly; - etc. BTW, I am not the first person talking about JMX in this list. Have fun, Paulo Gaspar -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 3:02 AM On Thu, 17 May 2001, Glenn Nielsen wrote: I guess he's refering to DOS attacks ( like a while(true); in java code or allocating lots of memory ). You won't have much of a templating language if you don't allow some sort of looping. Kind of hard to restrict that. True, but if you have a set of trusted tags, including looping tags, and no untrusted code except the one that calls the tags you could do a lot to control the resources. For example the tags ( or jasper generated code ) could check for time execution limits, or how many resources are allocated. It would be nice if self monitoring were built into Tomcat so sysads could track statistics on performance of the JVM, Tomcat in general, and individual servlets/JSP's. Even setting thresholds when automated email notifications could be done. Lets give sysadmins the information they need, then they can take action against problem users. Yes, that would be an interesting hack.. I was thinking about JPDA - it would be possible to check the memory use for each thread, associate it with the user code. Also, it is possible to store the time when entering/exiting user code, and have a deamon thread check if any thread is spending too much time. ( the time monitoring part can be done without jpda - but to monitor the memory I don't know other solution ). ( well, I know - I remember a certain tool that was used to manipluate bytecodes and add instrumentation before all allocations - but that's far too difficult for the time we have available ). I still think that using the SecurityManager implementation in Tomcat with a well tuned security polciy can provide one of the most secure environments available for running web based applications. This is just my opinion, feel free to try and convince me some other technology is more secure. I'll not even try :-) You're right, but there are some things that we could add to also control some resource usage ( memory and cpu time ). Costin
Re: Jasper performance
Jon Stevens wrote:
on 5/17/01 12:47 PM, Glenn Nielsen [EMAIL PROTECTED] wrote:
But now that both Tomcat 3.2 and Tomcat 4 support the Java SecurityManager
you can control security at the container level regardless of whether someone
is using the CFM servlet, velocity, CoCoon, JSP, etc.
Not true.
http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html
Hashtable strings = new Hashtable();
int i=0;
while (true)
{
strings.put (dead+i, new StringBuffer(99));
}
There is no amount of security that will prevent someone from putting that
into their JSP page other than disabling the ability to put scriptlets into
things. If you do that, then you are simply where you should have been in
the first place...using Velocity.
Yes, but using velocity templates limits a great deal what customers
can do when compared to a general purpose servlet container where
web applications can be deployed. There is a great deal more to
security than just preventing a 'trusted user' who can publish content
from doing something stupid. No where in your YMTD document do I see
anything about security, just your reference above to a trusted user
DoS. Heck, if one of my customers wants to use Velocity, they can do
so if it can be deployed as a web application, but it will have to
run within the security policies we set for the Tomcat Java SecurityManager. ;-)
Regards,
Glenn
--
Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder|
MOREnet System Programming | * if iz ina coment. |
Missouri Research and Education Network | */ |
--
Re: Jasper performance
At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. - Dennis Doubleday email: [EMAIL PROTECTED] yourfit.com, Inc. web: http://www.yourfit.com/
Re: Jasper performance
Dennis Doubleday wrote: At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
RE: Jasper performance - JMX
On Fri, 18 May 2001, Paulo Gaspar wrote: In the case of DoS, I don't believe a bit on trusted tags and such stuff. Why monitoring the tags at all if while(true) is so easy. I mean, the front door is wide open, why care about that little window? Well, what I said was trusted tags and only pure jsps, no user code. That means a webapplication will be allowed only if it has no libraries, no servlets, no user tags - and only JSPs that are not using % %, but only a limited set of tags. All the above is equivalent with what Jon described ( except syntactic sugar ). It'll be foreach instead of #foreach, etc - so at least you'll use the JSP syntax, and your app will work in normal containers as well. It will be a valid JSP in a valid webapp - but of course the container can't be called a servlet container ( it doesn't run even servlets :-) I doubt too many installations of Velocity are set up to disallow user code - it's not too much you can do. It'll be secure - probably because nobody will care to use such a thing :-) And if you allow any user code - all the #foreach prevention of DOS goes away. ( it's like the famous thin air firewall - it's the most secure, but pointless ). The only way to close everything is by monitoring the Servlets and allow setting some limitation on time per request. And that does not look so hard to do since Servlets are passive entities (meaning that they do NOT call Tomcat, Tomcat calls them) and their basic implementation (base class) belongs to Tomcat. I agree. With a security manager you control almost everything - except the amount of CPU and memory. It is possible ( and not very difficult ) to monitor the CPU amount used by a servlet - just a trivial interceptor that will log the moment of entry and exit in servlet code. You'll need a thread that periodically checks if a servlet/jsp is hunged or takes too much time, and disable the whole app if it is the case. It is also possible to periodically check the amount of memory used by each thread. ( it's much harder - but I think you can hack a bit with JPDA and do it ). ( like OptimizeIt work for example ). Both will be far more usefull than security by removing all usefull features. BTW, I am not the first person talking about JMX in this list. JMX is great - I like it a lot, it can be added easily by a module - but I don't think it's a big priority for now. After 3.3 is released we can discuss new features as add-on modules ( I have a few ). Costin
RE: Jasper performance
Velocity does do a lot to minimize the risk you mention, but while we're using stupid coding tricks, couldn't you do the following in Velocity? #* assume strings is a Vector *# #set ($strings = $request.getParameter(strings))) #foreach ($string in $strings) $strings.addElement($string.clone()); #end -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 8:50 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Dennis Doubleday wrote: At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
RE: Jasper performance
It isn't concurrent. -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 10:52 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Jef Newsom wrote: Velocity does do a lot to minimize the risk you mention, but while we're using stupid coding tricks, couldn't you do the following in Velocity? #* assume strings is a Vector *# #set ($strings = $request.getParameter(strings))) #foreach ($string in $strings) $strings.addElement($string.clone()); #end Good try :) Assuming it is a Vector, I am pretty convinced that wouldn't work because you will get a ConcurrentModificationException when you modified the Vector. geir -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 8:50 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Dennis Doubleday wrote: At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders... -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
RE: Jasper performance
I wrote a test script, and assuming (which the docs say it does) that Velocity uses the iterator() instead of elements() when it runs up against a vector, then all is well. If elements() is used, it goes into infinite loop land. My mistake. -Original Message- From: Jef Newsom Sent: Friday, May 18, 2001 11:44 AM To: [EMAIL PROTECTED] Subject: RE: Jasper performance It isn't concurrent. -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 10:52 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Jef Newsom wrote: Velocity does do a lot to minimize the risk you mention, but while we're using stupid coding tricks, couldn't you do the following in Velocity? #* assume strings is a Vector *# #set ($strings = $request.getParameter(strings))) #foreach ($string in $strings) $strings.addElement($string.clone()); #end Good try :) Assuming it is a Vector, I am pretty convinced that wouldn't work because you will get a ConcurrentModificationException when you modified the Vector. geir -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 8:50 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Dennis Doubleday wrote: At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders... -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
Re: Jasper performance
on 5/18/01 1:37 AM, Paulo Gaspar [EMAIL PROTECTED] wrote: All Velocity has is a #foreach. This is a fully functional looping construct that prevents you from screwing things up and still gets the job done. On the #foreach and DoS issues, I would use makes it harder instead of prevents in . But it also makes much harder to mix logic and presentation too. Have fun, Paulo Gaspar Please try to come up with a case where it isn't prevented and then give me a reason for makes it harder. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
on 5/18/01 6:50 AM, Geir Magnusson Jr. [EMAIL PROTECTED] wrote: Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir I guess the point that we are trying to make with Velocity is that the design of the Velocity tool is far cleaner than having to rely on improved compilers to do the work for you. -jon
Re: Jasper performance
(sorry for the response lag, unfortunatly I don't read tomcat very frequently) Hi Jon. The problem with taglibs is that there is no restriction on the ability to put Java code in the page. It is part of the JSP specification to be able to do that. Sure, you can disable it (as Costin said), but then you are breaking the JSP specification. And I know how important standards are to everyone... -jon I didn't see any follow-up clarifying this but apologies if I missed it. JSP 1.2 has the notion of a TagLibraryValidator that is associated with a tag library. This can be used to portably validate different assertions on your JSP page. This could be used to, for example, check that some actions are nested within others, or that some portions of your JSP page conform in one way or another. A TagLibraryValidator can also be used to disable scriptlets. === On some of the other points in this thread: * Using XSTL for templating... Like Jon and some others, I think that XSTL is a bit too complicated (and memory expensive) to be my favorite templating mechanism. But it is a widely used as a transformation mechanism. Some thoughts on the role of XML, JSP XSLT at http://java.sun.com/products/jsp/html/JSPXML.html * XML in Jasper... The JSP expert group has considered for a while adding a portable mechanism for transformations on a JSP page. Such a transformation would be defined as operating on the XML reprsetnation of a JSP page. I expect we will discuss again this feature for JSP 1.3, although there are no specific dates for that yet. IMHO, - eduard/o Jon Stevens wrote: on 5/17/01 6:46 AM, Dennis Doubleday [EMAIL PROTECTED] wrote: At 08:35 PM 5/16/01, Jon wrote: Also, there is a reason for the #foreach... http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html Jon, I agree with most of your points. I am a new Velocity user and I am very impressed by its combination of power and simplicity. Reading/writing XSLT specs is an exercise in masochism. However, I don't see how Velocity is really avoiding the fundamental problem described in the document you referenced. If you are an ISP hosting Velocity-based pages, you are certainly going to have to let that 14 year old kid install both templates and class files; templates by themselves won't accomplish much. So the incompetent or malicious client can easily make the same mistake in his Command class that he would have made in the JSP page, and therefore also create a DOS attack on all servlets hosted in that JVM. No? Controlling what goes into the Context is key. There is nothing stating that you have to give the 14 year old access to creating .java files. Instead, the alternative approach is to place certain objects within the Context which allow the 14 year old to do a limited set of actions. This follows along with the Pull Model: http://jakarta.apache.org/turbine/pullmodel.html This is the approach that Tea http://opensource.go.com/ uses as well as the general idea behind taglibs. The problem with taglibs is that there is no restriction on the ability to put Java code in the page. It is part of the JSP specification to be able to do that. Sure, you can disable it (as Costin said), but then you are breaking the JSP specification. And I know how important standards are to everyone... -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
on 5/18/01 3:01 PM, Eduardo Pelegri-Llopart [EMAIL PROTECTED] wrote: I didn't see any follow-up clarifying this but apologies if I missed it. JSP 1.2 has the notion of a TagLibraryValidator that is associated with a tag library. This can be used to portably validate different assertions on your JSP page. This could be used to, for example, check that some actions are nested within others, or that some portions of your JSP page conform in one way or another. A TagLibraryValidator can also be used to disable scriptlets. Making it easy to disable scriptlets and breaking the specification are two different things. It is part of the JSP specification to be able to use scriptlets. By disabling them, you are encouraging people to break the specification. If people open their eyes and realize that the specification is fundamentally flawed, then they will see that the alternative (and correct) approach would be to remove scriptlets from the specification entirely. If you go and do that, you are back at square one and might as well have used another technology (which took this into consideration from the start) in the first place. Again, we are back to my point that simply taking an already terrible technology (ASP) and trying to fit it into a Java model (JSP) is a really brain damaged thing to do. In order to fix the problems in ASP (and therefore JSP) one has two options: 1) Hack the exiting technology (JSP) until it kinda works. 2) Come up with an alternative technology that took consideration of these points from the start. Clearly Sun chooses to take the approach of #1 because Sun would look like a bunch of boneheads if they dropped JSP at this time. So, stick with what you have and *try* to push it on people for as long as you can until someone wakes up and realizes that what was pushed down their throat by Sun was a stupid decision. On some of the other points in this thread: * Using XSTL for templating... Like Jon and some others, I think that XSTL is a bit too complicated (and memory expensive) to be my favorite templating mechanism. But it is a widely used as a transformation mechanism. Some thoughts on the role of XML, JSP XSLT at http://java.sun.com/products/jsp/html/JSPXML.html The humor in that article is that it all falls down as people realize that they want to be able to take form input from something. If you notice, all the arrows point towards the client. There are NO arrows that point from the client towards the server. Once again, I'm not impressed because the document doesn't address the larger vision of what people need to do in the real world. * XML in Jasper... The JSP expert group has considered for a while adding a portable mechanism for transformations on a JSP page. Such a transformation would be defined as operating on the XML reprsetnation of a JSP page. I expect we will discuss again this feature for JSP 1.3, although there are no specific dates for that yet. Yawn. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
Sorry, Jon, we disagree. TagLibraryValidators *are* part of the JSP 1.2 specification. They are quite flexible and one of the simplest uses is to express that some tags cannot appear. Scriptlets are exposed as jsp:scriptlet tags. - eduard/o Jon Stevens wrote: on 5/18/01 3:01 PM, Eduardo Pelegri-Llopart [EMAIL PROTECTED] wrote: I didn't see any follow-up clarifying this but apologies if I missed it. JSP 1.2 has the notion of a TagLibraryValidator that is associated with a tag library. This can be used to portably validate different assertions on your JSP page. This could be used to, for example, check that some actions are nested within others, or that some portions of your JSP page conform in one way or another. A TagLibraryValidator can also be used to disable scriptlets. Making it easy to disable scriptlets and breaking the specification are two different things. It is part of the JSP specification to be able to use scriptlets. By disabling them, you are encouraging people to break the specification.
Re: Jasper performance
on 5/18/01 4:55 PM, Eduardo Pelegri-Llopart [EMAIL PROTECTED] wrote: Sorry, Jon, we disagree. TagLibraryValidators *are* part of the JSP 1.2 specification. Go back and read what I wrote again. I'm not saying that TagLibraryValidators aren't part of the specification. They are quite flexible and one of the simplest uses is to express that some tags cannot appear. Scriptlets are exposed as jsp:scriptlet tags. Correct. And Scriptlets are also part of the specification. See: JSP Specificaton: 2.11.2 So, making scriptlets disappear is invalidating the specification. No where does it say in 2.11.2 that it is legal to remove those entities. In fact, the document specifically states: All JSP containers must support scripting elements based on the Java programming language. I don't see how you can disagree with me on the point of removing scriptlets from an application will invalidate the specification. So, in effect, you are talking about modifications of the specification. If you are going to go to that length (which is removing something that has been in the specification since day one and is documented all over the place in examples), you might as well dump the entire JSP technology and start over. I also find humor how you simply ignore my other points. Maybe if you shove enough stuff under the rug, no one will figure it out. -jon
Re: Jasper performance - JMX
on 5/18/01 10:48 AM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: On Fri, 18 May 2001, Jon Stevens wrote: Correct, however some bright monkey decided to add % % into the JSP specification. So, if you disable that, you are breaking the specification. In other words, it is a bad design in the first place. That is the point. Not using % % doesn't brake the spec - it is still a valid JSP page. Preventing people from using % % breaks the spec. It is documented, read: 2.11.2 Not allowing user code ( servlets, user libs, user tags ) is indeed breaking the specification - and a server that is doing that shouldn't claim it supports server side java - but the same is true for turbine, isn't it ? Nope. Turbine doesn't require the use of user modifiable server side java in order to do its job. You can simply put tags into the context and make them available to the end users. I find % % quite usefull for prototyping - but if you don't like it - don't use it. I wouldn't tuch a server where it's disabled, or where all I can do is use an obscure scripting language. #1. Luckily you aren't developing web applications as your job. #2. obscure? The developer can make the decision - not a smart monkey that thinks he knows better. We aren't trying to work with java developers. We are trying to work with designers and template engineers. Please don't tell me mixing logic and presentation is the worst sin - It is. In all of the years that I have been doing web app development (about 6-7 years now), this is the most common complain that I have heard about. most systems allow that ( HTML + javascript, perl, php, jsp, asp, XSLT, servlets, and so on ). I don't know any successful system where this is not allowed. Velocity. Yes Costin, it has taken many years to get to the point of what we have been trying to get to, but it has finally happened. Eureka! Get used to it. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
RE: Jasper performance
... XSLT is not perfect - neither is HTTP or HTML or any other standard. But because Apache and many other organizations are implementing and using it - I think they'll be around for a while :-) Costin Costin, You are still missing the point (as I wrote in my previous post on this thread) of the big volume of jars XSLT use would add to Tomcat minimal distro for JSP compilation. Also, maybe the impact of using XSLT is quite larger than you expect. Especially via the huge memory demands and respective side effects. I know that you know well about the impact of creating a zillion of objects in memory (as it happens in even the most minimal use of XSLT), so I am not going to detail on what those side effects are. What is worse, XSLT is usualy way too much trouble because so many things are so counter intuitive. The trap is that the really simple stuff works as expected, but when you need something just a little bit more complicated, sudenly you bump into something that does not work as expected, and you spend the next few days trying to figure out what. Jon is a... er... politness challenged person and is a bit too obcessed with Velocity, but he is basically right. I used XSLT a lot and I can tell you that is easy for Velocity to be a better alternative (and it is not my favourite one!) just because XSLT is such a poor template mechanism. One year ago, I was in the XSLT-LIST, bought Michael Kay book (kind of a very essential XSLT bible) and I was thinking about writing everything I could in XSLT just as any other XSLT fanatic... until I found out that: - I was wasting way too much time with XSLT complexities (why do you think the XSLT-LIST has all that traffic? Do you know that list?); - The outcome was always slower and more resource intensive than I found natural (and that the alternatives, as I discovered later). I would still use XSLT for some XML transformation tasks, but never for templating. I use Velocity and, in terms of both functionality and resource usage, it would really be a much better choice than XSLT (which is not hard to do, anyway). But it would NOT be my 1st choice for this (see my previous posting for further detail). It would be my 2nd choice in a very cut down version. Notice that I was not born with this knowledge. I went trough a lot of trouble to learn this. Now, you can - Pay attention to what I say and chek it out; - Think that the W3C is always right (HAHAHAHAHA!); - Think that your intuition is always right, which is another way of saying that you were born knowing it all. Sorry Costin for assuming that you do not have much experience with XSLT and its alternatives, but I just think you are too smart to keep defending it if you had that experience. Have fun, Paulo Gaspar
RE: Jasper performance
I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps
RE: Jasper performance
From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
Re: Jasper performance
At 08:35 PM 5/16/01, Jon wrote: Also, there is a reason for the #foreach... http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html Jon, I agree with most of your points. I am a new Velocity user and I am very impressed by its combination of power and simplicity. Reading/writing XSLT specs is an exercise in masochism. However, I don't see how Velocity is really avoiding the fundamental problem described in the document you referenced. If you are an ISP hosting Velocity-based pages, you are certainly going to have to let that 14 year old kid install both templates and class files; templates by themselves won't accomplish much. So the incompetent or malicious client can easily make the same mistake in his Command class that he would have made in the JSP page, and therefore also create a DOS attack on all servlets hosted in that JVM. No? - Dennis Doubleday email: [EMAIL PROTECTED] yourfit.com, Inc. web: http://www.yourfit.com/
RE: Jasper performance
I agree with you, So I'll suggest you Forte IE, which has a great debugger. that allow you to trace exception form the stack to the class, then to the original source code. It also allow break point, which is the least. The only regrets about this tools : - need a big computer (600mz and more then 256 Mo, it's great with 512) - not GPL At 14:17 17/05/2001 +0100, you wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
answer inline... -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 1:51 PM I don't know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). Nops. The XSLT would just be the way to implement code templates for the java code generation step of compiling a JSP. (And the nice guy thing would not be a problem it was the other way since that text could be encoded to become valid XML.) If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Nothing like that. But Java Bytecode generation would probably be faster, not because we would be able to make the most performant compiler in the world, but because a lot of I/O operations would be saved (writing the java code, calling/loading the compiler, the compiler producing the output class file, loading the class file from disk). Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Welcome to the club. But it helps if you follow the list. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. About right. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. That is not a new idea. Just not possible - there is no such functionality on the Java compiler. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Costin complained about that in this list not long ago. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps I also hope for that and it never happens. =;o) Have fun, Paulo Gaspar
RE: Jasper performance
On Thu, 17 May 2001, Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. Yes, debugging is a problem in the current jasper implementation. One solution, as you mentioned, is to hack the class to include line numbers that match the JSP file ( the line number is an annotation in the class ). There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. I'm not an expert in debugging, but that sound like a reasonable solution. You'll have to wait a bit for the implementation - I'm still fighting with cleaning and merging the runtime. Costin - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
Answer inline: -Original Message- From: Christopher Kirk [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 3:17 PM From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. You still can take a look at the intermediate java file to understand what is going on. It is not trivial, but at least it makes clear if the bug is in your code or on Jasper's. Anyway, Jasper sure could probably be improved to give good enough information about JSP location of the source of an exception... and you could help implementing that. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. There are two problems with that approach: 1) The technical complexity of such project (even using tools like BCEL); 2) Then it would be harder for you to find out if an exception would be caused by some bug in your code or in Jasper's code. Anyway, you can also contribute to such thing. Some pointers that can help you starting: - You can find the above referred BCEL at http://bcel.sourceforge.net/ - You can check Rhino Javascript's compiler on how do they directly compile something (javascript in this case) into Java Bytecode: http://www.mozilla.org/rhino/ - Maybe Go's Tea template compiler would be a closer match for JSP: http://opensource.go.com/ Have fun, Paulo Gaspar - Chris. ...
RE: Jasper performance
Even less GPL (commercial) but still for free, not so clumsy and with a better GUI and Debugger... Borland's JBuilder 4 Foundation. =:o) Have fun, Paulo Gaspar -Original Message- From: Kaneda K [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 3:55 PM To: [EMAIL PROTECTED] Subject: RE: Jasper performance I agree with you, So I'll suggest you Forte IE, which has a great debugger. that allow you to trace exception form the stack to the class, then to the original source code. It also allow break point, which is the least. The only regrets about this tools : - need a big computer (600mz and more then 256 Mo, it's great with 512) - not GPL At 14:17 17/05/2001 +0100, you wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
Ok Chris. Now I undertand. Working with JSP when I found that type of mistakes I went to see the .java generated. There I matched the line number so I could see which was the bad code. Then I had to go to the jsp to fix it. It is not the best of the world, but is that that bad? On the other hand we can trick the compiler to make it fit the line numbers, just generating really hugly java code. package jklfe.fel.ef; import java.util.*; // line 1 user.code.here() // line 2 Just an idea. Chau, Gaston From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. - - Este mensaje ha sido enviado desde el servidor de Webmail de Tournet S.A. http://www.tournet.com.ar/
RE: Jasper performance
Hi Carlos, On Thu, 17 May 2001, Carlos Gaston Alvarez wrote: I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). The JspParser object knows how to deal with that. And it's easy to convert it into a SAX event generator. The script will be equivalent with a CDATA section ( and you can have tag in a CDATA ). In 1.2 you also have the XML representation ( and we can use it to generate exactly the same type of sax events from a non-xml jsp file ) If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Not quite - look at XSLTC ( part of xml-xalan project ). This is intended only for pure JSP pages, with no java fragments. We are in a very similar situation with XSLTC - the page-generated code itself it's not very complex ( in fact it's much simpler in our case ), so we don't benefit a lot from javac optimizations. And no, this is not going to be a compilator ( but some cutpaste from another apache project :-) One important thing to keep in mind: sometimes there is a tradeof between compilation time and runtime performance. The more optimizations you do at compile time - the slower it is to compile, faster to run. That mean that would be more of an option for development time, I would still use the real compiler before deployment. This is a long term project - it's definitely something cool, but I think we should hold that until we solve more important issues. ( and compilation time can be improved few times by just using jikes ). Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. I like guessing, but it would help if you'ld also read the code, and maybe write some patches :-) Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. Not quite. Almost all of the time is spent doing javac compilation. The jsp-java translation is insignifiant. What we care about is getting the response time for the first request below a second ( so developer will not feel the delay and get fast feedback ). On my computer, using jikes - this is exactly the case, and I have a 300MHz laptop. I run a lot of XSLTs ( that's part of my job ) and I never felt a delay ( and benchmarks for most transforms I do are well bellow a second ). If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Unfortunately we can't compile from and to memory with the current javac, and even if it would be possible - the compiler is a complex application, doing optimizations, verifications, etc: I don't think the time for read/write source/class is predominant. Costin
RE: Jasper performance
There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. That sounds very nice. How does it work? Where/how do you feed the map file? (Just some documentation pointers to whatever feature you use would do.) Thanks, Paulo Gaspar -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 9:24 AM On Thu, 17 May 2001, Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. Yes, debugging is a problem in the current jasper implementation. One solution, as you mentioned, is to hack the class to include line numbers that match the JSP file ( the line number is an annotation in the class ). There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. I'm not an expert in debugging, but that sound like a reasonable solution. You'll have to wait a bit for the implementation - I'm still fighting with cleaning and merging the runtime. Costin - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
On Thu, 17 May 2001 [EMAIL PROTECTED] wrote: There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. There is a current JSR (#45) with a goal of producing a standard mapping format like this for non-Java languages that are translated into Java, and JSP is a pretty classic use case of this. I haven't heard much lately about this (will check more), but conceptually this approach seems like a good direction to go. Costin Craig
Re: Jasper performance
on 5/17/01 8:01 AM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Ok Chris. Now I undertand. Working with JSP when I found that type of mistakes I went to see the .java generated. There I matched the line number so I could see which was the bad code. Then I had to go to the jsp to fix it. It is not the best of the world, but is that that bad? If you think about *who* the target JSP audience is, Yes. Do you expect page designers to figure out problems with line numbers by going to look at the generated page in some temp directory somewhere? Gee, I hope not. More humor for you JSP believers: http://www.theregister.co.uk/content/4/18996.html You may also wish to read a complete pile of bullshit here: http://java.sun.com/products/jsp/jsp-asp.html My favorite: ASP TechnologyJSP Technology Reusable, Cross-Platform ComponentsNoJavaBeans, Enterprise JavaBeans, custom JSP tags Security Against System CrashesNoYes Memory Leak ProtectionNoYes Scripting LanguageVBScript, JScriptJava Customizable TagsNoYes Yea, JSP protects you from memory leaks and System Crashes. Yea Right. Oh yea, and ASP is lacking customizable tags...as if customizable tags is a good thing? JSP sucks. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
on 5/17/01 6:46 AM, Dennis Doubleday [EMAIL PROTECTED] wrote: At 08:35 PM 5/16/01, Jon wrote: Also, there is a reason for the #foreach... http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html Jon, I agree with most of your points. I am a new Velocity user and I am very impressed by its combination of power and simplicity. Reading/writing XSLT specs is an exercise in masochism. However, I don't see how Velocity is really avoiding the fundamental problem described in the document you referenced. If you are an ISP hosting Velocity-based pages, you are certainly going to have to let that 14 year old kid install both templates and class files; templates by themselves won't accomplish much. So the incompetent or malicious client can easily make the same mistake in his Command class that he would have made in the JSP page, and therefore also create a DOS attack on all servlets hosted in that JVM. No? Controlling what goes into the Context is key. There is nothing stating that you have to give the 14 year old access to creating .java files. Instead, the alternative approach is to place certain objects within the Context which allow the 14 year old to do a limited set of actions. This follows along with the Pull Model: http://jakarta.apache.org/turbine/pullmodel.html This is the approach that Tea http://opensource.go.com/ uses as well as the general idea behind taglibs. The problem with taglibs is that there is no restriction on the ability to put Java code in the page. It is part of the JSP specification to be able to do that. Sure, you can disable it (as Costin said), but then you are breaking the JSP specification. And I know how important standards are to everyone... -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. Really? I was told that it was a complete myth - that *no one* has to debug the Java code - they just debug the JSP code. My JSP experience occurred a few years agoe, and I remember bewildering error messages - however, I just assumed that was straightened out. I am not trying to toss gasoline on what could be another conflagration - I am just generally interested in the subject. Thanks geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
RE: Jasper performance
On Thu, 17 May 2001, Paulo Gaspar wrote: ... XSLT is not perfect - neither is HTTP or HTML or any other standard. But because Apache and many other organizations are implementing and using it - I think they'll be around for a while :-) Costin Costin, You are still missing the point (as I wrote in my previous post on this thread) of the big volume of jars XSLT use would add to Tomcat minimal distro for JSP compilation. AFAIK tomcat4 already includes a xslt engine ( part of JAXP1.1 ), and I thing 3.3 will have to update to JAXP1.1 also ( we're still using the old JAXP1.0.1 ). But there is a point I'm not missing - I do understand the concern regarding the size of the code. I would point again to XSLTC - I think there are many similarities. The XSLT compiler itself is quite big ( i.e. the piece that converts a XSTL sheet into a .class file - same as Jasper is converting JSPs ). But the transformation itself ( the generated code ), including all the runtime, is small enough to run on a palm pilot. And that doesn't matter too much anyway - for a small-memory deployment ( like an appliance or a CD runner ) you shouldn't even include the JSP generator, XSLT or javac. That's why we have JSPC - to pre-process the WAR file and precompile it. There is even more - for this case you'll probably want to use EmbededTomcat instead of server.xml-based runner, and we can pre-process web.xml too - so it may be possible to eliminate even the parser from the runtime-only version of tomcat. ( many embedded devices are pre-processing the .class files before deployment, some are even generating binary code - so generating .class files wouldn't work anyway ). Also, maybe the impact of using XSLT is quite larger than you expect. Especially via the huge memory demands and respective side effects. I know that you know well about the impact of creating a zillion of objects in memory (as it happens in even the most minimal use of XSLT), so I am not going to detail on what those side effects are. Did I mentioned XSTL is a standard - with quite a few implementations, a lot of competition and smart people, etc ? If you look at Tomcat3.0 you may conclude servlets are quite bad - zillion of objects in memory, big overhead, etc. There is a huge difference between a xslt implementation one year ago and one today. The same way as tomcat3.3 ( or 4.0 ) will generate only few dozen objects at run time ( ok, fewer in 3.3 :-), most XSTL engines are starting to recycle most objects. Take a look at DTM work in xalan ( and xsltc ). What is worse, XSLT is usualy way too much trouble because so many things are so counter intuitive. Well, HTML is counter intutive too ( H1 instead of bigger font ? ) :-) But many of the counter intuitive things ( like no side efects, or the way they treat variables ) do have a good reason behind it. And I suppose many will find things that are counter intuitive in java ( you can't change a String ?). The trap is that the really simple stuff works as expected, but when you need something just a little bit more complicated, sudenly you bump into something that does not work as expected, and you spend the next few days trying to figure out what. Yes, I know the feeling. But it's not something specific to XSLT, it happens to most programmers in most languages. Jon is a... er... politness challenged person and is a bit too obcessed Too bad for him. That's why filters were invented ( or the delete key ). I used XSLT a lot and I can tell you that is easy for Velocity to be a better alternative (and it is not my favourite one!) just because XSLT is such a poor template mechanism. Sorry, but I doubt it. One year ago, I was in the XSLT-LIST, bought Michael Kay book (kind of a very essential XSLT bible) and I was thinking about writing everything I could in XSLT just as any other XSLT fanatic... until I found out that: - I was wasting way too much time with XSLT complexities (why do you think the XSLT-LIST has all that traffic? Do you know that list?); - The outcome was always slower and more resource intensive than I found natural (and that the alternatives, as I discovered later). One year ago tomcat couldn't handle 50 requests per second. Try again today. Notice that I was not born with this knowledge. I went trough a lot of trouble to learn this. Now, you can - Pay attention to what I say and chek it out; - Think that the W3C is always right (HAHAHAHAHA!); - Think that your intuition is always right, which is another way of saying that you were born knowing it all. Well, I do pay attention to what you say - and my day job involves working on xslt. I don't think W3C is always right ( XML Schema ? Soap ? ), but it wasn't that right with HTML or HTTP either ( one request per connection ?) And I wasn't born knowing it - I spent a lot of time looking around and reading, and I have my own doubts. Sorry
RE: Jasper performance
You should rarely have to worry about mistakes in JSP if you are using tag libraries and beans to factor out all business logic. -Original Message- From: Kaneda K [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 6:55 AM To: [EMAIL PROTECTED] Subject: RE: Jasper performance I agree with you, So I'll suggest you Forte IE, which has a great debugger. that allow you to trace exception form the stack to the class, then to the original source code. It also allow break point, which is the least. The only regrets about this tools : - need a big computer (600mz and more then 256 Mo, it's great with 512) - not GPL At 14:17 17/05/2001 +0100, you wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
On Thu, 17 May 2001, Craig R. McClanahan wrote: On Thu, 17 May 2001 [EMAIL PROTECTED] wrote: There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. There is a current JSR (#45) with a goal of producing a standard mapping format like this for non-Java languages that are translated into Java, and JSP is a pretty classic use case of this. I haven't heard much lately about this (will check more), but conceptually this approach seems like a good direction to go. Well, there is a very interesting discussion going on xalan-dev, regarding an XSLT debugger. I saw a demo - and you can execute a XSLT step-by-step, see the line in the source xml that is processed, etc. Yes, mapping the line numbers is a missing feature in our code, but there are already people who made this work ( while using jasper with a debugger ). I hope someone will contribute some code back. Costin
RE: Jasper performance
On Thu, 17 May 2001, Paulo Gaspar wrote: There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. That sounds very nice. How does it work? Where/how do you feed the map file? (Just some documentation pointers to whatever feature you use would do.) Jasper already generates the line number as comments. What you would need to do is generate them as code ( like a static Hashtable or [] ), and then change the catch() code a bit. You'll need to parse the stack trace and extract the line number/file, and identify the first time the generated JSP class name is found. Then use the line number there to get the source line number. Let me know if you need help - the first part is not very difficult, the second may be a bit tricky. Costin Thanks, Paulo Gaspar -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 9:24 AM On Thu, 17 May 2001, Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. Yes, debugging is a problem in the current jasper implementation. One solution, as you mentioned, is to hack the class to include line numbers that match the JSP file ( the line number is an annotation in the class ). There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. I'm not an expert in debugging, but that sound like a reasonable solution. You'll have to wait a bit for the implementation - I'm still fighting with cleaning and merging the runtime. Costin - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you
Re: Jasper performance
On Thu, 17 May 2001, Geir Magnusson Jr. wrote: Really? I was told that it was a complete myth - that *no one* has to debug the Java code - they just debug the JSP code. My JSP experience occurred a few years agoe, and I remember bewildering error messages - however, I just assumed that was straightened out. I am not trying to toss gasoline on what could be another conflagration - I am just generally interested in the subject. It has been resolved in debuggers ( I know Forte supports that, I'm sure other IDEs do too ), but so far nobody contributed code back to jasper. The problem is not difficult, as I said in a previous mail there are good solutions for more complex cases ( like XSTL debuggers - I just found out that there are quite a few already ). ( I'm hoping whoever does this to include some support for emacs too, I want it to jump to the bad line ! :-) Ovidiu, are you subscribed to this list ?? Costin
Re: Jasper performance
I totally agree with this. Improving performance of JSP page translation or compilation are very low priorities. Runtime performance is important, and as Christopher mentioned, good error reporting is very important. Glenn Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. - -- -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
