Re: Jasper performance/3.3 tag pooling
Hey Mel, I'll use this as a chance to explain some thoughts I've recently had on tag pooling. Maybe you and others have comments. Mel Martinez wrote: Hi folks! I'm still overwhelmed with other priorities (new job, house-hunting, moving, etc.) to be able to help again, but I managed over the last day or so to get caught up with the list (for the most part - skipped some major threads along the way). Hope things are working out well. I know where you're coming from. We just moved and are now in the process of putting in a yard. On tag-pooling: I am +1 on implementing a tag-cache on a per-page basis and tag-pooling on an application basis. If/when we move to the interfaces and toolkit metaphor I proposed for Jasper34, then page-based caching can best be done by enabling access to a tag-cache via the page-life-cycle handler, the JspPageHandler interface. I agree. I spent some time last week looking at possible optimizations. The general ideas were: - pool tag handler objects per application. This could still be turned on/off at runtime via module and is already available. - cache (re-use) handlers per page - i.e. only get the handler from the pool once (when it is needed). If application wide pooling is disabled then we'll just new a handler. To be clear: there won't be any data structures (stacks, etc.) for caching -- it will just be done via the rendering. - call most setters only once -- when the tag handler is first obtained. Attributes that are runtime expressions will need to be called every time. - in some situations we can pull out the tag handler initialization code from inside do loops. Currently, if a Tag is inside of a BodyTag, the initialization for the inner tag is done inside of the do-while(BodyTag.doEndBody() == EVAL_BODY_TAG). The inner tag initializations only need to be done once before the start of the do-while. Pulling the inner initializations/checks out of the do loop would be a nice thing, but it won't be as important once the tags and setter calls are re-used for the whole page. - maybe get rid of some redundant do-while(false) constructs and various other minor clean ups. So, to summarize: Each tag handler will be obtained once per page (either from pool or via new). Setters will be called once per handler per page. Both obtaining the handler and calling the setters will be done on an as needed basis. Yes, that will mean a synch call to get each handler when it's first used if application wide pooling is enabled. So far, I was thinking that there would still be one tag handler object per custom tag in the jsp page. For example, given: x:iterate count=100 x:someTag/ /x:iterate x:someTag/ there would still be 3 tag handler instances obtained. The spec says the someTag handler could be reused because tag scope allows it. But then we have the question of when to reuse handlers if scope allows it. For example, if the above was changed to: x:iterate count=100 x:someTag attrib=abc/ /x:iterate x:someTag attrib=xyz/ We could reuse the handler instance but would have to call the setters for attrib each time. What's more expensive, setter call or just using a different handler? Of course this depends on tag usage and what the setters are doing. So, for now, I was planning on not implementing tag handler reuse across multiple uses of a custom tag in the same jsp. This can, of course, be improved in the future. I am very much -1 on basing any sort of caching or pooling relative to the request thread using thread-local variables because it assumes the Thread will be pooled and reused - this may not be true in all containers and Jasper should be container-independent. I agree. After the above stuff gets done, I hope that we won't have to do any thread local variable stuff. I originally thought such optimizations would be nice, but am now hoping that they won't be needed. I'd like to make sure that all this stuff will (eventually) work with tomcat 4 and future tomcats as well. Basically, one should think in terms of what temporal and access scope different objects have: request, page, application, etc. The JspPageHandler persists between requests for a page and should be used to represent access to objects and services concerned with the life-cycle of a page beyond one request, such as modification checking, name mangling, recompilation/loading as needed and clearly, tag-object caching. Thus, it seems logical that tags could be retrieved via either directly from methods on the page handler or from a tag cache manager object retrieved from the page handler. Casey, do you see any problems following such a general design philosophy? You are the lord gawd of the tag-pooling code, so I will defer to your judgement. I agree with the general design philosophy. But don't defer to much :) (The more I learn about Tomcat the more I realize how much I don't know.) I haven't looked though all the new
Re: Jasper performance/3.3 tag pooling
I agree. I spent some time last week looking at possible optimizations. The general ideas were: - pool tag handler objects per application. This could still be turned on/off at runtime via module and is already available. - cache (re-use) handlers per page - i.e. only get the handler from the pool once (when it is needed). If application wide pooling is disabled then we'll just new a handler. To be clear: there won't be any data structures (stacks, etc.) for caching -- it will just be done via the rendering. - call most setters only once -- when the tag handler is first obtained. Attributes that are runtime expressions will need to be called every time. If you can get only these three in, and only that, it would help at least me/WebWork *immensely*. Heeding your advice regarding the examples I sent to you I've rewritten all tags to be reusable, and allow set-methods to be called only once, so I should see a huge improvement in performance. If it still performs bad, it's either my fault, or the rest of Tomcat :-) regards, Rickard
Re: Jasper performance/3.3 tag pooling
Thanks for sending the test application. I've been using monthlist.jsp while testing rendered then hand bashed code. After some preliminary bashing :) I think it will be possible to call all setters (except runtime expressions) only once per run of the page. Also, we can create / obtain tag handler instances only once per page run. That is an excellent start! Anyway... after tweaking the rendered code, I was disappointed that there wasn't much of a change in the performance when I hit the page directly. The optimized version was even a bit slower :( So, I hooked it up to a profiler. It said that most of the time was being spent in webwork.util.ValueStack.findValue which is called from some of the setters and some of the do[Start|End]Tag methods. Yeah, it's the by far most used method, and also where most of the magic happens (name-value conversions). I've been tweaking it some, but probably not enough. So, Is there any way you could send me a new webwork.jar (no need for the whole .war) that has tag implementations optimized for tag instance and setter call reuse?. If you can move some/many of the calls to ValueStack.findValue out of the doStart/doEnd methods, that should help. I will start working on this after the weekend. However I am also rewriting most of the tags for a new and simpler syntax, which is going to take some coding time too, since all the examples need to be modified, docs, etc. I will let you know as soon as possible though when the opts. are done. Yes, I know this doesn't solve the problem of why weblogic was faster for you, but any tag related optimizations we can make should help other tag users (me and my day job included). Yup, that was the basic idea when I started this thread :-) Glad to see things happen! :-) If you send a new jar, I'll test it out and let you know. Will do. /Rickard
Re: Jasper performance/3.3 tag pooling
[EMAIL PROTECTED] wrote: Jasper performance is a high priority thing for us (nr 1. on our system performance fixing list actually), so if possible, absolutely. I can't say I've delved that far into Jasper yet though. It was a bit hard to read. I'm working on a refactoring of jasper, and easy to read is a big priority. It's moving a bit slower than I expected - now I'm back on planning stage after hearing so much bad things about using XSLT :-) But I think there are ways to make ( almost ) everyone happy. I hope in few weeks ( after JavaOne ) we'll have something that works for both 1.1 and 1.2 ( the first part of refactoring ), and we can start the fun part - optimizing it. Sounds like a good plan :-) I'd be happy to help out with commenting on the code. Code contribution is tricky right now (lots of other OpenSource projects to manage :-). The main problem is to avoid one synchronized call for each tag ( new _is_also_ a synchronized call on most VMs - it synchronizes the heap !!! ). Whoa.. I didn't know that.. that's an important factor to consider then. Tomcat3.3 has one synchronized call to get the worker thread from the pool - and that doesn't seem to show up in any performance tests. As long as the synchs are few, and the block itself is short, you should be ok. We could keep a pool of pools ( probably for each context ). The local pools can be unsynchronized - so we'll have only one sync per page. Each context will have a set of pools ( with the size == the largest number of concurent requests for that context ). And we can of course be agressive in shrinking the pools that were not used recently. That could work. We can try to do some of this in 3.3 space - or in a branch of 3.3 ( jasper34 is going to take a while, and I have a feeling you need results fast ). The big date for me is 2002-01-01. Before that I need some kind of assurance that it'll improve greatly 'till that time, or we need to look at alternatives (*shiver*). So sooner than later, sure, but preferably righter than sooner. Get this stuff right, and I think you'll get lots of new supporters and keep old ones ;-) Regarding your application - I think we know the problem. There are few things you can do to reduce the memory use, and you could cache and recycle intermediary objects in each tag instance - you could benefit a lot from the reuse of the tags. I will start looking at optimizations on my side of things to, for sure. My code isn't extremely optimized, I know that, but OTOH I've tested the same code with WebLogic 5.0 (just to test) and it was wy faster, so there's definitely room on the JSP side to improve things. The memory savings of the thread pooling are not helping your application yet ( since they are very small compared with the rest ), yet you pay the synchronization price ( a lot - since you make heavy use of tags ). With few changes in your app and few changes in the tag pool we could reverse the situation - but if you disable the pool there's little chance you could increase the performance above the current level. Yup, that seems reasonable. You mentioned in your private email that there was a lot of MethodDescriptor's being created. Please look into where these are coming from. They could be associated with the tag setters. Pool/cache these and you'll probably see some immediate results. /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
RE: Jasper performance/3.3 tag pooling (XSLT)
Hi Costin, -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 23, 2001 10:20 AM ... I'm working on a refactoring of jasper, and easy to read is a big priority. It's moving a bit slower than I expected - now I'm back on planning stage after hearing so much bad things about using XSLT :-) But I think there are ways to make ( almost ) everyone happy. ... Considering your day job, I already had to change my diagnostic from you don't know what you are messing with to you know too much about XSLT and then you think it is easy for everybody. Still, I do not believe that using XSLT is a good idea. It goes a bit against your own motivation of making Jasper more readable to make the entry level lower for potential contributors. Besides learning about Jasper, many will have to learn about XSLT. And XSLT is not that easy to grasp. Worse, it is elusive - one tends to think to have it all figured out before time, and then fall easy prey of really big traps. I am not going to argue with you about the CPU cost issue, since your POV seems to be that everybody should manually compile JSPs for production and that developers have good enough machines anyway. We could start another holly war on this one, but I do not have the motivation. I think that many people prefer doing it without manual compilation, but we better just agree to disagree on that. Anyway, for me, the main issue is the complexity of XSLT being an obstacle for Jasper's code readability/maintenance. In case you are wondering, I do not intend to use JSP for presentation (I keep that Velocity is better at that) but I still think it as a lot of potential as a server side scripting mechanism, with taglibs providing an interesting way of defining simple dialects for common problems and embedded Java filling the gaps. For simple logic, it looks to have potential. Have fun, Paulo Gaspar
Re: Jasper performance/3.3 tag pooling
Rickard, Thanks for sending the test application. I've been using monthlist.jsp while testing rendered then hand bashed code. After some preliminary bashing :) I think it will be possible to call all setters (except runtime expressions) only once per run of the page. Also, we can create / obtain tag handler instances only once per page run. Anyway... after tweaking the rendered code, I was disappointed that there wasn't much of a change in the performance when I hit the page directly. The optimized version was even a bit slower :( So, I hooked it up to a profiler. It said that most of the time was being spent in webwork.util.ValueStack.findValue which is called from some of the setters and some of the do[Start|End]Tag methods. So even though the setters and tag handler news were optimized, there wasn't much change in page evaluation time. (A strange note: under the profiler, the optimized version ran in about 75% of the time it took the un-optimized version. Not sure why the big difference, but I have seen variations like that before when code is run under a profiler.) So, Is there any way you could send me a new webwork.jar (no need for the whole .war) that has tag implementations optimized for tag instance and setter call reuse?. If you can move some/many of the calls to ValueStack.findValue out of the doStart/doEnd methods, that should help. Yes, I know this doesn't solve the problem of why weblogic was faster for you, but any tag related optimizations we can make should help other tag users (me and my day job included). If you send a new jar, I'll test it out and let you know. -casey Rickard Öberg wrote: [EMAIL PROTECTED] wrote: Jasper performance is a high priority thing for us (nr 1. on our system performance fixing list actually), so if possible, absolutely. I can't say I've delved that far into Jasper yet though. It was a bit hard to read. I'm working on a refactoring of jasper, and easy to read is a big priority. It's moving a bit slower than I expected - now I'm back on planning stage after hearing so much bad things about using XSLT :-) But I think there are ways to make ( almost ) everyone happy. I hope in few weeks ( after JavaOne ) we'll have something that works for both 1.1 and 1.2 ( the first part of refactoring ), and we can start the fun part - optimizing it. Sounds like a good plan :-) I'd be happy to help out with commenting on the code. Code contribution is tricky right now (lots of other OpenSource projects to manage :-). The main problem is to avoid one synchronized call for each tag ( new _is_also_ a synchronized call on most VMs - it synchronizes the heap !!! ). Whoa.. I didn't know that.. that's an important factor to consider then. Tomcat3.3 has one synchronized call to get the worker thread from the pool - and that doesn't seem to show up in any performance tests. As long as the synchs are few, and the block itself is short, you should be ok. We could keep a pool of pools ( probably for each context ). The local pools can be unsynchronized - so we'll have only one sync per page. Each context will have a set of pools ( with the size == the largest number of concurent requests for that context ). And we can of course be agressive in shrinking the pools that were not used recently. That could work. We can try to do some of this in 3.3 space - or in a branch of 3.3 ( jasper34 is going to take a while, and I have a feeling you need results fast ). The big date for me is 2002-01-01. Before that I need some kind of assurance that it'll improve greatly 'till that time, or we need to look at alternatives (*shiver*). So sooner than later, sure, but preferably righter than sooner. Get this stuff right, and I think you'll get lots of new supporters and keep old ones ;-) Regarding your application - I think we know the problem. There are few things you can do to reduce the memory use, and you could cache and recycle intermediary objects in each tag instance - you could benefit a lot from the reuse of the tags. I will start looking at optimizations on my side of things to, for sure. My code isn't extremely optimized, I know that, but OTOH I've tested the same code with WebLogic 5.0 (just to test) and it was wy faster, so there's definitely room on the JSP side to improve things. The memory savings of the thread pooling are not helping your application yet ( since they are very small compared with the rest ), yet you pay the synchronization price ( a lot - since you make heavy use of tags ). With few changes in your app and few changes in the tag pool we could reverse the situation - but if you disable the pool there's little chance you could increase the performance above the current level. Yup, that seems reasonable. You mentioned in your private email that there was a lot of MethodDescriptor's being created. Please look into where these
Re: Jasper performance/3.3 tag pooling
[EMAIL PROTECTED] wrote: We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That is one solution, but what do you do with the pool after page request? I'm not sure I understand. Each thread has a number of associated objects that are recycled and reused - a Request object will stick with a thread. Same can be done for the tag pools - except that this may use a lot of memory ( but less than allocating/freeing ). It is possible to use a middle ground, or tune this - but for maximum performance you can have a local pool in each Request. What I was considering is this: a pool is a managed resource, since it can shrink and grow. After a page request, you will want to reuse the pool and its contents, but you can't stick the contents into a global pool, because at next request how are you to know how many instances of a tag to put in the request-associated pool. I.e., thread-associated pools perform local optimizations per request/page, but for pools to be of any use they need to do global optimizations, i.e. pool the tags in a global pool. Otherwise you'll have lots of pools of tags and no way to manage them globally. Example: Page request. No pools created yet. Page uses tag foo 10 times. Pool, associated with request/thread, created containing one foo tag instance. Concurrent page request with the above. Page uses tag foo 5 times. Another pool, associated with request/thread, created containing one foo tag instance. Both requests end: what to do with the pools? 1 Keep separate, each containing one instance 2 Merge so that there is only one pool for foo tags If 1, then there's no way to do proper global optimizations, i.e. say don't create more than x tag instances. If 2, then at next request, what should the page associated foo pool contain? 1 tag instance, 2 tag instances..? The only way to really know is for the page to do its thing, and pull the tags from the global pool. And then you're in synch land again. Do you understand now? I realize the above is a bit fuzzy... I hope that Costin will be able to reproduce what I found. I hope not :-) Again - thanks for doing the tests and checking the code, and hope to see more contributions and maybe few patches :-) Jasper performance is a high priority thing for us (nr 1. on our system performance fixing list actually), so if possible, absolutely. I can't say I've delved that far into Jasper yet though. It was a bit hard to read. IMHO the right answer is depends. And it depends on the actual use case, on how many objects are created and where the synchronization occurs. For tomcat, where we expect hundred of concurent requests and each would create almost a hundred object ( that was the case in 3.0 ) - I doubt any VM could make a difference. Have you then considered the middle way I proposed, where each page creates the tags it needs, but only once. In my experience, the performance kills is in iterative and recursive use of tags, e.g.: iterate foo/ /iterate which will create one foo tag per iteration in 3.2. *That's* the biggest problem IMHO. Creating the iterate and foo tag once for that page is not, considering that the overhead of managing pools of these tag instances is more resource draining, and hard to do properly on a global scale. I might be wrong though 8-) Nah. :-) /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
[EMAIL PROTECTED] wrote: But in tomcat 3.3 we do a different trick - the thread pool is maintaining a local storage for each thread, and it's passing it to the worker. The only synchronization in tomcat is in getting a thread from the thread pool - besides that we shouldn't need anything else. Right now we keep the Request/Request pair - so we have a one-to-one relation between main request and thread ( in other words, the request is allways in the same thread ). Whatever attributes/notes you store in the request will be equivalent with thread-local data, without any sync. That's good. But you still have the global vs local issue outlined in my last post. Purely local, as you outline above, and proper pooling management becomes tricky (=how to know how many tag instances to use for the particular handled request). Sorry, I should have been more specific. Of course object reuse can be a good performance optimization. I'm just saying that you gotta balance it with regard to modern JVM's ability to manage memory and new objects. Optimize locally in code, and let the JVM do it globally. That's just IMHO though, so feel free to disagree :-) :-) I was thinking the reverse - optimize globally, let the VM optimize locally, but it depends on definitions... The problem is that global management is a tricky business. Say you keep global pools of tag instances. Not only do you have to manage one particular tag pool properly (not too big, not too small), you also have to consider how many other pools are active (too many active pools with too many tags, and you have a memory consumption problem). Then factor in the temporal issues, i.e. that some tags are used more often than others sometimes, and you have a situation that I dare you to find an optimal algorithm to solve the pool management. And when/if you have that algorithm, I can bet you quite a bunch of dolares that it won't be more efficient than letting the JVM manage the memory and making sparse new's per page anyway. Or (...standard disclaimer...) what am I missing? :-) /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
Rickard Öberg wrote: [EMAIL PROTECTED] wrote: We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That is one solution, but what do you do with the pool after page request? I'm not sure I understand. Each thread has a number of associated objects that are recycled and reused - a Request object will stick with a thread. Same can be done for the tag pools - except that this may use a lot of memory ( but less than allocating/freeing ). It is possible to use a middle ground, or tune this - but for maximum performance you can have a local pool in each Request. What I was considering is this: a pool is a managed resource, since it can shrink and grow. After a page request, you will want to reuse the pool and its contents, but you can't stick the contents into a global pool, because at next request how are you to know how many instances of a tag to put in the request-associated pool. I.e., thread-associated pools perform local optimizations per request/page, but for pools to be of any use they need to do global optimizations, i.e. pool the tags in a global pool. Otherwise you'll have lots of pools of tags and no way to manage them globally. Example: Page request. No pools created yet. Page uses tag foo 10 times. Pool, associated with request/thread, created containing one foo tag instance. Concurrent page request with the above. Page uses tag foo 5 times. Another pool, associated with request/thread, created containing one foo tag instance. Both requests end: what to do with the pools? 1 Keep separate, each containing one instance 2 Merge so that there is only one pool for foo tags If 1, then there's no way to do proper global optimizations, i.e. say don't create more than x tag instances. If 2, then at next request, what should the page associated foo pool contain? 1 tag instance, 2 tag instances..? The only way to really know is for the page to do its thing, and pull the tags from the global pool. And then you're in synch land again. Do you understand now? I realize the above is a bit fuzzy... I hope that Costin will be able to reproduce what I found. I hope not :-) Again - thanks for doing the tests and checking the code, and hope to see more contributions and maybe few patches :-) Jasper performance is a high priority thing for us (nr 1. on our system performance fixing list actually), so if possible, absolutely. I can't say I've delved that far into Jasper yet though. It was a bit hard to read. IMHO the right answer is depends. And it depends on the actual use case, on how many objects are created and where the synchronization occurs. For tomcat, where we expect hundred of concurent requests and each would create almost a hundred object ( that was the case in 3.0 ) - I doubt any VM could make a difference. Have you then considered the middle way I proposed, where each page creates the tags it needs, but only once. In my experience, the performance kills is in iterative and recursive use of tags, e.g.: iterate foo/ /iterate which will create one foo tag per iteration in 3.2. *That's* the biggest problem IMHO. Creating the iterate and foo tag once for that page is not, considering that the overhead of managing pools of these tag instances is more resource draining, and hard to do properly on a global scale. I just had an idea (dangerous things) regarding tag pooling optimizations. When Jasper translates a page it should be able to generate information about which tag handler classes it needs, and for each tag handler, the profile of the attribute/value pairs. At runtime Jasper could make one call to a tag pooling synchronized method to get all available instances of the tag handlers it needs. Any tag handlers/attribute profiles it couldn't obtain, it would have to create. Then it would manage its own local tag pool during the execution of the request. On termination of the request, after the page has been committed to the remote client, it would make one call to a tag pooling synchronized method to recycle/add the tag handlers it used. So there would only be 2 synchronized methods per page for implementing tag pooling regardless of how many tags are used. And you have the benifits of minimizing JVM memory usage by sharing a global JVM tag pool. Regards, Glenn -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance/3.3 tag pooling
Glenn Nielsen wrote: I just had an idea (dangerous things) regarding tag pooling optimizations. When Jasper translates a page it should be able to generate information about which tag handler classes it needs, and for each tag handler, the profile of the attribute/value pairs. At runtime Jasper could make one call to a tag pooling synchronized method to get all available instances of the tag handlers it needs. Any tag handlers/attribute profiles it couldn't obtain, it would have to create. Then it would manage its own local tag pool during the execution of the request. On termination of the request, after the page has been committed to the remote client, it would make one call to a tag pooling synchronized method to recycle/add the tag handlers it used. So there would only be 2 synchronized methods per page for implementing tag pooling regardless of how many tags are used. And you have the benifits of minimizing JVM memory usage by sharing a global JVM tag pool. I would be ok with that, as long as I can set the size of the pool to be zero, since I don't trust Java coders to outperform the optimizations done by the JVM as outlined in earlier posts. /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
Glenn Nielsen wrote: [snip] I just had an idea (dangerous things) regarding tag pooling optimizations. When Jasper translates a page it should be able to generate information about which tag handler classes it needs, and for each tag handler, the profile of the attribute/value pairs. At runtime Jasper could make one call to a tag pooling synchronized method to get all available instances of the tag handlers it needs. Any tag handlers/attribute profiles it couldn't obtain, it would have to create. Then it would manage its own local tag pool during the execution of the request. On termination of the request, after the page has been committed to the remote client, it would make one call to a tag pooling synchronized method to recycle/add the tag handlers it used. So there would only be 2 synchronized methods per page for implementing tag pooling regardless of how many tags are used. And you have the benifits of minimizing JVM memory usage by sharing a global JVM tag pool. that sparked something.. If we just bite the bullet and modify Jasper to correctly determine tag handler usage within a page (i.e. not get a new (from pool or new) tag each time) then we can probably get the best of both worlds -- local reuse of tag handler variables (I don't like to call that a pool) plus application wide tag pooling. If application wide tag pooling is too expensive, then we can chunk it. Regardless, the per page reuse should be a big gain. Next step might be to determine if, in certain situations, we can not re-call tag setters. The spec says that attributes set should be persistent. Rickard's test case would really benefit from setter optimization because most time was spent in body of setter methods. Any comments? btw, If i start tinkering, should I work with jasper34 or the 3.3 stuff? -casey Regards, Glenn -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance/3.3 tag pooling
On Wed, 23 May 2001, Casey Lucas wrote: btw, If i start tinkering, should I work with jasper34 or the 3.3 stuff? Difficult question... The problem with jasper34 is that it doesn't work yet ( the one in proposals/jasper34 - I still have to move it in the new repository ). Mea culpa - I tried to make big changes instead of the old slow evolution... I'll start importing the current jasper33 in the new repository and make sure it builds, and use it as the first step for 34. Then I'll stick with the step-by-step evolution. Merging with jasper40 and xslt can wait a bit. Costin -casey Regards, Glenn -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance/3.3 tag pooling
On Wed, 23 May 2001 [EMAIL PROTECTED] wrote:
On Wed, 23 May 2001, Casey Lucas wrote:
btw, If i start tinkering, should I work with jasper34 or the 3.3 stuff?
Difficult question...
The problem with jasper34 is that it doesn't work yet ( the one in
proposals/jasper34 - I still have to move it in the new repository ). Mea
culpa - I tried to make big changes instead of the old slow evolution...
I'll start importing the current jasper33 in the new repository and make
sure it builds, and use it as the first step for 34. Then I'll stick with
the step-by-step evolution. Merging with jasper40 and xslt can wait a bit.
I know Costin loves evolutionary change :-), and it's certainly a valid
approach to Jasper.
But there is also another approach we should consider - a green-field
recoding of at least some of the major components (conforming to an
agreed-upon overall architecture, of course).
What has struck me about the custom tags pooling question is that we're
trying to make a non-optimizing compiler into an optimizing compiler from
the ground up, rather than taking advantage of the decades of compiler
writing experience and designing one from the top down.
Just as a for instance about why we might want to consider this, let's
look at a mythical iterate tag with a nested tag inside:
x:iterate from=0 to=999
x:foo name=bar value=%= ... an expression %/
/x:iterate
Today, Jasper will do a new FooTag() plus all of the associated tag
setup, inside the loop, 1000 times. Tomorrow, using tag pooling can mean
that there will only be at most one create, but you've still got the
allocate/deallocate overhead plus the redundant setXxxx calls.
However, it's perfectly legal for the pseudo-code generated by this page
to look like this (see the examples in the JSP spec):
IterateTag iteratetag = new IterateTag(); // or allocate from a pool
iterateTag.setPageContext(...);
iterateTag.setParent(null);
iterateTag.setFrom(0);
itearteTag.setTo(999);
FooTag fooTag = new FooTag(); // Or allocate from a pool
fooTag.setPageContext(...); // Lifted out of the loop
fooTag.setParent(iterateTag); // Lifted out of the loop
fooTag.setName(bar);// Lifted out of the loop
... iterate-start implementation ... {
fooTag.setValue(expression);
fooTag.doStartTag();
fooTag.doEndTag();
} ... iterate-end implementation ...
fooTag.release();
fooTag = null; // or recycle to a pool
iterateTag.release();
iterateTag = null; // or recycle to a pool
In other words, you pay either *one* object creation or *one*
allocate/deallocate for the x:foo tag instance, and you don't waste your
time doing stuff inside the loop that only needs to be done once.
Optimizing compilers can be made smart enough to do things like this, as
long as you take the time to build in the appropriate knowledge of the
language (in this case, JSP syntax and semantics) that you are
translating. If you happened to take a compiler class along a degree
path, this kind of thing will probably look familiar.
I think we owe it to ourselves to consider whether a completely new
effort, at least for the compiler, might get us to better results
(probably with less overall development effort) than an evolutionary
approach based on the current code.
NOTE: For most of the rest of the overall problem (the PageContext
implementation, how Jasper fits in with the servlet container, and so
on) evolution is probably a very reasonable strategy. On the compiler,
though, I'm not so sure.
Costin
Craig
Re: Jasper performance/3.3 tag pooling
On Wed, 23 May 2001, Craig R. McClanahan wrote: I know Costin loves evolutionary change :-), and it's certainly a valid approach to Jasper. But there is also another approach we should consider - a green-field recoding of at least some of the major components (conforming to an agreed-upon overall architecture, of course). NOTE: For most of the rest of the overall problem (the PageContext implementation, how Jasper fits in with the servlet container, and so on) evolution is probably a very reasonable strategy. On the compiler, though, I'm not so sure. If we are talking about the compiler ( or code generator ): I partially agree, the current architecture will get a lot of pressure from more complex optimizations or tricks. But before we can even start to change the generator we need to do the initial refactoring and get the other components in order ( runtime, etc). We can also get some optimizations in, and use that to learn what's needed from a new generator architecture. I just don't think the new generator can happen in a 3.4 space - my goal is just to enable an effort to rewrite it, and gather as much information as possible about it's requirements. In any case - whatever happens in the current generator with regard to generated code will still be usefull for any new generator architecture. And if certain optimizations can't be done - that's even better, because it would help us understand what's needed. I had big hopes for an XSLT based generator, and I still think it may be a good way to implement the code generator - and I hope to hear other ideas. Costin
Re: Jasper performance/3.3 tag pooling
Casey, We hope to freeze 3.3 for a release in the next weeks. If you feel the changes are reasonably small and will not create problems - you can still do it. What about: 1. We copy the code from 3.3 as it is in jasper34 area ( except that we rename the package ). 2. We make sure it builds and runs in 3.3 - with a minimal set of changes: - copy jasper34.jar in lib/container - edit server.xml and replace JspInterceptor with JspInterceptor34 3. You can start making the optimizations, and I can start doing the refactoring, making sure we keep everything functional. 4. In time we can merge the changes from 4.0, add the new interfaces proposed by Mel, add a new generator and so on - while making sure the tests are passing and jasper is stable. This is unlikely to be finished before 3.3 is released, but we can make sure we keep passing all the test suites and we can release milestones of jasper34 for who needs the performance enhancements. When the code is ready we can make a 3.3.1 release and replace the old jasper ( and same for when ajp14 is ready ). What I tried in proposals/jasper34 is just wrong and against the basic ideas of evolution - and I realize that. Just give me 2 days, I need to finish with xalan ( we just had the 2.1.0 release and I need to finish my work on it ) Costin On Wed, 23 May 2001, Casey Lucas wrote: Costin Craig, I agree with both of you. The optimizations that Craig mentioned are exactly the ones that I was hoping to add. Yes, the least fun part will probably be adding to the existing generator code. But, until a new generating architecture is in place, I think it would be good to go ahead and try to add the code to the existing jasper. When the next gen jasper gets a lot of momentum, we can add the tag related optimizations to it. -casey [EMAIL PROTECTED] wrote: On Wed, 23 May 2001, Craig R. McClanahan wrote: I know Costin loves evolutionary change :-), and it's certainly a valid approach to Jasper. But there is also another approach we should consider - a green-field recoding of at least some of the major components (conforming to an agreed-upon overall architecture, of course). NOTE: For most of the rest of the overall problem (the PageContext implementation, how Jasper fits in with the servlet container, and so on) evolution is probably a very reasonable strategy. On the compiler, though, I'm not so sure. If we are talking about the compiler ( or code generator ): I partially agree, the current architecture will get a lot of pressure from more complex optimizations or tricks. But before we can even start to change the generator we need to do the initial refactoring and get the other components in order ( runtime, etc). We can also get some optimizations in, and use that to learn what's needed from a new generator architecture. I just don't think the new generator can happen in a 3.4 space - my goal is just to enable an effort to rewrite it, and gather as much information as possible about it's requirements. In any case - whatever happens in the current generator with regard to generated code will still be usefull for any new generator architecture. And if certain optimizations can't be done - that's even better, because it would help us understand what's needed. I had big hopes for an XSLT based generator, and I still think it may be a good way to implement the code generator - and I hope to hear other ideas. Costin
Re: Jasper performance/3.3 tag pooling
I think the changes will be too big to add before the 3.3 freeze. I like your alternative and will start bashing the jsp-java files to get something optimal before changing the jasper34 generator code. I'll let you know when I have something that looks interesting so that it can be discussed before taking the time to modify the generator code. -casey [EMAIL PROTECTED] wrote: Casey, We hope to freeze 3.3 for a release in the next weeks. If you feel the changes are reasonably small and will not create problems - you can still do it. What about: 1. We copy the code from 3.3 as it is in jasper34 area ( except that we rename the package ). 2. We make sure it builds and runs in 3.3 - with a minimal set of changes: - copy jasper34.jar in lib/container - edit server.xml and replace JspInterceptor with JspInterceptor34 3. You can start making the optimizations, and I can start doing the refactoring, making sure we keep everything functional. 4. In time we can merge the changes from 4.0, add the new interfaces proposed by Mel, add a new generator and so on - while making sure the tests are passing and jasper is stable. This is unlikely to be finished before 3.3 is released, but we can make sure we keep passing all the test suites and we can release milestones of jasper34 for who needs the performance enhancements. When the code is ready we can make a 3.3.1 release and replace the old jasper ( and same for when ajp14 is ready ). What I tried in proposals/jasper34 is just wrong and against the basic ideas of evolution - and I realize that. Just give me 2 days, I need to finish with xalan ( we just had the 2.1.0 release and I need to finish my work on it ) Costin On Wed, 23 May 2001, Casey Lucas wrote: Costin Craig, I agree with both of you. The optimizations that Craig mentioned are exactly the ones that I was hoping to add. Yes, the least fun part will probably be adding to the existing generator code. But, until a new generating architecture is in place, I think it would be good to go ahead and try to add the code to the existing jasper. When the next gen jasper gets a lot of momentum, we can add the tag related optimizations to it. -casey [EMAIL PROTECTED] wrote: On Wed, 23 May 2001, Craig R. McClanahan wrote: I know Costin loves evolutionary change :-), and it's certainly a valid approach to Jasper. But there is also another approach we should consider - a green-field recoding of at least some of the major components (conforming to an agreed-upon overall architecture, of course). NOTE: For most of the rest of the overall problem (the PageContext implementation, how Jasper fits in with the servlet container, and so on) evolution is probably a very reasonable strategy. On the compiler, though, I'm not so sure. If we are talking about the compiler ( or code generator ): I partially agree, the current architecture will get a lot of pressure from more complex optimizations or tricks. But before we can even start to change the generator we need to do the initial refactoring and get the other components in order ( runtime, etc). We can also get some optimizations in, and use that to learn what's needed from a new generator architecture. I just don't think the new generator can happen in a 3.4 space - my goal is just to enable an effort to rewrite it, and gather as much information as possible about it's requirements. In any case - whatever happens in the current generator with regard to generated code will still be usefull for any new generator architecture. And if certain optimizations can't be done - that's even better, because it would help us understand what's needed. I had big hopes for an XSLT based generator, and I still think it may be a good way to implement the code generator - and I hope to hear other ideas. Costin
Re: Jasper performance/3.3 tag pooling
Hi! [EMAIL PROTECTED] wrote: Could you send a small page ( and the taglibs ) and the test conditions ? I have sent taglib+page and test conditions to Costin. We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That is one solution, but what do you do with the pool after page request? That was planned for later, in jasper34 space - but if you send code that shows a significant degradation in performance in some reasonable situations we can still fix it in 3.3. I hope that Costin will be able to reproduce what I found. Regarding the default - I think it should be enabled, mostly because that will require people to make sure their taglibs are working corectly in a pooled environment. It should be possible to disable it on a per application basis, and we need to make sure we document how to do that. I agree. I don't think the design is bad - just the implementation of the pool, but this is the first attempt with the goal of making sure the pooling work, optimizations will come later. The current experience with tomcat performance evolution from 3.1 to 3.3 shows that recycling objects has a huge effect ( if implemented corectly ) - and I see no reason why jasper would be different. Agree, object reuse can have nice effects. But the hash+sync overhead must be balanced against the benefits of the actual object reuse. I'm looking forward to the VM where the allocation and GC will be free ( so far most do synchronize on object allocation ). I had a quick chat with the JRockit guys (the server JVM dudes) last J1, and they went Waaah! when I told them that JBoss pools EJB instances. The JVM would do a much better job with it in their opinion. If that's true or not I don't know, but I have seen other statements like that from other people. /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
Casey Lucas wrote: Also, did you run the tests with and without tag pooling enabled on the same version of tomcat? (By adding removing TagPoolManagerInterceptor.) Yes. The only thing changed between the runs was the TPMI flag. This is a bad design. Basically, any gains you get from reusing tags are lost due to the overhead and general performance decrease you get by using hashtables+synchronized. Hash lookup is done once per jsp page - when the jsp page is first run. After that, it's basically a synchronized push / pop pair on all subsequent runs of the page. In the future we can even get rid of the synch by using thread local storage... one step at a time though. :) Yup, that might improve things. Although then it depends on how ThreadLocal is implemented. I guess at some point deep down it has to do synchronized, although I haven't checked. I've looked at pages generated by other more efficient JSP compilers (e.g. Resin), and they generally use another approach: Only reuse a tag within a particular page, and don't use a synchronized pool to do it. Just pass the instance around within the generated code. What is important is not primarily the global optimization gained by using pools, but the local optimization gained by not creating tags for each iteration in a iterative custom tag. Yes reusing a tag handler within a single page will be more efficient (for that particular page) but I would guess that once we change to pool per thread there's no way newing the tags at each use will be faster (at least for a busy site with lots of tags.) And didn't say you should either. Right now 3.2 new's tags, and it's slow. I'm saying, do it the middle way, but new'ing it once per page, and then reuse within that page. No global optimization problems (i.e. when to grow/shrink the pool), no synch problems, no shared pool considerations, no hash lookup. All the good stuff and none of the bad. As I said, last time I checked Resin worked this way, and I think we can all agree that it's pretty snappy. This is wy more efficient, and also avoids the suboptimization of trying to reuse objects using Java code, something which is more efficiently handled by modern JVMs' memory management (i.e. creating objects using new is pretty snappy compared with Java-coded pools). I disagree. I've found that object reuse can be a good performance optimization. Sorry, I should have been more specific. Of course object reuse can be a good performance optimization. I'm just saying that you gotta balance it with regard to modern JVM's ability to manage memory and new objects. Optimize locally in code, and let the JVM do it globally. That's just IMHO though, so feel free to disagree :-) So, please remove the tag pooling, and do it right. If you don't believe me, do some benchmarking or something. I have done benchmarking and pooling enabled always wins. :) Maybe you can send some examples so that we can try and track down the problems. As above, I've sent code to Costin. But I'll forward it to you to, so you check it out. Lastly, I would be absolutely thrilled to find out that the fault was mine, i.e. some silly coding or config error. :-) regards, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance/3.3 tag pooling
On Tue, 22 May 2001, Rickard Öberg wrote: Hi! [EMAIL PROTECTED] wrote: Could you send a small page ( and the taglibs ) and the test conditions ? I have sent taglib+page and test conditions to Costin. Thanks, I'll try to add it to the tests dir. We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That is one solution, but what do you do with the pool after page request? I'm not sure I understand. Each thread has a number of associated objects that are recycled and reused - a Request object will stick with a thread. Same can be done for the tag pools - except that this may use a lot of memory ( but less than allocating/freeing ). It is possible to use a middle ground, or tune this - but for maximum performance you can have a local pool in each Request. I hope that Costin will be able to reproduce what I found. I hope not :-) Again - thanks for doing the tests and checking the code, and hope to see more contributions and maybe few patches :-) I had a quick chat with the JRockit guys (the server JVM dudes) last J1, and they went Waaah! when I told them that JBoss pools EJB instances. The JVM would do a much better job with it in their opinion. If that's true or not I don't know, but I have seen other statements like that from other people. IMHO the right answer is depends. And it depends on the actual use case, on how many objects are created and where the synchronization occurs. For tomcat, where we expect hundred of concurent requests and each would create almost a hundred object ( that was the case in 3.0 ) - I doubt any VM could make a difference. Costin
Re: Jasper performance/3.3 tag pooling
On Tue, 22 May 2001, Rickard Öberg wrote: Hash lookup is done once per jsp page - when the jsp page is first run. After that, it's basically a synchronized push / pop pair on all subsequent runs of the page. In the future we can even get rid of the synch by using thread local storage... one step at a time though. :) Yup, that might improve things. Although then it depends on how ThreadLocal is implemented. I guess at some point deep down it has to do synchronized, although I haven't checked. It does ! :-) But in tomcat 3.3 we do a different trick - the thread pool is maintaining a local storage for each thread, and it's passing it to the worker. The only synchronization in tomcat is in getting a thread from the thread pool - besides that we shouldn't need anything else. Right now we keep the Request/Request pair - so we have a one-to-one relation between main request and thread ( in other words, the request is allways in the same thread ). Whatever attributes/notes you store in the request will be equivalent with thread-local data, without any sync. Sorry, I should have been more specific. Of course object reuse can be a good performance optimization. I'm just saying that you gotta balance it with regard to modern JVM's ability to manage memory and new objects. Optimize locally in code, and let the JVM do it globally. That's just IMHO though, so feel free to disagree :-) :-) I was thinking the reverse - optimize globally, let the VM optimize locally, but it depends on definitions... Costin
Re: Jasper performance/3.3 tag pooling
Hi Rickard, Could you send a small page ( and the taglibs ) and the test conditions ? Most tests I run show a (significant) improvement in performance. We know the pool is synchronized and that may create problems under heavy load, and we know how to fix this ( by using a per/thread pool without synchronization ). That was planned for later, in jasper34 space - but if you send code that shows a significant degradation in performance in some reasonable situations we can still fix it in 3.3. Regarding the default - I think it should be enabled, mostly because that will require people to make sure their taglibs are working corectly in a pooled environment. It should be possible to disable it on a per application basis, and we need to make sure we document how to do that. I don't think the design is bad - just the implementation of the pool, but this is the first attempt with the goal of making sure the pooling work, optimizations will come later. The current experience with tomcat performance evolution from 3.1 to 3.3 shows that recycling objects has a huge effect ( if implemented corectly ) - and I see no reason why jasper would be different. I'm looking forward to the VM where the allocation and GC will be free ( so far most do synchronize on object allocation ). Costin On Mon, 21 May 2001, Rickard Öberg wrote: Hi! Ok, so now I've tested the Jasper performance with the 3.3 tag pooling fix. The test was performed with a medium complexity page using lots of iterative custom tags in a hierarchical fashion (i.e. tags within tags). Results: The page ran slower, and above all the response time varied greatly (between 250ms and 460ms, whereas without tag pooling we got between 230ms and 340ms). Looking at the generated code, I'm not particularly surprised: it uses (it seems) a real shared tag pool, so using tags will execute code that needs to be synchronized. This is a bad design. Basically, any gains you get from reusing tags are lost due to the overhead and general performance decrease you get by using hashtables+synchronized. I've looked at pages generated by other more efficient JSP compilers (e.g. Resin), and they generally use another approach: Only reuse a tag within a particular page, and don't use a synchronized pool to do it. Just pass the instance around within the generated code. What is important is not primarily the global optimization gained by using pools, but the local optimization gained by not creating tags for each iteration in a iterative custom tag. This is wy more efficient, and also avoids the suboptimization of trying to reuse objects using Java code, something which is more efficiently handled by modern JVMs' memory management (i.e. creating objects using new is pretty snappy compared with Java-coded pools). So, please remove the tag pooling, and do it right. If you don't believe me, do some benchmarking or something. /Rickard
Re: Jasper performance/3.3 tag pooling
Rickard, Can you please send in some complete examples? Also, did you run the tests with and without tag pooling enabled on the same version of tomcat? (By adding removing TagPoolManagerInterceptor.) My experience has been that if the jsp uses many tags, then pooling is a big performance gain. See comments below. Rickard Öberg wrote: Hi! Ok, so now I've tested the Jasper performance with the 3.3 tag pooling fix. The test was performed with a medium complexity page using lots of iterative custom tags in a hierarchical fashion (i.e. tags within tags). Results: The page ran slower, and above all the response time varied greatly (between 250ms and 460ms, whereas without tag pooling we got between 230ms and 340ms). Looking at the generated code, I'm not particularly surprised: it uses (it seems) a real shared tag pool, so using tags will execute code that needs to be synchronized. Yes, everytime you obtain a tag, there is a synch that occurs for the pool that exists for each uniquely named tag and specific set of attributes. There is actually a set of named pools per application context. This is a bad design. Basically, any gains you get from reusing tags are lost due to the overhead and general performance decrease you get by using hashtables+synchronized. Hash lookup is done once per jsp page - when the jsp page is first run. After that, it's basically a synchronized push / pop pair on all subsequent runs of the page. In the future we can even get rid of the synch by using thread local storage... one step at a time though. :) I've looked at pages generated by other more efficient JSP compilers (e.g. Resin), and they generally use another approach: Only reuse a tag within a particular page, and don't use a synchronized pool to do it. Just pass the instance around within the generated code. What is important is not primarily the global optimization gained by using pools, but the local optimization gained by not creating tags for each iteration in a iterative custom tag. Yes reusing a tag handler within a single page will be more efficient (for that particular page) but I would guess that once we change to pool per thread there's no way newing the tags at each use will be faster (at least for a busy site with lots of tags.) This is wy more efficient, and also avoids the suboptimization of trying to reuse objects using Java code, something which is more efficiently handled by modern JVMs' memory management (i.e. creating objects using new is pretty snappy compared with Java-coded pools). I disagree. I've found that object reuse can be a good performance optimization. So, please remove the tag pooling, and do it right. If you don't believe me, do some benchmarking or something. I have done benchmarking and pooling enabled always wins. :) Maybe you can send some examples so that we can try and track down the problems. Thanks for taking a look at tag pooling. I look forward to refining the implementation. -casey
RE: Jasper performance
-Original Message- From: Eduardo Pelegri-Llopart [mailto:[EMAIL PROTECTED]] Sent: Saturday, May 19, 2001 12:01 AM (sorry for the response lag, unfortunatly I don't read tomcat very frequently) ... * Using XSTL for templating... Like Jon and some others, I think that XSTL is a bit too complicated (and memory expensive) to be my favorite templating mechanism. But it is a widely used as a transformation mechanism. Some thoughts on the role of XML, JSP XSLT at http://java.sun.com/products/jsp/html/JSPXML.html This follown-the-bunch atitude with XSLT and other technologies, always reminds me of lemmings. =;o) ... Have fun, Paulo Gaspar
RE: Jasper performance - JMX
In the case of DoS, I don't believe a bit on trusted tags and such stuff. Why monitoring the tags at all if while(true) is so easy. I mean, the front door is wide open, why care about that little window? The only way to close everything is by monitoring the Servlets and allow setting some limitation on time per request. And that does not look so hard to do since Servlets are passive entities (meaning that they do NOT call Tomcat, Tomcat calls them) and their basic implementation (base class) belongs to Tomcat. The standard way to do it would be by instrumenting the Servlets, either directly or with a wrapper using JMX. JMX is not a J2EE exclusive and it is not that complex either. Additional advantages of implementing JMX in Tomcat: - Collecting other statistics; - Diagnostics; - Changing settings on the fly; - etc. BTW, I am not the first person talking about JMX in this list. Have fun, Paulo Gaspar -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 3:02 AM On Thu, 17 May 2001, Glenn Nielsen wrote: I guess he's refering to DOS attacks ( like a while(true); in java code or allocating lots of memory ). You won't have much of a templating language if you don't allow some sort of looping. Kind of hard to restrict that. True, but if you have a set of trusted tags, including looping tags, and no untrusted code except the one that calls the tags you could do a lot to control the resources. For example the tags ( or jasper generated code ) could check for time execution limits, or how many resources are allocated. It would be nice if self monitoring were built into Tomcat so sysads could track statistics on performance of the JVM, Tomcat in general, and individual servlets/JSP's. Even setting thresholds when automated email notifications could be done. Lets give sysadmins the information they need, then they can take action against problem users. Yes, that would be an interesting hack.. I was thinking about JPDA - it would be possible to check the memory use for each thread, associate it with the user code. Also, it is possible to store the time when entering/exiting user code, and have a deamon thread check if any thread is spending too much time. ( the time monitoring part can be done without jpda - but to monitor the memory I don't know other solution ). ( well, I know - I remember a certain tool that was used to manipluate bytecodes and add instrumentation before all allocations - but that's far too difficult for the time we have available ). I still think that using the SecurityManager implementation in Tomcat with a well tuned security polciy can provide one of the most secure environments available for running web based applications. This is just my opinion, feel free to try and convince me some other technology is more secure. I'll not even try :-) You're right, but there are some things that we could add to also control some resource usage ( memory and cpu time ). Costin
Re: Jasper performance
Jon Stevens wrote:
on 5/17/01 12:47 PM, Glenn Nielsen [EMAIL PROTECTED] wrote:
But now that both Tomcat 3.2 and Tomcat 4 support the Java SecurityManager
you can control security at the container level regardless of whether someone
is using the CFM servlet, velocity, CoCoon, JSP, etc.
Not true.
http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html
Hashtable strings = new Hashtable();
int i=0;
while (true)
{
strings.put (dead+i, new StringBuffer(99));
}
There is no amount of security that will prevent someone from putting that
into their JSP page other than disabling the ability to put scriptlets into
things. If you do that, then you are simply where you should have been in
the first place...using Velocity.
Yes, but using velocity templates limits a great deal what customers
can do when compared to a general purpose servlet container where
web applications can be deployed. There is a great deal more to
security than just preventing a 'trusted user' who can publish content
from doing something stupid. No where in your YMTD document do I see
anything about security, just your reference above to a trusted user
DoS. Heck, if one of my customers wants to use Velocity, they can do
so if it can be deployed as a web application, but it will have to
run within the security policies we set for the Tomcat Java SecurityManager. ;-)
Regards,
Glenn
--
Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder|
MOREnet System Programming | * if iz ina coment. |
Missouri Research and Education Network | */ |
--
Re: Jasper performance
At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. - Dennis Doubleday email: [EMAIL PROTECTED] yourfit.com, Inc. web: http://www.yourfit.com/
Re: Jasper performance
Dennis Doubleday wrote: At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
RE: Jasper performance - JMX
On Fri, 18 May 2001, Paulo Gaspar wrote: In the case of DoS, I don't believe a bit on trusted tags and such stuff. Why monitoring the tags at all if while(true) is so easy. I mean, the front door is wide open, why care about that little window? Well, what I said was trusted tags and only pure jsps, no user code. That means a webapplication will be allowed only if it has no libraries, no servlets, no user tags - and only JSPs that are not using % %, but only a limited set of tags. All the above is equivalent with what Jon described ( except syntactic sugar ). It'll be foreach instead of #foreach, etc - so at least you'll use the JSP syntax, and your app will work in normal containers as well. It will be a valid JSP in a valid webapp - but of course the container can't be called a servlet container ( it doesn't run even servlets :-) I doubt too many installations of Velocity are set up to disallow user code - it's not too much you can do. It'll be secure - probably because nobody will care to use such a thing :-) And if you allow any user code - all the #foreach prevention of DOS goes away. ( it's like the famous thin air firewall - it's the most secure, but pointless ). The only way to close everything is by monitoring the Servlets and allow setting some limitation on time per request. And that does not look so hard to do since Servlets are passive entities (meaning that they do NOT call Tomcat, Tomcat calls them) and their basic implementation (base class) belongs to Tomcat. I agree. With a security manager you control almost everything - except the amount of CPU and memory. It is possible ( and not very difficult ) to monitor the CPU amount used by a servlet - just a trivial interceptor that will log the moment of entry and exit in servlet code. You'll need a thread that periodically checks if a servlet/jsp is hunged or takes too much time, and disable the whole app if it is the case. It is also possible to periodically check the amount of memory used by each thread. ( it's much harder - but I think you can hack a bit with JPDA and do it ). ( like OptimizeIt work for example ). Both will be far more usefull than security by removing all usefull features. BTW, I am not the first person talking about JMX in this list. JMX is great - I like it a lot, it can be added easily by a module - but I don't think it's a big priority for now. After 3.3 is released we can discuss new features as add-on modules ( I have a few ). Costin
RE: Jasper performance
Velocity does do a lot to minimize the risk you mention, but while we're using stupid coding tricks, couldn't you do the following in Velocity? #* assume strings is a Vector *# #set ($strings = $request.getParameter(strings))) #foreach ($string in $strings) $strings.addElement($string.clone()); #end -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 8:50 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Dennis Doubleday wrote: At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
RE: Jasper performance
It isn't concurrent. -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 10:52 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Jef Newsom wrote: Velocity does do a lot to minimize the risk you mention, but while we're using stupid coding tricks, couldn't you do the following in Velocity? #* assume strings is a Vector *# #set ($strings = $request.getParameter(strings))) #foreach ($string in $strings) $strings.addElement($string.clone()); #end Good try :) Assuming it is a Vector, I am pretty convinced that wouldn't work because you will get a ConcurrentModificationException when you modified the Vector. geir -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 8:50 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Dennis Doubleday wrote: At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders... -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
RE: Jasper performance
I wrote a test script, and assuming (which the docs say it does) that Velocity uses the iterator() instead of elements() when it runs up against a vector, then all is well. If elements() is used, it goes into infinite loop land. My mistake. -Original Message- From: Jef Newsom Sent: Friday, May 18, 2001 11:44 AM To: [EMAIL PROTECTED] Subject: RE: Jasper performance It isn't concurrent. -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 10:52 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Jef Newsom wrote: Velocity does do a lot to minimize the risk you mention, but while we're using stupid coding tricks, couldn't you do the following in Velocity? #* assume strings is a Vector *# #set ($strings = $request.getParameter(strings))) #foreach ($string in $strings) $strings.addElement($string.clone()); #end Good try :) Assuming it is a Vector, I am pretty convinced that wouldn't work because you will get a ConcurrentModificationException when you modified the Vector. geir -Original Message- From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]] Sent: Friday, May 18, 2001 8:50 AM To: [EMAIL PROTECTED] Subject: Re: Jasper performance Dennis Doubleday wrote: At 07:51 AM 5/18/01, Geir wrote: Those aren't comparable, 'Velocity templates' and 'general purpose servlet container', because Velocity is just a template tool - you still need the servlet and servlet container. That was exactly my point when I said Velocity doesn't really do anything to prevent DOS attacks, either. Any Velocity app requires a servlet back-end, and if I'm going to host user apps, I'm going to have to let them install servlets, in which case they can put in the same ever-looping code. Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders... -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
Re: Jasper performance
on 5/18/01 1:37 AM, Paulo Gaspar [EMAIL PROTECTED] wrote: All Velocity has is a #foreach. This is a fully functional looping construct that prevents you from screwing things up and still gets the job done. On the #foreach and DoS issues, I would use makes it harder instead of prevents in . But it also makes much harder to mix logic and presentation too. Have fun, Paulo Gaspar Please try to come up with a case where it isn't prevented and then give me a reason for makes it harder. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
on 5/18/01 6:50 AM, Geir Magnusson Jr. [EMAIL PROTECTED] wrote: Definitely. Agreed. There is no silver bullet. I guess the point is that you remove a little of the risk, as a designer can't % while(true); % (although as JSP compilers get better, I am sure this stuff can be found and flagged...) This is not intended to disparage designers : it's just a different talent set. My use of color has been described as dangerous, bordering on criminal :) geir I guess the point that we are trying to make with Velocity is that the design of the Velocity tool is far cleaner than having to rely on improved compilers to do the work for you. -jon
Re: Jasper performance
(sorry for the response lag, unfortunatly I don't read tomcat very frequently) Hi Jon. The problem with taglibs is that there is no restriction on the ability to put Java code in the page. It is part of the JSP specification to be able to do that. Sure, you can disable it (as Costin said), but then you are breaking the JSP specification. And I know how important standards are to everyone... -jon I didn't see any follow-up clarifying this but apologies if I missed it. JSP 1.2 has the notion of a TagLibraryValidator that is associated with a tag library. This can be used to portably validate different assertions on your JSP page. This could be used to, for example, check that some actions are nested within others, or that some portions of your JSP page conform in one way or another. A TagLibraryValidator can also be used to disable scriptlets. === On some of the other points in this thread: * Using XSTL for templating... Like Jon and some others, I think that XSTL is a bit too complicated (and memory expensive) to be my favorite templating mechanism. But it is a widely used as a transformation mechanism. Some thoughts on the role of XML, JSP XSLT at http://java.sun.com/products/jsp/html/JSPXML.html * XML in Jasper... The JSP expert group has considered for a while adding a portable mechanism for transformations on a JSP page. Such a transformation would be defined as operating on the XML reprsetnation of a JSP page. I expect we will discuss again this feature for JSP 1.3, although there are no specific dates for that yet. IMHO, - eduard/o Jon Stevens wrote: on 5/17/01 6:46 AM, Dennis Doubleday [EMAIL PROTECTED] wrote: At 08:35 PM 5/16/01, Jon wrote: Also, there is a reason for the #foreach... http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html Jon, I agree with most of your points. I am a new Velocity user and I am very impressed by its combination of power and simplicity. Reading/writing XSLT specs is an exercise in masochism. However, I don't see how Velocity is really avoiding the fundamental problem described in the document you referenced. If you are an ISP hosting Velocity-based pages, you are certainly going to have to let that 14 year old kid install both templates and class files; templates by themselves won't accomplish much. So the incompetent or malicious client can easily make the same mistake in his Command class that he would have made in the JSP page, and therefore also create a DOS attack on all servlets hosted in that JVM. No? Controlling what goes into the Context is key. There is nothing stating that you have to give the 14 year old access to creating .java files. Instead, the alternative approach is to place certain objects within the Context which allow the 14 year old to do a limited set of actions. This follows along with the Pull Model: http://jakarta.apache.org/turbine/pullmodel.html This is the approach that Tea http://opensource.go.com/ uses as well as the general idea behind taglibs. The problem with taglibs is that there is no restriction on the ability to put Java code in the page. It is part of the JSP specification to be able to do that. Sure, you can disable it (as Costin said), but then you are breaking the JSP specification. And I know how important standards are to everyone... -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
on 5/18/01 3:01 PM, Eduardo Pelegri-Llopart [EMAIL PROTECTED] wrote: I didn't see any follow-up clarifying this but apologies if I missed it. JSP 1.2 has the notion of a TagLibraryValidator that is associated with a tag library. This can be used to portably validate different assertions on your JSP page. This could be used to, for example, check that some actions are nested within others, or that some portions of your JSP page conform in one way or another. A TagLibraryValidator can also be used to disable scriptlets. Making it easy to disable scriptlets and breaking the specification are two different things. It is part of the JSP specification to be able to use scriptlets. By disabling them, you are encouraging people to break the specification. If people open their eyes and realize that the specification is fundamentally flawed, then they will see that the alternative (and correct) approach would be to remove scriptlets from the specification entirely. If you go and do that, you are back at square one and might as well have used another technology (which took this into consideration from the start) in the first place. Again, we are back to my point that simply taking an already terrible technology (ASP) and trying to fit it into a Java model (JSP) is a really brain damaged thing to do. In order to fix the problems in ASP (and therefore JSP) one has two options: 1) Hack the exiting technology (JSP) until it kinda works. 2) Come up with an alternative technology that took consideration of these points from the start. Clearly Sun chooses to take the approach of #1 because Sun would look like a bunch of boneheads if they dropped JSP at this time. So, stick with what you have and *try* to push it on people for as long as you can until someone wakes up and realizes that what was pushed down their throat by Sun was a stupid decision. On some of the other points in this thread: * Using XSTL for templating... Like Jon and some others, I think that XSTL is a bit too complicated (and memory expensive) to be my favorite templating mechanism. But it is a widely used as a transformation mechanism. Some thoughts on the role of XML, JSP XSLT at http://java.sun.com/products/jsp/html/JSPXML.html The humor in that article is that it all falls down as people realize that they want to be able to take form input from something. If you notice, all the arrows point towards the client. There are NO arrows that point from the client towards the server. Once again, I'm not impressed because the document doesn't address the larger vision of what people need to do in the real world. * XML in Jasper... The JSP expert group has considered for a while adding a portable mechanism for transformations on a JSP page. Such a transformation would be defined as operating on the XML reprsetnation of a JSP page. I expect we will discuss again this feature for JSP 1.3, although there are no specific dates for that yet. Yawn. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
Sorry, Jon, we disagree. TagLibraryValidators *are* part of the JSP 1.2 specification. They are quite flexible and one of the simplest uses is to express that some tags cannot appear. Scriptlets are exposed as jsp:scriptlet tags. - eduard/o Jon Stevens wrote: on 5/18/01 3:01 PM, Eduardo Pelegri-Llopart [EMAIL PROTECTED] wrote: I didn't see any follow-up clarifying this but apologies if I missed it. JSP 1.2 has the notion of a TagLibraryValidator that is associated with a tag library. This can be used to portably validate different assertions on your JSP page. This could be used to, for example, check that some actions are nested within others, or that some portions of your JSP page conform in one way or another. A TagLibraryValidator can also be used to disable scriptlets. Making it easy to disable scriptlets and breaking the specification are two different things. It is part of the JSP specification to be able to use scriptlets. By disabling them, you are encouraging people to break the specification.
Re: Jasper performance
on 5/18/01 4:55 PM, Eduardo Pelegri-Llopart [EMAIL PROTECTED] wrote: Sorry, Jon, we disagree. TagLibraryValidators *are* part of the JSP 1.2 specification. Go back and read what I wrote again. I'm not saying that TagLibraryValidators aren't part of the specification. They are quite flexible and one of the simplest uses is to express that some tags cannot appear. Scriptlets are exposed as jsp:scriptlet tags. Correct. And Scriptlets are also part of the specification. See: JSP Specificaton: 2.11.2 So, making scriptlets disappear is invalidating the specification. No where does it say in 2.11.2 that it is legal to remove those entities. In fact, the document specifically states: All JSP containers must support scripting elements based on the Java programming language. I don't see how you can disagree with me on the point of removing scriptlets from an application will invalidate the specification. So, in effect, you are talking about modifications of the specification. If you are going to go to that length (which is removing something that has been in the specification since day one and is documented all over the place in examples), you might as well dump the entire JSP technology and start over. I also find humor how you simply ignore my other points. Maybe if you shove enough stuff under the rug, no one will figure it out. -jon
Re: Jasper performance - JMX
on 5/18/01 10:48 AM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: On Fri, 18 May 2001, Jon Stevens wrote: Correct, however some bright monkey decided to add % % into the JSP specification. So, if you disable that, you are breaking the specification. In other words, it is a bad design in the first place. That is the point. Not using % % doesn't brake the spec - it is still a valid JSP page. Preventing people from using % % breaks the spec. It is documented, read: 2.11.2 Not allowing user code ( servlets, user libs, user tags ) is indeed breaking the specification - and a server that is doing that shouldn't claim it supports server side java - but the same is true for turbine, isn't it ? Nope. Turbine doesn't require the use of user modifiable server side java in order to do its job. You can simply put tags into the context and make them available to the end users. I find % % quite usefull for prototyping - but if you don't like it - don't use it. I wouldn't tuch a server where it's disabled, or where all I can do is use an obscure scripting language. #1. Luckily you aren't developing web applications as your job. #2. obscure? The developer can make the decision - not a smart monkey that thinks he knows better. We aren't trying to work with java developers. We are trying to work with designers and template engineers. Please don't tell me mixing logic and presentation is the worst sin - It is. In all of the years that I have been doing web app development (about 6-7 years now), this is the most common complain that I have heard about. most systems allow that ( HTML + javascript, perl, php, jsp, asp, XSLT, servlets, and so on ). I don't know any successful system where this is not allowed. Velocity. Yes Costin, it has taken many years to get to the point of what we have been trying to get to, but it has finally happened. Eureka! Get used to it. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
RE: Jasper performance
... XSLT is not perfect - neither is HTTP or HTML or any other standard. But because Apache and many other organizations are implementing and using it - I think they'll be around for a while :-) Costin Costin, You are still missing the point (as I wrote in my previous post on this thread) of the big volume of jars XSLT use would add to Tomcat minimal distro for JSP compilation. Also, maybe the impact of using XSLT is quite larger than you expect. Especially via the huge memory demands and respective side effects. I know that you know well about the impact of creating a zillion of objects in memory (as it happens in even the most minimal use of XSLT), so I am not going to detail on what those side effects are. What is worse, XSLT is usualy way too much trouble because so many things are so counter intuitive. The trap is that the really simple stuff works as expected, but when you need something just a little bit more complicated, sudenly you bump into something that does not work as expected, and you spend the next few days trying to figure out what. Jon is a... er... politness challenged person and is a bit too obcessed with Velocity, but he is basically right. I used XSLT a lot and I can tell you that is easy for Velocity to be a better alternative (and it is not my favourite one!) just because XSLT is such a poor template mechanism. One year ago, I was in the XSLT-LIST, bought Michael Kay book (kind of a very essential XSLT bible) and I was thinking about writing everything I could in XSLT just as any other XSLT fanatic... until I found out that: - I was wasting way too much time with XSLT complexities (why do you think the XSLT-LIST has all that traffic? Do you know that list?); - The outcome was always slower and more resource intensive than I found natural (and that the alternatives, as I discovered later). I would still use XSLT for some XML transformation tasks, but never for templating. I use Velocity and, in terms of both functionality and resource usage, it would really be a much better choice than XSLT (which is not hard to do, anyway). But it would NOT be my 1st choice for this (see my previous posting for further detail). It would be my 2nd choice in a very cut down version. Notice that I was not born with this knowledge. I went trough a lot of trouble to learn this. Now, you can - Pay attention to what I say and chek it out; - Think that the W3C is always right (HAHAHAHAHA!); - Think that your intuition is always right, which is another way of saying that you were born knowing it all. Sorry Costin for assuming that you do not have much experience with XSLT and its alternatives, but I just think you are too smart to keep defending it if you had that experience. Have fun, Paulo Gaspar
RE: Jasper performance
I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps
RE: Jasper performance
From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
Re: Jasper performance
At 08:35 PM 5/16/01, Jon wrote: Also, there is a reason for the #foreach... http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html Jon, I agree with most of your points. I am a new Velocity user and I am very impressed by its combination of power and simplicity. Reading/writing XSLT specs is an exercise in masochism. However, I don't see how Velocity is really avoiding the fundamental problem described in the document you referenced. If you are an ISP hosting Velocity-based pages, you are certainly going to have to let that 14 year old kid install both templates and class files; templates by themselves won't accomplish much. So the incompetent or malicious client can easily make the same mistake in his Command class that he would have made in the JSP page, and therefore also create a DOS attack on all servlets hosted in that JVM. No? - Dennis Doubleday email: [EMAIL PROTECTED] yourfit.com, Inc. web: http://www.yourfit.com/
RE: Jasper performance
I agree with you, So I'll suggest you Forte IE, which has a great debugger. that allow you to trace exception form the stack to the class, then to the original source code. It also allow break point, which is the least. The only regrets about this tools : - need a big computer (600mz and more then 256 Mo, it's great with 512) - not GPL At 14:17 17/05/2001 +0100, you wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
answer inline... -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 1:51 PM I don't know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). Nops. The XSLT would just be the way to implement code templates for the java code generation step of compiling a JSP. (And the nice guy thing would not be a problem it was the other way since that text could be encoded to become valid XML.) If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Nothing like that. But Java Bytecode generation would probably be faster, not because we would be able to make the most performant compiler in the world, but because a lot of I/O operations would be saved (writing the java code, calling/loading the compiler, the compiler producing the output class file, loading the class file from disk). Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Welcome to the club. But it helps if you follow the list. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. About right. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. That is not a new idea. Just not possible - there is no such functionality on the Java compiler. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Costin complained about that in this list not long ago. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps I also hope for that and it never happens. =;o) Have fun, Paulo Gaspar
RE: Jasper performance
On Thu, 17 May 2001, Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. Yes, debugging is a problem in the current jasper implementation. One solution, as you mentioned, is to hack the class to include line numbers that match the JSP file ( the line number is an annotation in the class ). There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. I'm not an expert in debugging, but that sound like a reasonable solution. You'll have to wait a bit for the implementation - I'm still fighting with cleaning and merging the runtime. Costin - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
Answer inline: -Original Message- From: Christopher Kirk [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 3:17 PM From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. You still can take a look at the intermediate java file to understand what is going on. It is not trivial, but at least it makes clear if the bug is in your code or on Jasper's. Anyway, Jasper sure could probably be improved to give good enough information about JSP location of the source of an exception... and you could help implementing that. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. There are two problems with that approach: 1) The technical complexity of such project (even using tools like BCEL); 2) Then it would be harder for you to find out if an exception would be caused by some bug in your code or in Jasper's code. Anyway, you can also contribute to such thing. Some pointers that can help you starting: - You can find the above referred BCEL at http://bcel.sourceforge.net/ - You can check Rhino Javascript's compiler on how do they directly compile something (javascript in this case) into Java Bytecode: http://www.mozilla.org/rhino/ - Maybe Go's Tea template compiler would be a closer match for JSP: http://opensource.go.com/ Have fun, Paulo Gaspar - Chris. ...
RE: Jasper performance
Even less GPL (commercial) but still for free, not so clumsy and with a better GUI and Debugger... Borland's JBuilder 4 Foundation. =:o) Have fun, Paulo Gaspar -Original Message- From: Kaneda K [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 3:55 PM To: [EMAIL PROTECTED] Subject: RE: Jasper performance I agree with you, So I'll suggest you Forte IE, which has a great debugger. that allow you to trace exception form the stack to the class, then to the original source code. It also allow break point, which is the least. The only regrets about this tools : - need a big computer (600mz and more then 256 Mo, it's great with 512) - not GPL At 14:17 17/05/2001 +0100, you wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
Ok Chris. Now I undertand. Working with JSP when I found that type of mistakes I went to see the .java generated. There I matched the line number so I could see which was the bad code. Then I had to go to the jsp to fix it. It is not the best of the world, but is that that bad? On the other hand we can trick the compiler to make it fit the line numbers, just generating really hugly java code. package jklfe.fel.ef; import java.util.*; // line 1 user.code.here() // line 2 Just an idea. Chau, Gaston From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. - - Este mensaje ha sido enviado desde el servidor de Webmail de Tournet S.A. http://www.tournet.com.ar/
RE: Jasper performance
Hi Carlos, On Thu, 17 May 2001, Carlos Gaston Alvarez wrote: I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). The JspParser object knows how to deal with that. And it's easy to convert it into a SAX event generator. The script will be equivalent with a CDATA section ( and you can have tag in a CDATA ). In 1.2 you also have the XML representation ( and we can use it to generate exactly the same type of sax events from a non-xml jsp file ) If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Not quite - look at XSLTC ( part of xml-xalan project ). This is intended only for pure JSP pages, with no java fragments. We are in a very similar situation with XSLTC - the page-generated code itself it's not very complex ( in fact it's much simpler in our case ), so we don't benefit a lot from javac optimizations. And no, this is not going to be a compilator ( but some cutpaste from another apache project :-) One important thing to keep in mind: sometimes there is a tradeof between compilation time and runtime performance. The more optimizations you do at compile time - the slower it is to compile, faster to run. That mean that would be more of an option for development time, I would still use the real compiler before deployment. This is a long term project - it's definitely something cool, but I think we should hold that until we solve more important issues. ( and compilation time can be improved few times by just using jikes ). Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. I like guessing, but it would help if you'ld also read the code, and maybe write some patches :-) Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. Not quite. Almost all of the time is spent doing javac compilation. The jsp-java translation is insignifiant. What we care about is getting the response time for the first request below a second ( so developer will not feel the delay and get fast feedback ). On my computer, using jikes - this is exactly the case, and I have a 300MHz laptop. I run a lot of XSLTs ( that's part of my job ) and I never felt a delay ( and benchmarks for most transforms I do are well bellow a second ). If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Unfortunately we can't compile from and to memory with the current javac, and even if it would be possible - the compiler is a complex application, doing optimizations, verifications, etc: I don't think the time for read/write source/class is predominant. Costin
RE: Jasper performance
There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. That sounds very nice. How does it work? Where/how do you feed the map file? (Just some documentation pointers to whatever feature you use would do.) Thanks, Paulo Gaspar -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 9:24 AM On Thu, 17 May 2001, Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. Yes, debugging is a problem in the current jasper implementation. One solution, as you mentioned, is to hack the class to include line numbers that match the JSP file ( the line number is an annotation in the class ). There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. I'm not an expert in debugging, but that sound like a reasonable solution. You'll have to wait a bit for the implementation - I'm still fighting with cleaning and merging the runtime. Costin - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
On Thu, 17 May 2001 [EMAIL PROTECTED] wrote: There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. There is a current JSR (#45) with a goal of producing a standard mapping format like this for non-Java languages that are translated into Java, and JSP is a pretty classic use case of this. I haven't heard much lately about this (will check more), but conceptually this approach seems like a good direction to go. Costin Craig
Re: Jasper performance
on 5/17/01 8:01 AM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Ok Chris. Now I undertand. Working with JSP when I found that type of mistakes I went to see the .java generated. There I matched the line number so I could see which was the bad code. Then I had to go to the jsp to fix it. It is not the best of the world, but is that that bad? If you think about *who* the target JSP audience is, Yes. Do you expect page designers to figure out problems with line numbers by going to look at the generated page in some temp directory somewhere? Gee, I hope not. More humor for you JSP believers: http://www.theregister.co.uk/content/4/18996.html You may also wish to read a complete pile of bullshit here: http://java.sun.com/products/jsp/jsp-asp.html My favorite: ASP TechnologyJSP Technology Reusable, Cross-Platform ComponentsNoJavaBeans, Enterprise JavaBeans, custom JSP tags Security Against System CrashesNoYes Memory Leak ProtectionNoYes Scripting LanguageVBScript, JScriptJava Customizable TagsNoYes Yea, JSP protects you from memory leaks and System Crashes. Yea Right. Oh yea, and ASP is lacking customizable tags...as if customizable tags is a good thing? JSP sucks. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
on 5/17/01 6:46 AM, Dennis Doubleday [EMAIL PROTECTED] wrote: At 08:35 PM 5/16/01, Jon wrote: Also, there is a reason for the #foreach... http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html Jon, I agree with most of your points. I am a new Velocity user and I am very impressed by its combination of power and simplicity. Reading/writing XSLT specs is an exercise in masochism. However, I don't see how Velocity is really avoiding the fundamental problem described in the document you referenced. If you are an ISP hosting Velocity-based pages, you are certainly going to have to let that 14 year old kid install both templates and class files; templates by themselves won't accomplish much. So the incompetent or malicious client can easily make the same mistake in his Command class that he would have made in the JSP page, and therefore also create a DOS attack on all servlets hosted in that JVM. No? Controlling what goes into the Context is key. There is nothing stating that you have to give the 14 year old access to creating .java files. Instead, the alternative approach is to place certain objects within the Context which allow the 14 year old to do a limited set of actions. This follows along with the Pull Model: http://jakarta.apache.org/turbine/pullmodel.html This is the approach that Tea http://opensource.go.com/ uses as well as the general idea behind taglibs. The problem with taglibs is that there is no restriction on the ability to put Java code in the page. It is part of the JSP specification to be able to do that. Sure, you can disable it (as Costin said), but then you are breaking the JSP specification. And I know how important standards are to everyone... -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. Really? I was told that it was a complete myth - that *no one* has to debug the Java code - they just debug the JSP code. My JSP experience occurred a few years agoe, and I remember bewildering error messages - however, I just assumed that was straightened out. I am not trying to toss gasoline on what could be another conflagration - I am just generally interested in the subject. Thanks geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
RE: Jasper performance
On Thu, 17 May 2001, Paulo Gaspar wrote: ... XSLT is not perfect - neither is HTTP or HTML or any other standard. But because Apache and many other organizations are implementing and using it - I think they'll be around for a while :-) Costin Costin, You are still missing the point (as I wrote in my previous post on this thread) of the big volume of jars XSLT use would add to Tomcat minimal distro for JSP compilation. AFAIK tomcat4 already includes a xslt engine ( part of JAXP1.1 ), and I thing 3.3 will have to update to JAXP1.1 also ( we're still using the old JAXP1.0.1 ). But there is a point I'm not missing - I do understand the concern regarding the size of the code. I would point again to XSLTC - I think there are many similarities. The XSLT compiler itself is quite big ( i.e. the piece that converts a XSTL sheet into a .class file - same as Jasper is converting JSPs ). But the transformation itself ( the generated code ), including all the runtime, is small enough to run on a palm pilot. And that doesn't matter too much anyway - for a small-memory deployment ( like an appliance or a CD runner ) you shouldn't even include the JSP generator, XSLT or javac. That's why we have JSPC - to pre-process the WAR file and precompile it. There is even more - for this case you'll probably want to use EmbededTomcat instead of server.xml-based runner, and we can pre-process web.xml too - so it may be possible to eliminate even the parser from the runtime-only version of tomcat. ( many embedded devices are pre-processing the .class files before deployment, some are even generating binary code - so generating .class files wouldn't work anyway ). Also, maybe the impact of using XSLT is quite larger than you expect. Especially via the huge memory demands and respective side effects. I know that you know well about the impact of creating a zillion of objects in memory (as it happens in even the most minimal use of XSLT), so I am not going to detail on what those side effects are. Did I mentioned XSTL is a standard - with quite a few implementations, a lot of competition and smart people, etc ? If you look at Tomcat3.0 you may conclude servlets are quite bad - zillion of objects in memory, big overhead, etc. There is a huge difference between a xslt implementation one year ago and one today. The same way as tomcat3.3 ( or 4.0 ) will generate only few dozen objects at run time ( ok, fewer in 3.3 :-), most XSTL engines are starting to recycle most objects. Take a look at DTM work in xalan ( and xsltc ). What is worse, XSLT is usualy way too much trouble because so many things are so counter intuitive. Well, HTML is counter intutive too ( H1 instead of bigger font ? ) :-) But many of the counter intuitive things ( like no side efects, or the way they treat variables ) do have a good reason behind it. And I suppose many will find things that are counter intuitive in java ( you can't change a String ?). The trap is that the really simple stuff works as expected, but when you need something just a little bit more complicated, sudenly you bump into something that does not work as expected, and you spend the next few days trying to figure out what. Yes, I know the feeling. But it's not something specific to XSLT, it happens to most programmers in most languages. Jon is a... er... politness challenged person and is a bit too obcessed Too bad for him. That's why filters were invented ( or the delete key ). I used XSLT a lot and I can tell you that is easy for Velocity to be a better alternative (and it is not my favourite one!) just because XSLT is such a poor template mechanism. Sorry, but I doubt it. One year ago, I was in the XSLT-LIST, bought Michael Kay book (kind of a very essential XSLT bible) and I was thinking about writing everything I could in XSLT just as any other XSLT fanatic... until I found out that: - I was wasting way too much time with XSLT complexities (why do you think the XSLT-LIST has all that traffic? Do you know that list?); - The outcome was always slower and more resource intensive than I found natural (and that the alternatives, as I discovered later). One year ago tomcat couldn't handle 50 requests per second. Try again today. Notice that I was not born with this knowledge. I went trough a lot of trouble to learn this. Now, you can - Pay attention to what I say and chek it out; - Think that the W3C is always right (HAHAHAHAHA!); - Think that your intuition is always right, which is another way of saying that you were born knowing it all. Well, I do pay attention to what you say - and my day job involves working on xslt. I don't think W3C is always right ( XML Schema ? Soap ? ), but it wasn't that right with HTML or HTTP either ( one request per connection ?) And I wasn't born knowing it - I spent a lot of time looking around and reading, and I have my own doubts. Sorry
RE: Jasper performance
You should rarely have to worry about mistakes in JSP if you are using tag libraries and beans to factor out all business logic. -Original Message- From: Kaneda K [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 6:55 AM To: [EMAIL PROTECTED] Subject: RE: Jasper performance I agree with you, So I'll suggest you Forte IE, which has a great debugger. that allow you to trace exception form the stack to the class, then to the original source code. It also allow break point, which is the least. The only regrets about this tools : - need a big computer (600mz and more then 256 Mo, it's great with 512) - not GPL At 14:17 17/05/2001 +0100, you wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. -
RE: Jasper performance
On Thu, 17 May 2001, Craig R. McClanahan wrote: On Thu, 17 May 2001 [EMAIL PROTECTED] wrote: There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. There is a current JSR (#45) with a goal of producing a standard mapping format like this for non-Java languages that are translated into Java, and JSP is a pretty classic use case of this. I haven't heard much lately about this (will check more), but conceptually this approach seems like a good direction to go. Well, there is a very interesting discussion going on xalan-dev, regarding an XSLT debugger. I saw a demo - and you can execute a XSLT step-by-step, see the line in the source xml that is processed, etc. Yes, mapping the line numbers is a missing feature in our code, but there are already people who made this work ( while using jasper with a debugger ). I hope someone will contribute some code back. Costin
RE: Jasper performance
On Thu, 17 May 2001, Paulo Gaspar wrote: There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. That sounds very nice. How does it work? Where/how do you feed the map file? (Just some documentation pointers to whatever feature you use would do.) Jasper already generates the line number as comments. What you would need to do is generate them as code ( like a static Hashtable or [] ), and then change the catch() code a bit. You'll need to parse the stack trace and extract the line number/file, and identify the first time the generated JSP class name is found. Then use the line number there to get the source line number. Let me know if you need help - the first part is not very difficult, the second may be a bit tricky. Costin Thanks, Paulo Gaspar -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 17, 2001 9:24 AM On Thu, 17 May 2001, Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. Yes, debugging is a problem in the current jasper implementation. One solution, as you mentioned, is to hack the class to include line numbers that match the JSP file ( the line number is an annotation in the class ). There is another solution - to generate the map ( java line number - JSP source line number). This is exactly how the .class annotation works, mapping bytecode to java source number. And will allow you to see both informations, and it's quite clean. I'm not an expert in debugging, but that sound like a reasonable solution. You'll have to wait a bit for the implementation - I'm still fighting with cleaning and merging the runtime. Costin - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you
Re: Jasper performance
On Thu, 17 May 2001, Geir Magnusson Jr. wrote: Really? I was told that it was a complete myth - that *no one* has to debug the Java code - they just debug the JSP code. My JSP experience occurred a few years agoe, and I remember bewildering error messages - however, I just assumed that was straightened out. I am not trying to toss gasoline on what could be another conflagration - I am just generally interested in the subject. It has been resolved in debuggers ( I know Forte supports that, I'm sure other IDEs do too ), but so far nobody contributed code back to jasper. The problem is not difficult, as I said in a previous mail there are good solutions for more complex cases ( like XSTL debuggers - I just found out that there are quite a few already ). ( I'm hoping whoever does this to include some support for emacs too, I want it to jump to the bad line ! :-) Ovidiu, are you subscribed to this list ?? Costin
Re: Jasper performance
I totally agree with this. Improving performance of JSP page translation or compilation are very low priorities. Runtime performance is important, and as Christopher mentioned, good error reporting is very important. Glenn Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. - Chris. -Original Message- From: Carlos Gaston Alvarez [mailto:[EMAIL PROTECTED]] Sent: 17 May 2001 12:51 To: [EMAIL PROTECTED] Subject: RE: Jasper performance I dont know if I really understood. You (some) are thinking to compile a jsp using xslt. I didnt know that that was possible. I mean, can a jsp be loaded as a dom object? I think that it cannot because a nice guy can write some code like % out.writeln(/html); % and that is, it is not more xml compliant. (please tellme if I am wrong). If the problem is the compilation time I can hardly believe that we can do something faster that a java compiler. Not because we dont have the skill, but because there are groups working in this kind of implementations. We should be blindly proud to think out compiler would be faster than theirs. Also, I consider that a waste of time. Let me do some guessing. I know, I should be reading the code, but I have been realy overloaded for the last months. Lets see if I have understood the problem. Saving a file for compilation is slow. So we need a faster solution. But, why is it slow? It sounds like we are saving the .java in file and then invoking a javacc process. The new process needs loads the compiler classes, compiles them (?) and then compiles our .java source. If we could have the compiler just loaded on our own memory space, we could invoke it saving the loading compiling (?) time. We could also send the .java 'file' in memory and expect the .class 'file' also in memory. Yes, may be we need some modifications in the compiler, but as part of the apache project we are at a good position for asking it an having a good answer. Chau, Gaston ps: sorry for doing so much guessing but next month I will have much more time. ps - This e-mail is intended only for the addressee(s) named above. It may contain confidential or privileged information and if you are not the intended addressee you are not authorized to disclose, copy, distribute or place any reliance on it whatsoever and we request that you inform our postmaster ([EMAIL PROTECTED]) that you have received this e-mail. Any attachment(s) to this message has been checked for viruses, but please rely on your own virus checker and procedures. If you contact us by e-mail, we will store your name and address to facilitate communications. - -- -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance
Jon Stevens wrote: on 5/17/01 6:46 AM, Dennis Doubleday [EMAIL PROTECTED] wrote: At 08:35 PM 5/16/01, Jon wrote: Also, there is a reason for the #foreach... http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html Jon, I agree with most of your points. I am a new Velocity user and I am very impressed by its combination of power and simplicity. Reading/writing XSLT specs is an exercise in masochism. However, I don't see how Velocity is really avoiding the fundamental problem described in the document you referenced. If you are an ISP hosting Velocity-based pages, you are certainly going to have to let that 14 year old kid install both templates and class files; templates by themselves won't accomplish much. So the incompetent or malicious client can easily make the same mistake in his Command class that he would have made in the JSP page, and therefore also create a DOS attack on all servlets hosted in that JVM. No? Controlling what goes into the Context is key. There is nothing stating that you have to give the 14 year old access to creating .java files. Instead, the alternative approach is to place certain objects within the Context which allow the 14 year old to do a limited set of actions. This follows along with the Pull Model: http://jakarta.apache.org/turbine/pullmodel.html This is the approach that Tea http://opensource.go.com/ uses as well as the general idea behind taglibs. The problem with taglibs is that there is no restriction on the ability to put Java code in the page. It is part of the JSP specification to be able to do that. Sure, you can disable it (as Costin said), but then you are breaking the JSP specification. And I know how important standards are to everyone... But now that both Tomcat 3.2 and Tomcat 4 support the Java SecurityManager you can control security at the container level regardless of whether someone is using the CFM servlet, velocity, CoCoon, JSP, etc. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html -- -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance
Geir Magnusson Jr. wrote: Christopher Kirk wrote: From my view, the problem with JSP-Java-Class isn't performance its debugging. JSP is hard to work with when you make a mistake, very often the error message is less than helpful. A very large step in improving this is by making the line number given by the stack trace match the line numbers of the JSP page. This currently is not the case because of the intermediate step of a java file. It would be beneficial to compile JSP straight to Java, complete with debugging information included in the class file. Really? I was told that it was a complete myth - that *no one* has to debug the Java code - they just debug the JSP code. My JSP experience occurred a few years agoe, and I remember bewildering error messages - however, I just assumed that was straightened out. I am not trying to toss gasoline on what could be another conflagration - I am just generally interested in the subject. In the interest of preemptive self defense, I really meant that. I have had a few, er, discussions on this topic with people, and have had people claim it never happens. It sounds a little disingenuous given the traffic - however, the majority of the thread messages hadn't yet been delivered when I sent it this morning... Ah well. shields up :) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ still climbing up to the shoulders...
Re: Jasper performance
Yea, JSP protects you from memory leaks and System Crashes. Yea Right. Oh yea, and ASP is lacking customizable tags...as if customizable tags is a good thing Jihad! If we're going to open that debate, then why use Java at all? After all, the computer scientist will tell you with C/C++, you are coding to an unsafe low-level substrate (machine code). With Java, you are coding to a safe high-level substrate (JVM bytecode). Isn't Java therefore just better? But then I could write perfect Java code yet if I have a problem with the JVM, I will crash my safe substrate. I admire the quixotic crusader in you Jon, really I do: The world needs more grail seekers. But you can cease insisting on magic programming bullets for the time being. And I think most people on this list will tell you ASP stands for A Slow Page. JSP sucks. -jon -- Nick Bauman Software Developer 3023 Lynn #22 Minneapolis, MN 55416 Mobile Phone: (612) 810-7406
Re: Jasper performance
On Thu, 17 May 2001, Glenn Nielsen wrote: This is the approach that Tea http://opensource.go.com/ uses as well as the general idea behind taglibs. The problem with taglibs is that there is no restriction on the ability to put Java code in the page. It is part of the JSP specification to be able to do that. Sure, you can disable it (as Costin said), but then you are breaking the JSP specification. And I know how important standards are to everyone... But now that both Tomcat 3.2 and Tomcat 4 support the Java SecurityManager you can control security at the container level regardless of whether someone is using the CFM servlet, velocity, CoCoon, JSP, etc. I guess he's refering to DOS attacks ( like a while(true); in java code or allocating lots of memory ). This can be slightly controlled by restricting access to % %, not allowing any servlet nor user code - but only a set of instrumented taglibs. I'm not sure how many users will host their webapps on sites that do not support servlets or any user code but only a very restricted templating system - and certainly that doesn't fit into the standard ( since servlets will have to be disabled, and only pure JSPs and no custom taglib can be allowed ). Extending jasper to allow the admin to control what tag extensions are allowed is possible - and of course tomcat can be hacked to not allow servlets, libs, webapps - except that it will no longer be tomcat, but a something that runs only pure JSPs and nothing else. There are few things to try - like using jpda or special VMs - but this is very experimental. Costin
Re: Jasper performance
[EMAIL PROTECTED] wrote: On Thu, 17 May 2001, Glenn Nielsen wrote: This is the approach that Tea http://opensource.go.com/ uses as well as the general idea behind taglibs. The problem with taglibs is that there is no restriction on the ability to put Java code in the page. It is part of the JSP specification to be able to do that. Sure, you can disable it (as Costin said), but then you are breaking the JSP specification. And I know how important standards are to everyone... But now that both Tomcat 3.2 and Tomcat 4 support the Java SecurityManager you can control security at the container level regardless of whether someone is using the CFM servlet, velocity, CoCoon, JSP, etc. I guess he's refering to DOS attacks ( like a while(true); in java code or allocating lots of memory ). You won't have much of a templating language if you don't allow some sort of looping. Kind of hard to restrict that. To handle both of the above cases it requires monitoring the performance of the JVM Tomcat is running in, monitoring Tomcat, and individual servlets/jsp's. Then a sysadmin can revoke priviliges for anyone who does something like the above intentionally or from ignorance. It would be nice if self monitoring were built into Tomcat so sysads could track statistics on performance of the JVM, Tomcat in general, and individual servlets/JSP's. Even setting thresholds when automated email notifications could be done. Lets give sysadmins the information they need, then they can take action against problem users. I still think that using the SecurityManager implementation in Tomcat with a well tuned security polciy can provide one of the most secure environments available for running web based applications. This is just my opinion, feel free to try and convince me some other technology is more secure. Regards, Glenn -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance
on 5/17/01 12:47 PM, Glenn Nielsen [EMAIL PROTECTED] wrote:
But now that both Tomcat 3.2 and Tomcat 4 support the Java SecurityManager
you can control security at the container level regardless of whether someone
is using the CFM servlet, velocity, CoCoon, JSP, etc.
Not true.
http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html
Hashtable strings = new Hashtable();
int i=0;
while (true)
{
strings.put (dead+i, new StringBuffer(99));
}
There is no amount of security that will prevent someone from putting that
into their JSP page other than disabling the ability to put scriptlets into
things. If you do that, then you are simply where you should have been in
the first place...using Velocity.
-jon
--
If you come from a Perl or PHP background, JSP is a way to take
your pain to new levels. --Anonymous
http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
on 5/17/01 5:21 PM, Glenn Nielsen [EMAIL PROTECTED] wrote: You won't have much of a templating language if you don't allow some sort of looping. Kind of hard to restrict that. All Velocity has is a #foreach. This is a fully functional looping construct that prevents you from screwing things up and still gets the job done. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
on 5/17/01 2:41 PM, Nick Bauman [EMAIL PROTECTED] wrote: Yea, JSP protects you from memory leaks and System Crashes. Yea Right. Oh yea, and ASP is lacking customizable tags...as if customizable tags is a good thing Jihad! If we're going to open that debate, then why use Java at all? Yes, I agree, designers shouldn't have to learn or use Java. All they should have to learn is how to stick a simple $variable or #directive into a webpage and have it generate the correct results. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
On Thu, 17 May 2001, Glenn Nielsen wrote: I guess he's refering to DOS attacks ( like a while(true); in java code or allocating lots of memory ). You won't have much of a templating language if you don't allow some sort of looping. Kind of hard to restrict that. True, but if you have a set of trusted tags, including looping tags, and no untrusted code except the one that calls the tags you could do a lot to control the resources. For example the tags ( or jasper generated code ) could check for time execution limits, or how many resources are allocated. It would be nice if self monitoring were built into Tomcat so sysads could track statistics on performance of the JVM, Tomcat in general, and individual servlets/JSP's. Even setting thresholds when automated email notifications could be done. Lets give sysadmins the information they need, then they can take action against problem users. Yes, that would be an interesting hack.. I was thinking about JPDA - it would be possible to check the memory use for each thread, associate it with the user code. Also, it is possible to store the time when entering/exiting user code, and have a deamon thread check if any thread is spending too much time. ( the time monitoring part can be done without jpda - but to monitor the memory I don't know other solution ). ( well, I know - I remember a certain tool that was used to manipluate bytecodes and add instrumentation before all allocations - but that's far too difficult for the time we have available ). I still think that using the SecurityManager implementation in Tomcat with a well tuned security polciy can provide one of the most secure environments available for running web based applications. This is just my opinion, feel free to try and convince me some other technology is more secure. I'll not even try :-) You're right, but there are some things that we could add to also control some resource usage ( memory and cpu time ). Costin
RE: Jasper performance
Ok, I missed the end users. Chau, Gaston - Original Message - From: Jon Stevens [EMAIL PROTECTED] To: tomcat-dev [EMAIL PROTECTED] Sent: Thursday, May 17, 2001 1:26 PM Subject: Re: Jasper performance on 5/17/01 8:01 AM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Ok Chris. Now I undertand. Working with JSP when I found that type of mistakes I went to see the .java generated. There I matched the line number so I could see which was the bad code. Then I had to go to the jsp to fix it. It is not the best of the world, but is that that bad? If you think about *who* the target JSP audience is, Yes. Do you expect page designers to figure out problems with line numbers by going to look at the generated page in some temp directory somewhere? Gee, I hope not. More humor for you JSP believers: http://www.theregister.co.uk/content/4/18996.html You may also wish to read a complete pile of bullshit here: http://java.sun.com/products/jsp/jsp-asp.html My favorite: ASP TechnologyJSP Technology Reusable, Cross-Platform ComponentsNoJavaBeans, Enterprise JavaBeans, custom JSP tags Security Against System CrashesNoYes Memory Leak ProtectionNoYes Scripting LanguageVBScript, JScriptJava Customizable TagsNoYes Yea, JSP protects you from memory leaks and System Crashes. Yea Right. Oh yea, and ASP is lacking customizable tags...as if customizable tags i s a good thing? JSP sucks. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
RE: Jasper performance
Have you said a Meg of html !!! Those are the test cases I need. Chau, Gaston - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, May 15, 2001 12:41 PM Subject: Re: Jasper performance We use Jasper (from a 3.1 build!) in another container. How big is the change if we wanted to move to the 3.3 version to get the tag pooling. I'm about to start performance testing our app, and currently the pages are taking ages to render (we build up to 1Meg of html.. :o), so we'll be looking at this area soon. In terms of a quick hit, anybody got opinions / experience of changing the generation to use an unsynchronized version of StringBuffer, I think this may show up as a possible easy enhancement. Ken. [EMAIL PROTECTED] on 15/05/2001 15:22:17 Please respond to [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc:(bcc: Ken X Horn) Subject: Re: Jasper performance Jasper performance has already been identified as an area needing improvement. Discussions and work on this has already started in the main tomcat branch in CVS jakarta-tomcat/proposals/jasper34, but this may be moving to the CVS repository jakarta-tomcat-jasper. This work just started recently, I don't know when it will be ready. It will take few months - it's not that easy. We already added tag pooling in tomcat3.3, and that have a significant effect on performance if you are using tags - but that's just the beginning. The first step is to reorganize the code. Then we'll try to make the code generator more customizable ( probably by using XSLT for some of the operations ). The real performance enhancement will come when we start tunning the generated code - there are many ideas around, but we need the refactoring first. BTW, jasper will share most of the code for the 1.1 and 1.2 APIs, so all enhancements will be available in both 3.x and 4.x ( and other containers as well ). If you have ideas, code or opinions - please get involved, we need you :-) Costin Rickard Öberg wrote: Hi! We are using Tomcat/JBoss and are pleased with the actual functionality. What is killing us right now is the performance of the code generated by Jasper, especially when using taglibs in complex ways. The generated code is way too unoptimized. So, if this has not been asked before (in which case a RTFA is ok, although I've looked already), my question is: When will Jasper be rewritten? Are there any such projects underway now? Have there been any discussions about this yet? Am I the only one seeing this problem, or are more people concernced about it? Thanks, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
RE: Jasper performance
I have some ideas to improbe performance. But it sounds like a should enter them at the CVS. Is that true? How can I do it? The JspCompacter is almost finished. (I also have to document it in english). When finished, that should I do. Should I send it to the list for evaluation? Chau, Gaston ps: I am adding also usage options. - Original Message - From: Glenn Nielsen [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, May 15, 2001 6:49 AM Subject: Re: Jasper performance Jasper performance has already been identified as an area needing improvement. Discussions and work on this has already started in the main tomcat branch in CVS jakarta-tomcat/proposals/jasper34, but this may be moving to the CVS repository jakarta-tomcat-jasper. This work just started recently, I don't know when it will be ready. Regards, Glenn Rickard Öberg wrote: Hi! We are using Tomcat/JBoss and are pleased with the actual functionality. What is killing us right now is the performance of the code generated by Jasper, especially when using taglibs in complex ways. The generated code is way too unoptimized. So, if this has not been asked before (in which case a RTFA is ok, although I've looked already), my question is: When will Jasper be rewritten? Are there any such projects underway now? Have there been any discussions about this yet? Am I the only one seeing this problem, or are more people concernced about it? Thanks, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED] -- -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
RE: Jasper performance
Given any reasonably timeframe for delivery on a new Jasper to production, jdk 1.1 is likely to be three cycles behind. Supporting legacy systems can only go so far. -Original Message- From: Glenn Nielsen [mailto:[EMAIL PROTECTED]] Sent: Tuesday, May 15, 2001 5:30 PM To: [EMAIL PROTECTED] Subject: Re: Jasper performance I would like to propose that the new Jasper require jdk 1.2. The current version of jasper can be used by those who have jdk 1.1.x. Then we don't have to worry about jumping through hoops trying to get the new jasper to run both in 1.1 and 1.2, plus we can optimize for 1.2. In addition JSP 1.2 will require jdk 1.2 or greater. Regards, Glenn [EMAIL PROTECTED] wrote: Jasper performance has already been identified as an area needing improvement. Discussions and work on this has already started in the main tomcat branch in CVS jakarta-tomcat/proposals/jasper34, but this may be moving to the CVS repository jakarta-tomcat-jasper. This work just started recently, I don't know when it will be ready. It will take few months - it's not that easy. We already added tag pooling in tomcat3.3, and that have a significant effect on performance if you are using tags - but that's just the beginning. The first step is to reorganize the code. Then we'll try to make the code generator more customizable ( probably by using XSLT for some of the operations ). The real performance enhancement will come when we start tunning the generated code - there are many ideas around, but we need the refactoring first. BTW, jasper will share most of the code for the 1.1 and 1.2 APIs, so all enhancements will be available in both 3.x and 4.x ( and other containers as well ). If you have ideas, code or opinions - please get involved, we need you :-) Costin Rickard Öberg wrote: Hi! We are using Tomcat/JBoss and are pleased with the actual functionality. What is killing us right now is the performance of the code generated by Jasper, especially when using taglibs in complex ways. The generated code is way too unoptimized. So, if this has not been asked before (in which case a RTFA is ok, although I've looked already), my question is: When will Jasper be rewritten? Are there any such projects underway now? Have there been any discussions about this yet? Am I the only one seeing this problem, or are more people concernced about it? Thanks, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED] -- -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | -- This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
RE: Jasper performance
Now, people are suggesting using something like XSLT to transform the .jsp XML/XHTML file into a .java file. Because you are introducing the XSLT layer into things, that will have a negative impact on the transformation performance (I'm not certain how much, but I am pretty much certain it will be more than the current system.). Given that this only happens when you are in development mode, I think that people developing JSP pages might not like the performance hit. I used XSLT in the past as a template mechanism and I think that Jon is right. XSLT is quite demanding on CPU and memory. (Yes, I am the TemplateMan: I already used XSLT, Freemarker, WebMacro and Velocity) MOREOVER, XSLT always adds a lot of baggage to the equation since the engines are usually quite big and demand for something more than a basic XML parser. And do not forget that, bad practice or not, a lot of people like to have compile-on-change JSPs even in production. So, what ever weight you add, it will be there all the time for a lot of people. And then, XSLT syntax is a pain in the ass, which pays even less when you are not trying to generate well formed XML. It is just too much trouble. I have been changing stuff from using XSLT to using Velocity and it is always a big relief. I do not believe that the code generation for JSPs will demand a very feature full template mechanism. OTOH, sometimes I often come across java products that have their own minimal template mechanism... which means it probably is not hard. So, I believe that Jasper needs a minimal template mechanism, and not a maximal one like XSLT. IF I was messing with Jasper, I would try to find out exactly what is the absolute minimum of templating features that Jasper would need to make the code generation easier. I would try to paths: 1) A very simple XML based template mechanism (with IFs, LOOPSs, VALUEs and so around/controlling-the-flow of the java templated text. No namespaces and no other complex XSLT sh*t; 2) A cut down version of the Webmacro/Velocity syntax. Since an XML parser is already used, 1) would not need much extra weight. If I would not be happy after playing with the 1) concept, I would go for a (very) cut down version of Velocity. My wild guess is that such cut down version could have less than 150 kB, which is much less that what you would have with a XSLT parser and it would be less memory and CPU intensive too. (Yes, Velocity as a load of baggage that can be thrown away for such simple mechanism, like resource management, app tools, part of the syntax features and so on.) For prototyping purposes... for 1) It might be possible to find some XML based mechanism in some OpenSource project somewhere. I bet that something like that exists; for 2) You have Velocity, of course. Well, it is just a suggestion. At the moment that is not my itch. Have fun, Paulo Gaspar -Original Message- From: Jon Stevens [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 16, 2001 7:19 AM on 5/15/01 8:32 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: +1 for the generator. Considerations on the generator: This will slow development down even further doing the transformation step. What ? Using JDK1.2 instead of JDK1.1 ? I doubt it, but thanks for worrying about this :-) Costin Sorry, let me quote the emails a bit better so that you can understand what I am trying to suggest. See above. Currently Jasper output's Java code from within Java code. This is about as fast as you are going to get because there is no intermediate transformation step going on, just conditional output of String data entirely within Java. While this is very fast, it also means that modifications to Jasper are damn near impossible without reading the source code in a fair amount of detail and doing quite a lot of testing. Now, people are suggesting using something like XSLT to transform the .jsp XML/XHTML file into a .java file. Because you are introducing the XSLT layer into things, that will have a negative impact on the transformation performance (I'm not certain how much, but I am pretty much certain it will be more than the current system.). Given that this only happens when you are in development mode, I think that people developing JSP pages might not like the performance hit. Just a word of consideration. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
On Tue, 15 May 2001, Jon Stevens wrote: Currently Jasper output's Java code from within Java code. This is about as fast as you are going to get because there is no intermediate transformation step going on, just conditional output of String data entirely within Java. While this is very fast, it also means that modifications to Jasper are damn near impossible without reading the source code in a fair amount of detail and doing quite a lot of testing. We do need to read source code and do testing in order to make modification in a code generator ??? :-)... XLST will not change that. Now, people are suggesting using something like XSLT to transform the .jsp XML/XHTML file into a .java file. Because you are introducing the XSLT layer into things, that will have a negative impact on the transformation performance (I'm not certain how much, but I am pretty much certain it will be more than the current system.). Given that this only happens when you are in development mode, I think that people developing JSP pages might not like the performance hit. The code generation stage is not significant - compared with the javac compilation. Some people are even using XSLT at runtime ( not that this is a good idea ) - so I doubt it'll have such a significant impact on development mode ( if it's ok for a runtime page, it should be ok for a developer ). In addition, for pure jsp ( if no java fragment is used ) it would be possible to use the same technique as XSLTC ( now part of xalan ) - and generate the bytecode directly ( since most of the generated code is glue ) - and avoid the javac. Of course, the big goal is to allow some taglib optimzations ( create alternate implementations for common-used tags, with same behavior but generating code directly ) - and that's why XSLT is need. The good news about JSP is that it is a specification. People can use a different implementation if they are not happy with one, or use one for development and one for deployment. ( I have a feeling Cocoon2 and Resin are both using XSLT in developing the pages - so I guess we will be safe, both are doing OK ) Costin
Re: Jasper performance
on 5/16/01 1:11 AM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: The code generation stage is not significant - compared with the javac compilation. Some people are even using XSLT at runtime ( not that this is a good idea ) - so I doubt it'll have such a significant impact on development mode ( if it's ok for a runtime page, it should be ok for a developer ). In addition, for pure jsp ( if no java fragment is used ) it would be possible to use the same technique as XSLTC ( now part of xalan ) - and generate the bytecode directly ( since most of the generated code is glue ) - and avoid the javac. Of course, the big goal is to allow some taglib optimzations ( create alternate implementations for common-used tags, with same behavior but generating code directly ) - and that's why XSLT is need. The good news about JSP is that it is a specification. People can use a different implementation if they are not happy with one, or use one for development and one for deployment. ( I have a feeling Cocoon2 and Resin are both using XSLT in developing the pages - so I guess we will be safe, both are doing OK ) Costin Costin, Once again, you impress me with your inability to understand a word of what I'm talking about. So, let me close this discussion with this: If the speed of generating a .java file (or directly to a .class file) from JSP ends up taking more resources (ie: memory) and more time after the conversion to using XSLT as the transformation step, I will forward your comments above to you again and tell you I told you so. I welcome the same response from you. As for your comments about Cocoon2, well, if you actually look at it, it is dog slow and *extremely* resource intensive. The only reason why there is *any* performance out of it is because there is a shit load of caching and compiled XSLT that goes on. This is something that I fear the development of JSP pages will not be able to take advantage of in the same way and as a result, development performance will suffer compared with what it is at today. I find it funny that I seem to be the only one who cares about this issue. :-) p.s. Over in Turbine land, we have a lot of experience with the dynamic generation of Java code and have found that using Texen has been a most favorable experience. It would be most humorous to me to see Velocity used to generate .java files from .jsp input. :-) http://jakarta.apache.org/velocity/texen.html -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
On Wed, 16 May 2001, Jon Stevens wrote: Costin, Once again, you impress me with your inability to understand a word of what I'm talking about. So, let me close this discussion with this: No problem, I'm not that good at explaining either. If the speed of generating a .java file (or directly to a .class file) from JSP ends up taking more resources (ie: memory) and more time after the conversion to using XSLT as the transformation step, I will forward your comments above to you again and tell you I told you so. I welcome the same response from you. And I tried to explain: the xslt transformation is not significant compared with the javac compilation ( it may add 5-10% extra ), and may end up saving the javac in some cases - which would be a significant speed improvement. And I also tried to explain: JSP doens't take more resources, a particular implementation ( generator ) may take more resources. The current java-only generator will still be available, there are many ways to compile the JSP page - and xslt has some benefits you can't get otherwise. As for your comments about Cocoon2, well, if you actually look at it, it is dog slow and *extremely* resource intensive. The only reason why there is *any* performance out of it is because there is a shit load of caching and compiled XSLT that goes on. This is something that I fear the development of JSP pages will not be able to take advantage of in the same way and as a result, development performance will suffer compared with what it is at today. Caching is good and will be done anyway ( at least for runtime ). Generating code is supposed to be resource intensive if you expect reasonable runtime performance - and I haven't heared too many people complaining Resin is too slow. I find it funny that I seem to be the only one who cares about this issue. :-) Yes, I find it funny too. Costin
Re: Jasper performance
on 5/16/01 4:02 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Yet another template generation language ??? #foreach ??? Costin, what rock have you been sleeping under for the last 5 years? Velocity is simply a cleaner implementation of WebMacro. It isn't YATGL. Also, there is a reason for the #foreach... http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html Explain how you are going to fix that little problem Mr. Smart Ass. Yes, that's really humorous... but I guess I'll stick with W3C's templating language for a while :-) Oh yea, I forgot! The W3C knows everything and everything that comes from the W3C is *perfect*. XSLT is oh so great and perfect. Yea. Right. Open your eyes dude. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
On Wed, 16 May 2001, Jon Stevens wrote: on 5/16/01 4:02 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Yet another template generation language ??? #foreach ??? Costin, what rock have you been sleeping under for the last 5 years? Velocity is simply a cleaner implementation of WebMacro. It isn't YATGL. Sorry, from that example I didn't realized it's a webmacro ( there are not too many languages to use # this way ) - and to be honest it's not on the list of languages I use. Also, there is a reason for the #foreach... http://jakarta.apache.org/velocity/ymtd/ymtd-hosting.html Explain how you are going to fix that little problem Mr. Smart Ass. Well, ignoring the Ass part - if you are talking about allowing only a restricted set of tags and not allowing java code in the page - that can be done ( and it's not even difficult ). Yes, that's really humorous... but I guess I'll stick with W3C's templating language for a while :-) Oh yea, I forgot! The W3C knows everything and everything that comes from the W3C is *perfect*. XSLT is oh so great and perfect. Yea. Right. Open your eyes dude. Oh, yea, I forgot - you are smarter than W3C and everyone else :-) XSLT is not perfect - neither is HTTP or HTML or any other standard. But because Apache and many other organizations are implementing and using it - I think they'll be around for a while :-) Costin
Re: Jasper performance
On Wed, 16 May 2001, Jon Stevens wrote: may? Do you have empirical evidence of that or did you pull that out of your ass? I guess that concludes our discussion. Costin , and may end up saving the javac in some cases - which would be a significant speed improvement. Ah. Something else you pulled out of your ass. And I also tried to explain: JSP doens't take more resources, a particular implementation ( generator ) may take more resources. Once again, you don't listen very well. I'm not talking about JSP taking more resources, I'm talking about Jasper taking more resources. In case you missed it, Xalan is not a lightweight piece of Java code. The current java-only generator will still be available, there are many ways to compile the JSP page - and xslt has some benefits you can't get otherwise. Oh, so now you are going to support both generators? You are so amazing! Generating code is supposed to be resource intensive if you expect reasonable runtime performance Is it supposed to be more intensive than it currently is? You are talking about slowing things down at least another 5-10%...I'm sure your users will really love that. -jon
Re: Jasper performance
on 5/16/01 7:06 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: On Wed, 16 May 2001, Jon Stevens wrote: on 5/16/01 4:02 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Yet another template generation language ??? #foreach ??? Costin, what rock have you been sleeping under for the last 5 years? Velocity is simply a cleaner implementation of WebMacro. It isn't YATGL. Sorry, from that example I didn't realized it's a webmacro ( there are not too many languages to use # this way ) - and to be honest it's not on the list of languages I use. Once again...sleeping under a rock. if you are talking about allowing only a restricted set of tags and not allowing java code in the page - that can be done ( and it's not even difficult ). Right now? Today? Show me how you are going to implement that and not break the specification or all of the examples. Oh, yea, I forgot - you are smarter than W3C and everyone else :-) Did I say that? I don't think so. However, I am going to state that XSLT in its current form is lacking in many ways. XSLT is not perfect - neither is HTTP or HTML or any other standard. But because Apache and many other organizations are implementing and using it - I think they'll be around for a while :-) Costin Luckily Velocity also comes from the Apache Software Foundation so it will be around for a while as well. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
Jasper performance has already been identified as an area needing improvement. Discussions and work on this has already started in the main tomcat branch in CVS jakarta-tomcat/proposals/jasper34, but this may be moving to the CVS repository jakarta-tomcat-jasper. This work just started recently, I don't know when it will be ready. It will take few months - it's not that easy. We already added tag pooling in tomcat3.3, and that have a significant effect on performance if you are using tags - but that's just the beginning. The first step is to reorganize the code. Then we'll try to make the code generator more customizable ( probably by using XSLT for some of the operations ). The real performance enhancement will come when we start tunning the generated code - there are many ideas around, but we need the refactoring first. BTW, jasper will share most of the code for the 1.1 and 1.2 APIs, so all enhancements will be available in both 3.x and 4.x ( and other containers as well ). If you have ideas, code or opinions - please get involved, we need you :-) Costin Rickard Öberg wrote: Hi! We are using Tomcat/JBoss and are pleased with the actual functionality. What is killing us right now is the performance of the code generated by Jasper, especially when using taglibs in complex ways. The generated code is way too unoptimized. So, if this has not been asked before (in which case a RTFA is ok, although I've looked already), my question is: When will Jasper be rewritten? Are there any such projects underway now? Have there been any discussions about this yet? Am I the only one seeing this problem, or are more people concernced about it? Thanks, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance
Rickard, As Glenn mentioned, Jasper is in the process of getting a rework. But in the mean time and since you mentioned taglibs, you may want to look at the tag pooling module that was added to tc 3.3. If your jsps use a lot of tags, you should see a big performance gain. Let me know if you have questions about tag pooling. I was hoping to add tag pooling to tc 4, but have been very busy. So unless I gain some spare time, you'll probably need to wait until the new Jasper is available (at least for tag pooling). -casey Rickard Öberg wrote: Hi! We are using Tomcat/JBoss and are pleased with the actual functionality. What is killing us right now is the performance of the code generated by Jasper, especially when using taglibs in complex ways. The generated code is way too unoptimized. So, if this has not been asked before (in which case a RTFA is ok, although I've looked already), my question is: When will Jasper be rewritten? Are there any such projects underway now? Have there been any discussions about this yet? Am I the only one seeing this problem, or are more people concernced about it? Thanks, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance
I, for one, would be very interested in making it easy/possible to uncouple Jasper from the servlet container itself. The question I have is How much of a scaffolding is required to use Jasper purely as a template engine without the baggage of the servlet container per se (or at worst, by being able to call the _jspservice method with a parameters of (HttpServletRequest, HttpServletResponse) from within an application instead of over the wire with http). Is there anyone who has done this already? Jasper performance has already been identified as an area needing improvement. Discussions and work on this has already started in the main tomcat branch in CVS jakarta-tomcat/proposals/jasper34, but this may be moving to the CVS repository jakarta-tomcat-jasper. This work just started recently, I don't know when it will be ready. It will take few months - it's not that easy. We already added tag pooling in tomcat3.3, and that have a significant effect on performance if you are using tags - but that's just the beginning. The first step is to reorganize the code. Then we'll try to make the code generator more customizable ( probably by using XSLT for some of the operations ). The real performance enhancement will come when we start tunning the generated code - there are many ideas around, but we need the refactoring first. BTW, jasper will share most of the code for the 1.1 and 1.2 APIs, so all enhancements will be available in both 3.x and 4.x ( and other containers as well ). If you have ideas, code or opinions - please get involved, we need you :-) Costin Rickard Öberg wrote: Hi! We are using Tomcat/JBoss and are pleased with the actual functionality. What is killing us right now is the performance of the code generated by Jasper, especially when using taglibs in complex ways. The generated code is way too unoptimized. So, if this has not been asked before (in which case a RTFA is ok, although I've looked already), my question is: When will Jasper be rewritten? Are there any such projects underway now? Have there been any discussions about this yet? Am I the only one seeing this problem, or are more people concernced about it? Thanks, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED] -- Nick Bauman Software Developer 3023 Lynn #22 Minneapolis, MN 55416 Mobile Phone: (612) 810-7406
Re: Jasper performance
We use Jasper (from a 3.1 build!) in another container. How big is the change if we wanted to move to the 3.3 version to get the tag pooling. I'm about to start performance testing our app, and currently the pages are taking ages to render (we build up to 1Meg of html.. :o), so we'll be looking at this area soon. In terms of a quick hit, anybody got opinions / experience of changing the generation to use an unsynchronized version of StringBuffer, I think this may show up as a possible easy enhancement. Ken. [EMAIL PROTECTED] on 15/05/2001 15:22:17 Please respond to [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc:(bcc: Ken X Horn) Subject: Re: Jasper performance Jasper performance has already been identified as an area needing improvement. Discussions and work on this has already started in the main tomcat branch in CVS jakarta-tomcat/proposals/jasper34, but this may be moving to the CVS repository jakarta-tomcat-jasper. This work just started recently, I don't know when it will be ready. It will take few months - it's not that easy. We already added tag pooling in tomcat3.3, and that have a significant effect on performance if you are using tags - but that's just the beginning. The first step is to reorganize the code. Then we'll try to make the code generator more customizable ( probably by using XSLT for some of the operations ). The real performance enhancement will come when we start tunning the generated code - there are many ideas around, but we need the refactoring first. BTW, jasper will share most of the code for the 1.1 and 1.2 APIs, so all enhancements will be available in both 3.x and 4.x ( and other containers as well ). If you have ideas, code or opinions - please get involved, we need you :-) Costin Rickard Öberg wrote: Hi! We are using Tomcat/JBoss and are pleased with the actual functionality. What is killing us right now is the performance of the code generated by Jasper, especially when using taglibs in complex ways. The generated code is way too unoptimized. So, if this has not been asked before (in which case a RTFA is ok, although I've looked already), my question is: When will Jasper be rewritten? Are there any such projects underway now? Have there been any discussions about this yet? Am I the only one seeing this problem, or are more people concernced about it? Thanks, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance
[EMAIL PROTECTED] wrote: We already added tag pooling in tomcat3.3, and that have a significant effect on performance if you are using tags - but that's just the beginning. This is great news! Our other options was to use another engine (like Orion or Resin), write our own JSP compiler, or not use JSP at all. Tag pooling will be a good start, and I'll try it out first thing tomorrow. The first step is to reorganize the code. Then we'll try to make the code generator more customizable ( probably by using XSLT for some of the operations ). The real performance enhancement will come when we start tunning the generated code - there are many ideas around, but we need the refactoring first. After having looked at the Jasper code, that seems like a good strategy :-) If you have ideas, code or opinions - please get involved, we need you :-) I will try, as time permits, as this is the single most performance-heavy part of our system right now. Any improvements would be incredibly useful. /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance
Jon Stevens wrote: Or you could use another tool that doesn't require you to generate .java files first, doesn't require a boat load of caching to get decent performance (imagine that using JSP the right way causes it to actually run slower???), doesn't need a complete re-write to fix it's problems, ...etc... http://jakarta.apache.org/velocity/ Somehow I'm not surprised by this comment ;-) And yes, we have considered it, but we really really would like to use the taglibs provided by WebWork (sourceforge.net/projects/webwork). One crucial feature I couldn't immediately find in Velocity is the ability to jsp:include output from other servlets. Using JSP also gives us a well-defined way to extend the possible tags, which is a must. /Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance
I would like to propose that the new Jasper require jdk 1.2. The current version of jasper can be used by those who have jdk 1.1.x. Then we don't have to worry about jumping through hoops trying to get the new jasper to run both in 1.1 and 1.2, plus we can optimize for 1.2. In addition JSP 1.2 will require jdk 1.2 or greater. Regards, Glenn [EMAIL PROTECTED] wrote: Jasper performance has already been identified as an area needing improvement. Discussions and work on this has already started in the main tomcat branch in CVS jakarta-tomcat/proposals/jasper34, but this may be moving to the CVS repository jakarta-tomcat-jasper. This work just started recently, I don't know when it will be ready. It will take few months - it's not that easy. We already added tag pooling in tomcat3.3, and that have a significant effect on performance if you are using tags - but that's just the beginning. The first step is to reorganize the code. Then we'll try to make the code generator more customizable ( probably by using XSLT for some of the operations ). The real performance enhancement will come when we start tunning the generated code - there are many ideas around, but we need the refactoring first. BTW, jasper will share most of the code for the 1.1 and 1.2 APIs, so all enhancements will be available in both 3.x and 4.x ( and other containers as well ). If you have ideas, code or opinions - please get involved, we need you :-) Costin Rickard Öberg wrote: Hi! We are using Tomcat/JBoss and are pleased with the actual functionality. What is killing us right now is the performance of the code generated by Jasper, especially when using taglibs in complex ways. The generated code is way too unoptimized. So, if this has not been asked before (in which case a RTFA is ok, although I've looked already), my question is: When will Jasper be rewritten? Are there any such projects underway now? Have there been any discussions about this yet? Am I the only one seeing this problem, or are more people concernced about it? Thanks, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED] -- -- Glenn Nielsen [EMAIL PROTECTED] | /* Spelin donut madder| MOREnet System Programming | * if iz ina coment. | Missouri Research and Education Network | */ | --
Re: Jasper performance
On Tue, 15 May 2001, Glenn Nielsen wrote: I would like to propose that the new Jasper require jdk 1.2. The current version of jasper can be used by those who have jdk 1.1.x. Then we don't have to worry about jumping through hoops trying to get the new jasper to run both in 1.1 and 1.2, plus we can optimize for 1.2. In addition JSP 1.2 will require jdk 1.2 or greater. +1 for the generator. +1 for the runtime implementation My only -1 is for the method signatures in the runtime - but that should be a tiny problem, and I'll watch and fix in case something slips by. The idea is that I want to have an alternate implementation that uses only JDK1.1 ( or J2ME ) - this is not a big priority, but it can't hurt anyone to keep a door open. I don't care too much about JDK1.1 - but it's very important for me to have a way to use a runtime that works in more restricted environments ( but that will not be the default - I can make such a think somewhere outside - as long as we don't add something in the signatures that would prevent that ) I already ordered the tini board - and it supports only JDK1.1 so far, so please don't try to make me change the vote :-) Costin Regards, Glenn [EMAIL PROTECTED] wrote: Jasper performance has already been identified as an area needing improvement. Discussions and work on this has already started in the main tomcat branch in CVS jakarta-tomcat/proposals/jasper34, but this may be moving to the CVS repository jakarta-tomcat-jasper. This work just started recently, I don't know when it will be ready. It will take few months - it's not that easy. We already added tag pooling in tomcat3.3, and that have a significant effect on performance if you are using tags - but that's just the beginning. The first step is to reorganize the code. Then we'll try to make the code generator more customizable ( probably by using XSLT for some of the operations ). The real performance enhancement will come when we start tunning the generated code - there are many ideas around, but we need the refactoring first. BTW, jasper will share most of the code for the 1.1 and 1.2 APIs, so all enhancements will be available in both 3.x and 4.x ( and other containers as well ). If you have ideas, code or opinions - please get involved, we need you :-) Costin Rickard Öberg wrote: Hi! We are using Tomcat/JBoss and are pleased with the actual functionality. What is killing us right now is the performance of the code generated by Jasper, especially when using taglibs in complex ways. The generated code is way too unoptimized. So, if this has not been asked before (in which case a RTFA is ok, although I've looked already), my question is: When will Jasper be rewritten? Are there any such projects underway now? Have there been any discussions about this yet? Am I the only one seeing this problem, or are more people concernced about it? Thanks, Rickard -- Rickard Öberg Software Development Specialist xlurc - Xpedio Linköping Ubiquitous Research Center Author of Mastering RMI Email: [EMAIL PROTECTED]
Re: Jasper performance
on 5/15/01 7:05 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: On Tue, 15 May 2001, Glenn Nielsen wrote: I would like to propose that the new Jasper require jdk 1.2. The current version of jasper can be used by those who have jdk 1.1.x. Then we don't have to worry about jumping through hoops trying to get the new jasper to run both in 1.1 and 1.2, plus we can optimize for 1.2. In addition JSP 1.2 will require jdk 1.2 or greater. +1 for the generator. Considerations on the generator: This will slow development down even further doing the transformation step. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
Re: Jasper performance
On Tue, 15 May 2001, Jon Stevens wrote: on 5/15/01 7:05 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: On Tue, 15 May 2001, Glenn Nielsen wrote: I would like to propose that the new Jasper require jdk 1.2. The current version of jasper can be used by those who have jdk 1.1.x. Then we don't have to worry about jumping through hoops trying to get the new jasper to run both in 1.1 and 1.2, plus we can optimize for 1.2. In addition JSP 1.2 will require jdk 1.2 or greater. +1 for the generator. Considerations on the generator: This will slow development down even further doing the transformation step. What ? Using JDK1.2 instead of JDK1.1 ? I doubt it, but thanks for worrying about this :-) Costin
Re: Jasper performance
on 5/15/01 8:32 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: +1 for the generator. Considerations on the generator: This will slow development down even further doing the transformation step. What ? Using JDK1.2 instead of JDK1.1 ? I doubt it, but thanks for worrying about this :-) Costin Sorry, let me quote the emails a bit better so that you can understand what I am trying to suggest. See above. Currently Jasper output's Java code from within Java code. This is about as fast as you are going to get because there is no intermediate transformation step going on, just conditional output of String data entirely within Java. While this is very fast, it also means that modifications to Jasper are damn near impossible without reading the source code in a fair amount of detail and doing quite a lot of testing. Now, people are suggesting using something like XSLT to transform the .jsp XML/XHTML file into a .java file. Because you are introducing the XSLT layer into things, that will have a negative impact on the transformation performance (I'm not certain how much, but I am pretty much certain it will be more than the current system.). Given that this only happens when you are in development mode, I think that people developing JSP pages might not like the performance hit. Just a word of consideration. -jon -- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
