Re: Tomcat Connector Benchmark
there isn't any official benchmark, but there's the benchmarks I ran this year and the results. peter On 7/12/05, Vicenc Beltran Querol [EMAIL PROTECTED] wrote: Hi, I would like to know if there is an official benchmark to compare the scalability/throughput of the new connectors (APR, Grizzly, ...) and Coyote. If not, maybe it's a good time to define one. Regards, Vicenç - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Initial apr results
if you need the test plan Tim, email me and I'll send it to you :) peter On 6/2/05, Bill Barker [EMAIL PROTECTED] wrote: You might also want to dig up Peter's JMeter test plan. This one is the opposite of the 'ab' test, in that it tests the ability to handle a lot of socket connections without necessarily much throughput. As Remy said, the 'ab -k' tests should be close unless either your JVM vendor or your APR implementation s*cks. Both connectors do much the same thing with this test execpt for sendfile (and 'tomcat.gif' is too small to for Tomcat to use sendfile by default). - Original Message - From: Tim Funk [EMAIL PROTECTED] To: Tomcat Developers List tomcat-dev@jakarta.apache.org Sent: Thursday, June 02, 2005 3:41 AM Subject: Re: Initial apr results Excellent. In my future tests, I'll keep the concurrency lower and the hits higher. I'll also use different size files. I am only able to use a 1.4.2 JVM. I might be able to get 1.5 on - but its highly doubtful. -Tim Remy Maucherat wrote: Tim Funk wrote: My test box was an HP-UX 9000/800/L1000-44 - Dual CPU (440 MHz) On my initial tests with the APR connector - the APR connector seemed slower the old http connector. But the difference is mild and my initial numbers are flaky. On the same hardware - I am running 6 other instances (of different versions) of tomcat at the same time which may be throwing my numbers off. During some slow time - I might be able to take most of them down and run some more tests to try to ensure I am hogging all the resources to the box and not sharing them. For those curious - for /tomcat.gif - my requests per second range anywhere from 1200+ to 5000+ - Due to such a large range - I have no confidence in my numbers so far to reach any conclusion. I was using the command: /usr/local/httpd/bin/ab -n 1000 -c 100 -k http://myserver:8090/tomcat.gif With keepalive off - I was still easily over 1000 requests per second for tomcat.gif. I can't assert yet that there are no bugs at the moment (performance or otherwise). So far, performance seems good on Windows, and Linux. To give a comparison on Windows with this test, APR HTTP is within 5% of regular HTTP, and gets closer depending on the JVM (I suppose if it's better at JNI - I've noticed slightly better results with Sun JDK 1.5 Server compared to 1.4.2 client). I think you should use -n 2 at least: if the test duration is too small, ab is going to produce random results. In this test, increasing concurrency isn't particularly useful either. Obviously, the results of this kind of testing are not really important as long as the results stay relatively close (for example, I think the -25% result I got when using polling exclusively was really good). - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] This message is intended only for the use of the person(s) listed above as the intended recipient(s), and may contain information that is PRIVILEGED and CONFIDENTIAL. If you are not an intended recipient, you may not read, copy, or distribute this message or any attachment. If you received this communication in error, please notify us immediately by e-mail and then delete all copies of this message and any attachments. In addition you should be aware that ordinary (unencrypted) e-mail sent through the Internet is not secure. Do not send confidential or sensitive information, such as social security numbers, account numbers, personal identification numbers and passwords, to us via ordinary (unencrypted) e-mail. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Initial apr results
once I finally get the APR build working on my laptop, I will run my benchmarks and publish the results. peter On 6/1/05, Tim Funk [EMAIL PROTECTED] wrote: My test box was an HP-UX 9000/800/L1000-44 - Dual CPU (440 MHz) On my initial tests with the APR connector - the APR connector seemed slower the old http connector. But the difference is mild and my initial numbers are flaky. On the same hardware - I am running 6 other instances (of different versions) of tomcat at the same time which may be throwing my numbers off. During some slow time - I might be able to take most of them down and run some more tests to try to ensure I am hogging all the resources to the box and not sharing them. For those curious - for /tomcat.gif - my requests per second range anywhere from 1200+ to 5000+ - Due to such a large range - I have no confidence in my numbers so far to reach any conclusion. I was using the command: /usr/local/httpd/bin/ab -n 1000 -c 100 -k http://myserver:8090/tomcat.gif With keepalive off - I was still easily over 1000 requests per second for tomcat.gif. -Tim - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Hybrid (NIO+Multithread, SSL enabled) architecture for Coyote
Am I reading the results correctly? tomcat 5.5.9 - 16,331.81/sec hybrid - 7,085.54/sec that means the hybrid connector is 2x slower. If those results are accurate, I would say the APR connector is much better choice. peter lin On 5/25/05, Vicenc Beltran Querol [EMAIL PROTECTED] wrote: Hi, The results of the AB benchmark configured with 20 concurrent clients are posted below, If somebody is interested in more configurations (from 20 to 1 concurrent clients) they are available at http://www.bsc.es/edragon/pdf/TestAb.tgz BTW, there is also available a comparison between Tomcat and the Hybrid (Tomcat+NIO) web servers at http://www.bsc.es/edragon/pdf/TestRubisDynamic.tgz. The comparison is based on the RUBiS benchmark and the httperf workload generator. Regards, -Vicenç ./ab -k -c 20 -n 200 http://pcbosch:8080/tomcat.gif Client: 2 way Xeon 2.4Ghz, 2GB RAM Server: 4 way Xeon 1.4Ghz, 2GB RAM Network: Gbit Java: build 1.5.0_03-b07 Tomcat 5.5.9 - - cument Path: /tomcat.gif Document Length:1934 bytes Concurrency Level: 20 Time taken for tests: 122.460403 seconds Complete requests: 200 Failed requests:0 Write errors: 0 Keep-Alive requests:1980006 Total transferred: 32937062 bytes HTML transferred: -426963428 bytes Requests per second:16331.81 [#/sec] (mean) Time per request: 1.225 [ms] (mean) Time per request: 0.061 [ms] (mean, across all concurrent requests) Transfer rate: 262.66 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect:00 0.0 0 14 Processing: 00 2.5 0 636 Waiting:00 2.4 0 636 Total: 00 2.5 0 636 Percentage of the requests served within a certain time (ms) 50% 0 66% 1 75% 1 80% 1 90% 1 95% 2 98% 6 99% 11 100%636 (longest request) - - Tomcat Hybrid 5.5.9 - - Document Path: /tomcat.gif Document Length:1934 bytes Concurrency Level: 20 Time taken for tests: 282.264843 seconds Complete requests: 200 Failed requests:0 Write errors: 0 Keep-Alive requests:200 Total transferred: 33032704 bytes HTML transferred: -426967296 bytes Requests per second:7085.54 [#/sec] (mean) Time per request: 2.823 [ms] (mean) Time per request: 0.141 [ms] (mean, across all concurrent requests) Transfer rate: 114.28 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect:00 0.0 0 1 Processing: 02 1.7 2 24 Waiting:02 1.7 2 24 Total: 02 1.7 2 24 Percentage of the requests served within a certain time (ms) 50% 2 66% 3 75% 4 80% 4 90% 5 95% 5 98% 6 99% 6 100% 24 (longest request) - - - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Hybrid (NIO+Multithread, SSL enabled) architecture for Coyote
On 5/25/05, Vicenc Beltran Querol [EMAIL PROTECTED] wrote: Hi, I'm absolutely disconcerted. In your previous answeryou agreed that the AB test is not good for comparing two different architectural approaches. And you still wanna compare the performance of the hybrid architecture using it. But when I look for APR results on the net, I find that in the message 70876 (http://www.mail-archive.com/tomcat-dev@jakarta.apache.org/msg70876.html) of this list that you're using JMeter and think-times in other experiments. Have you looked at any of the results I've post for realistic benchmarks? Why are you so obsessed with the AB results with concurrency level 20? Sorry, but I don't see the point on it... Using non-realistic benchmarks and very-oriented performance tricks only leads to winning few milliseconds in the response time of the server. But it's not a real benefit for the clients. When the server is overloaded (when performance improvements are really determinant), these benefits are negligible... In my opinion, following these development criterias is counterproductive and makes the server worse in the real world (where users put it into production). Surely, you disagree... I'm not sure I agree with that statement. The reason for using apache AB for small files under 2K is that JMeter is unable to max out the server with tiny files. You can see the original number I produced here http://people.apache.org/~woolfel/tc_results.html. Since the bulk of my work the last 4 years has been with large applications handling millions of pageviews a day, I can safely say that most large deployment will rarely exceed 50 concurrent requests for extended period of time. this is just my experience on real applications, but we generally buffer the entire page and then send it in one shot. this is done for several reasons. 1. WAN latency - as you already stated 2. improve accuracy of performance logging. we log the page generation to make sure we know exactly how much time is spent for the query, page markup and transfering the data. 3. allows us to track network bottleneck more accurately In my mind, the argument for tomcat supporting 1000 concurrent connections for an extended period of time isn't valid from my experience. There's typically a large cluster of servers that are load balanced behind a load balancing router. For me, throughput is far more important because most the images and files range from 5-15K in size. In these cases, maximizing throughput is more important. So small sites trying to deal with the /. effect, it's not worth it. I say that because the network will die long before tomcat will. Any site with serious performance requirements will host at a tier 1 provider and have a cluster of servers. small personal sites are shared hosted and often don't have enough bandwidth. my bias .02 cents. peter lin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Hybrid (NIO+Multithread, SSL enabled) architecture for Coyote
I took a look at the AB and Rubis numbers. Honestly I don't understand the rubis graphs. From the AB results, it looks like the connect, processing and wait times are lower for the hybrid. That's a good achievement and congrats to you on that. I'm not convinced of the benefit of the hybrid approach over APR. if both are equal, then it might be good to have both as options. it's nice to be able to support /. effect, but in reality that's achieved by distributing servers across multiple hosting facilities. It's not achieved through hosting a website on a single quad server supporting 10K concurrent connections. I'm not a committer, so I don't have a say in what goes into tomcat. thanks for researching NIO and taking time to post these results. peter lin On 5/25/05, Vicenc Beltran Querol [EMAIL PROTECTED] wrote: Hi, The results of the AB benchmark configured with 20 concurrent clients are posted below, If somebody is interested in more configurations (from 20 to 1 concurrent clients) they are available at http://www.bsc.es/edragon/pdf/TestAb.tgz BTW, there is also available a comparison between Tomcat and the Hybrid (Tomcat+NIO) web servers at http://www.bsc.es/edragon/pdf/TestRubisDynamic.tgz. The comparison is based on the RUBiS benchmark and the httperf workload generator. Regards, -Vicenç ./ab -k -c 20 -n 200 http://pcbosch:8080/tomcat.gif Client: 2 way Xeon 2.4Ghz, 2GB RAM Server: 4 way Xeon 1.4Ghz, 2GB RAM Network: Gbit Java: build 1.5.0_03-b07 Tomcat 5.5.9 - - cument Path: /tomcat.gif Document Length:1934 bytes Concurrency Level: 20 Time taken for tests: 122.460403 seconds Complete requests: 200 Failed requests:0 Write errors: 0 Keep-Alive requests:1980006 Total transferred: 32937062 bytes HTML transferred: -426963428 bytes Requests per second:16331.81 [#/sec] (mean) Time per request: 1.225 [ms] (mean) Time per request: 0.061 [ms] (mean, across all concurrent requests) Transfer rate: 262.66 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect:00 0.0 0 14 Processing: 00 2.5 0 636 Waiting:00 2.4 0 636 Total: 00 2.5 0 636 Percentage of the requests served within a certain time (ms) 50% 0 66% 1 75% 1 80% 1 90% 1 95% 2 98% 6 99% 11 100%636 (longest request) - - Tomcat Hybrid 5.5.9 - - Document Path: /tomcat.gif Document Length:1934 bytes Concurrency Level: 20 Time taken for tests: 282.264843 seconds Complete requests: 200 Failed requests:0 Write errors: 0 Keep-Alive requests:200 Total transferred: 33032704 bytes HTML transferred: -426967296 bytes Requests per second:7085.54 [#/sec] (mean) Time per request: 2.823 [ms] (mean) Time per request: 0.141 [ms] (mean, across all concurrent requests) Transfer rate: 114.28 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect:00 0.0 0 1 Processing: 02 1.7 2 24 Waiting:02 1.7 2 24 Total: 02 1.7 2 24 Percentage of the requests served within a certain time (ms) 50% 2 66% 3 75% 4 80% 4 90% 5 95% 5 98% 6 99% 6 100% 24 (longest request) - - - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Hybrid (NIO+Multithread, SSL enabled) architecture for Coyote
I'm not a committer, but I think evidence proves that native sockets + JNI is the way to go. To my knowledge, weblogic, websphere and Resin all use native sockets. having a pure Java approach sounds nice and all, but in the edge cases where high concurrent connection is needed, I much rather go with native + jni. my 1/10th of a cent worth. peter On 5/20/05, Remy Maucherat [EMAIL PROTECTED] wrote: Jeanfrancois Arcand wrote: I disagree ;-) I would like to see your implementation, because from what I'm seeing/measuring, it is completely the inverse. I would be interested to see how you did implement your NIO connector. The problem with HTTP is not NIO, but the strategy to use for predicting if you have read all the bytes or not. Falling to implement a good strategy will ends up parsing the bytes many times, calling the Selector.wakeup() too often, thus poor performance. The way you register your SelectionKey is also very important. Also, blocking IO required 1 thread per connection, which doesn't scale very well. That's why I think the new APR connector is interesting, since it fix that problem. But even if with APR, you did workaround the JNI bottleneck by using direct byte buffer, I suspect a pure NIO implementation will perform better than APR (except for static resource) just because of the C-Java overhead. But I don't have yet numbers to show...come to my session at JavaOne, I will :-) Sorry, but I agree with Mladen. There is simply no way a pure non blocking NIO strategy is going to work. It is an attempt to apply receipes which work really well for certain types of takss to other types of tasks. Translated, it would work spectacularly well for, say, a servlet like the default servlet and a small file (ie, small bufferable amount of data + no processing), but fail for a servlet which receives a lot of uploading or more generally has a long processing time. The main problem is that to keep contention on IO low, a low amount of processors has to be used, which is not compatible with the second type of tasks. The only way to apply NIO to Tomcat is to use it in blocking mode, as in the submitted patch. The only way to convince me your solution can work is to (also) write your own endpoint / protocol handler (this seems trendy these days for some reason, so I guess it's ok if everyone does it ;) ) so I can test it myself and see the results. As APR matches the numbers of classic blocking IO in the (100% throughput oriented, and worst case; at least, it's the case which favors regular blocking IO the most) ab -k -c 20 localhost/tomcat.gif, it seems hard to beat. rant BTW, about the NIO usage for optimizing JNI, I'm actually really mad about Sun. Why attempt to make any JNI calls safe and make performance suck in the first place, when with native code usage it is trivial to segfault the whole process anyway (example: feed APR a bad address) ? This really makes no sense to me, and seems simply a plot to try to force people to write 100% Java code. All that complexity and crappy performance for *nothing* (except helping .not and mono, of course) ... /rant You may tray that simply by using demo HTTP servers Blocking/Blocking Pool/NIO single thread/NIO multiple threads that comes with new Java6 (see java.net for sources). Right. This is actually a good example not to follow ;-). BTW the big patch use NIO blocking, which may improve scalability, but will impact throughput. The patch should be improved to use NIO non-blocking. And then we can compare ;-) You're going to have to prove your point in a big way ;) There were articles on the subject advocating the same thing, but if you looked a little into it, you could just see how to make the whole thing break really easily. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [OT] - benchmark on APR stuff
my apologies, I forgot to mention you need either the jmeter in my directory or a nightly build to use the testplans. good thing you found the solution. peter lin On 5/3/05, Peter Rossbach [EMAIL PROTECTED] wrote: Thanks it works with the cvs head from jmeter (2.1) Peter Jason Brittain schrieb: Peter Rossbach wrote: Hey Peter, I have download and install Jmeter 2.0.3 and want load your testplans, but I got this error: 2005/05/03 06:45:43 INFO - jmeter.gui.action.Load: Loading file: D:\peterlintestplan\concurrent_1.jmx 2005/05/03 06:45:43 ERROR - jmeter.save.SaveService: Problem loading part of file org.apache.avalon.framework.configuration.ConfigurationException: No attribute named class is associated with the configuration element testelement at - [snip] What I made wrong? Apparently JMeter 2.0.3 is too old. :) I downloaded and built JMeter from CVS head, and that allowed me to load and run his test plans. Cheers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Initial test of APR on Solaris
On 5/3/05, Bill Barker [EMAIL PROTECTED] wrote: - Original Message - From: Remy Maucherat [EMAIL PROTECTED] To: Tomcat Developers List tomcat-dev@jakarta.apache.org Sent: Tuesday, May 03, 2005 1:43 AM Subject: Re: Initial test of APR on Solaris Bill Barker wrote: Yeah, that works for me as well. My problem now is that the APRized HTTP Connector dies about 70% of the way through a test when I use the HTTPClient option in JMeter. A thread-dump shows all of the Workers waiting, and the Poller polling, but nothing is happening. It's a bit happier when I There's not much wrong with that situation, since it's a bit hard to tell what the client is doing. Maybe it's the 62 sized poller causing trouble if running on Windows (you should get a message about that), in which case you'd need to build APR from scratch. Is the Acceptor thread still accepting and giving that to the workers ? Only JMeter is running on Windows, and yes, the Acceptor is still accepting. I'm guessing it's a JMeter problem, since it's just too consistant. I'll try to verify whether this is really an issue with jmeter or something else. peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat/APR benchmark results
Cool. I assume you were finally able to my test plan mladen? I plan to run the benchmarks on my system this weekend, not that my laptop is healthy. peter On 4/21/05, Mladen Turk [EMAIL PROTECTED] wrote: Hi, Here are the brief results for Tomcat HEAD: Server Threads Pause (ms) Error(%) Rate (req/sec) Apache2.0.49500 1000 1.74 124.2 Http11Protocol 500 1000 0.20 139.5 Http11AprProtocol 500 1000 0.00 266.9 Tests has been run on SLES9/64-Bit/Java-1.5.0_02-b09 Client was WINXP/SP2 running JMeter and Peter Lin's test plans. Those tests simulate 500 concurrent users connecting to a 150 maxClient server. The connection is keep-alive with 1 second delay between the requests. So our JNI/APR http connectors is almost twice faster then both Tomcat standard http connector and Apache2/prefork. What is more important is that the error number is zero, meaning there were no refused connections, because of keepalive connection poller. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat/APR benchmark results
I'll be using my AMD 2ghz linux box running Fedora Core1. haven't updated to FC3 yet, though remy keeps suggesting I upgrade :) peter On 4/21/05, Mladen Turk [EMAIL PROTECTED] wrote: Peter Lin wrote: Cool. I assume you were finally able to my test plan mladen? Yes. I've run them extensively, thanks for sharing those. Been playing with the constant timer, number of threads, etc... Results show that for high number of connections the APR implementation is way ahead of any other. Me very happy :) I plan to run the benchmarks on my system this weekend, not that my laptop is healthy. Sure. If you are using WIN32 as server, you will need a patched APR to be able to handle more then a 64 FD_SETSIZE limit. Tell me, and I can mail it to you. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
APR + JNI benchmark
I was finally able to run some benchmarks after I fixed my laptop. So far I've only tested against the 5.5.4 to establish a base line. So far what I found is 1000 concurrent keep alive connections is the limit for my linux box. 500 is ok, but 1K concurrent connection over loads my linux server. i will write a new test plan with smaller steps from 200, 300, 400, 500. peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
test plans for new APR + JNI
I've posted 2 test plans designed to test out concurrent connections. http://cvs.apache.org/~woolfel/native_testplans.zip there's 2 test plans. both use the 20K.png file I created for the previous benchmark. there's 4 thread groups in test plan. the first one has a constant delay of 1second between requests the second has 500ms delay between requests the number of threads go from 250, 500, 1000, 2000 peter lin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[OT] - benchmark on APR stuff
I haven't been able to run the benchmarks so far due to BSOD on my laptop. I suspect SP2, since I just installed it friday and now I get IRQ errors related to the network interface. the test plan is in my apache directory, can someone else run the benchmarks until I figure out how to fix this stupid BSOD nightmare? http://cvs.apache.org/~woolfel/native_testplans.zip peter lin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat and APR
yeah, I can do that. ... I assume if i grab the nightly for 5.5.x and APR1.1.x I should be ready to go. In the event I need some assistance, you going to be around Mladen :) ? peter lin On 4/15/05, Remy Maucherat [EMAIL PROTECTED] wrote: Peter Lin wrote: I'll wait until that's fixed and then run the full set of benchmarks. that way we'll have direct comparison. All right, the performance is now more or less decent, and polling seems to work. You can try testing it :) Mladen recommends APR 1.1. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat and APR
if I have time this weekend, I'll try to run the same benchmarks on the latest code. is it included in the nightly build? if not, can someone post a build for me to benchmark on my system? peter On 4/14/05, Remy Maucherat [EMAIL PROTECTED] wrote: Hi, This has been hinted for a while ;) The purpose of this email is to propose using APR (Apache Portable Runtime) as the network IO used by Tomcat, instead of the JVM's IO. This will provide the following benefits (some small, others more significant): - zero GC IO API (APR is not object based) - flexible handling of socket reads/writes using pollers and non blocking IO - sendfile support Which will allow: - better scalability when using HTTP/1.1 keepalive, by eliminating the need for tying a thread blocked on an IO read (one or more (one on Unix, plenty on Windows) pollers will be used instead); this will allow certain single machine servers which couldn't use keealive in order to limit the amount of threads (to be able to have more simultaneous users) to benefit from the (large) performance benefits of keepalive - better performance and easier configuration when using AJP: one of the issues of AJP at the moment is that the number of AJP processors in Tomcat must be equal to the number of processors on the front end native servers, or keepalive has to be disabled (resulting in performance degradation); the benefit for AJP seems to actually be greater than for HTTP (special thanks to Mladen who explained to me the situation in detail) - efficient static file serving (not done yet) - (likely) better performance and reliability on free JVMs - probably slightly better performance (over NIO, Mladen measured 10%, while I measured 0%; as classic IO is a bit slower generally, I expect a small improvement); unfortunately, maximum throughput suffers (maximum throughput is measured by ab with keeplive enabled on localhost), as we cannot just block on the read until the next request comes; the benchmarks I use usually suck anyway ;) - SSL support through OpenSSL (not done right now) - access to certain useful native functions (detailed system status information, random data generation, etc) This would effectively make Tomcat more appealing to the lower end servers and, hopefully, web hosting companies (due to lower resource usage). Large sites would likely also benefit a little from the AJP improvements. The implementation would be an alternate endpoint implementation, replacing PoolTcpEndpoint. Alternate HTTP/1.1 processor and socket channel (for AJP) will be provided. Development required is actually fairly small (significant testing will be required, however). I didn't do the updated socket channel yet, but after 3 days of hacking, HTTP works (and the AJP stuff should be much faster to do). Now the tricky points: - is it appropriate for 5.5 ? I'd say yes, as it will be a separate independent implementation - will require APR, and a small JNI friendly wrapper library tentatively named libtcnative by Mladen (Windows binaries - and likely others - will be provided); portability should be good - we'll need to improve performance (and especially fix the kludge that is currently used to decide if a socket should be put inside a poller) Comments ? Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat and APR
I'll wait until that's fixed and then run the full set of benchmarks. that way we'll have direct comparison. peter On 4/14/05, Remy Maucherat [EMAIL PROTECTED] wrote: Peter Lin wrote: if I have time this weekend, I'll try to run the same benchmarks on the latest code. is it included in the nightly build? if not, can someone post a build for me to benchmark on my system? We're going to have to resolve the issue I mentioned with keepalive before doing serious benchmarking. If you're on Linux, then it should be easy to test: build APR (I'll let Mladen precise any version requirement), and then build the native stuff in j-t-c/jni. Then make sure both libs are available to the JVM. APR is more portable than certified JVMs at the moment, although gcj-4 looks to be a big improvement (although not certified). Then use protocol=org.apache.coyote.http11.Http11AprProtocol on the connector element. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: New TLP draft
not sure if this is important or not, but how should the migration of the mailing list happen? everyone subscribing to tomcat-user an tomcat-dev automatically get subscribed to the new one? peter On Apr 6, 2005 8:15 AM, Remy Maucherat [EMAIL PROTECTED] wrote: Hi, Here's a new draft with the necessary updates. I suppose this needs to be sent to the PMC for approval. If this draft is ok, I will send it there. Then there are infrastructure taks: - renaming mailing lists - moving CVS - new DNS and virtual host To summarize things: - I will be the proposed first PMC chair - PMC chairs will serve for one year, and cannot serve consecutive one year terms (shouldn't this be put in the resolution text ? - if so, please help writing it, as I can't write ASF resolution compliant language) - although the ASF infrastructure is quite aggressive in pushing Subversion, I find it harder to work with than CVS at the moment (including crucial - for me - revision graphs, and better tool support); as a result, I think I would prefer keeping CVS for the time being - feel free to take any further actions to update the PMC list (as proposed by Costin) Rémy --- Draft TLP resolution --- X. Establish the Apache Tomcat Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to the implementation of the Java Servlet and Java Server Pages specifications, for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache Tomcat PMC, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache Tomcat PMC be and hereby is responsible for the creation and maintenance of software related to creation and maintenance of open-source software related to the implementation of the Java Servlet and Java Server Pages specifications based on software licensed to the Foundation; and be it further RESOLVED, that the office of Vice President, Apache Tomcat be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache Tomcat PMC, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache Tomcat PMC; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache Tomcat PMC: Jean-Francois Arcand ([EMAIL PROTECTED]) Bill Barker ([EMAIL PROTECTED]) Ian Darwin ([EMAIL PROTECTED]) Jean-Frederic Clere ([EMAIL PROTECTED]) Tim Funk ([EMAIL PROTECTED]) Henri Gomez ([EMAIL PROTECTED]) Filip Hanik ([EMAIL PROTECTED]) Larry Isaacs ([EMAIL PROTECTED]) Jim Jagielski ([EMAIL PROTECTED]) Jan Luehe ([EMAIL PROTECTED]) Costin Manolache ([EMAIL PROTECTED]) Remy Maucherat ([EMAIL PROTECTED]) Kurt Miller ([EMAIL PROTECTED]) Glenn Nielsen ([EMAIL PROTECTED]) Amy Roh ([EMAIL PROTECTED]) Peter Rossbach ([EMAIL PROTECTED]) Yoav Shapira ([EMAIL PROTECTED]) Mark Thomas ([EMAIL PROTECTED]) Mladen Turk ([EMAIL PROTECTED]) Keith Wannamaker ([EMAIL PROTECTED]) NOW, THEREFORE, BE IT FURTHER RESOLVED, that Remy Maucherat be appointed to the office of Vice President, Apache Tomcat, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the initial Apache Tomcat PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache Tomcat Project; and be it further RESOLVED, that the initial Apache Tomcat PMC be and hereby is tasked with the migration and rationalization of the Apache Jakarta PMC Tomcat subproject; and be it further RESOLVED, that all responsibility pertaining to the Jakarta Tomcat sub-project and encumbered upon the Apache Jakarta PMC are hereafter discharged. --- End draft TLP resolution --- - To unsubscribe, e-mail:
Re: New TLP draft
you mean at the speed of light. It's already April, so they only have 5-6 months if they really want to get plugin out that matches the current CVS support. peter On Apr 6, 2005 11:37 AM, Remy Maucherat [EMAIL PROTECTED] wrote: They'd have to improve a lot very fast (from what was posted, CVS at the ASF dies at the end of the year) ;) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: TLP Draft Proposal
my non scientific tests with jdk1.5 using JMeter tells me it's a bit faster than jdk1.4, but I doubt it is faster than SWT. If I had more free time, I would definitely be interested in a SWT admin application. oh well, that's life :) peter On Wed, 23 Mar 2005 15:49:46 +0100, Henri Gomez [EMAIL PROTECTED] wrote: Should I read that now Swing on SDK 1.5 is faster than SWT ? On Wed, 23 Mar 2005 15:46:10 +0100, Remy Maucherat [EMAIL PROTECTED] wrote: Henri Gomez wrote: Did there is interest in a JFACE/SWT admin application ? If you ask me, Eclipse's strength is no longer SWT. At the moment, it's more like a liability, as Swing has become better. The exception is still on Windows (to some extent) where the Swing impl still needs some more polish (font antialias support, for example). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: TLP Draft Proposal
:) like I said, totally unscientific eye ball. but I'm also totally bias. I prefer the SWT API over swing. even though I have to use swing for JMeter. of course, not that my opinion really matters, since Im not a tomcat committer ;p peter On Wed, 23 Mar 2005 18:40:09 +0100, Remy Maucherat [EMAIL PROTECTED] wrote: Peter Lin wrote: my non scientific tests with jdk1.5 using JMeter tells me it's a bit faster than jdk1.4, but I doubt it is faster than SWT. If I had more free time, I would definitely be interested in a SWT admin application. oh well, that's life :) As you may have noticed, I do like some of the Eclipse stuff (and I use Eclipse-the-IDE itself). Regardless of its performance (which is actually bad on non Windows systems, from what I understand), I would however have to veto usage of SWT, given the better overall performance one gets with Swing. Obviously, you can try to demonstrate that this is not the case, and SWT is superior on all platforms, but somehow I doubt it. The only remaining benefit I see is portability to non Sun JVMs, as the free implementation of Swing isn't complete yet (this seems to be progressing very quickly at the moment, however). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: TLP Draft Proposal
sleep deprevation is good for you! sorry I couldn't resist. peter On Wed, 23 Mar 2005 13:28:23 -0500, Henri Yandell [EMAIL PROTECTED] wrote: The sleepless nights, the stress, the joy, the tears Although that might be more to do with my 5-month year old son than anything to do with Jakarta :) Hen - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Time for TLP?
I'm not a committer, but thought I'd give my half pence worth. Tomcat has come a long way and is second to none. TC5.5.x really has made tremendous strides and is even with the best servlet containers out there. it does seem like the java community at large would like to see tomcat as TLP, so might as well. Jakarta is pretty healthy, so I don't think it's necessary to use TC to boost the visibility of jakarta anymore. peter lin On Tue, 22 Mar 2005 13:23:29 +0100, Mladen Turk [EMAIL PROTECTED] wrote: Henri Gomez wrote: All in all, it will mean more work on our side, but I think It's worth the effort, because the Tomcat will be perceived as 'lean-and-mean-sexy-machine' :) What will be the extra works ? Mostly administrative and documentation namespace change, but think it's one-time-job. So the move is to go TLP ? Well, like Tim said, if not now, we'll come to this subject in couple of months again. IMO now, is as well as good as any other time. Let's do that admin task and move forward to what's count; coding. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat for professional large scale webapps?
100 concurrent users don't necessarily mean heavily loaded. Try 5000 concurrent users or something even higher. Keep in mind the bottleneck will be your database, so try to figure what 100 concurrent users means in terms of peak and average concurrent requests. in other words. What are the chances all 100 users will send a request at the same time? I'm gonna guess it's not very likely. If anything, I would be surprised if 100 concurrent users results in 25 concurrent requests during peak and 15 during average. In either case, remember your Network IO will be a major limitation. If the application is on a LAN, then you're fine. If it's hosted at an ISP and you only have 10mb link, it's not going to be able to handle 25 concurrent requests. If you're hosted at a nice ISP that gives you a 100mb link, you should be ok. The only real way to know if tomcat can handle the load and how many servers you need is to write the app and stress test it. Once you get a good measure of the average response time, you'll have more information to decide. If you look at the latest benchmarks I ran, it's going to be hard to find a servlet container that is significantly better. In fact, I would say 5.5.x is even with Resin now. hope that helps peter On Tue, 1 Mar 2005 07:13:37 -0500, Yoav Shapira [EMAIL PROTECTED] wrote: Hola, Numerous users/organizations have reported using Tomcat for that number of concurrent users and higher. Achieving good and efficient programming is usually the bottleneck. You might also want to try clustering and load-balancing your Tomcats. If you don't want to involve Apache you can use a Tomcat with the Balancer webapp as your front-end for load-balancing requests. Yoav -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Tuesday, March 01, 2005 3:52 AM To: tomcat-dev@jakarta.apache.org Subject: Tomcat for professional large scale webapps? Hello, I made very good experiences using Apache Tomcat for small scale webapps. However, I am now thinking of using it for a larger scale professional webapp: 100 conconcurrent Users. Haevy downloads, ... Assumed that programming is good and efficient, in what kind of difficulties may I run using Tomcat for a larger scale application? Is it a good choice (in terms of scalability, efficiency, memory usage, ...)? My App environment would be: Tomcat 5.x, Struts, Oracle, Lucene Greetings, Joos -- DSL Komplett von GMX +++ Superg|nstig und stressfrei einsteigen! AKTION Kein Einrichtungspreis nutzen: http://www.gmx.net/de/go/dsl - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Some benchmark results
I've started running a series of benchmarks for static files. Here are some early results. I plan to write up the results once it's all done. Server: AMD 2ghz RAM 1Gb jdk1.4.2 tomcat 5.0.x Client: gateway laptop 450 centrino 1.4ghz RAM 1Gb The basic setup Concurrent threads: 5, 10, 15, 20, 25, 30 page size in KB: 1, 5, 10, 20, 40, 80, 160 image size KB: 1, 5, 10, 20, 40, 80, 160 The results I have so far using JMeter. peter -- 1K --- protocol | samples | average | median | 90% line | min | max | error% | throughput | Kb/sec --- HTTP Request50002.4158 0 10 0 231 0.00% 1296.7/sec 1337.20 HTTP Request1 3.2866 0 10 0 490 0.00% 1384.8/sec 1428.13 HTTP Request15000 3.56353 0 10 0 36260.00% 1335.0/sec 1376.71 HTTP Request2 6.3406 0 20 0 120 0.00% 1352.1/sec 1394.33 HTTP Request25000 8.27288 0 20 0 281 0.00% 1309.1/sec 1350.02 HTTP Request3 4.6102 0 20 0 972 0.00% 1259.2/sec 1298.53 5k HTTP Request50002.9246 0 10 0 330 0.00% 1044.5/sec 5314.28 HTTP Request1 4.4717 0 10 0 11620.00% 1002.6/sec 5101.15 HTTP Request15000 6.7102 0 20 0 23730.00% 968.9/sec 4929.49 HTTP Request2 4.83515 0 10 0 15030.00% 950.1/sec 4834.10 HTTP Request25000 10.3578 0 30 0 19220.00% 933.6/sec 4749.89 HTTP Request3 5.86573 0 20 0 441 0.00% 939.4/sec 4779.46 10k HTTP Request50004.1644 0 10 0 742 0.00% 726.7/sec 7308.61 HTTP Request1 8.8132 10 20 0 181 0.00% 763.5/sec 7678.00 HTTP Request15000 7.7906 0 20 0 381 0.00% 718.0/sec 7221.14 HTTP Request2 10.8593 10 30 0 420 0.00% 705.2/sec 7091.88 HTTP Request25000 32.2276 30 50 0 110 0.00% 704.0/sec 7079.95 HTTP Request3 19.3326 10 50 0 16530.00% 686.8/sec 6906.56 20k HTTP Request50007.4202 0 10 0 400 0.00% 491.9/sec 9936.63 HTTP Request1 14.1984 10 20 0 982 0.00% 490.9/sec 9917.12 HTTP Request15000 17.698 10 40 0 531 0.00% 472.4/sec 9542.36 HTTP Request2 22.9608 20 50 0 471 0.00% 465.1/sec 9395.46 HTTP Request25000 32.8229 40 61 0 14320.00% 461.3/sec 9318.23 HTTP Request3 50.4059 50 90 0 701 0.00% 429.6/sec 8678.60 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [INTRODUCE] apr-java
awesome, i'll check it out tonight. You rock mladen! peter On Fri, 07 Jan 2005 15:25:07 +0100, Mladen Turk [EMAIL PROTECTED] wrote: Hi, Here is the work in progress for a new project I named apr-java. It offers a 'thin' layer using JNI over APR library. The initial code that I wrote over a year now had two way api, but I've decided to leave only Java-APR. I also have a working set of configure and make files for unixes, but since this is an overview of the technology, it's only a small subset of entire project, but you can get the picture :) . It will support around 90% of APR API (without things like strings, etc. that are well done inside java itself). The API itself on the Java side is done as close to the APR function call (apr_file_write - File.write) with almost the same function parameters as well. You can see that the wrapping code is very thin in most cases with only a couple of lines for each function, mostly caused by pointer issues. I'm sending the gz file (tried zip but failed) so you can see what the general idea is. Of course the library can be extended with functions that APR doesn't offer (for now or never will). One of the things would be setting user and group for a process, sending data from parent to child process, etc. The question is: Is it acceptable (think that previous discussions say it is) Where to put that in the cvs. Comments? Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat monitoring
I believe you would have to add those to the Thread MBean. What you see in the status servlet is what is maintained currently. peter On Wed, 05 Jan 2005 13:24:02 -0600, Jess Holle [EMAIL PROTECTED] wrote: I'd like to be able to monitor the maximum requests active (within a connector thread pool) that are used at any time (and reset this maximum). I'd like to be able to monitor the backlog (ala acceptCount) at a given point in time and the maximum to date (and reset this maximum). [I already noticed the ability to monitor the currentThreadsBusy in ThreadPool.] Am I (hopefully) overlooking this in the existing MBean stack? Or are these candidates for addition? -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
to elaborate a bit more my thoughts on the kind of stats would be useful from a monitoring perspective * system load * system freeram * system total ram * system free ram * open connections * # of connections timed_wait I'm sure are other stats that are useful. A combination of the existing stats from the status servlet plus a few system stats should go a long way and make life a little bit easier. peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
that sounds great. does it have support for sysinfo? if it does, I'll try using your apr-java package. peter On Sat, 01 Jan 2005 15:18:39 +0100, Mladen Turk [EMAIL PROTECTED] wrote: Costin Manolache wrote: Well, I'm working over a year now on a project that I've called apr-java. This is a thin (for now) wrapper around apr and apr-utils, so it will be supported on all platforms the apr is. BTW - it would be really great if it would use the SWT model, i.e. JNI methods matching exactly the APR signatures and param types, with minimal ammount of C wrapper code. It works really well, and it's the easiest to maintain and fastest of all JNI flavors I've seen. Exactly. IMO the SWT from Eclipse proved to be as effective and easy to use as any other Java package or library. And that is what I have done with my apr-java. Right now I've implemented it using the org.apache.apr.native with the Library, Pool, Error, File, Mutex, Mmap, Shm, Socket and Pollset as containers for specific APR functions with exactly the same function prototypes. The wrapping code is really minimal, and in lot cases done as macros. I did not tried to wrap the code that has better or direct implementation in Java like string, thread, etc... Also the project does not prevent that we add any additional functionality not present in the APR. Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
sysinfo on unix/linux should be pretty easy. I've used windows performance stats before when i tried to write the equivalent of the status servlet for IIS. I will try to write an exe named sysinfo that spits out similar performance stats. how can I help mladen? peter On Mon, 03 Jan 2005 15:55:44 +0100, Mladen Turk [EMAIL PROTECTED] wrote: Peter Lin wrote: that sounds great. does it have support for sysinfo? if it does, I'll try using your apr-java package. No, but it's up to us to decide what will go inside. APR is included, but I wish to leave that as open as it could be. It already have win32.c,unix.c and netware.c files for platform specific stuff that APR doesn't offer. Having sysinfo sounds good to me. WIN32 has also good performance data gathering, and I'm sure that Netware has them too. I also wish to include the OS specific things from httpd like setting group, user, sending data to child process, etc... What matters is that we'll have a generic native component, with well defined build and distribution specification. Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
hey mladen, is apr-java available in the normal APR distribution? peter On Mon, 03 Jan 2005 15:55:44 +0100, Mladen Turk [EMAIL PROTECTED] wrote: Peter Lin wrote: that sounds great. does it have support for sysinfo? if it does, I'll try using your apr-java package. No, but it's up to us to decide what will go inside. APR is included, but I wish to leave that as open as it could be. It already have win32.c,unix.c and netware.c files for platform specific stuff that APR doesn't offer. Having sysinfo sounds good to me. WIN32 has also good performance data gathering, and I'm sure that Netware has them too. I also wish to include the OS specific things from httpd like setting group, user, sending data to child process, etc... What matters is that we'll have a generic native component, with well defined build and distribution specification. Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
So which way would be best/better to proceed? Since mladen has his apr-java stuff, would it make sense to do this? 1. write native windows dll 2. write apr component 3. use apr-java to wrap apr 4. wrap apr-java with mbeans or 1. write apr component to call system level API 2. use apr-java to wrap apr 3. wrap apr-java with mbeans or something completely different? peter On Mon, 03 Jan 2005 11:21:55 -0800, Costin Manolache [EMAIL PROTECTED] wrote: Benson Margulies wrote: For systems with a /proc file system with these statistics, this doesn't require any JNI ... Yes, there are a lot of ways to workaround java limitations. /proc is one. But even on linux, a lot is not exposed via /proc, but ioctl. I guess the goal is not to implement sysinfo, but have a way to get more platform-specific information and access platform-specific features. Costin -Original Message- From: Peter Lin [mailto:[EMAIL PROTECTED] Sent: Monday, January 03, 2005 10:04 AM To: Tomcat Developers List Subject: Re: adding features to Status servlet sysinfo on unix/linux should be pretty easy. I've used windows performance stats before when i tried to write the equivalent of the status servlet for IIS. I will try to write an exe named sysinfo that spits out similar performance stats. how can I help mladen? peter On Mon, 03 Jan 2005 15:55:44 +0100, Mladen Turk [EMAIL PROTECTED] wrote: Peter Lin wrote: that sounds great. does it have support for sysinfo? if it does, I'll try using your apr-java package. No, but it's up to us to decide what will go inside. APR is included, but I wish to leave that as open as it could be. It already have win32.c,unix.c and netware.c files for platform specific stuff that APR doesn't offer. Having sysinfo sounds good to me. WIN32 has also good performance data gathering, and I'm sure that Netware has them too. I also wish to include the OS specific things from httpd like setting group, user, sending data to child process, etc... What matters is that we'll have a generic native component, with well defined build and distribution specification. Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
that sounds like a good idea :) look forward to trying it out once mladen checks in the code. peter On Mon, 03 Jan 2005 18:44:15 -0800, Costin Manolache [EMAIL PROTECTED] wrote: Or: 1. Wait for Mladen to check in the code 2. figure out how to build it and how to install it easily in tomcat ( and other java applications ). 3. probably add one native method. I don't think wrapping it as apr component makes sense ( apr is not a component system like xpcom ). 4. write the java mbean - using the native method and making all conversions between the native signature and java style ( if SWT-style of jni is used - i.e. using byte[], int pointers, etc - and doing java adaptation in java ). I'm as curious as you are to see the code and figure out how it can be used, I love jni :-) Costin Peter Lin wrote: So which way would be best/better to proceed? Since mladen has his apr-java stuff, would it make sense to do this? 1. write native windows dll 2. write apr component 3. use apr-java to wrap apr 4. wrap apr-java with mbeans or 1. write apr component to call system level API 2. use apr-java to wrap apr 3. wrap apr-java with mbeans or something completely different? peter On Mon, 03 Jan 2005 11:21:55 -0800, Costin Manolache [EMAIL PROTECTED] wrote: Benson Margulies wrote: For systems with a /proc file system with these statistics, this doesn't require any JNI ... Yes, there are a lot of ways to workaround java limitations. /proc is one. But even on linux, a lot is not exposed via /proc, but ioctl. I guess the goal is not to implement sysinfo, but have a way to get more platform-specific information and access platform-specific features. Costin -Original Message- From: Peter Lin [mailto:[EMAIL PROTECTED] Sent: Monday, January 03, 2005 10:04 AM To: Tomcat Developers List Subject: Re: adding features to Status servlet sysinfo on unix/linux should be pretty easy. I've used windows performance stats before when i tried to write the equivalent of the status servlet for IIS. I will try to write an exe named sysinfo that spits out similar performance stats. how can I help mladen? peter On Mon, 03 Jan 2005 15:55:44 +0100, Mladen Turk [EMAIL PROTECTED] wrote: Peter Lin wrote: that sounds great. does it have support for sysinfo? if it does, I'll try using your apr-java package. No, but it's up to us to decide what will go inside. APR is included, but I wish to leave that as open as it could be. It already have win32.c,unix.c and netware.c files for platform specific stuff that APR doesn't offer. Having sysinfo sounds good to me. WIN32 has also good performance data gathering, and I'm sure that Netware has them too. I also wish to include the OS specific things from httpd like setting group, user, sending data to child process, etc... What matters is that we'll have a generic native component, with well defined build and distribution specification. Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
right it is available in JDK5, but not everyone can use jdk5 :( I know plenty of people who are still using jdk1.3.1 and plenty are just moving to jdk1.4.2 now. many of the features are available in jdk5, but I believe what mladen is working on is beyond what jdk5 provides. I've only looked at some of the new features in jdk5. I guess i could target jdk5, but it would be nice to have a solution that can work with jdk1.4.2. peter On Tue, 4 Jan 2005 03:49:49 + (UTC), Kevin Offet [EMAIL PROTECTED] wrote: Peter Lin woolfel at gmail.com writes: that sounds like a good idea :) look forward to trying it out once mladen checks in the code. peter isn't what you are looking to do already available through the new 5.0 version jvm itself? startup tomcat with the additional jvm switch -Dcom.sun.management.jmxremote and then connect with your own custom MBean. to see what's possible, sun includes jconsole (a small gui app). it sure seems like everything you guys are talking about and more. Kevin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
hehe, well if it was just a simple JNI calling sysinfo, I would agree it would have limited lifespan. But considering how slow some people move, it would last atleast 3-4 years, which is still useful in my mind. given that mladen is working on apr-java, I would say it is more useful and would have a significantly longer lifespan. everything has a life span, but if it is useful, than i feel it's worth while. plus it's my own time, so not like anyone else is going to care if it only lasts for 2-4 years. I much rather have the option of using it with jdk1.4 and across all VM's than simply relying on Sun's jdk5. peter On Tue, 4 Jan 2005 04:46:48 + (UTC), Kevin Offet [EMAIL PROTECTED] wrote: Peter Lin woolfel at gmail.com writes: right it is available in JDK5, but not everyone can use jdk5 :( I know plenty of people who are still using jdk1.3.1 and plenty are just moving to jdk1.4.2 now. many of the features are available in jdk5, but I believe what mladen is working on is beyond what jdk5 provides. I've only looked at some of the new features in jdk5. I guess i could target jdk5, but it would be nice to have a solution that can work with jdk1.4.2. peter if it were only a matter of which version of jdk one uses, i'd say that you were proposing a solution that would have a limited lifetime. [as an aside - monitoring features could be a strong motivation for people to upgrade, which would be a good thing ;-) ] but having a solution independent of any one VM (sun/ibm/whoever) or version is the best. i do have a cold, but that is no excuse for being too fast of fingers and too slow of brain. ;-) i've been lurking the list for a while now and just itching to find some place where i might be able to help. so if you find you could use a little help, please drop me a note. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
it could be a separate module. It definitely should use MBean. In terms of getting the CPU load stats, I was thinking of calling the standard sysinfo loads[]. Is there some other way of getting the system load stats? or CPU stats? that doesn't require calling native code? peter On Fri, 31 Dec 2004 13:13:54 -0800, Costin Manolache [EMAIL PROTECTED] wrote: Peter Lin wrote: I'm thinking of adding system load stats to the status servlet. What do other's think about it? It would use JNI to call a native lib and it would only work on unix, but it would be good to have. I would also update JMeter in the process to display the system load average. peter lin Wouldn't be better to have a way to display an arbitrary mbean attribute, plus an mbean tracking system load ( and maybe memory/disk statistics ) ? Dealing with jni is allways tricky ( including build issues, etc ) - it is better to have it in separate modules. Costin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: adding features to Status servlet
that's why I decided it was a good idea to ask for other's thoughts. From a stress testing perspective, I find system load stats very valuable. breaking tomcat isn't something I find desirable either, but there has to be a better way to measure system load other than ssh into the server and use top. manually doing top or sysinfo is fine for load testing, but for performance monitoring, something automated is desirable. My thought was to make it optional and have it detect whether or not a native lib for that platform exists. If it doesn't it would affect the status servlet and would look exactly the same as it does now. on the otherhand, if the user enables it and a native lib exists, it could display the system load. the only other option is to lobby Sun to add system load stats to the VM, so that it is portable across platforms. peter On Fri, 31 Dec 2004 17:27:03 -0500, Frank W. Zammetti [EMAIL PROTECTED] wrote: I would personally have some reservations about doing this... It's a little better if it's a module you can activate and deactivate, but still... First, if it's not something you can do cross-platform, I'm not sure I'd like it. AFAIK Tomcat is nicely cross-platform now, anything that breaks that wouldn't be good I think. I understand this would be an optional component (right?), so it wouldn't actually be *breaking* anything, but the expectation I think is that anything in Tomcat works on all platforms, so would it be a good thing to introduce something that doesn't fit that mold? Second, and more importantly, it doesn't feel right to me to expose this type of information through an app server. Your talking about statistics that aren't truly related to the app server, although is certainly affected by the app server, so it's debatable whether they should be there or not. I agree this is useful information to have access to, but I'm not sure it'd be the right place for it. Have you considered maybe making this part of some Commons package? Make it something that any app could make use of, that might be a very nice thing. Heck, it might be somewhere in there already for all I know. Just my thoughts on it anyway. -- Frank W. Zammetti Founder and Chief Software Architect Omnytex Technologies http://www.omnytex.com Peter Lin wrote: it could be a separate module. It definitely should use MBean. In terms of getting the CPU load stats, I was thinking of calling the standard sysinfo loads[]. Is there some other way of getting the system load stats? or CPU stats? that doesn't require calling native code? peter On Fri, 31 Dec 2004 13:13:54 -0800, Costin Manolache [EMAIL PROTECTED] wrote: Peter Lin wrote: I'm thinking of adding system load stats to the status servlet. What do other's think about it? It would use JNI to call a native lib and it would only work on unix, but it would be good to have. I would also update JMeter in the process to display the system load average. peter lin Wouldn't be better to have a way to display an arbitrary mbean attribute, plus an mbean tracking system load ( and maybe memory/disk statistics ) ? Dealing with jni is allways tricky ( including build issues, etc ) - it is better to have it in separate modules. Costin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Leaving Millennium, going on vacation...
enjoy :) and eat plenty of good food. peter On Tue, 14 Dec 2004 13:02:26 -0500, Shapira, Yoav [EMAIL PROTECTED] wrote: Then you have a nice vacation as well ;) Well deserved, it's been an active and eventful year... Yoav Shapira http://www.yoavshapira.com -Original Message- From: Remy Maucherat [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 14, 2004 11:49 AM To: Tomcat Developers List Subject: Re: Leaving Millennium, going on vacation... Shapira, Yoav wrote: Hi, Tomorrow is my last day at Millennium, so this [EMAIL PROTECTED] address will become invalid. I'm going on vacation for most of the rest of December, and then I'll be back in January (although we'll see how busy school will be. This is just an FYI, I'll still be around for a few more days before vacation, but after the 19th I'm out of the country and not taking my laptop with me... Have a nice vacation :) I'll be on vacation too at the end of this week, but I'll have a laptop around. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 5.0.30 today...
LOL, man I couldn't help laughing. you guys are slacking off!! just kidding. peter On Fri, 03 Dec 2004 16:04:36 +0100, Remy Maucherat [EMAIL PROTECTED] wrote: Shapira, Yoav wrote: Bah ;) I posted a note to tomcat-user telling people of the fix, which is just a configuration. The next step would be to post the amended configuration file itself (struts-config.xml) on the download pages, and I'll do that today assuming no one objects. But a whole new release for this -- I hadn't planned on it, don't feel like it at the moment. I've been there already ;) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
glad jmeter helped
haha, glad it helped. I find myself using it quite a bit when I'm working on my own stuff on tomcat. it's a nice quick way to make sure the memory usage follows a regular pattern. It it doesn't I start up OptimizeIt :) peter On Tue, 9 Nov 2004 08:46:25 -0500, Shapira, Yoav [EMAIL PROTECTED] wrote: Hi, I'm also changing my vote to [ X ] Stable now that I've had more of a chance to test it over the weekend, including running some home-grown apps and their respective stress tests. (BTW Peter if you're watching this thread, the JMeter monitor thing works great!) Let's run this vote for 24 hours more, until tomorrow (Tuesday, 10 Nov 2004, around 1600h GMT), to give others a chance to vote. I'll post the vote results at that time, and make the relevant announcements as needed. Yoav Shapira http://www.yoavshapira.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: cvs commit: jakarta-tomcat-catalina/webapps/docs changelog.xml
where do I get a tinfoil hat? on a less silly note, thanks for such great software. starting next month I get to return to tomcat + java and get away from *cough* .NET + IIS peter On Wed, 15 Sep 2004 09:24:10 -0400, Tim Funk [EMAIL PROTECTED] wrote: I thought the setup time config was a reasonable compromise. It allows the tinfoil hat folks to be happy while providing no performance decrease. (Unless 2 new instance variables is an issue ;) ) -Tim Remy Maucherat wrote: [EMAIL PROTECTED] wrote: funkman 2004/09/15 05:59:46 Modified:webapps/docs changelog.xml Log: Start 5.5.3 - Server Header config You are simply ignoring my opinion, then. Fine :) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat Benchmarking / Load Testing
there's plenty of big sites using tomcat. They just don't say it. I know several sites getting millions of page views a day using tomcat just fine. peter On Tue, 31 Aug 2004 09:49:32 +0530, Gaurav Vaish [EMAIL PROTECTED] wrote: Hi Rémy, Thanks for your response. In anycase, is there a list of people / companies using Tomcat (standalone or with Apache)? Happy Hacking, Gaurav Vaish http://gallery.mastergaurav.net -- On Mon, 30 Aug 2004 23:51:22 +0200, Remy Maucherat [EMAIL PROTECTED] wrote: Gaurav Vaish wrote: Hi, I am looking for some good case-study on Tomcat loadtest and benchmarking. It may or may not be with mod_jk(2) however a study with the following paramters would be useful: - JDK Version - Tomcat version - OS (with version and SPs) - Apache Version (if not standalone) - Concurrent Users (Threads) - Response Time The problem is that we have a e-Learning application running on Tomcat 4.x (planning to migrate to 5.x) which faced severe problems when put on production server. Stress testing in labs were passed gracefully, however it gives several issues with around 500 concurrent users on the production server. In anycase, which would be more scalable (load) - standalone Tomcat or with Apache/mod_jk? The details of the production server are: - Red Hat Enterprise Server 9.0 - Kernel 2.4.9 - JDK 1.4.2 (Sun JDK) - Tomcat 4.0 (Standalone) - 2048MB RAM - 4-Processor CPU (2GHz each), Intel 386 Tomcat 5.0 is faster than 4.1 which is faster than 4.0, but we don't have any numbers to give you. Feel free to contribute results. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[OT] - anyone wants a gmail invite
I normally don't like to post off topic things, but I have a gmail invite. if any of the tomcat-developers want a gmail account, email me directly. peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Code Too Large for Try Statement in Catalina
the only way is to reorganize your jsp. this is an old issue dating
back quite a bit. are you using tomcat4 or 5?
if you're using tc4, I would recommend upgrading to tc4.1.x or 5.x.
the original jasper generated code which would easily exceed the
limit. the newer jasper2 which is used with tc4.1.x and 5.x does a
much better job.
peter
On Thu, 19 Aug 2004 12:28:32 -0700, Michael McGrady
[EMAIL PROTECTED] wrote:
I have the following error:
org.apache.jasper.JasperException: Unable to compile class for JSP
An error occurred at line: -1 in the jsp file: null
Generated servlet error:
[javac] Compiling 1 source file
C:\crackwillow\work\Catalina\localhost\_\pallet_jsp.java:12934: code too large for
try statement
} catch (Throwable t) {
^
C:\crackwillow\work\Catalina\localhost\_\pallet_jsp.java:30: code too large for try
statement
try {
^
C:\crackwillow\work\Catalina\localhost\_\pallet_jsp.java:17: code too large
public void _jspService(HttpServletRequest request, HttpServletResponse response)
^
3 errors
org.apache.jasper.compiler.DefaultErrorHandler.javacError(DefaultErrorHandler.java:130)
org.apache.jasper.compiler.ErrorDispatcher.javacError(ErrorDispatcher.java:293)
org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:340)
org.apache.jasper.compiler.Compiler.compile(Compiler.java:352)
org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:474)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:184)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
com.crackwillow.filter.GZIPFilter.doFilter(GZIPFilter.java:20)
Is there anything to do? I need this file to be a JSP file in Struts.
Michael
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Code Too Large for Try Statement in Catalina
what I meant was what Tim said.
peter
On Thu, 19 Aug 2004 12:59:59 -0700, Michael McGrady
[EMAIL PROTECTED] wrote:
Thanks, Tim,
The file actually is quite simple. It is a simple form with HTML radio
buttons for choosing colors. There are a LOT of colors, but the code is
pretty straightforward. I need to get the code into a JSP file so that
I can utilize Struts.
Michael
Tim Funk wrote:
1) don't use compile time includes
2) split your page into multiple files which can use jsp_includes. Any
file which needs to be this big is probably extrememly painful to debug.
3) followup to tomcat-user, not tomcat-dev
-Tim
Michael McGrady wrote:
I have the following error:
org.apache.jasper.JasperException: Unable to compile class for JSP
An error occurred at line: -1 in the jsp file: null
Generated servlet error:
[javac] Compiling 1 source file
C:\crackwillow\work\Catalina\localhost\_\pallet_jsp.java:12934: code
too large for try statement
} catch (Throwable t) {
^
C:\crackwillow\work\Catalina\localhost\_\pallet_jsp.java:30: code too
large for try statement
try {
^
C:\crackwillow\work\Catalina\localhost\_\pallet_jsp.java:17: code too
large
public void _jspService(HttpServletRequest request,
HttpServletResponse response)
^
3 errors
org.apache.jasper.compiler.DefaultErrorHandler.javacError(DefaultErrorHandler.java:130)
org.apache.jasper.compiler.ErrorDispatcher.javacError(ErrorDispatcher.java:293)
org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:340)
org.apache.jasper.compiler.Compiler.compile(Compiler.java:352)
org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:474)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:184)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
com.crackwillow.filter.GZIPFilter.doFilter(GZIPFilter.java:20)
Is there anything to do? I need this file to be a JSP file in Struts.
Michael
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [5.next] Refactoring save-to-XML
hey remy, have you tried XStream? the API is super simple and it's quite fast. Not sure if it would work, but we recently changed JMeter to use XStream instead. peter -Original Message- From: [EMAIL PROTECTED] Sent: 7/29/2004 6:56:40 AM To: [EMAIL PROTECTED] Subject: [5.next] Refactoring save-to-XML Hi, I'd like to make this feature less hardcoded (right now, it's ugly ;) ). A first step would be to extract the code to another helper class, but that's all I can think of. Does anyone have any ideas ? Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [5.next] Refactoring save-to-XML
On Thu, 29 Jul 2004 16:55:39 +0200, Remy Maucherat [EMAIL PROTECTED] wrote: I think it's a start. You identified the downsides well enough, but unfortunately, they're a showstopper for me. Additional tweaks are needed. I see that others suggested other technologies, but we'd have many issues as well (assuming XStream or the others can serialize the Catalina tree to a XML which would be readable, I don't see how it would possibly be able to restore it). the XStream API is super simple and I think it should be able to restore the object. XStream xstream = new XStream(); String xml = xstream.toXML(myObject); // serialize to XML Object myObject2 = xstream.fromXML(xml); // deserialize from XML the only catch I can see is it doesn't use attributes. In the case of JMeter, mike wrote a simple class to convert jmeter's hashtree to a simpler format. that resulted in smaller files and good performance. Another recurrent problems: - None of these things support manual editing of the XML file (well, they do, but any formatting, commets, etc is lost) - The way stuff is saved is clearly quite bad (save evrything, rather than just what is actually modified on one object) you do loose comments with XStream. Not sure how important comments are. XStream is xml from a java perspective, so it's the opposite of DOM in that respect. And this is where I think DOM might not suck after all: - it's a very good representation of the XML document (unlike all the other fast solutions, which usually discard the non meaningful stuff) - by keeping the elements in memory, and using the ContainerListener stuff that I want to remove, we could update the relevant DOM elements as needed rather than blindly save all the non default properties of every object (which doesn't work well with global config files to set defaults, and as soon as you press save all the defaults appear in every config file) - when asked, we serialize the DOM trees to files; if the serializer is good enough, we'll end up with the same as the original files, with only the needed modifications The problems: - it's slower (I think no, the files we're using are small) - we would need a DOM Digester to read the internal config (server.xml, all context files), while the usual Digester would handle the (much bigger) standard deployment descriptors with SAX Rémy another option might by Jibx, which is also xml from a Java perspective. It's much more flexible and allows for all sorts of mapping/conversion. peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Some benchs results : WAS: Invitation to HTTPD commiters in tomcat-dev
the nightly build of jmeter has an alpha sampler that uses Commons HTTPClient. you may want to try that one instead, if you use jmeter peter On Thu, 22 Jul 2004 15:09:19 +0200, Henri Gomez [EMAIL PROTECTED] wrote: Remy Maucherat wrote: Henri Gomez wrote: I made some benchs on my Linux Fedora Core 2 on a P4 2.8ghz / 1Gb RAM : Apache 2.0.50 in - Apache 2.0.50 alone (simple html file) - TC 3.3.2/Coyote 1.1 - Apache 2.0.50 + jk 1.2.6 + TC 3.3.2/jk2 JkMount /examples/* local worker.local.port=8009 worker.local.host=localhost worker.local.type=ajp13 worker.local.cachesize=16 worker.local.cache_timeout=600 worker.local.socket_keepalive=1 worker.local.socket_timeout=300 - Apache 2.0.50 + mod_proxy + TC 3.3.2 (Coyote 1.1). ProxyPass /tc3/ http://localhost:11011/ ProxyPassReverse /tc3/ http://localhost:11011/ Apache Bench is running on another machine, Windows 2000 P3 1Ghz, and both systems are on a switched 100Mbps network : Apache 2 alone 1202 req/s TC/Coyote 883 req/s Apache 2 + jk + TC906 req/s Apache 2 + proxy + TC497.req/s(but with 8000 errors ;( Constatation : - Remy make a tremendous works since Coyote HTTP 1.1 is only 15% slower than the Apache 2 native HTTP. - mod_proxy is 50% slower than mod_jk and that's a really bad news. Also many errors appears, about 4% errors. - Tomcat via jk or mod_proxy, when on the same machine make a cpu load of 60% system and 30% user. Tomcat alone is 33% system and 10% user. How could we optimize mod_proxy settings since I'm using the standard httpd.conf ? It's quite bad :( Did you check everything was ok using verbose ? ab -n 1 -v 10 All your tests show Keep-Alive requests:0 in the result. It should work ok with Tomcat standalone (to be honest, I didn't try 3.3 with the current HTTP/1.1 connector), and with Apache as well. ab uses HTTP/1.0 keepalive with the -k option. Well I was thinking ab (2.0.40) use HTTP 1.1. I'll retest it with JMeter :) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Some benchs results : WAS: Invitation to HTTPD commiters in tomcat-dev
you can run it in non-Gui mode with -n option. http://jakarta.apache.org/jmeter/usermanual/get-started.html#non_gui might help, or not. peter On Thu, 22 Jul 2004 15:33:41 +0200, Henri Gomez [EMAIL PROTECTED] wrote: Peter Lin wrote: the nightly build of jmeter has an alpha sampler that uses Commons HTTPClient. you may want to try that one instead, if you use jmeter peter made some tests with JMeter 2.0.1 but my laptop is way to slow. I need another smaller stress tool ;( - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Invitation to HTTPD commiters in tomcat-dev
although I'm not a commiter, I like to add 2 cents to the discussion. I like the idea of supporting JMX and the capbility of deploying a webapp without restarting the server. From the discussions so far, the task isn't simple, and may not fit the majority of users. if 80% of the users don't have this need, justifying the extra features and possibly added complexity is debatable. Would it be sufficient to create a hook for more advanced mods? There are people using tomcat's admin tool to deploy/redeploy webapps, so having the feature in mod_proxy or whatever would make the edge cases less painful. using JMX would make managing a cluster of servers easier and reduce the need to login to every single server to stop, edit conf files, and restart. peter On Tue, 20 Jul 2004 08:18:39 -0700, Costin Manolache [EMAIL PROTECTED] wrote: Well, the mod_proxy + enhancements for sticky session + enhancements for passing auth info sounds reasonable - and if nobody wants the JMX support, then maybe we won't need to write a new connector anyway :-) Remy will be happy - we'll only use the http connector. Costin Graham Leggett wrote: Henri Gomez wrote: jk was designed a long time ago so may be mod_proxy allready support persistant connections. Persistence will happen on the backend on the condition there was persistence on the frontend. Generally the networks between backend and frontend are fast enough that connection setup is not a problem - a bigger problem is having expensive backend processes hanging around attached to a persistent connection that is not being used (assuming these connections are held by a tomcat thread of course, which may or may not be the case, not sure how tomcat is built internally). Great. And if you handle in the proxy_loadbalancing.c the JSESSION_ID, (sticky session) to map next requests to the tomcat who set it, you'll have everything needed. Sticky sessions has been on my list of things to support for ages - perhaps a proxy_sticky.c module could take a single parameter (the name of the parameter and/or cookie) and keep track of which server served it. If you had redundant frontends you might have a mechanism to keep track of which server is handling which session stored in a shared mechanism. A separate module might keep track of which tomcat servers are up or down, removing a server from the list of available servers on certain events (connection refused, error 4xx, 5xx, whatever). Well LDAP could be use for configuration outside a file. JMX/CMX goes a bit farther since it let you update some characteristics at runtime. But I agree that providing a JMX/CMX facade to Apache 2.x modules will be a good starting point. Costin will certainly clarify this point with you. In fine the discussed mod_ajp module should detect topology change in a second phase to learn for example that a tomcat in a cluster started/stopped a web application, so next requests could be redirected to another tomcat in the cluster. Also you should be able to update the load factor for each tomcat, may be having a load factor by Webapplication. In theory this kind of thing should not be limited to tomcat only, but to web applications (whether PHP, whatever) in general. Perhaps a mechanism that allows the backend to connect to the frontend and say status has changed, I am offering webapp XXX now, so count me into the pool. Or a mechanism for the frontend to poll the characteristics of the backend so that it can autolearn what webapp can be found where (has the advantage of not requiring a special backend module, apart from a magic URL on the backend giving relevant the information) This opens up some interesting possiblities for virtual mappings. Something like this: ProxyPass /myWebapp virtual://myWebbapp (or something) Where proxy can hand out the request to a backend that has recently said hi proxy, I serve myWebapp, feel free to contact me to fulfil a request. Regards, Graham -- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Request for Inputs for a Design/Architecture for VXML Application Server
I think I get a better picture. I was guessing it might be a Text To Speech application. I use to work on a wireless platform and one of the features we explored was converting emails to .wav files and send them to the user. in my experience, the wav file would have been sent to a WAP phone with a modified WAP browser capable of playing wav files. That's the reason I assumed a stream. In your case, there's no streaming from the app server to other components, so I don't see any real risk. that doesn't mean there isn't. Just that I don't see any based on the info you've provided. chance are, you won't find a general purpose application server written in C++ that will provide significant performance benefit over the current crop of servlet containers. I know several people who have written EJB and servlet containers over the last 10 years. I have a cousin who has written EJB containers on 3 different occasions for telecom applications. In terms of network I/O performance. It should not be an issue. if concurrency is a concern, I would suggest looking at EmberIO or IBM's asyncIO. the only way you're going to know is to build a very quick prototype in 1-2 weeks and stress the heck out of it. I tend to take that approach and do the worse case scenario. If it passes the worse case scenario, you know it can only get better. good luck peter On Fri, 9 Jul 2004 11:19:23 -0700, Darshan Rawal [EMAIL PROTECTED] wrote: Hi Peter, I suppose there is some confusion. I believe you got it right, that the Media Server is the one which will interpret the VXML pages and not the Application server. But these VXML pages wont have any SPEECH in it. Instead they have specific TAGS related to speech for e.g a snipplet from a VXML page would look like as below: prompt Welcome to the Message Center, Please Enter your password. /prompt Note that the prompt is in just TEXT format. These pages when interpreted on the Media Server side will resut into SPEECH being heard by the end user by the use of TTS(text to speech server). Thus there is completely no streaming of Speech between the Media Server and Application Server. (if need to pass in some specific speech file, the Media Server will just recieve a file Handle, maybe an ID which will map to some file on the NAS which is shared by Application Server Media Server) I hope that I have not confused more :) For the context related to the performance impact of JAVA, its mainly CPU operations. What we essectially want to know is whether introduction of JAVA into our platform(which is completely C/C++) bring about significant load on the resources(mostly CPU power). And if yes how much ? Would it be some thing that I need to be worried about ? Also, can we say that the use of JIT in JAVA can nullify (within reasonable limits) the overhead of JVM in JAVA and at what cost. Also, instead of JNI what do you think about the use of Sockets(TCP/IP) to communicate with our C/C++ processes(which any way listen on some TCP/IP socket). Would you expect performance degradation, by use of sockets through JAVA and if yes is there any reasonable and objective measurements ? Also, just in case, do you know of any objective comparision of JAVA C/C++ ? I could not find one on Google, which was not Objective enough. Thanks again for all you help, it was really valuable... Regards, Darshan Rawal. On Thu, 8 Jul 2004 21:23:52 -0500, Peter Lin [EMAIL PROTECTED] wrote: On Thu, 8 Jul 2004 18:59:23 -0700, Darshan Rawal [EMAIL PROTECTED] wrote: Hello Peter, I completely agree on this, but this interpretation of VXML pages is going to happen on the Media Server side. Hence as of now I am just concerned with generation of VXML pages and transporting them to the Media Server, where the CPU intensive XML parsing is going to happen. ok, so if I understand the setup correctly. the app server will only generate VXML and not consume it. If that is the case, CPU usage should not be an issue. For detailed information about XML generation performance, google for XML articles by Dennis Sosnoski. He has written up detailed articles in the past on xml parsers and xml binding parser. We are currently utilizing a 650MHz Sun Sparc Blade Solaris 8 Core OS, and it will remain the same for the next release, hence I know that CPU load will be a reasonable main concern. Having done stress testing with XML on Solaris systems: 280R, X1, UltraSparc5, as long as you're not consuming XML, CPU should not be an issue. Since XML is very verbose and Sun blades typically come with 2 or 4 ethernet ports, I would recommend separating the network traffic so the XML uses a dedicated router to the media server. Our entire Engineering team has reasonablly well knowledge of C++/C.(not so much of JAVA).Also, complete development is going to be done by our team. We are a part of the architecture team
Re: Request for Inputs for a Design/Architecture for VXML Application Server
by the way, there is a standard Text To Speech API and reference implementing for Java. I know Leap Wireless uses that with an EJB container for some internal applications. I can't go into details, but I do know of cases where it scales just fine. peter On Fri, 9 Jul 2004 11:19:23 -0700, Darshan Rawal [EMAIL PROTECTED] wrote: Hi Peter, I suppose there is some confusion. I believe you got it right, that the Media Server is the one which will interpret the VXML pages and not the Application server. But these VXML pages wont have any SPEECH in it. Instead they have specific TAGS related to speech for e.g a snipplet from a VXML page would look like as below: prompt Welcome to the Message Center, Please Enter your password. /prompt Note that the prompt is in just TEXT format. These pages when interpreted on the Media Server side will resut into SPEECH being heard by the end user by the use of TTS(text to speech server). Thus there is completely no streaming of Speech between the Media Server and Application Server. (if need to pass in some specific speech file, the Media Server will just recieve a file Handle, maybe an ID which will map to some file on the NAS which is shared by Application Server Media Server) I hope that I have not confused more :) For the context related to the performance impact of JAVA, its mainly CPU operations. What we essectially want to know is whether introduction of JAVA into our platform(which is completely C/C++) bring about significant load on the resources(mostly CPU power). And if yes how much ? Would it be some thing that I need to be worried about ? Also, can we say that the use of JIT in JAVA can nullify (within reasonable limits) the overhead of JVM in JAVA and at what cost. Also, instead of JNI what do you think about the use of Sockets(TCP/IP) to communicate with our C/C++ processes(which any way listen on some TCP/IP socket). Would you expect performance degradation, by use of sockets through JAVA and if yes is there any reasonable and objective measurements ? Also, just in case, do you know of any objective comparision of JAVA C/C++ ? I could not find one on Google, which was not Objective enough. Thanks again for all you help, it was really valuable... Regards, Darshan Rawal. On Thu, 8 Jul 2004 21:23:52 -0500, Peter Lin [EMAIL PROTECTED] wrote: On Thu, 8 Jul 2004 18:59:23 -0700, Darshan Rawal [EMAIL PROTECTED] wrote: Hello Peter, I completely agree on this, but this interpretation of VXML pages is going to happen on the Media Server side. Hence as of now I am just concerned with generation of VXML pages and transporting them to the Media Server, where the CPU intensive XML parsing is going to happen. ok, so if I understand the setup correctly. the app server will only generate VXML and not consume it. If that is the case, CPU usage should not be an issue. For detailed information about XML generation performance, google for XML articles by Dennis Sosnoski. He has written up detailed articles in the past on xml parsers and xml binding parser. We are currently utilizing a 650MHz Sun Sparc Blade Solaris 8 Core OS, and it will remain the same for the next release, hence I know that CPU load will be a reasonable main concern. Having done stress testing with XML on Solaris systems: 280R, X1, UltraSparc5, as long as you're not consuming XML, CPU should not be an issue. Since XML is very verbose and Sun blades typically come with 2 or 4 ethernet ports, I would recommend separating the network traffic so the XML uses a dedicated router to the media server. Our entire Engineering team has reasonablly well knowledge of C++/C.(not so much of JAVA).Also, complete development is going to be done by our team. We are a part of the architecture team which has to decide the approach to take, and I was just looking for some pointer on that issue(especially related to the performance implications for JAVA). when you say performance implications, what do you mean? high concurrent load? memory usage? I/O? threading? the context plays a big factor in whether or not there will be any issues. An prototype of a VXML browser in the Media Server has already been implemeted. It does have some performance issues. But, as I said earlier this is not our main concern as of now. The questions that we have related to taking the approaches are as follows:- 1) Would JAVA be a performance hit in terms of CPU utilization(especially bcos of the JVM) ? Note that our VXML document server is not the one doing a lot of CPU intensive operations, it will be our legacy application written in C/C++. Also, we are not doing significant CPU intensive operation like math operations, number crunching, encryption, etc. Would JIT provide reasonable performance compared to C/C+, by probably consuming more memory
Re: Request for Inputs for a Design/Architecture for VXML Application Server
from where I am sitting, sounds like the project doesn't have enough information to make a good decision. Why move from MML/TCP to VXML/HTTP? if you read my performance article on the resources page, XML is very CPU and memory intensive. Even with XStream java-xml binding library, handling moderate concurrent load will easily consume 100% of the CPU. if you're serious about using VXML/HTTP, you're going to have to use some kind of hardware acceleration to get wire speed. even without implementing a single line of code, I can tell you right now it is going to be really hard scaling VXML/HTTP to handle 50 concurrent streams. On a 3ghz cpu, 30 concurrent stream will consume 100% of the CPU. With a 2.4ghz CPU, 10-15 concurrent xml parsers will consume 60-75% of the CPU regardless of the platform. it doesn't matter if you write it in C#, C++, C or Java. I keep a pretty close watch on XML technology and most of the parsers today are about the same. the differences are not significant to me. given the primary limitation is CPU consumption, your options for application server should be based on reliability, maturity and toolset. If J2EE provides the features and scalability you need, then use it. If writing your own application server in C++ is the best option, then find two guys who have 10 years of experience writing high performance app servers. I would guess using servlets/jsp is the least of your concerns at this point. Until you have the VXML driver written and know it's performance characteristics, any decision on app server will be premature. You may want to look at Xml Pull Parser3, XStream, Jixb, XBis and the new xml stream parser. do a quick implementation for VXML and see what kind of performance you get. if that is sufficient, you can move on. otherwise, you may have to write a highly optimized VXML parser in C using a finite state machine of some sort. good luck peter On Thu, 8 Jul 2004 18:19:23 -0700, Darshan Rawal [EMAIL PROTECTED] wrote: We implement a Messaging PLatform for Voice Applications. Our current platform has 2 Major Components as shown below:- Media Server --- MML/TCP-IP --- Application Server. We want to move to a new architecture for our Application Server where the interaction between App Server and Media Server is as shown below(i.e. based on VXML over HTTP) Media Server --- VXML/HTTP --- Application Server. To achieve this goal we need to implement an VXML Document Server, which will generate VXML (Voice XML) pages based on the business logic Data access layers within the Application server. We have 2 options to implement the VXML document Server:- 1) USing the J2EE presentation tier(i.e. JSP Servlets) 2) Writing our own VXML generation module in C++ based(probably based on the JAVA Servlet paradigm). Our Factors while deciding between the 2 would be as follows:- 1) Performance(This is the key for us, we ezpecially need to know exactly how slow JAVA is as Compared to C++) 2) Integration with our Business logic Data Access Layers.(All our business logic data access is written in C/C++. Hence if we chose JAVA, then we might have to use JNI or make use of TCP/IP to communicate with our different Processes written in C/C++) 3) Ease of Future Change(We want to develop new Application Features Quickly fast). 4) Fast better Development environment. I am a proponent of the use of Tomcat Servlet Container and hence the use of J2EE front end for our module. But I was hoping to get some pointers from you all, just in case somebody might have implemented something similiar or might just have some inputs nevertheless. We are in a time crunch here, as we have to make a decision by next Monday end. Thanks in advance. Regards, Darshan. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Request for Inputs for a Design/Architecture for VXML Application Server
On Thu, 8 Jul 2004 18:59:23 -0700, Darshan Rawal [EMAIL PROTECTED] wrote: Hello Peter, I completely agree on this, but this interpretation of VXML pages is going to happen on the Media Server side. Hence as of now I am just concerned with generation of VXML pages and transporting them to the Media Server, where the CPU intensive XML parsing is going to happen. ok, so if I understand the setup correctly. the app server will only generate VXML and not consume it. If that is the case, CPU usage should not be an issue. For detailed information about XML generation performance, google for XML articles by Dennis Sosnoski. He has written up detailed articles in the past on xml parsers and xml binding parser. We are currently utilizing a 650MHz Sun Sparc Blade Solaris 8 Core OS, and it will remain the same for the next release, hence I know that CPU load will be a reasonable main concern. Having done stress testing with XML on Solaris systems: 280R, X1, UltraSparc5, as long as you're not consuming XML, CPU should not be an issue. Since XML is very verbose and Sun blades typically come with 2 or 4 ethernet ports, I would recommend separating the network traffic so the XML uses a dedicated router to the media server. Our entire Engineering team has reasonablly well knowledge of C++/C.(not so much of JAVA).Also, complete development is going to be done by our team. We are a part of the architecture team which has to decide the approach to take, and I was just looking for some pointer on that issue(especially related to the performance implications for JAVA). when you say performance implications, what do you mean? high concurrent load? memory usage? I/O? threading? the context plays a big factor in whether or not there will be any issues. An prototype of a VXML browser in the Media Server has already been implemeted. It does have some performance issues. But, as I said earlier this is not our main concern as of now. The questions that we have related to taking the approaches are as follows:- 1) Would JAVA be a performance hit in terms of CPU utilization(especially bcos of the JVM) ? Note that our VXML document server is not the one doing a lot of CPU intensive operations, it will be our legacy application written in C/C++. Also, we are not doing significant CPU intensive operation like math operations, number crunching, encryption, etc. Would JIT provide reasonable performance compared to C/C+, by probably consuming more memory(which we can spare) ? since the app server is just producing the content, my guess is it's unlikely there will be any issues beyond concurrency requirements. since it's media, I'm assuming the servlet will produce a stream of data. You're probably going to have to use some sort of stream conversion to reduce memory usage. Regardless of the storage medium, I wouldn't recommend using DOM and would strongly suggest using the new xml stream parser/api. 2) How involving is the work related to integrating a JSP/Servlet based front end to legacy C/C++ code using JNI(debugging, performance issues ??) and/or TCP/IP sockets(performance Hit) ? Thanks Regards, - Darshan. I have no experience in JNI beyond reading examples and looking at the API/specs, so I can't provide anything useful there. You definitely can get good performance with JNI. Many servlet and EJB containers use JNI to call native network libs. Resin, weblogic and websphere all use it, so JNI won't be an issue. I hope that helps. peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
interesting article
http://www.webperformanceinc.com/library/ServletReport/ I'd like to thank all the developers for working so hard to improve tomcat. peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [5.next] Summary
I see your back from vacation. I hope it was restful. I've started working on porting CLIPS 6.2 from C to Java. Once that is done, I will donate it back to CLIPS community (since it's public domain software), donate it as a project to apache and use it in RuleML development. JBoss should be able to use it too :) peter On Wed, 23 Jun 2004 20:02:04 +0200, Remy Maucherat [EMAIL PROTECTED] wrote: Remy Maucherat wrote: Remy Maucherat wrote: - Move to concrete classes for request/response, and remove all of the RTTI done in Catalina's pipeline Ok, I'm almost done with the first part of the refatoring (thanks to Eclipse 3) :) I think I'll commit when I'm done with the next two items, so that all the big core changes are done (for now ;) ). - Use Tomcat 4.0 beta valve pattern - Redo logging using commons-logging, and remove Logger I'll do on with the next item: - Fork commons-digester, to remove what we don't need (many fancy features provided by beanutils, and in the process remove dependencies on beanutils and collections) I'll use the o.a.t.util.digester package for this, and hopefully not too much refactoring will be needed. Since modeler needs digester, I'll do special prebuilding of the JAR to rename the imports (a bit like the prebuilding which is done for the servlet JSP JARs). I forgot about one nasty dependency chain, BTW: DPCB - pool - collections, which resides right now in the commons loader :( I'll save this for later ;) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Next release
Remy Maucherat [EMAIL PROTECTED] wrote: I feel like starting working on the possible new codebase for Tomcat, now that Tomcat 5 is more or less stabilized. I have a few obvious items, but while a lot could be done to make the code more modern, it wouldn't make Tomcat actually run better, so I favor changing things only if it brings something very tangible. Note: This time, I favor changing the API, as with TC 5, I think we got as far as we could without. Some of my first items would be: - refactor the request and response API using concrete classes (CoyoteRequest and CoyoteResponse) rather than interfaces, in order to simplify the code and lower the amount of RTTI and casts - the current valve design mirrors the filters, but is actually different in what it can do (now that filters can work in request dispatchers), so I don't see the point of using the same pattern anymore; using the model from Tomcat pre-4.0 (yes, I know, it's old ;) ) would lower the number method calls and reduce significantly the stack trace - more JMX, such as redoing the notification model (questionable, since the gain isn't too evident) - (definitely) flexible configuration persistence (instead of the hack that is currently used ;) ) - I like the nested containers, and I think the server.xml structure is ok: I don't quite see how to simplify that without taking away from the functionality (Craig deserves some credit for the design, which aged rather well) - no non blocking IO for me, but introducing blocking NIO could be good Have you looked at EmberIO for this? It might be possible to use it in a way that doesn't break compliance. I started to look at it and thought it might be feasible. Not sure yet, since I don't understand the code well. peter - Do you Yahoo!? Yahoo! Movies - Buy advance tickets for 'Shrek 2'
Re: (movie)
what does the mail header say? or maybe some one else already unsubscribed the address :) peter lin Remy Maucherat [EMAIL PROTECTED] wrote: Anyone could find a good explanation as to why unsubscribing the address didn't work ? (the list manager says [EMAIL PROTECTED] is not subscribed) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Win a $20,000 Career Makeover at Yahoo! HotJobs
Re: 5.0.21 ?
hurray!!! that is good to hear. peter Remy Maucherat [EMAIL PROTECTED] wrote: Remy Maucherat wrote: I did post a message on the PMC list asking for a final word on our binary dependencies (installer + JMX). I'm also wondering if it whould be a good idea to pick up the latest Jasper fixes in a 5.0.21 tag. I got approval from Geir (the PMC chair) to ship with MX4J and the installer. So there will be a new 5.0.21 tag today, with new binaries :) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! Finance Tax Center - File online. File on time.
Re: Post on the geronimo list
can't wait to read more FUD articles proclaiming they're the savor of the world. I think I'll keep plugging along until the rubber hits the tires and some real traction is observed. peter Remy Maucherat [EMAIL PROTECTED] wrote: I saw this nice post on the Geronimo list, and I feel insulted. Two full-blown articles on the topic is forthcoming in the coming weeks. I want to take the time and care necessary to convey my various points in the most positive and constructive way I can. I'll send a link to this list and to the Tomcat list when it's published. and JBoss does not embed Tomcat. They bolt it onto the side of their server--this is not the same thing. If they say it is, they are either misleading you or they are honestly wrong. I know enough about Marketecture to safely say this. I've also read their source code, and I know how it works. Again, watch for the articles ; ) As far as competitive advantages go, once people get a taste of Geronimo's configuration and distribution features, once they understand what IoC-3 Components and GBeans buy them, they're never going to want to develop for anything else. Order-of-magnitude performance gains not included, I think the JBoss server is in serious trouble and I would hate to be holding ten million dollars of debt on my shoulders riding in *that* boat. The configuration advantages alone will challenge the way we think about application development. Yeah, they set the bar really high around here. I got a couple of nosebleeds trying to get over it. I'm sure the JBoss guys'll start seeing similar problems when they sit down to implement JSR-77. I feel inflamatory articles, FUD, and nonsense is a very unproductive way to interact (given your latest fine article on the ASF, I see you have a habit of taking people hostage using that method), and, given that, my answer to any of your proposals is very simple: no (aka, -1). As for the apparently marvellous G experience, we'll see and let the market decide. Quite frankly, you sound like an Avalon fan turned bad. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! Finance Tax Center - File online. File on time.
Re: [5.0] Problems with the next release
I have to agree. the decision affects a lot of apache projects. I hope ASF board changes the policy slightly and lengths the time for this to take place. It's good to have Apache equivalents to many of the libraries being used in apache projects, but it's going to take time. I may have to create a SF project for the monitor plug, write a custom SAX documentHandler to parse the tomcat status, or assist the existing jaxb project on apache. on jmeter-dev we were discussing the possibility of creating a SF project to distribute versions that don't need manual downloads, but most likely that might not be feasible. I would hate to see jakarta projects fork, just so we can provide complete distributions. peter lin Remy Maucherat [EMAIL PROTECTED] wrote:Hi, There are some problems with the next release, with the decision from the ASF board to mandate that all ASF releases are to be made of 100% ASL 2.0 licensed components (as a side note, I'd like to add that this is obviously a terrible decision). This has many consequences and some questions marks are left (such as for the JCP provided elements we are using). With Tomcat 5.0.x, the most pressing problem is with JMX, since there are no ASL 2.0 licensed implementations available. The options seem to be: A) Ship Tomcat 5.0.20 without JMX, and have it display a message with instructions on how to install JMX if it's not present (basically, everywhere but on JDK 1.5.0). B) Ship the binaries from non ASF servers (we could setup a project for that on Sourceforge). The sources can be shipped from the ASF servers as before. It is unclear to me if we can legally call these binaries Apache Tomcat or not. Comments ? Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! Search - Find what youre looking for faster.
tomcat monitor - Just in case
in case some commiters don't subscribe to tomcat-user. I plan to post the prototype tomcat monitor tonight. I just ran a couple of tests simulating 60 threads hitting the homepage on TC5.1.18. the monitor correctly registered an increase in activity while the 60 threads were hitting the server and went back down once the threads were done. http://cvs.apache.org/~woolfel/prototype_screencap.gif I will include two test plans with the zip file for people to play with. the code still needs to be cleaned up, but so far it appears to work correctly. peter lin - Do you Yahoo!? Yahoo! Search - Find what youre looking for faster.
Re: http://www-106.ibm.com/developerworks/library/j-nioserver/
I would have to agree with Remy here. The example given doesn't really prove anything in my mind. Sure, having NIO will allow you to use a set number of threads to handle inbound requests. Even without running a benchmark, it is obvious a NIO approach can scale better. A better example of how to use NIO is SEDA, which uses a staged event driven architecture with NIO. For those who don't know SEDA, it can handle 10K connections and over 2K req/second (http://www.eecs.harvard.edu/~mdw/proj/seda/httpload-bench/index.html). I took a look a the source included in the article. the implementation is just a very simple test, it doesn't really do anything and is in no way comparable to Tomcat, which has to comply to the servlet spec. writing non-blocking code is sufficiently difficult that it increases the development time compared to the current servlet specification. the reason the servlet spec team chose a single threaded approach is the ease of development, not because they weren't aware of NIO. This isn't the first time the topic has come up. peter lin Remy Maucherat [EMAIL PROTECTED] wrote: Shapira, Yoav wrote: Howdy, Have people read this article? I find it interesting. If you haven't, it's a benchmark comparison of a java.nio-based server with tomcat 5. The benchmark server is tiny and contains only a small subset of functionality. I am also not sure of how representative of the real-world this benchmark is, because all clients are set to keep-alive. Is that typical? What else do people think about this article? Complete BS ;) The model will only work for select few applications. What if you have processing that requires 1s ? We're building an application server here (I hope). NB I/O vs B I/O is simply the same as comparing cooperative mutitasking with preemptive multitasking. The first one is way easier to implement in a very efficient way, but in practice there's a reason why everyone switched: it just doesn't work, except for special, controlled environments. IMO, the underlying OSes need to be fixed (Linux 2.6 does this, I believe). Note that this is not about classic I/O versus NIO: NIO can be used in either blocking or non blocking modes. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it!
[OT] Re: http://www-106.ibm.com/developerworks/library/j-nioserver/
I'm inclined to wait until JCP comes up with a good way to migrate/support non-blocking approach within a servlet container. Event driven technique isn't new by any measure and has been proven to scale well. But most cases actually do not require the added complexity. Unless an application has to support a large number of concurrent requests/users in the 5K+ range, it's usually not cost effective. take SEDA for example, it breaks request processing into stages, so that each stage is profiled. If a request is for the same resource, the queue handler can by-pass the database call and just return the result. If you read Matt's paper on SEDA, you'll see the design goal was to handle /. like effect where a large number of users are requesting the same resource. The approach is powerful, but it would be difficult for someone with only ASP and CGI experience. Ideally, an API that hides the complexity of call-backs, thread sync and gives the appearance of single threaded processing would make it easier to develop and debug. For the last year I've been working on projects that have to support major scalability. Educating the other developers about async processing has been sufficiently difficult and a headache. My biased perspective :) peter lin Reshat Sabiq [EMAIL PROTECTED] wrote: What would this benchmark look like if Tomcat also was configured to use a max of x threads, just like sse? If the difference was negligible/none, then IMHO NIO effect is no different than playing with max threads value. However, if there was still a considerable difference for heavy loads, i would be inclined to changing the API to make it compatible w/ both, so that the container could toggle between using IO and NIO based on a config, load, etc. My undeserved 2 cents. :) -- Sincerely,Reshat.---If you see my certificate with this message, you should be able to send me encrypted e-mail. Please consult your e-mail client for details if you would like to do that. ATTACHMENT part 2 application/x-pkcs7-signature name=smime.p7s - Do you Yahoo!? Yahoo! Finance: Get your refund fast by filing online
Re: [5.0] Three proposals
As usual, these types of comparisons aren't really useful or even desirable. I remember there used to be a benchmark JSR, but it was with drawn. one of these days, a standard set of web and ejb benchmark apps should be created. that way, instead of the usual simple jsp tests, the performance measurements would be more indicative of how a webapp might perform. from my own experience with coyote connector, it is equal to SunOne, orion, resin, and weblogic. this is based on real applications benchmarked on several servlet containers. Compared to Tomcat 3.2 and older, TC5 has made great strides and remy has worked his butt off. I won't bother pointing out flaws in other servlet containers, since it leads to flame wars. you can google for that information yourself. peter lin Remy Maucherat [EMAIL PROTECTED] wrote: Jess Holle wrote: Any and all performance improvements would be greatly appreciated. For those who have not seen it, Sun is touting their SunONE is better performed / more scalable than Apache 2 + Tomcat benchmark. While Tomcat and mod_jk[2]'s sole goal is not to scale and perform every bit as well as every other web server / servlet engine alternative and benchmarks are often full of lies, I think it is in everyone's best interest to keep an eye out for opportunities to ensure Tomcat 5 remains competetive in terms of performance and scalability. Good for them. However, I'd like to point out that mod_jk 2 is not the best for throughtput. The HTTP connector is. Performance wise, it can't really improve anymore, but I'll try to optimize memory usage to get more scalability (I think it's good enough right now, though). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! Hotjobs: Enter the Signing Bonus Sweepstakes
Re: Performance Recommendation
I would have to concur with Remy on this one. The performance benefit is minimal and depending on how your JSP is written possibly no benefit at all. I know for a fact there are sites handling 10million+ pageviews a day using Tomcat. This is commercial sites and not some development/demo site. The developers work very hard to make make Tomcat robust and scalable. There are only 2 servlet containers that I know of first hand with better performance. But the difference was minimal. One was resin and the other was orion on a real application. In the end the performance benefit 1-5% wasn't worth the cost of a license and resin doesn't adhere to the official specs as closely as tomcat. You're better off looking at your pages and overall design to figure architectural inefficiencies. good luck. peter Remy Maucherat [EMAIL PROTECTED] wrote: Roozbeh Zabihollahi wrote: Hi, I am tomcat user, as all of readers know, tomcat is very good Jsp Container for developping and not good for production mode (cause of its performance). I installed and run 'RexIPAppServer' ,that says it has best performance, and compare Servlets it generates for JSPs and Tomcat Generated Servlets. I see 'RexIPAppServer' construct static strings that JSP page use with Static Strings. same as: static final char[] _jspText_18=\ \r\n .toCharArray(); but Tomcat doesn't distinguish (or it not want to do) between Static and Dynamic Strings and print them in dynamic way in service method. I want to ask from Tomcat Developers, that eather Tomcat Developers does not want to implement this feature or not? There's a flag in Jasper now for that. Look in conf/web.xml. and, could you help me to develop this feature, because i think this is exponentialy improve performance. ^-^ Hmmm, well, it won't, sorry. 5% at most is what you should expect ;-) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? New Yahoo! Photos - easier uploading and sharing
Re: Performance Recommendation
to put things in better context. Most of my work experience has been building complex applications with web front ends. In my case, most of the content is dynamic and the static html is minimal. Therefore, my experience with regard to performance is biased by the type of applications I work on. In 90% of the cases where I saw a performance issue, it was the result of how data was being fetched and cached. when I worked on a wireless platform, 90% of the site produced pages using multiple data sources. Some were third party and some local. In some cases, the data was scraped from a HTML site and presented to a variety of devices using html, hdml, wml and xhtml. I don't remember the exact performance numbers, but the query times were more than 75% of the overall request processing time (not counting sending the response). Obviously in these types of systems, trying to optimize 5% for static strings would result in no benefit in performance or scalability. On the otherhand, if the jsp pages are mostly static, then you it probably would be worth it. I tend to leave this type of optimizations as the last item, after I've exhausted all other options. peter Roozbeh Zabihollahi [EMAIL PROTECTED] wrote: I Think there is misUnderstandings here, first i didnt said that tomcat works ONLY for 'some development/demo site'. but I mean WHY Tomcat must not use for production mode? , as Apache, That it the best known Web Server in the world, in performance, security and many other major factors. I ask this question or recommend this option cause of I Love Tomcat, and I work with it several years. second, I think, 5% performance upgrade is very good for a version update, and i think if we use static byte arrays instead of normal strings, in normal JSPs(because they have very much static strings) performace will be more increased than 5%. anyway, JSPs is very small part of Web Site System. Application Server is very bigger part of it. then optimizing servers always increasing performance better and more significant. but I know, WebApp architecture and Design is very very important for performance. Thanx for your Reply. Roozbeh - Do you Yahoo!? New Yahoo! Photos - easier uploading and sharing
Re: Need Consulting
hmm, very interesting. What other processes are running outside the VM? do you have process monitors, intrusion detection or other memory/IO intensive processes running on the same box? you can rule out native drivers as a possible cause. the VM still could be causing it. tomcat is most likely not the cause. that still leaves a lot of variables you have to track down. what are the exact system specs again? peter jean-frederic clere [EMAIL PROTECTED] wrote: Peter Lin wrote: as someone else mentioned, these types of seg fault errors have been the result of database drivers in the past. Specifically, using native drivers. I am not using any native drivers. I am aware of this type of behavior occuring with other drivers like JMS that wrap native code. Make sure your database driver isn't the cause. Try using a third party pure java jdbc driver and see if the same behavior occurs. As the error messages states, something outside the VM caused the seg fault. Not exactly, the segfault occurs in a routine outside the VM but it is hard to know what causes the memory corruption. good luck. peter - Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard
Re: Need Consulting
as someone else mentioned, these types of seg fault errors have been the result of database drivers in the past. Specifically, using native drivers. I am aware of this type of behavior occuring with other drivers like JMS that wrap native code. Make sure your database driver isn't the cause. Try using a third party pure java jdbc driver and see if the same behavior occurs. As the error messages states, something outside the VM caused the seg fault. good luck. peter jean-frederic clere [EMAIL PROTECTED] wrote: Clere, Jean-Frederic wrote: Jeff Rogers wrote: Hello, I hate to post this to the dev list, but we need consulting help. I have no idea where to look for reputable Tomcat consulting. We are at our wits end and ready to make the jump to commercial software due to a signal 11 crashing problem with our Tomcat servers. signal 11 are normaly problems in the JVM's. Which java version are you using? BTW: I have tried to overload TC and I have got: An unexpected exception has been detected in native code outside the VM. Unexpected Signal : 11 occurred at PC=0x40098aa6 Function name=malloc Library=/lib/libc.so.6 Current Java thread: at java.net.SocketInputStream.socketRead(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:85) at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:767) at org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:428) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:569) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:392) at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java +++ That is with JVM 1.3.1_03... and 4.1.24. - Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard
RE: Http Connector / Gzip
interesting results. By any chance did you compare it to Perl regexp to see the difference? :) peter --- Chad Johnson [EMAIL PROTECTED] wrote: Some regex benchmark's I ran across: http://tusker.org/regex/regex_benchmark.html -Chad Johnson -Original Message- From: Henri Gomez [mailto:[EMAIL PROTECTED] Sent: Monday, October 06, 2003 11:15 AM To: Tomcat Developers List Subject: Re: Http Connector / Gzip Remy Maucherat a écrit : Henri Gomez wrote: Hi to all, What about using regexp in HTTP 1.1 connector to include/exclude browser which could/couldn't use gzip compression ? May be it could be use also to drop to HTTP 1.0 browser which claims to be 1.1 compatible, but are not. CF: Apache HTTPD server Well, I thought regexp was rather slow in Java, so it could be a problem if using a generic regexp library. jakarta-regexp is too slow ? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] __ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [PATCH] Performance Problems in Coyote Chunking
maybe I'm missing something, but http1.0 doesn't support chunking. isn't it feasible to just make tomcat respond with http1.0 instead for this particular problem? I'm probably being naive here, but you could use the older connector instead. peter --- Remy Maucherat [EMAIL PROTECTED] wrote: Steve Appling wrote: The following patch combines the 3 packets that were generated for each chunk into just one packet. There are more optimizations elsewhere - I'll keep looking. Patch of org.apache.coyote.http11.filters.ChunkedOutputFilter from 4.1.27 src. The patch is bad right now, because an exception will likely occur if the buffer needs to grow (as it has no sink). But I get the idea. Remy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Coyote Performance Problems with HTTP1.1
I don't know about others, but my feeling is chunking is useful for large files and not necessarily where the size of the content is unknown at the beginning of the response. I seriously doubt a 2-5K static file would see a real benefit. We all know the internet has a ton of packet collision. therefore, sending non-chunked response over a slow connection would have a higher rate of failure. I haven't been on a modem in a long time, but if memory serves me correctly, downloading a 200K zip file on a modem tends to be faster with chunked encoding than non-chunked. Even with smart download, I remember chunked encoding being faster for large files. I could be wrong. peter --- 3 - When to chunk I thought that chunking was invented to handle serving up dynamically created content that did not have a size known in advance. I believe that on both IIS and Apache static content is not chunked. Is there any way for tomcat to behave similarly - could the default servlet do something to prevent the connector from chunking the data it serves up? If you made it this far, thanks for taking the time to read this and consider my questions. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software
Re: Object pooling performance
Costin Manolache [EMAIL PROTECTED] wrote: On the other hand, I've seen pages where JSP tag pooling was worse. That's why I think per page customization, and supporting per-thread pooling are very important. This is a fine-tune operation for pages that are frequently accessed, not a one size fit all problem. Costin I've also seen this happen for pages that only have a couple tags. from my own benchmarks, pages with less than 40 tags are better off not using tag pooling. the per page fine-tuning is a nice way to maximize performance :) peter - Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month!
Re: tracking memory usage over time
Is there any particular reason the pages are recompiled frequently? If you're using tomcat 4.1.12 or newer, it should use Ant to compile the pages, which should get around the issue of memory leak due to page compilation. peter --- Aditya [EMAIL PROTECTED] wrote: I have the following JSP that I hit every 5 minutes and stuff the returned values into a RRD (www.rrdtool.org) to measure the memory (heap I presume) consumption of Tomcat over time. Is there a better way, short of using JMX in the newer Tomcat builds, of doing this? %@ page language=java % %@ page session=false % % long free = java.lang.Runtime.getRuntime().freeMemory(); long total = java.lang.Runtime.getRuntime().totalMemory(); out.print(free + | + total + |); % I can see a clear leak (about 20 contexts with a dozen or so hit constantly and recompiling JSPs very often) which necessitates (-Xmx and -Xms set to 256 MB) a restart of Tomcat every 4 days or so (with 4.1.14). I just upgraded to 4.1.20 thinking that the constant compiling was the source of the leak and that doesn't seem to have made a difference. Running 4.1.14 under jprobe doesn't evidence any leaks in our JSPs/filters. Hints on how to trace this leak down would be most welcome. Thanks, Adi - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] __ Do you Yahoo!? Yahoo! Shopping - Send Flowers for Valentine's Day http://shopping.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [5.0.1-dev] Testing with JRockit 8
I just ran a half dozen benchmarks with bea JRockit jdk1.4.1 for linux. The performance was better until it locked up bad and required a hard reset. One thing I noticed is the memory usage was almost double and the CPU usage was also higher. The numbers were consistently better than both sun and ibm jdk, but on my 8th benchmark it locked up bad. I have no idea why, but it would seem like there is a stability issue with the linux version of JRockit. The system i'm running on is RedHat 8.0 on a AMD XP 2ghz and 1gig of DDR RAM. peter Remy Maucherat [EMAIL PROTECTED] wrote:For my book, I have been testing Tomcat with JRockit. I tested on a Win2k / Linux dual boot box, so it has the advantage of allowing to compare OSes :) The result: - on Windows, its performance was pretty much amazing (it *did* actually beat IBM on Linux, which was previously making Windows look really bad). - on Linux, it locks up the computer after a few requests and I have to kill -9 all Java threads individually (why ?). I don't think there's a problem with Tomcat which would cause that. For the record, I was using RH 8 with the latest kernel patch (the one Costin says is bad in his weblog ;-) ). The main disadvantage of JRockit is its insane memory usage (basically, it's 2* Sun Hotspot client; maybe there will be a use for those Athlon 64s even for small servers after all ;-) ). The other result is that Tomcat on Linux appears to run more reliably than Tomcat on Windows (except for the socket lingering bug that did cause problems), as on Windows, with all VMs, a few connections are rejected (while it never happens on Linux). I don't believe this behavior is caused by a Tomcat bug. Any comments, or anyone able to reproduce this ? Remy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! Shopping - Send Flowers for Valentine's Day
Re: Tag reuse performance [Was: Re: DO NOT REPLY [Bug 16001] - Tag.release() not invoked]
I haven't read all the posts on this discussion, but here's some facts from personal observations. for pages with only a few tags, ie less than 30, tag pooling doesn't help. On the otherhand, if your page has 100+ tags, it improves performance. Some of the pages I benchmarked with had about 135 tags. In those situations, I saw a 20-50% improvement. I would argue that sites that don't have a lot of load should simply turn off tag pooling. Site that use tags extensively and get 1millions page views a day, will gain significantly from tag pooling. peter lin Costin Manolache [EMAIL PROTECTED] wrote:Hans Bergsten wrote: Without pooling With pooling Reuse w/o overhead - 5 threads Avg.: 330 ms 349 ms N/A Rate: 15.2/sec 13.6/sec N/A 20 threads Avg.: 1,752 ms 1,446 ms 1,265 ms Rate: 12.1/sec 13.6/sec 14.7/sec To me, this indicates that if you can avoid _all_ reuse overhead, there's some performace to be gained from reuse but not much. With the From 1.2s to 1.7s there is about 35% difference. I would call this quite significant. Even between 1.4 and 1.7 - you have 20%. Try to increase the thread count to 100 - and you'll see this going up. The difference ( 0.5s ) is probably 2-3 times the response time of apache for a static page. And most users will feel it. I agree that in percentage, the difference is somewhat significant, but don't make too much out of the real value. My test server is not representative of the type of hardware you would use for a site with this type of load. On hardware suitable for the task, the difference in And the test page is not representative of the type of pages that will run on a real site. I know that. All we can measure with relative accuracy is the overhead of the container/jsp implementation - at least in relative terms. Take as the reference the time ( or RPS ) for Apache to serve the same output as a static page. Or the time a servlet will take to generate the same output. Run your tests with 5, 20, 100 RPS ( and ab may be a better driver ). Compare the results - and most likely a production server will see similar ratios. I'll try to find some time ( next week - I hope ) and run the same tests with the no sync pool. the real values will likely be a lot smaller, and IMHO, insignificant. But please, let's not start a long debate about what's significant or not (that depends on too many factors). All I'm trying to show with these simple tests is that for pooling to really make a difference at all, you need to avoid all overhead, which may be very hard, and that the overhead with current pooling seems to eat all potential gain. Well - it shows pretty clearly that the _current_ implementation of thread pool is broken. Even if we don't take sync into account, the pool has a 5 object limit - what else could you expect ?? I ran 10,000 requests for each test case after a manual warm up (just a few requests to give the JIT a chance to kick in). If I rerun the tests to capture GC data (as Glen was asking for), I can run a longer warm-up as well. I didn't record the max values, but IIRC they were around 100 sec in all cases. The 1.4 JIT takes some time to kick in, if you run batches of 1000 requests you'll see the time keeps improving. I would do at leat 5000 request to warm up the jit. This is a very good start, thanks for bringing this up. I hope it at least gives us a better idea about what types of gains we can realistically expect from tag handler reuse. Most of the improvements in coyote ( or in 3.3 over 3.2 ) are due to object reuse. It is possible that tag handlers are different and the other overheads will obscure any benefit ( at least under low load ), but I can bet that under heavy load recycling will be very significant, if done correctly. Costin -- To unsubscribe, e-mail: For additional commands, e-mail: - Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now
Re: Tag reuse performance [Was: Re: DO NOT REPLY [Bug 16001] - Tag.release() not invoked]
these were all JSTL tags. Back when I ran the tests, I posted some of the results. I did tests that were synthetic, ie out 100 JSTL out tags in one page. Others were based on an actual page layout with lots of markup logic that use jstl c:choose in conjunction with jslt xml tags. the tests were with tomcat 4.1's jasper2 and with 4.0x jasper1. obviously the tag pooling was only with jasper2. I didn't have time to test tomcat 3.x tag pooling. peter lin Costin Manolache [EMAIL PROTECTED] wrote:Peter Lin wrote: I haven't read all the posts on this discussion, but here's some facts from personal observations. for pages with only a few tags, ie less than 30, tag pooling doesn't help. On the otherhand, if your page has 100+ tags, it improves performance. Some of the pages I benchmarked with had about 135 tags. In those situations, I saw a 20-50% improvement. I would argue that sites that don't have a lot of load should simply turn off tag pooling. Site that use tags extensively and get 1millions page views a day, will gain significantly from tag pooling. Is this based on the current tag pool implementation in jasper2 ? Because it is pretty clear that the tag pool has few problems. I would say the nature of the tags will also have a big impact. If your tag is very simple - you'll probably get some small benefits under load ( 20..30% ?). If the tag uses internal data structures, buffers, etc - it's very likely you'll see more ( since creating each tag instance will also create the additional hashtable, StringBuffers, etc ). I would bet that with complex tags that are specifically written to take advantage of the recycling you would see at least 2x better performance ( with a good sync-free and large enough tag pool ). If your tag is using any buffers or complex/expensive data structures that can be recycled - you'll save a lot. I don't think the number of tags in a page is too important - even if you have 1 complex tag - with 100 concurent users - you should see a difference. In an ideal world, all core tags would be recyclable and garbage-free - that may allow them to run at comparable speed with a hard-coded page. Costin - Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now
Re: Thread priority
My 2 cents. I rather a server deny connections and keep on running than accept connections until it dies :) peter lin Remy Maucherat [EMAIL PROTECTED] wrote:Glenn Nielsen wrote: Is there a reported bug you are trying to address? Not really. The test here reported a lot of connection failures: http://webperformanceinc.com/library/ServletReport/ Tomcat 4.1.17 should improve the results over 4.1.12 (it's faster and would probably lower slightly GC), but I think the number of errors was higher than it should have been. This could be caused by GC, or possibly because the accepting thread couldn't get the job done, or some other unknown factor. I have not noticed any problems related to thread scheduling with the medium volume site I administer. It is possible that this change could cause threads other than the socket accept thread to be starved. I would be -1 for making this change unless there is a significant reported bug that these changes would address. If it ain't broke, don't fix it. ;-) Remy -- To unsubscribe, e-mail: For additional commands, e-mail: - Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now
Re: [4.1.17] Tag soon ?
I wasn't able to run the additional benchmarks like I wanted, since the cpu fan on my workstation is dying and causing BSOD. I'm in the process of setting up my X1, so I will re-run the benchmarks on that server and post the results. peter lin Remy Maucherat [EMAIL PROTECTED] wrote:I don't see any major issues with 4.1.16, so I plan to tag 4.1.17 (which hopefully would be the next stable build) tomorrow after a few more minor tweaks. There was a report made against JK 2 immediately after 4.1.16 Beta was announced. Can someone confirm there's no major issue with JK ? Remy -- To unsubscribe, e-mail: For additional commands, e-mail: - Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now
Re: [4.1.16] Benchmarks
yeah, I can do that on a simple set this weekend and on a large webapp next week :) peter Remy Maucherat wrote: Could someone (Peter ?) do some quick benchmarks (with the HTTP/1.1 connector preferably) comparing 4.1.12 to 4.1.16 ? I'd like to see if my OptimizeIt work is translating into something in the real world, and unfortunately, I only have access to one computer at the moment (so I can't do any reliable benchmarking). Thanks, Remy -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: [Jasper2] framework for tag optimization
Yeah, I actually use that in some places, but it is a bit harder to read with pages that have a lot of tags. Actually, the whole page is tags with very little HTML and everything that is text is in resource bundles. Using that syntax doesn't really bother in when used sparsely, but with hunderds of JSP, it gets a bit gruesome as time goes on and pages change. Craig R. McClanahan [EMAIL PROTECTED] wrote: One textual approach to minimizing the extra newlines (I would not recommend this -- they don't bother me, but they might bother you): c:choosec:when ... /c:whenc:when ... /c:whenc:otherwise ... /c:otherwise/c:choose Craig I almost prefer patching JSPC or writing a pre-processor for JSPC than use tricks to get around extra \r\n. peter - Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now
RE: [Jasper2] framework for tag optimization
yeah, it definitely would risk breaking conformance, unless it became an official
spec. there is a need for a tag-like markup for designers (non-programmers), but
performs better for sites that get million hits a day or more. Most sites wouldn't
need it, but for heavy hitters it would save in resources.
peter lin
Peter Romianowski [EMAIL PROTECTED] wrote:Just a thought: Doesn't this breack the
standard conformance
tomcat is all about? One could easily be bound to tomcat after
writing some plugins that break the initial semantics of a tag.
I think there is no way to assure the correctness of a plugin-
implementation, so it would become impossible to switch to
another servlet-container. And I saw many improvement-discussion
on this list, which had been canceled with exact this reason.
But having plugins for standard tags (JSTL) out of the box
would be great in respect of performance.
Just my $0.02
Peter
-Original Message-
From: Peter Lin [mailto:[EMAIL PROTECTED]]
Sent: Saturday, November 23, 2002 4:59 AM
To: Tomcat Developers List; Kin-Man Chung
Subject: Re: [Jasper2] framework for tag optimization
hey kin-man,
that sounds great! I was actually thinking along those lines
a while back, but thought it was impracticle because the
project I was working one didn't have enough time to explore
that approach.
when I was doing performance analysis of jasper1 with jslt
and saw how bad the performance was (due to the nested
try/catch bug), I went through and manually wrote scriplet
code to do the same exact logic. The performance compared to
jasper1 + jstl was tremendous. I had full mockups of a 3
pages written in JSTL and pure scriplet.
If memory serves me correctly, the difference was 5-8x. the
JSTL version using jasper1 would take 900-1000ms+ to display
15 results. The same exact page using scriplet took about
100-150ms. I would definitely be interested in spending time
on this and assisting. I may have some time opening up next
year, so I hope to start contributing actively :) cross my fingers.
peter lin
Kin-Man Chung wrote:I am
designing a framework in Jasper for enabling plugins that
work closely with Jasper to generate Java codes instead of
calls to tag handlers. The main idea is to take take JSTL
tags, such as
${i}
and generates the Java codes
for (int i = 0; i = 100; i++) {
pageContext.setAttribute(i, String.valueOf(i));
out.print(evaluate(${i}));
}
or even
for (int i = 0; i = 100; i++) {
out.print(i);
}
The design is not to do the actual optimization in Jasper,
but to provide a framework for taglib writers to develop
plugins to Jasper that will do the actual optimization.
Eventually, Jasper will be bundled with 1 or 2 plugins for
JSTL, as test cases for the framework and as examples for
writing the plugins.
The plugins are specified in a xml file:
the name of the tag class
the name of the pkugin class
There are currently 3 interfaces:
TagPluginFactory
Used for creating a TagPlugin.
TagPlugin
Created at code generation time for a specific tag
invokation. Used by Jasper to generate java codes.
TagPlugContext
Created by Japser and used by the plugin to query properties
of the current tag, and to use resources in Jasper.
This work is at the very early stage of the design, and is
purely experimental. I'll be checking in sources for this
work, and they should not affect the other part of Jasper,
when plugins are not turned on.
I welcome comments and suggestions.
--
To unsubscribe, e-mail:
For additional commands, e-mail:
-
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now
--
To unsubscribe, e-mail:
For additional commands, e-mail:
-
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now
RE: [Jasper2] framework for tag optimization
So it would seam having a plugin framework wouldn't necessarily break conformance. On the otherhand, I would still like to see the expert group to seriously consider making plugin framework a standard feature of JSP compiler. Beyond performance, there are other good reasons to do so. For one, JSTL pages generate a lot of out.print(\r\n) when lt;c:choosegt; statemetns are used. Rather than have a tag handle the body and trim, I would prefer to see a plugin framework that gives application developers the ability to strip out extra control linefeeds or spaces. For example, some one may want to format their jsp code so it is easier to read, but often it breaks HTML formatting. There should be an easy way for page developers to write basic filters for jasper that allows a plugin to strip tabs and control linefeeds. Not only would this reduce the size of the generated page, but it should provide significant improvement for pages with lots and lots of tags. peter lin Craig R. McClanahan [EMAIL PROTECTED] wrote:Building the pluggability framework pretty much has to be done by the Jasper developers. Implementing plugins for a particular type of optimization (such as a JSTL recognizer/optimizer) could be done by Jasper developers or by anyone else. The way to ensure correctness of a Jasper+Plugin (of any type) is to ensure that it still passes the TCKs for the APIs it claims to implement (JSP x.y of course, and JSTL if we're talking about that particular kind of a plugin). There's really two proposals -- a pluggability interface into Jasper's page compiler, and a specific use of this interface for optimizing the code generated for JSTL tags. The pluggability would be available to anyone using Jasper (and, in the past at least, that includes more than a few J2EE platform vendors). None of this is likely to be done by typical application developers -- it's much more likely to be container developers that use it. On the other hand, I wonder if people running Struts-based apps on Tomcat would enjoy it if we did a pluggable optimizer for the Struts tags ... - Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now
RE: [Jasper2] framework for tag optimization
I sent the suggestion a while back to the expert group. AFAIK, the and other tags don't actually generate *any* output of their own -- the extra line breaks you are seeing are undoubtedly those you've put in the source JSP page itself. Yes, I was referring to the spaces that are generated by jasper because my original JSP source has: c:choose c:when/c:when c:otherwise/c:otherwise /c:choose Can't you do this with a standard javax.servlet.Filter, without needing to integrate it into a compiler? A custom BodyTag of your own could certainly do the job as well. I've considered it, but again it seems like a band-aid solution. It's a bit wasteful to have the servlet container generate out.print(\r\n) and then to have a filter strip it out. It might not matter for small sites. but if your website gets 1million+ hits a day, it adds up. It matters even more if other parts of your application are heavy weight and the server has to support a high number of concurrent requests. In my hand tweaked tests, it resulted in 10-20% performance improvement. Obviously this could easily be solved by putting c:choose statements all in one line, but that makes it a pain to read and debug. The test I ran had 400+ extra \r\n in the tag version vs hand tweaked. when the project requirements state you have to generate a complete page of results within 250ms and it takes 200ms to get XML from an app server, every bit helps :) Most projects/sites do not have these kids of performance requirements, but it would make life easier. I also considered writing a class to parse my JSP before calling JSPC, so that's another approach that wouldn't break spec compliance. peter - Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now
Re: [Jasper2] framework for tag optimization
hey kin-man,
that sounds great! I was actually thinking along those lines a while back, but
thought it was impracticle because the project I was working one didn't have enough
time to explore that approach.
when I was doing performance analysis of jasper1 with jslt and saw how bad the
performance was (due to the nested try/catch bug), I went through and manually wrote
scriplet code to do the same exact logic. The performance compared to jasper1 + jstl
was tremendous. I had full mockups of a 3 pages written in JSTL and pure scriplet.
If memory serves me correctly, the difference was 5-8x. the JSTL version using jasper1
would take 900-1000ms+ to display 15 results. The same exact page using scriplet took
about 100-150ms. I would definitely be interested in spending time on this and
assisting. I may have some time opening up next year, so I hope to start contributing
actively :)
cross my fingers.
peter lin
Kin-Man Chung [EMAIL PROTECTED] wrote:I am designing a framework in Jasper
for enabling plugins that
work closely with Jasper to generate Java codes instead of calls
to tag handlers. The main idea is to take take JSTL tags, such as
${i}
and generates the Java codes
for (int i = 0; i = 100; i++) {
pageContext.setAttribute(i, String.valueOf(i));
out.print(evaluate(${i}));
}
or even
for (int i = 0; i = 100; i++) {
out.print(i);
}
The design is not to do the actual optimization in Jasper, but to
provide a framework for taglib writers to develop plugins to Jasper
that will do the actual optimization. Eventually, Jasper will be
bundled with 1 or 2 plugins for JSTL, as test cases for the framework
and as examples for writing the plugins.
The plugins are specified in a xml file:
the name of the tag class
the name of the pkugin class
There are currently 3 interfaces:
TagPluginFactory
Used for creating a TagPlugin.
TagPlugin
Created at code generation time for a specific tag invokation. Used
by Jasper to generate java codes.
TagPlugContext
Created by Japser and used by the plugin to query properties of
the current tag, and to use resources in Jasper.
This work is at the very early stage of the design, and is purely
experimental. I'll be checking in sources for this work, and they
should not affect the other part of Jasper, when plugins are not
turned on.
I welcome comments and suggestions.
--
To unsubscribe, e-mail:
For additional commands, e-mail:
-
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now
Re: Javac memory leak
that is unusual. I've done baseline performance tests with the standard examples and haven't seen that kind of behavior with 4.1.10-12. Can you start tomcat with verbose:gc and see what kind of output you're getting. Just because the memory allocated to Java is 58megs at the end, it doesn't mean it's actually using all of it actively. getting the GC output will tell you how much memory is being used, when GC runs and whether or not a fullGC ran. in past experience with 4.0.4, after loading all the example pages tomcat's memory allocation is around 45megs. peter John Trollinger wrote: Ok, I did some testing running 4.1.14 as a service... Now my knowledge of memory management is not so good so this might mean nothing. When running as a service if I goto every example jsp page (the examples that come with the install) Compiled Memory starts at 37872 and ends at 39048 Not compiled Memory starts at 39635 and ends at 58760 Does this not look like a memory leak in the jsp compile code?? Thanks, John -Original Message- From: Holger Brozio [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 12:14 PM To: Tomcat Developers List Subject: Re: Javac memory leak I also have no memory leak problems now with Tomcat 4.1.12. But i'm using jikes instead of javac as jsp page compiler. Whats about switching to jikes. If the memory leak problem is gone with jikes, javac still leaks memory, otherwise it is a problem in the jsp pages. Cheers Holger - Original Message - From: John Trollinger [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, November 20, 2002 3:39 PM Subject: Javac memory leak Does anyone know if the javac memory leak still exists (1.4.1 docs say it does not). I have a test the goes through a bunch of jsp pages and if they are not precompiled I get a out of memory exception. Thanks, John -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:tomcat-dev- [EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Javac memory leak
Just to double check against tomcat 4.1.12, the memory usage seems normal to me John. After tomcat starts up, it is using 23megs. My webapp loads when tomcat starts up, so your numbers may be lower. The system has 256megs of memory and 750mhz cpu. after I I hit all the pages in /examples/jsp/ directory, the memory allocated to java is 40megs. I even ran it with OptimizeIt and I don't see any memory leaks. peter peter lin wrote: that is unusual. I've done baseline performance tests with the standard examples and haven't seen that kind of behavior with 4.1.10-12. Can you start tomcat with verbose:gc and see what kind of output you're getting. Just because the memory allocated to Java is 58megs at the end, it doesn't mean it's actually using all of it actively. getting the GC output will tell you how much memory is being used, when GC runs and whether or not a fullGC ran. in past experience with 4.0.4, after loading all the example pages tomcat's memory allocation is around 45megs. peter John Trollinger wrote: Ok, I did some testing running 4.1.14 as a service... Now my knowledge of memory management is not so good so this might mean nothing. When running as a service if I goto every example jsp page (the examples that come with the install) Compiled Memory starts at 37872 and ends at 39048 Not compiled Memory starts at 39635 and ends at 58760 Does this not look like a memory leak in the jsp compile code?? Thanks, John -Original Message- From: Holger Brozio [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 12:14 PM To: Tomcat Developers List Subject: Re: Javac memory leak I also have no memory leak problems now with Tomcat 4.1.12. But i'm using jikes instead of javac as jsp page compiler. Whats about switching to jikes. If the memory leak problem is gone with jikes, javac still leaks memory, otherwise it is a problem in the jsp pages. Cheers Holger - Original Message - From: John Trollinger [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, November 20, 2002 3:39 PM Subject: Javac memory leak Does anyone know if the javac memory leak still exists (1.4.1 docs say it does not). I have a test the goes through a bunch of jsp pages and if they are not precompiled I get a out of memory exception. Thanks, John -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:tomcat-dev- [EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Javac memory leak
This is from the examples jsp right? or are they from your webapp? the latest optimizeit has a leak finder feature. you may want to download the trail version and give it a shot. from the data it looks like a lot of objects are promoted to old memory from heap. It doesn't necessarily constitute a memory leak though. if you use a lot of static variables, that could easily account for objects getting promoted to old memory peter John Trollinger wrote: When I run the examples with -verbose:gc this is what I get (just the snippits at the end) From uncompiled jsp pages [GC 14707K-13665K(28412K), 0.0059002 secs] [GC 15521K-15518K(28412K), 0.0128997 secs] [GC 17374K-16086K(28412K), 0.0092266 secs] [GC 17942K-16258K(28412K), 0.0086307 secs] [GC 18114K-16213K(28412K), 0.0046327 secs] [GC 18069K-17934K(28412K), 0.0114056 secs] [GC 19790K-19672K(28412K), 0.0155838 secs] Did you see me on the stderr window? Did you see me on the browser window as well? [GC 21520K-20092K(28412K), 0.0094920 secs] [GC 21947K-20095K(28412K), 0.0075890 secs] [GC 21951K-21758K(28412K), 0.0131385 secs] [GC 23614K-23528K(28412K), 0.0137372 secs] [GC 25384K-23769K(28412K), 0.0079125 secs] [GC 25625K-24336K(28412K), 0.0073652 secs] [GC 26192K-26170K(28412K), 0.0153369 secs] [GC 28026K-27460K(29436K), 0.0190963 secs] [Full GC 27460K-15538K(29436K), 0.2839948 secs] [GC 17394K-15720K(28412K), 0.0060262 secs] [GC 17575K-15746K(28412K), 0.0046218 secs] [GC 17602K-15745K(28412K), 0.0027241 secs] [GC 17601K-15745K(28412K), 0.0025604 secs] [GC 17601K-15746K(28412K), 0.0026679 secs] Stopping service Tomcat-Standalone From compiled jsp pages [GC 11356K-10577K(14028K), 0.0061505 secs] [GC 11532K-11034K(14028K), 0.0046978 secs] [GC 11994K-11305K(14028K), 0.0060575 secs] [GC 12265K-11341K(14028K), 0.0059876 secs] [GC 12297K-11396K(14028K), 0.0065578 secs] [GC 12356K-11538K(14028K), 0.0063572 secs] [GC 12498K-11598K(14028K), 0.0040036 secs] [GC 12558K-11965K(14028K), 0.0052529 secs] Did you see me on the stderr window? Did you see me on the browser window as well? [GC 12925K-12020K(14028K), 0.0046872 secs] Stopping service Tomcat-Standalone As you can see there is about a 4 meg diff between the two This is using j2sdk 1.4.1 on windows XP with 512 ram using tomcat 4.1.14LE -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Javac memory leak
I haven't seen the memory leak on solaris or windows, but isn't the leak only on linux? I thought jasper2 fixes the problem with com.sun.tools.javac.Main since it uses the system native javac? peter John Trollinger wrote: Does anyone know if the javac memory leak still exists (1.4.1 docs say it does not). I have a test the goes through a bunch of jsp pages and if they are not precompiled I get a out of memory exception. Thanks, John -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Javac memory leak
Hmm, that sounds bizzare. I'm been using/testing 4.1.10-4.1.12 on both solaris and windows with jdk1.4.1. I've ran several stress test using JMeter simulating light to medium (64 concurrent connections) load for 1-10K requests without any problems. My pages are heavy on JSTL, so they are fairly heavy weight. Even when I run stress tests with 5K unique requests for 10K+ requests, I haven't seen memory leaks. the number of JSP range in 100-200 range. peter John Trollinger wrote: We are using Tomcat 4.1.12 on windows and when running an automated test that navigates through a set of about 75 jsp pages (they have includes in them that kick of more compiles) it gets a OutOfMemoryException halfway through the compile. Our system is quite complex so I don't know if I can get an example of this.. John -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Javac memory leak
by any chance do you have multiple webapps or just one? in my case, I only have one webapp which has some services it loads when tomcat starts. All of my pages are dynamic with all the text in resourceBundles. The dynamic data is retrieved from a middle layer in XML format. JSTL is used to get text from resourceBundles and parse the XML. I'm on win2K also. peter John Trollinger wrote: If the pages are precompiled I do not have any problems at all, it is only when the jsp cache has been deleted that this shows up. John -Original Message- From: peter lin [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 9:57 AM To: Tomcat Developers List Subject: Re: Javac memory leak Hmm, that sounds bizzare. I'm been using/testing 4.1.10-4.1.12 on both solaris and windows with jdk1.4.1. I've ran several stress test using JMeter simulating light to medium (64 concurrent connections) load for 1-10K requests without any problems. My pages are heavy on JSTL, so they are fairly heavy weight. Even when I run stress tests with 5K unique requests for 10K+ requests, I haven't seen memory leaks. the number of JSP range in 100-200 range. peter John Trollinger wrote: We are using Tomcat 4.1.12 on windows and when running an automated test that navigates through a set of about 75 jsp pages (they have includes in them that kick of more compiles) it gets a OutOfMemoryException halfway through the compile. Our system is quite complex so I don't know if I can get an example of this.. John -- To unsubscribe, e-mail: mailto:tomcat-dev- [EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Javac memory leak
John Trollinger wrote: We have 2, one is webdav and the other is our actual application. We use a lot of custom tags and a lot of both types of includes (when I say a lot we have jsp pages that include over 500 other jsp pages, and no, this is not my design :) ) I'm guessing there's some memory leak in one of the custom tags some where. early in the development of the project I did have memory leaks, but it was a bug in our code. Once I looked at the number of threads running and the GC output, it was obvious our custom tag was the cause. In my particular case, the connection to the middle layer wasn't getting garbaged immediately. Once I fixed that, my memory leak went away. could be something as simple as a tag.release() not releasing correctly :) peter -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Javac memory leak
you never know. if it's a slow leak, precompiled pages may not exhibit the leak. I only discovered the leak in our custom tag when I put the app under moderate/medium load. Under light load the bug wasn't apparent. I'm guessing if you hit each page individuall slowly, the bug doesn't appear. If that's the case, a slow leak may be cause. in my case a slow leak + load was required to expose the bug. peter John Trollinger wrote: But that would not account for the leak only occuring when pages need a compile. If the pages are compiled I never see the leak. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Personal Oracle9i and Tomcat 1.4
Oracle comes with it's own JDK and sets some environment variables. if you want both to coexist, hardcode the JDK in your catalina script. when you uninstall, it reverts the environment settings and fixes your tomcat problem. usually, that's the cause. peter [EMAIL PROTECTED] wrote: Hi guys, I had my Tomcat running well until the day I installed Personal Oracle9i on my machine. After that, I can not make Tomcat to run. When I try to start it, the startup window opens for a fraction of a second and then closes down without displaying any errors. There is nothing in the logs as well. I deinstalled Oracle and it started working again. What's happening here? Thanks Rafi PS: I installed Oracle without the HTTP server option. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: [4.1.15] Jasper flushing, tagging ?
thanks remy for the information that helps. What if I decide to buffer the response myself in a response wrapper? Does catalina try to call clearBuffer, reset, resetBuffer, flush or flushBuffer in the event a request is passed to my errorpage? Also, does the status code change and the exception set in the request object with request.setAttribute(Throwable) if html has already been sent to the browser? In cases where the exception occurs before any content is sent to the browser, it all works fine. I need to ensure exceptions display my custom errorpage, but I'd rather not set the page buffer to an arbitrarily large 64K or 100K. peter lin Remy Maucherat wrote: peter lin wrote: On a related note to flushing. I've discovered a bug, but don't want to file one until I know for sure it's not a duplicate of the other flush bugs already in bugzilla. The bug is the following: if a page throws an exception and an errorPage directive is set, the resulting error page will print out anything left over from the parent page and the errorpage. That means and everything preceding the exception is written to the browser. One way around this I found was to set the buffer to a large size, like buffer=64K. To reproduce the bug: create a page with some html (has to be more than the default buffer size), towards the bottom throw an exception. create a custom error page. using a browser load the page and the resulting out put should contain html from both pages. This bug can be reproduced with autFlush set to true and false. If people are busy, I'm willing to spend a couple hours going through the code to find a fix and submit a patch. If it's not a bug, any pointers would be appreciated. If the buffer size is exceeded, then some data (the top of your page) will get sent to the client. After that, when the exception occurs, there's no way to get the data back (obviously). The servlet container error handling will not do anything in that case (error pages and status reports will not be processed), while Jasper also attempts to include the error page if forwarding to it failed. IMO, that's not that great an idea because it creates messy output, and the inclusion shouldn't happen (the error should be logged and ignored). It's been like that for a while, AFAIK. Rémy -- To unsubscribe, e-mail: mailto:tomcat-dev-unsubscribe;jakarta.apache.org For additional commands, e-mail: mailto:tomcat-dev-help;jakarta.apache.org -- To unsubscribe, e-mail: mailto:tomcat-dev-unsubscribe;jakarta.apache.org For additional commands, e-mail: mailto:tomcat-dev-help;jakarta.apache.org
Re: [4.1.15] Jasper flushing, tagging ?
thanks again for the information. I will go with setting the buffer then. peter Remy Maucherat wrote: peter lin wrote: thanks remy for the information that helps. What if I decide to buffer the response myself in a response wrapper? Does catalina try to call clearBuffer, reset, resetBuffer, flush or flushBuffer in the event a request is passed to my errorpage? Also, does the status code change and the exception set in the request object with request.setAttribute(Throwable) if html has already been sent to the browser? If you buffer everything, then Jasper will do a forward instead of an include. In cases where the exception occurs before any content is sent to the browser, it all works fine. I need to ensure exceptions display my custom errorpage, but I'd rather not set the page buffer to an arbitrarily large 64K or 100K. I think you should, since in the end it should be the same (something will need a buffer to hold your bytes, right ?). If I were you, I'd try to set a big servlet buffer size rather than a big JSP page buffer size (that doesn't run into the issues a servlet like Jasper has when trying to recycle things while being thread safe, so it would be more efficient; plus it will compute the content-length now). Rémy -- To unsubscribe, e-mail: mailto:tomcat-dev-unsubscribe;jakarta.apache.org For additional commands, e-mail: mailto:tomcat-dev-help;jakarta.apache.org -- To unsubscribe, e-mail: mailto:tomcat-dev-unsubscribe;jakarta.apache.org For additional commands, e-mail: mailto:tomcat-dev-help;jakarta.apache.org
Re: JSPC problems with webapp option
Although I'm just a user, I would like to warnings also. On a related note, --compile isn't covered in the documentation. I stumbled across it when I look at the source code a month back. It might be nice to output the warnings to a file for those who have large webapps with thousands of jsp files. peter Eriksson, Michael wrote: Hi Remy, being another toyer I believe your second point is covered by the --compile flag. What I personally lack is the possibility to get warnings when compiling. (As opposed to errors only.) Grüsse, Michael -- To unsubscribe, e-mail: mailto:tomcat-dev-unsubscribe;jakarta.apache.org For additional commands, e-mail: mailto:tomcat-dev-help;jakarta.apache.org
[possible bug] -- Re: costin: fix reverted
On a related note, i just discovered some behavior that may be a bug. In my case, I have a request filter that buffers the output, so that I can gzip it and time the internal time. when a page hits an exception, it correctly forwards to my error page. but what ends up happening is the buffer isn't flushed, therefore the output has unwanted data. According to the spec, reset is supposed to be called on ServletOutputStream, but when I print out debug messages, I never see it call reset() or resetBuffer() in my output wrapper. It's unclear to me if this part of the spec applies to requests that are forwarded. the relevant section from the spec is below. 5.1 -- The reset method clears data in the buffer when the response is not committed. Headers and status codes set by the servlet prior to the reset call must be cleared as well. The resetBuffer method clears content in the buffer if the response is not committed without clearing the headers and status code. does someone know what the correct behavior should be? peter lin Costin Manolache wrote: Remy Maucherat wrote: Can you point me to the code doing this extra processing ? I'm a bit confused: - for jsps ( with flush-in-release ) that just can't work, since the flush is already done by the jsp page. - the comment ( or the message ) in forward is probably wrong. - my bigger question is when is the response not a facade ? My understanding is that facades are used all the time, so the second part of the if will never happen That code was already there before the facades. I don't think it is used anymore, but would leave it anyway in 4.1.x (it could be removed in 5). I already reverted the patch in 4.1.x - it's too dangerous for the stable tree. So you are saying that forward() used to do a flush, but it was changed when facades where added to just set the flag - to improve the error handling. Sounds consistent with the behavior we have now. Could you add a little comment ( or at least remove the debug message that says 'flushing' ) ? The new behavior ( no real flush ) seems correct. Costin I do see your point - if forward() flushes then the extra error processing is broken. That's another argument to not do the flush() in release() :-) And it does explain who is generating the stack trace. The error page and status report processing is in ErrorReport/DispatcherValve. That's where the stack trace is output unless the response has been committed (by flushing usually). I have a feeling we just need to clear the error flags after the jsp error page - I'm pretty sure the jsp error page was executed and the data is in the out buffer. Somehow the servlet error processing kicks in and overrides the jsp error page. Yes, that's likely the right explanation. Remy -- To unsubscribe, e-mail: mailto:tomcat-dev-unsubscribe;jakarta.apache.org For additional commands, e-mail: mailto:tomcat-dev-help;jakarta.apache.org -- To unsubscribe, e-mail: mailto:tomcat-dev-unsubscribe;jakarta.apache.org For additional commands, e-mail: mailto:tomcat-dev-help;jakarta.apache.org
AccessLog weirdness
I am experiencing some weird behavior with tomcat 4.1.x, but I don't know if it's a bug or not. What's happening is this. My setup is the following: solaris 8 win2K jdk 1.4.1 tomcat 4.1.12 standalone tomcat w/o any other server. I have a page which uses iframe/iframe to load some banner (evil I know). What ends up happening is two entries are logged in the access log. If I comment out the iframe/iframe only one entry is logged in access log. in the log I see an entry: w/o iframe banner --- ip - - [04/Nov/2002:12:08:38 -0500] GET /logtest.jsp HTTP/1.0 200 3938 - Mozilla/4.7 [en] (WinNT; U) w/ iframe banner --- ip - - [04/Nov/2002:12:13:05 -0500] GET /logtest.jsp HTTP/1.0 200 4039 - Mozilla/4.7 [en] (WinNT; U) ip - - [04/Nov/2002:12:13:05 -0500] GET /logtest.jsp HTTP/1.0 200 4039 http://darknight.gte.com:9080/logtest.jsp; Mozilla/4.7 [en] (WinNT; U) Now it gets more complicated. If I use Mozilla, or IE6, this behavior is not present and the access log only has one entry per page view. Is this a HTTP1.0 spec, netscape 4.7 or tomcat issue/bug? Thanks in advance to anyone that provides a clue. peter lin -- To unsubscribe, e-mail: mailto:tomcat-dev-unsubscribe;jakarta.apache.org For additional commands, e-mail: mailto:tomcat-dev-help;jakarta.apache.org
Re: DO NOT REPLY [Bug 13392] - When tag pooling is enabled, release() is not called on tag instances
I'm not sure what all the debate is over. Since you know for sure doStartTag() will be called, couldn't your tag call .release() as the first thing it does in doStartTag()? Wouldn't that easily solve the problem? since the spec does cover the issue as Jan pointed out, having doStartTag() call release() would be an quick and easy fix. maybe I'm just over simplifying the problem. peter lin [EMAIL PROTECTED] wrote: DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=13392. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=13392 When tag pooling is enabled, release() is not called on tag instances [EMAIL PROTECTED] changed: What|Removed |Added Status|CLOSED |REOPENED Resolution|INVALID | --- Additional Comments From [EMAIL PROTECTED] 2002-10-10 16:32 --- My preference is to have this fixed in the base release and to tell our customers that Tomcat works fine and that the tag pooling functionality works well with their applications without diabling this key feature. Why are you against such a practical change? -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
