RE: Slow upload speeds
Actually it shouldn't be the browser upload mechanism. I can't say about IE (closed-source) but the Mozilla/Netscape 6.0 are smart in the way they upload. There was a problem with Jason's library and Tomcat in some of the older versions (i.e. the version that ships in his book). If you go to the O'Reily site you can download a newer version that is smarter about how it receives the upload. This yielded a significant speed up. By the way, when uploading from the same machine as the server - you're right you should see almost 100% CPU utilization, but you should also see a significant amout of OS processing (moving from disk to memory, memory to device (NIC), device (NIC) to memory, memory to disk), which should be your major limitor (and will depend upon things like having DMA in your hardware architecture.) Randy -Original Message- From: Dmitry Rogatkin [mailto:[EMAIL PROTECTED]] Sent: Wednesday, February 21, 2001 11:16 PM To: [EMAIL PROTECTED] Subject: RE: Slow upload speeds I have feeling that it's a browser issue. Can somebody who knows browser upload mechanism confirm that? Dmitry R., [EMAIL PROTECTED] Chief Architect, MetricStream.COM Santa Clara, CA -Original Message- From:Tal Dayan [EMAIL PROTECTED] Sent:Wed, 21 Feb 2001 19:05:06 -0800 To: [EMAIL PROTECTED] Subject: RE: Slow upload speeds We have the same low performance problem with Jason's library. The strange things is that when the browser and server are on the same machine (using 127.0.0.1) the CPU usage is still low while transferring large files. Our experience with other client/server applications is that the CPU usage in such cases gets to 100% (since there is no real I/O to the NIC, all is handled by the CPU). I am not sure where the 'idle' time is spent. Tal -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 04, 2001 3:13 PM To: [EMAIL PROTECTED] Subject: Slow upload speeds Hi everyone. I am using Tomcat-3.2.1 on W2K, Solaris with or without apache. Since I need to provide users with functionality to upload fairly big files ( 10 MB), speed is essential. My uploader is based on Jason Hunter code. Maximum upload speed I could achieve is between 40 and 60 Kbytes/s (even when client and server are on the same machine). Bottleneck (quiet predictable) seems to be in ServletInputStream misc. "read()" methods. Speed does not seem to be dependant on a platform or web server. Anyone was able to achieve better speeds (at least around 100 kBytes/s)? Any ideas ? Thanks Andrus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] ___ Visit http://www.visto.com/info, your free web-based communications center. Visto.com. Life on the Dot. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
RE: Slow upload speeds
I have feeling that it's a browser issue. Can somebody who knows browser upload mechanism confirm that? Dmitry R., [EMAIL PROTECTED] Chief Architect, MetricStream.COM Santa Clara, CA -Original Message- From:Tal Dayan [EMAIL PROTECTED] Sent:Wed, 21 Feb 2001 19:05:06 -0800 To: [EMAIL PROTECTED] Subject: RE: Slow upload speeds We have the same low performance problem with Jason's library. The strange things is that when the browser and server are on the same machine (using 127.0.0.1) the CPU usage is still low while transferring large files. Our experience with other client/server applications is that the CPU usage in such cases gets to 100% (since there is no real I/O to the NIC, all is handled by the CPU). I am not sure where the 'idle' time is spent. Tal -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 04, 2001 3:13 PM To: [EMAIL PROTECTED] Subject: Slow upload speeds Hi everyone. I am using Tomcat-3.2.1 on W2K, Solaris with or without apache. Since I need to provide users with functionality to upload fairly big files ( 10 MB), speed is essential. My uploader is based on Jason Hunter code. Maximum upload speed I could achieve is between 40 and 60 Kbytes/s (even when client and server are on the same machine). Bottleneck (quiet predictable) seems to be in ServletInputStream misc. "read()" methods. Speed does not seem to be dependant on a platform or web server. Anyone was able to achieve better speeds (at least around 100 kBytes/s)? Any ideas ? Thanks Andrus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] ___ Visit http://www.visto.com/info, your free web-based communications center. Visto.com. Life on the Dot. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
RE: Slow upload speeds
I've never dealt with this in Tomcat, but from other network apps: How are you using the ServletInputStream - are you calling int read(), which reads the stream a byte at a time, or are you calling int read(byte[])? My experience is that you want to use the second option, preferably with the byte[] size equal to 1500 (Reason for 1500 - its the MTU (Maximum Transmission Unit) of Ethernet, probably the network the server is attached to). Reason this speeds things up: When calling int read(), to get 1500 bytes of input, Java has to make 1500 system calls. If you use int read(byte[]) then it can get 1500 bytes in one system call. The reason you want this byte[] to be the same length as the MTU - you can read each packet as it arrives and be processing it as the next packet finds its way across the network instead of waiting on the next one. One caveat: I have not looked at the source code for this class, but from the JavaDoc it appears that all the methods we are talking about are inherited from InputStream, so these techniques should work. Randy -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 04, 2001 6:13 PM To: [EMAIL PROTECTED] Subject: Slow upload speeds Hi everyone. I am using Tomcat-3.2.1 on W2K, Solaris with or without apache. Since I need to provide users with functionality to upload fairly big files ( 10 MB), speed is essential. My uploader is based on Jason Hunter code. Maximum upload speed I could achieve is between 40 and 60 Kbytes/s (even when client and server are on the same machine). Bottleneck (quiet predictable) seems to be in ServletInputStream misc. "read()" methods. Speed does not seem to be dependant on a platform or web server. Anyone was able to achieve better speeds (at least around 100 kBytes/s)? Any ideas ? Thanks Andrus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
RE: Slow upload speeds
Even though its on the same machine, the data still flows through the Operating System and into the TCP/IP stack to pop out the other side on the same machine, it just didn't cross any networking wires. That's the whole point of localhost/127.0.0.1, simulate traffic from the network without using the wires. The problem is still (probably, I haven't seen Jason's code), fundamentally the number of underlying system calls. This would be true if you wrote the program in Java or C or Visual Basic or Perl, Unix or NT. The User process has to ask the Operating System for the bytes. If it asks for each byte one at a time then it takes a lot more time than if it asked for (relatively) big blocks. If you could give me a pointer to the source I could probably find a little time today to look over this and see how his code works and if this is really the problem. (Its possible that he knew about this problem and accounted for it, but its a very commonly overlooked issue). Randy -Original Message- From: Jose Euclides da Silva Junior - DIGR.O [mailto:[EMAIL PROTECTED]] Sent: Friday, January 05, 2001 9:33 AM To: [EMAIL PROTECTED] Subject: RES: Slow upload speeds -BEGIN PGP SIGNED MESSAGE- Many people have been talked about bad performance when using Jason code.But, you should tell me some information like available memory and availabe disk space When you try to use client and server on the same machine and the problem still happens, we are able to say that bottleneck isnt the reason of the slow speed. Regards, José Euclides Júnior __ E-mail: [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] http://euclides.8m.com - -Mensagem original- De: Randy Layman [SMTP:[EMAIL PROTECTED]] Enviada em: Sexta-feira, 5 de Janeiro de 2001 10:32 Para: '[EMAIL PROTECTED]' Assunto:RE: Slow upload speeds I've never dealt with this in Tomcat, but from other network apps: How are you using the ServletInputStream - are you calling int read(), which reads the stream a byte at a time, or are you calling int read(byte[])? My experience is that you want to use the second option, preferably with the byte[] size equal to 1500 (Reason for 1500 - its the MTU (Maximum Transmission Unit) of Ethernet, probably the network the server is attached to). Reason this speeds things up: When calling int read(), to get 1500 bytes of input, Java has to make 1500 system calls. If you use int read(byte[]) then it can get 1500 bytes in one system call. The reason you want this byte[] to be the same length as the MTU - you can read each packet as it arrives and be processing it as the next packet finds its way across the network instead of waiting on the next one. One caveat: I have not looked at the source code for this class, but from the JavaDoc it appears that all the methods we are talking about are inherited from InputStream, so these techniques should work. Randy - -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 04, 2001 6:13 PM To: [EMAIL PROTECTED] Subject: Slow upload speeds Hi everyone. I am using Tomcat-3.2.1 on W2K, Solaris with or without apache. Since I need to provide users with functionality to upload fairly big files ( 10 MB), speed is essential. My uploader is based on Jason Hunter code. Maximum upload speed I could achieve is between 40 and 60 Kbytes/s (even when client and server are on the same machine). Bottleneck (quiet predictable) seems to be in ServletInputStream misc. "read()" methods. Speed does not seem to be dependant on a platform or web server. Anyone was able to achieve better speeds (at least around 100 kBytes/s)? Any ideas ? Thanks Andrus - - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] - - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: PGP 6.5.1 iQCVAwUBOlX0r90YhuJ3BUxtAQFLYAP9EPUYLOs+Sf6VhEoQt0LNvKG+sktEp5if oHR5kFLRrw58dd7utPfw8Bs96CZUuOtU/G4PfdKzvz3CyLfz7HsmOokIe4NnIMZ3 whhPy3Mo4ZJffczcSLCbTioxNmObxy7FavLvJGbXnVqSwJrbMwQRYQJLP1P23i+R 2UMnZ0P6r6s= =KmIk -END PGP SIGNATURE- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
RE: Slow upload speeds
Hi, thanks a lot for the replies. I will take a look at the links to the new version of uploader code. From: Randy Layman [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: "'[EMAIL PROTECTED]'" [EMAIL PROTECTED] Subject: RE: Slow upload speeds Date: Fri, 5 Jan 2001 08:20:44 -0500 The problem is still (probably, I haven't seen Jason's code), fundamentally the number of underlying system calls. This would be true if you wrote the program in Java or C or Visual Basic or Perl, Unix or NT. The User process has to ask the Operating System for the bytes. If it asks for each byte one at a time then it takes a lot more time than if it asked for (relatively) big blocks. Yes, Jason's code was calling readLine on ServletInputStream, so my original strategy was naturally to use in.read(buf, off, len) instead. I tried it, and it did not show any improvement. Call to "read" was still taking 99% of the method execution time. So I suspect this is the underlying stream implementation that is not buffered and still reads 1 byte at at a time from the socket. If you could give me a pointer to the source I could probably find a little time today to look over this and see how his code works and if this is really the problem. Thanks a lot. I'll clean up the code to remove dependancies (it is a part of a much bigger project) and send you the link. (Its possible that he knew about this problem and accounted for it, but its a very commonly overlooked issue). Randy Andrus _ Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
