Hi Derek, My changes are finally in place.. I need to do a profiling to see whether they have any good impact on performance...
If possible please drop the latest AXIOM binaries (from the nightly build or a manual build from Axiom svn head) in to your test scenerio and see whether if there's an improvment.. Thanks, Thilina On 3/7/07, Thilina Gunarathne <[EMAIL PROTECTED]> wrote:
Hi Derek, It's great that you started to nail down the attachment performance issues.. I've also started to do some refactorings to the Attachments handling classes.. I still have some test cases failing., which means it'll take some more time to get the changes in.. Let's crack this together :)... I'll also do some profiling to get more clues.. ~Thilina On 3/7/07, Derek Clayton <[EMAIL PROTECTED]> wrote: > Hi Davanum > > Unfortunately, the server is in .NET so I would only be able to provide the > client and some other sample data which alone might not be useful. > > However, I did some research in to the Axis2 and Axiom source using a client > that was trying to read a ~1.8M binary attachment (XOP/MTOM). It was taking > just over 3 seconds to read in and save the file. I finally tracked down > that 98% of the time involved was occuring in the Class: > > org.apache.axiom.attachments.MIMEBodyPartInputStream specifically in the > read() method at the line: > > // read the next value from stream > int value = inStream.read(); > > The inStream is a PushbackInputStream. On a simple isolated test (i.e. > plain old java reading from the file system) I tested reading in the file > using a BufferedInputStream vs a PushbackInputStream. It took 31 ms for the > BufferedInputStream and a whopping 1047 ms for the PushbackInputStream. (I > know there are reasons for the use of PushbackInputStream) > > I thought I had found the reason why it was slow and even in the Axiom > source just before it creates the PushbackInputStream it states in the Class > org.apache.axiom.attachments.Attachments: > > // do we need to wrap InputStream from a BufferedInputStream before > // wrapping from PushbackStream > > So I changed the source to first wrap in a BufferedInputStream however there > was no change in performance. The immediate Underlying InputStream is a > org.apache.commons.httpclient.AutoCloseInputStream and whatever it might > wrap (etc) I'm not sure. > > So basically I'm still not sure about what's going on. PushbackInputStream > is definitely a lot slower because it does not do any buffered reads. > However, I don't know why the performance wasn't improved when I wrapped it > in a BufferedInputStream. I was unable in my investigative time to find out > any other InputStreams that might have ultimately been wrapped by the > PushbackInputStream. > > Derek > > > >From: "Davanum Srinivas" <[EMAIL PROTECTED]> > >Reply-To: [EMAIL PROTECTED] > >To: [email protected] > >Subject: Re: MOTM and Encoding/Decoding > >Date: Thu, 1 Mar 2007 14:07:25 -0500 > > > >Derek, > > > >Could you please log a JIRA bug and upload the sample code? Let's try > >to create the scenario on our boxes... > > > >thanks, > >dims > > > >On 3/1/07, Derek Clayton <[EMAIL PROTECTED]> wrote: > >>Thank you for the very fast response. > >> > >> >If possible please post us some numbers of the time comparison. Make > >> >sure to avoid the System.out part when doing the comparison (this > >> >encoding takes time :( )... > >> > >>First let me explain the setup. The two machines are identical > >>hardware...Pentium 4 2.8GHz with 1Gig memory. Software differs since > >>these > >>are two different developer machines. > >> > >>Machine 1 acts as the client and is written in Java using Axis2. It sends > >>an Excel file to Machine 2. When it receives the XML file back (see next > >>paragraph) is simply saves it as a file. > >> > >>Machine 2 acts as the server and is using .NET. It receives an Excel file > >>as binary, saves that file, uses an ActiveX control to read in the file > >>and > >>parse it to generate an XML representation of the data. It then sends > >>that > >>XML back to the Machine 1 in the SOAP response. > >> > >>I haven't written an isolated test but this setup would favor Java/Axis2 > >>anyway since Machine 2 is having to do actual work in addition to reading > >>the binary file. > >> > >>For smaller Excel files the times are quite reasonable. > >> > >>Test 1 > >>-------------------------- > >>For a 302K send and a 922K response. > >> > >>Time to send and receive response: 1.6 secs. > >>Time to read response: 6 secs. > >> > >>It is taking about 3 times longer which is reasonable since the file size > >>is > >>3x as large even though the server is doing a lot more work than the > >>client. > >> > >>However, as the files get large the performance begins to break down. > >> > >>Test 2 > >>-------------------------- > >>For a 1.4M send and a 5.4M response. > >> > >>Time to send and receive response: 3 secs. > >>Time to read response: 37 secs. > >> > >>37 seconds to read the 5.4M binary file seems like a long time. As well > >>you > >>can see that the server processed a larger file (1.4M) in half the time as > >>did the client in Test 1. For even larger files the difference becomes > >>greater. > >> > >>Test 3 > >>-------------------------- > >>For a 8.5M and a 32M response. > >> > >>Time to send and receive response: 18 secs. > >>Time to read response: 220 secs. > >> > >>Here you can see the .NET is reading the 8.5M (along with parsing it and > >>generating XML) in 18 secs compared to where Java/Axis2 took 37 secs to > >>simply read and save a 5.4M file in Test 2. > >> > >>I know this is an informal test case. But is 37 seconds to read a 5.4M > >>binary attachment with no decoding similar to the experience of others? > >> > >>Thanks! > >> > >>Derek > >> > >>_________________________________________________________________ > >>Don't waste time standing in line—try shopping online. Visit Sympatico / > >>MSN > >>Shopping today! http://shopping.sympatico.msn.ca > >> > >> > >>--------------------------------------------------------------------- > >>To unsubscribe, e-mail: [EMAIL PROTECTED] > >>For additional commands, e-mail: [EMAIL PROTECTED] > >> > >> > > > > > >-- > >Davanum Srinivas :: http://wso2.org/ :: Oxygen for Web Services Developers > > > >--------------------------------------------------------------------- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > > > _________________________________________________________________ > Your Space. Your Friends. Your Stories. Share your world with Windows Live > Spaces. http://spaces.live.com/?mkt=en-ca > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Thilina Gunarathne WSO2, Inc.; http://www.wso2.com/ Home page: http://webservices.apache.org/~thilina/ Blog: http://thilinag.blogspot.com/
-- Thilina Gunarathne WSO2, Inc.; http://www.wso2.com/ Home page: http://webservices.apache.org/~thilina/ Blog: http://thilinag.blogspot.com/ --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
