RE: MTOM error java.lang.OutOfMemoryError: Java heap space
Hi Vikas, An OutOfMemoryError strongly suggests that cacheAttachments or one of the other 3 associated properties are not being set properly. On the client side, for receiving the file, one can code: Options options = new Options(); options.setTo(new EndpointReference(url)); options.setProperty(Constants.Configuration.ENABLE_MTOM, Constants.VALUE_TRUE); options.setProperty(Constants.Configuration.CACHE_ATTACHMENTS, Constants.VALUE_TRUE); options.setProperty(Constants.Configuration.ATTACHMENT_TEMP_DIR, AXIS2_ATTACHMENT_DIR); options.setProperty(Constants.Configuration.FILE_SIZE_THRESHOLD, 8000); One can modify axis2.xml on the server side similarly. Betsy -Original Message- From: Vikas [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 23, 2007 2:18 AM To: axis user Subject: Re: MTOM error java.lang.OutOfMemoryError: Java heap space Hi Thilina / Friends As you said earlier that you have tested Axis2 with 200 MB-700 MB file attachment, Can you tell me which configuration of axis you did it [means MTOM / SwA/ StAX / SAAJ versions] ? Can we do direct streaming using StAX / SAAJ to send such big files? With best regards From Vikas R. Khengare - Original Message - From: Thilina Gunarathne [EMAIL PROTECTED] To: axis-user@ws.apache.org; Vikas [EMAIL PROTECTED] Sent: Wednesday, May 23, 2007 1:39 PM Subject: Re: MTOM error java.lang.OutOfMemoryError: Java heap space We have successfully used the MTOM sample to transfer files with sizes in the range of 200MB to 700 MB... I have enabled FileCaching on server side by following code. parameter name=enableMTOM locked=falsetrue/parameter parameter name=enableSwA locked=falsetrue/parameter !--Uncomment if you want to enable file caching for attachments -- parameter name=cacheAttachments locked=falsetrue/parameter parameter name=attachmentDIR locked=falseC:\\TempWebService\\DestinationTemp/parameter parameter name=sizeThreshold locked=false4000/parameter At the same time I am continuing with client side file caching code. Client side caching is not needed for the MTOM sample... It is caching that file in given directory, but not writing on to output stream i.e. on specified location. Can you explain the above statement a bit more... Can you see the whole file on the caching directory... What do you mean by writing to the output stream... Thanks, Thilina Now also I am able send 2-3MB size of files but not bigger file size like 7MB-100MB. I am getting same error [java] Exception in thread main org.apache.axis2.AxisFault: java.lang.OutOfMemoryError: Java heap space [java] at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperatio n.java:271) [java] at org.apache.axis2.description.OutInAxisOperationClient.execute(OutInAxisOpera tion.java:202) [java] at sample.mtom.service.MTOMSampleStub.attachment(Unknown Source) [java] at sample.mtom.client.Client.transferFile(Unknown Source) [java] at sample.mtom.client.Client.main(Unknown Source) [java] Java Result: 1 my questions still unsolved? Can some one suggest solution? [Can we send file as attachment of size 5MB - 1GB?] With best regards Vikas R. Khengare - Original Message - From: Jochen Zink [EMAIL PROTECTED] To: axis-user@ws.apache.org Sent: Wednesday, May 23, 2007 11:51 AM Subject: Re: MTOM error java.lang.OutOfMemoryError: Java heap space Hi Vikas. you have to enabled FileCaching on ServerSide. This is the only way to handle large attachments int axis2, sadly. -Ursprüngliche Nachricht- Von: axis-user@ws.apache.org Gesendet: 23.05.07 07:23:12 An: axis user axis-user@ws.apache.org Betreff: MTOM error java.lang.OutOfMemoryError: Java heap space Hi friends, I am running sample application mtom from standard axis2 samples [axis2-1.1.1\samples\mtom]. This sample is working for sending attachment of size 2-3 MB, but giving error for more than 3MB size of files. I want to send file of size 50-100 MB. error is as follows [java] Exception in thread main org.apache.axis2.AxisFault: java.lang.Out OfMemoryError: Java heap space [java] at org.apache.axis2.description.OutInAxisOperationClient.send(Ou tInAxisOperation.java:271) [java] at org.apache.axis2.description.OutInAxisOperationClient.execute (OutInAxisOperation.java:202) [java] at sample.mtom.service.MTOMSampleStub.attachment(Unknown Source) [java] at sample.mtom.client.Client.transferFile(Unknown Source) [java] at sample.mtom.client.Client.main(Unknown Source) [java] Java Result: 1 I
RE: MTOM error java.lang.OutOfMemoryError: Java heap space
There is a known bug with MTOM attachments in Axis 1.1.1, fixed in Axis 1.2. -Original Message- From: Jochen Zink [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 23, 2007 1:19 AM To: axis-user@ws.apache.org Subject: Re: MTOM error java.lang.OutOfMemoryError: Java heap space Hi Vikas, okay, I tryed a example with a attachment larger than 50mb. ... I got the same error. I'm sure that I have enabled FileCaching in the same way as you did (configured in axis2.xml). I'm pretty sure that this worked for month ago. The only thing I have changed is to copy newer versions of the axiom libs. Currently I use axis2 1.1.1 with axiom 1.2.4. I tryed the same szenarop with axis2 1.1.1 and axiom 1.2.2 and it does not work too. I'm not sure what is going wrong... my axis2.xml (server): parameter name=enableMTOM locked=falsetrue/parameter parameter name=enableSwA locked=falsefalse/parameter parameter name=cacheAttachments locked=falsetrue/parameter parameter name=attachmentDIR locked=falseD:\\cache/parameter parameter name=sizeThreshold locked=false3000/parameter my services.xml has no parameter which deals with attachments.. Is there any bug known? Thanks -Ursprüngliche Nachricht- Von: axis-user@ws.apache.org Gesendet: 23.05.07 09:22:04 An: axis-user@ws.apache.org Betreff: Re: MTOM error java.lang.OutOfMemoryError: Java heap space Hi Jochen Friends, I have enabled FileCaching on server side by following code. parameter name=enableMTOM locked=falsetrue/parameter parameter name=enableSwA locked=falsetrue/parameter !--Uncomment if you want to enable file caching for attachments -- parameter name=cacheAttachments locked=falsetrue/parameter parameter name=attachmentDIR locked=falseC:\\TempWebService\\DestinationTemp/parameter parameter name=sizeThreshold locked=false4000/parameter At the same time I am continuing with client side file caching code. It is caching that file in given directory, but not writing on to output stream i.e. on specified location. Now also I am able send 2-3MB size of files but not bigger file size like 7MB-100MB. I am getting same error [java] Exception in thread main org.apache.axis2.AxisFault: java.lang.OutOfMemoryError: Java heap space [java] at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperatio n.java:271) [java] at org.apache.axis2.description.OutInAxisOperationClient.execute(OutInAxisOpera tion.java:202) [java] at sample.mtom.service.MTOMSampleStub.attachment(Unknown Source) [java] at sample.mtom.client.Client.transferFile(Unknown Source) [java] at sample.mtom.client.Client.main(Unknown Source) [java] Java Result: 1 my questions still unsolved? Can some one suggest solution? [Can we send file as attachment of size 5MB - 1GB?] With best regards Vikas R. Khengare - Original Message - From: Jochen Zink [EMAIL PROTECTED] To: axis-user@ws.apache.org Sent: Wednesday, May 23, 2007 11:51 AM Subject: Re: MTOM error java.lang.OutOfMemoryError: Java heap space Hi Vikas. you have to enabled FileCaching on ServerSide. This is the only way to handle large attachments int axis2, sadly. -Ursprüngliche Nachricht- Von: axis-user@ws.apache.org Gesendet: 23.05.07 07:23:12 An: axis user axis-user@ws.apache.org Betreff: MTOM error java.lang.OutOfMemoryError: Java heap space Hi friends, I am running sample application mtom from standard axis2 samples [axis2-1.1.1\samples\mtom]. This sample is working for sending attachment of size 2-3 MB, but giving error for more than 3MB size of files. I want to send file of size 50-100 MB. error is as follows [java] Exception in thread main org.apache.axis2.AxisFault: java.lang.Out OfMemoryError: Java heap space [java] at org.apache.axis2.description.OutInAxisOperationClient.send(Ou tInAxisOperation.java:271) [java] at org.apache.axis2.description.OutInAxisOperationClient.execute (OutInAxisOperation.java:202) [java] at sample.mtom.service.MTOMSampleStub.attachment(Unknown Source) [java] at sample.mtom.client.Client.transferFile(Unknown Source) [java] at sample.mtom.client.Client.main(Unknown Source) [java] Java Result: 1 I have enabled MTOM filecaching both in the client side, but not server side caching means in axis2.xml. 1) Can some one figureout solution? 2) Can we send file as attachment of size 50MB - 1GB? With best regards From Vikas R. Khengare ___ SMS schreiben mit WEB.DE FreeMail - einfach, schnell und kostenguenstig. Jetzt gleich testen! http://f.web.de/?mc=021192
RE: [Axis2] ConfigurationContext configuration
Michele - I don't know if this will help, but I modify values in the AxisConfiguration using code like this: MessageContext msgContext = MessageContext.getCurrentMessageContext(); AxisConfiguration axisConfig = msgContext.getConfigurationContext() .getAxisConfiguration(); Parameter param = axisConfig.getParameter(Constants.Configuration.CACHE_ATTACHMENTS); param.setValue(true); Betsy -Original Message- From: Michele Mazzucco [mailto:[EMAIL PROTECTED] Sent: Thursday, March 29, 2007 8:00 AM To: axis2 Subject: [Axis2] ConfigurationContext configuration Hi all, I've already seen this document [1] and I would like to know if there's any way to create a ConfigurationContext, without loading it from a file, with all default parameters apart from the http server port. Thanks, Michele [1] http://wso2.org/library/585 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [Axis2] ConfigurationContext configuration
Michele, I use code like this on the client: Options options = new Options(); options.setProperty(Constants.Configuration.CACHE_ATTACHMENTS, Constants.VALUE_TRUE); serviceClient.setOptions(options); Betsy -Original Message- From: Michele Mazzucco [mailto:[EMAIL PROTECTED] Sent: Thursday, March 29, 2007 8:34 AM To: axis-user@ws.apache.org Subject: RE: [Axis2] ConfigurationContext configuration Betsy, thanks. However I would like to change the parameters on the client side. Any clue? Michele On Thu, 2007-03-29 at 08:13 -0700, Betsy Frey wrote: Michele - I don't know if this will help, but I modify values in the AxisConfiguration using code like this: MessageContext msgContext = MessageContext.getCurrentMessageContext(); AxisConfiguration axisConfig = msgContext.getConfigurationContext() .getAxisConfiguration(); Parameter param = axisConfig.getParameter(Constants.Configuration.CACHE_ATTACHMENTS); param.setValue(true); Betsy -Original Message- From: Michele Mazzucco [mailto:[EMAIL PROTECTED] Sent: Thursday, March 29, 2007 8:00 AM To: axis2 Subject: [Axis2] ConfigurationContext configuration Hi all, I've already seen this document [1] and I would like to know if there's any way to create a ConfigurationContext, without loading it from a file, with all default parameters apart from the http server port. Thanks, Michele [1] http://wso2.org/library/585 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [AXIS2] MTOM problems
Both problems below seem to be fixed by Axis2-1.2-RC1. Thilina was right when he guessed I had not replaced all the jars when I said I had tested with RC1. Betsy -Original Message- From: Thilina Gunarathne [mailto:[EMAIL PROTECTED] Sent: Tuesday, March 27, 2007 12:12 PM To: axis-user@ws.apache.org Subject: Re: [AXIS2] MTOM problems Hi, 1. Sometimes 1 extra byte is received. This is a 255 (0xff). It would be great if you can isolate the failing scenario and log a Jira with a test case.. 2. When I set the client option CACHE_ATTACHMENTS to true, a client-side thread loops, appending bytes of 0xff to a .ATT file in the attachments directory. I did some fixes to the Attachment caching after the 1.1 release.. They are available in the latest svn head of Axiom.. And should be available with the Axis2-1.2-RC1.. Can you please retest the 2 with Axis2-1.2-RC1.. Make sure to replace each and every jar with the once that came with 1.2-rc1, specially the Axiom jars.. -- Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[AXIS2] MTOM problems
I have some MTOM sample code in which a client sends all files in its transfer directory to a service. The server sends the files back to the client, which stores them in its received directory, and does a byte-to-byte compare to make sure the files in the two directories match. 1. Sometimes 1 extra byte is received. This is a 255 (0xff). 2. When I set the client option CACHE_ATTACHMENTS to true, a client-side thread loops, appending bytes of 0xff to a .ATT file in the attachments directory. The problems occur in both 1.1.1 and in 1.2 RC1. My guess is both problems might have the same cause: an incorrect cast of an int to a byte, and then testing for -1 (eof); bytes have value 0 to 255, not -1. Are these known problems? Thanks, Betsy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [AXIS2] MTOM performance
Thilina - thanks for this information. Betsy -Original Message- From: Thilina Gunarathne [mailto:[EMAIL PROTECTED] Sent: Friday, March 23, 2007 2:18 AM To: axis-user@ws.apache.org Subject: Re: [AXIS2] MTOM performance I did a quick test against the snapshots and the results I got were in the range of ~6.5 MB/s, which translates in to 375 to 400 MB/Min.. My environments is Ubuntu Linux 6.10 java version 1.5.0_10 file system: ext3 Apache Tomcat 5.5.20 FileCacheThreshold 4 bytes The program I used was the SoapWithAttachments sample of Axis2. Thanks, Thilina On 3/20/07, Betsy Frey [EMAIL PROTECTED] wrote: Thanks, Thilina. The latest Axiom jars make a 3x difference. I'm now seeing about 10mb/minute. Are more optimizations planned? Betsy -Original Message- From: Thilina Gunarathne [mailto:[EMAIL PROTECTED] Sent: Tuesday, March 20, 2007 4:16 AM To: axis-user@ws.apache.org Subject: Re: [AXIS2] MTOM performance Betsy, copying it. I continue to see it grow by about 3.7mb/minute. I did some testing after the 1.0 release and the numbers I got were ~20 mb/min... MAy be something went wrong from 1.0 to 1.1... It would be really great if you can let us know the transfer rates you got when using the latest axiom jars [1].. ~Thilina ttp://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axio m-api-SNAPSHOT.jar http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi om-dom-SNAPSHOT.jar http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi om-impl-SNAPSHOT.jar On 3/19/07, Betsy Frey [EMAIL PROTECTED] wrote: Thanks, Rodirigo, but the problem is not my disks. I changed the code to generate bytes on the fly, instead of from a source file. I changed the reference from localhost to 127.0.0.1. I am just looking at the file in the attachment directory; not moving or copying it. I continue to see it grow by about 3.7mb/minute. Separately, I have confirmed that the transfer is in binary in a MIME section; MTOM is enabled. So the question remains: why is MTOM performance so sluggish, and is there anything to be done about it? Thanks. Betsy -Original Message- From: Rodrigo Ruiz [mailto:[EMAIL PROTECTED] Sent: Monday, March 19, 2007 8:57 AM To: axis-user@ws.apache.org Subject: Re: [AXIS2] MTOM performance Several reasons :-) First, a file system copy can make use of low-level OS mechanisms to speed up transfers, and even in some cases, delegate the job to the disk manager chipset. You will never get these speeds with a web service, as it will never have direct access to the client file system. Said this, if you think about what the client and server do, leaving aside the overhead from XML transfer and processing, you will notice that having the client and server on the same machine is not a good testcase. On one side, the client must read the file from disk. If the client is correctly configured and programmed, the client will send the file contents to the server at the same time it is reading it (concurrently), so to avoid putting the whole file in memory. On the other side, the server will write the incoming bytes to a file while it is receiving the client request. Finally, once the whole file is received in the server side, your service will usually move the file to its final destination. If you copy the file instead of moving it, the service will perform even worse ;-) Now, if you join both sides, you will see the system must read and write the same file at different locations of the disk. This will make your test waste a lot of time just in disk head seeks. I would propose you to make sure that you follow these guides: * Make sure reads and writes are, at least, from/to different (physical) disks. If you have only one disk per machine, but you have several machines, try to put client and server in different hosts. Take into account that a fast ethernet bandwidth is far better than 3.7MB per minute ;-) * If you want to do some tests for only one of the sides, another approach could be to create mockups for the file readers and writers. For example, if you use a DataHandler in the client side, you can implement a fake DataSource that generates random bytes on the fly, instead of reading them from a file. Such a DataSource will allow you to test your server side performance without the client interfering in your disk accesses. * Always use the localhost address (127.0.0.1) to contact the server. In many environments you will get different (and worse) results if you use the machine public IP address. * Make sure you do your best to move the file in the server side to its final location instead of copying it. The performance is totally different. Hope this helps you, Rodrigo Ruiz
RE: [AXIS2] MTOM performance
Thanks, Thilina. The latest Axiom jars make a 3x difference. I'm now seeing about 10mb/minute. Are more optimizations planned? Betsy -Original Message- From: Thilina Gunarathne [mailto:[EMAIL PROTECTED] Sent: Tuesday, March 20, 2007 4:16 AM To: axis-user@ws.apache.org Subject: Re: [AXIS2] MTOM performance Betsy, copying it. I continue to see it grow by about 3.7mb/minute. I did some testing after the 1.0 release and the numbers I got were ~20 mb/min... MAy be something went wrong from 1.0 to 1.1... It would be really great if you can let us know the transfer rates you got when using the latest axiom jars [1].. ~Thilina ttp://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axio m-api-SNAPSHOT.jar http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi om-dom-SNAPSHOT.jar http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi om-impl-SNAPSHOT.jar On 3/19/07, Betsy Frey [EMAIL PROTECTED] wrote: Thanks, Rodirigo, but the problem is not my disks. I changed the code to generate bytes on the fly, instead of from a source file. I changed the reference from localhost to 127.0.0.1. I am just looking at the file in the attachment directory; not moving or copying it. I continue to see it grow by about 3.7mb/minute. Separately, I have confirmed that the transfer is in binary in a MIME section; MTOM is enabled. So the question remains: why is MTOM performance so sluggish, and is there anything to be done about it? Thanks. Betsy -Original Message- From: Rodrigo Ruiz [mailto:[EMAIL PROTECTED] Sent: Monday, March 19, 2007 8:57 AM To: axis-user@ws.apache.org Subject: Re: [AXIS2] MTOM performance Several reasons :-) First, a file system copy can make use of low-level OS mechanisms to speed up transfers, and even in some cases, delegate the job to the disk manager chipset. You will never get these speeds with a web service, as it will never have direct access to the client file system. Said this, if you think about what the client and server do, leaving aside the overhead from XML transfer and processing, you will notice that having the client and server on the same machine is not a good testcase. On one side, the client must read the file from disk. If the client is correctly configured and programmed, the client will send the file contents to the server at the same time it is reading it (concurrently), so to avoid putting the whole file in memory. On the other side, the server will write the incoming bytes to a file while it is receiving the client request. Finally, once the whole file is received in the server side, your service will usually move the file to its final destination. If you copy the file instead of moving it, the service will perform even worse ;-) Now, if you join both sides, you will see the system must read and write the same file at different locations of the disk. This will make your test waste a lot of time just in disk head seeks. I would propose you to make sure that you follow these guides: * Make sure reads and writes are, at least, from/to different (physical) disks. If you have only one disk per machine, but you have several machines, try to put client and server in different hosts. Take into account that a fast ethernet bandwidth is far better than 3.7MB per minute ;-) * If you want to do some tests for only one of the sides, another approach could be to create mockups for the file readers and writers. For example, if you use a DataHandler in the client side, you can implement a fake DataSource that generates random bytes on the fly, instead of reading them from a file. Such a DataSource will allow you to test your server side performance without the client interfering in your disk accesses. * Always use the localhost address (127.0.0.1) to contact the server. In many environments you will get different (and worse) results if you use the machine public IP address. * Make sure you do your best to move the file in the server side to its final location instead of copying it. The performance is totally different. Hope this helps you, Rodrigo Ruiz Betsy Frey wrote: I've just made some measurements, transferring files using MTOM in Axis2 on tomcat. My client and servlet are on the same machine. Attachments are cached. For files of 1mb or more, the transfer rate, client-to-servlet, is about 3.7mb per minute. In contrast, a file system copy transfers at about 2.5gb per minute: 675 times faster than Axis2. Why is MTOM performance is so sluggish? Thanks for any assistance, Betsy ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review
[AXIS2] MTOM performance
I've just made some measurements, transferring files using MTOM in Axis2 on tomcat. My client and servlet are on the same machine. Attachments are cached. For files of 1mb or more, the transfer rate, client-to-servlet, is about 3.7mb per minute. In contrast, a file system copy transfers at about 2.5gb per minute: 675 times faster than Axis2. Why is MTOM performance is so sluggish? Thanks for any assistance, Betsy ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [AXIS2] MTOM performance
Thanks, Rodirigo, but the problem is not my disks. I changed the code to generate bytes on the fly, instead of from a source file. I changed the reference from localhost to 127.0.0.1. I am just looking at the file in the attachment directory; not moving or copying it. I continue to see it grow by about 3.7mb/minute. Separately, I have confirmed that the transfer is in binary in a MIME section; MTOM is enabled. So the question remains: why is MTOM performance so sluggish, and is there anything to be done about it? Thanks. Betsy -Original Message- From: Rodrigo Ruiz [mailto:[EMAIL PROTECTED] Sent: Monday, March 19, 2007 8:57 AM To: axis-user@ws.apache.org Subject: Re: [AXIS2] MTOM performance Several reasons :-) First, a file system copy can make use of low-level OS mechanisms to speed up transfers, and even in some cases, delegate the job to the disk manager chipset. You will never get these speeds with a web service, as it will never have direct access to the client file system. Said this, if you think about what the client and server do, leaving aside the overhead from XML transfer and processing, you will notice that having the client and server on the same machine is not a good testcase. On one side, the client must read the file from disk. If the client is correctly configured and programmed, the client will send the file contents to the server at the same time it is reading it (concurrently), so to avoid putting the whole file in memory. On the other side, the server will write the incoming bytes to a file while it is receiving the client request. Finally, once the whole file is received in the server side, your service will usually move the file to its final destination. If you copy the file instead of moving it, the service will perform even worse ;-) Now, if you join both sides, you will see the system must read and write the same file at different locations of the disk. This will make your test waste a lot of time just in disk head seeks. I would propose you to make sure that you follow these guides: * Make sure reads and writes are, at least, from/to different (physical) disks. If you have only one disk per machine, but you have several machines, try to put client and server in different hosts. Take into account that a fast ethernet bandwidth is far better than 3.7MB per minute ;-) * If you want to do some tests for only one of the sides, another approach could be to create mockups for the file readers and writers. For example, if you use a DataHandler in the client side, you can implement a fake DataSource that generates random bytes on the fly, instead of reading them from a file. Such a DataSource will allow you to test your server side performance without the client interfering in your disk accesses. * Always use the localhost address (127.0.0.1) to contact the server. In many environments you will get different (and worse) results if you use the machine public IP address. * Make sure you do your best to move the file in the server side to its final location instead of copying it. The performance is totally different. Hope this helps you, Rodrigo Ruiz Betsy Frey wrote: I've just made some measurements, transferring files using MTOM in Axis2 on tomcat. My client and servlet are on the same machine. Attachments are cached. For files of 1mb or more, the transfer rate, client-to-servlet, is about 3.7mb per minute. In contrast, a file system copy transfers at about 2.5gb per minute: 675 times faster than Axis2. Why is MTOM performance is so sluggish? Thanks for any assistance, Betsy ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- --- GRID SYSTEMS, S.A. Rodrigo Ruiz Parc Bit - Edificio 17 Research Coordinator 07121 Palma de Mallorca[EMAIL PROTECTED] Baleares - Spain Tel: +34 971 435 085 http://www.gridsystems.com/Fax: +34 971 435 082 --- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: service request location
In Axis 1.4, you can use code like this:
boolean isClientLocal = true;
MessageContext msgContext =
MessageContext.getCurrentContext();
String remoteIP =
msgContext.getStrProp(org.apache.axis.Constants.MC_REMOTE_ADDR);
if (!remoteIP.equals(127.0.0.1))
{
InetAddress serverAddress =
InetAddress.getLocalHost();
remoteAddress = InetAddress.getByName(remoteIP);
// See if the caller's IP address matches that of
the server.
if (!serverAddress.equals(remoteAddress))
isClientLocal = false;
}
From: Nate - Hotmail [mailto:[EMAIL PROTECTED]
Sent: Monday, March 19, 2007 9:55 AM
To: axis-user@ws.apache.org
Subject: service request location
Is there a way to find out where a client is making a request from, ie:
localhost or somewhere else?
Thanks,
Nate
**
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed. Any
unauthorized review, use, disclosure or distribution is prohibited. If you are
not the intended recipient, please contact the sender by reply e-mail and
destroy all copies of the original message.
RE: WS debugging in Eclipse
To use eclipse to debug a tomcat servlet: 1. Be sure that tomcat is started with the below. One way to do that is to edit tomcat/bin/catalina.bat, where JAVA_OPTS is defined. -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=n 2. Set a breakpoint in the servlet code. Note that servlet timeouts may start to occur when you debug, so you want the breakpoint close to the code being debugged. 3. Start the eclipse debugger. Open Run | Debug... Select the section for remote java application. Select the project. Connection type is socket. Set host and address (8000 in the above case). 4. Run the test that invokes the server. The breakpoint should show up in the Eclipse debugger. Betsy From: VF [mailto:[EMAIL PROTECTED] Sent: Thursday, December 07, 2006 12:24 AM To: axis-user@ws.apache.org Subject: RE: WS debugging in Eclipse Hi Rajith, Im using tcp monitor,but sometimes I need to step into code. Now I make it so,that i construct skeleton class direct and run my web service as local application. But sometimes I need to debbug it on Tomcat as it would run in bussiness with all settings. Regards Vladi From: Rajith Attapattu [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 06, 2006 7:36 PM To: axis-user@ws.apache.org Subject: Re: WS debugging in Eclipse Vladi, Another more easy option would be for you to deploy your services on axis2 and then remote debug using eclipse. You can also use tcpmon http://ws.apache.org/commons/tcpmon/ to check the SOAP messages that are exchanged between the service and the client. Regards, Rajith On 12/6/06, VF [EMAIL PROTECTED] wrote: Hi all, Is it possible to debug web services in Eclipse? Vladi - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[Axis2] SAX or DOM
In Axis2, is there a way for the server to start receiving a very long request, before the entire request has been transmitted? Similarly, is there a way for the client to start receiving a very long response, before the entire response has been transmitted? Perhaps this comes down to asking whether SAX or DOM is used for XML parsing in Axis2. Thanks. Betsy Frey ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: about Axis2's binary attachment
One does not have to use an ImageDataSource or FileDataSource. You can code your own DataSource, implementing javax.activation.DataSource. When receiving MTOM data, one gets a DataHandler from the OMText element, and one can get the input stream from that: OMText data = (OMText) child.getFirstOMChild(); DataHandler dh = (DataHandler) data.getDataHandler(); InputStream istream = dh.getDataSource().getInputStream(); An intermediate file may still be created. This is because, when transferring binary data using MTOM, there is the issue that a very large amount of data could cause an OutOfMemoryError, without storing it on disk as it is being received. One controls this by these configuration settings: parameter name=cacheAttachments locked=falsetrue/parameter parameter name=attachmentDIR locked=falsetemp directory/parameter parameter name=sizeThreshold locked=false4000/parameter Betsy From: scott ken [mailto:[EMAIL PROTECTED] Sent: Thursday, September 28, 2006 7:53 PM To: axis-user@ws.apache.org Subject: Re: about Axis2's binary attachment Usually we will have an InputStream object. But I didn't make it work. I tried using a byte array as input. you can easily convert an InputStream to a byte array. After that, use ByteArrayDataSource. // Build OMText to hold binary data byte[] binaryBytes = ...; ByteArrayDataSource dataSource = new ByteArrayDataSource(binaryBytes ); DataHandler expectedDH = new DataHandler(dataSource); OMText textData = omFactory.createOMText(expectedDH, true); On 9/28/06, sam wang [EMAIL PROTECTED] wrote: Hi, there, I have a question about transferring binary attachment through Axis2. I want to transfer binary attachment from server side to client side. My question is that if I can attach any kind of binary data? from Axis2's api, I see there are only two datasource: file and image. theoritically, Axis2 can transfer any kind of object because we can first serialize any object to disk, save it as a file, and then construct file datasource to attach into soap as binary attachment. but it will not be efficient. so my question is that if there is a way to attach the object (for example, CachedRowSet) to soap directly and return to client, instead of saving as file first. if it can, could anyone give me a java snippet? thank you very much! Sam __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Axis using more than 256MB for a 9MB file
Hi Thilina, Are you using the Axis2 tip? I ask because when I use MTOM in Axis2 v 1.0, with file caching on, transferring 1gb of data causes my tomcat 5.5.17 server to see: java.lang.OutOfMemoryError: Java heap space. Thanks, Betsy -Original Message- From: Thilina Gunarathne [mailto:[EMAIL PROTECTED] Sent: Friday, September 15, 2006 11:51 PM To: axis-user@ws.apache.org Subject: Re: Axis using more than 256MB for a 9MB file Try Axis2 with File Caching Switched On.. We have tested Axis2 with files ~1GB without much hassel. IIRC Axis1.x too has the file caching option.. ~Thilina On 9/15/06, Neil Aggarwal [EMAIL PROTECTED] wrote: Hello: Does Axis2 solve these problems? Thanks, Neil -- Neil Aggarwal, (214)986-3533, www.JAMMConsulting.com FREE! Eliminate junk email and reclaim your inbox. Visit http://www.spammilter.com for details. -Original Message- From: Neil Aggarwal [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 13, 2006 4:43 PM To: axis-user@ws.apache.org Subject: Axis using more than 256MB for a 9MB file Hello: I have a web service deployed using Axis 1.4 In that service, I have a method that accepts a String containing an XML document. When I try to upload a 9MB XML file into my service, Axis seems to be taking up a HUGE amount of memory. If I set the JVM to 256MB of heap, it runs out of memory and throws an OutOfMemoryException. If I set the JVM to 512MB of heap, I get the file just fine. The problem occurs *before* my web service method is even invoked. The service is running on Tomcat on Windows. Any ideas why Axis would consume so much memory? Thanks, Neil -- Neil Aggarwal, (214)986-3533, www.JAMMConsulting.com FREE! Eliminate junk email and reclaim your inbox. Visit http://www.spammilter.com for details. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- May the SourcE be with u http://webservices.apache.org/~thilina/ http://thilinag.blogspot.com/ http://www.bloglines.com/blog/Thilina - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Axis using more than 256MB for a 9MB file
Hi Thilina, I retract my question; sorry for the bother. My note below was incorrect; file caching was not on. I did see a client timeout when transferring 1gb of data using MTOM in Axis2 v 1.0, but I'm thinking that I need to configure a larger timeout value. Betsy -Original Message- From: Betsy Frey Sent: Saturday, September 16, 2006 3:29 AM To: 'axis-user@ws.apache.org' Subject: RE: Axis using more than 256MB for a 9MB file Hi Thilina, Are you using the Axis2 tip? I ask because when I use MTOM in Axis2 v 1.0, with file caching on, transferring 1gb of data causes my tomcat 5.5.17 server to see: java.lang.OutOfMemoryError: Java heap space. Thanks, Betsy -Original Message- From: Thilina Gunarathne [mailto:[EMAIL PROTECTED] Sent: Friday, September 15, 2006 11:51 PM To: axis-user@ws.apache.org Subject: Re: Axis using more than 256MB for a 9MB file Try Axis2 with File Caching Switched On.. We have tested Axis2 with files ~1GB without much hassel. IIRC Axis1.x too has the file caching option.. ~Thilina On 9/15/06, Neil Aggarwal [EMAIL PROTECTED] wrote: Hello: Does Axis2 solve these problems? Thanks, Neil -- Neil Aggarwal, (214)986-3533, www.JAMMConsulting.com FREE! Eliminate junk email and reclaim your inbox. Visit http://www.spammilter.com for details. -Original Message- From: Neil Aggarwal [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 13, 2006 4:43 PM To: axis-user@ws.apache.org Subject: Axis using more than 256MB for a 9MB file Hello: I have a web service deployed using Axis 1.4 In that service, I have a method that accepts a String containing an XML document. When I try to upload a 9MB XML file into my service, Axis seems to be taking up a HUGE amount of memory. If I set the JVM to 256MB of heap, it runs out of memory and throws an OutOfMemoryException. If I set the JVM to 512MB of heap, I get the file just fine. The problem occurs *before* my web service method is even invoked. The service is running on Tomcat on Windows. Any ideas why Axis would consume so much memory? Thanks, Neil -- Neil Aggarwal, (214)986-3533, www.JAMMConsulting.com FREE! Eliminate junk email and reclaim your inbox. Visit http://www.spammilter.com for details. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- May the SourcE be with u http://webservices.apache.org/~thilina/ http://thilinag.blogspot.com/ http://www.bloglines.com/blog/Thilina - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is SOAP appropriate for large data transmission
SOAP without an attachment is inappropriate for sending large amounts of binary data, because the XML requires it to be converted to Base64, which is inefficient. With an attachment, SOAP over http has the overhead of the protocol, which is not as fast as ftp, for instance. I have used DIME attachments in Axis 1.1. They worked fine up to some limit, 2gb or so as I recall; 2 ^ 31 or 32. Betsy -Original Message- From: Rodrigo Ruiz [mailto:[EMAIL PROTECTED] Sent: Thursday, June 22, 2006 2:51 AM To: axis-user@ws.apache.org Subject: Is SOAP appropriate for large data transmission Hi all, I usually read in this forum that web services and SOAP are not appropriate for large data transmission, but I have never read about any good technical reasons behind these sentences. Most of the issues I have seen in this matter are related to the SOAP stack implementation, rather than to SOAP itself: - Progress bar for file download: You can split the download in smaller chunks and transmit them using several calls. It should be possible to modify the attachments implementation of SOAP and add the logic for asynchronous status notification, so it is technically possible to track the progress of attachments download/upload. - SOAP XML overhead: In fact, the overhead of a SOAP envelope containing just a reference to the attachment part is more important for small attachments than for large ones. Just one or two kilobytes of XML are not too much when you are trying to transfer an 8GB stream. - Partial/broken transmissions: Again, if the attachments implementation is able to track the progress of a download, it may be possible to resume it after a transmission error. We would just need to be able to specify the range we are interested in, just like with old plain HTML. Could somebody shed some light on the reasons that make SOAP inappropriate for this task? Regards, Rodrigo Ruiz -- --- GRIDSYSTEMSRodrigo Ruiz Aguayo Parc Bit - Son Espanyol 07120 Palma de Mallorcamailto:[EMAIL PROTECTED] Baleares - España Tel:+34-971435085 Fax:+34-971435082 http://www.gridsystems.com --- -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.394 / Virus Database: 268.9.2/372 - Release Date: 21/06/2006 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is SOAP appropriate for large data transmission
Hi German, I have not tried sending attachments with Axis2. Betsy -Original Message- From: German Sakaryan [mailto:[EMAIL PROTECTED] Sent: Thursday, June 22, 2006 4:43 AM To: axis-user@ws.apache.org Subject: Re: Is SOAP appropriate for large data transmission Hi,Betsey have tried to send large files with axis2 ? I get fast always broken pipe exception. German Sakaryan Betsy Frey schrieb: SOAP without an attachment is inappropriate for sending large amounts of binary data, because the XML requires it to be converted to Base64, which is inefficient. With an attachment, SOAP over http has the overhead of the protocol, which is not as fast as ftp, for instance. I have used DIME attachments in Axis 1.1. They worked fine up to some limit, 2gb or so as I recall; 2 ^ 31 or 32. Betsy -Original Message- From: Rodrigo Ruiz [mailto:[EMAIL PROTECTED] Sent: Thursday, June 22, 2006 2:51 AM To: axis-user@ws.apache.org Subject: Is SOAP appropriate for large data transmission Hi all, I usually read in this forum that web services and SOAP are not appropriate for large data transmission, but I have never read about any good technical reasons behind these sentences. Most of the issues I have seen in this matter are related to the SOAP stack implementation, rather than to SOAP itself: - Progress bar for file download: You can split the download in smaller chunks and transmit them using several calls. It should be possible to modify the attachments implementation of SOAP and add the logic for asynchronous status notification, so it is technically possible to track the progress of attachments download/upload. - SOAP XML overhead: In fact, the overhead of a SOAP envelope containing just a reference to the attachment part is more important for small attachments than for large ones. Just one or two kilobytes of XML are not too much when you are trying to transfer an 8GB stream. - Partial/broken transmissions: Again, if the attachments implementation is able to track the progress of a download, it may be possible to resume it after a transmission error. We would just need to be able to specify the range we are interested in, just like with old plain HTML. Could somebody shed some light on the reasons that make SOAP inappropriate for this task? Regards, Rodrigo Ruiz - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Axis2: can client read from open stream?
Is there a way in which Axis 2 will allow the client to start receiving the response while the server is still adding children to the SOAP response? Thanks you. Betsy Frey ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
