RE: sending very large attachments

[Liu, Scott]

Hi, Mike,

I have been considering supporting file attachments just like the way
you have described. The schema will only contain a list of file names
and other file attributes. Intuitively as long as Axis can handle the
attachment I will be able to retrieve them from the MessageContext
object on the server.
Since you have already done it you may help me to understand how it
works.

1. How would Axis know that it needs to process attachments if there is
no indication in the schema? Does it know whether or not the attachment
is encrypted in MIME or DIME format?

2. Is your change on attachment directory made in the Axis code and is
there any change in Axis code required to make attachment work?

On a separate note but related I have a web service (Axis 1.1) to send
messages from a client to a server. It works without any attachment.
Once I attached a simple text file using a .NET client Axis failed with
a NullPointerException in parsing the soap document. This was what has
prompted me to think whether or not I should somehow tell Axis that
there is a file attached in a coming message. My web service is of
doc/literal.

Thanks,

Scott

P.S. In case someone happens to know what may be the problem here is the
stacktrace:

 soapenv:Body  
soapenv:Fault   
   faultcodesoapenv:Server.userException/faultcode   
   faultstringjava.lang.NullPointerException/faultstring

   detail
  ns1:stackTrace
xmlns:ns1=http://xml.apache.org/axis/;java.lang.NullPointerException
at java.io.FilterInputStream.available(FilterInputStream.java:146)
at oracle.xml.jaxp.JXSAXParser.parse(JXSAXParser.java:277)  at
org.apache.axis.encoding.DeserializationContextImpl.parse(Deserializatio
nContextImpl.java:243)  at
org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:538)   at
org.apache.axis.Message.getSOAPEnvelope(Message.java:376)   at
org.apache.axis.server.AxisServer.invoke(AxisServer.java:318)   at
org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:854)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760) at
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.j
ava:339)at
javax.servlet.http.HttpServlet.service(HttpServlet.java:853)at 
com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(Releasab
leResourcePooledExecutor.java:192)  at
java.lang.Thread.run(Thread.java:534)/ns1:stackTrace   
   /detail  
/soapenv:Fault 
 /soapenv:Body


-Original Message-
From: Vivek Singhai [mailto:[EMAIL PROTECTED] 
Sent: Saturday, April 09, 2005 9:30 AM
To: axis-user@ws.apache.org
Subject: RE: sending very large attachments

We also have a similar requirement to send large attachments and they
also
should not be as a separate parameter in the SOAP message 
But we are using Axis 1.1 Release verion, My question is Does this piece
of
code holds good in Axis 1.1??

On the client side you can set chunked encoding by setting the transport

to http 1.1, and adding chunked encoding to the transfer headers

Hashtable chunkedTable = new Hashtable();
chunkedTable.put(HTTPConstants.HEADER_TRANSFER_ENCODING,
 
HTTPConstants.HEADER_TRANSFER_ENCODING_CHUNKED);

call.setProperty(MessageContext.HTTP_TRANSPORT_VERSION,
 HTTPConstants.HEADER_PROTOCOL_V11);
call.setProperty(HTTPConstants.REQUEST_HEADERS,chunkedTable);

call.addAttachmentPart(
 new DataHandler(new FileDataSource(file)));

Thanks,
Vivek


-Original Message-
From: Sai Giddu [mailto:[EMAIL PROTECTED] 
Sent: Friday, April 08, 2005 6:47 PM
To: axis-user@ws.apache.org
Subject: RE: sending very large attachments


Thanks..
This would be helpful.

Cheers,
Sai


-Original Message-
From: Mike Smorul [mailto:[EMAIL PROTECTED]
Sent: 08 April 2005 14:03
To: axis-user@ws.apache.org
Subject: RE: sending very large attachments



Section 3.6 in http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html
does 
a better job that I could of explaining transfer codings.

One difference that we did to handle large encodings was to divorce the 
attachments from actual message, ala no DataHandlers as parameters. 
Instead we send an array of filenames that correspond to the
attachments. 
Normally, this loose coupling of attachments and parameter wouldn't be a

good idea, but we didn't want to lock ourselves into using attachments
for 
a bulk data transport, and we can verify attachments through previously 
passed checksums.

On the client side you can set chunked encoding by setting the transport

to http 1.1, and adding chunked encoding to the transfer headers

Hashtable chunkedTable = new Hashtable();
chunkedTable.put(HTTPConstants.HEADER_TRANSFER_ENCODING,
 
HTTPConstants.HEADER_TRANSFER_ENCODING_CHUNKED);

call.setProperty(MessageContext.HTTP_TRANSPORT_VERSION,
 HTTPConstants.HEADER_PROTOCOL_V11

RE: sending very large attachments

[Vivek Singhai]

We also have a similar requirement to send large attachments and they also
should not be as a separate parameter in the SOAP message 
But we are using Axis 1.1 Release verion, My question is Does this piece of
code holds good in Axis 1.1??

On the client side you can set chunked encoding by setting the transport 
to http 1.1, and adding chunked encoding to the transfer headers

Hashtable chunkedTable = new Hashtable();
chunkedTable.put(HTTPConstants.HEADER_TRANSFER_ENCODING,
 HTTPConstants.HEADER_TRANSFER_ENCODING_CHUNKED);

call.setProperty(MessageContext.HTTP_TRANSPORT_VERSION,
 HTTPConstants.HEADER_PROTOCOL_V11);
call.setProperty(HTTPConstants.REQUEST_HEADERS,chunkedTable);

call.addAttachmentPart(
 new DataHandler(new FileDataSource(file)));

Thanks,
Vivek


-Original Message-
From: Sai Giddu [mailto:[EMAIL PROTECTED] 
Sent: Friday, April 08, 2005 6:47 PM
To: axis-user@ws.apache.org
Subject: RE: sending very large attachments


Thanks..
This would be helpful.

Cheers,
Sai


-Original Message-
From: Mike Smorul [mailto:[EMAIL PROTECTED]
Sent: 08 April 2005 14:03
To: axis-user@ws.apache.org
Subject: RE: sending very large attachments



Section 3.6 in http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html does 
a better job that I could of explaining transfer codings.

One difference that we did to handle large encodings was to divorce the 
attachments from actual message, ala no DataHandlers as parameters. 
Instead we send an array of filenames that correspond to the attachments. 
Normally, this loose coupling of attachments and parameter wouldn't be a 
good idea, but we didn't want to lock ourselves into using attachments for 
a bulk data transport, and we can verify attachments through previously 
passed checksums.

On the client side you can set chunked encoding by setting the transport 
to http 1.1, and adding chunked encoding to the transfer headers

Hashtable chunkedTable = new Hashtable();
chunkedTable.put(HTTPConstants.HEADER_TRANSFER_ENCODING,
 HTTPConstants.HEADER_TRANSFER_ENCODING_CHUNKED);

call.setProperty(MessageContext.HTTP_TRANSPORT_VERSION,
 HTTPConstants.HEADER_PROTOCOL_V11);
call.setProperty(HTTPConstants.REQUEST_HEADERS,chunkedTable);

call.addAttachmentPart(
 new DataHandler(new FileDataSource(file)));

On the server side, by not including datahandlers as parameters, Axis 
ony reads attachments on demand. This lets us toggle the location where 
Axis writes the Axis*att files.

MessageContext msgContext  = MessageContext.getCurrentContext();
msgContext.setProperty(
MessageContext.ATTACHMENTS_DIR,newPath);

// causes Axis to read in rest of attachment stream
IteratorAttachmentPart iap = reqMsg.getAttachments();

-Mike

On Fri, 8 Apr 2005, Sai Giddu wrote:

 Mike,
Could you please elaborate more on chunked-encoding. I'm facing
similar performance issues while trying to send large data using SOAP 
protocol.

 Thanks,
 Sai

 -Original Message-
 From: Mike Smorul [mailto:[EMAIL PROTECTED]
 Sent: 07 April 2005 22:52
 To: 'axis-user@ws.apache.org'
 Subject: RE: sending very large attachments



 Most implementations tend to choke on large http transfers because 
 they tend to buffer the entire transmission prior to sending. Usually 
 done under the assumption you are transfering single web pages and not 
 large files.

 We've have fairly good results at sending large (1G) attachments 
 through axis. This was done using chunked-encoding and redirecting the 
 axis attachment directory per attachment to avoid recopying data into 
 a final location. Although when dealing with large attachments, you 
 should consider implementing some type of checkpointing during the 
 transfer in case of failure.

 -Mike

 On Thu, 7 Apr 2005, THOMAS, JAI [AG-Contractor/1000] wrote:

 More than Axis, it would be a problem with HTTP assuming you are 
 using http transfer. HTTP has a limitation on size that varies 
 slightly by implementation but from experience, anything over 10meg 
 would be a problem.

 Jai



 -Original Message-
 From: Alex Milanovic [mailto:[EMAIL PROTECTED]
 Sent: Thursday, April 07, 2005 4:33 PM
 To: axis-user@ws.apache.org
 Subject: sending very large attachments


 Hi All,
 I was wondering if it would make sense to use the SOAP attachment 
 method for sending very large files from one host to another over the 
 Internet? How would AXIS deal with a file of 1GB in size? Alex




_
This message and any attachments are intended only for the use of the
addressee and may contain information that is privileged and confidential.
If the reader of the 
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have

RE: sending very large attachments

[Sai Giddu]

Mike,
Could you please elaborate more on chunked-encoding. I'm facing similar 
performance issues while trying to send large data using SOAP protocol.

Thanks,
Sai

-Original Message-
From: Mike Smorul [mailto:[EMAIL PROTECTED]
Sent: 07 April 2005 22:52
To: 'axis-user@ws.apache.org'
Subject: RE: sending very large attachments



Most implementations tend to choke on large http transfers because they 
tend to buffer the entire transmission prior to sending. Usually done 
under the assumption you are transfering single web pages and not large 
files.

We've have fairly good results at sending large (1G) attachments through 
axis. This was done using chunked-encoding and redirecting the 
axis attachment directory per attachment to avoid recopying data 
into a final location. Although when dealing with large attachments, you 
should consider implementing some type of checkpointing during the 
transfer in case of failure.

-Mike

On Thu, 7 Apr 2005, THOMAS, JAI [AG-Contractor/1000] wrote:

 More than Axis, it would be a problem with HTTP assuming you are using http 
 transfer.
 HTTP has a limitation on size that varies slightly by implementation but from 
 experience, anything over 10meg would be a problem.

 Jai



 -Original Message-
 From: Alex Milanovic [mailto:[EMAIL PROTECTED]
 Sent: Thursday, April 07, 2005 4:33 PM
 To: axis-user@ws.apache.org
 Subject: sending very large attachments


 Hi All,
 I was wondering if it would make sense to use the SOAP attachment method for
 sending very large files from one host to another over the Internet? How
 would AXIS deal with a file of 1GB in size?
 Alex

RE: sending very large attachments

[Mike Smorul]

Section 3.6 in http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html does 
a better job that I could of explaining transfer codings.

One difference that we did to handle large encodings was to divorce the 
attachments from actual message, ala no DataHandlers as parameters. 
Instead we send an array of filenames that correspond to the attachments. 
Normally, this loose coupling of attachments and parameter wouldn't be a 
good idea, but we didn't want to lock ourselves into using attachments for 
a bulk data transport, and we can verify attachments through previously 
passed checksums.

On the client side you can set chunked encoding by setting the transport 
to http 1.1, and adding chunked encoding to the transfer headers

   Hashtable chunkedTable = new Hashtable();
   chunkedTable.put(HTTPConstants.HEADER_TRANSFER_ENCODING,
HTTPConstants.HEADER_TRANSFER_ENCODING_CHUNKED);
   call.setProperty(MessageContext.HTTP_TRANSPORT_VERSION,
HTTPConstants.HEADER_PROTOCOL_V11);
   call.setProperty(HTTPConstants.REQUEST_HEADERS,chunkedTable);
   call.addAttachmentPart(
new DataHandler(new FileDataSource(file)));
On the server side, by not including datahandlers as parameters, Axis 
ony reads attachments on demand. This lets us toggle the location where 
Axis writes the Axis*att files.

   MessageContext msgContext  = MessageContext.getCurrentContext();
   msgContext.setProperty(
   MessageContext.ATTACHMENTS_DIR,newPath);
   // causes Axis to read in rest of attachment stream
   IteratorAttachmentPart iap = reqMsg.getAttachments();
-Mike
On Fri, 8 Apr 2005, Sai Giddu wrote:
Mike,
   Could you please elaborate more on chunked-encoding. I'm facing 
similar performance issues while trying to send large data using SOAP 
protocol.
Thanks,
Sai
-Original Message-
From: Mike Smorul [mailto:[EMAIL PROTECTED]
Sent: 07 April 2005 22:52
To: 'axis-user@ws.apache.org'
Subject: RE: sending very large attachments

Most implementations tend to choke on large http transfers because they
tend to buffer the entire transmission prior to sending. Usually done
under the assumption you are transfering single web pages and not large
files.
We've have fairly good results at sending large (1G) attachments through
axis. This was done using chunked-encoding and redirecting the
axis attachment directory per attachment to avoid recopying data
into a final location. Although when dealing with large attachments, you
should consider implementing some type of checkpointing during the
transfer in case of failure.
-Mike
On Thu, 7 Apr 2005, THOMAS, JAI [AG-Contractor/1000] wrote:
More than Axis, it would be a problem with HTTP assuming you are using http 
transfer.
HTTP has a limitation on size that varies slightly by implementation but from 
experience, anything over 10meg would be a problem.
Jai

-Original Message-
From: Alex Milanovic [mailto:[EMAIL PROTECTED]
Sent: Thursday, April 07, 2005 4:33 PM
To: axis-user@ws.apache.org
Subject: sending very large attachments
Hi All,
I was wondering if it would make sense to use the SOAP attachment method for
sending very large files from one host to another over the Internet? How
would AXIS deal with a file of 1GB in size?
Alex

RE: sending very large attachments

[THOMAS, JAI [AG-Contractor/1000]]

More than Axis, it would be a problem with HTTP assuming you are using http 
transfer.
HTTP has a limitation on size that varies slightly by implementation but from 
experience, anything over 10meg would be a problem.

Jai



-Original Message-
From: Alex Milanovic [mailto:[EMAIL PROTECTED]
Sent: Thursday, April 07, 2005 4:33 PM
To: axis-user@ws.apache.org
Subject: sending very large attachments


Hi All,
I was wondering if it would make sense to use the SOAP attachment method for
sending very large files from one host to another over the Internet? How
would AXIS deal with a file of 1GB in size?
Alex

RE: sending very large attachments

[Mike Smorul]

Most implementations tend to choke on large http transfers because they 
tend to buffer the entire transmission prior to sending. Usually done 
under the assumption you are transfering single web pages and not large 
files.

We've have fairly good results at sending large (1G) attachments through 
axis. This was done using chunked-encoding and redirecting the 
axis attachment directory per attachment to avoid recopying data 
into a final location. Although when dealing with large attachments, you 
should consider implementing some type of checkpointing during the 
transfer in case of failure.

-Mike
On Thu, 7 Apr 2005, THOMAS, JAI [AG-Contractor/1000] wrote:
More than Axis, it would be a problem with HTTP assuming you are using http 
transfer.
HTTP has a limitation on size that varies slightly by implementation but from 
experience, anything over 10meg would be a problem.
Jai

-Original Message-
From: Alex Milanovic [mailto:[EMAIL PROTECTED]
Sent: Thursday, April 07, 2005 4:33 PM
To: axis-user@ws.apache.org
Subject: sending very large attachments
Hi All,
I was wondering if it would make sense to use the SOAP attachment method for
sending very large files from one host to another over the Internet? How
would AXIS deal with a file of 1GB in size?
Alex