Thanks for the reply, Unfortunately, zipping the messages is not an option for us... We have cnosidered it but it is not possible, mainly due to security issues...
Note that if signing a 100MB message takes even 500MB of ram but they are released once the message is sent - this is something we can manage, we can make sure no more than 100MB of messages are signed per server per unit time and so make sure no out of memory will occur... So even if WSS4J will need a lot of memory to sign, but will be released once the message is sent, it will suit us well. Idan. ----- Original Message ----- From: "Werner Dittmann" <[EMAIL PROTECTED]> To: "Granqvist, Hans" <[EMAIL PROTECTED]> Cc: "Idan Miller" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>; <[email protected]> Sent: Friday, September 02, 2005 9:39 PM Subject: Re: Using WSS4J for signing large XML messages > Just some thoughts: if we are going to sign such a large XML then > IMO the Signature class needs to read in the whole data set > to compute the digest. This of course depends which part > of the message needs to be signed. If the body contains > this full 100MB and you sign the body - then the Signature > class needs to compute the digest for this 100MB. IMHO > deferred DOM processing wouldn't help here, also a SAX > style parsing (or pull parsing) wouldn't help much. > > Also a full serialization of the SOAP request must take > place before signing (or encrypting) the whole thing. After > signing the signed content must not be modified in any > way. This last requirement by the way was and still is one of > the real tricky things we do in the handlers for Axis. We needed > a modification of Axis code to overcome this problem. > > The SOAP engine must not perform an additional serialization pass, > as this may lead to content modification, e.g. new linefeeds, blanks, > or even worse additional data elements could be inserted. > > This is one reason why WSS4J takes the fully serialized SOAP request > from Axis (or JAX-RPC implementation), performs all necessary > security actions, adds wsu:Id attributes, etc. and sets up > a new SOAP request and serializes it. This new, security enhanced, > serialized SOAP request is then given back to Axis to sent it over > the wire (or any other medium, e.g. air carriers :-) ). > > Thus, I personally would go another way: construct the XML > document by some other means, ZIP it, perform a digest over > this ZIP file. Then define a method that accepts the digest, > and maybe a URL of the file (as an alternative you may send > the file as attachment (SwA)). Use Signature/encryption for > this short SOAP request. This is more difficult in the first > run, but IMO performs much better in the long run > than signing a 100MB XML SOAP request. > > Regards, > Werner > > > Granqvist, Hans wrote: > > Can you provide an example message without disclosing anything > > private? How big is it zipped up -- can you email it? > > > > I'm curious about the depth of the DOM, that is, the level of > > nested elements, its flatness and the like. Normally deferred > > DOM processing can handle huge documents, but that breaks > > if the c14n process constantly backtracks up the ancestor axis > > to collect namespaces. > > > > (In a different world, in a different project ;) someone solved > > a similar problem by sending signatures over their own hashes > > of the document, and of course that works but is logistically > > tedious.) > > > > Hans > > > > > >>-----Original Message----- > >>From: Idan Miller [mailto:[EMAIL PROTECTED] > >>Sent: Friday, September 02, 2005 4:17 AM > >>To: [EMAIL PROTECTED] > >>Cc: [email protected] > >>Subject: Re: Using WSS4J for signing large XML messages > >> > >>The hardware is a server with 4GB of memory and 2 processors. > >>We would love for you to test and see if WSS4J handles large > >>messages. We are most interested in seeing if the memory gets > >>released once the message has done processing... > >> > >>Thanks, > >>Idan. > >>----- Original Message ----- > >>From: "Davanum Srinivas" <[EMAIL PROTECTED]> > >>To: "Idan Miller" <[EMAIL PROTECTED]> > >>Cc: <[email protected]> > >>Sent: Thursday, September 01, 2005 3:20 AM > >>Subject: Re: Using WSS4J for signing large XML messages > >> > >> > >>very interesting...what sort of hardware do you run this on currently? > >>we haven't tested this large messages. internally yes we use DOM. > >>Xerces does have some deferred load capabilities, depends on the jvm > >>performance as well. would be worth our time to try this out and see > >>if we can help you. let us know. If we see stuff that fails, am > >>positive we can get it fixed one way or another (since all components > >>in wss4j are open source as well) > >> > >>thanks, > >>dims > >> > >>On 8/31/05, Idan Miller <[EMAIL PROTECTED]> wrote: > >> > >>>Hi everyone, > >>> > >>>Our project uses web services to transfer XML messages that > >> > >>can be of a > >>very > >> > >>>large size (up to 100MB). > >>> > >>>Currently, we are using WSE over IIS for verifying the signing for > >> > >>incoming > >> > >>>messages. Due to a problem with WSE loading the XML using a > >> > >>DOM Document > >> > >>>(probably due to cannonization), combined with memory being > >> > >>held in the > >> > >>>large object heap in .NET, we are unable to transfer very > >> > >>large messages, > >> > >>>since we simply run out of memory that doesn't get released. > >>> > >>>Now that WSS4J is out, we would like to know if this problem will be > >> > >>solved > >> > >>>by changing our architecture to use apache axis and WSS4J: > >>> > >>>How does WSS4J handle large messages? > >>>Does it use DOM or does it use a reader (SAX) for cannonization? > >>>Also, if it does use DOM, has it been tested for out of > >> > >>memory errors? > >> > >>>p.s. we are using X509 certificates for the signing. > >>> > >>>Thanks, > >>>Idan. > >> > >> > >>-- > >>Davanum Srinivas : http://wso2.com/ - Oxygenating The Web > >>Service Platform > >> > >> > >>--------------------------------------------------------------------- > >>To unsubscribe, e-mail: [EMAIL PROTECTED] > >>For additional commands, e-mail: [EMAIL PROTECTED] > >> > >> > >> > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
