DO NOT REPLY [Bug 36526] - Out of memory error when signing or verifying big files

[bugzilla]

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=36526>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=36526





------- Additional Comments From [EMAIL PROTECTED]  2005-09-19 17:41 -------
Created an attachment (id=16449)
 --> (http://issues.apache.org/bugzilla/attachment.cgi?id=16449&action=view)
Solution idea

I have an idea to solve signing and verifying big files without Base64
transformation and without OutOfMemoryError.



1) In XMLSignatureInput.java should be a new member:

        File InputFile = null;

2) In XMLSignatureInput.java should be a new constructor:

        public XMLSignatureInput(File inputfile)
        {
                InputFile = inputfile;
        }

3) In XMLSignatureInput.java, updateOutputStream method should be modified:

        /**
         * @param diOs
         * @throws IOException
         * @throws CanonicalizationException
         */
        public void updateOutputStream(OutputStream diOs) throws
CanonicalizationException, IOException {        
        if (diOs==outputStream) {
                return;
        }
        if (bytes!=null) {
            diOs.write(bytes);
            return;      
         }else if (this.isElement()) {                    
             Canonicalizer20010315OmitComments c14nizer =
                    new Canonicalizer20010315OmitComments();       
             c14nizer.setWriter(diOs);
            c14nizer.engineCanonicalizeSubTree(this._subNode,this.excludeNode);

            return;
          } else if (this.isNodeSet()) {        
             /* If we have a node set but an octet stream is needed, we MUST
c14nize
              * without any comments.
              *
              * We don't use the factory because direct instantiation should be
a
              * little bit faster...
              */
             Canonicalizer20010315OmitComments c14nizer =
                new Canonicalizer20010315OmitComments();         
             c14nizer.setWriter(diOs);
             if (this._inputNodeSet.size() == 0) {
                // empty nodeset
                return;
             }                              
             c14nizer.engineCanonicalizeXPathNodeSet(this._inputNodeSet);      
         
             return;             
          } else {

/* --> begin of new lines */
            if ( InputFile != null )
            {
              byte[] buffer = new byte[32*1024];
              BufferedInputStream bis = null;
              try
              {
                bis = new BufferedInputStream(new FileInputStream(InputFile));
                int bytesread = 0;
                while ( (bytesread = bis.read(buffer)) != -1 )
                {
                  diOs.write(buffer,0,bytesread);
                }
              }
              finally
              {
                if ( bis != null )
                {
                  bis.close();
                  bis = null;
                }
              }
              return;
            }
/* <-- end of new lines */

            InputStream is = getResetableInputStream();
            if (bytes!=null) {
                //already read write it, can be rea.
                diOs.write(bytes,0,bytes.length);
                return;
            }            
            is.reset();            
            int num;
            byte[] bytesT = new byte[1024];
            while ((num=is.read(bytesT))>0) {
                diOs.write(bytesT,0,num);
            }
                
          }
                
        }


4) In ResolverLocalFileSystem.java, line 65 and 66 should be modified:

/* original, line 65-66:
        FileInputStream inputStream = new FileInputStream(fileName);
        XMLSignatureInput result = new XMLSignatureInput(inputStream);
*/
/* new, line 65: */
        XMLSignatureInput result = new XMLSignatureInput(new File(fileName));




It is only an idea, not a real patch. I tried it with signing and verifying 
two big files (jdk-1_5_0-doc.zip, size=45635523 and lnx_920_disk2.cpio.gz,
size=588798999) and it works fine in this case:

<ds:SignedInfo Id="BigFile_2_Signature-1__SignedInfo-1">
  <ds:CanonicalizationMethod
Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"; /> 
  <ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"; />

  <ds:Reference Id="BigFile_2_Signature-1__Reference-1"
Type="http://uri.etsi.org/01903/v1.2.2#SignedProperties";
URI="#BigFile_2_Signature-1__SignedProperties-1">
  <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"; /> 
  <ds:DigestValue>NmZTo8j7D61PP1opL1HTs2YphN0=</ds:DigestValue> 
  </ds:Reference>
  <ds:Reference Id="BigFile_2_Signature-1__Reference-2"
URI="file:jdk-1_5_0-doc.zip">
  <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"; /> 
  <ds:DigestValue>HZqRglwK5G0XFQ36wdGzx00w+kQ=</ds:DigestValue> 
  </ds:Reference>
  <ds:Reference Id="BigFile_2_Signature-1__Reference-3"
URI="file:lnx_920_disk2.cpio.gz">
  <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"; /> 
  <ds:DigestValue>gaHz/04SteF1VCpuLY/+fkWksT8=</ds:DigestValue> 
  </ds:Reference>
</ds:SignedInfo>


Maybe something like this idea should be useful for other people or should be
applied in a new version of the library.

-- 
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.