Is there some way to break up the processing? I noticed that signing large
documents takes about
20-40 times memory. This seems very inefficient. They should be a flag or
way to process large
documents. For example, GNU soap can use DIME to stream data such that it
can be broken up.
It can send and receive arbitrarily large documents.
Bill
----- Original Message -----
From: <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Thursday, May 13, 2004 2:59 PM
Subject: What's next? (Some advice needed).
> Hi,
> Now that I the patches of c14n has been included I have detected several
> places where some improvements can he done:
> 1. Refactor the org.apache.xml.security.Init: Right now is very slow
> in what it does. The use of XpathApi makes it very slow, I can rewrite
> it to use DOM, and use internal structure objects to reduce the memory
> footprint. Also it can be good to have another method of initialization
> (like Init.initGreedy() ) that do the expensive steps so it doesn't need
> to happened at execution time. It can also be correct so it compile
> safely with the current CVS version of xalan (and get rid of the
> integration failed messages in the maillist).
> 2. Reduce memory footprint of the library: It seems that the big xml
> files or not well handled by the current version. During my refactoring
> of c14n I see that the c14n bytes are copy back and forth (for example
> for digesting a byte array is copy 3 times before the final digest).
> This in big documents can be expensive in terms of memory. A refactoring
> avoiding these copies can be done easily.
> 3. Refactor the CachedXPathApi handling: It seems that the way that
> the cachedxpathapi is cache in the case is needed (like in
> transformXPath) is not very efficient. A simple (but incorrect although
> makes all test passed) patch makes the runtime of all test goes from
> 61.43sec to 43.9 sec. Perhaps make the CachedXPathApi cached for
> document or for manifest instead for transformation can improve the
> things in this regard.
> 4. Keep refactoring the c14n: it seems that the NameSpaceSymb that I
> use in c14nInclusive with xpath is not the better structure for this
> c14n method. Perhaps a more simple approach can speed up the things a
> little and reduce the memory footprint in this particular case. Other
> improvements can be do the xpath selection and c14n at the same time.
> This can makes avoid the circumvent bug in all c14n methods (right now
> is only needed with an xpath transformation), this can reduces the
> memory footprint and time at the same time. Or the way that the c14n
> creates the byte array using a Printer over a bytearray can be speed-up
> using a string buffer, or ad hoc string buffer(like xerces or xalan
> seems to do).
> 5. See synergies between other Apache projects: For example the our
> B64 decoder relays on stringtokenizer(this method is slow) perhaps using
> the B64 decoder from Jakarta commons can improve the thing a little.
> Perhaps there are more examples of easy reutilization.
>
> What do you think is the most important improvement right now? I can
> begin working in any of these areas, but some steering is appreciated.
>
> Regards,
> Raul Benito.
>
> p.s- Here it is the simple patch that improves the time. This patch is
> not thread safe (and I think it isn't even xalan version safe).
>
> Index: TransformXPath.java
> ===================================================================
> RCS file:
>
/home/cvspublic/xml-security/src/org/apache/xml/security/transforms/implemen
tations/TransformXPath.java,v
> retrieving revision 1.15
> diff -u -r1.15 TransformXPath.java
> --- TransformXPath.java 12 May 2004 12:00:45 -0000 1.15
> +++ TransformXPath.java 13 May 2004 21:38:17 -0000
> @@ -63,6 +63,7 @@
> /** Field implementedTransformURI */
> public static final String implementedTransformURI =
> Transforms.TRANSFORM_XPATH;
> + static CachedXPathAPI cx=new CachedXPathAPI();
>
> //J-
> public boolean wantsOctetStream () { return false; }
> @@ -105,7 +106,7 @@
> */
> Set inputSet = input.getNodeSet();
> CachedXPathFuncHereAPI xPathFuncHereAPI =
> - new CachedXPathFuncHereAPI(new CachedXPathAPI()
> /*input.getCachedXPathAPI()*/);
> + new CachedXPathFuncHereAPI(cx /*input.getCachedXPathAPI()*/);
> if (inputSet.size() == 0) {
> Object exArgs[] = { "input node set contains no nodes" };
>
> @@ -172,7 +173,7 @@
> }
>
> XMLSignatureInput result = new XMLSignatureInput(resultNodes,
> - input.getCachedXPathAPI());
> + cx);
>
> result.setSourceURI(input.getSourceURI());
>
>
