Hello,
i had pretty much the same problem, a large number of small files to aggregate, and i finally found the xpathdirectorygenerator, which is pretty straightforward to use, doesn't have the caching issues of document() function, and is faster than the technique of xinclude and cinclude on a large number of files.
Hope this help,
Marc
Derek Hohls a �crit :
Chris
Yes... the CInclude transformer output isn't cached, and I have found that the approach you suggested is only marginally faster than using the xinclude approach I had adopted originally.
I guess that working with information from numerous XML files on disc is not really a viable operation using Cocoon... which is, I think, a pity.
In my case its impossible to predict when content can be changed...
could be a whole lot of changes in a few minutes, and then nothing for a few months. This makes it very hard to simply set
a time parameter as suggested in the documents. There needs to
be something a little more straightforward than that to make it
usable/useful.
Any more thoughts?
Derek
I misspoke in my last email (actually, mis-wrote :)[EMAIL PROTECTED] 2005/05/07 04:17:27 AM >>>
The CInclude transformer output isn't, by default, cached, but the
input to it, "cocoon:/101.meta.xml", below, would be. In your case, where
you have fifty or so inputs of that form, that's what you'd want. However,
you can also get the CInclude transformer to cache its output, as described here:
http://cocoon.apache.org/2.1/userdocs/transformers/cinclude-transformer.html#Caching
I agree when you said "surely its Cocoon's "job" to check ....", but I've gotten myself into a bit of a mess, with lots of these cached pipeline fragments daisy-chained. I've also implemented "preemptive caching" on some of them, and now I just haven't spent the time to dig
in and see where the cache isn't getting properly invalidated.
Cheers!
Derek Hohls wrote:
techniqueChris(?)
Thanks - I will give this a try (cannot be slower than what I have
now and looks pretty straightforward). It had not occurred to me
because it seemed this would require 50+ calls to the same pipeline.. instead of just one pass through one... but I'll check.
Re the caching - surely its Cocoon's "job" to check the files being used (if they are static, as in this case) and send through the latest version... but I will run some tests and see what occurs.
Thanks Derek
[EMAIL PROTECTED] 2005/05/06 02:44:53 PM >>>
Is the XPath expression the same in every case ("//inml:ind/meta")? If so, then it would be easy to switch to using CInclude, which is cached:
<file name='101.xml'> <ci:include src='cocoon:/101.meta.xml'/> </file>
And then define a new pipeline to produce 101.meta.xml:
<map:match pattern='*.meta.xml'> <map:generator src='{1}.xml'/> <map:transform src='pull-out-ind-meta.xslt'/> <map:serialize> </map:match>
I'm pretty new to Cocoon, actually, and I've been using this
a lot,it,
for example, to generate my nav bar. I'm not altogether happy with
though,
mainly because I can't figure out how to control the cache -- i.e. to
make sure that it gets invalidated whenever {1}.xml changes. But, it's pretty fast.
Derek Hohls wrote:
---------------------------------------------------------------------I am looking to find a way to speed up a key step in a pipeline:
The one in question has the following steps:
<map:match pattern="ind-list">
<map:parameter name="handler" value="myindhandler"/>
<map:generate src="inds" type="directory" include="*.xml"/> <map:transform src="stylesheets/ind/ind-xincludes.xsl" >
<map:parameter name="ind-dir" value="inds"/>
</map:transform>
<!-- *NOW SLOW* -->
<map:transform type="xinclude"/>
<map:serialize type="xml"/> </map:match>
The pipeline is fast up to the end of the first transform,
resulting in XML which contains a number of tags of the form:
<file name="101.xml"> <xi:include href="inds/101.xml#xmlns(inml=http://www.myschema.com)xpointer(//inml:ind/meta)"/> </file>
The number of tags varies by directory, but is typically about 50. The files themselves are small - about 50k - and the "meta" tags have only a few bytes of text.
However, this last step takes over a minute! on a fast server
(2Gb memory, 3Ghz processor)...
What can I do to ensure that this is speeded up significantly...? ie at least a factor of 10!
Thanks Derek
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
