Re: how to list all sitemap components
Carsten Ziegeler wrote: David Crossley wrote: That is the trouble - i don't know what is correct. :) The file called components-javadoc-sitemaptask-diff.txt is the difference between the list produced by scanning javadocs (finding extra clutter, maybe missing some) and the list produced by the SitemapTask. diff components-source.txt components-sitemaptask.txt Yepp. This is not to say that the scanning javadocs is finding all components. It is just to provide a method for comparison. Ok. Here are some examples of components that are not found by the SitemapTask ... org/apache/cocoon/matching/RegexpTargetHostMatcher org/apache/cocoon/transformation/JXTemplateTransformer org/apache/cocoon/selection/SessionStateSelector org/apache/cocoon/transformation/CachingCIncludeTransformer org/apache/cocoon/forms/formmodel/RepeaterAction Ah, so it's the other way round and the list I provided above are those that are not found in the javadocs, right? I'm still trying to make sense of the diff file: what line is missing in which file. The *Pipeline components that you listed above are found by the SitemapTask but not by the scan of javadocs. That is because the correlate-table.sh script forgot to look for pipelines. But do we consider them sitemap components that should be documented in the Userdocs? Yes, we should :) These are sitemap components, you can selected between different pipeline implementations in your sitemap (caching, non-caching etc.), so imho it makes sense to document them as well. That is what i thought but wasn't sure. Thanks. I will generate some fresh files and put them at my web space perhaps i can remove a bit more of the clutter. --David
RE: how to list all sitemap components
David Crossley wrote: So if I read the diff correctly, these classes are not found by the sitemap task: 65a64,71 org/apache/cocoon/components/pipeline/impl/AbstractCachingProcessingPi peline org/apache/cocoon/components/pipeline/impl/BaseCachingProcessingPipe line org/apache/cocoon/components/pipeline/impl/CachingPointProcessingPip eline org/apache/cocoon/components/pipeline/impl/CachingProcessingPipeline org/apache/cocoon/components/pipeline/impl/ExpiresCachingProcessingPip eline org/apache/cocoon/components/pipeline/impl/NonCachingProcessingPipel ine org/apache/cocoon/components/profiler/ProfilingCachingProcessingPipe line org/apache/cocoon/components/profiler/ProfilingNonCachingProcessingPip eline org/apache/cocoon/woody/acting/AbstractWoodyAction org/apache/cocoon/woody/acting/HandleFormSubmitAction org/apache/cocoon/woody/acting/MakeFormAction org/apache/cocoon/woody/generation/WoodyGenerator org/apache/cocoon/woody/samples/InitForm1Action org/apache/cocoon/woody/transformation/WoodyTemplateTransformer org/apache/garbage/serializer/AbstractSerializer org/apache/garbage/serializer/EncodingSerializer org/apache/garbage/serializer/HTMLSerializer org/apache/garbage/serializer/XHTMLSerializer org/apache/garbage/serializer/XMLSerializer Is this correct? That is the trouble - i don't know what is correct. :) The file called components-javadoc-sitemaptask-diff.txt is the difference between the list produced by scanning javadocs (finding extra clutter, maybe missing some) and the list produced by the SitemapTask. diff components-source.txt components-sitemaptask.txt Yepp. This is not to say that the scanning javadocs is finding all components. It is just to provide a method for comparison. Ok. Here are some examples of components that are not found by the SitemapTask ... org/apache/cocoon/matching/RegexpTargetHostMatcher org/apache/cocoon/transformation/JXTemplateTransformer org/apache/cocoon/selection/SessionStateSelector org/apache/cocoon/transformation/CachingCIncludeTransformer org/apache/cocoon/forms/formmodel/RepeaterAction Ah, so it's the other way round and the list I provided above are those that are not found in the javadocs, right? I'm still trying to make sense of the diff file: what line is missing in which file. The *Pipeline components that you listed above are found by the SitemapTask but not by the scan of javadocs. That is because the correlate-table.sh script forgot to look for pipelines. But do we consider them sitemap components that should be documented in the Userdocs? Yes, we should :) These are sitemap components, you can selected between different pipeline implementations in your sitemap (caching, non-caching etc.), so imho it makes sense to document them as well. Carsten
Re: how to list all sitemap components
Carsten Ziegeler wrote: David Crossley wrote: Okay, i uploaded them to my ASF committer space www.apache.org ~crossley/review-sitemap-docs/ See the notes in SVN cocoon-2_1_X/tools/review-sitemap-docs/README.txt and cocoon-2_1_X/tools/review-sitemap-docs/TODO.txt Thanks!! So if I read the diff correctly, these classes are not found by the sitemap task: 65a64,71 org/apache/cocoon/components/pipeline/impl/AbstractCachingProcessingPipeline org/apache/cocoon/components/pipeline/impl/BaseCachingProcessingPipeline org/apache/cocoon/components/pipeline/impl/CachingPointProcessingPipeline org/apache/cocoon/components/pipeline/impl/CachingProcessingPipeline org/apache/cocoon/components/pipeline/impl/ExpiresCachingProcessingPipeline org/apache/cocoon/components/pipeline/impl/NonCachingProcessingPipeline org/apache/cocoon/components/profiler/ProfilingCachingProcessingPipeline org/apache/cocoon/components/profiler/ProfilingNonCachingProcessingPipeline org/apache/cocoon/woody/acting/AbstractWoodyAction org/apache/cocoon/woody/acting/HandleFormSubmitAction org/apache/cocoon/woody/acting/MakeFormAction org/apache/cocoon/woody/generation/WoodyGenerator org/apache/cocoon/woody/samples/InitForm1Action org/apache/cocoon/woody/transformation/WoodyTemplateTransformer org/apache/garbage/serializer/AbstractSerializer org/apache/garbage/serializer/EncodingSerializer org/apache/garbage/serializer/HTMLSerializer org/apache/garbage/serializer/XHTMLSerializer org/apache/garbage/serializer/XMLSerializer Is this correct? That is the trouble - i don't know what is correct. The file called components-javadoc-sitemaptask-diff.txt is the difference between the list produced by scanning javadocs (finding extra clutter, maybe missing some) and the list produced by the SitemapTask. diff components-source.txt components-sitemaptask.txt This is not to say that the scanning javadocs is finding all components. It is just to provide a method for comparison. Here are some examples of components that are not found by the SitemapTask ... org/apache/cocoon/matching/RegexpTargetHostMatcher org/apache/cocoon/transformation/JXTemplateTransformer org/apache/cocoon/selection/SessionStateSelector org/apache/cocoon/transformation/CachingCIncludeTransformer org/apache/cocoon/forms/formmodel/RepeaterAction The woody and garbage stuff are probably not relevant. The PhpGenerator is listed as missing because it caused 'build docs' to crash, so i moved it out of the way. The *Pipeline components that you listed above are found by the SitemapTask but not by the scan of javadocs. That is because the correlate-table.sh script forgot to look for pipelines. But do we consider them sitemap components that should be documented in the Userdocs? --David
RE: how to list all sitemap components
David Crossley wrote: I am working on this in cocoon-2_1_X branch. In tools/targets/docs-build.xml uncomment the sitemap-components task at line 56 to also scan the blocks. Then run 'build javadocs'. Then run 'build docs'. This will also produce lists of all possible sitemap components at build/all-sitemap-components*.txt via the SitemapTask. Then follow tools/review-sitemap-docs/README.txt to run the shell script correlate-table.sh which attempts to scan javadocs and match with what we have manually identified as being possible sitemap components. It seems that my linux machine isn't working the way it should be (perhaps it's already on holiday), so I can't even get Cocoon build using unix right now. Can you provide the result of the comparison? Thanks Carsten
Re: how to list all sitemap components
Carsten Ziegeler wrote: David Crossley wrote: I am working on this in cocoon-2_1_X branch. In tools/targets/docs-build.xml uncomment the sitemap-components task at line 56 to also scan the blocks. Then run 'build javadocs'. Then run 'build docs'. This will also produce lists of all possible sitemap components at build/all-sitemap-components*.txt via the SitemapTask. Then follow tools/review-sitemap-docs/README.txt to run the shell script correlate-table.sh which attempts to scan javadocs and match with what we have manually identified as being possible sitemap components. It seems that my linux machine isn't working the way it should be (perhaps it's already on holiday), so I can't even get Cocoon build using unix right now. Can you provide the result of the comparison? Okay, i uploaded them to my ASF committer space www.apache.org ~crossley/review-sitemap-docs/ See the notes in SVN cocoon-2_1_X/tools/review-sitemap-docs/README.txt and cocoon-2_1_X/tools/review-sitemap-docs/TODO.txt --David
RE: how to list all sitemap components
David Crossley wrote: Okay, i uploaded them to my ASF committer space www.apache.org ~crossley/review-sitemap-docs/ See the notes in SVN cocoon-2_1_X/tools/review-sitemap-docs/README.txt and cocoon-2_1_X/tools/review-sitemap-docs/TODO.txt Thanks!! So if I read the diff correctly, these classes are not found by the sitemap task: 65a64,71 org/apache/cocoon/components/pipeline/impl/AbstractCachingProcessingPipeline org/apache/cocoon/components/pipeline/impl/BaseCachingProcessingPipeline org/apache/cocoon/components/pipeline/impl/CachingPointProcessingPipeline org/apache/cocoon/components/pipeline/impl/CachingProcessingPipeline org/apache/cocoon/components/pipeline/impl/ExpiresCachingProcessingPipeline org/apache/cocoon/components/pipeline/impl/NonCachingProcessingPipeline org/apache/cocoon/components/profiler/ProfilingCachingProcessingPipeline org/apache/cocoon/components/profiler/ProfilingNonCachingProcessingPipeline org/apache/cocoon/woody/acting/AbstractWoodyAction org/apache/cocoon/woody/acting/HandleFormSubmitAction org/apache/cocoon/woody/acting/MakeFormAction org/apache/cocoon/woody/generation/WoodyGenerator org/apache/cocoon/woody/samples/InitForm1Action org/apache/cocoon/woody/transformation/WoodyTemplateTransformer org/apache/garbage/serializer/AbstractSerializer org/apache/garbage/serializer/EncodingSerializer org/apache/garbage/serializer/HTMLSerializer org/apache/garbage/serializer/XHTMLSerializer org/apache/garbage/serializer/XMLSerializer Is this correct? Carsten
Re: how to list all sitemap components
Carsten Ziegeler wrote: David Crossley wrote: ... which misses src/java/o/a/c/serialization/XMLSerializer.java perhaps because that extends AbstractTextSerializer. Ah, ok, that's strange. My understanding was that this should work and if I'm not mistaken it works for other classes. For example the transformers that are found inherit from abstract class as well; those abstract classes are declaring to implement the Transformer interface. Hmm, can you provide a list of which classes are found? (Perhaps with a list of which classes are not found - so we could see if we find similarities) I am working on this in cocoon-2_1_X branch. In tools/targets/docs-build.xml uncomment the sitemap-components task at line 56 to also scan the blocks. Then run 'build javadocs'. Then run 'build docs'. This will also produce lists of all possible sitemap components at build/all-sitemap-components*.txt via the SitemapTask. Then follow tools/review-sitemap-docs/README.txt to run the shell script correlate-table.sh which attempts to scan javadocs and match with what we have manually identified as being possible sitemap components. --David
Re: how to list all sitemap components
David Crossley wrote: Bertrand Delacretaz wrote: Upayavira a écrit : ...cd build/cocoon-2.1.7-dev/javadocs/ grep -rl SitemapModelComponent * Good one! Here's the pretty listing then: for i in $(grep -rl SitemapModelComponent * | grep org/apache) do echo $i | sed 's/\//\./g' | sed 's/\.html$//' done There's still a bit of junk left (package-frame etc) but it's a good start. Thanks, that will be suitable. Fiddling with the SitemapTask.java i can also getting a listing. So i will be able to work out a solution. Thanks. Hmmm, no joy so far. Using the grep javadocs method does not find any serializers, actions, matchers, selectors. Using the SitemapTask finds components only if their package name is o.a.c.transformation.Transformer etc. --David
Re: how to list all sitemap components
Bertrand Delacretaz wrote: David Crossley a ?crit : ...Using the grep javadocs method does not find any serializers, actions, matchers, selectors. Why, can't you grep their base interfaces like: for i in $(egrep -rl 'SitemapModelComponent|OtherBaseInterface|Other) ? Oh, i haven't tried that. I expected all sitemap components to have SitemapModelComponent, but evidently not. Also, how about defining a marker interface for components that need to be documented in this way? AutoDocumentedComponent extends Component or something like this. At the moment i am trying to get a list of all possible sitemap components, so that we know which components should be documented. When we know that, then we can add such a marker to the relevant components. Actually the SitemapTask will handle that because it searches for the javadoc-like comments, e.g. @cocoon.sitemap.component.name http://cocoon.apache.org/2.1/plan/review-sitemap-docs.html#Introduction --David
RE: how to list all sitemap components
David Crossley wrote: Thanks, that will be suitable. Fiddling with the SitemapTask.java i can also getting a listing. So i will be able to work out a solution. Thanks. Hmmm, no joy so far. Using the grep javadocs method does not find any serializers, actions, matchers, selectors. Using the SitemapTask finds components only if their package name is o.a.c.transformation.Transformer etc. Hmmm, are you sure? The SitemapTaks is able (should be able to) to generate docs for all sitemap components. It does so by using qdox and introspecting the classes (if a class implements the corresponding interface). For example, the task creates an xml document for a sitemap component if the javadoc containes the corresponding javadoc tags. I think the only sources that have this tag right now are transformers in the o.a.c.transformation package, so this might be the reason. But changing the SitemapTask to just spit out a list of all sitemap components should be fairly easy. I can't help you with this today - I'm now awake for more than 36 hours, but later this week should work. Carsten
Re: how to list all sitemap components
Carsten Ziegeler wrote: David Crossley wrote: ...Fiddling with the SitemapTask.java i can also getting a listing. So i will be able to work out a solution. Thanks. Hmmm, no joy so far. Using the grep javadocs method does not find any serializers, actions, matchers, selectors. Using the SitemapTask finds components only if their package name is o.a.c.transformation.Transformer etc. Hmmm, are you sure? The SitemapTaks is able (should be able to) to generate docs for all sitemap components. It does so by using qdox and introspecting the classes (if a class implements the corresponding interface). Welcome back Carsten. You probably have stacks of email to catch up with. Here is the reason for my questions. Re: review of sitemap component documentation http://marc.theaimsgroup.com/?t=11012753601 http://marc.theaimsgroup.com/?t=11019584291 http://cocoon.apache.org/2.1/plan/review-sitemap-docs.html For example, the task creates an xml document for a sitemap component if the javadoc containes the corresponding javadoc tags. That part i understand. The part that i am having trouble with is the mechanism by which it determines which java files to investigate to see if they do contain those tags. I think the only sources that have this tag right now are transformers in the o.a.c.transformation package, so this might be the reason. That is the next stage of my mission. Once all possible components are identified, i will add the javadoc-like tags to them and we can start adding the documentation. The abovementioned table lists the components that do currently have the tags: not many. But changing the SitemapTask to just spit out a list of all sitemap components should be fairly easy. That is what i thought too. I have local modifications that does that, but it seems to not find all components. The SitemapTask has ... // Class Constants private static final String SERIALIZER = org.apache.cocoon.serialization.Serializer; ... if ( javaClass.isA(SERIALIZER) ) { ... which misses src/java/o/a/c/serialization/XMLSerializer.java perhaps because that extends AbstractTextSerializer. I can't help you with this today - I'm now awake for more than 36 hours, but later this week should work. Thanks, i was hoping that you would help when you returned. I will commit the work that i have done so far in the 2.1 branch. --David
RE: how to list all sitemap components
David Crossley wrote: Welcome back Carsten. You probably have stacks of email to catch up with. Yes I had, now I'm done to 50 (I just deleted all the stuff dealing with jxtg, tags, templates blabla :) ) Here is the reason for my questions. Re: review of sitemap component documentation http://marc.theaimsgroup.com/?t=11012753601 http://marc.theaimsgroup.com/?t=11019584291 http://cocoon.apache.org/2.1/plan/review-sitemap-docs.html Thanks for the pointers! But changing the SitemapTask to just spit out a list of all sitemap components should be fairly easy. That is what i thought too. I have local modifications that does that, but it seems to not find all components. The SitemapTask has ... // Class Constants private static final String SERIALIZER = org.apache.cocoon.serialization.Serializer; ... if ( javaClass.isA(SERIALIZER) ) { ... which misses src/java/o/a/c/serialization/XMLSerializer.java perhaps because that extends AbstractTextSerializer. Ah, ok, that's strange. My understanding was that this should work and if I'm not mistaken it works for other classes. For example the transformers that are found inherit from abstract class as well; those abstract classes are declaring to implement the Transformer interface. Hmm, can you provide a list of which classes are found? (Perhaps with a list of which classes are not found - so we could see if we find similarities) Carsten
Re: how to list all sitemap components
Le 13 déc. 04, à 09:20, David Crossley a écrit : ...Using the grep javadocs method does not find any serializers, actions, matchers, selectors. Why, can't you grep their base interfaces like: for i in $(egrep -rl 'SitemapModelComponent|OtherBaseInterface|Other) ? Also, how about defining a marker interface for components that need to be documented in this way? AutoDocumentedComponent extends Component or something like this. -Bertrand smime.p7s Description: S/MIME cryptographic signature
Re: how to list all sitemap components
Upayavira wrote: David Crossley wrote: Bertrand Delacretaz wrote: David Crossley a ?crit : ...Using the grep javadocs method does not find any serializers, actions, matchers, selectors. Why, can't you grep their base interfaces like: for i in $(egrep -rl 'SitemapModelComponent|OtherBaseInterface|Other)? Oh, i haven't tried that. I expected all sitemap components to have SitemapModelComponent, but evidently not. I just used SitemapModelComponent as something of a throwaway comment, Ah, i see now. Your comment sounded definitive. actually, if you search for http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/generation/Generator.html, http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/transformation/Transformer.html, etc, in the Javadoc files, you _will_ get everything! Also, how about defining a marker interface for components that need to be documented in this way? AutoDocumentedComponent extends Component or something like this. At the moment i am trying to get a list of all possible sitemap components, so that we know which components should be documented. When we know that, then we can add such a marker to the relevant components. Actually the SitemapTask will handle that because it searches for the javadoc-like comments, e.g. @cocoon.sitemap.component.name http://cocoon.apache.org/2.1/plan/review-sitemap-docs.html#Introduction Yup, once marked up, we'll be there. But a simple script we can run everyso often to ensure that we haven't added new ones without markup would be useful. Yes, my first task is twofold: - find all possible components - develop a shell script which will scan the code and match against the coordination table. --David
Re: how to list all sitemap components
David Crossley wrote: Geoff Howard wrote: I'd just use eclipse to find every class that implements the right interface(s). Would that work? Thanks Geoff. However, command-line tools only because i need to script it. Sorry, i forgot to specify that. A bit of BCEL should do the trick. J.Pietschmann
Re: how to list all sitemap components
I'd just use eclipse to find every class that implements the right interface(s). Would that work? Geoff On Thu, 09 Dec 2004 10:46:36 +1100, David Crossley [EMAIL PROTECTED] wrote: I am trying to create a list of all sitemap components in the Cocoon core and blocks. So far i have tried to use 'find and grep' by looking for well-known filenames, e.g. *Transformer.java and also searching in well-known directories, e.g. /transformation/ However, that misses some components and gets too much extra stuff. Using the package name inside the *.java also misses some components. Is there a way to uniquely identify the sitemap components by grepping the *.java e.g. perhaps a unique method name? --David
Re: how to list all sitemap components
Geoff Howard wrote: I'd just use eclipse to find every class that implements the right interface(s). Would that work? Or, what we need is a tool that lists all the interfaces that a class implements. Hmm. Javadoc. For the CastorTransformer, you get a line saying: *All Implemented Interfaces:* Component http://avalon.apache.org/framework/api/org/apache/avalon/framework/component/Component.html, Configurable http://avalon.apache.org/framework/api/org/apache/avalon/framework/configuration/Configurable.html, ContentHandler http://java.sun.com/j2ee/sdk_1.3/techdocs/api/org/xml/sax/ContentHandler.html, LexicalHandler http://java.sun.com/j2ee/sdk_1.3/techdocs/api/org/xml/sax/ext/LexicalHandler.html, LogEnabled http://avalon.apache.org/framework/api/org/apache/avalon/framework/logger/LogEnabled.html, Poolable http://excalibur.apache.org/apidocs/org/apache/avalon/excalibur/pool/Poolable.html, Recyclable http://excalibur.apache.org/apidocs/org/apache/avalon/excalibur/pool/Recyclable.html, SitemapModelComponent http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/sitemap/SitemapModelComponent.html, Transformer http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/transformation/Transformer.html, XMLConsumer http://excalibur.apache.org/apidocs/org/apache/excalibur/xml/sax/XMLConsumer.html, XMLConsumer http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/xml/XMLConsumer.html, XMLPipe http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/xml/XMLPipe.html, XMLProducer http://cocoon.apache.org/2.1/apidocs/org/apache/cocoon/xml/XMLProducer.html The word Transformer is in there, as is SitemapModelComponent, each of which tell you what you want to know. Am I correct? Regards, Upayavira Geoff On Thu, 09 Dec 2004 10:46:36 +1100, David Crossley [EMAIL PROTECTED] wrote: I am trying to create a list of all sitemap components in the Cocoon core and blocks. So far i have tried to use 'find and grep' by looking for well-known filenames, e.g. *Transformer.java and also searching in well-known directories, e.g. /transformation/ However, that misses some components and gets too much extra stuff. Using the package name inside the *.java also misses some components. Is there a way to uniquely identify the sitemap components by grepping the *.java e.g. perhaps a unique method name? --David
Re: how to list all sitemap components
Le 9 déc. 04, à 00:46, David Crossley a écrit : I am trying to create a list of all sitemap components in the Cocoon core and blocks... Doesn't qdox list the interfaces that a class implements? If it's the case, it should be possible to create a pipeline (using the qdox block) or use the qdox ant task to generate the list, by finding all classes which implement the Generator interface. Is there a way to uniquely identify the sitemap components by grepping the *.java e.g. perhaps a unique method name? Probably not, as the method could be implemented in a base class and as such not be visible in the derived class's source code. I'd go the qdox route, assuming it finds the inherited interfaces. -Bertrand smime.p7s Description: S/MIME cryptographic signature
Re: how to list all sitemap components
Bertrand Delacretaz wrote: Le 9 déc. 04, à 00:46, David Crossley a écrit : I am trying to create a list of all sitemap components in the Cocoon core and blocks... Doesn't qdox list the interfaces that a class implements? If it's the case, it should be possible to create a pipeline (using the qdox block) or use the qdox ant task to generate the list, by finding all classes which implement the Generator interface. Is there a way to uniquely identify the sitemap components by grepping the *.java e.g. perhaps a unique method name? Probably not, as the method could be implemented in a base class and as such not be visible in the derived class's source code. I'd go the qdox route, assuming it finds the inherited interfaces. -Bertrand cd build/cocoon-2.1.7-dev/javadocs/ grep -rl SitemapModelComponent * That's pretty easy, isn't it? Regards, Upayavira
Re: how to list all sitemap components
Le 9 déc. 04, à 16:56, Upayavira a écrit : ...cd build/cocoon-2.1.7-dev/javadocs/ grep -rl SitemapModelComponent * Good one! Here's the pretty listing then: for i in $(grep -rl SitemapModelComponent * | grep org/apache) do echo $i | sed 's/\//\./g' | sed 's/\.html$//' done There's still a bit of junk left (package-frame etc) but it's a good start. -Bertrand smime.p7s Description: S/MIME cryptographic signature
Re: how to list all sitemap components
Geoff Howard wrote: I'd just use eclipse to find every class that implements the right interface(s). Would that work? Thanks Geoff. However, command-line tools only because i need to script it. Sorry, i forgot to specify that. --David
Re: how to list all sitemap components
Bertrand Delacretaz wrote: Upayavira a écrit : ...cd build/cocoon-2.1.7-dev/javadocs/ grep -rl SitemapModelComponent * Good one! Here's the pretty listing then: for i in $(grep -rl SitemapModelComponent * | grep org/apache) do echo $i | sed 's/\//\./g' | sed 's/\.html$//' done There's still a bit of junk left (package-frame etc) but it's a good start. Thanks, that will be suitable. Fiddling with the SitemapTask.java i can also getting a listing. So i will be able to work out a solution. Thanks. --David