[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
Hi Brice, Brice Figureau wrote: Or I could change puppetdoc rdoc to take more arguments like --modulepath /path/to/modules along with the path where to find the global (or site) manifests. If --modulepath is not mentionned then we assume that analysed path contains only modules. +1 It does indeed make sense to accept the same parameters that puppet and puppetmasterd already use. François --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
On Nov 4, 2008, at 2:24 AM, Brice Figureau wrote: My concern is that puppetdoc right now doesn't use puppet.conf at all, and I'd like to keep this behavior. I want to be able to generate docs for a pile of manifests, modules or not. If I tie puppetdoc rdoc generation to puppet.conf :modulepath, then the only rdoc puppetdoc will be able to generate will be the manifests hosted on the current puppetmaster for a given environment. We could go this way (that'd be way simpler to know where the manifests are, or where the modules are), but then it fixes the documentation puppetdoc can generate (ie it'd be impossible to put some modules in a path and run puppetdoc over it). Or I could change puppetdoc rdoc to take more arguments like --modulepath /path/to/modules along with the path where to find the global (or site) manifests. If --modulepath is not mentionned then we assume that analysed path contains only modules. I'd do that. Just generate docs for what people specify. I'd also probably have puppetdoc accept standard ARGV arguments, and consider those to be normal manifests, which it could then produce output for on stdout. E.g., you could do: puppetdoc manifests/site.pp and it would give you docs for just that file, or, as you say, you could do: puppetdoc --modulepath ... --manifest ... Maybe the --manifest becomes redundant at that point, though. -- Finn's Law: Uncertainty is the final test of innovation. - Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
On Nov 4, 2008, at 9:28 AM, Brice Figureau wrote: On Tue, 2008-11-04 at 09:05 -0600, Luke Kanies wrote: I'd do that. Just generate docs for what people specify. I'd also probably have puppetdoc accept standard ARGV arguments, and consider those to be normal manifests, which it could then produce output for on stdout. E.g., you could do: puppetdoc manifests/site.pp and it would give you docs for just that file, Yes, that is planned, but that's a complete different thing than the current rdoc implementation (although it is simple to have). What is still unknown at this stage for this mode is what to output in this mode... All comments? Only classes, defines and nodes ? in which order (I'm afraid that's not something I can control though) ? I'd do whatever you'd normally show. or, as you say, you could do: puppetdoc --modulepath ... --manifest ... Maybe the --manifest becomes redundant at that point, though. My idea was: puppetdoc --modulepath module path1 --modulepath module path2 path1 path2... path1: contains manifests path2: contains manifests too modulepath1: contains only a module hierarchy modulepath2: contains only a module hierarchy If path1 or path2 encompass any module specified in modulepath, those get treated as module of course. The question that remains is, if path1 and path2 don't encompass any module path, should I treat those module path as information, or should I go and scan also those to produce documentation. I'm not sure I understand the complication; module paths contain modules, non-module paths contain normal manifests, right? -- Patriotism is often an arbitrary veneration of real estate above principles. -- George Jean Nathan - Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
On Nov 4, 2008, at 10:54 AM, Brice Figureau wrote: Right, of course :-) My problem is: Let's say I have: /etc/puppet/modules/module1 /etc/puppet/modules/module2 and some global manifests /etc/puppet/manifests/site.pp ... and some other global manifests /tmp/manifests/site.pp If I run: puppetdoc /etc/puppet It will scan both the modules and the global manifests, but have no way to understand that module1 and module2 are modules. If I run: puppetdoc --modulepath /etc/puppet/modules /etc/puppet Puppetdoc now knows that module1 and module2 are modules. And finally, if I run: puppetdoc --modulepath /etc/puppet/modules /tmp/manifests Then it will scan /tmp/manifests/site.pp which is OK, but should it scan /etc/puppet/modules? Urgh, I would just have the software throw up all over someone who did that. Manifest paths and module paths shouldn't overlap, and if they happen to, I definitely would skip any module paths when scanning for normal manifests. -- Neonle will continue to be rude, and will nretend that you had a small stroke which makes you unable to say or see the letter n. Stunid nractical joke, if you ask me. Bunch of noon-heads, huh? -- Fred Barling, Humorscope - Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
Brice Figureau schrieb: On Tue, 2008-11-04 at 10:04 -0600, Luke Kanies wrote: On Nov 4, 2008, at 9:28 AM, Brice Figureau wrote: On Tue, 2008-11-04 at 09:05 -0600, Luke Kanies wrote: I'd do that. Just generate docs for what people specify. I'd also probably have puppetdoc accept standard ARGV arguments, and consider those to be normal manifests, which it could then produce output for on stdout. E.g., you could do: puppetdoc manifests/site.pp and it would give you docs for just that file, Yes, that is planned, but that's a complete different thing than the current rdoc implementation (although it is simple to have). What is still unknown at this stage for this mode is what to output in this mode... All comments? Only classes, defines and nodes ? in which order (I'm afraid that's not something I can control though) ? I'd do whatever you'd normally show. or, as you say, you could do: puppetdoc --modulepath ... --manifest ... Maybe the --manifest becomes redundant at that point, though. My idea was: puppetdoc --modulepath module path1 --modulepath module path2 path1 path2... path1: contains manifests path2: contains manifests too modulepath1: contains only a module hierarchy modulepath2: contains only a module hierarchy If path1 or path2 encompass any module specified in modulepath, those get treated as module of course. The question that remains is, if path1 and path2 don't encompass any module path, should I treat those module path as information, or should I go and scan also those to produce documentation. I'm not sure I understand the complication; module paths contain modules, non-module paths contain normal manifests, right? Right, of course :-) My problem is: Let's say I have: /etc/puppet/modules/module1 /etc/puppet/modules/module2 and some global manifests /etc/puppet/manifests/site.pp ... and some other global manifests /tmp/manifests/site.pp If I run: puppetdoc /etc/puppet It will scan both the modules and the global manifests, but have no way to understand that module1 and module2 are modules. puppetdoc should use the same code as other puppet* binaries to read and interpret puppet.conf and commandline parameters and it should use the modulepath and friends from there. They contain (by definition) all necessary information and all site-local peculiarities. That should generalise properly to create standalone module documentation with puppetdoc --manifest /dev/null --modulepath /path/to/modules/ Regards, DavidS --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
On Tue, 2008-11-04 at 18:13 +0100, David Schmitt wrote: Brice Figureau schrieb: On Tue, 2008-11-04 at 10:04 -0600, Luke Kanies wrote: On Nov 4, 2008, at 9:28 AM, Brice Figureau wrote: On Tue, 2008-11-04 at 09:05 -0600, Luke Kanies wrote: I'd do that. Just generate docs for what people specify. I'd also probably have puppetdoc accept standard ARGV arguments, and consider those to be normal manifests, which it could then produce output for on stdout. E.g., you could do: puppetdoc manifests/site.pp and it would give you docs for just that file, Yes, that is planned, but that's a complete different thing than the current rdoc implementation (although it is simple to have). What is still unknown at this stage for this mode is what to output in this mode... All comments? Only classes, defines and nodes ? in which order (I'm afraid that's not something I can control though) ? I'd do whatever you'd normally show. or, as you say, you could do: puppetdoc --modulepath ... --manifest ... Maybe the --manifest becomes redundant at that point, though. My idea was: puppetdoc --modulepath module path1 --modulepath module path2 path1 path2... path1: contains manifests path2: contains manifests too modulepath1: contains only a module hierarchy modulepath2: contains only a module hierarchy If path1 or path2 encompass any module specified in modulepath, those get treated as module of course. The question that remains is, if path1 and path2 don't encompass any module path, should I treat those module path as information, or should I go and scan also those to produce documentation. I'm not sure I understand the complication; module paths contain modules, non-module paths contain normal manifests, right? Right, of course :-) My problem is: Let's say I have: /etc/puppet/modules/module1 /etc/puppet/modules/module2 and some global manifests /etc/puppet/manifests/site.pp ... and some other global manifests /tmp/manifests/site.pp If I run: puppetdoc /etc/puppet It will scan both the modules and the global manifests, but have no way to understand that module1 and module2 are modules. puppetdoc should use the same code as other puppet* binaries to read and interpret puppet.conf and commandline parameters and it should use the modulepath and friends from there. They contain (by definition) all necessary information and all site-local peculiarities. That should generalise properly to create standalone module documentation with puppetdoc --manifest /dev/null --modulepath /path/to/modules/ That might be the best solution. I wasn't keen about using puppet.conf for puppetdoc (it wasn't using it before my patch), as it would tie documentation generation to a specific installation. And there is the problem of environments, too. But, since you can override puppet.conf directives with the command line arguments, I think that will works. OK, to summarize: By default puppetdoc uses the default puppet.conf, except if overriden with --configdir or --config (default puppet* behavior). If --modulepath or --manifests are provided, they override puppet.conf defaults. Now, the behavior is to generate documentation for both site manifests _and_ modulepath (if not empty). If modulepath is overriden and defaults to /dev/null or an inexistant path, don't produce doc for modules. Same for manifests. If --mode is rdoc and path are passed in ARGV, then the additionnal path are scanned for global manifests. If those path overlaps with any configured modulepaths, then we exit with an error. If --mode is not rdoc and filenames are passed in ARGV, then perform a straight documentation extract (no fancy formatting and so) on stdout. Luke, do you agree with this? Did I forgot anything? If not then I'll follow this specs. -- Brice Figureau [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
On Tue, 2008-11-04 at 10:04 -0600, Luke Kanies wrote: On Nov 4, 2008, at 9:28 AM, Brice Figureau wrote: On Tue, 2008-11-04 at 09:05 -0600, Luke Kanies wrote: I'd do that. Just generate docs for what people specify. I'd also probably have puppetdoc accept standard ARGV arguments, and consider those to be normal manifests, which it could then produce output for on stdout. E.g., you could do: puppetdoc manifests/site.pp and it would give you docs for just that file, Yes, that is planned, but that's a complete different thing than the current rdoc implementation (although it is simple to have). What is still unknown at this stage for this mode is what to output in this mode... All comments? Only classes, defines and nodes ? in which order (I'm afraid that's not something I can control though) ? I'd do whatever you'd normally show. or, as you say, you could do: puppetdoc --modulepath ... --manifest ... Maybe the --manifest becomes redundant at that point, though. My idea was: puppetdoc --modulepath module path1 --modulepath module path2 path1 path2... path1: contains manifests path2: contains manifests too modulepath1: contains only a module hierarchy modulepath2: contains only a module hierarchy If path1 or path2 encompass any module specified in modulepath, those get treated as module of course. The question that remains is, if path1 and path2 don't encompass any module path, should I treat those module path as information, or should I go and scan also those to produce documentation. I'm not sure I understand the complication; module paths contain modules, non-module paths contain normal manifests, right? Right, of course :-) My problem is: Let's say I have: /etc/puppet/modules/module1 /etc/puppet/modules/module2 and some global manifests /etc/puppet/manifests/site.pp ... and some other global manifests /tmp/manifests/site.pp If I run: puppetdoc /etc/puppet It will scan both the modules and the global manifests, but have no way to understand that module1 and module2 are modules. If I run: puppetdoc --modulepath /etc/puppet/modules /etc/puppet Puppetdoc now knows that module1 and module2 are modules. And finally, if I run: puppetdoc --modulepath /etc/puppet/modules /tmp/manifests Then it will scan /tmp/manifests/site.pp which is OK, but should it scan /etc/puppet/modules? -- Brice Figureau [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
On Sun, 2008-11-02 at 21:53 -0600, Luke Kanies wrote: On Oct 31, 2008, at 5:40 PM, Brice Figureau wrote: Note: * the current version only knows how to parse modules. It can't parse sole manifests or manifests that are not under a standard module hierarchy. This is a great start, though. Are you planning on adding these? Just supporting the standard site.pp in the 'manifest' setting would be sufficient. Yes, I'd like to be able to parse everything. My concern is that right now everything must be logically part of a module. I can invent a special module for the manifests that don't belong to this module (something along a site module for instance). * the current version fails if you have ruby plugins (facter or other) in your modules. This will be fixed soon I hope. I think I have an idea for this, but I don't know if that will work. I think it will be quite fragile, though as it will depend in the current plugin API: I think I'll parse the ruby code in a crude way, looking for strings ala: Facter.add(something) or newfunction(:blah, :type = :rvalue) and backtrack to fetch the comments that are just before. Then I'll associate those documentation objects to the module they belong to. You should send a note to the user list, so the rest of the community has a chance to test. I will do as soon as it will be a little bit more stable (ie right now if you have ruby plugins it fails). Thanks for your comments ;-) -- Brice Figureau [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
This is utterly awesome Brice. Great work. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
On 3/11/08 4:53, Luke Kanies wrote: On Oct 31, 2008, at 5:40 PM, Brice Figureau wrote: Hi, As promised a couple of weeks ago, here is my first attempt, just in time for a creepy Halloween, to provide an application to generate documentation for manifests. It is based upon a parser modification that associates comments with statements, and a RDoc based html output tailored for puppet language. An example output generated from David Schmidtt modules[*] is available here: http://www.masterzen.fr/puppet/rdoc/index.html This is *awesome*. Once David goes through and comprehensively comments his whole code base, it'll be even better. :) It is to be noted that David Schmidtt modules don't contains RDoc style markup comments, so sometimes the generated output can seem strange. The development version is hosted on my github repository: http://github.com/masterzen/puppet/tree/feature%2Finline-doc Usage: % puppetdoc --rdoc --outputdir /tmp/doc /path/to/modules The generated output ends up in doc by default or into the provided outputdir. At first, the output shows the list of all modules and the list of all classes. Once you click on a module link in the All modules, the bottom left screen shows all the defined defines and classes of this module and the right frame shows the module documentation. If you click on a specific class or define it then shows the documentation for this specific class or define. This mimics a little bit the Javadoc output , which I found really pleasant to use (way better than the ruby-doc RDoc output IMHO). Note: * the current version only knows how to parse modules. It can't parse sole manifests or manifests that are not under a standard module hierarchy. This is a great start, though. Are you planning on adding these? Just supporting the standard site.pp in the 'manifest' setting would be sufficient. I'm currently working on it, unfortunately I'm stumbling across an issue.: I want the rdoc generator to be independent of any configuration (ie modulepath), so that we can point it at any directory and let it scan. That means that I have to find an heuristic that says, this manifest file is part of a module or this manifest is part of the global 'site' part. My first thought was to check that there is a 'files' and a 'templates' directory in the same path as the manifests dir. For an unknown reason some of my own modules only have the manifests directory, so it seems Puppet doesn't enforce the presence of this dirs. Hence, I can't use this heuristic as is. I can check for a 'modules' directory and infer from that I'm scanning a module, but that seems quite artificial and error-prone. Any idea of a better heuristic, or should I bite the bullet and use the current puppet.conf to check for modulepath? -- Brice Figureau Days of Wonder http://www.daysofwonder.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
On Oct 31, 2008, at 5:40 PM, Brice Figureau wrote: Hi, As promised a couple of weeks ago, here is my first attempt, just in time for a creepy Halloween, to provide an application to generate documentation for manifests. It is based upon a parser modification that associates comments with statements, and a RDoc based html output tailored for puppet language. An example output generated from David Schmidtt modules[*] is available here: http://www.masterzen.fr/puppet/rdoc/index.html This is *awesome*. Once David goes through and comprehensively comments his whole code base, it'll be even better. :) It is to be noted that David Schmidtt modules don't contains RDoc style markup comments, so sometimes the generated output can seem strange. The development version is hosted on my github repository: http://github.com/masterzen/puppet/tree/feature%2Finline-doc Usage: % puppetdoc --rdoc --outputdir /tmp/doc /path/to/modules The generated output ends up in doc by default or into the provided outputdir. At first, the output shows the list of all modules and the list of all classes. Once you click on a module link in the All modules, the bottom left screen shows all the defined defines and classes of this module and the right frame shows the module documentation. If you click on a specific class or define it then shows the documentation for this specific class or define. This mimics a little bit the Javadoc output , which I found really pleasant to use (way better than the ruby-doc RDoc output IMHO). Note: * the current version only knows how to parse modules. It can't parse sole manifests or manifests that are not under a standard module hierarchy. This is a great start, though. Are you planning on adding these? Just supporting the standard site.pp in the 'manifest' setting would be sufficient. * the current version fails if you have ruby plugins (facter or other) in your modules. This will be fixed soon I hope. * only defines, includes, classes and modules (ie README content) documentation are generated for this version. * there is no rspec tests coverage for the RDoc integration. I didn't really think it was necessary. If really needed, please let me know. This is ok for most of it, but some basic tests might be nice. * the RDoc integration files are not in the right place (puppet/ rdoc). I plan to move them into (puppet/util/rdoc) to not be in the way of the important parts of puppet. If needed we can even externalize those files as they are only needed by puppetdoc. The very next action will be to provide, based on this an ala 'perldoc' feature that knows to parse a manifest and produce directly on the terminal its documentation. What is unknown at this stage is what documentation to extract from the manifests in 'perldoc' mode, nor which formatting to apply (if there is one to provide). Please review and comment as usual. Testing would be great also, as I only generated documentations for my own modules and David Schmidtt ones. This is great. You should send a note to the user list, so the rest of the community has a chance to test. -- Zeilinger's Fundamental Law: There is no Fundamental Law. - Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
[Puppet-dev] Re: [PATCH 0/3] Fix #1664 - Inline documentation
On 1/11/08 8:54, David Schmitt wrote: Brice Figureau schrieb: [*]: I generated the output from its git repository located here: http://git.black.co.at/ I didn't ask for permission, so I hope this isn't an issue to republish his comments. No problem at all! Thanks for choosing my repos :) I couldn't publish my own (because it's my company one and it has no _comments_ at all :-D). Yours was big enough that it was a challenge to make my code working with it. Yours also has comments, which is good :-) The docs really look great and are already a great asset for making the modules more accessible. On the downside, I cannot ignore anymore that my comments are _really_ lacking :-D lol Where can I find more info on how to author markup to improve the infos that are parsed? The markup used is the RDoc one. You'll find the documentation here: http://rdoc.sourceforge.net/doc/index.html You can ignore pretty much everything, and scroll down to Markup. The ruby only directives of the markup (:yield:, :notnew:...) are ignored by my system. -- Brice Figureau Days of Wonder http://www.daysofwonder.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Developers group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~--~~~~--~~--~--~---
