Hi All, Getting in on this thread a bit late. This is interesting work.
Is this functionality still slated to be rolled into a maven plugin? Could this be something that's implemented as a javadoc doclet, or does that api tie everything too closely to the static approach? Drew On Mon, Jan 19, 2015 at 11:49 AM, Daniel Bress <[email protected]> wrote: > Mark, > I've got the API for generating > Processor/ControllerService/ReportingTask documentation, and now am looking > at integrating that into the Framework during startup. I've got a few > questions about how to tie everything together. > > 1) Where should my code reside in the tree? I was going to create a new > jar/project under framework-bundle/framework called documentation. > 2) My stuff requires initialized instances of ConfigurableComponents. The > reason being most Processors set up their properties in their init() > methods. So what's the best way to get an instance of a > Processor/ControllerService/Reporting task? I was planning on kicking my > stuff out of the nifi-runtime/NiFi class. To get things going I've been > using stubbed out versions of the InitializationContext interfaces that I > wrote. Should I get the instances from the FlowController? > > I've got it tied in right now in a maybe hacky way, but the docs looks > good. I went through all the existing html, and deleted anything that > didn't add additional detail. Anything that did provide additional detail > I renamed to "additionalDetails.html" which my code detects and links from > the generated documentation. Also having NIFI-6 and NIFI-4 resolved will > make the documentation look better for ControllerServices and > ReportingTasks. > > Once I get it integrated and tested a bit I'll add the patch to the > ticket. Thanks for your help! > > Dan Bress > Software Engineer > ONYX Consulting Services > > ________________________________________ > From: Daniel Bress <[email protected]> > Sent: Friday, January 16, 2015 4:14 PM > To: [email protected] > Subject: Re: Processor documentation plugin > > Mark & Matt, > Cool. While I like the idea of @SupportsDynamicProperties, especially > if it has a description, I'd also like a guaranteed way to get the dynamic > PropertyDescriptor so I can interrogate it further. I'd be interested in > knowing whether it supports expression language, and/or how the value of > the property is validated. > > On that note, I was thinking it may be helpful to add either an annotation > or a method to the Validator interface that could provide a text > description about what it is validating. Is this useful or overkill? > > Dan Bress > Software Engineer > ONYX Consulting Services > > ________________________________________ > From: Mark Payne <[email protected]> > Sent: Friday, January 16, 2015 11:14 AM > To: [email protected] > Subject: RE: Processor documentation plugin > > It might make sense to have an annotation for this: > @SupportsDynamicProperties > Perhaps even allow developer to provide a string for how they are used: > @SupportsDynamicProperties("Name of dynamic property becomes attribute > name; value becomes attribute value.") > I've begun ticket #6: https://issues.apache.org/jira/browse/NIFI-6 but I > have not pushed anything to a branch or anything yet. Basically, I > deprecated the existing annotations and created 3 new packages for them. > Instead of org.apache.nifi.processor.annotation, we will have: > > org.apache.nifi.annotation.documentationorg.apache.nifi.annotation.behaviororg.apache.nifi.annotation.lifecycle > So the @SupportsDynamicProperties could go under 'behavior' if that makes > sense. > > > Date: Fri, 16 Jan 2015 11:09:28 -0500 > > Subject: Re: Processor documentation plugin > > From: [email protected] > > To: [email protected] > > > > Dan, > > > > We discussed item 1 awhile back and even created a ticket for it. > However, > > there was always higher priority work that took precedence. I am not sure > > if that ticket was ever created in JIRA for Apache NiFi. If not, we > should. > > I believe the current mechanism allows the processor to decide if a given > > dynamic property should be a supported. I think we would need a more > > general indicator that dynamic properties are supported at all to trigger > > that button state. This would likely help with item 2 as well. > > > > Matt > > > > On Fri, Jan 16, 2015 at 10:45 AM, Daniel Bress <[email protected]> > > wrote: > > > > > Mark, > > > Sounds good. I'm on board. I was thinking of having the > > > ControllerServices/ReportingTask's documentation include an XML > template, > > > but if those will be configurable via the UI then that will not be > needed. > > > I'm on board with your rev 1 / rev 2 suggestions too. I'll get to > work > > > accordingly. > > > > > > > > > Matt/Mark, > > > Two UI/API questions: > > > 1) If a processor does not support dynamic properties, why does the UI > let > > > them hit [+] New Property? > > > 2) If I have an instance of Processor class, I can call > > > getSupportedProperties() on it to get a list of the required/optional > > > properties. But I don't have a way to know if that processor supports > > > dynamic properties or not. Supporting dynamic properties or not was > > > something I was planning on reflecting in the documentation. Is there > any > > > way to detect this currently? > > > > > > Thanks, > > > Dan Bress > > > Software Engineer > > > ONYX Consulting Services > > > > > > ________________________________________ > > > From: Mark Payne <[email protected]> > > > Sent: Wednesday, January 14, 2015 8:51 PM > > > To: [email protected] > > > Subject: RE: Processor documentation plugin > > > > > > Dan, > > > I think this is great! > > > I would recommend we slate this for the 0.1.0 version, which includes > > > putting Controller Services and Reporting Tasks in the UI. But by the > same > > > token I think that makes your first bulletin for "rev 2" more of a > priority > > > for "rev 1" if we can get it there. I could pitch in and help if > needed. > > > I'd also recommend you put the Capability Description in for Rev 1 as > > > that's arguably the most important part of the docs (which may make the > > > Tags very easy at that point??) > > > The rest of what you have listed for "rev 2" i would agree should be > there. > > > > > > Very excited to see this happening! > > > Thanks-Mark > > > > > > > From: [email protected] > > > > To: [email protected] > > > > Subject: Re: Processor documentation plugin > > > > Date: Thu, 15 Jan 2015 01:39:44 +0000 > > > > > > > > Mark et al, > > > > Ok, I think I hear everyone's concerns and ideas. I will > challenge > > > myself to come up with a solution that makes everyone happy all the > time. > > > > > > > > The goals of this effort will be: > > > > - Machine generate as much of the basic(processor description, > > > property description, relationship description) info as possible. This > > > will make the developers lives better by storing this information in > one > > > place, and make users lives better by having the "inline" documentation > > > match the "usage" documentation. This will also ensure that all the > > > documentation is formatted consistently. > > > > - Allow the developer to optionally provide additional html and > > > images that would be linked from the machine generated documentation > > > > - Provide a way for the documentation to be generated > statically > > > and linked from the Apache NiFi webpage > > > > > > > > A few other 'nice-to-have' goals(probably for rev 2): > > > > - Support documenting ControllerServices and ReportingTasks is > a > > > similar way > > > > -Maybe leverage existing annotations on these types? E.g > > > @Tags, @CapabilityDescription... > > > > - Provide new 'documentation' annotations that would allow a > > > developer to describe the FlowFile attributes a processor uses on > input, > > > and which attributes a processor modifies on output > > > > - Provide new 'documentation' annotations that describe the > > > expected input format of the flow file content > > > > - Provide new 'documentation' annotations that describe whether > > > the processor modifies the flow file content > > > > - Provide new 'documentation' annotations that describe the > format > > > of the outbound flow file content > > > > - Provide new 'documentation' annotations that describe other > NiFi > > > artifacts(processors, controller services, reporting tasks) that are > > > related to this one, e.g. "See Also ..." > > > > - Consider if/how current Processor annotations should be > > > displayed (@SideEffectFree, @TriggerSerially, @TriggerWhenEmpty..., > > > @EventDriven) > > > > > > > > > > > > High level implementation plan: > > > > - Develop a utility class to read an AbstractProcessor(and > > > existing HTML documention) and generated the basic documentation with > links > > > to existing HTML > > > > - Integrate this class into the framework such that all > > > documentation displayed within a NiFi instance is generated by the > utility > > > > - Integrate this utility into something that can take a NiFi > > > build, dig through the nars, find the > > > Processors/ControllerServices/ReportingTasks and generate their > appropriate > > > HTML documentation so that it can be stored somewhere and linked from > the > > > Apache NiFi website. > > > > > > > > If this feels like the wrong path, or if you have any other ideas > > > please speak up. Thanks! > > > > > > > > Dan Bress > > > > Software Engineer > > > > ONYX Consulting Services > > > > > > > > ________________________________________ > > > > From: Mark Payne <[email protected]> > > > > Sent: Monday, January 12, 2015 3:40 PM > > > > To: [email protected] > > > > Subject: RE: Processor documentation plugin > > > > > > > > Dan, > > > > Yes, that's what I'm saying. Right now, it's all done manually by > hand, > > > and it's a pain. The down side i think to doing it via a Maven plugin > is > > > that if the user wants to modify it, they have to rebuild, pull the > > > generated html out of the target directory, and then modify it and put > it > > > in the src/main/resources/docs. Then, if they change something they > have to > > > do this again. If it's done on-the-fly, rather than building static > content > > > with a plugin, that doesn't have to happen. > > > > But you may come up with something much better than I'm envisioning > :) > > > Having a util to do it is a good call though. Then it could be used in > a > > > maven plugin, by the app, or wherever is most appropriate. > > > > Thanks-Mark > > > > > > > > > From: [email protected] > > > > > To: [email protected] > > > > > Subject: Re: Processor documentation plugin > > > > > Date: Mon, 12 Jan 2015 20:34:03 +0000 > > > > > > > > > > Mark, > > > > > Sorry.. I missed your reply in the thread. > > > > > > > > > > Are you saying that going forward, you want to auto generate the > > > doc and allow the developer to include a "more info" link? I wanted to > > > confirm that it doesn't work that way now. > > > > > > > > > > I'll think about this a little more. Might write some code to > > > generate the documentation from an AbstractProcessor class, then decide > > > later where to integrate it(maven plugin, or somewhere else) > > > > > > > > > > Thanks! > > > > > > > > > > Dan Bress > > > > > Software Engineer > > > > > ONYX Consulting Services > > > > > > > > > > ________________________________________ > > > > > From: Mark Payne <[email protected]> > > > > > Sent: Sunday, January 11, 2015 1:23 PM > > > > > To: [email protected] > > > > > Subject: Re: Processor documentation plugin > > > > > > > > > > Dan, > > > > > > > > > > Matt and I discussed this approach before but ultimately decided > that > > > we like the idea of just auto generating the docs on demand with a > "More > > > Info" type of link that lets you add more in html format if you care > to do > > > so. > > > > > > > > > > Others' thoughts and ideas are welcome though! > > > > > > > > > > Sent from my iPhone > > > > > > > > > > > On Jan 11, 2015, at 12:32 PM, Daniel Bress < > [email protected]> > > > wrote: > > > > > > > > > > > > Hi All, > > > > > > > > > > > > Just pulled down the code out of Git and got everything up and > > > running on Windows. This was pretty simple, and the UI looks great > under > > > chrome. > > > > > > > > > > > > > > > > > > I was thinking of writing a maven plugin to generate the > processor > > > usage HTML documentation. > > > > > > > > > > > > > > > > > > Looking through the existing processor documentation, it seems > > > like most of this information comes directly from the code in the > > > processors, via @CapabilityDescription, and the PropertyDescriptors and > > > Relationships. Then if your processor only requires this level of > > > documentation in the HTML format, you could tie this plugin to the > build, > > > and keep your inline processor documentation in sync with the HTML > > > documentation. > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > Dan Bress > > > > > > Software Engineer > > > > > > ONYX Consulting Services > > > >
