Mark & Matt, Cool. While I like the idea of @SupportsDynamicProperties, especially if it has a description, I'd also like a guaranteed way to get the dynamic PropertyDescriptor so I can interrogate it further. I'd be interested in knowing whether it supports expression language, and/or how the value of the property is validated.
On that note, I was thinking it may be helpful to add either an annotation or a method to the Validator interface that could provide a text description about what it is validating. Is this useful or overkill? Dan Bress Software Engineer ONYX Consulting Services ________________________________________ From: Mark Payne <[email protected]> Sent: Friday, January 16, 2015 11:14 AM To: [email protected] Subject: RE: Processor documentation plugin It might make sense to have an annotation for this: @SupportsDynamicProperties Perhaps even allow developer to provide a string for how they are used: @SupportsDynamicProperties("Name of dynamic property becomes attribute name; value becomes attribute value.") I've begun ticket #6: https://issues.apache.org/jira/browse/NIFI-6 but I have not pushed anything to a branch or anything yet. Basically, I deprecated the existing annotations and created 3 new packages for them. Instead of org.apache.nifi.processor.annotation, we will have: org.apache.nifi.annotation.documentationorg.apache.nifi.annotation.behaviororg.apache.nifi.annotation.lifecycle So the @SupportsDynamicProperties could go under 'behavior' if that makes sense. > Date: Fri, 16 Jan 2015 11:09:28 -0500 > Subject: Re: Processor documentation plugin > From: [email protected] > To: [email protected] > > Dan, > > We discussed item 1 awhile back and even created a ticket for it. However, > there was always higher priority work that took precedence. I am not sure > if that ticket was ever created in JIRA for Apache NiFi. If not, we should. > I believe the current mechanism allows the processor to decide if a given > dynamic property should be a supported. I think we would need a more > general indicator that dynamic properties are supported at all to trigger > that button state. This would likely help with item 2 as well. > > Matt > > On Fri, Jan 16, 2015 at 10:45 AM, Daniel Bress <[email protected]> > wrote: > > > Mark, > > Sounds good. I'm on board. I was thinking of having the > > ControllerServices/ReportingTask's documentation include an XML template, > > but if those will be configurable via the UI then that will not be needed. > > I'm on board with your rev 1 / rev 2 suggestions too. I'll get to work > > accordingly. > > > > > > Matt/Mark, > > Two UI/API questions: > > 1) If a processor does not support dynamic properties, why does the UI let > > them hit [+] New Property? > > 2) If I have an instance of Processor class, I can call > > getSupportedProperties() on it to get a list of the required/optional > > properties. But I don't have a way to know if that processor supports > > dynamic properties or not. Supporting dynamic properties or not was > > something I was planning on reflecting in the documentation. Is there any > > way to detect this currently? > > > > Thanks, > > Dan Bress > > Software Engineer > > ONYX Consulting Services > > > > ________________________________________ > > From: Mark Payne <[email protected]> > > Sent: Wednesday, January 14, 2015 8:51 PM > > To: [email protected] > > Subject: RE: Processor documentation plugin > > > > Dan, > > I think this is great! > > I would recommend we slate this for the 0.1.0 version, which includes > > putting Controller Services and Reporting Tasks in the UI. But by the same > > token I think that makes your first bulletin for "rev 2" more of a priority > > for "rev 1" if we can get it there. I could pitch in and help if needed. > > I'd also recommend you put the Capability Description in for Rev 1 as > > that's arguably the most important part of the docs (which may make the > > Tags very easy at that point??) > > The rest of what you have listed for "rev 2" i would agree should be there. > > > > Very excited to see this happening! > > Thanks-Mark > > > > > From: [email protected] > > > To: [email protected] > > > Subject: Re: Processor documentation plugin > > > Date: Thu, 15 Jan 2015 01:39:44 +0000 > > > > > > Mark et al, > > > Ok, I think I hear everyone's concerns and ideas. I will challenge > > myself to come up with a solution that makes everyone happy all the time. > > > > > > The goals of this effort will be: > > > - Machine generate as much of the basic(processor description, > > property description, relationship description) info as possible. This > > will make the developers lives better by storing this information in one > > place, and make users lives better by having the "inline" documentation > > match the "usage" documentation. This will also ensure that all the > > documentation is formatted consistently. > > > - Allow the developer to optionally provide additional html and > > images that would be linked from the machine generated documentation > > > - Provide a way for the documentation to be generated statically > > and linked from the Apache NiFi webpage > > > > > > A few other 'nice-to-have' goals(probably for rev 2): > > > - Support documenting ControllerServices and ReportingTasks is a > > similar way > > > -Maybe leverage existing annotations on these types? E.g > > @Tags, @CapabilityDescription... > > > - Provide new 'documentation' annotations that would allow a > > developer to describe the FlowFile attributes a processor uses on input, > > and which attributes a processor modifies on output > > > - Provide new 'documentation' annotations that describe the > > expected input format of the flow file content > > > - Provide new 'documentation' annotations that describe whether > > the processor modifies the flow file content > > > - Provide new 'documentation' annotations that describe the format > > of the outbound flow file content > > > - Provide new 'documentation' annotations that describe other NiFi > > artifacts(processors, controller services, reporting tasks) that are > > related to this one, e.g. "See Also ..." > > > - Consider if/how current Processor annotations should be > > displayed (@SideEffectFree, @TriggerSerially, @TriggerWhenEmpty..., > > @EventDriven) > > > > > > > > > High level implementation plan: > > > - Develop a utility class to read an AbstractProcessor(and > > existing HTML documention) and generated the basic documentation with links > > to existing HTML > > > - Integrate this class into the framework such that all > > documentation displayed within a NiFi instance is generated by the utility > > > - Integrate this utility into something that can take a NiFi > > build, dig through the nars, find the > > Processors/ControllerServices/ReportingTasks and generate their appropriate > > HTML documentation so that it can be stored somewhere and linked from the > > Apache NiFi website. > > > > > > If this feels like the wrong path, or if you have any other ideas > > please speak up. Thanks! > > > > > > Dan Bress > > > Software Engineer > > > ONYX Consulting Services > > > > > > ________________________________________ > > > From: Mark Payne <[email protected]> > > > Sent: Monday, January 12, 2015 3:40 PM > > > To: [email protected] > > > Subject: RE: Processor documentation plugin > > > > > > Dan, > > > Yes, that's what I'm saying. Right now, it's all done manually by hand, > > and it's a pain. The down side i think to doing it via a Maven plugin is > > that if the user wants to modify it, they have to rebuild, pull the > > generated html out of the target directory, and then modify it and put it > > in the src/main/resources/docs. Then, if they change something they have to > > do this again. If it's done on-the-fly, rather than building static content > > with a plugin, that doesn't have to happen. > > > But you may come up with something much better than I'm envisioning :) > > Having a util to do it is a good call though. Then it could be used in a > > maven plugin, by the app, or wherever is most appropriate. > > > Thanks-Mark > > > > > > > From: [email protected] > > > > To: [email protected] > > > > Subject: Re: Processor documentation plugin > > > > Date: Mon, 12 Jan 2015 20:34:03 +0000 > > > > > > > > Mark, > > > > Sorry.. I missed your reply in the thread. > > > > > > > > Are you saying that going forward, you want to auto generate the > > doc and allow the developer to include a "more info" link? I wanted to > > confirm that it doesn't work that way now. > > > > > > > > I'll think about this a little more. Might write some code to > > generate the documentation from an AbstractProcessor class, then decide > > later where to integrate it(maven plugin, or somewhere else) > > > > > > > > Thanks! > > > > > > > > Dan Bress > > > > Software Engineer > > > > ONYX Consulting Services > > > > > > > > ________________________________________ > > > > From: Mark Payne <[email protected]> > > > > Sent: Sunday, January 11, 2015 1:23 PM > > > > To: [email protected] > > > > Subject: Re: Processor documentation plugin > > > > > > > > Dan, > > > > > > > > Matt and I discussed this approach before but ultimately decided that > > we like the idea of just auto generating the docs on demand with a "More > > Info" type of link that lets you add more in html format if you care to do > > so. > > > > > > > > Others' thoughts and ideas are welcome though! > > > > > > > > Sent from my iPhone > > > > > > > > > On Jan 11, 2015, at 12:32 PM, Daniel Bress <[email protected]> > > wrote: > > > > > > > > > > Hi All, > > > > > > > > > > Just pulled down the code out of Git and got everything up and > > running on Windows. This was pretty simple, and the UI looks great under > > chrome. > > > > > > > > > > > > > > > I was thinking of writing a maven plugin to generate the processor > > usage HTML documentation. > > > > > > > > > > > > > > > Looking through the existing processor documentation, it seems > > like most of this information comes directly from the code in the > > processors, via @CapabilityDescription, and the PropertyDescriptors and > > Relationships. Then if your processor only requires this level of > > documentation in the HTML format, you could tie this plugin to the build, > > and keep your inline processor documentation in sync with the HTML > > documentation. > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > Dan Bress > > > > > Software Engineer > > > > > ONYX Consulting Services > >
