+1 it's so obvious it should've always been there :) As for offline access. Well, every time I showed an offline help section to every user thry were startled :) tells about awareness. The first hunch is always to google insread of going to a locally hosted doc, so it won't be an issue, IMO.
Andrew On Mon, Jan 28, 2019, 8:13 AM Bryan Bende <bbe...@gmail.com> wrote: > What does everyone think about bumping the "Developer" section of the > docs ahead of "Processors" so that it's easier to find? > > Here is what it would look like - > https://gist.github.com/bbende/279c983f5c80d4fad1431ae81862060f > > I also added links to the "Contributor Guide" and the "Maven Projects" > page since I think it would be helpful to make the Developer section > be the one-stop place people look for development help, although I can > see an argument for not mixing wiki content with the docs content. > > One issue would be if we want the docs to be fully usable in an > off-line environment, then linking to the wiki won't work, so we could > remove those links, or convert those pages to be part of the docs now > that they are more stable. > > For reference, we already have some links in the docs to the wiki: > > > https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#supplying-a-contribution > > On Sat, Jan 26, 2019 at 10:49 AM Joe Witt <joe.w...@gmail.com> wrote: > > > > ...we can. but the discussion is about how to both lower the bar and > offer > > more routes to the bar. > > > > On Sat, Jan 26, 2019, 10:45 AM Otto Fowler <ottobackwa...@gmail.com > wrote: > > > > > Why wouldn’t we make the archetypes do this? > > > > > > > > > On January 25, 2019 at 18:04:25, Andy LoPresto (alopre...@apache.org) > > > wrote: > > > > > > James, > > > > > > I’m wondering if a page outlining a toy processor (something like > > > `CountText` or `ReverseContent`) and doing a line-by-line annotation > from a > > > developer’s perspective would be helpful. It could be a few sections: > > > > > > 1. How to get to this point > > > * running the maven archetype > > > * choosing the directory to install to > > > * putting the class name in the manifest file > > > * etc. > > > 2. The code > > > * here’s the class > > > * here’s what extending AbstractProcessor gets you, etc. A lot of this > is > > > currently in the Developer Guide, but in textbook mode > > > * here’s how to modify some code (i.e. write one line of Java that > switches > > > it from straight content pass-through to reversing the text) > > > * here’s how to make a unit test (introduce the TestRunner framework, > etc.) > > > 3. Running, building, installing > > > * Run your unit test from the IDE/maven > > > * Build the NAR module > > > * Install the NAR in NiFi lib/ or custom/ > > > * Restart NiFi > > > * See the NAR loaded in the log > > > * Deploy the component on the canvas > > > > > > I imagine this being written more conversationally/blog-like than most > of > > > our current reference documentation to be used as a split-screen > > > walkthrough. Each section could certainly link to the existing detailed > > > documentation for various topics, like the processor lifecycle, etc. > > > > > > Does this sounds like something that would have helped you? > > > > > > Andy LoPresto > > > alopre...@apache.org > > > alopresto.apa...@gmail.com > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > > > > On Jan 25, 2019, at 1:59 PM, James Srinivasan < > > > james.sriniva...@gmail.com> > > > wrote: > > > > > > > > 9) Oh, and the wiki is a little hard to navigate and the contents > rather > > > patchy > > > > > > > > On Fri, 25 Jan 2019 at 21:57, James Srinivasan > > > > <james.sriniva...@gmail.com> wrote: > > > >> > > > >> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I > > > >> realise I could and possibly should submit PRs :) > > > >> > > > >> 1) I'm used to Java and Maven, so used the archetype. It worked > fine, > > > >> it would have been nice it if set up unit tests for me. > > > >> 2) The User and Developer documentation is great and comprehensive. > > > >> Finding the developer docs is a little painful (handful of items at > > > >> the end of a scrolling list of 200+ processors) > > > >> 3) The Developer docs could possibly do with a little more clarity > on > > > >> processor lifetime i.e. what is called when ^h^h^h - skimming back > > > >> over the docs, it looks pretty clear now > > > >> 4) Some example code for common operations e.g. getting/setting > > > >> attributes or reading/writing/modifying flowfile content would be > > > >> great. > > > >> 5) When using existing processors for inspiration, best practices > > > >> weren't always clear e.g. some generated properties inside > > > >> getSupportedPropertyDescriptors(), others generated a private static > > > >> list on init and returned that. Such differences are inevitable in a > > > >> large project, but it would be nice to have something blessed to > start > > > >> from. > > > >> 6) (Minor niggle - layout of the docs doesn't work great on a phone > > > screen) > > > >> 7) I couldn't find (m?)any docs about the Groovy scripting API, but > > > >> the great blog posts by Matt Burgess and others were invaluable > > > >> 8) In case this all sounds too negative, NiFi is fab! > > > >> > > > >> On Fri, 25 Jan 2019 at 18:47, Andrew Grande <apere...@gmail.com> > wrote: > > > >>> > > > >>> I am not against the archetype. But we need to spell out every > step of > > > the > > > >>> way. I'd like to see a user thinking about their custom logic ASAP > > > rather > > > >>> than fighting the tools to get started. Those steps should be > > > brain-dead, > > > >>> just reflexes, if you know what I mean. Hell, let them create a > custom > > > >>> processor project or prototype in a script by accident even! :) > > > >>> > > > >>> On Fri, Jan 25, 2019, 10:43 AM Bryan Bende <bbe...@gmail.com> > wrote: > > > >>> > > > >>>> That makes sense about the best practice for deploying to an > > > >>>> additional lib directory. > > > >>>> > > > >>>> So for the project structure you are saying it would be easier to > have > > > >>>> a repo somewhere with essentially the same thing that is in the > > > >>>> archetype, but they just clone it and rename it themselves (what > the > > > >>>> archetype does for you)? > > > >>>> > > > >>>> Something that I think would be awesome is if we could provide a > > > >>>> web-based project initializer that would essentially run the > archetype > > > >>>> behind the scenes and then let you download the archive of the > code, > > > >>>> just like the spring-boot starter [1]. Not sure if their > initializr is > > > >>>> something that can be re-used and customized [2]. > > > >>>> > > > >>>> The problem is we would need to host that somewhere. > > > >>>> > > > >>>> [1] https://start.spring.io/ > > > >>>> [2] https://github.com/spring-io/initializr > > > >>>> > > > >>>> On Fri, Jan 25, 2019 at 12:56 PM Andrew Grande < > apere...@gmail.com> > > > wrote: > > > >>>>> > > > >>>>> We assume they create new projects from archetypes every day. > They > > > don't. > > > >>>>> > > > >>>>> We also assume they know how to deploy new NARs. Most don't. > > > Especially > > > >>>> if > > > >>>>> we want them to follow best practices and create an additional > NAR > > > >>>> bundles > > > >>>>> directory entry im the config (vs dumping into nifi lib). > > > >>>>> > > > >>>>> I can attest that I feel a bit lost myself every time I need to > come > > > back > > > >>>>> to this and refresh my brain synapses. If we could make these not > > > require > > > >>>>> any of that and make simple thinga dead simple.... > > > >>>>> > > > >>>>> Andrew > > > >>>>> > > > >>>>> On Fri, Jan 25, 2019, 9:47 AM Bryan Bende <bbe...@gmail.com> > wrote: > > > >>>>> > > > >>>>>> Andrew, > > > >>>>>> > > > >>>>>> I'm not disagreeing with your points, but I'm curious how you > see > > > >>>>>> those two ideas being different from the processor archetype > and the > > > >>>>>> wiki page with the archetype commands? > > > >>>>>> > > > >>>>>> Is it just that people don't know about it? > > > >>>>>> > > > >>>>>> -Bryan > > > >>>>>> > > > >>>>>> [1] > > > >>>>>> > > > >>>> > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions > > > >>>>>> > > > >>>>>> On Fri, Jan 25, 2019 at 12:23 PM Otto Fowler < > > > ottobackwa...@gmail.com> > > > > > > >>>>>> wrote: > > > >>>>>>> > > > >>>>>>> I think this ties into my other discuss thread on refreshing > the > > > >>>>>> archetypes > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> On January 25, 2019 at 11:50:10, Andrew Grande ( > apere...@gmail.com > > > ) > > > >>>>>> wrote: > > > >>>>>>> > > > >>>>>>> I consistently see my users struggling when they move up the > nifi > > > >>>> food > > > >>>>>>> chain and start looking at custom processors. The good content > > > about > > > >>>>>>> prototyping processsors via scripting processors and finalizing > > > with > > > >>>> a > > > >>>>>> full > > > >>>>>>> NAR bundle is everywhere but where it should be. > > > >>>>>>> > > > >>>>>>> A few simple changes could help (not *more* docs). They are > great, > > > >>>> much > > > >>>>>>> better than in many other projecta, but people are already > drowning > > > >>>> in > > > >>>>>>> those. > > > >>>>>>> > > > >>>>>>> How about: > > > >>>>>>> > > > >>>>>>> + ISP has a pre-populated processor sceleton. A simple no-op to > > > fill > > > >>>> in > > > >>>>>> is > > > >>>>>>> miles better than a blank text area (which invokes a blank > stare). > > > >>>>>>> > > > >>>>>>> + As much as we may loook down on this, but... A simple guide > to a > > > >>>> full > > > >>>>>> NAR > > > >>>>>>> build as a series of copy/paste commands. > > > >>>>>>> > > > >>>>>>> There's more, but this should fit the context for now. > > > >>>>>>> > > > >>>>>>> Andrew > > > >>>>>>> > > > >>>>>>> On Fri, Jan 25, 2019, 8:13 AM Mike Thomsen < > mikerthom...@gmail.com > > > > > > > >>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> One of the changes we should make is to create a separate > guide > > > for > > > >>>>>>> product > > > >>>>>>>> vendors on how to build and maintain a bundle. We're at that > point > > > >>>>>> where > > > >>>>>>>> vendors will have to do it on their own as extension > providers, so > > > >>>> it > > > >>>>>>> would > > > >>>>>>>> be very helpful for them to have a simple and straight forward > > > >>>> document > > > >>>>>>>> showing them what should be there, best practices for > > > >>>> maintainability > > > >>>>>> and > > > >>>>>>>> where to announce it. > > > >>>>>>>> > > > >>>>>>>> On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende <bbe...@gmail.com > > > > > >>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> I think we have a lot more documentation than most projects, > but > > > >>>> I > > > >>>>>>>>> think an issue is that content is scattered in many different > > > >>>>>>>>> locations, and some of the docs are huge reference guides > where > > > >>>> it > > > >>>>>> can > > > >>>>>>>>> be hard to find all the pieces of what you are trying to do. > > > >>>>>>>>> > > > >>>>>>>>> The first thing a new contributor wants to do is get the code > > > >>>> and run > > > >>>>>>>>> a build, and we do have a quick-start guide linked to on the > > > >>>> site, > > > >>>>>> but > > > >>>>>>>>> I think there is a lot of extra information in there that is > not > > > >>>>>>>>> really relevant to someone just wanting get the code and > build. > > > >>>> We > > > >>>>>>>>> could have separate guides per OS like "Build NiFi on Linux", > > > >>>> "Build > > > >>>>>>>>> NiFi on Windows", etc, where each guide was 4-5 steps like: > > > >>>>>>>>> > > > >>>>>>>>> - Clone repo > > > >>>>>>>>> - checkout master > > > >>>>>>>>> - run maven > > > >>>>>>>>> - cd to assembly > > > >>>>>>>>> - ./bin/nifi.sh > > > >>>>>>>>> > > > >>>>>>>>> The next thing they want to do is contribute a change, and we > > > >>>> have a > > > >>>>>>>>> great contributor guide, but again I think there could be a > very > > > >>>>>> short > > > >>>>>>>>> tutorial for the most common steps: > > > >>>>>>>>> > > > >>>>>>>>> - fork repo > > > >>>>>>>>> - clone fork > > > >>>>>>>>> - create branch > > > >>>>>>>>> - make changes > > > >>>>>>>>> - push branch > > > >>>>>>>>> - submit pr > > > >>>>>>>>> > > > >>>>>>>>> and then say something like "for a more detailed description > of > > > >>>> the > > > >>>>>>>>> contribution process, please reference the Contributor > Guide". > > > >>>>>>>>> > > > >>>>>>>>> If we then make these getting started guides more prominent > > > >>>> right in > > > >>>>>>>>> the middle of the NiFi homepage, then maybe they will be > easier > > > >>>> to > > > >>>>>>>>> find for new community members. > > > >>>>>>>>> > > > >>>>>>>>> We can keep extending this idea to other common tasks beyond > just > > > >>>>>>>>> building and contributing. > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> On Thu, Jan 24, 2019 at 8:03 PM Andy LoPresto < > > > >>>> alopre...@apache.org> > > > >>>>>>>>> wrote: > > > >>>>>>>>>> > > > >>>>>>>>>> Hi folks, > > > >>>>>>>>>> > > > >>>>>>>>>> Based on some recent (and long-term) experiences, I wanted > to > > > >>>>>> discuss > > > >>>>>>>>> with the community what we could do to lower the barrier of > > > >>>> entry to > > > >>>>>>>> using > > > >>>>>>>>> & contributing to NiFi. I hope to get some good feedback from > > > >>>> both > > > >>>>>>>>> long-time and newer members, and determine some immediate > > > >>>> concrete > > > >>>>>>> steps > > > >>>>>>>> we > > > >>>>>>>>> can take. > > > >>>>>>>>>> > > > >>>>>>>>>> Problems identified: > > > >>>>>>>>>> * NiFi has a number of custom profiles, so a simple “mvn > clean > > > >>>>>>> install” > > > >>>>>>>>> in project root doesn’t get a new developer up and running > > > >>>>>> immediately > > > >>>>>>>>>> * The API is very well defined, but for new contributors, it > > > >>>> can > > > >>>>>> be a > > > >>>>>>>>> challenge to know where to put functionality, and building a > > > >>>> custom > > > >>>>>>>>> processor + NAR and deploying isn’t a one-step process > > > >>>>>>>>>> * Project size (and build size/time) is large. This can > > > >>>> restrict > > > >>>>>> the > > > >>>>>>>>> minimum hardware necessary, elongate the development cycle, > etc. > > > >>>>>>>>>> * Some new users do not receive mailing list replies > > > >>>>>>>>>> > > > >>>>>>>>>> Possible solutions: > > > >>>>>>>>>> * On a clean git clone, “mvn clean install” should build a > > > >>>> working > > > >>>>>>>>> instance. Maybe we provide a quickstart.sh script to handle > the > > > >>>>>> default > > > >>>>>>>>> maven build, change to the target directory, and start NiFi? > > > >>>>>>>>>> * Individual contributors have written excellent blogs, and > > > >>>>>>>>> documentation exists, but making it more prominent or more > easily > > > >>>>>>>> accessed > > > >>>>>>>>> could help? > > > >>>>>>>>>> * Extension registry will solve all the world’s problems > > > >>>> (related > > > >>>>>> to > > > >>>>>>>>> bundling and build time) > > > >>>>>>>>>> * Not sure about this one — I don’t know if it’s because > > > >>>> they’re > > > >>>>>> not > > > >>>>>>>>> subscribed, their mail client is blocking them, etc. > > > >>>>>>>>>> > > > >>>>>>>>>> I’ve said my bit, now I am eager to hear from other > community > > > >>>>>> members > > > >>>>>>>> on > > > >>>>>>>>> their experiences, steps that helped them, and suggestions > for > > > >>>> the > > > >>>>>>> future > > > >>>>>>>>> to continue to make the NiFi community welcoming to new > users. > > > >>>>>> Thanks. > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> Andy LoPresto > > > >>>>>>>>>> alopre...@apache.org > > > >>>>>>>>>> alopresto.apa...@gmail.com > > > >>>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B > 2F7D > > > >>>> EF69 > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>> > > > >>>> > > > >
