Re: Cron syntax

[Joe Witt]

Alan nailed it.  Quartz supported the finer resolution and some folks
found that useful.  It would be totally fair to ask that given 5
fields we just add a 6th with an implied 0 (for example).  A good JIRA
if you feel that is helpful.

On Wed, Sep 9, 2015 at 8:01 AM, Alan Jackoway  wrote:
> I believe that NiFi uses the Quartz cron syntax, which is described here:
> http://quartz-scheduler.org/api/2.2.0/org/quartz/CronExpression.html
>
> As for why, I believe it is because NiFi uses the Quartz java library to
> schedule processes, so it kept with Quartz syntax rather than translating
> it. I'm not sure why Quartz uses 6 instead of 5. Probably for more precision
> so that you can schedule down to the second if you want to.
>
> Alan
>
> On Wed, Sep 9, 2015 at 1:00 PM, John Titus 
> wrote:
>>
>> Just curious - why does the NiFi cron syntax differ from crontab syntax,
>> in that there is a "seconds" field in NiFi cron that I don't think crontab
>> uses? It always throws me that there's an extra field there.
>
>

[DISCUSS] Feature proposal: Streamline visual flow design

[Rob Moran]

There has been recent discussion around UI enhancements with the goal of
streamlining visual flow design. Please consider the following enhancements
and concepts for proposed solutions. Do you have any objections? If so,
please share your thoughts and ideas for alternate solutions to streamline
visual flow design in NiFi's GUI.


*Enhancement 1*Enable quicker, more efficient access to both known and not
yet known processors.


*Issue*The current interaction of dropping a processor on the graph and
being prompted with a dialog helps a user who does not know exactly which
one they need. However, as the number of processors increase, the current
methods of finding what you need become increasingly difficult. And for
those users who know exactly what processor they want, routine interaction
with the dialog becomes rather cumbersome.

*Concept for Proposed Solution*
Present logical groupings of processors to the user. Ideas include
usage-generated categories like ‘recent’ and ‘popular,’ along with
categories such as those defined by the Enterprise Integration Patterns
(e.g., mediate, route, transform) and perhaps further subcategories if
applicable. These options would be accessible from the main UI as well as
the add processor dialog.

Other ideas include 'pinning' processors you routinely use for quick
access, setting a default drag-n-drop processor, and assigning keyboard
shortcuts to quickly add a favorite to the graph.

Design decisions made here could also serve as a model for placing other
elements onto the graph such as templates.


*Enhancement 2*Provide visual distinction to processor types.


*Issue*When viewing a flow on the graph, all processor blocks look the
same. As a result, users must rely on processor names alone to interpret
what they are doing and how the given flow is working together.

*Concept for Proposed Solution*
Introduce some combination of iconography, unique styling, and more
descriptive labeling to processor blocks. As mentioned earlier, looking to
the Enterprise Integration Patterns could provide cues for visually
distinct icons and labeling. Unique styling could occur at various zoom
levels and/or screen resolution to better respond to user needs.

*Enhancement 3*
Give users the choice to be prompted immediately with a configuration
dialog after they place a processor, draw a connection, etc. on the graph.

*Issue*
Currently there is inconsistency with the interaction. Place a processor -
nothing. Draw a connection - configuration dialog pops up.

*Concept for Proposed Solution*
Part 1 - Decide on a consistent default behavior. Part 2 - Provide the user
the ability to reverse the behavior. One thought is to include a toggle in
each configuration dialog giving the user control over the behavior while
in context. Additionally, there could be a user preferences area where they
could make global changes. A user preferences area could come into play
with potential solutions proposed in Enhancement 1 as well.
-- 
Rob

Re: [DISCUSS] Feature proposal: Streamline visual flow design

[Corey Flowers]

I also think 1 and 2 seem like good ideas but having built hundreds of
flows, I would not want a configuration window to open automatically when a
processor is added to the graph. From an operations standpoint, you are
designing the flow was you add and remove the processor to work through
different ideas because there maybe 15 different ways to accomplish the
desired result. You don't want configuration windows to pop up every time
you are working through an idea or process on the graph.

There's my 2 cents worth a penny.
Corey

Sent from my iPhone

On Sep 10, 2015, at 1:31 PM, Brandon DeVries  wrote:

Rob,

I have no real objections to (1) or (2).  (1) seems useful and can probably
be implemented without radical changes.  (2) might be harder, but if done
well could be a good thing.

I object to the characterization of "inconsistency" in (3).  Placing a
processor is fundamentally different than drawing a connection.  I would
think of drawing the connection as a configuration step in itself
(configuring an outgoing relationship from one processor to connect to
another).  In other words, drawing a connection could be seen as saying you
want to connect a relationship on one processor to another... but which
one?  Being able to establish an unconfigured, "invalid" connection between
processors seems like a recipe for confusion.  Obviously this is all my
opinion, but I don't think you should ever be able to establish a
connection that isn't valid.  Having said that, if forcing immediate
configuration of a processor after placing it was an option that could be
enabled, I don't see any harm.  However, I would vote that the default
behavior remain as it is.

Brandon


On Thu, Sep 10, 2015 at 12:55 PM, Rob Moran  wrote:

> There has been recent discussion around UI enhancements with the goal of
> streamlining visual flow design. Please consider the following enhancements
> and concepts for proposed solutions. Do you have any objections? If so,
> please share your thoughts and ideas for alternate solutions to streamline
> visual flow design in NiFi's GUI.
>
>
> *Enhancement 1*Enable quicker, more efficient access to both known and
> not yet known processors.
>
>
> *Issue*The current interaction of dropping a processor on the graph and
> being prompted with a dialog helps a user who does not know exactly which
> one they need. However, as the number of processors increase, the current
> methods of finding what you need become increasingly difficult. And for
> those users who know exactly what processor they want, routine interaction
> with the dialog becomes rather cumbersome.
>
> *Concept for Proposed Solution*
> Present logical groupings of processors to the user. Ideas include
> usage-generated categories like ‘recent’ and ‘popular,’ along with
> categories such as those defined by the Enterprise Integration Patterns
> (e.g., mediate, route, transform) and perhaps further subcategories if
> applicable. These options would be accessible from the main UI as well as
> the add processor dialog.
>
> Other ideas include 'pinning' processors you routinely use for quick
> access, setting a default drag-n-drop processor, and assigning keyboard
> shortcuts to quickly add a favorite to the graph.
>
> Design decisions made here could also serve as a model for placing other
> elements onto the graph such as templates.
>
>
> *Enhancement 2*Provide visual distinction to processor types.
>
>
> *Issue*When viewing a flow on the graph, all processor blocks look the
> same. As a result, users must rely on processor names alone to interpret
> what they are doing and how the given flow is working together.
>
> *Concept for Proposed Solution*
> Introduce some combination of iconography, unique styling, and more
> descriptive labeling to processor blocks. As mentioned earlier, looking to
> the Enterprise Integration Patterns could provide cues for visually
> distinct icons and labeling. Unique styling could occur at various zoom
> levels and/or screen resolution to better respond to user needs.
>
> *Enhancement 3*
> Give users the choice to be prompted immediately with a configuration
> dialog after they place a processor, draw a connection, etc. on the graph.
>
> *Issue*
> Currently there is inconsistency with the interaction. Place a processor -
> nothing. Draw a connection - configuration dialog pops up.
>
> *Concept for Proposed Solution*
> Part 1 - Decide on a consistent default behavior. Part 2 - Provide the
> user the ability to reverse the behavior. One thought is to include a
> toggle in each configuration dialog giving the user control over the
> behavior while in context. Additionally, there could be a user preferences
> area where they could make global changes. A user preferences area could
> come into play with potential solutions proposed in Enhancement 1 as well.
> --
> Rob
>

Re: [DISCUSS] Feature proposal: Streamline visual flow design

[Jennifer Barnabee]

Rob,
I also like enhancements 1 & 2. For the ability to pin processors or pull
recent/popular processors from a user-generated list, can we make that
something that is expandable/collapsible? While building a flow, I think
people might want that type of thing open. But then later, while working
with a flow, they'd want it out of the way.
Great ideas!
-Jenn

On Thu, Sep 10, 2015 at 12:55 PM, Rob Moran  wrote:

> There has been recent discussion around UI enhancements with the goal of
> streamlining visual flow design. Please consider the following enhancements
> and concepts for proposed solutions. Do you have any objections? If so,
> please share your thoughts and ideas for alternate solutions to streamline
> visual flow design in NiFi's GUI.
>
>
> *Enhancement 1*Enable quicker, more efficient access to both known and
> not yet known processors.
>
>
> *Issue*The current interaction of dropping a processor on the graph and
> being prompted with a dialog helps a user who does not know exactly which
> one they need. However, as the number of processors increase, the current
> methods of finding what you need become increasingly difficult. And for
> those users who know exactly what processor they want, routine interaction
> with the dialog becomes rather cumbersome.
>
> *Concept for Proposed Solution*
> Present logical groupings of processors to the user. Ideas include
> usage-generated categories like ‘recent’ and ‘popular,’ along with
> categories such as those defined by the Enterprise Integration Patterns
> (e.g., mediate, route, transform) and perhaps further subcategories if
> applicable. These options would be accessible from the main UI as well as
> the add processor dialog.
>
> Other ideas include 'pinning' processors you routinely use for quick
> access, setting a default drag-n-drop processor, and assigning keyboard
> shortcuts to quickly add a favorite to the graph.
>
> Design decisions made here could also serve as a model for placing other
> elements onto the graph such as templates.
>
>
> *Enhancement 2*Provide visual distinction to processor types.
>
>
> *Issue*When viewing a flow on the graph, all processor blocks look the
> same. As a result, users must rely on processor names alone to interpret
> what they are doing and how the given flow is working together.
>
> *Concept for Proposed Solution*
> Introduce some combination of iconography, unique styling, and more
> descriptive labeling to processor blocks. As mentioned earlier, looking to
> the Enterprise Integration Patterns could provide cues for visually
> distinct icons and labeling. Unique styling could occur at various zoom
> levels and/or screen resolution to better respond to user needs.
>
> *Enhancement 3*
> Give users the choice to be prompted immediately with a configuration
> dialog after they place a processor, draw a connection, etc. on the graph.
>
> *Issue*
> Currently there is inconsistency with the interaction. Place a processor -
> nothing. Draw a connection - configuration dialog pops up.
>
> *Concept for Proposed Solution*
> Part 1 - Decide on a consistent default behavior. Part 2 - Provide the
> user the ability to reverse the behavior. One thought is to include a
> toggle in each configuration dialog giving the user control over the
> behavior while in context. Additionally, there could be a user preferences
> area where they could make global changes. A user preferences area could
> come into play with potential solutions proposed in Enhancement 1 as well.
> --
> Rob
>

Re: Hacker News

[Joe Witt]

Yeah John thanks for pointing this out.  That resulted in the apache
nifi site having 30,000+ visits yesterday.  Wow!

Some good questions worth responding too but I think this is the right
forum so probably will hold off until the questions occur here on
users@nao.

Thanks
Joe

On Wed, Sep 9, 2015 at 10:45 AM, John Titus
 wrote:
> Just wanted to make sure you all saw this:
>
> https://news.ycombinator.com/item?id=10190846
>
> It can add a lot of visibility to a project, especially if the maintainers 
> jump in to answer questions.

RE: [DISCUSS] Feature proposal: Streamline visual flow design

[Rick Braddy]

I am very supportive of #1 and #2.  On #1, it would be very helpful for 
Templates to be available for drag and drop operation and treated similarly to 
other classes of processors.

I would also suggest that toolbox drag/drop approach that Visio uses to 
organize libraries of icons is a proven approach and may be a useful design 
pattern to consider.  It lets the user open up several tool palettes and also 
control the order the palettes are displayed (top to bottom), so depending upon 
what project one is working on, the relevant objects are available above the 
fold for quick access.

I really like the idea of being able to “pin” certain processors/templates to 
the toolbar (or some area) for ease of access.  Again, I think processors and 
templates should be treated similarly, because certain use cases will involve 
deploying lots of prefabricated process groups as templates vs. wiring up 
mostly individual processors.

On #3, I understand the issue of auto-configuration upon drop of a processor 
could be obtrusive to initial flow layout.  The original suggestion was two 
parts: a) for processors that have required properties that must be configured, 
when the Configure menu item is selected by user, automatically open the 
Properties tab first.  Part b) was to auto-prompt for configuration upon drop 
of processor to canvas.

I believe 3.a would improve usability and save time.  3.b may be better as a 
user preference, but defaulted to current behavior (do not auto-open upon drop, 
as it creates too much friction around initial flow layout).  Perhaps 
double-clicking an un-configured processor would also make it faster to 
configure it, without having to right-click and press Configure, eliminating 
the need for 3.b altogether.  Today, double-click on a processor does nothing, 
so it seems to be available, if that’s its best use.

Thanks for writing this proposal up and circulating it, Rob.

Rick

From: Rob Moran [mailto:rmo...@gmail.com]
Sent: Thursday, September 10, 2015 11:56 AM
To: users@nifi.apache.org
Subject: [DISCUSS] Feature proposal: Streamline visual flow design


There has been recent discussion around UI enhancements with the goal of 
streamlining visual flow design. Please consider the following enhancements and 
concepts for proposed solutions. Do you have any objections? If so, please 
share your thoughts and ideas for alternate solutions to streamline visual flow 
design in NiFi's GUI.

Enhancement 1
Enable quicker, more efficient access to both known and not yet known 
processors.

Issue
The current interaction of dropping a processor on the graph and being prompted 
with a dialog helps a user who does not know exactly which one they need. 
However, as the number of processors increase, the current methods of finding 
what you need become increasingly difficult. And for those users who know 
exactly what processor they want, routine interaction with the dialog becomes 
rather cumbersome.

Concept for Proposed Solution
Present logical groupings of processors to the user. Ideas include 
usage-generated categories like ‘recent’ and ‘popular,’ along with categories 
such as those defined by the Enterprise Integration Patterns (e.g., mediate, 
route, transform) and perhaps further subcategories if applicable. These 
options would be accessible from the main UI as well as the add processor 
dialog.

Other ideas include 'pinning' processors you routinely use for quick access, 
setting a default drag-n-drop processor, and assigning keyboard shortcuts to 
quickly add a favorite to the graph.

Design decisions made here could also serve as a model for placing other 
elements onto the graph such as templates.

Enhancement 2
Provide visual distinction to processor types.

Issue
When viewing a flow on the graph, all processor blocks look the same. As a 
result, users must rely on processor names alone to interpret what they are 
doing and how the given flow is working together.

Concept for Proposed Solution
Introduce some combination of iconography, unique styling, and more descriptive 
labeling to processor blocks. As mentioned earlier, looking to the Enterprise 
Integration Patterns could provide cues for visually distinct icons and 
labeling. Unique styling could occur at various zoom levels and/or screen 
resolution to better respond to user needs.

Enhancement 3
Give users the choice to be prompted immediately with a configuration dialog 
after they place a processor, draw a connection, etc. on the graph.

Issue
Currently there is inconsistency with the interaction. Place a processor - 
nothing. Draw a connection - configuration dialog pops up.

Concept for Proposed Solution
Part 1 - Decide on a consistent default behavior. Part 2 - Provide the user the 
ability to reverse the behavior. One thought is to include a toggle in each 
configuration dialog giving the user control over the behavior while in 
context. Additionally, there could be a user preferences area where they could 
make