Re: Advice on ListS3 Processor

[Sivaprasanna]

Jim,

${filename} is a NiFi expression language that is used to get the name of
the flowfile that is in the flow. For ex: assume this simple pipeline.
GetFile -> UpdateAttribute -> PutFile. GetFile will read the files from the
input directory and each flowfile will have an attribute ‘filename’ which
is the actual name of the file. You can use ${filename} in the downstream
processoes .. in this case UpdateAttribute and/or PutFile ans manipulate it
to whatever you want. This can’t b used in ListS3 since that is the source
processor and it’s not aware of what ${filename} contains so it will be
replaced with empty string. Hence, it will actually look for
‘folder1/folder1a/ .txt‘.

Having said that , for your usecase, what you can do is: ListS3 (provide
the source *directory* alone in the configuration) -> RouteOnAttribute
(create a property with a name like ‘txtFilesOnly’ and its value as
${filename:endsWith(‘txt’)} -> to any downstream processor you want but for
the relationship, choose ‘txtFilesOnly’ while connecting to the downstream
processor.

How this work is, ListS3 just lists files present in the S3 buckets’
provided path as a separate flowfile. These flowfiles will have an
attribute on them called ‘filename’ and you are using that attribute in the
‘RouteOnAttribute’ processor, saying only if the filename ends with .txt,
route thos particular flowfiles alone to the downstream processors. BTW,
ListS3 doesn’t fetch the content along so you probably want to use FetchS3
after ‘RouteOnAttribute’ to actually read the needed files i.e. txt files
from S3 bucket.

On Sat, 24 Feb 2018 at 4:45 AM, Jim Murphy  wrote:

> Hey all,
>
> I'm a bit of a newbie to nifi . But I am hoping someone might have
> some advice for me here. I am using the ListS3 processor and the prefix
> field just doesn't seem to like anything but static filenames. I cannot
> reference directories or use the nifi expression language, where the docs
> say that I should be able to.
>
> I'd ideally like to reference files like this:
>
> /folder1/folder1a/${filename:endsWith('txt')}
>
> to find all files with a .txt extension 2 levels down in that specific
> path.
>
> Any advice?
>
> I totally know it's something stupid I am doing (or not doing). But
> research shows scant examples for this processor out there.
>
> Any help is appreciated greatly.
>
> Thanks,
>
> Jim
>

Advice on ListS3 Processor

[Jim Murphy]

Hey all,

I'm a bit of a newbie to nifi . But I am hoping someone might have
some advice for me here. I am using the ListS3 processor and the prefix
field just doesn't seem to like anything but static filenames. I cannot
reference directories or use the nifi expression language, where the docs
say that I should be able to.

I'd ideally like to reference files like this:

/folder1/folder1a/${filename:endsWith('txt')}

to find all files with a .txt extension 2 levels down in that specific path.

Any advice?

I totally know it's something stupid I am doing (or not doing). But
research shows scant examples for this processor out there.

Any help is appreciated greatly.

Thanks,

Jim

Re: Implementation of ListFile's Primary Node only in a cluster

[Sivaprasanna]

I have started working on an annotation implementation wherein the
developer can use that annotation to indicate that processor is supposed to
be set to run only on 'Primary node'. Framework side of things work just
fine. However, for UI side there are a couple of questions and issues:

   1. nf-processor-details.js

and nf-processor-configuration.js

checks if the setup 'isClustered' or 'executionNode === PRIMARY' which
confuses me. Checking ' nfClusterSummary.isClustered()' alone is enough,
right? The reason is, since we are also checking 'executionNode ===
Primary', even for single instance NiFi i.e. non clustered setup, the
'execution-node-options' will be rendered for processors marked with this
annotation.
   2. In order to avoid this, I made a change to the code and removed the
'executionNode === PRIMARY' condition check in the mentioned files. Even
after that, 'execution-node-options' is being rendered. Am I missing
something?

I have pushed these changes to my remote repo. Here is the link:
https://github.com/zenfenan/nifi/commit/e09e85960fb394eeef89d9cb6aa7acdfc5d4dad3

BTW, right now I have implemented it in this way : If the annotation is
present, at the time of processor creation/instantiation, the executionNode
will be set to 'PRIMARY'. However this can be changed later by configuring
the processor from the UI. Should we think about disabling the 'Execution
Node' configuration altogether (from UI) for a processor marked with this
annotation (which makes more sense to me but kinda seems to be restricting
the users' liberty from choosing according their wish) ?


On Sun, Feb 11, 2018 at 12:59 AM, Bryan Bende  wrote:

> Currently it means that the dataflow manager/developer is expected to
> set the 'Execution Nodes' strategy to "Primary Node" at the time of
> flow design.
>
> We don't have anything that restricts the scheduling strategy of a
> processor, but we probably should consider having an annotation like
> @PrimaryNodeOnly that you can put on a processor and then the
> framework will enforce that it can only be scheduled on primary node.
>
> In the case of ListFile, I think the statement in the documentation is
> only partially true...
>
> When "Input Directory Location" is set to local, there should be no
> issue with scheduling the processor on all nodes in the cluster, as it
> would be listing a local directory and storing state locally.
>
> When "Input Directory Location" is set to remote, it wouldn't make
> sense to have all nodes listing the same remote directory and getting
> the same results, and also the state is then stored in ZooKeeper under
> a ZNode using the processor's UUID, and the processor has the same
> UUID on each node so they would be overwriting each other's state in
> ZK.
>
> So ListFile probably can't be restricted to primary node only, where
> as something like ListHDFS probably could because it is always listing
> a remote destination.
>
>
> On Fri, Feb 9, 2018 at 10:55 PM, Sivaprasanna 
> wrote:
> > I was going through ListFile processor's code and found out that in the
> > documentation
> >  bundles/nifi-standard-bundle/nifi-standard-processors/src/
> main/java/org/apache/nifi/processors/standard/ListFile.java#L72-L76>,
> > it is mentioned that "this processor is designed to run on Primary Node
> > only in a cluster". I want to understand what "designed" stands for here.
> > Does that mean the processor was built in a way that it only runs on the
> > Primary node regardless of the "Execution Nodes" strategy set to
> otherwise
> > or does it mean that dataflow manager/developer is expected to set the
> > 'Execution Nodes' strategy to "Primary Node" at the time of flow design?
> If
> > it is of the former case, how is it handled in the code? If it is
> handled,
> > it should be in the framework side but I don't see any annotation
> > indicating anything related to such mechanism in the processor code and
> > more over a related JIRA NIFI-543
> >  is also open so I want
> > clear my doubt.
> >
> > -
> > Sivaprasanna
>

Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Fluid Design System

[Scott Aslan]

TonyK,

The intent is to use this this NgModule in NiFi Registry and eventually
NiFi. However, this would be released under ASLv2 so yes other projects
could use it.

On Thu, Feb 22, 2018 at 10:46 PM, Tony Kurc  wrote:

> Is some of the thinking that projects other than nifi projects would use
> this?
>
> On Feb 22, 2018 10:00 PM, "Scott Aslan"  wrote:
>
> > Sivaprasanna,
> >
> > I am not sure I follow exactly what you are saying...
> >
> > NiFi Registry would no longer continue to host a copy of the FDS
> NgModule.
> > Instead, NiFi Registry would just add the NiFi FDS sub-project as a
> client
> > side dependency in its package.json. This would be analogous to how NiFi
> > Registry depends on Angular Material, etc. npm supports the ability to
> > download published packages which are current with the latest stable
> > release of a package. npm *also* supports the ability to develop off
> > of the *master
> > branch (or any other branch really)* of the NiFi FDS. An example of this
> > can be found in the github.io demo here
> >  system/blob/gh-pages/package.
> > json#L45>
> > . By placing that dependency in the package.json for the NiFi Registry
> each
> > subsequent clean build would automatically download the latest master
> > branch of the NiFi FDS sub-project and developers can leverages the
> latest
> > NiFi FDS components.
> >
> > This also brings up a good point about release management. I have also
> > included a prototype of one possible implementation of automating the
> > tagging of a branch and automatically updating release version numbers
> etc.
> > leveraging grunt-bump [1]. The FDS-0.0.1-RC.0 tag [2] was created with
> the
> > described grunt task.
> >
> > [1]
> > https://github.com/scottyaslan/fluid-design-system/blob/master/Gruntfile
> .
> > js#L47
> >
> > [2] https://github.com/scottyaslan/fluid-design-
> system/tree/FDS-0.0.1-RC.0
> >
> > On Thu, Feb 22, 2018 at 12:39 PM, Sivaprasanna <
> sivaprasanna...@gmail.com>
> > wrote:
> >
> > > I agree with Matt. With clear documentation and guides, contributions
> on
> > > the sub-projects can be streamlined and be ensured that the necessary
> > > changes are already available on the core project i.e NiFi. One
> challenge
> > > is that the committer of the sub-project should have the courtesy to
> > check
> > > wether the supporting changes are made available to the core project
> and
> > > track its status but given how contributions are being handled in
> > > nifi-registry project, I don’t think it’s going to be that much of a
> > > headache.
> > >
> > > We could also add to the helper doc mentioning that if the contribution
> > is
> > > going to affect a core component, the contributor needs to add the JIRA
> > id
> > > of the core project’s supporting changes in the sub-projects’ issue
> > > description.
> > >
> > > On Thu, 22 Feb 2018 at 10:42 PM, Matt Gilman 
> > > wrote:
> > >
> > > > Joe, Joe,
> > > >
> > > > Regarding the release process... I think it could be similar to how
> > folks
> > > > verified and validated the NiFi Registry release. Guidance was given
> > in a
> > > > helper guide regarding how to obtain/build a branch or PR that
> > references
> > > > the new components. For the Registry release, there was a PR for NiFi
> > > that
> > > > had the supporting changes already available.
> > > >
> > > > We may have this issue any time we release new versions that depend
> on
> > > > another (sub)project.
> > > >
> > > > Matt
> > > >
> > > > On Thu, Feb 22, 2018 at 11:39 AM, Joe Percivall <
> jperciv...@apache.org
> > >
> > > > wrote:
> > > >
> > > > > Scott,
> > > > >
> > > > > Definitely like the direction of standardizing NiFi and Registry
> > around
> > > > the
> > > > > same set of components, so the user has a common UX. Is there
> > precedent
> > > > for
> > > > > creating a new sub-project just for common components/modules to be
> > > used
> > > > by
> > > > > the top-level, and it's other sub-projects? My concerns are similar
> > to
> > > > > Joe's. Along those lines, if the core problems we're trying to
> > address
> > > is
> > > > > the release process and distribution, is there a less
> "heavy-weight"
> > > > > avenue?
> > > > >
> > > > > In the past, we've also talked about pulling out the core NiFi
> > > framework
> > > > to
> > > > > be shared between NiFi and MiNiFi-Java for similar reasons. How we
> go
> > > > about
> > > > > solving this for the UI could be used a model for the core
> framework
> > as
> > > > > well.
> > > > >
> > > > > - Joe
> > > > >
> > > > > On Thu, Feb 22, 2018 at 10:58 AM, Mike Thomsen <
> > mikerthom...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Also, what sort of framework is the UI being standardized on?
> > > Angular?
> > > > > > React? Something else?
> > > > > >
> > > > > > On Thu, Feb 22, 2018 at 10:03 AM, Joe Witt 
> > > wrote:
> > > > 

Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Fluid Design System

[Kevin Doran]

Hey Scott,

I got a chance to take a look at your demo app. It's nice! I think your 
proposal is a good idea. A lot of effort went into creating this library for 
the use in NiFi Registry (awesome work on that!), and its pretty cool that 
something came out of it that can stand on its own to be shared across all NiFi 
UIs... and potentially other projects if they're interested.

Others have pointed out that every sub-project we add will create a bit of 
additional effort in order to be supported by the community. This is true, so 
these decisions should be thoughtfully considered. A like this particular case, 
because in addition to the project structure benefits, a standalone sub-project 
also has the potential to create a nice sub-community for our front-end 
contributors within the NiFi project, where they can explore new ideas, develop 
a roadmap for consistency across all NiFi UIs, and grow a group of front-end 
specialists [1].

Lastly, I think Scott makes a good point that automated testing good coverage 
and a RTC pattern of introducing code changes can and should reduce the burden 
on reviewers at release time [2]. I've noticed in Scott's example repo that 
even the process of versioning and packaging has been automated to work easily 
with NPM, so even the package structure should be reliably generated. There are 
still things that need to be verified manually in every release, such as the 
LICENSE/NOTICE (although hopefully a lot of that is getting verified in PR 
reviews), but if we focus on identifying in the Release Helper's Guide exactly 
what the manual review needs to include and how to do it (versus which things 
we are confident that our tests running on Travis CI have verified) we could 
streamline the validation process. With more subprojects, the amount to 
validate in each individual one should be smaller, too (ie, there are fewer 
front-end libraries to verify license compatibility in NiFi Registry because 
that will be done as part of Fluid Design System releases), and we can spread 
that burden out to a larger group over time.

[1] I don't want to be misunderstood - I'm not implying that our current 
front-end contributors aren't awesome (quite the opposite!), or that there 
aren't benefits to having "everyone work a bit on everything." I just think 
it's important to keep in mind that as a project's codebase grows, it can be 
hard for newcomers to find exactly where and how to contribute, and this would 
create a nice self-contained project for someone interested in exploring the UX 
code. If you want to work on everything in this codebase, you'll still be able 
to :)

[2] "Master is always releasable" is the philosophy of those who practice 
continuous deployment. I'm *not* suggesting we adopt CD, but there are valuable 
takeaways to learn from that approach, and it's possible to adopt some of them 
as a means of keeping the master branch extremely stable, even if we're only 
cutting releases periodically.


On 2/22/18, 13:43, "Scott Aslan"  wrote:

Joe,

Yes, extract out the FDS.

As for a release schedule, I don't think there would need to be one. We
would put out new releases as needed for new features or components. These
releases would be totally independent of NiFi or NiFi Registry. The
intention with this project is to follow semantic versioning and avoid
making breaking changes so using this library in NiFi or the NiFi Registry
would be as simple as updating the version number in the package.json and
rebuilding the application.

As for validation of releases I have a couple of ideas. I envisioned this
code base would follow a RTC paradigm and the initial release of this FDS
NgModule would include unit test coverage of all the existing
features/components/utils. Any new features/components/utils would require
adequate test coverage before being merged to NiFi FDS master. We could
also provide a demo application that users can build and deploy locally to
allow for human verification or even e2e testing...

I took the liberty of standing up a repo to give everyone a better idea of
what we are all talking about.
https://github.com/scottyaslan/fluid-design-system

Since no server or backend is required to run these UI/UX components I also
stood a demo of this as a github.io page here:
https://scottyaslan.github.io/fluid-design-system/


-Scotty

On Thu, Feb 22, 2018 at 10:03 AM, Joe Witt  wrote:

> Scott
>
> Ok so extract out the fluid design work you started with NiFi Registry
> to its own codebase which can be rev'd and published to NPM making it
> easier to consume/reuse across NiFi projects and offers better
> consistency.  This sounds interesting.
>
> In thinking through the additional community effort or the 

Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Fluid Design System

[Scott Aslan]

MikeT,

Good question. Right now NiFi has a mix of design systems from Angular
Material, multiple jquery plugins (some of them custom and all of them have
their own approach to styling), jquery UI, and custom D3 visualizations. In
order to add a new UX feature in NiFi a developer needs to know each and
everyone of these different yet available widgets and when and how to use
them. The slider widget we use in NiFi is the jquery UI
https://jqueryui.com/slider/. However, the processors on the canvas are
custom visualizations (svg) drawn with D3 so the jquery UI slider will not
work in this case.

There is currently no documentation around when to use which design system
nor any documentation around adding custom controls to a processor but this
could certainly be something we can try to add some documentation around to
provide guidance to UI/UX contributors.

On Fri, Feb 23, 2018 at 8:42 AM, Joe Witt  wrote:

> MikeT
>
> Not disagreeing with your point about the UX/sliders for the case you
> mention - that would be cool - but as a heads up you can do what you
> describe today with RouteOnAttribute and specifying EL statements to
> cover the flowfile size ranges - would work pretty nicely.
>
> Thanks
>
> On Fri, Feb 23, 2018 at 8:41 AM, Mike Thomsen 
> wrote:
> > Scott,
> >
> > Where does the UI/documentation stand on adding custom controls to
> > processors? That's a feature that I could see being really useful. An
> > example is a processor I've thought about writing called
> > RouteOnFlowfileSize that would let users define relationships based on
> > flowfile size ranges. Having sliders would be critical to making that
> > user-friendly.
> >
> > On Fri, Feb 23, 2018 at 8:28 AM, Scott Aslan 
> wrote:
> >
> >> Mike Thomsen,
> >>
> >> The Ui is being standardized on Angular. NiFi is currently running
> >> AngularJS v1.5.11 while NiFi Registry is running Angular v4.4.6.
> >>
> >> On Fri, Feb 23, 2018 at 8:23 AM, Scott Aslan 
> >> wrote:
> >>
> >> > Micheal M.,
> >> >
> >> > A *Design System* is a collection of utilities, components, and
> >> > guidelines which define the overall structure and experience of an
> >> > application(s). Fluid is the name of this design system.
> >> >
> >> >
> >> >
> >> > On Thu, Feb 22, 2018 at 1:44 PM, Michael Moser 
> >> wrote:
> >> >
> >> >>  There are compelling pros and easily identifiable cons to placing UI
> >> >> components into their own project.  I don't have anything to add
> there.
> >> >>
> >> >> Please, however, consider a different name.  "Fluid Design System" is
> >> >> generic to the point of giving no cognitive clue about what it
> actually
> >> >> is.  And without that clue, it's no different than a shorter made-up
> >> word.
> >> >> Also, a quick Google search doesn't indicate that it's an industry
> >> >> accepted
> >> >> phrase that conveys meaning.
> >> >>
> >> >> Consider:
> >> >>
> >> >> Fluidifi
> >> >> NiFi Fluid UI
> >> >> NiFi UI Components
> >> >> NiFi FDS
> >> >>
> >> >> Thanks,
> >> >> -- Mike
> >> >>
> >> >>
> >> >> On Thu, Feb 22, 2018 at 1:43 PM, Scott Aslan 
> >> >> wrote:
> >> >>
> >> >> > Joe,
> >> >> >
> >> >> > Yes, extract out the FDS.
> >> >> >
> >> >> > As for a release schedule, I don't think there would need to be
> one.
> >> We
> >> >> > would put out new releases as needed for new features or
> components.
> >> >> These
> >> >> > releases would be totally independent of NiFi or NiFi Registry. The
> >> >> > intention with this project is to follow semantic versioning and
> avoid
> >> >> > making breaking changes so using this library in NiFi or the NiFi
> >> >> Registry
> >> >> > would be as simple as updating the version number in the
> package.json
> >> >> and
> >> >> > rebuilding the application.
> >> >> >
> >> >> > As for validation of releases I have a couple of ideas. I
> envisioned
> >> >> this
> >> >> > code base would follow a RTC paradigm and the initial release of
> this
> >> >> FDS
> >> >> > NgModule would include unit test coverage of all the existing
> >> >> > features/components/utils. Any new features/components/utils would
> >> >> require
> >> >> > adequate test coverage before being merged to NiFi FDS master. We
> >> could
> >> >> > also provide a demo application that users can build and deploy
> >> locally
> >> >> to
> >> >> > allow for human verification or even e2e testing...
> >> >> >
> >> >> > I took the liberty of standing up a repo to give everyone a better
> >> idea
> >> >> of
> >> >> > what we are all talking about.
> >> >> > https://github.com/scottyaslan/fluid-design-system
> >> >> >
> >> >> > Since no server or backend is required to run these UI/UX
> components I
> >> >> also
> >> >> > stood a demo of this as a github.io page here:
> >> >> > https://scottyaslan.github.io/fluid-design-system/
> >> >> > 
> >> >> >
> >> >> > 

Re: PutSyslog IP Address

[Andre]

John,

In addition to what Andrew said, if by ESM you refer to McAfee ESM, then
you need to be mindful it expects the system to send data in a particular
format and the data source to be configured on what the call a Forwarder.

Cheers



On Fri, Feb 23, 2018 at 11:49 PM, John Smith  wrote:

> Was just wondering how you (or anyone else) managed to solve this problem?
> We're doing something similar in that we're using Nifi to collect all our
> syslogs (using GetSysLog) and processing and forwarding it on to our ESM
> (using PutSysLog). The IP address which shows up in our ESM is the IP
> address of our Nifi box sending the syslog packets which is not ideal to
> say
> the least! My current thought is to write a custom processor but it would
> be
> good if I didn't have to do this!
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>

Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Fluid Design System

[Joe Witt]

MikeT

Not disagreeing with your point about the UX/sliders for the case you
mention - that would be cool - but as a heads up you can do what you
describe today with RouteOnAttribute and specifying EL statements to
cover the flowfile size ranges - would work pretty nicely.

Thanks

On Fri, Feb 23, 2018 at 8:41 AM, Mike Thomsen  wrote:
> Scott,
>
> Where does the UI/documentation stand on adding custom controls to
> processors? That's a feature that I could see being really useful. An
> example is a processor I've thought about writing called
> RouteOnFlowfileSize that would let users define relationships based on
> flowfile size ranges. Having sliders would be critical to making that
> user-friendly.
>
> On Fri, Feb 23, 2018 at 8:28 AM, Scott Aslan  wrote:
>
>> Mike Thomsen,
>>
>> The Ui is being standardized on Angular. NiFi is currently running
>> AngularJS v1.5.11 while NiFi Registry is running Angular v4.4.6.
>>
>> On Fri, Feb 23, 2018 at 8:23 AM, Scott Aslan 
>> wrote:
>>
>> > Micheal M.,
>> >
>> > A *Design System* is a collection of utilities, components, and
>> > guidelines which define the overall structure and experience of an
>> > application(s). Fluid is the name of this design system.
>> >
>> >
>> >
>> > On Thu, Feb 22, 2018 at 1:44 PM, Michael Moser 
>> wrote:
>> >
>> >>  There are compelling pros and easily identifiable cons to placing UI
>> >> components into their own project.  I don't have anything to add there.
>> >>
>> >> Please, however, consider a different name.  "Fluid Design System" is
>> >> generic to the point of giving no cognitive clue about what it actually
>> >> is.  And without that clue, it's no different than a shorter made-up
>> word.
>> >> Also, a quick Google search doesn't indicate that it's an industry
>> >> accepted
>> >> phrase that conveys meaning.
>> >>
>> >> Consider:
>> >>
>> >> Fluidifi
>> >> NiFi Fluid UI
>> >> NiFi UI Components
>> >> NiFi FDS
>> >>
>> >> Thanks,
>> >> -- Mike
>> >>
>> >>
>> >> On Thu, Feb 22, 2018 at 1:43 PM, Scott Aslan 
>> >> wrote:
>> >>
>> >> > Joe,
>> >> >
>> >> > Yes, extract out the FDS.
>> >> >
>> >> > As for a release schedule, I don't think there would need to be one.
>> We
>> >> > would put out new releases as needed for new features or components.
>> >> These
>> >> > releases would be totally independent of NiFi or NiFi Registry. The
>> >> > intention with this project is to follow semantic versioning and avoid
>> >> > making breaking changes so using this library in NiFi or the NiFi
>> >> Registry
>> >> > would be as simple as updating the version number in the package.json
>> >> and
>> >> > rebuilding the application.
>> >> >
>> >> > As for validation of releases I have a couple of ideas. I envisioned
>> >> this
>> >> > code base would follow a RTC paradigm and the initial release of this
>> >> FDS
>> >> > NgModule would include unit test coverage of all the existing
>> >> > features/components/utils. Any new features/components/utils would
>> >> require
>> >> > adequate test coverage before being merged to NiFi FDS master. We
>> could
>> >> > also provide a demo application that users can build and deploy
>> locally
>> >> to
>> >> > allow for human verification or even e2e testing...
>> >> >
>> >> > I took the liberty of standing up a repo to give everyone a better
>> idea
>> >> of
>> >> > what we are all talking about.
>> >> > https://github.com/scottyaslan/fluid-design-system
>> >> >
>> >> > Since no server or backend is required to run these UI/UX components I
>> >> also
>> >> > stood a demo of this as a github.io page here:
>> >> > https://scottyaslan.github.io/fluid-design-system/
>> >> > 
>> >> >
>> >> > -Scotty
>> >> >
>> >> > On Thu, Feb 22, 2018 at 10:03 AM, Joe Witt 
>> wrote:
>> >> >
>> >> > > Scott
>> >> > >
>> >> > > Ok so extract out the fluid design work you started with NiFi
>> Registry
>> >> > > to its own codebase which can be rev'd and published to NPM making
>> it
>> >> > > easier to consume/reuse across NiFi projects and offers better
>> >> > > consistency.  This sounds interesting.
>> >> > >
>> >> > > In thinking through the additional community effort or the effort
>> >> > > trade-off:
>> >> > > How often do you anticipate we'd be doing releases (and thus
>> >> > > validation/voting) for this?
>> >> > > How often would those differ from when we'd want to do a NiFi or
>> NiFi
>> >> > > Registry release?
>> >> > > How do you envision the community would be able to help vet/validate
>> >> > > releases of these modules?
>> >> > >
>> >> > > Thanks
>> >> > > Joe
>> >> > >
>> >> > > On Thu, Feb 22, 2018 at 8:18 AM, Scott Aslan > >
>> >> > > wrote:
>> >> > > > NiFi Community,
>> >> > > >
>> >> > > > I'd like to initiate a discussion around creating a sub-project of
>> >> NiFi
>> >> > > to
>> >> > > > 

Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Fluid Design System

[Mike Thomsen]

Scott,

Where does the UI/documentation stand on adding custom controls to
processors? That's a feature that I could see being really useful. An
example is a processor I've thought about writing called
RouteOnFlowfileSize that would let users define relationships based on
flowfile size ranges. Having sliders would be critical to making that
user-friendly.

On Fri, Feb 23, 2018 at 8:28 AM, Scott Aslan  wrote:

> Mike Thomsen,
>
> The Ui is being standardized on Angular. NiFi is currently running
> AngularJS v1.5.11 while NiFi Registry is running Angular v4.4.6.
>
> On Fri, Feb 23, 2018 at 8:23 AM, Scott Aslan 
> wrote:
>
> > Micheal M.,
> >
> > A *Design System* is a collection of utilities, components, and
> > guidelines which define the overall structure and experience of an
> > application(s). Fluid is the name of this design system.
> >
> >
> >
> > On Thu, Feb 22, 2018 at 1:44 PM, Michael Moser 
> wrote:
> >
> >>  There are compelling pros and easily identifiable cons to placing UI
> >> components into their own project.  I don't have anything to add there.
> >>
> >> Please, however, consider a different name.  "Fluid Design System" is
> >> generic to the point of giving no cognitive clue about what it actually
> >> is.  And without that clue, it's no different than a shorter made-up
> word.
> >> Also, a quick Google search doesn't indicate that it's an industry
> >> accepted
> >> phrase that conveys meaning.
> >>
> >> Consider:
> >>
> >> Fluidifi
> >> NiFi Fluid UI
> >> NiFi UI Components
> >> NiFi FDS
> >>
> >> Thanks,
> >> -- Mike
> >>
> >>
> >> On Thu, Feb 22, 2018 at 1:43 PM, Scott Aslan 
> >> wrote:
> >>
> >> > Joe,
> >> >
> >> > Yes, extract out the FDS.
> >> >
> >> > As for a release schedule, I don't think there would need to be one.
> We
> >> > would put out new releases as needed for new features or components.
> >> These
> >> > releases would be totally independent of NiFi or NiFi Registry. The
> >> > intention with this project is to follow semantic versioning and avoid
> >> > making breaking changes so using this library in NiFi or the NiFi
> >> Registry
> >> > would be as simple as updating the version number in the package.json
> >> and
> >> > rebuilding the application.
> >> >
> >> > As for validation of releases I have a couple of ideas. I envisioned
> >> this
> >> > code base would follow a RTC paradigm and the initial release of this
> >> FDS
> >> > NgModule would include unit test coverage of all the existing
> >> > features/components/utils. Any new features/components/utils would
> >> require
> >> > adequate test coverage before being merged to NiFi FDS master. We
> could
> >> > also provide a demo application that users can build and deploy
> locally
> >> to
> >> > allow for human verification or even e2e testing...
> >> >
> >> > I took the liberty of standing up a repo to give everyone a better
> idea
> >> of
> >> > what we are all talking about.
> >> > https://github.com/scottyaslan/fluid-design-system
> >> >
> >> > Since no server or backend is required to run these UI/UX components I
> >> also
> >> > stood a demo of this as a github.io page here:
> >> > https://scottyaslan.github.io/fluid-design-system/
> >> > 
> >> >
> >> > -Scotty
> >> >
> >> > On Thu, Feb 22, 2018 at 10:03 AM, Joe Witt 
> wrote:
> >> >
> >> > > Scott
> >> > >
> >> > > Ok so extract out the fluid design work you started with NiFi
> Registry
> >> > > to its own codebase which can be rev'd and published to NPM making
> it
> >> > > easier to consume/reuse across NiFi projects and offers better
> >> > > consistency.  This sounds interesting.
> >> > >
> >> > > In thinking through the additional community effort or the effort
> >> > > trade-off:
> >> > > How often do you anticipate we'd be doing releases (and thus
> >> > > validation/voting) for this?
> >> > > How often would those differ from when we'd want to do a NiFi or
> NiFi
> >> > > Registry release?
> >> > > How do you envision the community would be able to help vet/validate
> >> > > releases of these modules?
> >> > >
> >> > > Thanks
> >> > > Joe
> >> > >
> >> > > On Thu, Feb 22, 2018 at 8:18 AM, Scott Aslan  >
> >> > > wrote:
> >> > > > NiFi Community,
> >> > > >
> >> > > > I'd like to initiate a discussion around creating a sub-project of
> >> NiFi
> >> > > to
> >> > > > encompass the Fluid Design System NgModule created during the
> >> > development
> >> > > > of the NiFi Registry. A possible name for this sub-project is
> simply
> >> > > > "NiFi Fluid
> >> > > > Design System". The idea would be to create a sub-project that
> >> > > distributes
> >> > > > an atomic set of high quality, reuse-able, theme-able, and
> testable
> >> > UI/UX
> >> > > > components, fonts, and other JS modules for use across the various
> >> web
> >> > > > applications throughout the NiFi 

Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Fluid Design System

[Scott Aslan]

Mike Thomsen,

The Ui is being standardized on Angular. NiFi is currently running
AngularJS v1.5.11 while NiFi Registry is running Angular v4.4.6.

On Fri, Feb 23, 2018 at 8:23 AM, Scott Aslan  wrote:

> Micheal M.,
>
> A *Design System* is a collection of utilities, components, and
> guidelines which define the overall structure and experience of an
> application(s). Fluid is the name of this design system.
>
>
>
> On Thu, Feb 22, 2018 at 1:44 PM, Michael Moser  wrote:
>
>>  There are compelling pros and easily identifiable cons to placing UI
>> components into their own project.  I don't have anything to add there.
>>
>> Please, however, consider a different name.  "Fluid Design System" is
>> generic to the point of giving no cognitive clue about what it actually
>> is.  And without that clue, it's no different than a shorter made-up word.
>> Also, a quick Google search doesn't indicate that it's an industry
>> accepted
>> phrase that conveys meaning.
>>
>> Consider:
>>
>> Fluidifi
>> NiFi Fluid UI
>> NiFi UI Components
>> NiFi FDS
>>
>> Thanks,
>> -- Mike
>>
>>
>> On Thu, Feb 22, 2018 at 1:43 PM, Scott Aslan 
>> wrote:
>>
>> > Joe,
>> >
>> > Yes, extract out the FDS.
>> >
>> > As for a release schedule, I don't think there would need to be one. We
>> > would put out new releases as needed for new features or components.
>> These
>> > releases would be totally independent of NiFi or NiFi Registry. The
>> > intention with this project is to follow semantic versioning and avoid
>> > making breaking changes so using this library in NiFi or the NiFi
>> Registry
>> > would be as simple as updating the version number in the package.json
>> and
>> > rebuilding the application.
>> >
>> > As for validation of releases I have a couple of ideas. I envisioned
>> this
>> > code base would follow a RTC paradigm and the initial release of this
>> FDS
>> > NgModule would include unit test coverage of all the existing
>> > features/components/utils. Any new features/components/utils would
>> require
>> > adequate test coverage before being merged to NiFi FDS master. We could
>> > also provide a demo application that users can build and deploy locally
>> to
>> > allow for human verification or even e2e testing...
>> >
>> > I took the liberty of standing up a repo to give everyone a better idea
>> of
>> > what we are all talking about.
>> > https://github.com/scottyaslan/fluid-design-system
>> >
>> > Since no server or backend is required to run these UI/UX components I
>> also
>> > stood a demo of this as a github.io page here:
>> > https://scottyaslan.github.io/fluid-design-system/
>> > 
>> >
>> > -Scotty
>> >
>> > On Thu, Feb 22, 2018 at 10:03 AM, Joe Witt  wrote:
>> >
>> > > Scott
>> > >
>> > > Ok so extract out the fluid design work you started with NiFi Registry
>> > > to its own codebase which can be rev'd and published to NPM making it
>> > > easier to consume/reuse across NiFi projects and offers better
>> > > consistency.  This sounds interesting.
>> > >
>> > > In thinking through the additional community effort or the effort
>> > > trade-off:
>> > > How often do you anticipate we'd be doing releases (and thus
>> > > validation/voting) for this?
>> > > How often would those differ from when we'd want to do a NiFi or NiFi
>> > > Registry release?
>> > > How do you envision the community would be able to help vet/validate
>> > > releases of these modules?
>> > >
>> > > Thanks
>> > > Joe
>> > >
>> > > On Thu, Feb 22, 2018 at 8:18 AM, Scott Aslan 
>> > > wrote:
>> > > > NiFi Community,
>> > > >
>> > > > I'd like to initiate a discussion around creating a sub-project of
>> NiFi
>> > > to
>> > > > encompass the Fluid Design System NgModule created during the
>> > development
>> > > > of the NiFi Registry. A possible name for this sub-project is simply
>> > > > "NiFi Fluid
>> > > > Design System". The idea would be to create a sub-project that
>> > > distributes
>> > > > an atomic set of high quality, reuse-able, theme-able, and testable
>> > UI/UX
>> > > > components, fonts, and other JS modules for use across the various
>> web
>> > > > applications throughout the NiFi universe (uNiFiverse???). Both NiFi
>> > and
>> > > > NiFi Registry web applications would eventually leverage this module
>> > via
>> > > > npm. This approach will enable us to provide our users with a
>> > consistent
>> > > > experience across web applications. Creating a sub-project would
>> also
>> > > allow
>> > > > the FDS code to evolve independently of NiFi/NiFi registry and be
>> > > released
>> > > > on it's own timeline. In addition, it would make tracking
>> issues/work
>> > > much
>> > > > clearer through a separate JIRA.
>> > > >
>> > > > Please discuss and provide and thoughts or feedback.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Scotty
>> > >
>> >
>>
>
>

Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Fluid Design System

[Scott Aslan]

Micheal M.,

A *Design System* is a collection of utilities, components, and guidelines
which define the overall structure and experience of an application(s).
Fluid is the name of this design system.



On Thu, Feb 22, 2018 at 1:44 PM, Michael Moser  wrote:

>  There are compelling pros and easily identifiable cons to placing UI
> components into their own project.  I don't have anything to add there.
>
> Please, however, consider a different name.  "Fluid Design System" is
> generic to the point of giving no cognitive clue about what it actually
> is.  And without that clue, it's no different than a shorter made-up word.
> Also, a quick Google search doesn't indicate that it's an industry accepted
> phrase that conveys meaning.
>
> Consider:
>
> Fluidifi
> NiFi Fluid UI
> NiFi UI Components
> NiFi FDS
>
> Thanks,
> -- Mike
>
>
> On Thu, Feb 22, 2018 at 1:43 PM, Scott Aslan 
> wrote:
>
> > Joe,
> >
> > Yes, extract out the FDS.
> >
> > As for a release schedule, I don't think there would need to be one. We
> > would put out new releases as needed for new features or components.
> These
> > releases would be totally independent of NiFi or NiFi Registry. The
> > intention with this project is to follow semantic versioning and avoid
> > making breaking changes so using this library in NiFi or the NiFi
> Registry
> > would be as simple as updating the version number in the package.json and
> > rebuilding the application.
> >
> > As for validation of releases I have a couple of ideas. I envisioned this
> > code base would follow a RTC paradigm and the initial release of this FDS
> > NgModule would include unit test coverage of all the existing
> > features/components/utils. Any new features/components/utils would
> require
> > adequate test coverage before being merged to NiFi FDS master. We could
> > also provide a demo application that users can build and deploy locally
> to
> > allow for human verification or even e2e testing...
> >
> > I took the liberty of standing up a repo to give everyone a better idea
> of
> > what we are all talking about.
> > https://github.com/scottyaslan/fluid-design-system
> >
> > Since no server or backend is required to run these UI/UX components I
> also
> > stood a demo of this as a github.io page here:
> > https://scottyaslan.github.io/fluid-design-system/
> > 
> >
> > -Scotty
> >
> > On Thu, Feb 22, 2018 at 10:03 AM, Joe Witt  wrote:
> >
> > > Scott
> > >
> > > Ok so extract out the fluid design work you started with NiFi Registry
> > > to its own codebase which can be rev'd and published to NPM making it
> > > easier to consume/reuse across NiFi projects and offers better
> > > consistency.  This sounds interesting.
> > >
> > > In thinking through the additional community effort or the effort
> > > trade-off:
> > > How often do you anticipate we'd be doing releases (and thus
> > > validation/voting) for this?
> > > How often would those differ from when we'd want to do a NiFi or NiFi
> > > Registry release?
> > > How do you envision the community would be able to help vet/validate
> > > releases of these modules?
> > >
> > > Thanks
> > > Joe
> > >
> > > On Thu, Feb 22, 2018 at 8:18 AM, Scott Aslan 
> > > wrote:
> > > > NiFi Community,
> > > >
> > > > I'd like to initiate a discussion around creating a sub-project of
> NiFi
> > > to
> > > > encompass the Fluid Design System NgModule created during the
> > development
> > > > of the NiFi Registry. A possible name for this sub-project is simply
> > > > "NiFi Fluid
> > > > Design System". The idea would be to create a sub-project that
> > > distributes
> > > > an atomic set of high quality, reuse-able, theme-able, and testable
> > UI/UX
> > > > components, fonts, and other JS modules for use across the various
> web
> > > > applications throughout the NiFi universe (uNiFiverse???). Both NiFi
> > and
> > > > NiFi Registry web applications would eventually leverage this module
> > via
> > > > npm. This approach will enable us to provide our users with a
> > consistent
> > > > experience across web applications. Creating a sub-project would also
> > > allow
> > > > the FDS code to evolve independently of NiFi/NiFi registry and be
> > > released
> > > > on it's own timeline. In addition, it would make tracking issues/work
> > > much
> > > > clearer through a separate JIRA.
> > > >
> > > > Please discuss and provide and thoughts or feedback.
> > > >
> > > > Thanks,
> > > >
> > > > Scotty
> > >
> >
>

Re: PutSyslog IP Address

[John Smith]

Was just wondering how you (or anyone else) managed to solve this problem?
We're doing something similar in that we're using Nifi to collect all our
syslogs (using GetSysLog) and processing and forwarding it on to our ESM
(using PutSysLog). The IP address which shows up in our ESM is the IP
address of our Nifi box sending the syslog packets which is not ideal to say
the least! My current thought is to write a custom processor but it would be
good if I didn't have to do this!



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/