Re: [ANNOUNCE] New Apache NiFi Committer Yolanda Davis

[Yolanda Davis]

Thank you Joe, Tony, and to everyone in this community! It has been a great
experience contributing to NiFi and I am so excited to be a part of this
project. I look forward to all the great work to come.

Thanks Again!

-yolanda

On Sat, Jul 30, 2016 at 8:22 AM, Joe Witt <joe.w...@gmail.com> wrote:

> Congrats and thanks Yolanda!
>
> On Jul 30, 2016 8:16 AM, "Tony Kurc" <tk...@apache.org> wrote:
>
> > On behalf of the Apache NiFI PMC, I am very pleased to announce that
> > Yolanda Davis has accepted the PMC's invitation to become a committer on
> > the Apache NiFi project. We greatly appreciate all of Yolanda's hard work
> > and generous contributions and look forward to continued involvement in
> the
> > project.
> >
> > In addition to contributing variable registry support and json
> conversion,
> > many of you may have heard one of her talks and experienced her
> enthusiasm
> > firsthand at a myriad of conferences and meetups.
> >
> >
> > Tony
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [DISCUSS] 1.0 Beta Release

[Yolanda Davis]

+1

On Wed, Aug 3, 2016 at 11:01 AM, Jeff  wrote:

> +1
>
> On Tue, Aug 2, 2016 at 9:17 AM Joe Skora  wrote:
>
> > +1
> >
> > On Mon, Aug 1, 2016 at 10:49 PM, Tony Kurc  wrote:
> >
> > > seems reasonable to me
> > >
> > > On Mon, Aug 1, 2016 at 10:14 PM, Joe Percivall <
> > > joeperciv...@yahoo.com.invalid> wrote:
> > >
> > > > Hey Team,
> > > >
> > > > The big ticket elements of 1.0 look to be in a pretty good state.
> Lots
> > of
> > > > things left to tighten up, primarily docs and bug fixes, but, it
> seems
> > > wise
> > > > for a major release like this to first kick out a Beta[1]. The
> relevant
> > > > section of the Apache.org link:
> > > >
> > > > Releases that are believed to be usable by testers and developers
> > > > outside the project, but perhaps not yet stable in terms of
> > features
> > > or
> > > > functionality, are usually referred to as "beta" or "unstable".
> > > >
> > > > This will enable us to will open up to the broader base of users that
> > > > normally don't build from source to help test and give us a couple
> > weeks
> > > of
> > > > feedback to work with before we do a formal 1.0 release vote.
> > > >
> > > > If no one has any objections, I will start an RC vote for it later
> this
> > > > week.
> > > >
> > > > [1] http://www.apache.org/dev/release.html#release-types
> > > >
> > > > Joe
> > > > - - - - - -
> > > > Joseph Percivall
> > > > linkedin.com/in/Percivall
> > > > e: joeperciv...@yahoo.com
> > > >
> > >
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [DISCUSS] MiNiFi C++ Initial Release - cpp-0.0.1

[Yolanda Davis]

+1 !

On Thu, Aug 4, 2016 at 11:42 PM, Aldrin Piri  wrote:

> Hey folks,
>
> There's been great efforts to get an initial working implementation of
> MiNiFi in C++.  This currently provides a great start on the core
> functionalities that the Java equivalent provides.  Perhaps the neatest of
> all these items is the C++ implementation of the Site to Site protocol
> allowing transmission from a native instance to the NiFi we have all come
> to use and love in a more bite-sized binary.
>
> To that end and to aid in making the MiNiFi effort a bit more concrete, I
> would like to get a vote started on 0.0.1 (JIRA cpp-0.0.1) in the next few
> days and would be happy to act as the RM. There are still a few issues to
> tackle mostly tailored around documentation
>
> The hope is that with a Java and C++ implementation in hand there will be a
> larger foundation on which people can get involved and participate.  This
> is especially crucial as we start expanding the framework, look to solidify
> APIs, and consider protocols to facilitate the design and implementation of
> centralized management and bi-directional communications.
>
> There is still much to be done both with our MiNiFi agent implementations
> and everything that follows.  This marks another nice milestone of progress
> and will further generate a multitude of avenues for people to get
> involved.
>
> Looking forward to the new possibilities and challenges ahead.
>
> --Aldrin
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi Committer Andre Fucs de Miranda

[Yolanda Davis]

Congratulations Andre!

Sent from my iPhone

> On Aug 6, 2016, at 10:40 PM, Tony Kurc  wrote:
> 
> On behalf of the Apache NiFI PMC, I am very pleased to announce that Andre
> Fucs de Miranda has accepted the PMC's invitation to become a committer on
> the Apache NiFi project. We greatly appreciate all of Andre's hard work and
> generous contributions and look forward to continued involvement in the
> project.
> 
> Many of you have already had the pleasure of interacting with him on the
> mailing lists, pull requests or jiras, and have benefited from his
> knowledge and contributions on log processing such as the Lumberjack bundle.
> 
> Welcome Andre!
> 
> Tony

Re: [ANNOUNCE] New Apache NiFi Committer Jeff Storck

[Yolanda Davis]

Congratulations Jeff!

On Tue, Feb 21, 2017 at 2:41 PM, Aldrin Piri  wrote:

> On behalf of the Apache NiFI PMC, I am very pleased to announce that Jeff
> Storck has accepted the PMC's invitation to become a committer on the
> Apache NiFi project. We greatly appreciate all of Jeff's hard work and
> generous contributions and look forward to continued involvement in the
> project.
>
> Jeff's contributions include significant efforts toward upgrade and
> migration processes inclusive of flow layout when upgrading from 0.x to 1.x
> and the ZooKeeper migration toolkit.
>
> Congrats, Jeff!
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Release Apache NiFi 1.1.2

[Yolanda Davis]

+1 (non-binding)

Confirmed hashes and signatures, ran full build and executed a simple flow.
all looked good.

On Thu, Feb 16, 2017 at 10:37 PM, Andy LoPresto 
wrote:

> Hello,
>
> I am pleased to be calling this vote for the source release of Apache NiFi
> nifi-1.1.2.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1098
>
> The Git tag is nifi-1.1.2-RC1
> The Git commit ID is 51fad01f5daf33716b8b5379c32ee932d91c8c63
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
> 51fad01f5daf33716b8b5379c32ee932d91c8c63
>
> Checksums of nifi-1.1.2-source-release.zip:
> MD5: 31d20a09955fac32d5b4147c497deede
> SHA1: 9f8f2ce00397d990dfb0890f9b1ede70dca4f25e
> SHA256: 0f57b5ae7f5e1d7f1289d779df39f20d70af0ffd92cb01b265beb90b309b747e
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/alopresto.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 2 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> version=12339600=12316020
>
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-Version1.1.2
>
> The vote will be open for 96 hours (over holiday weekend).
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test.  The please vote:
>
> [ ] +1 Release this package as nifi-1.1.2
> [ ] +0 no opinion
> [ ] -1 Do not release this package because because…
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Release Apache NiFi 0.7.2

[Yolanda Davis]

+1 (non-binding)

Confirmed hashes and signatures, ran build and executed a simple flow

On Thu, Feb 16, 2017 at 11:42 PM, Andy LoPresto 
wrote:

> Hello,
>
> I am pleased to be calling this vote for the source release of Apache NiFi
> nifi-0.7.2.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1099
>
> The Git tag is nifi-0.7.2-RC1
> The Git commit ID is 831ac6939df7d418cadb336eedb4e09fb97541a1
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
> 831ac6939df7d418cadb336eedb4e09fb97541a1
>
> Checksums of nifi-0.7.2-source-release.zip:
> MD5: efe9ea1cf0698a57a6829fe3829fc136
> SHA1: aee51164af34394c7017dae491b239a88b614865
> SHA256: 8cd02675003fca837ea0092b6225600a4700b905e5214a751ae7ff833263193b
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/alopresto.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 2 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> version=12339601=12316020
>
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-Version0.7.2
>
> The vote will be open for 96 hours (over holiday weekend).
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test.  The please vote:
>
> [ ] +1 Release this package as nifi-0.7.2
> [ ] +0 no opinion
> [ ] -1 Do not release this package because because…
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Establish Registry, a sub-project of Apache NiFi

[Yolanda Davis]

+1 (non-binding)

On Fri, Feb 10, 2017 at 11:40 AM, Bryan Bende  wrote:

> All,
>
> Following a solid discussion for the past few days [1] regarding the
> establishment of Registry as a sub-project of Apache NiFi, I'd like to
> call a formal vote to record this important community decision and
> establish consensus.
>
> The scope of this project is to define APIs for interacting with
> resources that one or more NiFi instances may be interested in, such
> as a flow registry for versioned flows, an extension registry for
> extensions, and possibly other configuration resources in the future.
> In addition, this project will provide reference implementations of
> these registries, with the goal of allowing the community to build a
> diverse set of implementations, such as a Git provider for versioned
> flows, or a bintray provider for an extension registry.
>
> I am a +1 and looking forward to the future work in this area.
>
> The vote will be open for 72 hours and be a majority rule vote.
>
> [ ] +1 Establish Registry, a subproject of Apache NiFi
> [ ]   0 Do not care
> [ ]  -1 Do not establish Registry, a subproject of Apache NiFi
>
> Thanks,
>
> Bryan
>
> [1] http://mail-archives.apache.org/mod_mbox/nifi-dev/201702.mbox/%3CCALo_
> M19euo2LLy0PVWmE70FzeLhQRcCtX6TC%3DqoiBVfn4zFQMA%40mail.gmail.com%3E
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi PMC Member - Matt Burgess

[Yolanda Davis]

Congratulations Matt B.!!

On Fri, Oct 7, 2016 at 10:28 AM, Tony Kurc  wrote:

> On behalf of the Apache NiFI PMC, I am very pleased to announce that Matt
> Burgess has accepted the PMC's invitation to join the Apache NiFi PMC. We
> greatly appreciate all of Matt's hard work and generous contributions to
> the project and all the positive interaction with the community. We look
> forward to his continued involvement in the project.
>
> Welcome and congratulations!
> Tony
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi Committer Rob Moran

[Yolanda Davis]

Congratulations Rob!

On Mon, Sep 19, 2016 at 3:36 PM, Rob Moran  wrote:

> Thanks everyone! I look forward to continuing the great progress we've
> made, and trying to make us look a little better while we do it ;)
>
> Rob
>
> On Mon, Sep 19, 2016 at 3:08 PM, Tony Kurc  wrote:
>
> > On behalf of the Apache NiFI PMC, I am very pleased to announce that Rob
> > Moran has accepted the PMC's invitation to become a committer on the
> Apache
> > NiFi project. We greatly appreciate all of Rob's hard work and generous
> > contributions to the project. We look forward to his continued
> involvement
> > in the project.
> >
> > Among other things, Rob has been responsible for a lot of the look and
> feel
> > of the NiFi user interface, to include the awesome logos!
> >
> > Welcome and congratulations!
> >
> > Tony
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: Apache NiFi 1.0.0 RC1 Release Helper Guide

[Yolanda Davis]

+1 (non-binding)

Ran full test suite, with rat and contrib-check.  Verified, readme, notice,
license, commit ID, hashes and checksums. Tested flows and templates in
standalone cluster (on Mac OS).

On Fri, Aug 26, 2016 at 12:26 PM, Joe Percivall <
joeperciv...@yahoo.com.invalid> wrote:

> Hello Apache NiFi community,
>
> Please find the associated guidance to help those interested in
> validating/verifying the release so they can vote.
>
> # Download latest KEYS file:
> https://dist.apache.org/repos/dist/dev/nifi/KEYS
>
> # Import keys file:
> gpg --import KEYS
>
> # [optional] Clear out local maven artifact repository
>
> # Pull down nifi-1.0.0 source release artifacts for review:
>
> wget https://dist.apache.org/repos/dist/dev/nifi/1.0.0/nifi-1.0.
> 0-source-release.zip
> wget https://dist.apache.org/repos/dist/dev/nifi/1.0.0/nifi-1.0.
> 0-source-release.zip.asc
> wget https://dist.apache.org/repos/dist/dev/nifi/1.0.0/nifi-1.0.
> 0-source-release.zip.md5
> wget https://dist.apache.org/repos/dist/dev/nifi/1.0.0/nifi-1.0.
> 0-source-release.zip.sha1
> wget https://dist.apache.org/repos/dist/dev/nifi/1.0.0/nifi-1.0.
> 0-source-release.zip.sha256
>
> # Verify the signature
> gpg --verify nifi-1.0.0-source-release.zip.asc
>
> # Verify the hashes (md5, sha1, sha256) match the source and what was
> provided in the vote email thread
> md5sum nifi-1.0.0-source-release.zip
> sha1sum nifi-1.0.0-source-release.zip
> sha256sum nifi-1.0.0-source-release.zip
>
> # Unzip nifi-1.0.0-source-release.zip
>
> # Verify the build works including release audit tool (RAT) checks
> cd nifi-1.0.0
> mvn clean install -Pcontrib-check
>
> # Verify the contents contain a good README, NOTICE, and LICENSE.
>
> # Verify the git commit ID is correct
>
> # Verify the RC was branched off the correct git commit ID
>
> # Look at the resulting convenience binary as found in nifi-assembly/target
>
> # Make sure the README, NOTICE, and LICENSE are present and correct
>
> # Run the resulting convenience binary and make sure it works as expected
>
> # Send a response to the vote thread indicating a +1, 0, -1 based on your
> findings.
>
> Thank you for your time and effort to validate the release!
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Release Apache NiFi 1.0.0 (RC1)

[Yolanda Davis]

Resending on appropriate thread:

+1 (non-binding)

Ran full test suite, with rat and contrib-check.  Verified, readme, notice,
license, commit ID, hashes and checksums. Tested flows and templates in
standalone cluster (on Mac OS).

On Fri, Aug 26, 2016 at 12:25 PM, Joe Percivall <
joeperciv...@yahoo.com.invalid> wrote:

> Hello Apache NiFi Community,
>
> I am pleased to be calling this vote for the source release of Apache NiFi,
> nifi-1.0.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1090/
>
> Note: There is a second binary being distributed now, the NiFi Toolkit. It
> can be used to facilitate securing a NiFi instance.
>
> The Git tag is nifi-1.0.0-RC1
> The Git commit hash is 74d5224783dfdc513f6b3ad5ed96671d3c581707
> * https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
> 74d5224783dfdc513f6b3ad5ed96671d3c581707
> * https://github.com/apache/nifi/commit/74d5224783dfdc513f6b3ad5ed9667
> 1d3c581707
>
> Checksums of nifi-1.0.0-source-release.zip:
> MD5: 8bdba49a73b94d036fad6c63b0ebe39d
> SHA1: 504c58f9b2fb305c41598a17f5b78f68f2b2fa3d
> SHA256: 22167ede5127683ca8de6dbd2fb9112cb1de650b7cfff7e640c905521447af92
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jpercivall
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 595 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12316020=12332640
>
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-Version1.0.0
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build from source, and test. Then
> please vote:
>
> [ ] +1 Release this package as nifi-1.0.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
> Thanks!
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi Committer Bryan Rosander

[Yolanda Davis]

Congrats Bryan!

On Wed, Oct 26, 2016 at 1:15 PM, Joe Witt  wrote:

> On behalf of the Apache NiFI PMC, I am very pleased to announce that
> Bryan Rosander has accepted the PMC's invitation to become a committer
> on the Apache NiFi project. We greatly appreciate all of Andre's hard
> work and generous contributions and look forward to continued
> involvement in the project.
>
> Bryan has contributed to a range of features and discussions such as
> the TLS Toolkit, Site to Site, JSON Conversion, Execute Script,
> Expression Language, MiNiFi, and event log parsing.
>
> Welcome Bryan!
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi Committer Scott Aslan

[Yolanda Davis]

Congrats Scott!!

Sent from my iPhone

> On Nov 4, 2016, at 11:25 AM, Tony Kurc  wrote:
> 
> On behalf of the Apache NiFI PMC, I am very pleased to announce that Scott
> Aslan has accepted the PMC's invitation to become a committer on the Apache
> NiFi project. We greatly appreciate all of Scott's hard work and generous
> contributions to the project. We look forward to his continued involvement
> in the project.
> 
> For those familiar with the UI improvements in 1.x, a lot of the
> implementation was contributed by Scott, so another set of very visible
> contributions! (Sorry for the pun, I couldn't help myself).
> 
> Welcome and congratulations!
> Tony

Re: [VOTE] Release Apache NiFi 1.1.0 (RC2)

[Yolanda Davis]

Aldrin,

I saw the same exception on that test on OS X but it resolved when I
upgraded my version of java (upgraded to 1.8.0_111-b14)


On Sun, Nov 27, 2016 at 5:13 PM, Aldrin Piri  wrote:

> +1, binding
>
> Hashes and signatures looked good.
> Contrib passed
> Builds and functionality were okay on OS X and Linux.
>
> I could not get a successful build on Windows 10 without removing some
> tests.  I ran into the item covered in
> https://issues.apache.org/jira/browse/NIFI-2692 from the 1.0 release as
> well as the following in TestInvokeJavascript.  This was with JDK
> 1.8.0u77.  Will create a ticket to capture but was not reproducible in any
> other environments inclusive of a Windows 7 VM.  Removing those tests
> produced a working build in Windows 10.
>
> [main] INFO org.apache.nifi.processors.script.InvokeScriptedProcessor -
> InvokeScriptedProcessor[id=11fe6a03-cb69-4b78-8b74-b16eeacc3710] logger is
> set! logger=org.apache.nifi.util.MockComponentLog@645aa696
> [main] ERROR org.apache.nifi.processors.script.InvokeScriptedProcessor -
> [InvokeScriptedProcessor[id=11fe6a03-cb69-4b78-8b74-b16eeacc3710]] Unable
> to validate the script Processor: java.lang.ClassCastException: Cannot cast
> jdk.nashorn.internal.objects.NativeArray to java.util.Collection
> java.lang.ClassCastException: Cannot cast
> jdk.nashorn.internal.objects.NativeArray
> to java.util.Collection
> at java.lang.invoke.MethodHandleImpl.newClassCastException(
> MethodHandleImpl.java:361)
> at java.lang.invoke.MethodHandleImpl.castReference(
> MethodHandleImpl.java:356)
> at org.apache.nifi.processor.Processor$$NashornJavaAdapter.
> validate(Unknown
> Source)
> at org.apache.nifi.processors.script.InvokeScriptedProcessor.
> customValidate(
> InvokeScriptedProcessor.java:464)
> at org.apache.nifi.components.AbstractConfigurableComponent.validate(
> AbstractConfigurableComponent.java:126)
> at org.apache.nifi.util.MockProcessContext.validate(
> MockProcessContext.java:226)
> at org.apache.nifi.util.MockProcessContext.assertValid(
> MockProcessContext.java:243)
> at org.apache.nifi.util.StandardProcessorTestRunner.assertValid(
> StandardProcessorTestRunner.java:334)
> at org.apache.nifi.processors.script.TestInvokeJavascript.
> testReadFlowFileContentAndStoreInFlowFileAttribute(
> TestInvokeJavascript.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> FrameworkMethod.java:50)
> at org.junit.internal.runners.model.ReflectiveCallable.run(
> ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(
> FrameworkMethod.java:47)
> at org.junit.internal.runners.statements.InvokeMethod.
> evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.
> evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:78)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.internal.runners.statements.RunBefores.
> evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(
> JUnit4IdeaTestRunner.java:78)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(
> JUnitStarter.java:212)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
>
> java.lang.AssertionError: Processor has 1 validation failures:
> 'Validation' validated against
> 'target/test/resources/javascript/test_reader.js'
> is invalid because An error occurred calling validate in the configured
> script Processor.
>
> at org.junit.Assert.fail(Assert.java:88)
> at org.apache.nifi.util.MockProcessContext.assertValid(
> MockProcessContext.java:251)
> at org.apache.nifi.util.StandardProcessorTestRunner.assertValid(
> 

Re: [VOTE] Release Apache NiFi 1.1.0 (RC2)

[Yolanda Davis]

+1 (non-binding)

Verified signatures, checksums, LICENSE and NOTICE
Build was successful  on OS X  (JDK 1.8.0_111-b14)
Tested basic flows in standalone and three node cluster (using GetTwitter,
JoltTransformJson, PutMongo)
Tested upgrade use case of moving from 1.0.0 to 1.1.0  (See Andre's email
from 11/25 @ 10:13 am for use case details)

Yolanda


On Sat, Nov 26, 2016 at 1:11 AM, Joe Witt  wrote:

> Hello Apache NiFi Community,
>
> I am pleased to be calling this vote for the source release of Apache NiFi,
> nifi-1.1.0.
>
> The source release zip and convenience binaries, including signatures
> and digests can be found at:
>   https://dist.apache.org/repos/dist/dev/nifi/1.0.0-rc2/
>
> The Git tag is nifi-1.1.0-RC2
> The Git commit hash is f61e42c65e1c2387591ee368c2148cd5bda646bd
> * https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
> f61e42c65e1c2387591ee368c2148cd5bda646bd
> * https://github.com/apache/nifi/commit/f61e42c65e1c2387591ee368c2148c
> d5bda646bd
>
> Checksums of nifi-1.1.0-source-release.zip:
> MD5: 371fb856d9c3947603239ea98f171f6f
> SHA1: 532c2e14e915dfa522254745bbc068aa6620babb
> SHA256: dd1d0569f209fd7f179b85a50fe4bf81b3d850c79b13d32cad88982a8234a719
>
> Release artifacts are signed with the following key:
>   https://people.apache.org/keys/committer/joewitt
>
> KEYS file available here:
>   https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 268 issues were closed/resolved for this release:
>   https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12316020=12337875
>
>
> Release note highlights can be found here:
>   https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-Version1.1.0
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build from source, and test. Then
> please vote:
>
> [ ] +1 Release this package as nifi-1.1.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
> Thanks!
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: nifi configuration file for process group

[Yolanda Davis]

Hello Bala,

If you are using a processor that supports expression language for the IP
address property you can configure NiFi to use a variable within a custom
property file that you define.  Once that is set you can refer to the
custom variable in the IP field instead of the raw IP value.  For more
information on how to set that up here is a link to the admin guide:

https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#custom_properties

I hope this is helpful.  Please let me know if you have any questions.

Thanks,

Yolanda

On Thu, Nov 17, 2016 at 7:21 AM, balacode63 
wrote:

> Hi all,
>
> My use case is,
>
> 1)Process group A is having 10 processors which uses same ip address
> ex(192.168.1.1) for some processing. ex http post
> 2) if this ip address is changed, i need to update all the processors.
> 3) is there any way i can handle this in a configuration file in nifi?
> ex: this configuration data will be used across different processors
>
> Please guide me
> Thanks,
> Bala
>
>
>
>
>
>
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/nifi-configuration-file-for-
> process-group-tp13908.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: nifi configuration file for process group

[Yolanda Davis]

Bala,

Let me add to the below in saying that in the current implementation of
custom properties, variables are bound at startup. So if the custom
property file is changed then nifi would need to be restarted. However
there are jiras and discussion around improving this in the future.

Yolanda

On Thu, Nov 17, 2016 at 10:42 AM, Yolanda Davis <yolanda.m.da...@gmail.com>
wrote:

> Hello Bala,
>
> If you are using a processor that supports expression language for the IP
> address property you can configure NiFi to use a variable within a custom
> property file that you define.  Once that is set you can refer to the
> custom variable in the IP field instead of the raw IP value.  For more
> information on how to set that up here is a link to the admin guide:
>
> https://nifi.apache.org/docs/nifi-docs/html/administration-
> guide.html#custom_properties
>
> I hope this is helpful.  Please let me know if you have any questions.
>
> Thanks,
>
> Yolanda
>
> On Thu, Nov 17, 2016 at 7:21 AM, balacode63 <balamurugan.c...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> My use case is,
>>
>> 1)Process group A is having 10 processors which uses same ip address
>> ex(192.168.1.1) for some processing. ex http post
>> 2) if this ip address is changed, i need to update all the processors.
>> 3) is there any way i can handle this in a configuration file in nifi?
>> ex: this configuration data will be used across different processors
>>
>> Please guide me
>> Thanks,
>> Bala
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-nifi-developer-l
>> ist.39713.n7.nabble.com/nifi-configuration-file-for-process-
>> group-tp13908.html
>> Sent from the Apache NiFi Developer List mailing list archive at
>> Nabble.com.
>>
>
>
>
> --
> --
> yolanda.m.da...@gmail.com
> @YolandaMDavis
>
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Release Apache NiFi 1.1.0 (RC1)

[Yolanda Davis]

Andre,

I attempted to reproduce your test case with a three node cluster
(unsecured in my case since I figured it may not be relevant). Upon initial
startup of nodes in 1.1.0, using the linked 1.0.0 conf directory, each node
was disconnected from the cluster until the vote determined which flow was
the correct one to use.  Once that completed all nodes connected to the
cluster and showed 3/3 connected as expected in the UI.

I can also attempt to do this with a secured cluster to see if that could
be a factor but would need to know what method of authentication was used
(TLS client cert, LDAP, etc)? Also I can send the actual terminal output
from my setup if that would aid in comparing environments.

Thanks,

Yolanda


On Fri, Nov 25, 2016 at 10:13 AM, Andre  wrote:

> Joe,
>
> -0 (non-binding)
>
> When testing "rolling upgrade" I noticed that as nodes restarted they were
> given new Node Ids (I suspect the NodeId is related to the version they
> run?). This results on a cluster with 50% of nodes showing up as
> disconnected.
>
>
> Not sure if this is particular to my test environment but would be great if
> someone try to reproduce:
>
>
>
> On a working 3 node secure cluster with embedded zookeeper and "easy
> upgrades directory structure":
>
> - Untar nifi snapshot
> - mv the output to under /path/to/nifi/
>
> you should now have
>
> /path/to/nifi/config
> /path/to/nifi/nifi-1.0.0
> /path/to/nifi/nifi-1.1.0-SNAPSHOT
>
> - ensure permissions of nifi-1.1.0-SNAPSHOT are correct (i.e. chown, chmod,
> etc)
> - cd nifi-1.1.0-SNAPSHOT
> - move original nifi config: mv conf conf.old
> - ln -s ../config conf
>
> - stop old nifi /path/to/nifi/nifi-1.0.0/bin/nifi.sh stop
>
> - edit /path/to/nifi/config so that nifi.version is set to 1.1.0-SNAPSHOT
>
> - start new nifi  /path/to/nifi/nifi-1.1.0-SNAPSHOT/bin/nifi.sh start
>
>
> NiFi should start working but 3/6 nodes will show as down, preventing
> modifications to the data flow.
>
>
>
> Cheers
>
>
>
> On Thu, Nov 24, 2016 at 6:04 PM, Joe Witt  wrote:
>
> > Hello Apache NiFi Community,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi,
> > nifi-1.1.0.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> >   https://repository.apache.org/content/repositories/orgapachenifi-1092
> >
> > The Git tag is nifi-1.1.0-RC1
> > The Git commit hash is 1b2b9f136df73a8b0d5d21419b6ebc2206f78393
> > * https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
> > 1b2b9f136df73a8b0d5d21419b6ebc2206f78393
> > * https://github.com/apache/nifi/commit/1b2b9f136df73a8b0d5d21419b6ebc
> > 2206f78393
> >
> > Checksums of nifi-1.1.0-source-release.zip:
> > MD5: b7a83412a088634835f6510974282408
> > SHA1: c538fedf990250dbd75090d7f03aeb2554802c8d
> > SHA256: 39dc521230c9936f9178b9fe34a410b932cf0d0807db4aeba9a1b41051023ec6
> >
> > Release artifacts are signed with the following key:
> >   https://people.apache.org/keys/committer/joewitt
> >
> > KEYS file available here:
> >   https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 265 issues were closed/resolved for this release:
> >   https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12316020=12337875
> >
> > Release note highlights can be found here:
> >   https://cwiki.apache.org/confluence/display/NIFI/
> > Release+Notes#ReleaseNotes-Version1.1.0
> >
> > The vote will be open for 88 hours (a bit longer than usual due to
> timing).
> >
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [ ] +1 Release this package as nifi-1.1.0
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi PMC Member - Joe Skora

[Yolanda Davis]

Congratulations Joe!

On Thu, Dec 29, 2016 at 1:02 PM, Aldrin Piri  wrote:

> All,
>
> On behalf of the Apache NiFi PMC, I am pleased to announce that Joe Skora has
> accepted the PMC's invitation to join the Apache NiFi PMC.  Joe has been
> with NiFi for quite some time, even before its arrival in the ASF and
> became a committer in February.  We are most pleased he brought his
> knowledge and supported the community once open sourced and has has
> provided continuous and excellent contributions in all facets of the
> community.  Of note, Joe was our first community member to carry out a
> release without being a PMC member for 0.7.1.
>
> Please join us in congratulating and welcoming Joe to the Apache NiFi PMC.
>
> Again, congratulations Joe and well deserved!
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi Committer Jeremy Dyer

[Yolanda Davis]

Congratulations Jeremy!

On Mon, Dec 19, 2016 at 8:23 PM, Aldrin Piri  wrote:

> On behalf of the Apache NiFI PMC, I am very pleased to announce that Jeremy
> Dyer has accepted the PMC's invitation to become a committer on the
> Apache NiFi project. We greatly appreciate all of Jeremy's hard work and
> generous contributions and look forward to continued involvement in the
> project.
>
> Jeremy’s contributions include creation a suite of the processors that
> aided in parsing HTML, build improvement and testing for MiNiFi, as well as
> many articles and presentations on using NiFi in new and novel ways.
>
> Welcome Jeremy!
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Release Apache NiFi 1.0.1 (RC1)

[Yolanda Davis]

+1 (non-binding)

Ran through release helper guide, verified signatures and hashes, then
built and ran simple flow on OS X (java 1.8 build 111).

-yolanda

On Fri, Dec 16, 2016 at 10:18 AM, Joe Percivall <
joeperciv...@yahoo.com.invalid> wrote:

> Hello Apache NiFi Community,
> I am pleased to be calling this vote for the source release of Apache
> NiFi,nifi-1.0.1.
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1096
> The Git tag is nifi-1.0.1-RC1The Git commit hash is
> 1890f6c522514027ae46f86601f4771f62cadc6d* https://git-wip-us.apache.org/
> repos/asf?p=nifi.git;a=commit;h=1890f6c522514027ae46f86601f4771f62cadc6d*
> https://github.com/apache/nifi/commit/1890f6c522514027ae46f86601f477
> 1f62cadc6d
> Checksums of nifi-1.0.1-source-release.zip:MD5:
> cc7fea9a22c0b48f87dd7152ab83c28cSHA1: 
> 88c35d5d3ff9d350473a742cdd8c38204628d343SHA256:
> d9d9628ced5bf3f0f3e0eae7729f4eb507120b072e883e287198fee80fbf9d15
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jpercivall
> KEYS file available here:https://dist.apache.org/
> repos/dist/release/nifi/KEYS
> 6 issues were closed/resolved for this release:https://issues.apache.
> org/jira/secure/ReleaseNote.jspa?projectId=12316020&
> version=12338865Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-Version1.0.1
> The vote will be open for 72 hours.Please download the release candidate
> and evaluate the necessary itemsincluding checking hashes, signatures,
> build from source, and test. Thenplease vote:
> [ ] +1 Release this package as nifi-1.0.1[ ] +0 no opinion[ ] -1 Do not
> release this package because...
> Thanks!
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Release Apache NiFi 1.1.1 (RC1)

[Yolanda Davis]

+1 (non-binding)

Verified signatures, hashes, README, NOTICE and LICENSE. Built and ran on
OS X (java 1.8 build 111).

-yolanda

On Mon, Dec 19, 2016 at 5:35 PM, Joe Percivall <
joeperciv...@yahoo.com.invalid> wrote:

> Hello Apache NiFi Community,
>
> I am pleased to be calling this vote for the source release of Apache
> NiFi, nifi-1.1.1.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1097
>
> The Git tag is nifi-1.1.1-RC1
> The Git commit hash is a92f2e36ed6be695e4dc6f624f6b3a96e6d1a57c
> * https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
> a92f2e36ed6be695e4dc6f624f6b3a96e6d1a57c
> * https://github.com/apache/nifi/commit/a92f2e36ed6be695e4dc6f624f6b3a
> 96e6d1a57c
>
> Checksums of nifi-1.1.1-source-release.zip:
> MD5: 74955060d8ee295d77a23607ac644a6e
> SHA1: 82efc0dc3141d0fad0205b33539e5928da87ad17
> SHA256: 25fab8d7abfecf4c0ccef1ed9cd5f0849c829c0741142ed4074bc8dd0781f7d0
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jpercivall
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 16 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12316020=12338797
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-Version1.1.1
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build from source, and test. Then
> please vote:
>
> [ ] +1 Release this package as nifi-1.1.1
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
> Thanks!




-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi Committer Bin Qiu

[Yolanda Davis]

Congrats Bin!

On Tue, Apr 4, 2017 at 4:17 PM, Tony Kurc  wrote:

> Apache NiFi community,
>
>
> On behalf of the Apache NiFI PMC, I am very pleased to announce that Bin
> Qiu has accepted the PMC's invitation to become a committer on the Apache
> NiFi project. We greatly appreciate all of Bin’s hard work and generous
> contributions and look forward to continued involvement in the project.
>
> Bin has been one of the driving forces behind MiNiFi - C++, contributing a
> massive amount in terms of code and helping with release verification and
> always being responsive to the community.
>
>
> Congrats, Bin!
>
>
> - Tony
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi PMC Member - Oleg Zhurakousky

[Yolanda Davis]

Congratulations Oleg!

On Tue, Apr 4, 2017 at 3:51 PM, Tony Kurc  wrote:

> Apache NiFi Community,
>
> On behalf of the Apache NiFi PMC, I am pleased to announce that Oleg
> Zhurakousky has accepted the PMC's invitation to join the Apache NiFi PMC.
> We greatly appreciate all of Oleg’s hard work and generous contributions to
> the project. We look forward to his continued involvement in the project.
>
> Oleg started out contributing in late 2015, diving in deep on framework
> issues and contributing exciting new capabilities. I had the pleasure of
> reviewing some of those early contributions and always found the dialog
> with Oleg on reviews and ideas to be fun and stimulating. Since accepting
> the PMC’s invitation to become a committer, he’s maintained a steady hand
> in reviewing new contributions, staying active contributing code and on the
> mailing lists, and becoming an ever more valuable member of our community.
>
>
> Please join us in congratulating and welcoming Oleg to the Apache NiFi PMC.
>
> Congratulations and welcome, Oleg!
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [DRAFT][REPORT] Apache NiFi - July 2017

[Yolanda Davis]

+1 LGTM

On Fri, Jul 7, 2017 at 8:10 AM, Marc <phroc...@apache.org> wrote:

> +1 Looks good!
>
> On Fri, Jul 7, 2017 at 7:43 AM, Andrew Psaltis <psaltis.and...@gmail.com>
> wrote:
>
> > +1
> >
> > On Thu, Jul 6, 2017 at 08:20 Pierre Villard <pierre.villard...@gmail.com
> >
> > wrote:
> >
> > > +1
> > >
> > > 2017-07-06 14:09 GMT+02:00 Joe Skora <jsk...@gmail.com>:
> > >
> > > > +1  Release this package as NiFi Board Report July 2017
> > > >
> > > > On Thu, Jul 6, 2017 at 7:46 AM, Tony Kurc <trk...@gmail.com> wrote:
> > > >
> > > > > I don't have any concerns, looks good!
> > > > >
> > > > > On Thu, Jul 6, 2017 at 2:01 AM, Aldrin Piri <aldrinp...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Looks good.  Thanks for compiling this, Joe.
> > > > > >
> > > > > > On Tue, Jul 4, 2017 at 9:50 AM, Joe Witt <joew...@apache.org>
> > wrote:
> > > > > >
> > > > > > > Team,
> > > > > > >
> > > > > > > It's that time again to submit our board report for Apache
> NiFi.
> > > > > > > Please see below draft.  If you have any suggestions/fixes,
> edits
> > > > > > > please advise.
> > > > > > >
> > > > > > > I'll submit the report in a few days.
> > > > > > >
> > > > > > > Thanks
> > > > > > > Joe
> > > > > > >
> > > > > > > +_+_+_+_+_+_+_
> > > > > > >
> > > > > > > ## Description:
> > > > > > >  - Apache NiFi is an easy to use, powerful, and reliable system
> > to
> > > > > > >process and distribute data.
> > > > > > >  - Apache NiFi MiNiFi is an edge data collection agent built to
> > > > > > seamlessly
> > > > > > >integrate with and leverage the command and control of NiFi.
> > > There
> > > > > are
> > > > > > >both Java and C++ implementations.
> > > > > > >  - Apache NiFi Registry is a centralized registry for key
> > > > configuration
> > > > > > > items
> > > > > > >including flow versions, assets, and extensions for Apache
> > NiFi
> > > > > > >and Apache MiNiFi.
> > > > > > >  - Apache NiFi Nar Maven Plugin is a release artifact used for
> > > > > supporting
> > > > > > >the NiFi classloader isolation model.
> > > > > > >
> > > > > > > ## Issues:
> > > > > > >  - There are no issues requiring board attention at this time.
> > > > > > >
> > > > > > > ## Activity:
> > > > > > >  - Based on board feedback in last report the descriptions
> above
> > > have
> > > > > > been
> > > > > > >simplified to avoid confusing child/subproject terminology.
> > We
> > > > have
> > > > > > >successfully conducted and completed name searches for the
> > > MiNiFi
> > > > > and
> > > > > > >Registry efforts.
> > > > > > >  - Conducted numerous releases as listed below.  The 0.7.x
> > releases
> > > > > > focused
> > > > > > >on providing simple bug fixes and resolving reported CVEs.
> > The
> > > > 1.x
> > > > > > >releases brought about stability, performance improvements,
> > and
> > > > > > > addressed
> > > > > > >reported CVEs.  Further, in the 1.x releases numerous
> > > substantive
> > > > > > > features
> > > > > > >were introduced including the ability to run multiple
> versions
> > > of
> > > > > the
> > > > > > > same
> > > > > > >components and support for both format and schema aware
> record
> > > > > > > processors
> > > > > > >to easily handle cases like acquisition, SQL based routing,
> > > > > > enrichment,
> > > > > > >conversion, and delivery of record oriented data.  These
> make
> > > many
> 

Re: File Filter regular expression - GetFile

[Yolanda Davis]

Hello Shankha,

Give the below a try:

ab\.[\w]*

That should match any files that start with "ab." (with any word character
in the extension).

Please let me know if that works for you.

Best Regards,

Yolanda


On Wed, Jun 28, 2017 at 5:55 AM, shankhamajumdar <
shankha.majum...@lexmark.com> wrote:

> Hi,
>
> Just a small modification, I am using ListFile processor. For example in
> the
> directory I have 5 files which are
> test.txt, ab.txt, ab1.txt. If I use [ab].* in the regular expression I am
> getting both the files i.e. ab.txt and ab1.txt. But I just want to get
> ab.txt not ab1.txt. Please let me know how to resolve this.
>
> Regards,
> Shankha
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/File-Filter-regular-expression-
> GetFile-tp16279p16280.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: Making NiFi upgrade smoother

[Yolanda Davis]

Hi Andre,

Realized I responded on the wrong thread...

Just wanted to add my thoughts on this.  One of the things I have been
thinking and working through in creating utilities to automate upgrades
(see https://issues.apache.org/jira/browse/NIFI-3663) is creating the
ability to migrate to new configurations/settings, especially in the
instance that  bootstrap.conf or a nifi.properties have been changed. Even
master bootstrap and nifi.properties may be subject to upgrade. I think a
large pain point is that users have needed to address those changes
manually, so automating the migration might help in making an upgrade
better.

Also when thinking about smoother upgrades, I think there are some
additional considerations that may need to be accounted for, such as
whether or not certain repos should stay in place or be relocated (e.g. if
a repository lives under the conf folder in the older installation, and not
external, should that be moved to the newer one?). Or whether an existing
flow template or state information (for zookeeper) needs to be migrated
into the new installation as well. And how to easily manage a need to
rollback if upgrade did not work for whatever reason (e.g. a flow is
impacted by upgraded libraries). My hope is that upgrade tool will have a
comprehensive approach for these things and also lead us down the road
towards supporting rolling upgrades in a cluster in the future.

-yolanda

On Tue, Apr 25, 2017 at 11:03 AM, Andre F de Miranda  wrote:

> dev,
>
> What do you think about the idea of allowing an user to configure overrides
> for the "master" configuration files in order to smooth upgrades?
>
> In my opinion candidates would be_ bootstrap.conf_ and  _nifi.properties_
>
> as they contain the links to all other files (logback.xml excluded).
>
> Upon starting bootstrap and nifi would load first the standard files as it
> does today, but once the load is done, the process would proceed to load
> the local overrides (reaching its final configuration).
>
> This way that whenever a user decides to upgrade, it would simply copy or
> link the override file to the NiFi folder and voilà!
>
> I realise this is not particularly a new idea, (YARN uses the "*-site.xml"
> to configure local properties and so does a number of other JAVA based
> applications I have dealt with) so I am keen to hear your thoughts
>
> Cheers
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: Making NiFi upgrade smoother

[Yolanda Davis]

Hi Andre,

Just wanted to add my thoughts on this.  One of the things I have been
thinking and working through in creating utilities to automate upgrades
(see https://issues.apache.org/jira/browse/NIFI-3663) is creating the
ability to migrate to new configurations/settings, especially in the
instance that  bootstrap.conf or a nifi.properties have been changed. Even
master bootstrap and nifi.properties may be subject to upgrade. I think a
large pain point is that users have needed to address those changes
manually, so automating the migration might help in making an upgrade
better.

Also when thinking about smoother upgrades, I think there are some
additional considerations that may need to be accounted for, such as
whether or not certain repos should stay in place or be relocated (e.g. if
a repository lives under the conf folder in the older installation, and not
external, should that be moved to the newer one?). Or whether an existing
flow template or state information (for zookeeper) needs to be migrated
into the new installation as well. And how to easily manage a need to
rollback if upgrade did not work for whatever reason (e.g. a flow is
impacted by upgraded libraries). My hope is that upgrade tool will have a
comprehensive approach for these things and also lead us down the road
towards supporting rolling upgrades in a cluster.

-yolanda





On Tue, Apr 25, 2017 at 11:06 AM, Andre  wrote:

> dev,
>
> What do you think about the idea of allowing an user to configure overrides
> for the "master" configuration files in order to smooth upgrades?
>
> In my opinion candidates would be_ bootstrap.conf_ and  _nifi.properties_
>
> as they contain the links to all other files (logback.xml excluded).
>
> Upon starting bootstrap and nifi would load first the standard files as it
> does today, but once the load is done, the process would proceed to load
> the local overrides (reaching its final configuration).
>
> This way that whenever a user decides to upgrade, it would simply copy or
> link the override file to the NiFi folder and voilà!
>
> I realise this is not particularly a new idea, (YARN uses the "*-site.xml"
> to configure local properties and so does a number of other JAVA based
> applications I have dealt with) so I am keen to hear your thoughts
>
> Cheers
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi PMC Member - Yolanda Davis

[Yolanda Davis]

Thank you everyone! It has been my pleasure to continue to contribute to
and support this awesome community. Definitely looking forward to all the
great things ahead!

On Wed, May 17, 2017 at 10:32 PM, Koji Kawamura <ijokaruma...@gmail.com>
wrote:

> Congratulations Yolanda, well deserved! I have a great respect for
> your excellent work.
>
> On Thu, May 18, 2017 at 11:15 AM, Andy LoPresto <alopre...@apache.org>
> wrote:
> > Congratulations Yolanda. Well deserved and very excited for your
> continued
> > excellent contributions to all aspects of the project.
> >
> > Andy LoPresto
> > alopre...@apache.org
> > alopresto.apa...@gmail.com
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > On May 17, 2017, at 7:14 PM, Joe Witt <joe.w...@gmail.com> wrote:
> >
> > Agree very much with Tony's excellent writeup.
> >
> > Congrats and thank you Yolanda!
> >
> > On Wed, May 17, 2017 at 10:11 PM, Tony Kurc <tk...@apache.org> wrote:
> >
> > On behalf of the Apache NiFi PMC, I am pleased to announce that Yolanda
> > Davis has accepted the PMC's invitation to join the Apache NiFi PMC.  We
> > greatly appreciate all of Yolanda's hard work and generous contributions
> to
> > the project. We look forward to continued involvement in the project.
> >
> > I know many people have interacted with Yolanda maybe at one of her
> > conference talks, or had the pleasure of working with her on reviews, or
> > used some of the features she's contributed and put a lot of work into
> > making them well tested and easy to use. If so, I think you'd agree it is
> > hard to interact with Yolanda and not get excited about not only NiFi but
> > the entire Apache way. If not, you're in for a treat when you get the
> > chance!
> >
> > Please join us in congratulating and welcoming Yolanda to the Apache NiFi
> > PMC.
> >
> > Congratulations and welcome, Yolanda!
> >
> > -- Tony
> >
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi PMC Member - Pierre Villard

[Yolanda Davis]

Congrats Pierre!

On Mon, May 22, 2017 at 3:17 PM, Tony Kurc  wrote:

> On behalf of the Apache NiFi PMC, I am pleased to announce that Pierre
> Villard has accepted the PMC's invitation to join the Apache NiFi PMC.  We
> greatly appreciate all of Pierre's hard work and generous contributions to
> the project. We look forward to continued involvement in the project.
>
> Pierre has been a long time code contributor to both NiFi and MiNiFi. In
> addition to that, he's spurred the community through his blogging -
> teaching people how to use, operate, and monitor NiFi. He's a prolific
> reviewer of contributions, and active in the important task of verifying
> releases.
>
> Please join us in congratulating and welcoming Pierre to the Apache NiFi
> PMC.
>
> Congratulations and welcome, Pierre!
>
> -- Tony
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi PMC Member - Koji Kawamura

[Yolanda Davis]

Congratulations Koji!

Sent from my iPhone

> On May 26, 2017, at 10:58 PM, Andy LoPresto  wrote:
> 
> Congratulations Koji. For anyone not familiar with some of his greatest hits, 
> it’s not just the (impressive) code contributions like the Wait & Notify 
> processors and massive Site-to-site refactoring, but his detailed blog posts 
> explaining complex topics very clearly. Well-deserved. 
> 
> 
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On May 26, 2017, at 7:18 PM, Andre  wrote:
>> 
>> Congratulations Koji
>> 
>> Cheers
>> 
>> 
>> 
>> 
>>> On Sat, May 27, 2017 at 12:09 PM, Tony Kurc  wrote:
>>> 
>>> On behalf of the Apache NiFi PMC, I am pleased to announce that Koji
>>> Kawamura has accepted the PMC's invitation to join the Apache NiFi PMC.  We
>>> greatly appreciate all of Koji's hard work and generous contributions to
>>> the project. We look forward to continued involvement in the project.
>>> 
>>> Koji has been contributing code NiFi since 2015, taking point on important
>>> parts of the project like improving site-to-site. Beyond code, Koji is
>>> active in building the community through blogging on technical topics,
>>> generating interest on social media (@apachenifi #NiFi) like twitter, and
>>> giving talks at conferences. Koji has been keeping the project strong by
>>> being active in reviewing new contributions and verifying releases.
>>> 
>>> Please join us in congratulating and welcoming Koji to the Apache NiFi PMC.
>>> 
>>> Congratulations and welcome, Koji!
>>> 
>>> -- Tony
>>> 
> 

Re: [VOTE] Release Apache NiFi MiNiFi 0.2.0

[Yolanda Davis]

+1 (non-binding)

Verified signatures and hashes
Reviewed README, NOTICE, LICENSE
Ran Build with contrib-check
Converted and ran simple flow, no issues

On Mon, May 15, 2017 at 12:55 PM, Aldrin Piri  wrote:

> Hello,
>
> I am pleased to be calling this vote for the source release of Apache NiFi
> MiNiFi, minifi-0.2.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1107
>
> The Git tag is minifi-0.2.0-RC1
> The Git commit ID is 1813e87ccddffcc9ab75d25948c31901faf2f1ab
> * https://git-wip-us.apache.org/repos/asf?p=nifi-minifi.git;a=commit;h=
> 1813e87ccddffcc9ab75d25948c31901faf2f1ab
> * https://github.com/apache/nifi-minifi/commit/
> 1813e87ccddffcc9ab75d25948c31901faf2f1ab
>
> Checksums of minifi-0.2.0-source-release.zip:
> MD5:  e0f9bbd7afeb9547126a710b3e30face
> SHA1:  bcba77ccd4254de3275ecb2a732005e6b404706d
> SHA256:  c182d71ba298a41a336da139f435f10a253997d677fc4ae8820b261ea8c3a4c2
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/aldrin.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 26 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> version=12338791=12319921
>
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/MINIFI/
> Release+Notes#ReleaseNotes-Version0.2.0
>
> The vote will be open for at least 72 hours and close at 5PM EDT, 18 May
> 2017 [1].
>
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test.  Then please vote:
>
> [ ] +1 Release this package as minifi-0.2.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
> Thanks!
>
> [1] You can determine this time for your local time zone at
> https://s.apache.org/minifi-0.2.0-rc1-close
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi Committer Drew Lim

[Yolanda Davis]

Congratulations Drew!


> On May 31, 2017, at 4:20 PM, Bryan Rosander  wrote:
> 
> Congrats Drew!
> 
> On Wed, May 31, 2017 at 3:51 PM, Andrew Psaltis 
> wrote:
> 
>> Congrats Drew! I spend a lot of time teaching people about NiFi and can
>> honestly say the documentation -- built-in and on the wiki is a great
>> selling point. Thanks for all the hard work.
>> 
>>> On Wed, May 31, 2017 at 3:08 PM, Tony Kurc  wrote:
>>> 
>>> On behalf of the Apache NiFI PMC, I am very pleased to announce that Drew
>>> Lim has accepted the PMC's invitation to become a committer on the Apache
>>> NiFi project. We greatly appreciate all of Drew's hard work and generous
>>> contributions to the project. We look forward to his continued
>> involvement
>>> in the project.
>>> 
>>> In my opinion, one of the strengths of NiFi is that someone can launch
>> the
>>> UI, do some light (often built-in) reading, and get started "making data
>>> flow" in a short amount of time. Drew has been instrumental in making
>> this
>>> happen, over the past year contributing to the wiki and making the
>> built-in
>>> documentation correct and approachable. We all know the Apache adage
>>> "Community > Code", and Drew's contributions are key to ensuring the
>> health
>>> of the community.
>>> 
>>> Welcome and congratulations!
>>> 
>>> --Tony
>>> 
>> 
>> 
>> 
>> --
>> Thanks,
>> Andrew
>> 
>> Subscribe to my book: Streaming Data 
>> 
>> twiiter: @itmdata 
>> 

Re: [VOTE] Release Apache NiFi 0.7.4

[Yolanda Davis]

+1 (binding)

-Ran through release helper
-Full build with contrib check on OSX
-Ran simple flow, worked as expected

On Tue, Jun 6, 2017 at 1:59 PM, Scott Aslan  wrote:

> +1 (non-binding)
>
> - Verified signatures, checksums, LICENSE, NOTICE, and README
> - Built using a full clean build with contrib check successfully on OSX
> - Run example data flows on OS X with Chrome Version 58.0.3029.110
> (64-bit)
>  on unsecured instance
> - Run example data flows on OS X with Chrome Version 58.0.3029.110
> (64-bit) on
> secured instance
>
> On Tue, Jun 6, 2017 at 1:17 PM, Pierre Villard <
> pierre.villard...@gmail.com>
> wrote:
>
> > +1 (binding)
> > Release this package as nifi-0.7.4
> >
> > - Ran through release helper, all looks good (build on OSX)
> > - Basic workflows running as expected
> >
> >
> > 2017-06-06 17:44 GMT+02:00 Bryan Bende :
> >
> > > +1 (binding) Release this package as nifi-0.7.4
> > >
> > > - Ran through release helper and everything checked out
> > > - Ran basic flow to test that binary works
> > >
> > > Minor note - The years in the NOTICE files still say 2014-2016.
> > >
> > >
> > > On Tue, Jun 6, 2017 at 11:30 AM, Joe Skora  wrote:
> > > > +1 (binding)
> > > >
> > > > * Verified GPG signature.
> > > > * Verified hashes.
> > > > * Built source with contrib-check profile.
> > > > * Verified README, NOTICE, and LICENSE.
> > > > * Verified commit ID.
> > > > * Verified RC was branched from "support/nifi-0.7.x".
> > > > * Ran binary and tested basic functionality.
> > > >
> > > > On Mon, Jun 5, 2017 at 9:29 PM, Koji Kawamura <
> ijokaruma...@gmail.com>
> > > > wrote:
> > > >
> > > >> +1 (binding)
> > > >>
> > > >> Verified gpg signature.
> > > >> Verified all hashes on source zipfile.
> > > >> Performed mvn clean install -Pcontrib-checks
> > > >> Builds cleanly, all tests pass on Mac OS Sierra (10.12.4)
> > > >> Oracle JDK 1.8.0_121-b13, maven 3.2.5
> > > >> Ran simple flow and worked as expected.
> > > >>
> > > >> Thanks,
> > > >> Koji
> > > >>
> > > >> On Tue, Jun 6, 2017 at 7:31 AM, Adam Taft 
> wrote:
> > > >> > +1 (binding)
> > > >> >
> > > >> > Verified gpg signature.
> > > >> > Verified all hashes on source zipfile.
> > > >> > Performed mvn clean install -Pcontrib-checks
> > > >> > Builds cleanly, all tests pass in docker container centos:latest
> w/
> > > >> > openjdk-1.8.0 and maven 3.5.0
> > > >> > LICENSE, NOTICE, README look good.
> > > >> > Binary runs as expected with a simple dataflow.
> > > >> >
> > > >> > Cheers,
> > > >> >
> > > >> > Adam
> > > >> >
> > > >>
> > >
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi Committer Marc Parisi

[Yolanda Davis]

Congrats Marc!

On Thu, Jun 1, 2017 at 9:36 AM, Bryan Rosander  wrote:

> Congrats Marc!  Thanks for all the C++ help so far :)
>
> On Thu, Jun 1, 2017 at 8:48 AM, Scott Aslan  wrote:
>
> > Congrats Marc!
> >
> > On Thu, Jun 1, 2017 at 8:39 AM, Marc  wrote:
> >
> > > Thank you, everyone! Amazing to be part of such a cohesive and
> > flourishing
> > > community.
> > >
> > > I look forward to contributing and working with all of you!
> > >
> > > On Thu, Jun 1, 2017 at 8:35 AM, Mark Payne 
> wrote:
> > >
> > > > Welcome, Marc! Happy to have you aboard!
> > > >
> > > > > On Jun 1, 2017, at 5:20 AM, Pierre Villard <
> > > pierre.villard...@gmail.com>
> > > > wrote:
> > > > >
> > > > > Congrats Marc!
> > > > >
> > > > > 2017-06-01 6:14 GMT+02:00 Andrew Lim :
> > > > >
> > > > >> Congrats Marc!
> > > > >>
> > > > >>
> > > > >>> On May 31, 2017, at 11:45 PM, Joe Percivall <
> jperciv...@apache.org
> > >
> > > > >> wrote:
> > > > >>>
> > > > >>> Congrats Marc, excited to see what else you can create for
> MiNiFi!
> > > > >>>
> > > > >>> On Wed, May 31, 2017 at 11:00 PM, Joe Witt 
> > > wrote:
> > > > >>>
> > > >  Congrats and welcome.  Kevin said it very well so I'll simply +1
> > > that.
> > > > 
> > > >  On Wed, May 31, 2017 at 10:45 PM, Kevin Doran <
> > > > kdoran.apa...@gmail.com>
> > > >  wrote:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >   Congratulations, Marc! Very well deserved. Thanks
> > for
> > > >  your contributions and generosity with sharing your technical
> > > > knowledge.
> > > > >
> > > > >
> > > > >
> > > > >   _
> > > > > From: Jeremy Dyer 
> > > > > Sent: Wednesday, May 31, 2017 10:23 PM
> > > > > Subject: Re: [ANNOUNCE] New Apache NiFi Committer Marc Parisi
> > > > > To:  
> > > > >
> > > > >
> > > > > Congratulations Marc! Have really enjoyed working and learning
> > from
> > > > you
> > > >  on the C++ side of the house. Look forward to your further work!
> > > > >
> > > > > Sent from my iPhone
> > > > >
> > > > >> On May 31, 2017, at 9:37 PM, Tony Kurc 
> > wrote:
> > > > >>
> > > > >> On behalf of the Apache NiFI PMC, I am very pleased to
> announce
> > > that
> > > >  Marc
> > > > >> Parisi has accepted the PMC's invitation to become a committer
> > on
> > > > the
> > > > >> Apache NiFi project. We greatly appreciate all of Marc's hard
> > work
> > > > and
> > > > >> generous contributions to the project. We look forward to his
> > > > >> continued
> > > > >> involvement in the project.
> > > > >>
> > > > >> Marc has been contributing heavily to MiNiFi C++ for the past
> > > > several
> > > > >> months, really bringing energy to that piece of the project.
> > He's
> > > > been
> > > > >> making heavy code contributions, and as many of you have
> > > discovered,
> > > >  active
> > > > >> and helpful on the mailing lists and is a willing reviewer.
> Marc
> > > > also
> > > > >> brings with him experience with another Apache project,
> > Accumulo,
> > > > >> where
> > > >  he
> > > > >> is a committer and PMC member.
> > > > >>
> > > > >> Welcome and congratulations!
> > > > >>
> > > > >> Tony
> > > > >
> > > > >
> > > > >
> > > > >
> > > > 
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> --
> > > > >>> *Joe Percivall*
> > > > >>> linkedin.com/in/Percivall
> > > > >>> e: jperciv...@apache.com
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > *Scott Aslan = new WebDeveloper(*
> > *{"location": {"city": "Saint Cloud","state": "FL",
> > "zip": "34771"},"contact": {"email":
> > "scottyas...@gmail.com ","linkedin":
> > "http://www.linkedin.com/in/scottyaslan
> > "}});*
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [DRAFT][REPORT] Apache NiFi Board Report Oct 2017

[Yolanda Davis]

LGTM as well with suggested update.

-yolanda

On Mon, Oct 9, 2017 at 11:03 AM, Rob Moran  wrote:

> Joe, looks good with the couple changes Drew mentioned--thanks.
>
> On Mon, Oct 9, 2017 at 10:30 AM Andrew Lim 
> wrote:
>
> > Noticed the following:
> >
> > -In Health Report section: "seeing a very strong pipeline to both both
> > committer and PMC” (remove extra both)
> > -"Apache NiFi 1.4.0 was released Oct 1 2017” (Change? Release Notes says
> > Oct 2)
> >
> > Otherwise, looks great!
> >
> > -Drew
> >
> > > On Oct 9, 2017, at 10:23 AM, Marc  wrote:
> > >
> > > Joe
> > >
> > >  I think this looks great.
> > >
> > >  Thanks for putting this together,
> > >
> > >
> > >
> > > On Mon, Oct 9, 2017 at 10:12 AM, Joe Witt  wrote:
> > >
> > >> Team,
> > >>
> > >> Please see draft board report.  Am looking for feedback/edits quickly
> > >> as this needs to be submitted by Wed.  I'll probably send it in
> > >> tonight though as I might not have time over the next couple of days.
> > >>
> > >> Thanks
> > >> Joe
> > >>
> > >> **==**==**==**==**
> > >>
> > >> ## Description:
> > >> - Apache NiFi is an easy to use, powerful, and reliable system to
> > >>   process and distribute data.
> > >> - Apache NiFi MiNiFi is an edge data collection agent built to
> > seamlessly
> > >>   integrate with and leverage the command and control of NiFi. There
> are
> > >>   both Java and C++ implementations.
> > >> - Apache NiFi Registry is a centralized registry for key configuration
> > >> items
> > >>   including flow versions, assets, and extensions for Apache NiFi
> > >>   and Apache MiNiFi.
> > >> - Apache NiFi Nar Maven Plugin is a release artifact used for
> supporting
> > >>   the NiFi classloader isolation model.
> > >>
> > >> ## Issues:
> > >> - There are no issues requiring board attention at this time.
> > >>
> > >> ## Activity:
> > >> - Released Apache NiFi 1.4.0 which is another very heavy feature,
> > >> stability,
> > >>   and bug fix release.  It includes improvements on the web UI coming
> > >> from new
> > >>   contributors which is a great sign.
> > >> - The Apache NiFi downloads have been quite large and growing for some
> > >> time.
> > >>   The groundwork to solve this is the Apache NiFi Registry effort
> which
> > is
> > >>   coming along nicely and which the foundational elements of NiFi to
> > >> integrate
> > >>   with it are present.  We're probably looking at two more releases
> for
> > >> that
> > >>   to land and for which we can drop the NiFi build size down.
> > >> - We continue to hover around the 100 outstanding pull requests
> number.
> > >> What
> > >>   is encouraging is we've seen an uptick in contributions both in code
> > and
> > >>   review feedback from new community members.
> > >> - The Apache NiFi MiNiFi C++ agent has rapidly matured over the past
> > >> couple of
> > >>   months and is serving as a new source for contributions and
> > >> contributors.
> > >>
> > >> ## Health report:
> > >> - As stated in our past report, health of the community is strong and
> > >>   indicators of strength continue trending in the right direction.
> > Mailing
> > >>   list and JIRA activity is strong. ASF Hipchat is serving as an
> on-ramp
> > >> for
> > >>   new users to our mailing list and JIRA systems. We continue to see
> new
> > >>   users and contributors.
> > >> - The PMC and committer ranks did not grow during the reporting
> period.
> > >> This
> > >>   is a reflection of the PMC not focusing enough on the topic though
> as
> > >> we're
> > >>   seeing a very strong pipeline to both both committer and PMC. Expect
> > >> both
> > >>   new committers and PMC members during our next reporting period.
> > >>
> > >> ## PMC changes:
> > >>
> > >> - Currently 22 PMC members.
> > >> - Last PMC member added Fri May 26 2017.
> > >>
> > >> ## Committer base changes:
> > >>
> > >> - Currently 34 committers.
> > >> - Last committer added Thu Jun 01 2017.
> > >>
> > >> ## Releases:
> > >>
> > >> - Apache NiFi 1.4.0 was released Oct 1 2017.
> > >>
> > >> ## Mailing list activity:
> > >>
> > >> - Activity on the mailing lists remains extremely high with a mixture
> > >>   of new users, contributors, and deeper more experienced users and
> > >>   contributors sparking discussion and questions and filing bugs or
> > >>   new features.
> > >>
> > >> - us...@nifi.apache.org:
> > >>- 553 subscribers (up 34 in the last 3 months):
> > >>- 748 emails sent to list (862 in previous quarter)
> > >>
> > >> - dev@nifi.apache.org:
> > >>- 383 subscribers (up 12 in the last 3 months):
> > >>- 604 emails sent to list (947 in previous quarter)
> > >>
> > >> - iss...@nifi.apache.org:
> > >>- 46 subscribers (up 4 in the last 3 months):
> > >>- 5775 emails sent to list (7457 in previous quarter)
> > >>
> > >>
> > >> ## JIRA activity:
> > >>
> > >> - 335 JIRA tickets created in the last 3 months
> > >> - 263 JIRA tickets 

Re: [ANNOUNCE] New NiFi PMC member Scott Aslan

[Yolanda Davis]

Congrats Scott!!

Sent from my iPhone

> On Dec 2, 2017, at 8:43 PM, Marc  wrote:
> 
> Congrats Scott!
> 
>> On Dec 2, 2017 8:14 PM, "Kevin Doran"  wrote:
>> 
>> Congrats, Scott! Keep crushing it!
>> 
>> Get Outlook for iOS
>> 
>> From: Joe Witt 
>> Sent: Saturday, December 2, 2017 7:27:35 PM
>> To: dev@nifi.apache.org
>> Subject: Re: [ANNOUNCE] New NiFi PMC member Scott Aslan
>> 
>> congrats scott and thanks for all the awesome ui and community work youve
>> done!
>> 
>>> On Dec 2, 2017 2:48 PM, "Matt Burgess"  wrote:
>>> 
>>> Congratulations Scotty! Glad to have you aboard!
>>> 
>>> 
 On Dec 2, 2017, at 2:46 PM, Tony Kurc  wrote:
 
 NiFi community,
 On behalf of the Apache NiFi PMC, I am pleased to announce that Scott
>>> Aslan
 has accepted the PMC's invitation to join the Apache NiFi PMC.  We
>>> greatly
 appreciate all of Scotts's hard work and generous contributions to the
 project. We look forward to continued involvement in the project.
 
 Scott has been delivering as a strong committer for quite a long time,
>> is
 making huge progress on nifi-registry, and has had wide reach to the
 community in many ways, to include our mailing lists.
 
 Congratulations and welcome, Scott!
 
 Tony
>>> 
>> 

Re: [ANNOUNCE] New NiFi PMC member Jeff Storck

[Yolanda Davis]

Congrats Jeff!

On Wed, Dec 6, 2017 at 5:12 PM, Jeff  wrote:

> Thanks, all!  It's great to be a part of this amazing community.
>
> On Tue, Dec 5, 2017 at 12:27 PM Matt Burgess  wrote:
>
> > Congrats Jeff!  Welcome aboard!
> >
> > On Tue, Dec 5, 2017 at 11:49 AM, Tony Kurc  wrote:
> > > NiFi community,
> > > On behalf of the Apache NiFi PMC, I am pleased to announce that Jeff
> > Storck
> > > has accepted the PMC's invitation to join the Apache NiFi PMC.  We
> > greatly
> > > appreciate all of Jeff's hard work and generous contributions to the
> > > project. We look forward to continued involvement in the project.
> > >
> > > For those of you enjoying NiFi 1.4.0 release, Jeff was the release
> > manager,
> > > which is a huge complicated set of work. He's also been a long time
> > > contributor, committer, and a strong member of the community.
> > >
> > > Congratulations and welcome, Jeff!
> > >
> > > Tony
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: SSL and Zookeeper

[Yolanda Davis]

Hi Mark,

Just adding to what Andy said below. Currently there is support for
Kerberos/SASL authentication with Zookeeper.  The admin guide provides
details on that configuration and setup for embedded zookeeper and links
for guidance on external Zookeeper installations.

https://nifi.apache.org/docs/nifi-docs/html/administration-
guide.html#securing_zookeeper

-yolanda

On Tue, Oct 31, 2017 at 3:07 PM, Andy LoPresto  wrote:

> Hi Mark,
>
> I believe SSL ZK connections are only supported in 3.5.0+ [1] and
> currently NiFi uses ZK 3.4.6 [2]. I don’t know the details on making a TLS
> connection to ZK, but my first thoughts would be to update the host/port
> combination in your NiFi configs to reference the ZK HTTPS port, and ensure
> that the certificate(s) used to identify ZK are in your NiFi truststore.
>
> This warrants raising a Jira to request the feature. Thanks.
>
> [1] http://zookeeper-user.578899.n2.nabble.com/SSL-between-
> java-client-and-zookeeper-td7582421.html
> [2] https://github.com/apache/nifi/blob/master/pom.xml#L748
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Oct 31, 2017, at 11:37 AM, Mark Bean  wrote:
>
> Has anyone setup zookeeper connections for a NiFi Cluster over SSL? We have
> ZK itself running over SSL. How do we get the NiFi to ZK connections
> secure? Is this possible? Advice, suggestions and/or documentation greatly
> appreciated.
>
> Thanks,
> Mark
>
>
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Release Apache NiFi Registry 0.1.0

[Yolanda Davis]

+1, binding

Have been looking forward to this contribution; it is a great addition to
the community!

Following the release helper I verified signatures,  hashes as well as
the README,
NOTICE, and LICENSE. Successfully built the project per instructions and
built the NiFi project using the referenced PR.  From there I tested in
unsecured mode the basic test suggested in the helper guide.  Also tried
minor things such as deleting a flow from registry that was associated to a
PG to ensure NiFi updated as expected.

One thing that would be helpful is additional documentation on the NiFi
side on how to integrate with Registry. I also noticed on my test to delete
a flow where it seemed there was a pretty big lag when NiFi recognized the
change, even beyond my attempt to refresh.  Not sure if this is
configurable or just odd behavior on my end.

Another minor issue is when someone adds a process group, enters a name for
it and then selects import to use a flow in registry.  The name gets
overwritten by the flow (which makes sense) but wondering if there's a way
to either allow user to keep new name or at least let them know it will be
overwritten by version in registry?

Again great job!

-yolanda



On Thu, Dec 28, 2017 at 1:09 PM, Bryan Bende  wrote:

> Hello,
>
> I am pleased to be calling this vote for the source release of Apache
> NiFi Registry 0.1.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1115/
>
> The Git tag is nifi-registry-0.1.0-RC1
> The Git commit ID is 81b99e7b04491eabb72ddf30754053ca12d0fcca
> https://git-wip-us.apache.org/repos/asf?p=nifi-registry.git;a=commit;h=
> 81b99e7b04491eabb72ddf30754053ca12d0fcca
>
> Checksums of nifi-registry-0.1.0-source-release.zip:
> MD5: 56244c3c296cdc9c3fcc6d22590b80d1
> SHA1: 6354e91f868f40d6656ec2467bde307260ad63ca
> SHA256: 2c680e441e6c4bfa2381bf004e9b19a6a79401a6a83e04597d0a714a95efd301
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/bbende.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 65 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12320920=12340217
>
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-NiFiRegistry0.1.0
>
> The vote will be open for 96 hours.
>
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build from source, and test.
>
> The please vote:
>
> [ ] +1 Release this package as nifi-registry-0.1.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New NiFi PMC member Jeremy Dyer

[Yolanda Davis]

Congratulations Jeremy!

On Wed, Aug 1, 2018 at 7:23 AM, Rob Moran  wrote:

> Congratulations, Jeremy!
>
> On Tue, Jul 31, 2018 at 1:17 PM Sivaprasanna 
> wrote:
>
> > Congratulations, Jeremy. Well deserved.
> >
> > On Tue, 31 Jul 2018 at 9:44 PM, Scott Aslan 
> wrote:
> >
> > > Congrats Jeremy!
> > >
> > > On Tue, Jul 31, 2018 at 11:02 AM Michael Moser 
> > wrote:
> > >
> > > >  Congrats, welcome, and thank you for your work.
> > > >
> > > >
> > > > On Tue, Jul 31, 2018 at 9:21 AM Otto Fowler  >
> > > > wrote:
> > > >
> > > > > Congratulations!
> > > > >
> > > > >
> > > > > On July 31, 2018 at 08:36:48, Tony Kurc (tk...@apache.org) wrote:
> > > > >
> > > > > All,
> > > > >
> > > > > On behalf of the Apache NiFi PMC, I am pleased to announce that
> > Jeremy
> > > > Dyer
> > > > > has accepted the PMC's invitation to join the Apache NiFi PMC.
> > > > >
> > > > > Jeremy has been a long-time contributor to the project - across
> many
> > > > > different parts of the project to include NiFi and and MiNiFi,
> > > > contributing
> > > > > code, reviews, release verification, and help on the mailing lists.
> > > He's
> > > > > performed the challenging, detail oriented work of acting as a
> > release
> > > > > manager for both the Java and C++ versions of MiNiFi (0.5.0 of
> each).
> > > > >
> > > > > Congratulations Jeremy and well deserved!
> > > > >
> > > > > Tony
> > > > >
> > > >
> > >
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New NiFi PMC member Kevin Doran

[Yolanda Davis]

Congratulations Kevin!

On Wed, Aug 1, 2018 at 7:23 AM, Rob Moran  wrote:

> Congratulations, Kevin!
>
> On Tue, Jul 31, 2018 at 1:16 PM Sivaprasanna 
> wrote:
>
> > Congratulations, Kevin.
> >
> > On Tue, 31 Jul 2018 at 9:43 PM, Scott Aslan 
> wrote:
> >
> > > Congrats Kevin!
> > >
> > > On Tue, Jul 31, 2018 at 11:01 AM Michael Moser 
> > wrote:
> > >
> > > > Congrats, welcome, and thank you for your work.
> > > >
> > > >
> > > > On Tue, Jul 31, 2018 at 9:21 AM Otto Fowler  >
> > > > wrote:
> > > >
> > > > > Congratulations!
> > > > >
> > > > >
> > > > > On July 31, 2018 at 08:26:34, Tony Kurc (tk...@apache.org) wrote:
> > > > >
> > > > > NiFi Community,
> > > > >
> > > > > On behalf of the Apache NiFi PMC, I am pleased to announce that
> Kevin
> > > > Doran
> > > > > has accepted the PMC's invitation to join the Apache NiFi PMC.
> > > > >
> > > > > In addition to being a regular code contributor to the project for
> > > quite
> > > > > some time, Kevin has been hard to miss on the mailing lists,
> > especially
> > > > on
> > > > > NiFi Registry threads. We all appreciate his hard work getting
> > releases
> > > > > "out the door", helping verify releases and recently doing release
> > > > manager
> > > > > duty for the NiFi Registry 0.2.0.
> > > > >
> > > > > Please join us in congratulating and welcoming Kevin to the Apache
> > NiFi
> > > > > PMC.
> > > > >
> > > > > Tony
> > > > >
> > > >
> > >
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Establish Fluid Design System, a sub-project of Apache NiFi

[Yolanda Davis]

+1

On Fri, Mar 9, 2018 at 3:22 PM, Pierre Villard 
wrote:

> +1
>
> Le ven. 9 mars 2018 à 21:16, Bryan Bende  a écrit :
>
> > +1
> >
> > On Fri, Mar 9, 2018 at 3:11 PM, Joe Witt  wrote:
> > > +1
> > >
> > > On Mar 9, 2018 3:10 PM, "Scott Aslan"  wrote:
> > >
> > > All,
> > >
> > > Following a solid discussion for the past couple of weeks [1] regarding
> > the
> > > establishment of Fluid Design System as a sub-project of Apache NiFi,
> I'd
> > > like to
> > > call a formal vote to record this important community decision and
> > > establish consensus.
> > >
> > > The scope of this project is to define a theme-able set of high quality
> > UI
> > > components and utilities for use across the various Apache NiFi web
> > > applications in order to provide a more consistent user experience.
> > >
> > > I am a +1 and looking forward to the future work in this area.
> > >
> > > The vote will be open for 72 hours and be a majority rule vote.
> > >
> > > [ ] +1 Establish Fluid Design System, a subproject of Apache NiFi
> > > [ ]   0 Do not care
> > > [ ]  -1 Do not establish Fluid Design System, a subproject of Apache
> NiFi
> > >
> > > Thanks,
> > >
> > > ScottyA
> > >
> > > [1] *http://mail-archives.apache.org/mod_mbox/nifi-dev/201802.mbox/%
> > > 3CCAKeSr4ibXX9xzGN1GhdVv5uTmWvfB3QULXF9orzw4FYD0n7taQ%40mail.gmail.com
> > %3E
> > >  > > 3CCAKeSr4ibXX9xzGN1GhdVv5uTmWvfB3QULXF9orzw4FYD0n7taQ%40mail.gmail.com
> > %3E>*
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [DISCUSS] Predictive Analytics for NiFi Metrics

[Yolanda Davis]

Hi Craig,

Thanks for your feedback and insight on your use cases.  What version of
MiNiFi are you running?  Concerning performing edge ML this may be possible
for you with MiNiFi C++ version 0.6.0.  That release supports the creation
of python processors which can be added to your flow to execute models.
Andy Christianson sent me info from a blog written by Marc Parisi on this
topic here: https://www.parisi.io/index.php/2019/03/27/hey-bro/

In creating an analytics framework for models we may look to simplify
things further where instead of creating a processor for models you could
perhaps just implement a simple interface and rely on the engine to execute
things as needed.  But for now perhaps the python processsor could help
fill the gap for you?

-yolanda


On Wed, Jul 31, 2019 at 6:01 AM Craig Knell  wrote:

> Sounds. Great
>
> Let me know if you need some help
>
> Best regards
>
> Craig
>
>
>
> > On 31 Jul 2019, at 17:31, Arpad Boda  wrote:
> >
> > Craig,
> >
> > OPC ( https://issues.apache.org/jira/browse/MINIFICPP-819 ) and Modbus (
> > https://issues.apache.org/jira/browse/MINIFICPP-897 ) are on the way for
> > MiNiFi c++, hopefully both will be part of next release (0.7.0).
> > It's gonna be legen... wait for it! :)
> >
> > Regards,
> > Arpad
> >
> >> On Wed, Jul 31, 2019 at 2:30 AM Craig Knell 
> wrote:
> >>
> >> Hi Folks
> >>
> >> That's our use case now.  All our Models are run in python.
> >> Currently we send events to the ML via http, although this is not
> optimal
> >>
> >> Our use case is edge ML where we want a light weight wrapper for
> >> Python code base.
> >> Jython however does not work with the code base
> >> I'm think of changing the interface to some thing like REDIS for pub/sub
> >> Id also like this to be a push deployment via minifi
> >>
> >> Also support for sensors via protocols via Modbus and OPC would be great
> >>
> >> Craig
> >>
> >>> On Wed, Jul 31, 2019 at 1:43 AM Joe Witt  wrote:
> >>>
> >>> Definitely something that I think would really help the community.  It
> >>> might make sense to frame/structure these APIs such that an internal
> >> option
> >>> could be available to reduce dependencies and get up and running but
> that
> >>> also just as easily a remote implementation where the engine lives and
> is
> >>> managed externally could also be supported.
> >>>
> >>> Thanks
> >>>
> >>>
> >>> On Tue, Jul 30, 2019 at 1:40 PM Andy LoPresto 
> >> wrote:
> >>>
> >>>> Yolanda,
> >>>>
> >>>> I think this sounds like a great idea and will be very useful to
> >>>> admins/users, as well as enabling some interesting next-level
> >> functionality
> >>>> and insight generation. Thanks for putting this out there.
> >>>>
> >>>> Andy LoPresto
> >>>> alopre...@apache.org
> >>>> alopresto.apa...@gmail.com
> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>>
> >>>>> On Jul 30, 2019, at 5:55 AM, Yolanda Davis <
> >> yolanda.m.da...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Hello Everyone,
> >>>>>
> >>>>> I wanted to reach out to the community to discuss potentially
> >> enhancing
> >>>>> NiFi to include predictive analytics that can help users assess and
> >>>> predict
> >>>>> NiFi behavior and performance. Currently NiFi has lots of metrics
> >>>> available
> >>>>> for areas including jvm and flow component usage (via component
> >> status)
> >>>> as
> >>>>> well as provenance data which NiFi makes available either through
> >> the UI
> >>>> or
> >>>>> reporting tasks (for consumption by other systems). Past discussions
> >> in
> >>>> the
> >>>>> community cite users shipping this data to applications such as
> >>>> Prometheus,
> >>>>> ELK stacks, or Ambari metrics for further analysis in order to
> >>>>> capture/review performance issues, detect anomalies, and send alerts
> >> or
> >>>>> notifications.  These systems are efficient in capturing and helping
> >> to
> >>>>> analyze these metrics however it requires custo

Re: Re:[EXT] [DISCUSS] Predictive Analytics for NiFi Metrics

[Yolanda Davis]

Hi Mark and Rob

Mark thanks so much for the info on your work and Rob thanks for jumping in
on the UI! I just wanted to add, Mark, that looking at your branch I think
we also may have some opportunities to exchange notes or collaborate on the
backend as well.  The work in the feature branch is still in progress (with
some decoupling to ensure we can allow flexible configuration of models).
Please feel free to review and leave comments under the parent JIRA.  At
the same time I'll take a deeper dive on your branch and perhaps we can
exchange notes on potential areas for improvement/collaboration if it makes
sense?

Thanks Again,

-yolanda


On Mon, Aug 19, 2019 at 3:34 PM Robert Fellows 
wrote:

> Hey Mark,
>   I've started working on some UI based on the initial commit for this
> proposal. What you have done and what I am working on have a bit of
> overlap, but not much.
> I'm working on getting the predicted count and bytes into the existing
> connection metric display that is already on the canvas. The only overlap
> looks like it might be in the
> Summary table. I plan on adding a PR for my additions hopefully tomorrow.
> Maybe once it is up we can discuss how we bring the them together where it
> makes sense?
>
> This is the main JIRA case:
> https://issues.apache.org/jira/browse/NIFI-6510
> And this is the subtask that I am working toward:
> https://issues.apache.org/jira/browse/NIFI-6568
>
>
> -- Rob Fellows
>
> On Mon, Aug 19, 2019 at 2:26 PM Owens, Mark  wrote:
>
> > The images from the preview email do not appear to be displaying. They
> can
> > be viewed at:
> > https://github.com/jmark99/nifi-images
> >
> > From: Owens, Mark 
> > Sent: Monday, August 19, 2019 2:25 PM
> > To: dev@nifi.apache.org
> > Subject: RE: Re:[EXT] [DISCUSS] Predictive Analytics for NiFi Metrics
> >
> >
> > Hi Yolanda,
> >
> >
> >
> > I've been working on a feature that appears to possibly overlap with the
> > work you are pursuing. Perhaps we should see if/should we try to
> coordinate
> > our efforts. I've been updating NiFi to predict the time to queue
> overflow
> > for both flowfiles and bytes and displaying that information in the GUI.
> > For the initial attempt, I’ve been using a simple model of straight line
> > prediction over a sliding window of 15 minutes to predict when flows will
> > fail. This estimate is then displayed on both the NiFi Summary page under
> > the connections tab and in the status history graphs.  Below are examples
> > of what would be displayed to the user.
> >
> >
> >
> > [cid:image001.png@01D55696.E4CCD550]
> >
> >
> >
> > The Connection tab contains a new column on the right that displays the
> > prediction for both flow files and data size. The user can select a
> maximum
> > time at which specific times are no longer displayed. In this example, if
> > the prediction lies beyond 12 hours then the display simply indicates
> that
> > the flow is greater than 12 hours away from failure at the moment.
> >
> >
> >
> > [cid:image002.png@01D55697.2C8AC500]
> >
> >
> >
> > This display graphs the prediction for byte overflow over time. Note that
> > if the estimate is greater than the user provided maximum value of
> interest
> > the graph maxes out at that time, effectively indicating no overflow
> > concerns.
> >
> >
> >
> > [cid:image003.png@01D55697.965C27D0]
> >
> >
> >
> > A similar display for flowfile count is displayed as well.
> >
> >
> >
> > The current state of work can be found at
> > https://github.com/jmark99/nifi/tree/time-to-overflow
> >
> >
> >
> > I welcome your (or any others) feedback on this effort.
> >
> >
> >
> > Thanks,
> > Mark
> >
> >
> >
> > P.S. If the images are not displaying, they can be viewed at
> > https://github.com/jmark99/nifi-images
> >
> >
> >
> >
> >
> >
> >
> > -Original Message-
> > From: Yolanda Davis  > yolanda.m.da...@gmail.com>>
> > Sent: Monday, August 19, 2019 11:29 AM
> > To: dev@nifi.apache.org<mailto:dev@nifi.apache.org>
> > Subject: Re:[EXT] [DISCUSS] Predictive Analytics for NiFi Metrics
> >
> >
> >
> > Hello All,
> >
> >
> >
> > I just wanted to follow up on the discussion we started a couple of weeks
> > ago concerning an analytics framework for NiFi metrics.  Working with
> Andy
> > Christianson and Matt Burgess we shaped our ideas and drafted a proposal
> > for

Re:[EXT] [DISCUSS] Predictive Analytics for NiFi Metrics

[Yolanda Davis]

Hello All,

I just wanted to follow up on the discussion we started a couple of weeks
ago concerning an analytics framework for NiFi metrics.  Working with Andy
Christianson and Matt Burgess we shaped our ideas and drafted a proposal
for this feature on the Apache NiFi Wiki [1] . We've also begun
implementing some of these ideas in a feature branch (which is work in
progress) [2].  We’d appreciate any questions or feedback you may have.

Thanks,

-yolanda

[1] -
https://cwiki.apache.org/confluence/display/NIFI/Operational+Analytics+Framework+for+NiFi
[2] - https://github.com/apache/nifi/commits/analytics-framework

On Wed, Jul 31, 2019 at 9:58 AM Andy Christianson
 wrote:

> As someone who operated a 24/7 mission-critical NiFi flow, this feature
> would have been a life saver. If I'm heading home on a Friday, it would be
> great to have some blinking red lights to let me know that the system
> predicts that it is going to experience backpressure sometime over the
> weekend, so that corrective action could be taken before leaving.
>
> Since there is support in the community for this, I created a JIRA to
> track the effort:
>
> https://issues.apache.org/jira/browse/NIFI-6510
>
> I also created a JIRA to track the remote protocol:
>
> https://issues.apache.org/jira/browse/NIFI-6511
>
>
> Regards,
>
> Andy
>
>
> Sent from ProtonMail, Swiss-based encrypted email.
>
> ‐‐‐ Original Message ‐‐‐
> On Wednesday, July 31, 2019 6:57 AM, Arpad Boda  wrote:
>
> > If you could share a bit more details about your OPC and Modbus usage,
> that
> > would be highly appreciated!
> >
> > On Wed, Jul 31, 2019 at 12:01 PM Craig Knell craig.kn...@gmail.com
> wrote:
> >
> > > Sounds. Great
> > > Let me know if you need some help
> > > Best regards
> > > Craig
> > >
> > > > On 31 Jul 2019, at 17:31, Arpad Boda ab...@cloudera.com.invalid
> wrote:
> > > > Craig,
> > > > OPC ( https://issues.apache.org/jira/browse/MINIFICPP-819 ) and
> Modbus (
> > > > https://issues.apache.org/jira/browse/MINIFICPP-897 ) are on the
> way for
> > > > MiNiFi c++, hopefully both will be part of next release (0.7.0).
> > > > It's gonna be legen... wait for it! :)
> > > > Regards,
> > > > Arpad
> > > >
> > > > > On Wed, Jul 31, 2019 at 2:30 AM Craig Knell craig.kn...@gmail.com
> > > > > wrote:
> > > >
> > > > > Hi Folks
> > > > > That's our use case now. All our Models are run in python.
> > > > > Currently we send events to the ML via http, although this is not
> > > > > optimal
> > > >
> > > > > Our use case is edge ML where we want a light weight wrapper for
> > > > > Python code base.
> > > > > Jython however does not work with the code base
> > > > > I'm think of changing the interface to some thing like REDIS for
> pub/sub
> > > > > Id also like this to be a push deployment via minifi
> > > > > Also support for sensors via protocols via Modbus and OPC would be
> great
> > > > > Craig
> > > > >
> > > > > > On Wed, Jul 31, 2019 at 1:43 AM Joe Witt joe.w...@gmail.com
> wrote:
> > > > > > Definitely something that I think would really help the
> community. It
> > > > > > might make sense to frame/structure these APIs such that an
> internal
> > > > > > option
> > > > > > could be available to reduce dependencies and get up and running
> but
> > > > > > that
> > > >
> > > > > > also just as easily a remote implementation where the engine
> lives and
> > > > > > is
> > > >
> > > > > > managed externally could also be supported.
> > > > > > Thanks
> > > > > > On Tue, Jul 30, 2019 at 1:40 PM Andy LoPresto
> alopre...@apache.org
> > > > > > wrote:
> > > > > >
> > > > > > > Yolanda,
> > > > > > > I think this sounds like a great idea and will be very useful
> to
> > > > > > > admins/users, as well as enabling some interesting next-level
> > > > > > > functionality
> > > > > >
> > > > > > > and insight generation. Thanks for putting this out there.
> > > > > > > Andy LoPresto
> > > > > > > alopre...@apache.org
> > > > > > > alopresto.apa...@gmail.com
> > > > 

Re: [VOTE] Create NiFi Standard Libraries sub-project

[Yolanda Davis]

+1 Create NiFi Standard Libraries (binding)

On Tue, Sep 3, 2019 at 7:03 PM Koji Kawamura  wrote:

> +1 Create NiFi Standard Libraries (binding)
>
> On Wed, Sep 4, 2019 at 7:25 AM Mike Thomsen 
> wrote:
> >
> > +1 binding
> >
> > On Tue, Sep 3, 2019 at 5:33 PM Andy LoPresto 
> wrote:
> >
> > > +1, create NiFi Standard Libraries (binding)
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > alopresto.apa...@gmail.com
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >
> > > > On Sep 3, 2019, at 2:16 PM, Bryan Bende  wrote:
> > > >
> > > > All,
> > > >
> > > > In a previous thread there was a plan discussed to restructure some
> of
> > > > the repositories in order to address several different issues, such
> as
> > > > build time, reusability of code, and eventually separating how the
> > > > framework and extensions are released [1][2].
> > > >
> > > > The overall plan requires many steps to get there, so I'd like to
> > > > propose starting with a small actionable step - the creation of a new
> > > > sub-project called NiFi Standard Libraries (formerly referred to as
> > > > nifi-commons).
> > > >
> > > > Project Name: Apache NiFi Standard Libraries
> > > > Git Repository: nifi-standard-libraries
> > > > JIRA: NIFILIBS
> > > >
> > > > Description:
> > > >
> > > > A collection of standard implementations used across the NiFi
> ecosystem.
> > > >
> > > > Candidate Libraries:
> > > >
> > > > In general, each library may consist of multiple Maven modules, and
> > > > should be independent from the rest of the ecosystem, and from other
> > > > libraries within NiFi Standard Libraries.
> > > >
> > > > In addition, each library may make it's own decision about whether it
> > > > is considered a public facing extension point/API, or an internal
> > > > library that may be changed at any time. This should be documented in
> > > > a README at the root of each library, such as
> > > > nifi-standard-libraries/nifi-xyz/README.
> > > >
> > > > An initial library that has been discussed was referred to as
> > > > 'nifi-security' and would centralize much of the security related
> code
> > > > shared by NiFi and NiFi Registry, such as shared security APIs, and
> > > > implementations for various providers, such as LDAP/Kerberos/etc.
> > > >
> > > > A second candidate library would be an optimistic-locking library
> > > > based on NiFi's revision concept. Currently this has been created
> > > > inside nifi-registry for now [3], but could be moved as soon as
> > > > nifi-standard-libraries exists.
> > > >
> > > > (This list does not have to be final in order to decide if we are
> > > > creating NiFi Standard Libraries or not)
> > > >
> > > > Integration & Usage:
> > > >
> > > > Once NiFi Standard Libraries is created, the community can start
> > > > creating and/or moving code there and perform releases as necessary.
> A
> > > > release will consist of the standard Apache source release, plus
> > > > artifacts released to Maven central. The community can then decide
> > > > when it is appropriate to integrate these released libraries into one
> > > > of our downstream projects.
> > > >
> > > > For example, if we create a nifi-security library in
> > > > nifi-standard-libraries, we can release that whenever we decide, but
> > > > we may not integrate it into NiFi or NiFi Registry until it makes
> > > > sense for a given release of those projects.
> > > >
> > > > This vote will be open for 48 hours, please vote:
> > > >
> > > > [ ] +1 Create NiFi Standard Libraries
> > > > [ ] +0 no opinion
> > > > [ ] -1 Do not create NiFi Standard Libraries because...
> > > >
> > > > [1]
> > >
> http://apache-nifi.1125220.n5.nabble.com/discuss-Splitting-NiFi-framework-and-extension-repos-and-releases-td27499.html
> > > > [2]
> > >
> https://cwiki.apache.org/confluence/display/NIFIREG/NiFi+Project+and+Repository+Restructuring
> > > > [3]
> > >
> https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-revision
> > >
> > >
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

[DISCUSS] Predictive Analytics for NiFi Metrics

[Yolanda Davis]

Hello Everyone,

I wanted to reach out to the community to discuss potentially enhancing
NiFi to include predictive analytics that can help users assess and predict
NiFi behavior and performance. Currently NiFi has lots of metrics available
for areas including jvm and flow component usage (via component status) as
well as provenance data which NiFi makes available either through the UI or
reporting tasks (for consumption by other systems). Past discussions in the
community cite users shipping this data to applications such as Prometheus,
ELK stacks, or Ambari metrics for further analysis in order to
capture/review performance issues, detect anomalies, and send alerts or
notifications.  These systems are efficient in capturing and helping to
analyze these metrics however it requires customization work and knowledge
of NiFi operations to provide meaningful analytics within a flow context.

In speaking with Matt Burgess and Andy Christianson on this topic we feel
that there is an opportunity to introduce an analytics framework that could
provide users reasonable predictions on key performance indicators for
flows, such as back pressure and flow rate, to help administrators improve
operational management of NiFi clusters.  This framework could offer
several key features:

   - Provide a flexible internal analytics engine and model api which
   supports the addition of or enhancement to onboard models
   - Support integration of remote or cloud based ML models
   - Support both traditional and online (incremental) learning methods
   - Provide support for model caching  (perhaps later inclusion into a
   model repository or registry)
   - UI enhancements to display prediction information either in existing
   summary data, new data visualizations, or directly within the flow/canvas
   (where applicable)

For an initial target we thought that back pressure prediction would be a
good starting point for this initiative, given that back pressure detection
is a key indicator of flow performance and many of the metrics currently
available would provide enough data points to create a reasonable
performing model.  We have some ideas on how this could be achieved however
we wanted to discuss this more with the community to get thoughts about
tackling this work, especially if there are specific use cases or other
factors that should be considered.

Looking forward to everyone's thoughts and input.

Thanks,

-yolanda

--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: Rules engine documentation?

[Yolanda Davis]

Hi Mike,

In the 1.10 release the EasyRules Controller service has documentation
related to how rules can be defined and how the service can be configured.
Also I am working on a blog post detailing the Rules API in general and how
it can be leveraged using InvokeScriptedProcessor.  Hopefully this can help
supplement information that didn't make 1.10.  There have been
contributions since 1.10 that have additional functionality, including
action handler implementations and scripted rules/handlers, which also have
more detailed documentation.

I hope this helps. Please let me know if you have any other questions.

Thanks,

-yolanda

On Tue, Nov 19, 2019 at 7:28 AM Mike Thomsen  wrote:

> Are there any docs yet on the rules engine-related changes that have been
> added?
>
> Thanks,
>
> Mike
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [VOTE] Release Apache NiFi 1.10.0 (rc3)

[Yolanda Davis]

+1 (binding)

Ran through release helper and verified on secure cluster with samples
flows and recent contributions for analytics. All looks good.

3rd times a charm :). Thanks Joe for RMing this release!

On Fri, Nov 1, 2019 at 12:22 PM Scott Aslan  wrote:

> +1 (binding)
>
> Verified signatures, hashes, build, etc. Tested various parameter use
> cases.
>
> Thanks for RMing Joe!
>
> On Fri, Nov 1, 2019 at 10:40 AM Matt Burgess  wrote:
>
> > +1 (binding)
> >
> > Ran thru the release helper and tried a number of flows with various
> > controller services and processors, LGTM
> >
> > Thanks for RM'ing Joe!
> >
> > On Tue, Oct 29, 2019 at 1:32 PM Joe Witt  wrote:
> > >
> > > Hello,
> > >
> > > I am pleased to be calling this vote for the source release of Apache
> > NiFi
> > > nifi-1.10.0.
> > >
> > > As they say 'third time's a charm'.
> > >
> > > The source zip, including signatures, digests, etc. can be found at:
> > > https://repository.apache.org/content/repositories/orgapachenifi-1151
> > >
> > > The source being voted upon and the convenience binaries can be found
> at:
> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.10.0/
> > >
> > > The Git tag is nifi-1.10.0-RC3
> > > The Git commit ID is b217ae20ad6a04cac874b2b00d93b7f7514c0b88
> > >
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=b217ae20ad6a04cac874b2b00d93b7f7514c0b88
> > >
> > > Checksums of nifi-1.10.0-source-release.zip:
> > > SHA256:
> e9b0a14b3029acd69c6693781b6b6487c14dda12676db8b4a015bce23b1029c1
> > > SHA512:
> > >
> >
> b07258cbc21d2e529a1aa3098449917e2d059e6b45ffcfcb6df094931cf16caa8970576555164d3f2290cfe064b5780ba1a8bf63dad04d20100ed559a1cfe133
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/joewitt.asc
> > >
> > > KEYS file available here:
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > > 384 issues were closed/resolved for this release:
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12344993
> > >
> > > Release note highlights can be found here:
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.10.0
> > >
> > > The vote will be open for 72 hours.
> > > Please download the release candidate and evaluate the necessary items
> > > including checking hashes, signatures, build
> > > from source, and test. Then please vote:
> > >
> > > [ ] +1 Release this package as nifi-1.10.0
> > > [ ] +0 no opinion
> > > [ ] -1 Do not release this package because...
> >
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: [ANNOUNCE] New Apache NiFi Committer Rob Fellows

[Yolanda Davis]

Congratulations Rob!


> On Sep 25, 2019, at 10:09 AM, Marc Parisi  wrote:
> 
> Congrats Rob!
> 
>> On Wed, Sep 25, 2019 at 9:59 AM Rob Moran  wrote:
>> 
>> Thanks for all your work, Rob -- congrats!
>> 
>> On Wed, Sep 25, 2019 at 9:29 AM Andrew Lim 
>> wrote:
>> 
>>> Congrats Rob!
>>> 
>>> 
 On Sep 24, 2019, at 7:56 PM, Tony Kurc  wrote:
 
 Apache NiFi community,
 On behalf of the Apache NiFI PMC, I am very pleased to announce that
>> Rob
 has accepted the PMC's invitation to become a committer on the Apache
>>> NiFi
 project. We greatly appreciate all of Rob's hard work and generous
 contributions to the project. We look forward to his continued
>>> involvement
 in the project.
 
 What stood out with Rob are his regular code contributions and reviews
>> on
 many parts of the project to include NiFi, NiFi FDS, and NiFi Registry
 since early this year. Additionally, he's been doing the
 not-always-glamorous work of helping verify releases, which was a huge
 assist in getting NiFi 1.9.1, NiFi Registry 0.4.0 and 0.5.0, and NiFi
>> FDS
 0.2.0 out to the community.
 
 Welcome and congratulations!
 Tony
>>> 
>>> 
>> 

Re: [VOTE] Release Apache NiFi 1.11.0 (rc3)

[Yolanda Davis]

+1 (binding)

Ran through release guide, confirming signatures and hashes.  Built with
Java 11 and ran on Mac Os.
Created simple flows on a single node instance and confirmed some of the
latest additions around rules with no issues encountered.

On Wed, Jan 22, 2020 at 9:59 AM Tony Kurc  wrote:

> +1 (binding)
>
> successfully went through the helper guide on mxlinux 18, did not discover
> any issues with the signature, hashes, or build.
>
> On Wed, Jan 22, 2020 at 8:48 AM Aldrin Piri  wrote:
>
> > +1, binding
> >
> > comments:
> > signature - good
> > hashes - good
> > L - good
> > built and tested some simple flows in conjunction with registry on macos,
> > debian
> >
> > thanks for organizing, Joe!
> >
> >
> > On Wed, Jan 22, 2020 at 8:02 AM Mike Thomsen 
> > wrote:
> >
> > > +1 binding. Ran a flow with a custom processor and verified the fix for
> > > NIFI-7044.
> > >
> > > On Wed, Jan 22, 2020 at 3:15 AM 高橋喜秋  wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > - [x] Non-RPM (usual) build with Java 8.
> > > > - [x] Deploy on 3 node secured cluster in CentOS 7.7.0 and run on
> Java
> > 8.
> > > > - [x] Non-RPM (usual) build with Java 11.
> > > > - [x] Deploy on 1 node non secured cluster in CentOS 7.7.0 and run on
> > > Java
> > > > 11.
> > > >
> > > > Thanks for working for the release!
> > > >
> > > > On 2020/01/19 20:21:02, Joe Witt  wrote:
> > > > > Hello,
> > > > >
> > > > > I am pleased to be calling this vote for the source release of
> Apache
> > > > NiFi
> > > > > nifi-1.11.0.
> > > > >
> > > > > The source zip, including signatures, digests, etc. can be found
> at:
> > > > >
> > https://repository.apache.org/content/repositories/orgapachenifi-1155
> > > > >
> > > > > The source being voted upon and the convenience binaries can be
> found
> > > at:
> > > > > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.11.0/
> > > > >
> > > > > A helpful reminder on how the release candidate verification
> process
> > > > works:
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> > > > >
> > > > > The Git tag is nifi-1.11.0-RC3
> > > > > The Git commit ID is 633408bce7ad34dad727ed9c4edfd36a224f3f12
> > > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=633408bce7ad34dad727ed9c4edfd36a224f3f12
> > > > >
> > > > > Checksums of nifi-1.11.0-source-release.zip:
> > > > > SHA256:
> > > 0e2d77265fc7cedfbdb9588df1dd7f456fd18b6288d65eb5e21befe23af7c567
> > > > > SHA512:
> > > > >
> > > >
> > >
> >
> 4880fa3482b3e8d8eed439848fe0a6596826d7ad46425a91b0dd4a4bcd178259327380b24045b7991dbdf8449abdfdda145786b6863eb603f6cef3b9e0ae8ec1
> > > > >
> > > > > Release artifacts are signed with the following key:
> > > > > https://people.apache.org/keys/committer/joewitt.asc
> > > > >
> > > > > KEYS file available here:
> > > > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > > > >
> > > > > 129 issues were closed/resolved for this release:
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12346451
> > > > >
> > > > > Release note highlights can be found here:
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.11.0
> > > > >
> > > > > The vote will be open for 72 hours.
> > > > > Please download the release candidate and evaluate the necessary
> > items
> > > > > including checking hashes, signatures, build
> > > > > from source, and test. Then please vote:
> > > > >
> > > > > [ ] +1 Release this package as nifi-1.11.0
> > > > > [ ] +0 no opinion
> > > > > [ ] -1 Do not release this package because...
> > > > >
> > > >
> > >
> >
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: In memoriam of Jeff Storck

[Yolanda Davis]

I echo everyone in saying I am heartbroken over the loss of Jeff. I'm so
happy to have known him.  We had lots of conversations around all things
code, food and one of our favorite TV shows "Atlanta".  At times if anyone
were listening in they'd hear us go from having a fierce (but friendly)
debate to laughing hysterically.  He was (and to me will always be) such a
good guy.

Rest in Peace, Jeff

On Mon, Jun 15, 2020 at 3:13 PM Jeremy Dyer  wrote:

> This is shocking and heartbreaking news. Jeff was a great guy and will be
> deeply missed.
>
> The last time I saw Jeff in person was with Aldrin. We were eating at
> Bonchon chicken and he was mocking me for how little spice I could handle
> XD. I could always count on him for a good Dumb and Dumber reference and
> laugh. We also shared a common hatred for conference food.
>
> RIP Jeff
>
> On Mon, Jun 15, 2020 at 2:33 PM Joe Witt  wrote:
>
>> You will be greatly missed.  Your impact to this community has been
>> tremendous.  The items Andy summarizes were huge efforts that you drove
>> over periods of many many months if not a year or more and they make NiFi
>> so much more accessible than before.
>>
>> RIP Jeff.
>>
>>
>>
>> On Mon, Jun 15, 2020 at 11:24 AM Andy LoPresto 
>> wrote:
>>
>>> It is with a heavy heart that I write to the NiFi community today. Jeff
>>> Storck, a PMC member, committer, and genuine and helpful presence in the
>>> community, has passed away.
>>>
>>> I was lucky enough to know Jeff personally for many years, and his
>>> absence is a huge loss to all of us who did. Jeff was incredibly
>>> intelligent, but also kind and willing to share his experience with
>>> everyone. Whether playing volleyball (I am nowhere near as good but he
>>> humored me), discussing the best ramen and sushi spots, or evaluating a new
>>> exercise regime, Jeff brought passion to everything. A number of us are
>>> sharing stories of our favorite times with Jeff, and I am touched by how
>>> many people have a memory of Jeff reaching out and patiently helping them
>>> when they were new or struggling with a task.
>>>
>>> While other colleagues would happily transition to any topic _but_ work
>>> when we went to a nearby brewery at the end of a long day, Jeff would sit
>>> down next to me and say with a smile, "Ok Andy, work's done, now we can
>>> _really_ talk about Groovy unit testing." He never shied away from
>>> expressing his perspective and stood on conviction, but he was also open
>>> and genuinely wanted to hear other views to expand his mind.
>>>
>>> If you come across a Spock test in the NiFi codebase, that was most
>>> likely Jeff's work. He was intimately involved in much of the most
>>> challenging code - especially Kerberos integration, making the difficult
>>> but critical processes easier for our users. Anyone running NiFi on Java 11
>>> should thank Jeff, as that was a labor of love, pushing against the
>>> headwinds of so many compatibility issues and language changes. The ease
>>> with which NiFi runs on multiple versions and platforms belies the immense
>>> amount of effort and dedication that he put into making this happen.
>>>
>>> There are so many aspects to Jeff that a note like this could never
>>> capture, but one that stands above the rest to me is Jeff's passion for
>>> learning and growth. He devoted himself to doing the best he could and
>>> constantly improving that. That is a noble philosophy that I know I will
>>> remember and admire moving forward. I’ve already started learning Kotlin
>>> because of Jeff’s enthusiasm and encouragement.
>>>
>>> Jeff’s family has created a GoFundMe page [1] and there they describe
>>> their intent to celebrate his life. I think that message is very positive
>>> and uplifting. To anyone wondering how they can honor Jeff's legacy, I
>>> suggest offering a helping hand to someone who needs it. Something as
>>> simple as responding to an extra "newbie" mailing list question at the end
>>> of a long day, or taking on a challenging task because your neighbor has
>>> their plate full. That's how Jeff lived, and he made the world a better
>>> place.
>>>
>>>
>>> Andy
>>>
>>> [1] https://www.gofundme.com/f/in-memory-of-the-awesome-jeff-storck
>>>
>>> Andy LoPresto
>>> alopre...@apache.org
>>> alopresto.apa...@gmail.com
>>> He/Him
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>>

-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: Analytic Framework Metrics / Logging

[Yolanda Davis]

Hello Aliesha,

Thank you for reaching out!  What specific analytics settings did you
change in nifi.properties? For example by default the
*nifi.analytics.predict.enabled* property is set to false. Enabling that to
be true by itself should turn on predictions (after a restart of nifi).

Also if you are interested in seeing debug for the analytics package you
can add the following entry to the logback.xml file under the conf folder:

**

Finally I do have a blog which may offer more insights on how this works
here:
http://lonnifi.blogspot.com/2019/11/back-pressure-prediction-deep-dive.html.
It includes more information on the feature and its settings. It also has
an accompanying video.

Please don't hesitate to reach out if you have any further questions.

Best Regards

-yolanda

On Wed, Feb 10, 2021 at 4:18 PM Aliesha Alli 
wrote:

> Hello NiFi Team,
> I have downloaded both NiFi Linux and Windows on separate systems and
> configured "analytics.predict" properties, created a FlowFile, and changed
> the processors to Debug. 100% of the threshold object is met, but I don't
> see any events for predictions or R-Square score in the nifi-app.log file.
> In Linux specifically as this is my main environment, what do I need to
> enable in order to see the analytic prediction or events associated with
> the analytic framework?
> Thank you,Aliesha
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: Processing JSON Files Embedded with Base64 Encoded Files

[Yolanda Davis]

Hello Chris,

Depending on the structure of your Json file, you may be able to achieve
what you need using processors such as JoltTransformJSON

paired with NiFi's expression language which supports base64

encoding and decoding.  JoltTransform is very useful if you're looking to
convert or transform incoming JSON data to another json structure. Also
EvaluateJsonPath

may
be useful as well if you're looking to extract data first from json then
potentially decode/encode

Hope this helps!

-yolanda

On Tue, Aug 15, 2023 at 9:45 AM Reid, Chris 
wrote:

> Hello.
>
> I understand that there's currently no Apache NiFi processor that can
> process JSON files embedded with Base64 encoded files.
>
> I'd like to know if it's possible to develop a custom Apache NiFi
> processor to process JSON files embedded with Base64 files?
>
> Thank you.
>
> Chris
>


-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis