Re: Automated Testing of Nifi Flow by injecting data into Processors

[Andrew Psaltis]

Hi Alexei,
If you're talking about testing your flow with simulated data as if it's
deployed, basically not unit tests. I've seen this done in several
different ways:
1. Using the GenerateFlowFile processor and adding in custom text. At
first, that may seem obvious, however, if you think about it in that custom
text field you can add in a block of JSON data that then in a subsequent
processor you can manipulate to populate various values and then convert it
to the format you need. From here you can have the data continue on to the
rest of your flow, or if you want to test the reading of data from a type
of source, perhaps File then you can write the data out to test location
and have your regular flow read it in. There are various ways you can take
this to make it as elaborate as necessary.

2. Build a custom processor to do the data generation. Here is an example
for TimeSeries data:
https://medium.com/hashmapinc/its-here-an-apache-nifi-simulator-for-generating-random-and-realistic-time-series-data-d7e463aa5e78

Does this help with idea on how to do it?



On Wed, Mar 6, 2019 at 11:50 AM Zenin, Alexei 
wrote:

> Hi,
>
> Is there any way in Nifi to test out your whole flow without having the
> source systems online and running. I need this kind of capability to run
> automated integration tests which tests if the flow behaves as expected.
> None of the tools or libraries I have seen accomplish this and the Nifi API
> does not seem to allow to inject data directly into a processor (Nifi's
> TestRunner seems to be able to create FlowFiles but can we inject mock data
> to a processor through the Nifi REST API?). I think this would be a
> critical feature to make this system more production ready and abide by
> best practices for CICD.
>
> Regards,
> Alexei
> To unsubscribe from receiving further commercial electronic messages from
> The Bank of Nova Scotia in Canada, please click here:
> https://unsubscribe.scotiabank.com?entid=BNS=SBNK
>
> To unsubscribe from receiving further commercial electronic messages from
> certain other senders set out in the attached list, please click here:
> https://www.unsubscribe.gwm.scotiabank.com?page=gwm
>
> This email may contain confidential information the use of which by an
> unintended recipient is unauthorized. This email may also contain important
> disclosure information for the records of the intended recipient(s).  For
> details please click here:
> http://www.scotiabank.com/email_disclaimer/email_english.html
>
> Pour vous désabonner et cesser de recevoir des messages électroniques
> commerciaux de La Banque de Nouvelle-Écosse, veuillez cliquer ici :
> https://desabonnement.banquescotia.com?entid=BNS=SBNK
>
> Pour vous désabonner et cesser de recevoir des messages électroniques
> commerciaux de certains autres expéditeurs figurant dans la liste
> ci-jointe, veuillez cliquer ici :
> https://www.unsubscribe.gwm.scotiabank.com?page=gwm=fr
>
> Cette transmission peut contenir de l'information confidentielle et son
> utilisation par toute personne autre que la personne à laquelle cette
> transmission est destinée est interdite. Le présent courriel peut aussi
> contenir des renseignements importants pour les dossiers du ou des
> destinataires prévus.  Pour plus de détails, veuillez vous diriger vers:
> http://www.scotiabank.com/email_disclaimer/email_francais.html
>
> Para dejar de recibir mensajes electrónicos comerciales de The Bank of
> Nova Scotia en Canadá, haga clic aquí:
> https://cancelarsuscripcion.scotiabank.com?entid=BNS=SBNK
>
> Para dejar de recibir mensajes electrónicos comerciales de otras entidades
> incluidas en la lista adjunta, haga clic aquí:
> https://www.unsubscribe.gwm.scotiabank.com?page=gwm=es
>
> Este correo electrónico puede contener información confidencial cuyo uso
> por parte de personas distintas de los destinatarios del mismo está
> prohibido. El mensaje puede tener también datos importantes para los
> archivos   del(de los) destinatario(s) a quien(es) está dirigido.  Para más
> detalles, por favor dirigirse a:
> http://www.scotiabank.com/email_disclaimer/email_espanol.html

Re: Apache NiFi MiNiFi 0.5.0 RC2 Release Helper Guide

[Andrew Psaltis]

Sorry for the goof on response thread.

Andy,
Thanks for the key education, greatly appreciated.

On Tue, Jul 3, 2018 at 02:10 Andy LoPresto  wrote:

> Hi Andrew,
>
> A couple things:
>
> * You accidentally replied to the release helper guide; I think you meant
> to vote on the [VOTE] thread
> * the warning message you received during GPG verification simply means
> that you had not previously marked Jeremy’s key as “trusted” via your GPG
> application. The intended process is:
>
> * Jeremy posts his public key on a key server
> * You verify Jeremy’s key via a different channel (chat/in-person/voice
> verification) — this is where the key fingerprint is useful; he can read it
> over the phone and you, knowing his voice, can verify that he is using the
> key ostensibly published by him
> * If you do not know Jeremy or cannot contact him, you can delegate that
> trust verification to someone else. For example, I have verified the key
> fingerprint with Jeremy offline, so I trust that this key is his. I have
> signed that public key using my private key (key ID 0x2F7DEF69) and I can
> publish that signature to public key servers. Now, if you trust my key, you
> can accept that transitive trust as well. (The servers are under stress
> right now but this link should show that when the server is up:
> https://pgp.mit.edu/pks/lookup?search=0x6B295AD5=index).
> * Once you have verified or trust that the key represents Jeremy, you can
> assign it a level of “owner trust” in your GPG application, ranging from
> Never -> Marginal -> Full, representing how seriously you believe this is
> Jeremy’s key.
> * After a trust level has been assigned, you will not get the message you
> did. You will get a message like the one below:
>
> hw12203:/Users/alopresto/Workspace/scratch/release_verification/minifi-java-0.5.0
> (master) alopresto
>  0s @ 11:09:55 $ gpg --verify -v minifi-0.5.0-source-release.zip.asc
> gpg: assuming signed data in 'minifi-0.5.0-source-release.zip'
> gpg: Signature made Thu Jun 28 09:31:10 2018 PDT
> gpg:using RSA key 50AA60AD5D58311187B0BEB5C6E550DA6B295AD5
> gpg:issuer "jeremyd...@apache.org"
> gpg: using pgp trust model
> gpg: Good signature from "Jeremy Dyer (CODE SIGNING KEY) <
> jeremyd...@apache.org>" [full]
> gpg: binary signature, digest algorithm SHA512, key algorithm rsa4096
>
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Jul 1, 2018, at 8:35 PM, Andrew Psaltis 
> wrote:
>
> +1 (non-binding)
>
> - verified keys
> - verified signatures
> - verified README's, NOTICE and LICENSE
> - tested c2 NiFiRestConfigurationProvider with NiFi 1.6.0 and minifi from
> this build, various changes to template -- bumping versions, etc.
>
> One thing I noticed when verifying the keys, which I am not sure is an
> issue is the WARNING that the key is not certified with a trusted
> signature. The following is the output from the command:
>
> gpg: assuming signed data in 'minifi-0.5.0-source-release.zip'
> gpg: Signature made Fri Jun 29 00:31:10 2018 +08
> gpg:using RSA key 50AA60AD5D58311187B0BEB5C6E550DA6B295AD5
> gpg:issuer "jeremyd...@apache.org"
> gpg: Good signature from "Jeremy Dyer (CODE SIGNING KEY) <
> jeremyd...@apache.org>" [unknown]
> gpg: WARNING: This key is not certified with a trusted signature!
> gpg:  There is no indication that the signature belongs to the
> owner.
> Primary key fingerprint: 50AA 60AD 5D58 3111 87B0  BEB5 C6E5 50DA 6B29 5AD5
>
>
> On Fri, Jun 29, 2018 at 1:39 AM Jeremy Dyer  wrote:
>
> Hello Apache NiFi community,
>
> Please find the associated guidance to help those interested in
> validating/verifying the release so they can vote.
>
> # Download latest KEYS file:
>  https://dist.apache.org/repos/dist/dev/nifi/KEYS
>
> # Import keys file:
>  gpg --import KEYS
>
> # [optional] Clear out local maven artifact repository
>
> # Pull down minifi-0.5.0 source release artifacts for review:
>
>  wget
>
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi/0.5.0/minifi-0.5.0-source-release.zip
>  wget
>
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi/0.5.0/minifi-0.5.0-source-release.zip.asc
>  wget
>
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi/0.5.0/minifi-0.5.0-source-release.zip.sha1
>  wget
>
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi/0.5.0/minifi-0.5.0-source-release.zip.sha256
>
> # Verify the signature
>  gpg --verify minifi-0.5.0-source-release.zip.asc
>
> # Veri

Re: Apache NiFi MiNiFi 0.5.0 RC2 Release Helper Guide

[Andrew Psaltis]

+1 (non-binding)

- verified keys
- verified signatures
- verified README's, NOTICE and LICENSE
- tested c2 NiFiRestConfigurationProvider with NiFi 1.6.0 and minifi from
this build, various changes to template -- bumping versions, etc.

One thing I noticed when verifying the keys, which I am not sure is an
issue is the WARNING that the key is not certified with a trusted
signature. The following is the output from the command:

gpg: assuming signed data in 'minifi-0.5.0-source-release.zip'
gpg: Signature made Fri Jun 29 00:31:10 2018 +08
gpg:using RSA key 50AA60AD5D58311187B0BEB5C6E550DA6B295AD5
gpg:issuer "jeremyd...@apache.org"
gpg: Good signature from "Jeremy Dyer (CODE SIGNING KEY) <
jeremyd...@apache.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:  There is no indication that the signature belongs to the
owner.
Primary key fingerprint: 50AA 60AD 5D58 3111 87B0  BEB5 C6E5 50DA 6B29 5AD5


On Fri, Jun 29, 2018 at 1:39 AM Jeremy Dyer  wrote:

> Hello Apache NiFi community,
>
> Please find the associated guidance to help those interested in
> validating/verifying the release so they can vote.
>
> # Download latest KEYS file:
>   https://dist.apache.org/repos/dist/dev/nifi/KEYS
>
> # Import keys file:
>   gpg --import KEYS
>
> # [optional] Clear out local maven artifact repository
>
> # Pull down minifi-0.5.0 source release artifacts for review:
>
>   wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi/0.5.0/minifi-0.5.0-source-release.zip
>   wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi/0.5.0/minifi-0.5.0-source-release.zip.asc
>   wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi/0.5.0/minifi-0.5.0-source-release.zip.sha1
>   wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi/0.5.0/minifi-0.5.0-source-release.zip.sha256
>
> # Verify the signature
>   gpg --verify minifi-0.5.0-source-release.zip.asc
>
> # Verify the hashes (sha1 and sha256) match the source and what was
> provided in the vote email thread
>   sha1sum minifi-0.5.0-source-release.zip
>   sha256sum minifi-0.5.0-source-release.zip
>
> # Unzip minifi-0.5.0-source-release.zip
>
> # Verify the build works including release audit tool (RAT) checks
>   cd minifi-0.5.0
>   mvn clean install -Pcontrib-check
>
> # Verify the contents contain a good README, NOTICE, and LICENSE.
>
> # Verify the git commit ID is correct
>
> # Verify the RC was branched off the correct git commit ID
>
>
> There are three convenience binaries generated as part of this process.
> The MiNiFi assembly, a MiNiFi Toolkit assembly, and a MiNiFi C2 Assembly.
>
> For the MiNiFi assembly:
>
> # Look at the resulting convenience binary as found in
> minifi-assembly/target
>
> # Make sure the README, NOTICE, and LICENSE are present and correct
>
> # Run the resulting convenience binary and make sure it works as expected
>
>
> For the MiNiFi Toolkit assembly:
>
> # Look at the resulting convenience binary as found in
> minifi-toolkit/minifi-toolkit-assembly/target
>
> # Make sure the README, NOTICE, and LICENSE are present and correct
>
> # Run the resulting convenience binary and make sure it works as expected
>
>
> For the MiNiFi C2 assembly:
>
> # Look at the resulting convenience binary as found in
> minifi-c2/minifi-c2-assembly/target
>
> # Make sure the README, NOTICE, and LICENSE are present and correct
>
> # Run the resulting convenience binary and make sure it works as expected
>
>
>
> # Send a response to the vote thread indicating a +1, 0, -1 based on your
> findings.
>
>
> Thank you for your time and effort to validate the release!
>

Re: [ANNOUNCE] New Apache NiFi Committer Sivaprasanna Sethuraman

[Andrew Psaltis]

Congratulations Sivaprasanna!

On Tue, Jun 5, 2018 at 12:25 PM, Pierre Villard  wrote:

> Congrats, well deserved!
>
> 2018-06-05 18:14 GMT+02:00 Mike Thomsen :
>
> > Congratulations, Sivaprasanna!
> >
> > On Tue, Jun 5, 2018 at 10:22 AM Bryan Bende  wrote:
> >
> > > Congrats! and thank you for your contributions to the NiFi community.
> > >
> > > On Tue, Jun 5, 2018 at 10:16 AM, Kevin Doran 
> wrote:
> > > > Congrats, Sivaprasanna!
> > > >
> > > > On 6/5/18, 10:09, "Tony Kurc"  wrote:
> > > >
> > > > On behalf of the Apache NiFI PMC, I am very pleased to announce
> > that
> > > > Sivaprasanna has accepted the PMC's invitation to become a
> > committer
> > > on the
> > > > Apache NiFi project. We greatly appreciate all of Sivaprasanna's
> > > hard work
> > > > and generous contributions to the project. We look forward to
> > > continued
> > > >  involvement in the project.
> > > >
> > > > Sivaprasanna has been working with the community on the mailing
> > > lists, and
> > > > has a big mix of code and feature contributions to include
> features
> > > and
> > > > improvements to cloud service integrations like Azure, AWS, and
> > > Google
> > > > Cloud.
> > > >
> > > > Welcome and congratulations!
> > > >
> > > >
> > > >
> > >
> >
>

Re: Publishing Kafka Topics via Apache Nifi cluster.

[Andrew Psaltis]

Pierre,
Mentioning that JIRA is very timely, I was just talking with some yesterday
about NiFi and Kafka and this exact scenario came up and was something they
were keenly interested in being able to do. Route data for say different
stocks to different NiFi nodes via an RPG.  Happy to help our here with
finding time to work on that JIRA.

On Thu, May 31, 2018 at 10:55 AM, Pierre Villard <
pierre.villard...@gmail.com> wrote:

> As an additional note to what Bryan said, there is a JIRA [1] that could
> help in this case.
> I'm trying to find time to work on it... but no luck so far :)
>
> [1] https://issues.apache.org/jira/browse/NIFI-4026
>
>
>
> 2018-05-31 15:43 GMT+02:00 Bryan Bende :
>
> > Hello,
> >
> > If I'm understanding the situation correctly, you want ordering within
> > a key, but not necessarily total ordering across all your data?
> >
> > I'm making this assumption since you said you have 9 partitions on
> > your Kafka topic and you are partitioning by key, so the data for each
> > key is in order per partition.
> >
> > The list + fetch pattern with redistribution doesn't have a way to
> > control how the data is distributed, it is just round-robin and you
> > can control the batch size, but you can't partition the data to nodes
> > based on a key.
> >
> > There is an EnforceOrder processor [1] which was made to help with
> > this kind of scenario, I believe specifically for CDC scenarios where
> > the event log has to be processed in order. I haven't used it myself
> > so maybe others can help here, but I believe you would use your "key"
> > as the "Group Identifier" and then somehow you need to get an integer
> > value on each flow file that represents the order within the group. So
> > for example your A-event flow file would need some kind of attribute
> > like "order = 1" and then the B-event flow file would need an
> > attribute like "order = 2". You might be able to assign this order
> > using an UpdateAttribute processor right after the ListSFTP, but you
> > have to do it per key somehow.
> >
> > Another option is to just run the whole flow on primary node without
> > doing the site-to-site redistribution, but then you lose out on
> > parallel processing, and even on a single node I believe there are
> > cases where ordering is not guaranteed.
> >
> > Thanks,
> >
> > Bryan
> >
> > [1] https://nifi.apache.org/docs/nifi-docs/components/org.
> > apache.nifi/nifi-standard-nar/1.6.0/org.apache.nifi.processors.standard.
> > EnforceOrder/index.html
> >
> >
> > On Wed, May 30, 2018 at 4:35 PM, rey26  wrote:
> > > Hello Team,
> > >
> > > We have a apache Nifi cluster with 3 nodes and 3 nodes kafka cluster.We
> > are
> > > receiving some files which has transactions in orders.(A-type first and
> > than
> > > B-type)
> > > These events are in order but may come is different files.For example
> > > A-event for id 111 can be present in file 1 and B-event can come in
> > > immedaite file2 [B will always come after
> > > A-type for any ID].We want data need to be puslished in the same order
> > as it
> > > is received.
> > >
> > > We developed a flow using ListSFTP+FecthFTP+publishkafka combination in
> > > order ,have also done partitioning on kafka topic[9 partitions] on the
> > > basics of a key column
> > > and same key is used in Publish Kafka Processor.
> > >
> > > Al the events are published to the same partition but are going out of
> > order
> > > but within the partition are out of order.
> > > Example B-type events are coming before A-Type in kafka topic TEST.
> > >
> > > Now i have some queries regarding the above
> > >
> > > What i understood is that since the ListSFTP+FecthFTP improves load
> > > balancing but does it ensures ordering?
> > > File1 may go to Node1 and File2 may go to Node2 , and Node 2 can
> publish
> > the
> > > record to the same partition on kafka before Node1?
> > > Is there any way to gaurantee load order of files in Apache Nifi in
> > cluster
> > > Mode keeping perfomance in mind.?
> > >
> > > Since each task in PublishKafka processor is one publisher , if we run
> > the
> > > publish kafka on only primary node and pass only one broker-id does it
> > will
> > > do the trick?
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
> >
>

Re: [ANNOUNCE] New Apache NiFi Committer Kevin Doran

[Andrew Psaltis]

Congrats Kevin!

On Wed, Jan 3, 2018 at 9:17 AM, Bryan Bende  wrote:

> On behalf of the Apache NiFi PMC, I am very pleased to announce that
> Kevin Doran has accepted the PMC's invitation to become a committer on
> the Apache NiFi project. We greatly appreciate all of Kevin's hard
> work and generous contributions to the project. We look forward to his
> continued involvement in the project.
>
> Kevin has made significant contributions to the NiFi Registry project,
> especially around setting up the security model & REST APIs, and was a
> major contributor to getting the first release out. You also may have
> interacted with him on the mailing lists already, as he is always
> willing to dig into questions/issues and help out.
>
> Welcome and congratulations!
>



-- 
Thanks,
Andrew

Re: [ANNOUNCE] New NiFi PMC member Scott Aslan

[Andrew Psaltis]

Congrats Scott!! All of your work on the UI is greatly appreciated by
everyone that uses NiFi.

On Sat, Dec 2, 2017 at 2:46 PM, Tony Kurc  wrote:

> NiFi community,
> On behalf of the Apache NiFi PMC, I am pleased to announce that Scott Aslan
> has accepted the PMC's invitation to join the Apache NiFi PMC.  We greatly
> appreciate all of Scotts's hard work and generous contributions to the
> project. We look forward to continued involvement in the project.
>
> Scott has been delivering as a strong committer for quite a long time, is
> making huge progress on nifi-registry, and has had wide reach to the
> community in many ways, to include our mailing lists.
>
> Congratulations and welcome, Scott!
>
> Tony
>



-- 
Thanks,
Andrew

Re: Syslog processing from cisco switches to Splunk

[Andrew Psaltis]

Dave,
To clarify you are using the PutUDP processor, not the PutSplunk processor?

On Thu, Oct 19, 2017 at 7:31 AM, DAVID SMITH 
wrote:

> Hi
> We are trying to do something which on the face of it seems fairly simple
> but will not work.We have a cisco switch which is producing syslogs,
> normally we use zoneranger to send them to Splunk and the records are
> shown.However we want to do a bit of content routing, so we are using NiFi
> 0.7.3 with a ListenUDP on port 514 and we can see the records coming in to
> NiFi. Without doing anything to the records we use a putUDP to send records
> to the Splunk server, NiFi says they have sent successfully but they never
> show in Splunk.We have used a listenUDP on another NiFi and the records
> transfer and look exactly the same as they were sent.We have also used
> listenSyslog and putSyslog, but the listenSyslog says the records are
> invalid.
> Has anyone ever to do this, and can you give us any guidance on what we
> may be missing?
> Many thanksDave




-- 
Thanks,
Andrew

Re: [DRAFT][REPORT] Apache NiFi Board Report Oct 2017

[Andrew Psaltis]

Joe -- LGTM

On Mon, Oct 9, 2017 at 11:03 AM, Rob Moran  wrote:

> Joe, looks good with the couple changes Drew mentioned--thanks.
>
> On Mon, Oct 9, 2017 at 10:30 AM Andrew Lim 
> wrote:
>
> > Noticed the following:
> >
> > -In Health Report section: "seeing a very strong pipeline to both both
> > committer and PMC” (remove extra both)
> > -"Apache NiFi 1.4.0 was released Oct 1 2017” (Change? Release Notes says
> > Oct 2)
> >
> > Otherwise, looks great!
> >
> > -Drew
> >
> > > On Oct 9, 2017, at 10:23 AM, Marc  wrote:
> > >
> > > Joe
> > >
> > >  I think this looks great.
> > >
> > >  Thanks for putting this together,
> > >
> > >
> > >
> > > On Mon, Oct 9, 2017 at 10:12 AM, Joe Witt  wrote:
> > >
> > >> Team,
> > >>
> > >> Please see draft board report.  Am looking for feedback/edits quickly
> > >> as this needs to be submitted by Wed.  I'll probably send it in
> > >> tonight though as I might not have time over the next couple of days.
> > >>
> > >> Thanks
> > >> Joe
> > >>
> > >> **==**==**==**==**
> > >>
> > >> ## Description:
> > >> - Apache NiFi is an easy to use, powerful, and reliable system to
> > >>   process and distribute data.
> > >> - Apache NiFi MiNiFi is an edge data collection agent built to
> > seamlessly
> > >>   integrate with and leverage the command and control of NiFi. There
> are
> > >>   both Java and C++ implementations.
> > >> - Apache NiFi Registry is a centralized registry for key configuration
> > >> items
> > >>   including flow versions, assets, and extensions for Apache NiFi
> > >>   and Apache MiNiFi.
> > >> - Apache NiFi Nar Maven Plugin is a release artifact used for
> supporting
> > >>   the NiFi classloader isolation model.
> > >>
> > >> ## Issues:
> > >> - There are no issues requiring board attention at this time.
> > >>
> > >> ## Activity:
> > >> - Released Apache NiFi 1.4.0 which is another very heavy feature,
> > >> stability,
> > >>   and bug fix release.  It includes improvements on the web UI coming
> > >> from new
> > >>   contributors which is a great sign.
> > >> - The Apache NiFi downloads have been quite large and growing for some
> > >> time.
> > >>   The groundwork to solve this is the Apache NiFi Registry effort
> which
> > is
> > >>   coming along nicely and which the foundational elements of NiFi to
> > >> integrate
> > >>   with it are present.  We're probably looking at two more releases
> for
> > >> that
> > >>   to land and for which we can drop the NiFi build size down.
> > >> - We continue to hover around the 100 outstanding pull requests
> number.
> > >> What
> > >>   is encouraging is we've seen an uptick in contributions both in code
> > and
> > >>   review feedback from new community members.
> > >> - The Apache NiFi MiNiFi C++ agent has rapidly matured over the past
> > >> couple of
> > >>   months and is serving as a new source for contributions and
> > >> contributors.
> > >>
> > >> ## Health report:
> > >> - As stated in our past report, health of the community is strong and
> > >>   indicators of strength continue trending in the right direction.
> > Mailing
> > >>   list and JIRA activity is strong. ASF Hipchat is serving as an
> on-ramp
> > >> for
> > >>   new users to our mailing list and JIRA systems. We continue to see
> new
> > >>   users and contributors.
> > >> - The PMC and committer ranks did not grow during the reporting
> period.
> > >> This
> > >>   is a reflection of the PMC not focusing enough on the topic though
> as
> > >> we're
> > >>   seeing a very strong pipeline to both both committer and PMC. Expect
> > >> both
> > >>   new committers and PMC members during our next reporting period.
> > >>
> > >> ## PMC changes:
> > >>
> > >> - Currently 22 PMC members.
> > >> - Last PMC member added Fri May 26 2017.
> > >>
> > >> ## Committer base changes:
> > >>
> > >> - Currently 34 committers.
> > >> - Last committer added Thu Jun 01 2017.
> > >>
> > >> ## Releases:
> > >>
> > >> - Apache NiFi 1.4.0 was released Oct 1 2017.
> > >>
> > >> ## Mailing list activity:
> > >>
> > >> - Activity on the mailing lists remains extremely high with a mixture
> > >>   of new users, contributors, and deeper more experienced users and
> > >>   contributors sparking discussion and questions and filing bugs or
> > >>   new features.
> > >>
> > >> - us...@nifi.apache.org:
> > >>- 553 subscribers (up 34 in the last 3 months):
> > >>- 748 emails sent to list (862 in previous quarter)
> > >>
> > >> - dev@nifi.apache.org:
> > >>- 383 subscribers (up 12 in the last 3 months):
> > >>- 604 emails sent to list (947 in previous quarter)
> > >>
> > >> - iss...@nifi.apache.org:
> > >>- 46 subscribers (up 4 in the last 3 months):
> > >>- 5775 emails sent to list (7457 in previous quarter)
> > >>
> > >>
> > >> ## JIRA activity:
> > >>
> > >> - 335 JIRA tickets created in the last 3 months
> > >> - 263 JIRA tickets closed/resolved in the last 3 

Re: Using GregorianCalendar in apache nifi

[Andrew Psaltis]

Hi,
I'm not 100% sure I understand what your trying to accomplish.  However, if
you are wanting to use the Gregorian package from a script that is executed
with the executescript processor then please take a look at the
ExecuteScript cookbook, specifically part 3 [1].

Thanks,
Andrew

[1]
https://community.hortonworks.com/articles/77739/executescript-cookbook-part-3.html

On Fri, Aug 11, 2017 at 14:48 sally <salome.tkhilaishv...@gmail.com> wrote:

> Is  it possible to download apache-nifi open source project from  github
> and
> then add "import java.util.GregorianCallendar" inside ExecuteScript.java
> code and then deploy project  I  have tried it  but it throws errors what
> should i do?
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/Using-GregorianCalendar-in-apache-nifi-tp16629p16633.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Documents

[Andrew Psaltis]

Nardeep,
One option would be to go to the user guide, choose to print it, and then
change the destination from your printer to a PDF.

Hope that helps,
Andrew

On Thu, Aug 3, 2017 at 18:43 Nardeep Mandal <nardeep...@hotmail.com> wrote:

> Hi,
>
>
> I am wondering if you can let me have PDF or word doc for 'Apache NiFi
> User Guide' instead of going to internet browser.
>
> I lose track every time I start to read again between yesterday and today
>
>
> Thank you,
>
> Nardeep
>
-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Confluence wiki edit permission

[Andrew Psaltis]

Thanks Joe.

On Fri, Jul 7, 2017 at 11:38 PM, Joe Witt <joe.w...@gmail.com> wrote:

> Andrew
>
> You should now have permissions to edit the wiki.
>
> Thanks
> Joe
>
> On Fri, Jul 7, 2017 at 11:34 PM, Andrew Psaltis
> <psaltis.and...@gmail.com> wrote:
> > Hi,
> > I am interested in fixing an issue and adding more content in the
> > contributor guide on the wiki. One change is pretty simple an invalid
> URL.
> > The additional content I am thinking of at this time is around this
> process
> > in particular. I searched around trying to understand the process of how
> to
> > contribute wiki changes and the only references I can find were several
> > year old emails from this mailing list of folks asking to be given edit
> > permissions to contribute changes.
> >
> > Am I approaching this correctly? If not can you please point me in the
> > correct direction?
> >
> > --
> > Thanks,
> > Andrew
>



-- 
Thanks,
Andrew

Confluence wiki edit permission

[Andrew Psaltis]

Hi,
I am interested in fixing an issue and adding more content in the
contributor guide on the wiki. One change is pretty simple an invalid URL.
The additional content I am thinking of at this time is around this process
in particular. I searched around trying to understand the process of how to
contribute wiki changes and the only references I can find were several
year old emails from this mailing list of folks asking to be given edit
permissions to contribute changes.

Am I approaching this correctly? If not can you please point me in the
correct direction?

-- 
Thanks,
Andrew

Re: [DRAFT][REPORT] Apache NiFi - July 2017

[Andrew Psaltis]

ollows
> > > > >ASF security team published processes effectively.
> > > > >
> > > > > ## PMC changes:
> > > > >
> > > > >  - Currently 22 PMC members.
> > > > >  - New PMC members:
> > > > > - Koji Kawamura was added to the PMC on Fri May 26 2017
> > > > > - Pierre Villard was added to the PMC on Sun May 21 2017
> > > > > - Yolanda Davis was added to the PMC on Wed May 17 2017
> > > > >
> > > > > ## Committer base changes:
> > > > >
> > > > >  - Currently 34 committers.
> > > > >  - New commmitters:
> > > > > - Marc Parisi was added as a committer on Thu Jun 01 2017
> > > > > - Andrew M. Lim was added as a committer on Wed May 31 2017
> > > > >
> > > > > ## Releases:
> > > > >
> > > > >  - Apache NiFi 1.3.0 was released June 9 2017.
> > > > >  - Apache NiFi 0.7.4 was released June 9 2017.
> > > > >  - Apache NiFi MiNiFi (java) 0.2.0 was released May 19 2017.
> > > > >  - Apache NiFi 0.7.3 was released May 19 2017.
> > > > >  - Apache NiFi MiNiFi (cpp) 0.2.0 was released May 14 2017.
> > > > >  - Apache NiFi 1.2.0 was released May 10 2017.
> > > > >
> > > > > ## Mailing list activity:
> > > > >
> > > > >  - Activity on the mailing lists remains extremely high with a
> > mixture
> > > > >of new users, contributors, and deeper more experienced users
> and
> > > > >contributors sparking discussion and questions and filing bugs
> or
> > > > >new features.
> > > > >
> > > > >  - us...@nifi.apache.org:
> > > > > - 512 subscribers (up 39 in the last 3 months):
> > > > > - 936 emails sent to list (846 in previous quarter)
> > > > >
> > > > >  - dev@nifi.apache.org:
> > > > > - 367 subscribers (up 24 in the last 3 months):
> > > > > - 1018 emails sent to list (925 in previous quarter)
> > > > >
> > > > >  - iss...@nifi.apache.org:
> > > > > - 42 subscribers (up 4 in the last 3 months):
> > > > > - 7936 emails sent to list (6592 in previous quarter)
> > > > >
> > > > >
> > > > > ## JIRA activity:
> > > > >
> > > > >  - 573 JIRA tickets created in the last 3 months
> > > > >  - 421 JIRA tickets closed/resolved in the last 3 months
> > > > >
> > > >
> > >
> >
>
-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: AvroSchemaRegistry doesn't enjoy copy and paste?

[Andrew Psaltis]

Andre,
Thanks for the clarification. I missed the salient point of "NiFi simply
failed to validate the second schema (with
nested records)"



On Fri, Jun 23, 2017 at 3:41 AM, Andre <andre-li...@fucs.org> wrote:

> Andrew,
>
> Your are correct The JSON's must be different... but I have not even gone
> to that level yet! NiFi simply failed to validate the second schema (with
> nested records)
>
> The thing that was not clear to my is why the Avro Registry Service would
> not validate the second schema, event after minifying it and confirming
> both single quotes and double quotes are pure ASCII characters.
>
> It turned out to be the issue is fixed by NIFI-4029
>
> Are we targeting releasing 1.3.1 by any chance? 
>
> Cheers
>
>
> PS-I wasn't familiar with avro-tools random, so thank for that! 
>
>
> On Fri, Jun 23, 2017 at 2:57 PM, Andrew Psaltis <psaltis.and...@gmail.com>
> wrote:
>
> > Andre,
> > I may be totally off-base here as it relates to the record related
> > processing in NiFi. From a pure Avro viewpoint the same JSON will not
> work
> > between those two schemas that you provided. The second one has a nested
> > header record and the first does not. I used the java AVRO tools to do
> the
> > following:
> >
> >
> >1. Generate sample Avro for schema 1
> >
> >  java -jar avro-tools-1.7.7.jar random --count 1 --schema-file
> schema1.asvc
> > test1.avro
> >
> >
> > 2.  Convert binary Avro to JSON
> >
> >
> > java -jar avro-tools-1.7.7.jar tojson  test1.avro
> >
> >
> > This produces JSON like such:
> >
> >
> > {"version":297340384,"deviceVendor":null}
> >
> >
> > 3. Generate sample Avro for schema 2
> >
> > java -jar avro-tools-1.7.7.jar random --count 1 --schema-file
> schema2.asvc
> > test2.avro
> >
> >
> > 4. Convert binary Avro to JSON
> >
> >
> > java -jar avro-tools-1.7.7.jar tojson  test2.avro
> >
> >
> > This produces JSON like such:
> >
> >
> > {"header":{"version":-492928400,"deviceVendor":{"float":0.59934044}}}
> >
> >
> > To then have valid JSON with the second schema, you would represent it
> as:
> >
> > {"header":{"version":-492928400,"deviceVendor":null}}
> >
> >
> > If you try and generate Avro with JSON from step 2 above
> > ({"version":297340384,"deviceVendor":null}) using the second schema:
> >
> >
> > java -jar avro-tools-1.7.7.jar fromjson --schema-file schema2.asvc
> > test1.json
> >
> > It will throw an exception:
> >
> >
> > avro.codenull3+�MW���o���c��Exception in thread "main"
> > org.apache.avro.AvroTypeException: Expected field name not found: header
> >
> >
> > Hopefully this did not totally muddy the water.
> >
> > Thanks,
> > Andrew
> >
> >
> >
> > On Thu, Jun 22, 2017 at 11:00 PM, Andre <andre-li...@fucs.org> wrote:
> >
> > > All,
> > >
> > > So it turns out the reason for the validation failure seems to be
> arising
> > > here:
> > >
> > > https://github.com/apache/nifi/blob/b1901d5fe0bf87be3dcce144f13b74
> > > eb995be168/nifi-commons/nifi-record/src/main/java/org/
> > > apache/nifi/serialization/record/RecordField.java#L44
> > >
> > > When using the following schema:
> > >
> > > {
> > > "type":"record",
> > > "name":"header",
> > > "fields":[
> > >   {
> > > "name":"version",
> > > "type":"int",
> > > "doc":"The CEF version extracted from (CEF:0) where 0 is the
> > > version"
> > >   },
> > >   {
> > > "name":"deviceVendor",
> > > "type":[
> > >   "null",
> > >   "float"
> > > ],
> > > "default":null
> > >   }
> > > ]
> > >
> > > }
> > >
> > > The float field with default Null will parse correctly.
> > >
> > > However
> > >
> > > When using this:
> > >
> > > {
> > >   "name":"nifiCommonEventFormat",
> > >   "namespace&qu

Re: AvroSchemaRegistry doesn't enjoy copy and paste?

[Andrew Psaltis]

is. Any chance you’re using Safari?
>>>
>>> > On Jun 22, 2017, at 8:53 AM, Andrew Grande <apere...@gmail.com> wrote:
>>> >
>>> > Definitely something is auto replacing quotes, I can confirm pasting
>>> worked
>>> > fine before from a programmer's editor.
>>> >
>>> > Andrew
>>> >
>>> > On Thu, Jun 22, 2017, 9:06 AM Mark Payne <marka...@hotmail.com> wrote:
>>> >
>>> >> Andre,
>>> >>
>>> >> I've not seen this personally. I just clicked on the link you sent,
>>> copied
>>> >> the schema,
>>> >> and pasted it in, and it did not have any problems. What application
>>> are
>>> >> you copying
>>> >> the text from? I've certainly seen that some applications
>>> (specifically
>>> >> Microsoft Outlook
>>> >> and Office) love to take double-quotes and change them into other
>>> >> characters so that
>>> >> they look nicer. But if you then copy that and paste it, it is not
>>> pasting
>>> >> a double-quote but
>>> >> some other unicode character.
>>> >>
>>> >> Would recommend you open the below link in Chrome and copy from there
>>> and
>>> >> see if
>>> >> that works?
>>> >>
>>> >> Thanks
>>> >> -Mark
>>> >>
>>> >>
>>> >>
>>> >>> On Jun 22, 2017, at 8:56 AM, Andre <andre-li...@fucs.org> wrote:
>>> >>>
>>> >>> All,
>>> >>>
>>> >>> I was playing with the AvroSchemaRegistry and noticed it seems to not
>>> >> play
>>> >>> ball when the DFM pastes the schema into the dynamic property value.
>>> >>>
>>> >>> To test it I basically copied the demo schema from Mark's blog
>>> post[1]
>>> >> and
>>> >>> pasted into a NiFi 1.3.0 instance. To my surprise the controller
>>> would
>>> >> not
>>> >>> validate, instead it displayed:
>>> >>>
>>> >>> "was expecting double-quote to start field name"
>>> >>>
>>> >>> I also faced similar errors using the following schema:
>>> >>>
>>> >>>
>>> >> https://github.com/fluenda/SecuritySchemas/blob/master/CEFRe
>>> v23/cefRev23_nifi.avsc
>>> >>>
>>> >>> Has anyone else seen this?
>>> >>>
>>> >>> Cheers
>>> >>
>>> >>
>>>
>>>
>>
>


-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [ANNOUNCE] New Apache NiFi Committer Drew Lim

[Andrew Psaltis]

Congrats Drew! I spend a lot of time teaching people about NiFi and can
honestly say the documentation -- built-in and on the wiki is a great
selling point. Thanks for all the hard work.

On Wed, May 31, 2017 at 3:08 PM, Tony Kurc <tk...@apache.org> wrote:

> On behalf of the Apache NiFI PMC, I am very pleased to announce that Drew
> Lim has accepted the PMC's invitation to become a committer on the Apache
> NiFi project. We greatly appreciate all of Drew's hard work and generous
> contributions to the project. We look forward to his continued involvement
> in the project.
>
> In my opinion, one of the strengths of NiFi is that someone can launch the
> UI, do some light (often built-in) reading, and get started "making data
> flow" in a short amount of time. Drew has been instrumental in making this
> happen, over the past year contributing to the wiki and making the built-in
> documentation correct and approachable. We all know the Apache adage
> "Community > Code", and Drew's contributions are key to ensuring the health
> of the community.
>
> Welcome and congratulations!
>
> --Tony
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [VOTE] Release Apache NiFi MiNiFi 0.2.0

[Andrew Psaltis]

+1 (non-binding)


Verified signatures
Verified hashes
Built cleanly using contrib check
Built simple flow to communicate with NiFi 1.2

converted flow with toolkit
ran flow

Verified flowStatus JSON



On Wed, May 17, 2017 at 4:18 PM, Tony Kurc <tk...@apache.org> wrote:

> +1 (binding)
>
> - verified signatures
> - verified hashes
> - build cleanly
> - verified a simple flow
> - I think c2 worked as expected
> - ran toolkit's config.sh and did some converting without issue
>
>
> On Mon, May 15, 2017 at 12:55 PM, Aldrin Piri <ald...@apache.org> wrote:
>
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > MiNiFi, minifi-0.2.0.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1107
> >
> > The Git tag is minifi-0.2.0-RC1
> > The Git commit ID is 1813e87ccddffcc9ab75d25948c31901faf2f1ab
> > * https://git-wip-us.apache.org/repos/asf?p=nifi-minifi.git;a=commit;h=
> > 1813e87ccddffcc9ab75d25948c31901faf2f1ab
> > * https://github.com/apache/nifi-minifi/commit/
> > 1813e87ccddffcc9ab75d25948c31901faf2f1ab
> >
> > Checksums of minifi-0.2.0-source-release.zip:
> > MD5:  e0f9bbd7afeb9547126a710b3e30face
> > SHA1:  bcba77ccd4254de3275ecb2a732005e6b404706d
> > SHA256:  c182d71ba298a41a336da139f435f10a253997d677fc4ae8820b261ea8c3
> a4c2
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/aldrin.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 26 issues were closed/resolved for this release:
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > version=12338791=12319921
> >
> > Release note highlights can be found here:
> > https://cwiki.apache.org/confluence/display/MINIFI/
> > Release+Notes#ReleaseNotes-Version0.2.0
> >
> > The vote will be open for at least 72 hours and close at 5PM EDT, 18 May
> > 2017 [1].
> >
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build
> > from source, and test.  Then please vote:
> >
> > [ ] +1 Release this package as minifi-0.2.0
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
> > [1] You can determine this time for your local time zone at
> > https://s.apache.org/minifi-0.2.0-rc1-close
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Pulling out hair over slf4j...

[Andrew Psaltis]

r:4.1.2:compile
> [INFO] |  \- org.apache.httpcomponents:httpmime:jar:4.1.2:compile
> [INFO] +- com.perfectsearchcorp:psjbase:jar:195:compile
> [INFO] |  +- org.fusesource:sigar:jar:1.6.4:compile
> [INFO] |  |  \- log4j:log4j:jar:1.2.15:compile
> [INFO] |  +- jaxen:jaxen:jar:1.1.4:compile
> [INFO] |  \- org.databene:contiperf:jar:1.06:compile
> [INFO] +- org.apache.nifi:nifi-api:jar:1.1.2:compile
> [INFO] +- org.apache.nifi:nifi-utils:jar:1.1.2:compile
> [INFO] +- org.apache.nifi:nifi-processor-utils:jar:1.1.2:compile
> [INFO] |  \- org.apache.nifi:nifi-security-utils:jar:1.1.2:compile
> [INFO] | +- org.bouncycastle:bcprov-jdk15on:ja

Re: [VOTE] Release Apache NiFi 1.2.0 (RC2)

[Andrew Psaltis]

Cool -- not sure if it is worth calling that out in next Release Guideline,
as it may trip up others or most likely myself again :)

On Mon, May 8, 2017 at 9:54 AM, Joe Witt <joe.w...@gmail.com> wrote:

> Rat check has never worked on windows and is flakey in parallel builds at
> best.
>
> On May 8, 2017 9:05 AM, "Andrew Psaltis" <psaltis.and...@gmail.com> wrote:
>
> > +0 (non-binding) -- failed building on Windows 10 using mvn clean install
> > -Pcontrib-check -DskipTests (skipped test as build with tests failed)
> >
> > Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426;
> > 2017-04-03T15:39:06-04:00)
> > Maven home: c:\tools\apache-maven-3.5.0\bin\..
> > Java version: 1.8.0_131, vendor: Oracle Corporation
> > Java home: C:\Program Files\Java\jdk1.8.0_131\jre
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Build failed  with RAT check against nifi-poi-processors. Not sure if
> this
> > is expected or not. Details of rat.txt output are:
> >
> >
> > *
> > Summary
> > ---
> > Generated at: 2017-05-08T08:22:12-04:00
> > Notes: 0
> > Binaries: 2
> > Archives: 0
> > Standards: 6
> >
> > Apache Licensed: 5
> > Generated Documents: 0
> >
> > JavaDocs are generated and so license header is optional
> > Generated files do not required license headers
> >
> > 1 Unknown Licenses
> >
> > ***
> >
> > Unapproved licenses:
> >
> >
> > C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/nifi-
> > poi-bundle/nifi-poi-processors/src/test/resources/Unsupported.xls
> >
> > ***
> >
> > Archives:
> >
> > *
> >   Files with Apache License headers will be marked AL
> >   Binary files (which do not require AL headers) will be marked B
> >   Compressed archives will be marked A
> >   Notices, licenses etc will be marked N
> >   AL
> >  C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/nifi-
> > poi-bundle/nifi-poi-processors/pom.xml
> >   AL
> >  C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/nifi-
> > poi-bundle/nifi-poi-processors/src/main/java/org/
> > apache/nifi/processors/poi/ConvertExcelToCSVProcessor.java
> >   AL
> >  C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/nifi-
> > poi-bundle/nifi-poi-processors/src/main/resources/
> > META-INF/services/org.apache.nifi.processor.Processor
> >   AL
> >  C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/nifi-
> > poi-bundle/nifi-poi-processors/src/test/java/org/
> > apache/nifi/processors/poi/ConvertExcelToCSVProcessorTest.java
> >   B
> > C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/nifi-
> > poi-bundle/nifi-poi-processors/src/test/resources/CollegeScorecard.xlsx
> >   AL
> >  C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/nifi-
> > poi-bundle/nifi-poi-processors/src/test/resources/logback-test.xml
> >   B
> > C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/nifi-
> > poi-bundle/nifi-poi-processors/src/test/resources/TwoSheets.xlsx
> >  !?
> > C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/nifi-
> > poi-bundle/nifi-poi-processors/src/test/resources/Unsupported.xls
> >
> > *
> >  Printing headers for files without AL header...
> >
> >
> > ===
> > ==C:/Users/apsal/Downloads/nifi-1.2.0/nifi-nar-bundles/
> > nifi-poi-bundle/nifi-poi-processors/src/test/resources/Unsupported.xls
> > ===
> > ÐÏ?ࡱ?á>???þÿ
> > ?1???þÿÿÿþÿÿÿ0???
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > ???£DÍ?Á???á???°?Á?â???\?pjames
> >
> >
> >  B???°?a?À???=?œ?

Re: [VOTE] Release Apache NiFi 1.2.0 (RC2)

[Andrew Psaltis]

 !unreadableKeyProviderIsValid
> > - Built RPM with `mvn -T 4.0C clean install -Prpm,generateArchives
> -DskipTests` and tested RPM install
> > - Tested GenerateTableFetch and QueryDatabaseTable with MS SQL 2008
> [NIFI-3585], among other flows.
> >
> >> On May 5, 2017, at 9:07 PM, Bryan Bende <bbe...@apache.org> wrote:
> >>
> >> Hello,
> >>
> >> I am pleased to be calling this vote for the source release of Apache
> >> NiFi nifi-1.2.0 (RC2).
> >>
> >> The source zip, including signatures, digests, etc. can be found at:
> >> https://repository.apache.org/content/repositories/orgapachenifi-1104
> >>
> >> The Git tag is nifi-1.2.0-RC2
> >> The Git commit ID is 3a605af8e0ac024fb0ba67262d49dab2727b2576
> >> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
> 3a605af8e0ac024fb0ba67262d49dab2727b2576
> >>
> >> Checksums of nifi-1.2.0-source-release.zip:
> >> MD5: 90e298a9e23a9dab65358daddd8b5990
> >> SHA1: e138941f576bdb1dff17df6674c19ffae3ef6719
> >> SHA256: c18398800c435dabff9f45ec55a450ca78c3c1aec222aa295c0e2057912f
> eeb6
> >>
> >> Release artifacts are signed with the following key:
> >> https://people.apache.org/keys/committer/bbende.asc
> >>
> >> KEYS file available here:
> >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> >>
> >> 381 issues were closed/resolved for this release:
> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12316020=12338432
> >>
> >> Release note highlights can be found here:
> >> https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-Version1.2.0
> >>
> >> The vote will be open for 72 hours.
> >> Please download the release candidate and evaluate the necessary items
> >> including checking hashes, signatures, build
> >> from source, and test.  The please vote:
> >>
> >> [ ] +1 Release this package as nifi-1.2.0
> >> [ ] +0 no opinion
> >> [ ] -1 Do not release this package because because...
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [ANNOUNCE] New Apache NiFi PMC Member - Oleg Zhurakousky

[Andrew Psaltis]

Congrats Oleg! Very well deserved!

On Tue, Apr 4, 2017 at 5:16 PM, Jeff <jtsw...@gmail.com> wrote:

> Congrats, Oleg!
>
> On Tue, Apr 4, 2017 at 6:33 PM Andy LoPresto <alopre...@apache.org> wrote:
>
> > Congratulations Oleg. Very excited for your continued excellent work on
> > the project and help for everyone on the lists.
> >
> > Andy LoPresto
> > alopre...@apache.org
> > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>*
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > On Apr 4, 2017, at 3:22 PM, Joe Witt <joe.w...@gmail.com> wrote:
> >
> > Oleg
> >
> > Thanks for all you've done and congrats on joining the PMC!
> >
> > Thanks
> > Joe
> >
> > On Apr 4, 2017 3:51 PM, "Tony Kurc" <tk...@apache.org> wrote:
> >
> > Apache NiFi Community,
> >
> > On behalf of the Apache NiFi PMC, I am pleased to announce that Oleg
> > Zhurakousky has accepted the PMC's invitation to join the Apache NiFi
> PMC.
> > We greatly appreciate all of Oleg’s hard work and generous contributions
> to
> > the project. We look forward to his continued involvement in the project.
> >
> > Oleg started out contributing in late 2015, diving in deep on framework
> > issues and contributing exciting new capabilities. I had the pleasure of
> > reviewing some of those early contributions and always found the dialog
> > with Oleg on reviews and ideas to be fun and stimulating. Since accepting
> > the PMC’s invitation to become a committer, he’s maintained a steady hand
> > in reviewing new contributions, staying active contributing code and on
> the
> > mailing lists, and becoming an ever more valuable member of our
> community.
> >
> >
> > Please join us in congratulating and welcoming Oleg to the Apache NiFi
> PMC.
> >
> > Congratulations and welcome, Oleg!
> >
> >
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [ANNOUNCE] New Apache NiFi Committer Bin Qiu

[Andrew Psaltis]

Congrats Bin!

On Tue, Apr 4, 2017 at 4:17 PM, Tony Kurc <tk...@apache.org> wrote:

> Apache NiFi community,
>
>
> On behalf of the Apache NiFI PMC, I am very pleased to announce that Bin
> Qiu has accepted the PMC's invitation to become a committer on the Apache
> NiFi project. We greatly appreciate all of Bin’s hard work and generous
> contributions and look forward to continued involvement in the project.
>
> Bin has been one of the driving forces behind MiNiFi - C++, contributing a
> massive amount in terms of code and helping with release verification and
> always being responsive to the community.
>
>
> Congrats, Bin!
>
>
> - Tony
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Nifi on Raspberry pi - jetty issues

[Andrew Psaltis]

Pushkar,
I just realized after sending the email, that I did not provide any
resources where you can learn more about MiNiFi outside of the Apache site.
Here are some resources that may make your experience a little smoother:

Edge Intelligence with MiNiFi --
https://hortonworks.com/webinar/edge-intelligence-iot-apache-minifi/
Reading Sense HAT data -
https://community.hortonworks.com/articles/85984/using-minifi-to-read-data-from-a-sense-hat-on-a-ra.html
NiFi on a Rasp. PI Best Practices -
https://community.hortonworks.com/articles/32605/running-nifi-on-raspberry-pi-best-practices.html
-- this does not cover MiNiFi, thus it may have some pointers for solving
the current problem you are having.

Hope that helps.

On Fri, Mar 3, 2017 at 3:27 PM, Andrew Psaltis <psaltis.and...@gmail.com>
wrote:

> Pushkar,
> While it certainly should and has been possible to run regular NiFi on a
> PI. Perhaps the best way forward is to actually use MiNiFi [1]. MiNiFi is
> available in as a C++ and Java agent and designed explicitly to run on
> devices.
>
> I'm sorry I could not be of more assistance with the actually config
> changes you may want to make, I am traveling at the present time and do not
> have access to a PI.
>
>
> [1] https://nifi.apache.org/minifi/index.html
>
> Thanks,
> Andrew
>
> On Fri, Mar 3, 2017 at 3:21 PM, Pushkara R <pushkar1...@gmail.com> wrote:
>
>> Hi,
>>
>> I'm getting NiFi to run on a raspberry pi. the bootstrap takes a very long
>> time. But at the end, the port doesn't open because of a jetty issue and
>> NiFi shuts down. I've pasted the exception below.
>>
>> A quick look at stackoverflow tells me I need to insrease the timeout
>> value
>> to go around this. But I do not know how to change it, I'm not sure if it
>> is picking it up at runtime or not.
>>
>> Any assistance would be helpful.
>>
>> Thanks
>> Pushkar
>>
>> PS : the stacktrace
>>
>> java.lang.Exception: Timeout scanning annotations
>> at
>> org.eclipse.jetty.annotations.AnnotationConfiguration.scanFo
>> rAnnotations(AnnotationConfiguration.java:575)
>> ~[jetty-annotations-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.annotations.AnnotationConfiguration.config
>> ure(AnnotationConfiguration.java:447)
>> ~[jetty-annotations-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.webapp.WebAppContext.configure(WebAppContext.java:494)
>> ~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppCo
>> ntext.java:1361)
>> ~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doStart(
>> ContextHandler.java:772)
>> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.servlet.ServletContextHandler.doStart(Serv
>> letContextHandler.java:262)
>> ~[jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520)
>> ~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(
>> AbstractLifeCycle.java:68)
>> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(
>> ContainerLifeCycle.java:132)
>> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(
>> ContainerLifeCycle.java:114)
>> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(
>> AbstractHandler.java:61)
>> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(
>> AbstractLifeCycle.java:68)
>> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(
>> ContainerLifeCycle.java:132)
>> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(
>> ContainerLifeCycle.java:106)
>> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(
>> AbstractHandler.java:61)
>> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
>> at
>> org.eclipse.jetty.server.handler.gzip.Gz

Re: Nifi on Raspberry pi - jetty issues

[Andrew Psaltis]

rt(JettyServer.java:675)
> ~[nifi-jetty-1.1.2.jar:1.1.2]
> at org.apache.nifi.NiFi.(NiFi.java:156)
> [nifi-runtime-1.1.2.jar:1.1.2]
> at org.apache.nifi.NiFi.main(NiFi.java:262)
> [nifi-runtime-1.1.2.jar:1.1.2]
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Need help with Custom Processor

[Andrew Psaltis]

Hi Pushkar,
By chance is the "A" dependency marked with a scope in "B"'s pom, such as
test, compile, or provided? If so then you would want to include "A" in
your pom as a dependency. As long as "A" is not marked with a scope that
excludes it from "B"'s final packaging or it is not excluded by one of the
maven packaging plugins then it should pick up the dependency and include
it in your NAR when you package your processor. If "A" is indeed marked
with a scope that prevents it from being included in the final NAR, you
would either want to modify B's pom or explicitly add it as a dependency to
your pom file.

Thanks,
Andrew

On Tue, Feb 28, 2017 at 2:56 PM, Pushkara R <pushkar1...@gmail.com> wrote:

> Hi Andrew,
>
> The particular class that isn't being found is part of an artifact (let's
> call it A) which is a dependency of another artifact B that my processors
> depend on. A is in B's pom file and B is in my processors' pom file. Yet I
> am getting the class not found issue.
>
> I am currently working by adding A's jar in the lib folder as I am only
> doing a few benchmarks on NiFi. But please do let me know how I could fix
> this problem.
>
> Regards
> Pushkar
>
> On Tue, Feb 28, 2017 at 12:40 PM, Andrew Psaltis <psaltis.and...@gmail.com
> >
> wrote:
>
> > Hi Pushkar,
> > The preferred approach is to add the external dependency to the pom file
> > for your processor. After doing that when you build your nar bundle, the
> > resulting nar file will contain all of the 3rd party dependencies
> required
> > by your processor. Can you double check that you are indeed including it
> as
> > a maven dependency?
> >
> > Thanks,
> > Andrew
> >
> >
> > On Tue, Feb 28, 2017 at 12:31 PM, Pushkara R <pushkar1...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I'm writing a new processor for NiFi which has a dependency on an
> > external
> > > library. When I build the NAR and add it to the lib folder and restart
> > > NiFi, the external library classes that my processor depends on fail
> > > getting loaded with a "ClassNotFound" exception.
> > >
> > > How do I go about fixing this issue? Do I add the jar for the external
> > > library separately in the lib folder of NiFi? Or is there something
> else
> > to
> > > it?
> > >
> > > Thanks
> > > Pushkar
> > >
> >
> >
> >
> > --
> > Thanks,
> > Andrew
> >
> > Subscribe to my book: Streaming Data <http://manning.com/psaltis>
> > <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
> > twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Need help with Custom Processor

[Andrew Psaltis]

Hi Pushkar,
The preferred approach is to add the external dependency to the pom file
for your processor. After doing that when you build your nar bundle, the
resulting nar file will contain all of the 3rd party dependencies required
by your processor. Can you double check that you are indeed including it as
a maven dependency?

Thanks,
Andrew


On Tue, Feb 28, 2017 at 12:31 PM, Pushkara R <pushkar1...@gmail.com> wrote:

> Hi,
>
> I'm writing a new processor for NiFi which has a dependency on an external
> library. When I build the NAR and add it to the lib folder and restart
> NiFi, the external library classes that my processor depends on fail
> getting loaded with a "ClassNotFound" exception.
>
> How do I go about fixing this issue? Do I add the jar for the external
> library separately in the lib folder of NiFi? Or is there something else to
> it?
>
> Thanks
> Pushkar
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [VOTE] Establish Registry, a sub-project of Apache NiFi

[Andrew Psaltis]

+1 non-binding This will be an awesome addition.  Looking forward to
helping out.


On Fri, Feb 10, 2017 at 17:26 u...@moosheimer.com <u...@moosheimer.com> wrote:

> +1 (non-binding)
>
> Uwe
>
> > Am 10.02.2017 um 22:18 schrieb Koji Kawamura <ijokaruma...@apache.org>:
> >
> > +1 (non-binding)
> >
> > On Feb 11, 2017 5:37 AM, "Jennifer Barnabee" <
> jennifer.barna...@gmail.com>
> > wrote:
> >
> > +1 binding
> >
> > Sent from my iPhone
> >
> >> On Feb 10, 2017, at 2:55 PM, Joe Skora <jsk...@gmail.com> wrote:
> >>
> >> +1 binding
> >>
> >> On Fri, Feb 10, 2017 at 2:09 PM, Peter Wicks (pwicks) <
> pwi...@micron.com>
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>>
> >>> -Original Message-
> >>> From: Bryan Bende [mailto:bbe...@gmail.com]
> >>> Sent: Friday, February 10, 2017 9:41 AM
> >>> To: dev@nifi.apache.org
> >>> Subject: [VOTE] Establish Registry, a sub-project of Apache NiFi
> >>>
> >>> All,
> >>>
> >>> Following a solid discussion for the past few days [1] regarding the
> >>> establishment of Registry as a sub-project of Apache NiFi, I'd like to
> >>> call a formal vote to record this important community decision and
> >>> establish consensus.
> >>>
> >>> The scope of this project is to define APIs for interacting with
> >>> resources that one or more NiFi instances may be interested in, such
> >>> as a flow registry for versioned flows, an extension registry for
> >>> extensions, and possibly other configuration resources in the future.
> >>> In addition, this project will provide reference implementations of
> >>> these registries, with the goal of allowing the community to build a
> >>> diverse set of implementations, such as a Git provider for versioned
> >>> flows, or a bintray provider for an extension registry.
> >>>
> >>> I am a +1 and looking forward to the future work in this area.
> >>>
> >>> The vote will be open for 72 hours and be a majority rule vote.
> >>>
> >>> [ ] +1 Establish Registry, a subproject of Apache NiFi
> >>> [ ]   0 Do not care
> >>> [ ]  -1 Do not establish Registry, a subproject of Apache NiFi
> >>>
> >>> Thanks,
> >>>
> >>> Bryan
> >>>
> >>> [1] http://mail-archives.apache.org/mod_mbox/nifi-dev/201702.
> > mbox/%3CCALo_
> >>> M19euo2LLy0PVWmE70FzeLhQRcCtX6TC%3DqoiBVfn4zFQMA%40mail.gmail.com%3E
> >>>
>
-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [ANNOUNCE] New Apache NiFi Committer Joey Frazee

[Andrew Psaltis]

Congrats Joey!

On Tue, Jan 3, 2017 at 3:10 PM, Oleg Zhurakousky <
ozhurakou...@hortonworks.com> wrote:

> Congrats Joey!
>
> Nice to have you on board!
> Oleg
>
> On Jan 3, 2017, at 2:42 PM, Aldrin Piri <ald...@apache.org<mailto:aldr
> i...@apache.org>> wrote:
>
> On behalf of the Apache NiFI PMC, I am very pleased to announce that Joey
> Frazee has accepted the PMC's invitation to become a committer on the
> Apache NiFi project. We greatly appreciate all of Joey's hard work and
> generous contributions and look forward to continued involvement in the
> project.
>
> Joey's contributions including support for HL7, JMS, and EventHub
> extensions.  Joey can also be found assisting with the lists, as well as
> articles and repositories based around the NiFi community.
>
> Congrats, Joey!
>
>


-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [VOTE] Release Apache NiFi 1.1.0 (RC1)

[Andrew Psaltis]

+1 (non-binding)

Ran full clean build w/contrib-check
Built on OSX
Ran a variety of use case importing templates, processing data on OSX
Tested using Chrome and Safari.

Lots of great features -- UI seems snappier

Thanks

On Thu, Nov 24, 2016 at 2:04 AM, Joe Witt <joew...@apache.org> wrote:

> Hello Apache NiFi Community,
>
> I am pleased to be calling this vote for the source release of Apache NiFi,
> nifi-1.1.0.
>
> The source zip, including signatures, digests, etc. can be found at:
>   https://repository.apache.org/content/repositories/orgapachenifi-1092
>
> The Git tag is nifi-1.1.0-RC1
> The Git commit hash is 1b2b9f136df73a8b0d5d21419b6ebc2206f78393
> * https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
> 1b2b9f136df73a8b0d5d21419b6ebc2206f78393
> * https://github.com/apache/nifi/commit/1b2b9f136df73a8b0d5d21419b6ebc
> 2206f78393
>
> Checksums of nifi-1.1.0-source-release.zip:
> MD5: b7a83412a088634835f6510974282408
> SHA1: c538fedf990250dbd75090d7f03aeb2554802c8d
> SHA256: 39dc521230c9936f9178b9fe34a410b932cf0d0807db4aeba9a1b41051023ec6
>
> Release artifacts are signed with the following key:
>   https://people.apache.org/keys/committer/joewitt
>
> KEYS file available here:
>   https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 265 issues were closed/resolved for this release:
>   https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12316020=12337875
>
> Release note highlights can be found here:
>   https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-Version1.1.0
>
> The vote will be open for 88 hours (a bit longer than usual due to timing).
>
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build from source, and test. Then
> please vote:
>
> [ ] +1 Release this package as nifi-1.1.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
> Thanks!
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Need Information - Apache NiFi

[Andrew Psaltis]

Hi M. Mothilal,
Attached is a template that filters based on a Twitter ID (my id, which I
retrieved from [1]). As I am sure you already know, you need to change the
tokens and keys to match your application. After setting up the
configuration, I went to Twitter and just re-tweeted a tweet and moments
later it appeared in my NiFi flow. The different actions that should cause
data to appear are identified here [2].

Hopefully this helps.



[1] http://gettwitterid.com/?user_name=itmdata=GET+USER+ID
[2] https://dev.twitter.com/streaming/overview/request-parameters#follow

Thanks,
Andrew

On Tue, Nov 8, 2016 at 6:59 PM, Mothilal M <moth...@gmail.com> wrote:

> Hi Bryan Bende,
>
> Greetings. I did try filter endpoint and it certainly needs numeric ID and
> not twitter ID. Processor shows error if twitter ID - so I had converted it
> to respective numeric. Please advice if there is any sample worked on this
> use case - filtered tweets based on specific IDs.
>
> Warm Regards,
> M. Mothilal
>
> On Tue, Nov 8, 2016 at 2:45 PM, Mothilal M <moth...@gmail.com> wrote:
>
> > Hi Joe,
> >
> > Greetings and appreciate your reply. Totally agree with your opinion on
> > comparisons.
> >
> > Look forward for other developers to comment on suggestions of getTwitter
> > and getHTTP processor issues for twitter and facebook integration.
> >
> > Warm Regards,
> > M. Mothilal
> >
> > On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <moth...@gmail.com> wrote:
> >
> >> Greetings Joe,
> >>
> >> Glad to connect with you from your reply in Apache NiFi community. I am
> >> integrating NiFi with social media - twitter and Facebook to achieve
> >> analysis but I am struck at a point. Request your assistance in solving
> the
> >> same.
> >>
> >> getTwitter processor - i am not able to get tweets filtered by twitter
> >> IDs. (IDs to follow field value is not working as expected. I did get
> the
> >> numeric value of twitter IDs and input in the value as comma separated
> but
> >> no results). Please advice as I look to filter my tweets on particular
> IDs
> >> and also by particular location by cities like Toronto, calgary.
> >>
> >> getHTTP processor for facebook integration works fine but i am unable to
> >> fetch data from my personal facebook ID. I tried accessing the Graph API
> >> and it mentions user permissions are not granted but I did provide the
> >> essential permissions in app settings. Please advice if you could help
> out
> >> in setting the facebook app permissions to ensure I can fetch
> user_status,
> >> user_location fields. On success, I would need to venture out few
> options
> >> further.
> >>
> >> General question - How could Apache NiFi be useful in replacing Splunk.
> I
> >> understand most of the current companies have a paid version of Splunk
> and
> >> NiFi is an open-source but please let me know features which are key in
> >> replacing Splunk.
> >>
> >> I appreciate your time and would be more helpful if you could let me
> know
> >> your comfortable timing for a technical call discussion. I am available
> and
> >> reachable on +1(647)641-0910. Look forwardly for your reply.
> >>
> >> Warm Regards,
> >> M. Mothilal
> >>
> >> 
> >> --
> >>
> >> Hello,
> >>
> >> I think as a community we're all quite happy to help address questions
> >> or ideas you might have for Apache NiFi.  Please feel free to ask your
> >> questions here and if they're more from the perspective of a user
> >> please use the us...@nifi.apache.org list.
> >>
> >> For the technical questions it is probably best to align them with
> >> some portion of the overview, administration, user, best practice, or
> >> developer guide so we can use that as a basis for further discussion.
> >> You can find those here: https://nifi.apache.org/docs.html
> >>
> >> Thanks
> >> Joe
> >>
> >>
> >> -- Forwarded message --
> >> From: Mothilal M <moth...@gmail.com>
> >> Date: Tue, Nov 8, 2016 at 1:25 PM
> >> Subject: Need Information - Apache NiFi
> >> To: dev@nifi.apache.org
> >>
> >>
> >> Hi Apache NiFi team,
> >>
> >> Greetings. I did my hands on experience with multiple tutorials around
> >> sentimental analysis integrating social 

Re: [ANNOUNCE] New Apache NiFi Committer Rob Moran

[Andrew Psaltis]

Congrats Rob!! Your UI work is greatly appreciated, looking forward for
your continued contribution.

On Mon, Sep 19, 2016 at 1:08 PM, Tony Kurc <tk...@apache.org> wrote:

> On behalf of the Apache NiFI PMC, I am very pleased to announce that Rob
> Moran has accepted the PMC's invitation to become a committer on the Apache
> NiFi project. We greatly appreciate all of Rob's hard work and generous
> contributions to the project. We look forward to his continued involvement
> in the project.
>
> Among other things, Rob has been responsible for a lot of the look and feel
> of the NiFi user interface, to include the awesome logos!
>
> Welcome and congratulations!
>
> Tony
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [VOTE] Release Apache NiFi MiNiFI C++ 0.0.1 (RC1)

[Andrew Psaltis]

: 'test/Server.cpp'
> > make: [tests] Error 1 (ignored)
> >
> > In minifi.properties (is there a reason this is nifi.version instead of
> > minifi.version?):
> >
> > # Core Properties #
> > nifi.version=0.0.1
> >
> > Andy LoPresto
> > alopre...@apache.org
> > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>*
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > On Aug 29, 2016, at 4:47 PM, Koji Kawamura <ijokaruma...@gmail.com>
> wrote:
> >
> > +1 (non-binding)
> >
> > Downloaded the source archive, verified signature, md5, sha1, and
> > sha256 hashes are all good.
> > LICENSE file looks good
> > Built the source successfully with the steps written in README, on Mac
> > OS X El Captain
> > Used brew to install boost: $ brew install boost
> > Created conf/flow.yml:
> > - copy & paste from README
> > - modified Input Ports id so that it matches with the remote NiFi's
> > input port uuid
> > - enabled Site-to-Site at the remote NiFi
> > Started minifi, created a file under /tmp/getfile dir, then confirmed
> > the file content was sent to the remote NiFi via Site-to-Site
> >
> >
> >
> > On Mon, Aug 29, 2016 at 5:52 AM, Aldrin Piri <ald...@apache.org> wrote:
> >
> > Hello Apache NiFi Community,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > MiNiFi C++, nifi-minifi-cpp-0.0.1.
> >
> > The source archive, signature, and digests can be located at:
> >
> > Source Archive:
> >https://dist.apache.org/repos/dist/dev/nifi/nifi-
> > minifi-cpp/0.0.1/nifi-minifi-cpp-0.0.1-source.tar.gz
> > GPG armored signature:
> >https://dist.apache.org/repos/dist/dev/nifi/nifi-
> > minifi-cpp/0.0.1/nifi-minifi-cpp-0.0.1-source.tar.gz.asc
> > Source MD5:
> >https://dist.apache.org/repos/dist/dev/nifi/nifi-
> > minifi-cpp/0.0.1/nifi-minifi-cpp-0.0.1-source.tar.gz.md5
> > Source SHA1:
> >https://dist.apache.org/repos/dist/dev/nifi/nifi-
> > minifi-cpp/0.0.1/nifi-minifi-cpp-0.0.1-source.tar.gz.sha1
> > Source SHA256:
> >https://dist.apache.org/repos/dist/dev/nifi/nifi-
> > minifi-cpp/0.0.1/nifi-minifi-cpp-0.0.1-source.tar.gz.sha256
> >
> > The Git tag is minifi-cpp-0.0.1-RC1
> > The Git commit hash is 61f0e450ef6604e7028be38f0335e70713ebdea3
> > * https://git-wip-us.apache.org/repos/asf?p=nifi-minifi-cpp.
> > git;a=commit;h=61f0e450ef6604e7028be38f0335e70713ebdea3
> > * https://github.com/apache/nifi-minifi-cpp/commit/
> > 61f0e450ef6604e7028be38f0335e70713ebdea3
> >
> > Checksums of nifi-minifi-cpp-0.0.1-source.tar.gz:
> > MD5: 24121f1ebe3a678d5e2dc84fb8479ef6
> > SHA1: 32c1fb584c9bb484c456cdc4f5dfdd9e91e13364
> > SHA256: c72046d86a9dc34033fa40ad1b2300f1f7915acd369f763e7fc02095d09bb91e
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/aldrin
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 11 issues were closed/resolved for this release:
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > version=12337990=12319921
> >
> > Release note highlights can be found here:
> > https://cwiki.apache.org/confluence/display/MINIFI/
> > Release+Notes#ReleaseNotes-Versioncpp-0.0.1
> >
> > The vote will be open for 72 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [ ] +1 Release this package as nifi-minifi-cpp-0.0.1
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
> >
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: SOAP Service through InvokeHTTP

[Andrew Psaltis]

Hello,
I have written a SOAP processor [1] that you are free to use. It has gotten
a bit of testing under it's belt, however, it can always use more. I am
totally open to fixing issues you may find and/or accepting PR's. If it
makes enough sense and is complete enough I have no problem contributing
it.

Thanks,
Andrew

[1] https://github.com/apsaltis/nifi-soap

On Wed, Aug 24, 2016 at 7:40 PM, Jeff <jtsw...@gmail.com> wrote:

> Hello,
>
> With InvokeHTTP, you can add dynamic properties, which will be sent in the
> request as headers.  You can use dynamic properties to set values for the
> Content-Type and SOAPAction headers, just use the header names for the
> names of the dynamic properties.  InvokeHTTP lets you control the HTTP
> method, so you can set that to POST.  The remaining step would be to get
> the content of request.xml to be sent to the InvokeHTTP as a flowfile.  One
> way to do this is to use a GetFile processor to fetch requeset.xml from
> some location on the filesystem, and pass the success relationship of
> GetFile to InvokeHTTP.
>
> On Wed, Aug 24, 2016 at 7:06 PM mliem90 <matt.l...@gmail.com> wrote:
>
> > Hello,
> >
> > I am looking to do a SOAP Web service call and was told invokeHTTP may be
> > able to work. Wonder how I would be able to do this. For example, using
> > curl
> > I could invoke SOAP like the below:
> >
> > curl -X POST -H "Content-Type: text/xml" -H "SOAPAction:
> > "http://www.webserviceX.NET/GetCitiesByCountry"; --data-binary
> > @request.xml
> > http://www.webservicex.net/globalweather.asmx
> >
> > the request.xml:
> > http://www.w3.org/2003/05/soap-envelope;
> > xmlns:web="http://www.webserviceX.NET;>
> >
> >
> >   
> >
> >  CANADA
> >   
> >
> > 
> >
> > I Appreciate the help!
> >
> >
> >
> > --
> > View this message in context:
> > http://apache-nifi-developer-list.39713.n7.nabble.com/SOAP-
> Service-through-InvokeHTTP-tp13129.html
> > Sent from the Apache NiFi Developer List mailing list archive at
> > Nabble.com.
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Max Kafka message size

[Andrew Psaltis]

ledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
>
> This occurs using PublishKafka, and PutKafka.  Setting the Max Record Size
> property in the PutKafka processor has no affect on this.  Note the stack
> trace above is from the PutKafka processor with Max Record Size set to 10MB.
>
> I believe that this a regression from 0.6.0.
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
> On 8/20/16, 3:48 PM, "Andrew Psaltis" <psaltis.and...@gmail.com> wrote:
>
> Hi Chris,
> Regarding the PutKafka processor looking at this block[1] of the
> PutKafka
> code, it has a default size of 1 MB, but it does not restrict the
> size. The
> DATA_SIZE_VALIDATOR does a sanity check and also enforces that
> the supported value entered is the correct format  [B|
> KB|MB|GB|TB].
> Later on in the code at this block[2], the value is set on the Kafka
> config, again this does not enforce a value maximum.
>
> In regards to the PublishKafka processor I do not see where it accepts
> a
> size nor restrict the size at all.
>
> Have you adjusted the 'message.max.bytes' config value for your
> broker(s)?
> The default value for that is 1 MB [3] (The url references the 0.8
> Kafka,
> however I believe this default has been stable since the early days of
> the
> project.)
>
> If you really do want to send messages that are larger than 1 MB in
> size, I
> would highly recommending reading this post[4] from Gwen Shapira.  It
> does
> a great job of outlining the things you need to take into
> consideration.
> This will also point you to the relevant configs in Kafka that will
> need to
> be adjusted if you decide to go this route.
>
>
> Thanks,
> Andrew
>
> [1]
> https://github.com/apache/nifi/blob/rel/nifi-0.7.0/nifi-
> nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/
> main/java/org/apache/nifi/processors/kafka/PutKafka.java#L174-L180
> [2]
> https://github.com/apache/nifi/blob/rel/nifi-0.7.0/nifi-
> nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/
> main/java/org/apache/nifi/processors/kafka/PutKafka.java#L495
> [3] https://kafka.apache.org/08/configuration.html
> [4] http://ingest.tips/2015/01/21/handling-large-messages-kafka/
>
> On Sat, Aug 20, 2016 at 3:25 PM, McDermott, Chris Kevin (MSDU -
> STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote:
>
> > Hi folks,
> >
> >
> >
> > From experimentation and looking at the code it seems that the max
> message
> > size that can be sent via the PublishKafka and PutKafka processors
> in 0.7.0
> > is 1MB.  Can someone please confirm my read on this?
> >
> >
> >
> > Thanks,
> >
> >
> >
> > Chris McDermott
> >
> >
>     >
> > Remote Business Analytics
> >
> > STaTS/StoreFront Remote
> >
> > HPE Storage
> >
> > Hewlett Packard Enterprise
> >
> > Mobile: +1 978-697-5315
> >
> >
> >
> >
>
>
> --
> Thanks,
> Andrew
>
> Subscribe to my book: Streaming Data <http://manning.com/psaltis>
> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
>
>
>


-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Max Kafka message size

[Andrew Psaltis]

Hi Chris,
Regarding the PutKafka processor looking at this block[1] of the PutKafka
code, it has a default size of 1 MB, but it does not restrict the size. The
DATA_SIZE_VALIDATOR does a sanity check and also enforces that
the supported value entered is the correct format  [B| KB|MB|GB|TB].
Later on in the code at this block[2], the value is set on the Kafka
config, again this does not enforce a value maximum.

In regards to the PublishKafka processor I do not see where it accepts a
size nor restrict the size at all.

Have you adjusted the 'message.max.bytes' config value for your broker(s)?
The default value for that is 1 MB [3] (The url references the 0.8 Kafka,
however I believe this default has been stable since the early days of the
project.)

If you really do want to send messages that are larger than 1 MB in size, I
would highly recommending reading this post[4] from Gwen Shapira.  It does
a great job of outlining the things you need to take into consideration.
This will also point you to the relevant configs in Kafka that will need to
be adjusted if you decide to go this route.


Thanks,
Andrew

[1]
https://github.com/apache/nifi/blob/rel/nifi-0.7.0/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/main/java/org/apache/nifi/processors/kafka/PutKafka.java#L174-L180
[2]
https://github.com/apache/nifi/blob/rel/nifi-0.7.0/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/main/java/org/apache/nifi/processors/kafka/PutKafka.java#L495
[3] https://kafka.apache.org/08/configuration.html
[4] http://ingest.tips/2015/01/21/handling-large-messages-kafka/

On Sat, Aug 20, 2016 at 3:25 PM, McDermott, Chris Kevin (MSDU -
STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote:

> Hi folks,
>
>
>
> From experimentation and looking at the code it seems that the max message
> size that can be sent via the PublishKafka and PutKafka processors in 0.7.0
> is 1MB.  Can someone please confirm my read on this?
>
>
>
> Thanks,
>
>
>
> Chris McDermott
>
>
>
> Remote Business Analytics
>
> STaTS/StoreFront Remote
>
> HPE Storage
>
> Hewlett Packard Enterprise
>
> Mobile: +1 978-697-5315
>
>
>
>


-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [ANNOUNCE] Apache NiFi 1.0.0-Beta release

[Andrew Psaltis]

Congrats on a really big step!! Excited to play with the bits.

On Wed, Aug 10, 2016 at 12:07 AM, Joe Percivall <jperciv...@apache.org>
wrote:

>
> Hello
> The Apache NiFi team would like to announce the release of Apache NiFi
> 1.0.0-BETA.
> The upcoming 1.0.0 release will mark the culmination of a lot of work over
> the last few months with many new framework level features being added.
> This Beta release was cut to give our Apache NiFi users a chance to help
> test this upcoming major release. We encourage users to download the
> binary, give the new UI a try, and report any bugs to the Apache NiFi Jira.
> https://issues.apache.org/jira/browse/NIFI/
> Disclaimer: This release is meant for testing and may not be stable in
> terms of features or functionality.
> The release artifacts can be downloaded from here:https://nifi.apache.org/
> download.html
> Maven artifacts have been made available here:https://repository.
> apache.org/content/repositories/releases/org/apache/nifi/
> Issues closed/resolved for this list can be found here:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020;
> version=12338066
> Release note highlights can be found here:https://cwiki.apache.org/
> confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.0.0-Beta
> Thank youThe Apache NiFi team
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [ANNOUNCE] New Apache NiFi Committer Andre Fucs de Miranda

[Andrew Psaltis]

Congrats Andre!

On Sunday, August 7, 2016, Andre <andre-li...@fucs.org> wrote:

> Tony,
>
> I am very honoured by the PMC's invitation but truth be told:
>
> I wouldn't be sending this email if it wasn't for the great assistance
> provided by this community.
>
> So thank you all for your support and assistance (and constructive
> criticism!).
>
> It has been a long journey since I first heard of NiFi and I continue
> looking forward to contribute to the project.
>
> Once again, thank you all for the support along the journey.
>
> On Sun, Aug 7, 2016 at 12:40 PM, Tony Kurc <tk...@apache.org
> <javascript:;>> wrote:
>
> > On behalf of the Apache NiFI PMC, I am very pleased to announce that
> Andre
> > Fucs de Miranda has accepted the PMC's invitation to become a committer
> on
> > the Apache NiFi project. We greatly appreciate all of Andre's hard work
> and
> > generous contributions and look forward to continued involvement in the
> > project.
> >
> > Many of you have already had the pleasure of interacting with him on the
> > mailing lists, pull requests or jiras, and have benefited from his
> > knowledge and contributions on log processing such as the Lumberjack
> > bundle.
> >
> > Welcome Andre!
> >
> > Tony
> >
>


-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [DISCUSS] MiNiFi C++ Initial Release - cpp-0.0.1

[Andrew Psaltis]

+1 a fantastic step forward. Will be great to see how close the the true
edge this can go ;)

On Fri, Aug 5, 2016 at 2:17 AM, Pierre Villard <pierre.villard...@gmail.com>
wrote:

> +1, awesome!
>
> 2016-08-05 6:16 GMT+02:00 Matt Burgess <mattyb...@gmail.com>:
>
> > +1, also we need documentation on the S2S protocol so we can further
> > extend the reach (Go, Python, Erlang/Elixir, etc)
> >
> >
> > > On Aug 4, 2016, at 11:42 PM, Aldrin Piri <aldrinp...@gmail.com> wrote:
> > >
> > > Hey folks,
> > >
> > > There's been great efforts to get an initial working implementation of
> > > MiNiFi in C++.  This currently provides a great start on the core
> > > functionalities that the Java equivalent provides.  Perhaps the neatest
> > of
> > > all these items is the C++ implementation of the Site to Site protocol
> > > allowing transmission from a native instance to the NiFi we have all
> come
> > > to use and love in a more bite-sized binary.
> > >
> > > To that end and to aid in making the MiNiFi effort a bit more
> concrete, I
> > > would like to get a vote started on 0.0.1 (JIRA cpp-0.0.1) in the next
> > few
> > > days and would be happy to act as the RM. There are still a few issues
> to
> > > tackle mostly tailored around documentation
> > >
> > > The hope is that with a Java and C++ implementation in hand there will
> > be a
> > > larger foundation on which people can get involved and participate.
> This
> > > is especially crucial as we start expanding the framework, look to
> > solidify
> > > APIs, and consider protocols to facilitate the design and
> implementation
> > of
> > > centralized management and bi-directional communications.
> > >
> > > There is still much to be done both with our MiNiFi agent
> implementations
> > > and everything that follows.  This marks another nice milestone of
> > progress
> > > and will further generate a multitude of avenues for people to get
> > > involved.
> > >
> > > Looking forward to the new possibilities and challenges ahead.
> > >
> > > --Aldrin
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [ANNOUNCE] New Apache NiFi Committer Yolanda Davis

[Andrew Psaltis]

Congrats Yolanda!!

On Sun, Jul 31, 2016 at 9:58 AM, Pierre Villard <pierre.villard...@gmail.com
> wrote:

> Congrats Yolanda, that's a great news!
>
> 2016-07-30 16:43 GMT+02:00 Andy LoPresto <alopresto.apa...@gmail.com>:
>
> > Congratulations Yolanda. Well-deserved and I am excited for all of your
> > upcoming contributions, both code and to the community.
> >
> > Andy LoPresto
> > alopre...@apache.org
> > alopresto.apa...@gmail.com
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > > On Jul 30, 2016, at 09:01, Yolanda Davis <yolanda.m.da...@gmail.com>
> > wrote:
> > >
> > > Thank you Joe, Tony, and to everyone in this community! It has been a
> > great
> > > experience contributing to NiFi and I am so excited to be a part of
> this
> > > project. I look forward to all the great work to come.
> > >
> > > Thanks Again!
> > >
> > > -yolanda
> > >
> > >> On Sat, Jul 30, 2016 at 8:22 AM, Joe Witt <joe.w...@gmail.com> wrote:
> > >>
> > >> Congrats and thanks Yolanda!
> > >>
> > >>> On Jul 30, 2016 8:16 AM, "Tony Kurc" <tk...@apache.org> wrote:
> > >>>
> > >>> On behalf of the Apache NiFI PMC, I am very pleased to announce that
> > >>> Yolanda Davis has accepted the PMC's invitation to become a committer
> > on
> > >>> the Apache NiFi project. We greatly appreciate all of Yolanda's hard
> > work
> > >>> and generous contributions and look forward to continued involvement
> in
> > >> the
> > >>> project.
> > >>>
> > >>> In addition to contributing variable registry support and json
> > >> conversion,
> > >>> many of you may have heard one of her talks and experienced her
> > >> enthusiasm
> > >>> firsthand at a myriad of conferences and meetups.
> > >>>
> > >>>
> > >>> Tony
> > >
> > >
> > >
> > > --
> > > --
> > > yolanda.m.da...@gmail.com
> > > @YolandaMDavis
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Jira privileges request

[Andrew Psaltis]

Thank you sir!

On Tue, Jul 12, 2016 at 5:11 PM, Tony Kurc <trk...@gmail.com> wrote:

> Andrew, I added you to Contributors.
>
> On Tue, Jul 12, 2016 at 5:06 PM, Andrew Psaltis <psaltis.and...@gmail.com>
> wrote:
>
> > Bryan,
> > While you are in the giving mood -- can you please add my id: apsaltis
> >
> > Thanks,
> > Andrew
> >
> > On Tue, Jul 12, 2016 at 4:58 PM, Bryan Bende <bbe...@gmail.com> wrote:
> >
> > > Haimo,
> > >
> > > You are now added as well.
> > >
> > > Thanks,
> > >
> > > Bryan
> > >
> > > On Tue, Jul 12, 2016 at 4:40 PM, Haimo Liu <haimoli...@gmail.com>
> wrote:
> > >
> > > > Hi Bryan,
> > > >
> > > > I would be more than glad to become a contributor too, can you help
> > grant
> > > > me the same JIRA privilege? Username: h_liu
> > > >
> > > > Thanks!
> > > > Haimo
> > > >
> > > > On Tue, Jul 12, 2016 at 1:12 PM, Andrew Lim <
> > andrewlim.apa...@gmail.com>
> > > > wrote:
> > > >
> > > > > Thanks Bryan!  I think I’m all set now.
> > > > >
> > > > > -Drew
> > > > >
> > > > > > On Jul 12, 2016, at 1:09 PM, Bryan Bende <bbe...@gmail.com>
> wrote:
> > > > > >
> > > > > > Drew,
> > > > > >
> > > > > > I added you the contributors role in JIRA. Let us know if you
> still
> > > > can't
> > > > > > assign tickets.
> > > > > >
> > > > > > -Bryan
> > > > > >
> > > > > > On Tue, Jul 12, 2016 at 1:06 PM, Andrew Lim <
> > > > andrewlim.apa...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > >> There are some tickets that I would like to assist on and assign
> > to
> > > > > >> myself.  Could I be given the privileges in Jira to do so?
> > > > > >>
> > > > > >> My username is:   andrewmlim
> > > > > >>
> > > > > >> Thanks!
> > > > > >>
> > > > > >> -Drew
> > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > Andrew
> >
> > Subscribe to my book: Streaming Data <http://manning.com/psaltis>
> > <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
> > twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Jira privileges request

[Andrew Psaltis]

Bryan,
While you are in the giving mood -- can you please add my id: apsaltis

Thanks,
Andrew

On Tue, Jul 12, 2016 at 4:58 PM, Bryan Bende <bbe...@gmail.com> wrote:

> Haimo,
>
> You are now added as well.
>
> Thanks,
>
> Bryan
>
> On Tue, Jul 12, 2016 at 4:40 PM, Haimo Liu <haimoli...@gmail.com> wrote:
>
> > Hi Bryan,
> >
> > I would be more than glad to become a contributor too, can you help grant
> > me the same JIRA privilege? Username: h_liu
> >
> > Thanks!
> > Haimo
> >
> > On Tue, Jul 12, 2016 at 1:12 PM, Andrew Lim <andrewlim.apa...@gmail.com>
> > wrote:
> >
> > > Thanks Bryan!  I think I’m all set now.
> > >
> > > -Drew
> > >
> > > > On Jul 12, 2016, at 1:09 PM, Bryan Bende <bbe...@gmail.com> wrote:
> > > >
> > > > Drew,
> > > >
> > > > I added you the contributors role in JIRA. Let us know if you still
> > can't
> > > > assign tickets.
> > > >
> > > > -Bryan
> > > >
> > > > On Tue, Jul 12, 2016 at 1:06 PM, Andrew Lim <
> > andrewlim.apa...@gmail.com>
> > > > wrote:
> > > >
> > > >> There are some tickets that I would like to assist on and assign to
> > > >> myself.  Could I be given the privileges in Jira to do so?
> > > >>
> > > >> My username is:   andrewmlim
> > > >>
> > > >> Thanks!
> > > >>
> > > >> -Drew
> > >
> > >
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: [ANNOUNCE] New Apache NiFi PMC Member - Andy LoPresto

[Andrew Psaltis]

Congrats Andy!!!

On Thursday, July 7, 2016, Tony Kurc <trk...@gmail.com> wrote:

> Glad to have you aboard, Andy!
>
> On Thu, Jul 7, 2016 at 12:44 PM, Joe Witt <joew...@apache.org
> <javascript:;>> wrote:
>
> > On behalf of the Apache NiFI PMC, I am very pleased to announce that
> > Andy LoPresto has accepted the PMC's invitation to join the Apache
> > NiFi PMC. We greatly appreciate all of Andy's hard work and generous
> > contributions to the project with a specific focus on security related
> > elements. We look forward to his continued involvement in the project.
> >
> > Welcome and congratulations!
> >
>


-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Build error with processor project

[Andrew Psaltis]

while executing
> org.apache.maven.plugins:maven-site-plugin:3.4:attach-descriptor:
> Lorg/apache/maven/doxia/tools/SiteTool;
>
> [ERROR] -
>
> [ERROR] realm =plugin>org.apache.maven.plugins:maven-site-plugin:3.4
>
> [ERROR] strategy =
> org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
>
> [ERROR] urls[0] =
> file:/Users/ben.meng/.m2/repository/org/apache/maven/plugins/maven-site-plugin/3.4/maven-site-plugin-3.4.jar
>
> [ERROR] urls[1] =
> file:/Users/ben.meng/.m2/repository/org/apache/maven/maven-archiver/2.5/maven-archiver-2.5.jar
>
> [ERROR] urls[2] =
> file:/Users/ben.meng/.m2/repository/org/apache/maven/reporting/maven-reporting-api/2.0.6/maven-reporting-api-2.0.6.jar
>
> [ERROR] urls[3] =
> file:/Users/ben.meng/.m2/repository/commons-cli/commons-cli/1.0/commons-cli-1.0.jar
>
> [ERROR] urls[4] =
> file:/Users/ben.meng/.m2/repository/org/codehaus/plexus/plexus-interactivity-api/1.0-alpha-4/plexus-interactivity-api-1.0-alpha-4.jar
>
> [ERROR] urls[5] =
> file:/Users/ben.meng/.m2/repository/org/codehaus/plexus/plexus-utils/3.0/plexus-utils-3.0.jar
>
> [ERROR] urls[6] =
> file:/Users/ben.meng/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.15/plexus-interpolation-1.15.jar
>
> [ERROR] urls[7] =
> file:/Users/ben.meng/.m2/repository/org/codehaus/plexus/plexus-archiver/2.4.4/plexus-archiver-2.4.4.jar
>
> [ERROR] urls[8] =
> file:/Users/ben.meng/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar
>
> [ERROR] urls[9] =
> file:/Users/ben.meng/.m2/repository/org/codehaus/plexus/plexus-io/2.0.10/plexus-io-2.0.10.jar
>
> [ERROR] urls[10] =
> file:/Users/ben.meng/.m2/repository/org/apache/commons/commons-compress/1.5/commons-compress-1.5.jar
>
> [ERROR] urls[11] =
> file:/Users/ben.meng/.m2/repository/org/tukaani/xz/1.2/xz-1.2.jar
>
> [ERROR] urls[12] =
> file:/Users/ben.meng/.m2/repository/org/apache/maven/doxia/doxia-core/1.6/doxia-core-1.6.jar
>
> [ERROR] urls[13] =
> file:/Users/ben.meng/.m2/repository/org/apache/maven/doxia/doxia-sink-api/1.6/doxia-sink-api-1.6.jar
>
> [ERROR] urls[14] =
> file:/Users/ben.meng/.m2/repository/org/apache/maven/doxia/doxia-logging-api/1.6/doxia-logging-api-1.6.jar
>
> [ERROR] urls[15] =
> file:/Users/ben.meng/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar
>
> [ERROR] urls[16] =
> file:/Users/ben.meng/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar
>
> [ERROR] urls[17] =
> file:/Users/ben.meng/.m2/repository/org/apache/httpcomponents/httpclient/4.0.2/httpclient-4.0.2.jar
>
> [ERROR] urls[18] =
> file:/Users/ben.meng/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar
>
> [ERROR] urls[19] =
> file:/Users/ben.meng/.m2/repository/commons-codec/commons-codec/1.3/commons-codec-1.3.jar
>
> [ERROR] urls[20] =
> file:/Users/ben.meng/.m2/repository/org/apache/httpcomponents/httpcore/4.0.1/httpcore-4.0.1.jar
>
> [ERROR] urls[21] =
> file:/Users/ben.meng/.m2/repository/xmlunit/xmlunit/1.5/xmlunit-1.5.jar
>
> [ERROR] Number of foreign imports: 1
>
> [ERROR] import: Entry[import  from realm
> ClassRealm[project>com.mydomain:testprocessor:1.0-SNAPSHOT, parent:
> ClassRealm[maven.api, parent: null]]]
>
> [ERROR]
>
> [ERROR] -:
> org.apache.maven.doxia.tools.SiteTool
>
> [ERROR] -> [Help 1]
>
> [ERROR]
>
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e switch.
>
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>
> [ERROR]
>
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
>
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException
>
> 
> The information contained in this transmission may contain privileged and
> confidential information. It is intended only for the use of the person(s)
> named above. If you are not the intended recipient, you are hereby notified
> that any review, dissemination, distribution or duplication of this
> communication is strictly prohibited. If you are not the intended
> recipient, please contact the sender by reply email and destroy all copies
> of the original message.
> 
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: KITESDK fails creation of schema

[Andrew Psaltis]

That makes sense. You may want to take into consideration the following
lifted from the spec:


The name portion of a fullname, record field names, and enum symbols must:


   - start with [A-Za-z_]
  - subsequently contain only [A-Za-z0-9_]


Thus, you may want to replace any other characters with perhaps a "_". It
wold be nice if the kiteSDK or another Avro processor provided an option to
substitute illegal characters with a valid on.

Hope that helps.

Andrew

On Wed, Jun 22, 2016 at 12:21 PM, pradeepbill <pradeep.b...@gmail.com>
wrote:

> I probably will replace incoming json text "Blah_bc_sc-status" to
> "Blah_bc_sc_status" and then operate on it,
>
> THanks
> Pradeep
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/KITESDK-fails-creation-of-schema-tp12105p12113.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: KITESDK fails creation of schema

[Andrew Psaltis]

Hi Pradeep,
That error is due to "-" being an illegal character for an Avro name, per
the spec here[1]. Unfortunately the error message is not as descriptive as
it could be.

Thanks,
Andrew

[1] https://avro.apache.org/docs/1.7.7/spec.html#Names

On Wed, Jun 22, 2016 at 12:02 PM, pradeepbill <pradeep.b...@gmail.com>
wrote:

> if my JSON has   "Blah_bc_sc-status":"407" , I can not create a valid
> schema
> using kiteSDK , fails saying
>
> [parumalla@sxmn5 ~]$ ./kite-dataset json-schema --class BCSchemaClass
> hdfs:/user/parumalla/test_bc.txt  --output
> hdfs:/user/parumalla/nifi/schemas/bc.avsc
>
> Unknown error: Illegal character in: Blah_bc_sc-status
>
> If I replace the '-' with '_' , it works, I probably will try modifying the
> schema by hand and see what happens.
>
> Thanks
> Pradeep
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/KITESDK-fails-creation-of-schema-tp12105.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Target version for new procesors

[Andrew Psaltis]

Thanks Joe and Andre -- this definitely helps point me in the correct
direction.

On Fri, Jun 17, 2016 at 6:19 PM, Andre <andre-li...@fucs.org> wrote:

> Andrew,
>
> If you plan to submit it as a PR I would say go with 1.0 (I have been doing
> that).
>
> The wrapping up of 0.7 is at full steam  and I think targeting it would
> just add to the stress.
>
> On the the other hand, targeting 1.x gives the reviewers the small courtesy
> of extra time to go through your code.
>
> Cheers
> On 18 Jun 2016 05:15, "Andrew Psaltis" <psaltis.and...@gmail.com> wrote:
>
> > All,
> > With the what appears to be a soon cut of 0.7.0 and 1.0.0 just around the
> > corner, if one were to start on developing a new processor what
> version(s)
> > should be targeted?
> >
> > --
> > Thanks,
> > Andrew
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Target version for new procesors

[Andrew Psaltis]

All,
With the what appears to be a soon cut of 0.7.0 and 1.0.0 just around the
corner, if one were to start on developing a new processor what version(s)
should be targeted?

-- 
Thanks,
Andrew

Re: Help Wanted

[Andrew Psaltis]

Totally makes sense. My only comment was just in regards to the fact that
on that Wiki page there is no mention of looking at GH PR's. I agree that
in this day and age looking to GH, may be first inclination for many.
However, perhaps calling that out in the contribution guide would eliminate
ambiguity. For example this JIRA
https://issues.apache.org/jira/browse/NIFI-1837, has a patch, but not a GH
PR.

Totally cool looking at both, just seems that it can be easy for someone
new to the project to miss something.



On Tue, May 3, 2016 at 11:35 AM, Bryan Bende <bbe...@gmail.com> wrote:

> The "Patch Available" state in JIRA can mean a patch is attached to the
> JIRA, or a PR is submitted.
>
> It is really just a manual state transition on the ticket after
> In-Progress... the next state is patch available which tells people there
> is something to review.
>
> On Tue, May 3, 2016 at 11:29 AM, Andrew Psaltis <psaltis.and...@gmail.com>
> wrote:
>
>> Totally agree on all fronts. Would seem like it makes sense for a
>> documentation PR to be opened soon with updates to the
>> https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-CodeReviewProcess
>> page to remove the ambiguity.
>>
>>
>> On Tue, May 3, 2016 at 11:27 AM, Bryan Bende <bbe...@gmail.com> wrote:
>>
>>> They are treated with same priority, but as Oleg mentioned, the PRs do
>>> make it easier for collaborative review and has the built in integration
>>> with Travis, although currently some issues to get it consistently working.
>>>
>>> On Tue, May 3, 2016 at 11:26 AM, Suneel Marthi <smar...@apache.org>
>>> wrote:
>>>
>>>> PR is the standard now across most Apache projects.
>>>>
>>>> On Tue, May 3, 2016 at 11:25 AM, Oleg Zhurakousky <
>>>> ozhurakou...@hortonworks.com> wrote:
>>>>
>>>>> Andrew
>>>>>
>>>>> Regarding PR vs. Patch.
>>>>>
>>>>> This has been an ongoing discussion and i’ll let other’s to contribute
>>>>> to this. Basically we support both. That said, personally (and it appears
>>>>> to be embraced by the rest of the community) PR is the preference
>>>>> specifically due to the inline review/comment capabilities provided by
>>>>> GitHub.
>>>>>
>>>>> Cheers
>>>>> Oleg
>>>>>
>>>>> > On May 3, 2016, at 11:18 AM, Andrew Psaltis <
>>>>> psaltis.and...@gmail.com> wrote:
>>>>> >
>>>>> > Thank you Oleg!
>>>>> >
>>>>> > Yeah, that page with the Code Review, has a little refresh link, but
>>>>> it
>>>>> > really just points to this JIRA query:
>>>>> > https://issues.apache.org/jira/browse/NIFI-1837?filter=12331874
>>>>> >
>>>>> > As a community is there a preference given to JIRA's with Patch or
>>>>> GH PR's
>>>>> > or are they all treated with the same priority?
>>>>> >
>>>>> > Thanks,
>>>>> > Andrew
>>>>> >
>>>>> > On Tue, May 3, 2016 at 11:12 AM, Oleg Zhurakousky <
>>>>> > ozhurakou...@hortonworks.com> wrote:
>>>>> >
>>>>> >> Andrew
>>>>> >>
>>>>> >> Thank you so much for following up on this.
>>>>> >> I am assuming you have GitHub account. If not please create one as
>>>>> most of
>>>>> >> our contributions deal with pull requests (PR).
>>>>> >> Then you can go to https://github.com/apache/nifi , click on “Pull
>>>>> >> Requests” and review them by commenting in line (you can see plenty
>>>>> of
>>>>> >> examples there of PRs that are already in review process).
>>>>> >>
>>>>> >> I would also suggest to get familiar with Contributor’s guideline
>>>>> for NiFi
>>>>> >> -
>>>>> https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide.
>>>>> But
>>>>> >> it appears you have already done so and I think there may be small
>>>>> >> discrepancy in the link you provided or may be it is not as dynamic.
>>>>> >> In any event JIRA and GutHub are good resources to use.
>>>>> >>
>>>>> >> As for the last question, the best case scen

Re: Help Wanted

[Andrew Psaltis]

Totally agree on all fronts. Would seem like it makes sense for a
documentation PR to be opened soon with updates to the
https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-CodeReviewProcess
page to remove the ambiguity.


On Tue, May 3, 2016 at 11:27 AM, Bryan Bende <bbe...@gmail.com> wrote:

> They are treated with same priority, but as Oleg mentioned, the PRs do
> make it easier for collaborative review and has the built in integration
> with Travis, although currently some issues to get it consistently working.
>
> On Tue, May 3, 2016 at 11:26 AM, Suneel Marthi <smar...@apache.org> wrote:
>
>> PR is the standard now across most Apache projects.
>>
>> On Tue, May 3, 2016 at 11:25 AM, Oleg Zhurakousky <
>> ozhurakou...@hortonworks.com> wrote:
>>
>>> Andrew
>>>
>>> Regarding PR vs. Patch.
>>>
>>> This has been an ongoing discussion and i’ll let other’s to contribute
>>> to this. Basically we support both. That said, personally (and it appears
>>> to be embraced by the rest of the community) PR is the preference
>>> specifically due to the inline review/comment capabilities provided by
>>> GitHub.
>>>
>>> Cheers
>>> Oleg
>>>
>>> > On May 3, 2016, at 11:18 AM, Andrew Psaltis <psaltis.and...@gmail.com>
>>> wrote:
>>> >
>>> > Thank you Oleg!
>>> >
>>> > Yeah, that page with the Code Review, has a little refresh link, but it
>>> > really just points to this JIRA query:
>>> > https://issues.apache.org/jira/browse/NIFI-1837?filter=12331874
>>> >
>>> > As a community is there a preference given to JIRA's with Patch or GH
>>> PR's
>>> > or are they all treated with the same priority?
>>> >
>>> > Thanks,
>>> > Andrew
>>> >
>>> > On Tue, May 3, 2016 at 11:12 AM, Oleg Zhurakousky <
>>> > ozhurakou...@hortonworks.com> wrote:
>>> >
>>> >> Andrew
>>> >>
>>> >> Thank you so much for following up on this.
>>> >> I am assuming you have GitHub account. If not please create one as
>>> most of
>>> >> our contributions deal with pull requests (PR).
>>> >> Then you can go to https://github.com/apache/nifi , click on “Pull
>>> >> Requests” and review them by commenting in line (you can see plenty of
>>> >> examples there of PRs that are already in review process).
>>> >>
>>> >> I would also suggest to get familiar with Contributor’s guideline for
>>> NiFi
>>> >> - https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide.
>>> But
>>> >> it appears you have already done so and I think there may be small
>>> >> discrepancy in the link you provided or may be it is not as dynamic.
>>> >> In any event JIRA and GutHub are good resources to use.
>>> >>
>>> >> As for the last question, the best case scenario is both (code review
>>> and
>>> >> test). Having said that we do realize that your time and the time of
>>> every
>>> >> contributor may be limited, so I say whatever you can. Some time
>>> quick code
>>> >> scan can uncover the obvious that doesn’t need testing.
>>> >>
>>> >> Thanks again
>>> >> Cheers
>>> >> Oleg
>>> >>
>>> >> On May 3, 2016, at 11:07 AM, Andrew Psaltis <psaltis.and...@gmail.com
>>> >
>>> >> wrote:
>>> >>
>>> >> Oleg,
>>> >> I would love to help -- couple of quick questions:
>>> >>
>>> >> The GH PR's are ~60 as you indicated, but the How To Contribute guide
>>> (Code
>>> >> review process --
>>> >>
>>> >>
>>> https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-CodeReviewProcess
>>> >> ) shows a JIRA list with patches available.
>>> >>
>>> >> Which should be reviewed first? For the PR's on GH are you just
>>> looking for
>>> >> code review or same process of apply local merge and test?
>>> >>
>>> >> Thanks,
>>> >> Andrew
>>> >>
>>> >> On 5/3/16, 9:58 AM, "Oleg Zhurakousky" <ozhurakou...@hortonworks.com>
>>> >> wrote:
>>> >>
>>> >> Guys
>>&g

Re: Help Wanted

[Andrew Psaltis]

Thank you Oleg!

Yeah, that page with the Code Review, has a little refresh link, but it
really just points to this JIRA query:
https://issues.apache.org/jira/browse/NIFI-1837?filter=12331874

As a community is there a preference given to JIRA's with Patch or GH PR's
or are they all treated with the same priority?

Thanks,
Andrew

On Tue, May 3, 2016 at 11:12 AM, Oleg Zhurakousky <
ozhurakou...@hortonworks.com> wrote:

> Andrew
>
> Thank you so much for following up on this.
> I am assuming you have GitHub account. If not please create one as most of
> our contributions deal with pull requests (PR).
> Then you can go to https://github.com/apache/nifi , click on “Pull
> Requests” and review them by commenting in line (you can see plenty of
> examples there of PRs that are already in review process).
>
> I would also suggest to get familiar with Contributor’s guideline for NiFi
> - https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide. But
> it appears you have already done so and I think there may be small
> discrepancy in the link you provided or may be it is not as dynamic.
> In any event JIRA and GutHub are good resources to use.
>
> As for the last question, the best case scenario is both (code review and
> test). Having said that we do realize that your time and the time of every
> contributor may be limited, so I say whatever you can. Some time quick code
> scan can uncover the obvious that doesn’t need testing.
>
> Thanks again
> Cheers
> Oleg
>
> On May 3, 2016, at 11:07 AM, Andrew Psaltis <psaltis.and...@gmail.com>
> wrote:
>
> Oleg,
> I would love to help -- couple of quick questions:
>
> The GH PR's are ~60 as you indicated, but the How To Contribute guide (Code
> review process --
>
> https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-CodeReviewProcess
> ) shows a JIRA list with patches available.
>
> Which should be reviewed first? For the PR's on GH are you just looking for
> code review or same process of apply local merge and test?
>
> Thanks,
> Andrew
>
> On 5/3/16, 9:58 AM, "Oleg Zhurakousky" <ozhurakou...@hortonworks.com>
> wrote:
>
> Guys
>
> I’d like to use this opportunity to address all members of the NiFi
>
> community hence this email is sent to both mailing lists (dev/users)
>
>
> While somewhat skeptical when I started 6 month ago, I have to admit that
>
> now I am very excited to observe the growth and adaption of the Apache NiFi
> and say that in large part it’s because of the healthy community that we
> have here - committers and contributors alike representing variety of
> business domains.
>
> This is absolutely great news for all of us and I am sure some if not all
>
> of you share this sentiment.
>
>
> That said and FWIW we need help!
> While it’s great to wake up every morning to a set of new PRs and patches,
>
> we now have a bit of a back log. In large this is due to the fact that most
> of our efforts are spent in development as we all try to grow NiFi feature
> base. However we need to remember that PRs and patches will remain as they
> are unless and until they are reviewed/agreed to be merged by this same
> community and that is where we need help. While “merge" responsibilities
> are limited to “committers”, “review” is the responsibility of every member
> of this community and I would like to ask you if at all possible to
> redirect some of your efforts to this process.
>
> We currently have 61 outstanding PRs and this particular development cycle
>
> is a bit more complex then the previous ones since it addresses 0.7.0 and
> 1.0.0 releases in parallel (so different approach to breaking changes if
> any etc.)
>
>
> Cheers
> Oleg
>
>
> --
> Thanks,
> Andrew
>
>
>


-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Help Wanted

[Andrew Psaltis]

Oleg,
I would love to help -- couple of quick questions:

The GH PR's are ~60 as you indicated, but the How To Contribute guide (Code
review process --
https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-CodeReviewProcess
) shows a JIRA list with patches available.

Which should be reviewed first? For the PR's on GH are you just looking for
code review or same process of apply local merge and test?

Thanks,
Andrew

On 5/3/16, 9:58 AM, "Oleg Zhurakousky"  wrote:

>Guys
>
>I’d like to use this opportunity to address all members of the NiFi
community hence this email is sent to both mailing lists (dev/users)
>
>While somewhat skeptical when I started 6 month ago, I have to admit that
now I am very excited to observe the growth and adaption of the Apache NiFi
and say that in large part it’s because of the healthy community that we
have here - committers and contributors alike representing variety of
business domains.
>This is absolutely great news for all of us and I am sure some if not all
of you share this sentiment.
>
>That said and FWIW we need help!
>While it’s great to wake up every morning to a set of new PRs and patches,
we now have a bit of a back log. In large this is due to the fact that most
of our efforts are spent in development as we all try to grow NiFi feature
base. However we need to remember that PRs and patches will remain as they
are unless and until they are reviewed/agreed to be merged by this same
community and that is where we need help. While “merge" responsibilities
are limited to “committers”, “review” is the responsibility of every member
of this community and I would like to ask you if at all possible to
redirect some of your efforts to this process.
>We currently have 61 outstanding PRs and this particular development cycle
is a bit more complex then the previous ones since it addresses 0.7.0 and
1.0.0 releases in parallel (so different approach to breaking changes if
any etc.)
>
>Cheers
>Oleg
>

-- 
Thanks,
Andrew