Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.13.0 (RC2)
Just a note, but the issues closed/resolved list is inaccessible to the public. Not sure if it is meant to be that way or not. The release notes link right below it is accessible though. Thanks, John On Monday, December 5, 2022, 05:40:42 AM EST, Ferenc Gerlits wrote: Hello, I am pleased to be calling this vote for the source release of Apache NiFi MiNiFi C++ 0.13.0. The code and the binary package are identical to RC1, I have fixed a problem with the source package. Sorry about the extra work. The source tarball, the binary build, plus signatures and digests can be found at: https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp/0.13.0/ The Git tag is minifi-cpp-0.13.0-RC2 The Git commit ID is 4389b9ac05e43e1dbf908e9014787d07c8a9cd05 https://github.com/apache/nifi-minifi-cpp/commit/4389b9ac05e43e1dbf908e9014787d07c8a9cd05 Checksums of nifi-minifi-cpp-0.13.0-source.tar.gz: SHA256: a0c4bf28d3a7e761951027c572f7716abac326a84da4c4a7c8a35e0aea10067f SHA512: 86637bbfbe910f076279174f102328bef4471852cfa8af67fda208dcd4ec4086986ea6eddce30f8f432f573f7b36ed40fa9d8270670e126d69efe235469c3170 Release artifacts are signed with the following key: https://people.apache.org/keys/committer/fgerlits.asc KEYS file available here: https://dist.apache.org/repos/dist/release/nifi/KEYS 85 issues were closed/resolved for this release: https://issues.apache.org/jira/projects/MINIFICPP/versions/12351771 Release note highlights can be found here: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65145325#ReleaseNotesMiNiFi(C++)-Versioncpp-0.13.0 The vote will be open for 72 hours. Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build from source, and test. Then please vote: [ ] +1 Release this package as nifi-minifi-cpp-0.13.0 [ ] +0 no opinion [ ] -1 Do not release this package because...
Re: [discuss] we need to enable secure by default...
I'm on the opposite side of this discussion. I disagree that NiFi has to implement something immediately. Now, I admit that I am old school, and feel that if you aren't capable of doing what is necessary to secure your own things, it's your own fault. But, I also realize, everyone on the internet seems to be coding now without the true understanding of what that means. I totally agree that a NiFi server open in the wild is a bad thing, and can be used by hackers. But, truly, NiFi isn't broken in the sense of security vulnerability. Mark mentions a default username password with a basic auth, which is actually no better than what that Twitter post had. The default username/password will be known to all, and that will be the first thing they attempt to get into those servers. (Or using the API.) My company has looked into the basic auth side of things, just so that even in a local network, people couldn't just find the server and do stuff on it. But, since there wasn't a NiFi "approved" method, we didn't pursue it. Instead, we run NiFi on a single machine that you have to log in to to be able to get to it, and the single machine has very limited users and the no access to NiFi from outside that machine. In a second instance, we have done a SSH tunnel into a Linux machine to access the NiFi running on that Linux machine when firewalls are not open to the NiFi port. Also, I run NiFi at home, and I don't have it connected to the Internet. I wouldn't want to deal with user/pass or HTTPS because there is no need for it. HTTP on its own isn't bad, it is the data individuals are sending that determines if HTTP shouldn't be used. My small website is just simple data with absolutely no usefulness to hackers, and therefore, doesn't really need HTTPS. My opinion is that we shouldn't take a knee-jerk reaction to this one Twitter post about what appears to be ONE NiFi server, unsecured on the Internet. Just my opposing opinion. John On Wednesday, February 10, 2021, 09:24:01 AM EST, Mark Payne wrote: > I would be in favor of this as well. I agree that there is no need to wait > for a 2.0 version - there is no inherit backward compatibility challenge here. > Requiring that new application configs be set, etc. I think is completely > fair game between minor versions. > > Personally, though, I would very much stray away from auto-generating > certificates. For those of us who have dealt with certificates significantly > In the past, minor configuration issues are easy to address. But for someone > who hasn’t spent much time dealing with certificates, the error messages > that are often intentionally vague can quickly result in users being > overwhelmed. This is especially true if browser configurations are already > setup to > automatically offer certificates that area already installed - this can > result in weird error messages about untrusted certificates when the user > thinks > they haven’t even provided a certificate, etc. It gets really hairy. > > I am more in favor of a default username/password personally. It would > require implementing a new auth provider. And it’s one that historically we > have > avoided implementing because a basic auth type of approach is less secure > than mutual auth, ldap, etc. But it’s more secure than nothing. > > Thanks > -Mark
Re: [VOTE] Release Apache NiFi 1.13.0
Just a note that the "helpful reminder" Confluence page stills lists the artifact names as 1.12.0. On Wednesday, January 27, 2021, 11:15:56 PM EST, Joe Witt wrote: Hello, I am pleased to be calling this vote for the source release of Apache NiFi 1.13.0. (cut) A helpful reminder on how the release candidate verification process works: https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
Re: [board report] Apache NiFi - July 2019
Joe, Minor squabble, but, NiFi is getting setup for 1.10.0 release, not 0.10.0 release. On Wednesday, July 10, 2019, 1:16:04 PM EDT, Joe Witt wrote: Team, I was running late so submitted the report already. Here is what I sent to the board for Apache NiFi July 2019 report. Great work and great progress all! ... ## Activity: - Apache NiFi 0.10.0 is progressing nicely with nearly 200 JIRAs already included. This brings powerful features such as sourcing extensions from the latest NiFi Registry at runtime, far better model for paramaterized version controlled flows, Java 11 compatibility, and much more.
Re: Compiling custom processor
Thanks Bryan. That was a mistake on my part. I thought that I could use part of the NiFi standard processors in my custom processor. I suppose this also means it's not possible to extend a standard processor class, say Wait, and override a method, for instance, onTrigger(), to do different work. Instead, as you state, I should copy the portions I would need to my custom processor. Is that accurate? In the end, as I was looking through how to modify Wait, I think it is bigger than the need that I have, so I restarted with a new generated processor project, and just added the cache service, and appropriate relationships, and simple logic to control flow through my processor. I'll work on getting a github account and uploading my simple processor. Thanks for the assistance John McGinn On Tue, 10/16/18, Bryan Bende wrote: Subject: Re: Compiling custom processor To: figgie...@sbcglobal.net Date: Tuesday, October 16, 2018, 4:45 PM You shouldn't really try to depend on code inside other NARs like standard processors, you'll want to copy WaitNotifyProtocol into your own NAR. On Tue, Oct 16, 2018 at 4:40 PM John McGinn wrote: > > The commons-lang3-3.7.jar file is located in the lib/bootstrap directory, therefore, I figured it could be considered provided. I've since change it to normal scope, and it is included in my NAR and that error goes away. So, this > then leans on the next error I get, which is no class found for the WaitNotifyProtocol in nifi.processors.standard. Is there an easy way to figure out what NiFi class/object I'm using, and what NAR should be listed as provided? I tried to add nifi-standard-processors-nar, but that doesn't exist for maven, nor does it exist in the lib directory. I looked inside the nifi-standard-nar NAR file, and the nifi-standard-processors is there, and inside that is WaitNotifyProtocol, but yet, my maven won't build if I have nifi-standard-nar listed, either with type of NAR, or scope of provided. > > On Tue, 10/16/18, Bryan Bende wrote: > > Subject: Re: Compiling custom processor > To: dev@nifi.apache.org, figgie...@sbcglobal.net > Date: Tuesday, October 16, 2018, 4:01 PM > > Any reason why the commons-lang3 > dependency is provided and not normal > compile scope? > > Generally the only cases where a dependency > should be provided are > when a parent NAR > provides it (the API scenario earlier), or when it > is a jar directly in lib which means it is on > the classpath of > everything (like nifi-api > or slf4j). > On Tue, Oct 16, 2018 at 3:47 PM John McGinn > > wrote: > > > > I have a > directory /home/user/nifi-nars, which is listed in the > nifi.properties file located under the nifi-assembly > directory from the nifi-master code. That's where I > redropped my NAR file at. > > > > The error was "due to uncaught > Exception: java.lang.NoClassDefFoundError: > org/apache/commons/lang3/StringUtils". I added a > provided dependency of version 3.4 for commons-lang3 in my > processors pom.xml. That worked initially, but with the > redropping of the NAR file, the error occurs. The stacktrace > points to the session.get() lambda function trying to use > StringUtils.isBlank(). (Again, this is a copy and renamed > class of the Wait Processor. Trying to turn it into a Yield, > as it were, hold unless it's free to move forward.) > > > > > On Tue, 10/16/18, Bryan Bende > wrote: > > > > Subject: > Re: Compiling custom processor > > To: dev@nifi.apache.org > > Date: Tuesday, October 16, 2018, 3:34 > PM > > > > There > shouldn't be any issue > > doing > that, generally you just copy the > > > new > > NAR into the lib directory and > restart. > > > > What > were the specifics of the class not found > > error? > > > > On Tue, Oct 16, 2018 at > > 3:24 PM John McGinn > > wrote: > > > > > > And I'm > > > back. > > > > > > > So, I had my > > processor running, > no errors, so things were good. > > > > > > > I then add 2 if > > statements with logger messages, run the > same mvn command, > > copied the new NAR > on top of the old one, same versionId and > > all, and now I get a class not found > error, that didn't > > occur before I > stopp
Re: Compiling custom processor
I have a directory /home/user/nifi-nars, which is listed in the nifi.properties file located under the nifi-assembly directory from the nifi-master code. That's where I redropped my NAR file at. The error was "due to uncaught Exception: java.lang.NoClassDefFoundError: org/apache/commons/lang3/StringUtils". I added a provided dependency of version 3.4 for commons-lang3 in my processors pom.xml. That worked initially, but with the redropping of the NAR file, the error occurs. The stacktrace points to the session.get() lambda function trying to use StringUtils.isBlank(). (Again, this is a copy and renamed class of the Wait Processor. Trying to turn it into a Yield, as it were, hold unless it's free to move forward.) On Tue, 10/16/18, Bryan Bende wrote: Subject: Re: Compiling custom processor To: dev@nifi.apache.org Date: Tuesday, October 16, 2018, 3:34 PM There shouldn't be any issue doing that, generally you just copy the new NAR into the lib directory and restart. What were the specifics of the class not found error? On Tue, Oct 16, 2018 at 3:24 PM John McGinn wrote: > > And I'm back. > > So, I had my processor running, no errors, so things were good. > > I then add 2 if statements with logger messages, run the same mvn command, copied the new NAR on top of the old one, same versionId and all, and now I get a class not found error, that didn't occur before I stopped, redeployed and started. And I can't delete the event in the queue even though source and destination are stopped. > > Is there problems when redeploying a custom processor NAR on top of itself? Even if the NiFi instance is stopped? Serialization issues? > > Frustratedly yours, > John McGinn > > ---- > On Tue, 10/16/18, John McGinn wrote: > > Subject: Re: Compiling custom processor > To: dev@nifi.apache.org > Date: Tuesday, October 16, 2018, 1:21 PM > > Thanks Bryan, > > I restarted with a fresh maven > generate, and only added the API to the processor pom, and > then the API-NAR to the NAR pom. I then figured out a > subsequent dependency (lang3), and got that in, and now my > processor shows up in local NiFi instance. > > Regarding the copying of the standard > processor, when I copied Wait over, I renamed it from Wait > to MyProcessorWait to be unique, and not copied. I'm just > attempting changes to Wait without modifying the actual one. > Also, thanks on the META-INF/services file information. > > Now I can debug and test. > > Sincerely, > John McGinn > > > On Tue, 10/16/18, Bryan Bende > wrote: > > Subject: Re: Compiling custom > processor > To: dev@nifi.apache.org > Date: Tuesday, October 16, 2018, 9:52 > AM > > In general, if your processor > uses a controller service then the > processors pom file needs a provided > dependency > on the API of the CS, > and your NAR pom needs > a NAR dependency on the NAR where the > CS API > is. > > Example is > shown here in the section linking > processors and > controller > services - >https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions > > Also, you do not want to > include NiFI's standard processors in > your > own NAR, this will result in two > copies of > every standard processor. > You will want to > copy whatever code you need into your > own NAR. > > In you last statement, if you > made a new processor in > nifi-standard-processors and it didn't > show > up, it is most likely > because you didn't > update META-INF/services file to > include the new > processor. > On Tue, Oct 16, 2018 at > 9:36 AM John McGinn > > wrote: > > > > Ok, I am > far from a maven expert, and am > struggling on this > problem. > > > > I created > a new project using the maven generate > process, and compiled > that sample processor and everything > was fine, and I could > see it in my local NiFi instance. I > then copied over the > Wait processor on top of the > MyProcessor.java class, changed > the package name, and the class name, > and attempted to > compile. Got errors due to dependency > issues. This is where > I get confused. > > > > In > the processor directory, I modify the > pom.xml with a > dependency line for nifi-api, > nifi-utils, > > nifi-distri
Re: Compiling custom processor
And I'm back. So, I had my processor running, no errors, so things were good. I then add 2 if statements with logger messages, run the same mvn command, copied the new NAR on top of the old one, same versionId and all, and now I get a class not found error, that didn't occur before I stopped, redeployed and started. And I can't delete the event in the queue even though source and destination are stopped. Is there problems when redeploying a custom processor NAR on top of itself? Even if the NiFi instance is stopped? Serialization issues? Frustratedly yours, John McGinn On Tue, 10/16/18, John McGinn wrote: Subject: Re: Compiling custom processor To: dev@nifi.apache.org Date: Tuesday, October 16, 2018, 1:21 PM Thanks Bryan, I restarted with a fresh maven generate, and only added the API to the processor pom, and then the API-NAR to the NAR pom. I then figured out a subsequent dependency (lang3), and got that in, and now my processor shows up in local NiFi instance. Regarding the copying of the standard processor, when I copied Wait over, I renamed it from Wait to MyProcessorWait to be unique, and not copied. I'm just attempting changes to Wait without modifying the actual one. Also, thanks on the META-INF/services file information. Now I can debug and test. Sincerely, John McGinn On Tue, 10/16/18, Bryan Bende wrote: Subject: Re: Compiling custom processor To: dev@nifi.apache.org Date: Tuesday, October 16, 2018, 9:52 AM In general, if your processor uses a controller service then the processors pom file needs a provided dependency on the API of the CS, and your NAR pom needs a NAR dependency on the NAR where the CS API is. Example is shown here in the section linking processors and controller services - https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions Also, you do not want to include NiFI's standard processors in your own NAR, this will result in two copies of every standard processor. You will want to copy whatever code you need into your own NAR. In you last statement, if you made a new processor in nifi-standard-processors and it didn't show up, it is most likely because you didn't update META-INF/services file to include the new processor. On Tue, Oct 16, 2018 at 9:36 AM John McGinn wrote: > > Ok, I am far from a maven expert, and am struggling on this problem. > > I created a new project using the maven generate process, and compiled that sample processor and everything was fine, and I could see it in my local NiFi instance. I then copied over the Wait processor on top of the MyProcessor.java class, changed the package name, and the class name, and attempted to compile. Got errors due to dependency issues. This is where I get confused. > > In the processor directory, I modify the pom.xml with a dependency line for nifi-api, nifi-utils, nifi-distribute-cache-client-service-api and nifi-standard-processors. This seemed to make things work, and I get a 33 meg NAR file, which contains lots of JARs. I load that up, and I get a problem with CalculateStats, or similar, because of RecordReaderFactory. I go back in and include nifi-record-serialization-services, and it compiles, and the JAR is included in the NAR file now, and I still get class not found. > > I've also tried to change the dependencies to a status of provided, and therefore get a minimal 25k or so NAR file with no JARs included. I get the same issue with the class not found. > > I had also attempted to copy Wait.java to AnotherWait.java within the actually NiFi source code, and it compiles, and shows up in the JARs and NARs as I'd expect, but I cannot get to it within my NiFi instance. > > Can someone let me know what I'm doing wrong. > > Thanks, > John McGinn
Re: Compiling custom processor
Thanks Bryan, I restarted with a fresh maven generate, and only added the API to the processor pom, and then the API-NAR to the NAR pom. I then figured out a subsequent dependency (lang3), and got that in, and now my processor shows up in local NiFi instance. Regarding the copying of the standard processor, when I copied Wait over, I renamed it from Wait to MyProcessorWait to be unique, and not copied. I'm just attempting changes to Wait without modifying the actual one. Also, thanks on the META-INF/services file information. Now I can debug and test. Sincerely, John McGinn On Tue, 10/16/18, Bryan Bende wrote: Subject: Re: Compiling custom processor To: dev@nifi.apache.org Date: Tuesday, October 16, 2018, 9:52 AM In general, if your processor uses a controller service then the processors pom file needs a provided dependency on the API of the CS, and your NAR pom needs a NAR dependency on the NAR where the CS API is. Example is shown here in the section linking processors and controller services - https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions Also, you do not want to include NiFI's standard processors in your own NAR, this will result in two copies of every standard processor. You will want to copy whatever code you need into your own NAR. In you last statement, if you made a new processor in nifi-standard-processors and it didn't show up, it is most likely because you didn't update META-INF/services file to include the new processor. On Tue, Oct 16, 2018 at 9:36 AM John McGinn wrote: > > Ok, I am far from a maven expert, and am struggling on this problem. > > I created a new project using the maven generate process, and compiled that sample processor and everything was fine, and I could see it in my local NiFi instance. I then copied over the Wait processor on top of the MyProcessor.java class, changed the package name, and the class name, and attempted to compile. Got errors due to dependency issues. This is where I get confused. > > In the processor directory, I modify the pom.xml with a dependency line for nifi-api, nifi-utils, nifi-distribute-cache-client-service-api and nifi-standard-processors. This seemed to make things work, and I get a 33 meg NAR file, which contains lots of JARs. I load that up, and I get a problem with CalculateStats, or similar, because of RecordReaderFactory. I go back in and include nifi-record-serialization-services, and it compiles, and the JAR is included in the NAR file now, and I still get class not found. > > I've also tried to change the dependencies to a status of provided, and therefore get a minimal 25k or so NAR file with no JARs included. I get the same issue with the class not found. > > I had also attempted to copy Wait.java to AnotherWait.java within the actually NiFi source code, and it compiles, and shows up in the JARs and NARs as I'd expect, but I cannot get to it within my NiFi instance. > > Can someone let me know what I'm doing wrong. > > Thanks, > John McGinn
Compiling custom processor
Ok, I am far from a maven expert, and am struggling on this problem. I created a new project using the maven generate process, and compiled that sample processor and everything was fine, and I could see it in my local NiFi instance. I then copied over the Wait processor on top of the MyProcessor.java class, changed the package name, and the class name, and attempted to compile. Got errors due to dependency issues. This is where I get confused. In the processor directory, I modify the pom.xml with a dependency line for nifi-api, nifi-utils, nifi-distribute-cache-client-service-api and nifi-standard-processors. This seemed to make things work, and I get a 33 meg NAR file, which contains lots of JARs. I load that up, and I get a problem with CalculateStats, or similar, because of RecordReaderFactory. I go back in and include nifi-record-serialization-services, and it compiles, and the JAR is included in the NAR file now, and I still get class not found. I've also tried to change the dependencies to a status of provided, and therefore get a minimal 25k or so NAR file with no JARs included. I get the same issue with the class not found. I had also attempted to copy Wait.java to AnotherWait.java within the actually NiFi source code, and it compiles, and shows up in the JARs and NARs as I'd expect, but I cannot get to it within my NiFi instance. Can someone let me know what I'm doing wrong. Thanks, John McGinn
