Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.99.0 (RC3)

2024-05-07 Thread Joe Witt
Hello Sigs/Hashes are good. Building source on macos I'm getting a build failure. The only option selected was to skip tests otherwise left as defaults. Am following the readme and release guide. ninja: build stopped: subcommand failed. Build was unsuccessful Traceback (most recent call

Re: [discuss] identifying key components needed updates which require deeper review/consideration

2024-05-07 Thread Joe Witt
Awesome Chris and David - both of those are done and dusted! Amazing. Matt; Thanks yes for Hive it should just be simplifying/updating the Iceberg dependency on Hive to the latest or even removing if not necessary. Iceberg itself can be updated but my understanding is the 1.5.1 release has a

Re: [DISCUSS] Release Timeline for NiFi 2.0.0-M3

2024-05-07 Thread David Handermann
Team, There has been a significant amount of progress on the new UI, and it is now part of the default build in the main branch. There are a couple more dependency updates to be incorporated, and I plan initiating a release candidate build for 2.0.0-M3 in the next day or two at the latest.

Re: [discuss] identifying key components needed updates which require deeper review/consideration

2024-05-07 Thread David Handermann
Joe, Thanks for highlighting these areas to be updated. I was able to address the QuestDB update with a couple minor adjustments, and submitted a PR [1]. Regards, David Handermann [1] https://github.com/apache/nifi/pull/8773 On Tue, May 7, 2024 at 2:58 PM Chris Sampson wrote: > > I’ll pick up

Re: [discuss] removal of nifi-external module in 2.x/deprecate in 1.x

2024-05-07 Thread David Handermann
Joe, Thanks for highlighting the Kafka Connect external module. I concur with your evaluation, I have not observed community usage or questions, so it seems best to remove from the main branch. Regards, David Handermann On Tue, May 7, 2024 at 2:18 PM Otto Fowler wrote: > > I would say nuke it,

Re: [discuss] identifying key components needed updates which require deeper review/consideration

2024-05-07 Thread Chris Sampson
I’ll pick up the Elasticsearch dependency upgrade. I’ll try to test it again OpenSearch as well, if for no reason other than to prove either way whether the current NiFi components will work with the AWS service, although I may not get to do that and instead testing will focus on Elasticsearch

Re: [discuss] identifying key components needed updates which require deeper review/consideration

2024-05-07 Thread Joe Witt
Yeah sorry - what I wrote for the Hive one was a bit nonsensical. I tried to clarify it better in the JIRA. Thanks On Tue, May 7, 2024 at 12:38 PM Matt Burgess wrote: > For [1] Hive 3 was removed in > https://issues.apache.org/jira/browse/NIFI-12981, if you mean the Hive > dependency from

Re: [discuss] identifying key components needed updates which require deeper review/consideration

2024-05-07 Thread Matt Burgess
For [1] Hive 3 was removed in https://issues.apache.org/jira/browse/NIFI-12981, if you mean the Hive dependency from Apache Iceberg we probably need a release from them that brings in Hive 4 dependencies? Or do you suggest we bring the Hive components back as long as we update the version? I'll

Re: [discuss] removal of nifi-external module in 2.x/deprecate in 1.x

2024-05-07 Thread Otto Fowler
I would say nuke it, as you are saying. I would also say you should cross post this to users@ as they will have relevant input On May 7, 2024 at 1:32:11 PM, Joe Witt (joew...@apache.org) wrote: Team, The only component remaining is kafka-connect in the nifi-external module. As noted in [1] it

Re: ...not the most recent version of this FlowFile within this session...

2024-05-07 Thread Russell Bateman
Yes, what you described is what was happening, Mark. I didn't display all of the code to the session methods, and I did re-read the in-coming flowfile for different purposes than I had already read and written it. So, I wasn't helpful enough. In the end, however, I had forgotten, immediately

[discuss] identifying key components needed updates which require deeper review/consideration

2024-05-07 Thread Joe Witt
Team, We have made massive strides going from thousands of out of date libraries to dozens in the 2.x line vs any prior releases. And we can/should stay on top of these better going forward. A few key bits that are outstanding with no obvious immediate work underway are as follows. If anyone

Re: ...not the most recent version of this FlowFile within this session...

2024-05-07 Thread Mark Payne
Russell, May not be too bad to fix :) Any time you use a method on ProcessSession to modify a FlowFile in some way (such as putAttribute, putAllAttributes, write, removeAttribute, removeAllAttributes, etc.) the ProcessSession returns to you a new FlowFile. So you *should* write code such as:

Re: ...not the most recent version of this FlowFile within this session...

2024-05-07 Thread Joe Witt
Russ FlowFile result = session.*write*( flowfile, new StreamCallback() session.*read*( flowfile, new InputStreamCallback() session.*removeAllAttributes*( result, keys ); session.*putAllAttributes*( result, attributes );

...not the most recent version of this FlowFile within this session...

2024-05-07 Thread Russell Bateman
In /pom.xml/ I specify using NiFi framework libraries from 1.13.2. There are peculiar reasons (that we are trying to fix) that inhibit us from moving forward as all of our customer machines are running 1.1.2. (Don't shoot me, I'm not DevOps, but just the guy who writes custom processors.) I

[discuss] removal of nifi-external module in 2.x/deprecate in 1.x

2024-05-07 Thread Joe Witt
Team, The only component remaining is kafka-connect in the nifi-external module. As noted in [1] it does not see much active maintenance and I never see in the apache community any questions/usage related either. Kafka is currently on 3.7.0 and this component is Kafka 2.6 for example. We should

Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.99.0 (RC3)

2024-05-07 Thread Arpad Boda
+1 (binding) Built on ubuntu, verified signature and checksums. All tests passed, started and ran a simple flow with no issues. Nice job, team! On Tue, May 7, 2024 at 11:35 AM Gábor Gyimesi wrote: > Hello, > > I am pleased to be calling this vote for the source release of Apache NiFi > MiNiFi

[VOTE] Release Apache NiFi MiNiFi C++ 0.99.0 (RC3)

2024-05-07 Thread Gábor Gyimesi
Hello, I am pleased to be calling this vote for the source release of Apache NiFi MiNiFi C++ 0.99.0. The source tarball, some binary builds, plus signatures and digests can be found at: https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp/0.99.0/ The Git tag is minifi-cpp-0.99.0-RC3 The