Re: Facing Issue with Ni-Fi

2024-08-01 Thread Matt Burgess
Raja, To what system are you migrating? If it is a straight DB migration, you should be able to use ExecuteSQL -> PutDatabaseRecord. Just make sure your Fetch Size property on ExecuteSQL is not set to the default of zero or the PostgreSQL driver will try to fetch the whole thing into memory. For 1

Re: Flow comparison

2024-07-23 Thread Matt Burgess
json.gz, and these are the differences it found. > It is a bit confusing, admittedly, but it is functioning as intended. > > Thanks > -Mark > > > > On Jul 23, 2024, at 2:56 PM, Matt Burgess wrote: > > > > I recently noticed this on NiFi startup of a standalone insta

Flow comparison

2024-07-23 Thread Matt Burgess
I recently noticed this on NiFi startup of a standalone instance even though I hadn't changed the flow in the meantime: 2024-07-23 14:54:29,559 INFO [main] o.a.n.c.s.StandardProcessScheduler Enabling StandardControllerServiceNode[service=JsonRecordSetWriter[id=e09a1886-0190-1000-cf4b-219dcc5cd740]

Re: [DISCUSS] 1.x Maintenance

2024-07-18 Thread Matt Burgess
t; > > > can > > > > > > > > > > find contributors and committers that want to put in > extra > > > work > > > > > to > > > > > > &g

Re: [DISCUSS] 1.x Maintenance

2024-07-18 Thread Matt Burgess
gt; > > > Notably though it is also a very different concern from > > critical > > > > > > > > dependencies we have and how they evolve. For example we rely > > on > > > > > things > >

Re: [VOTE] Release Apache NiFi 1.27.0 (RC2)

2024-07-05 Thread Matt Burgess
+1 (binding) ran through the release helper and tested various flows including NIFI-13422. Apache Maven 3.9.8 (36645f6c9b5079805ea5009217e36f2cffd34256) Java version: 1.8.0_372, vendor: Azul Systems, Inc., runtime: /Users/mburgess/.sdkman/candidates/java/8.0.372-zulu/zulu-8.jdk/Contents/Home/jre D

[DISCUSS] 1.x Maintenance

2024-07-02 Thread Matt Burgess
There have been some ongoing discussions [1,2] about what to bring back for PRs to 1.x vs trying to push forward with 2.x. There are of course great points from everyone. On the 2.x front, namely that 2.x has many improvements not just to components but the framework and UI as well, and that we've

Re: [VOTE] Release Apache NiFi 2.0.0-M4 (RC1)

2024-06-28 Thread Matt Burgess
+1 (binding) Ran through release helper and tried various flows, verified versioning against a NiFi Registry (not the Git-backed one) Apache Maven 3.9.7 (8b094c9513efc1b9ce2d952b3b9c8eaedaf8cbf0) Java version: 21, vendor: Oracle Corporation, runtime: /Users/mburgess/.sdkman/candidates/java/21-orac

Re: [VOTE] Release Apache NiFi 1.27.0 (RC1)

2024-06-27 Thread Matt Burgess
I'm seeing SHA256 and SHA512 mismatches, also my automated RC script noticed your GPG key is expired. On Thu, Jun 27, 2024 at 5:11 PM Pierre Villard wrote: > Team, > > I am pleased to be calling this vote for the source release of Apache NiFi > 1.27.0. > > Please review the following guide for h

Re: processors authoring advice

2024-06-24 Thread Matt Burgess
Wei, I have looked at adding a processor for calling stored procedures, but the problem I encountered was how to allow the user to specify input and/or output parameters to stored procedures and functions. You can use ExecuteSQL to execute a stored procedure with no input/output parameters as you

Re: SplitPCAP processor bug with large files and proposed fix

2024-06-24 Thread Matt Burgess
Jack, For 1, check one of the existing Split processors such as SplitRecord for the "fragment.*" attribute pattern, it should be consistent with that behavior and code. For 2, if an invalid packet should "spoil the bunch" so to speak, you can use session.rollback() then transfer the original Flow

Re: Thoughts on various ways to help drive NiFi and the community forward

2024-06-16 Thread Matt Burgess
Mathew, certainly! I've been meaning to update the blog too. Email me at mattyb...@apache.org and we'll get 'em up there :) Thanks, Matt On Sat, Jun 15, 2024 at 2:46 PM Mathew Kiprop wrote: > +1 to all subjects addressed Joe. > > I have been planning to improve developer advocacy around scripti

Unfinished process

2024-06-10 Thread Matt Burgess
I just built from the latest main and haven't run it yet but I still have a running process many minutes later: 503 19756 1 0 6:12PM ?? 0:02.10 /Users/mburgess/git/nifi/nifi-frontend/target/frontend-working-directory/node/node /Users/mburgess/git/nifi/nifi-frontend/target/frontend

Re: [VOTE] Release Apache NiFi NAR Maven Plugin 2.0.0

2024-05-23 Thread Matt Burgess
+1 binding, verified the commit hash, applied the patch to NiFi main and verified the NAR Maven plugin works as expected. Thanks for RM'ing Pierre! On Wed, May 22, 2024 at 7:31 AM Pierre Villard wrote: > Hello Apache NiFi Community, > > I am pleased to be calling this vote for the source releas

Re: [DISCUSS] Release NiFi NAR Maven Plugin

2024-05-20 Thread Matt Burgess
+1 for a 1.6.0 release On Mon, May 20, 2024 at 1:25 PM Bryan Bende wrote: > Thanks Pierre. I agree it would be good to kick out a release. I would > lean towards 1.6.0 since the commits seem to be improvements rather > than bug fixes for a patch. > > On Mon, May 20, 2024 at 1:08 PM Pierre Villar

Re: [discuss] What to do with the Cassandra components

2024-05-20 Thread Matt Burgess
ot of progress on such an an approach I can post to my repo if > you're interested. > > On Fri, May 17, 2024 at 7:31 AM Matt Burgess wrote: > > > Now that the Couchbase PR is up I can continue my work on this if > > everyone's ok with the approach. > > >

Re: [discuss] What to do with the Cassandra components

2024-05-17 Thread Matt Burgess
gt; > > Thanks > > > Joe > > > > > > On Fri, Mar 22, 2024 at 11:42 AM Mike Thomsen > > > wrote: > > > > > > > The scope tag was probably copy pasta. You raise a valid point about > > the > > > > processor dependencies t

Re: [VOTE] Release Apache NiFi 2.0.0-M3 (RC1)

2024-05-16 Thread Matt Burgess
+1 (binding) Ran through release helper, verified various flows. Thanks for RM'ing David! On Mon, May 13, 2024 at 11:48 PM David Handermann < exceptionfact...@apache.org> wrote: > Team, > > I am pleased to be calling this vote for the source release of Apache > NiFi 2.0.0-M3. > > Please review t

Re: Inquiry Regarding Missing Documentation for selectHive3QL in nifi-hive3-nar for NiFi 1.25.0

2024-05-16 Thread Matt Burgess
Jing, Because SelectHive3QL (and the nifi-hive3-nar in general) is not included with the convenience binary for Apache NiFi, the documentation for it is not published on the website. If you downloaded and installed the NAR into your instance, the documentation is available by dragging a SelectHive

Re: [discuss] identifying key components needed updates which require deeper review/consideration

2024-05-07 Thread Matt Burgess
For [1] Hive 3 was removed in https://issues.apache.org/jira/browse/NIFI-12981, if you mean the Hive dependency from Apache Iceberg we probably need a release from them that brings in Hive 4 dependencies? Or do you suggest we bring the Hive components back as long as we update the version? I'll t

Re: [VOTE] Release Apache NiFi 1.26.0 (RC1)

2024-05-03 Thread Matt Burgess
+1 (binding) Ran through release helper, verified various flows for fixes including https://issues.apache.org/jira/browse/NIFI-13121 Thanks for RM'ing Pierre! On Fri, May 3, 2024 at 8:47 AM Pierre Villard wrote: > Team, > > I am pleased to be calling this vote for the source release of Apache

Re: [VOTE] Release Apache NiFi 1.26.0 (RC1)

2024-05-03 Thread Matt Burgess
You may be running into [1], can you upgrade your JDK? Regards, Matt [1] https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8219013 On Fri, May 3, 2024 at 2:54 PM Dan S wrote: > II am encountering the exception below when building with Java 8 > My environment per maven -version > > Mave

Re: [DISCUSS] Release Timeline for NiFi 2.0.0-M3

2024-05-01 Thread Matt Burgess
One thing that was mentioned was an included Jira for "providing a clearer distinction between framework and extensions," This involved moving extensions to a new module and for community folks this is causing problems with any new improvements in-flight before that. Seems like it needed more anno

Re: [discuss] What to do with the Cassandra components

2024-03-22 Thread Matt Burgess
t were that simple, they would probably have just gone with that > > >> solution. That said, the API is functionally vendor agnostic at this > point > > >> at the Java API level. So I see no need to add abstraction above > that. I've > > >> got prob

Re: [discuss] What to do with the Cassandra components

2024-03-20 Thread Matt Burgess
gt; should be doable to reuse the existing bundle, but rename it to the > "CQL > > > Bundle" and just add a second controller service for Scylla that is > > > otherwise 100% the same codewise. > > > > > > On Tue, Mar 19, 2024 at 6:41 PM Mike Thomsen

Re: [discuss] What to do with the Cassandra components

2024-03-19 Thread Matt Burgess
, Mar 19, 2024 at 12:37 PM Mike Thomsen < > mikerthom...@gmail.com > > > > > > >> wrote: > > > >> > > > > >> > Matt, > > > >> > > > > >> > I got that. My point was that the Java changes appear

Re: [discuss] What to do with the Cassandra components

2024-03-19 Thread Matt Burgess
en I looked at the driver, it appeared > to > > copy verbatim the entire public API of that driver. So I think before we > > dive into abstractions, it's worth doing a bit more validation of these > > details. IMHO, this might be a much lighter lift than anticipated. > >

Re: DB Dynamic Connection

2024-03-18 Thread Matt Burgess
values for db connection from a secret vault in the cloud; 2) make a DB > connection; 3) Read the data and send to processor relationship; 4) close > connection. > > On Fri, Mar 15, 2024 at 5:23 PM Matt Burgess wrote: > > > True, but my concern is that you might see performanc

Re: [discuss] What to do with the Cassandra components

2024-03-18 Thread Matt Burgess
rrent > vulnerability findings with the legacy driver, so this approach is > helpful on several fronts. > > Regards, > David Handermann > > On Mon, Mar 18, 2024 at 3:30 PM Matt Burgess wrote: > > > > Totally agree, that's what my branch does (see link in previ

Re: [discuss] What to do with the Cassandra components

2024-03-18 Thread Matt Burgess
te for new Service and Processor implementations, without > concern for uncertain compatibility questions. > > Regards, > David Handermann > > On Mon, Mar 18, 2024 at 2:35 PM Matt Burgess wrote: > > > > What do y'all think about removing the individual connect

Re: [discuss] What to do with the Cassandra components

2024-03-18 Thread Matt Burgess
se and > could have been done slightly differently but with work around. > > https://github.com/steven-matison/nifi/tree/nifi-10120-1 > > > > > > > On Fri, Mar 15, 2024 at 2:30 PM Matt Burgess wrote: > > > Oops used the wrong email address so if there have be

Re: DB Dynamic Connection

2024-03-15 Thread Matt Burgess
'll try to implement this using DBCPService Controller Interface. > > Thanks for your consideration. > > Eduardo Fontes > > On Tue, Mar 5, 2024 at 11:10 PM Matt Burgess wrote: > > > Eduardo, > > > > It doesn't sound like DBCPConnectionPoolLookup will

Re: [discuss] What to do with the Cassandra components

2024-03-15 Thread Matt Burgess
Oops used the wrong email address so if there have been responses to the Cassandra thread since mine I missed them, my bad! On Fri, Mar 15, 2024 at 2:00 PM Matt Burgess wrote: > I believe the CQL protocol is backwards compatible but the Java API is > not. For e

Re: [discuss] What to do with the Cassandra components

2024-03-15 Thread Matt Burgess
ms worth exploring. Also I dont know if the > 4.x drivers get rid of the vulnerable bits. > > Thanks > > On Fri, Mar 15, 2024 at 10:39 AM Matt Burgess > wrote: > > > At the very least we should upgrade to Cassandra 3.11.6: > > https://github.com/apache/cassandra/blob

Re: [discuss] What to do with the Cassandra components

2024-03-15 Thread Matt Burgess
At the very least we should upgrade to Cassandra 3.11.6: https://github.com/apache/cassandra/blob/cassandra-3.11.16/CHANGES.txt On Fri, Mar 15, 2024 at 1:31 PM Matt Burgess wrote: > If the community agrees to get rid of Cassandra 3 that'll save me effort > on the refactor after I ad

Re: [discuss] What to do with the Cassandra components

2024-03-15 Thread Matt Burgess
andra 3 > implies the current set of dependencies would remain too right? > > Is the current set of components we have ones we want to retain? We > certainly need Cassandra components - but are the ones we have now the > right ones? > > Thanks > Joe > > On Fri,

Re: [discuss] What to do with the Cassandra components

2024-03-15 Thread Matt Burgess
I'm actively working this, I pushed my branch up in case anyone wants to take a look [1]. The idea is to abstract the Cassandra API "up a couple levels" and provide implementations for Cassandra 3, 4, and eventually 5. For JDBC-like interfaces this is a PITA because of the API (Statement, PreparedS

Re: DB Dynamic Connection

2024-03-05 Thread Matt Burgess
Eduardo, It doesn't sound like DBCPConnectionPoolLookup will work for you because of all the different connection strings. I don't know if there's a good reason why we couldn't create the BasicDataSource when getConnection() is called, passing in a Map of FlowFile attributes (that's how the Lookup

Re: UpdateAttribute Failure Relationship

2024-02-08 Thread Matt Burgess
Mike's option #2 seems solid but would take a lot of work and there will always be inputs we don't account for. I support that work but in code sometimes we just do a "catch(Throwable)" just so it doesn't blow up. What about a subjectless "try" or "trycatch" function you can wrap around your whole

Re: [VOTE] Release Apache NiFi 2.0.0-M2 (RC4)

2024-01-26 Thread Matt Burgess
+1 (binding) Ran full build Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae) Maven home: /Users/mburgess/.sdkman/candidates/maven/current Java version: 21, vendor: Oracle Corporation Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "14.2.1", arch: "aarch6

Re: [VOTE] Release Apache NiFi 1.25.0 (RC1)

2024-01-26 Thread Matt Burgess
+1 (binding) Ran full clean install -Pcontrib-check Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae) Maven home: /Users/mburgess/.sdkman/candidates/maven/current Java version: 1.8.0_372, vendor: Azul Systems, Inc., Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x",

Re: [VOTE] Release Apache NiFi 1.24.0 (RC5)

2023-11-24 Thread Matt Burgess
+1 (binding) Ran through release helper (verifying keys and hashes and such) and tested with a handful of different flows. Thanks for seeing this through Pierre! - Matt On Thu, Nov 23, 2023 at 10:13 AM Pierre Villard wrote: > > Team, > > I am pleased to be calling this vote for the source releas

Re: [VOTE] Release Apache NiFi 1.24.0 (RC3)

2023-11-17 Thread Matt Burgess
+1 (binding) Ran through release helper, ran NiFi standalone with Java 8, tested various flows and components. Thanks for RM'ing Pierre! On Thu, Nov 16, 2023 at 2:01 PM Pierre Villard wrote: > > Team, > > I am pleased to be calling this vote for the source release of Apache NiFi > 1.24.0. > > P

Re: Removing JRuby?

2023-11-06 Thread Matt Burgess
ourse. Regards, Matt On Mon, Nov 6, 2023 at 7:12 AM Mike Thomsen wrote: > > If we deprecate ExecuteScript, I think we need to have groovyx be ready to > function as a drop-in replacement if it's not there already. > > On Sun, Nov 5, 2023 at 9:21 PM Matt Burgess wrote: >

Re: Removing JRuby?

2023-11-05 Thread Matt Burgess
IIRC the removal of these engines was mostly due to lack of use or at least the perception thereof. If JRuby is being used by the community actively, I'm happy to revisit that discussion. Luaj's JSR-223 interface left something to be desired, but JRuby just needed a system variable set or something

Re: Provenance events without FlowFiles?

2023-10-26 Thread Matt Burgess
e for example (in order to improve efficiency of > data movement between cloud providers) and, in this case, not sure we would > have a 'File' object. Unless you're talking about a more generic File > object here and not the object for local file system. > > Le jeu. 26 o

Re: Provenance events without FlowFiles?

2023-10-26 Thread Matt Burgess
AFAIK it is fine and appropriate to issue multiple provenance events for a single FlowFile. In the case for PutAzureBlobStorage uploading a file to Azure, it is the incoming FlowFile that triggers the upload. Before reporting a provenance event, attributes are added to the FlowFile, so that "versio

Re: JOLTTransformRecord problem

2023-10-10 Thread Matt Burgess
For some reason I don't have the original thread, I must've inadvertently deleted it. IIRC your example input was a single JSON object and I said if that were the case you could use JoltTransformJSON instead. However if that is NOT the case (which is your point c above) then you have a couple of op

Re: JOLTTransformRecord problem

2023-10-06 Thread Matt Burgess
mew, What version of NiFi are you using? We may have fixed the bug in newer versions but if you are using the latest, this is bug. We shouldn't infer CHOICE[STRING,RECORD] when the only entry is a RECORD. Since you only have one top-level record in your FlowFile you can use JoltTransformJSON inst

Re: How to Connect Apache NiFi to PostgreSql and using nifi insert records from csv file to database

2023-10-06 Thread Matt Burgess
Gennady, That is a great explanation by Lars, also note that by default GetFile will remove the CSV from your system. If you want to keep it but only fetch it once, use ListFile -> FetchFile rather than GetFile. Regards, Matt On Fri, Oct 6, 2023 at 12:44 PM Lars Winderling wrote: > > Hi Gennady

Re: NIFI Jira ticket not updated with link to NIFI PR code in Github

2023-09-07 Thread Matt Burgess
Dan, That happens sometimes, it can take a while for the link to be added to the Jira, but usually it does happen eventually. Sometimes I'll add the link manually then remove it when the system catches up and adds the link to the Jira. Regards, Matt On Thu, Sep 7, 2023 at 10:49 AM Dan S wrote:

Re: PGVector and Database Driver

2023-08-31 Thread Matt Burgess
java.sql.SQLException: Cannot unwrap to org.postgresql.PGConnection > > On 31.08.23 18:53, Matt Burgess wrote: > > This means the JDBC driver you're using does not support the use of > > the two-argument setObject() call when the object is a PGVector. Did > > you regist

Re: PGVector and Database Driver

2023-08-31 Thread Matt Burgess
This means the JDBC driver you're using does not support the use of the two-argument setObject() call when the object is a PGVector. Did you register the Vector type by calling: PGvector.addVectorType(conn); The documentation [1] says that the two-argument setObject() should work if you have regi

Re: Jira contributor access

2023-08-27 Thread Matt Burgess
Umar, Welcome to the NiFi project! Please use the Self-Service Portal at https://selfserve.apache.org/jira-account.html to request access to Jira per the current Apache rules. A PMC member will review and approve as prudent. Thank you, Matt On Sun, Aug 27, 2023 at 3:35 PM Umar Hussain wrote: >

Re: Apache NiFi compatibility with SQL Server 2008 R2

2023-08-24 Thread Matt Burgess
Frank, Yes NiFi is compatible with MSSQL 2008, there's even a specific database adapter in some processors to generate the correct SQL for it (it's called "MSSQL2008DatabaseAdapter"). Regards, Matt On Thu, Aug 24, 2023 at 11:23 AM Frank Mansilla wrote: > > Dear Apache NiFi Community, > > I am i

Re: FileMaker and Apache NiFi

2023-08-23 Thread Matt Burgess
Samuel, Where do you see that DBCPConnectionPool uses JNDI? As far as I know it just uses Apache DBCP and JDBC DataSources using the supplied driver. Regards, Matt On Wed, Aug 23, 2023 at 9:14 AM Namazi, Samuel wrote: > > To whom it may concern, > > I am trying to connect my FileMaker database

Re: [VOTE] Release Apache NiFi 1.23.2 (RC1)

2023-08-22 Thread Matt Burgess
+1 (binding) Ran through release helper and my usual flows verifying no regressions. Thanks for RM'ing David! On Mon, Aug 21, 2023 at 4:16 PM David Handermann wrote: > > Team, > > NOTE: This is a shortened vote window given the narrow scope of changes. > > I am pleased to be calling this vote fo

Re: [DISCUSS] Release Apache NiFi 1.23.2 for NIFI-11971

2023-08-21 Thread Matt Burgess
+1 On Mon, Aug 21, 2023 at 9:58 AM David Handermann wrote: > > Team, > > Thanks for the work on release for NiFi 1.23.1 last week. > > As it happens, another important bug surfaced last week with Content > Repository handling of empty files, described in NIFI-11971 [1]. This > bug was introduced

Re: JoltTransformJSON EL when using file

2023-08-17 Thread Matt Burgess
; manual schemas and using postgres as the perm data store. I'm using json as > mqtt does it intrinsically and I can jolt transform other data into json > form and then store it easily in postgres. Well, that was the idea 😁 > > Kr, > > On Sun, 13 Aug 2023, 10:44 Matt Burgess, wro

Re: [VOTE] Release Apache NiFi 1.23.1 (RC1)

2023-08-16 Thread Matt Burgess
+1 (binding) Ran through release helper, verified NIFI-11922 [1] and stored flows in Registry. Apache Maven 3.9.1 (2e178502fcdbffc201671fb2537d0cb4b4cc58f8) Java version: 17.0.7, vendor: Oracle Corporation, runtime: /Users/mburgess/.sdkman/candidates/java/17.0.7-oracle Default locale: en_US, plat

Re: [DISCUSS] Prepare Apache NiFi 1.23.1 Release?

2023-08-13 Thread Matt Burgess
+1, thank you David! On Sat, Aug 12, 2023 at 10:26 AM David Handermann wrote: > > Team, > > Following the release of Apache NiFi 1.23.0, several important bugs have > been resolved in a few Processors [1], and there have been a handful of > incremental dependency upgrades [2]. Preparing an increm

Re: JoltTransformJSON EL when using file

2023-08-12 Thread Matt Burgess
Just to follow up, I added a unit test to put EL in a JOLT spec and it worked. I noticed you referred to "attrname" in your post but your spec refers to "firstname", is that a typo? Regards, Matt On Thu, Aug 10, 2023 at 3:03 PM Matt Burgess wrote: > > I added file su

Re: JoltTransformJSON EL when using file

2023-08-10 Thread Matt Burgess
I added file support to JoltTransformJSON under NIFI-4957 [1], a first glance at the code seems like it should work, but I'll try to reproduce it and follow up, thanks for bringing this to our attention! Regards, Matt [1] https://issues.apache.org/jira/browse/NIFI-4957 On Thu, Aug 10, 2023 at 6:

Re: Java 17 features in 2.x

2023-08-07 Thread Matt Burgess
few Java 11 and 17 references on the main branch for > > things like List.of(), and most of these are easy to adjust when > > backporting, but they do require careful attention. > > > > Regards, > > David Handermann > > > > On Mon, Aug 7, 2023 at 4:04 PM Matt B

Re: Java 17 features in 2.x

2023-08-07 Thread Matt Burgess
In my opinion that's ok, but I think it would be helpful if a PR is going to be backported to support/nifi-1.x that the PR author provides two PRs, one against main with Java 17 features and one against support/nifi-1.x with Java 8 features. That being said, allowing Java 17 features may make maint

Re: [VOTE] Release Apache NiFi 1.23.0 (RC3)

2023-07-26 Thread Matt Burgess
+1 (binding) Ran through release helper, verified H2 migration on Registry, ExtractRecordSchema, various other flows to ensure no regressions. Thanks for RM'ing David! On Tue, Jul 25, 2023 at 5:33 PM David Handermann wrote: > > Team, > > I am pleased to be calling this vote for the source relea

Re: [VOTE] Release Apache NiFi 1.23.0 (RC2)

2023-07-20 Thread Matt Burgess
+1 (binding), verified hashes, keys, and commit IDs, ran flows to verify NIFI-11783 [1], NIFI-11807 [2], and verified no regression in NIFI-11621 [3]. Apache Maven 3.9.1 (2e178502fcdbffc201671fb2537d0cb4b4cc58f8) Maven home: /Users/mburgess/.sdkman/candidates/maven/current Java version: 17.0.7, ve

Re: Use of attribute uuid and other "native" attributes

2023-07-18 Thread Matt Burgess
In general I recommend only sending on those attributes that will be used at some point downstream (unless you have an "original" relationship that should maintain the original state with respect to provenance). If you don't know that ahead of time you'll probably need to send all/most of the attri

Re: board report for July 2023

2023-07-05 Thread Matt Burgess
Looks good to me, this project always has impressive things to report to the board and they have consistently appreciated it as well, it clearly indicates a healthy and active community. Thanks for this! On Tue, Jul 4, 2023 at 12:52 PM Joe Witt wrote: > > Team, > > Here is the submitted board rep

Re: Jira contributor access

2023-07-04 Thread Matt Burgess
It appears you have an existing Jira account [1], I have added you as a Contributor to the NiFi projects in Jira. If you cannot log into your Jira account and cannot reset your password I believe you have to create a new one via [2]. Regards, Matt [1] https://issues.apache.org/jira/secure/ViewPr

Re: [VOTE] Release Apache NiFi NAR Maven Plugin 1.5.1

2023-06-15 Thread Matt Burgess
+1 (binding) Verified signatures and hashes, built with the system below, built NiFi NARs and verified the issue I was encountering is fixed. Thanks for RM'ing Nandor! Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39) Maven home: /Users/mburgess/.sdkman/candidates/maven/current Java v

Re: [DISCUSS] Release NiFi NAR Maven Plugin 1.5.1

2023-06-13 Thread Matt Burgess
+1 from me, I ran into the issue so would like to have that fix available. Thanks for volunteering to RM! On Mon, Jun 12, 2023 at 5:36 AM Nandor Soma Abonyi wrote: > > Hello Apache NiFi Community, > > I'd like to initiate a discussion about the next release of > nifi-nar-maven-plugin. > > Curre

Re: [VOTE] Release Apache NiFi 1.22.0 (RC1)

2023-06-07 Thread Matt Burgess
+1 (binding) Ran through release helper and tested various flows including verifying NIFI-11621 [1] for nested array inference and NIFI-11653 [2] for removing H2 support for DBCPConnectionPool. Also tested some version control capability for NiFi Registry. Apache Maven 3.8.2 (ea98e05a04480131370a

Re: User-defined ProvenanceReporter

2023-04-30 Thread Matt Burgess
All, We've been discussing this in Slack [1], we can continue there and if anyone is interested and not on our Apache Slack channel, I encourage you to join [2] but I can also copy-paste into this thread if need be. Regards, Matt [1] https://apachenifi.slack.com/archives/C0L9VCD47/p1682895885163

Re: Issue upgrading Nifi(1.7.0) & Minifi(0.5.0) to 1.20.0

2023-04-29 Thread Matt Burgess
Ignasi, A lot has changed between MiNiFi 0.5.0 and MiNiFi 1.20.0, including changes to NARs, how they are loaded, etc. For custom NARs I would try to rebuild them using the same version of NiFi and MiNiFi dependencies. The latest Apache NiFi release is now 1.21.0 so I would change the NiFi and MiN

Re: [VOTE] Release Apache NiFi 1.21.0 (RC2)

2023-04-05 Thread Matt Burgess
+1 (binding), ran through the release helper and a few flows (including one with the CaptureChangeMySQL changes) on a Mac with Java 11. Thanks for RM'ing Joe! On Mon, Apr 3, 2023 at 8:11 PM Joe Witt wrote: > > Hello, > > I am pleased to be calling this vote for the source release of Apache > NiF

Re: [discuss] NiFi support for Hadoop ecosystem components

2023-03-24 Thread Matt Burgess
As one of the small number of people that fight the battle, I like the idea of Option 1 (full disclosure: I work for a vendor). From a community standpoint (I'm on the PMC) I'm not strongly opposed to Option 2 although I wouldn't want to be the one managing and releasing the artifacts :) Having sai

Re: [VOTE] Release Apache NiFi NAR Maven Plugin 1.5.0 - RC2

2023-03-14 Thread Matt Burgess
+1 (binding) Ran through release helper, built NiFi bundles/NARs using this version of the plugin, all looks good. Thanks for RM'ing Kevin! On Tue, Mar 14, 2023 at 5:00 PM Kevin Doran wrote: > > Hello Apache NiFi Community, > > Issues with RC1 have been addressed in RC2, and I am pleased to agai

Re: [VOTE] Release Apache NiFi 1.20.0 (RC1)

2023-02-07 Thread Matt Burgess
+1 (binding) Ran through the release helper and a few flows. I did see this as a race condition for InvokeScriptedProcessor but it fixes itself on the first validation so I don't think we need to tank the RC as a result, I wrote up [1] to investigate: 2023-02-07 16:26:39,825 ERROR [NiFi Web Serve

Re: [VOTE] Release Apache NiFi NAR Maven Plugin 1.4.0

2023-02-01 Thread Matt Burgess
+1 binding, ran through release helper (noticed the tag hadn't been pushed but is now, thanks!) Built NiFi with 1.4.0-RC1 plugin, verified all worked successfully. Thanks for RM'ing Kevin! On Mon, Jan 30, 2023 at 6:19 PM Kevin Doran wrote: > > Hello Apache NiFi Community, > > I am pleased to be c

Re: Line number to identify where CSVReader failed

2023-01-19 Thread Matt Burgess
I was thinking the same, CSVRecordReader could keep track of the number of records read and if an exception is thrown during iteration over reading the records, we can output the number of records read successfully. On Thu, Jan 19, 2023 at 3:47 PM Joe Witt wrote: > > Dan, > > Seems like our recor

Re: Getting a NiFi Jira account

2023-01-16 Thread Matt Burgess
Kim, According to the new Jira guidelines [1], please submit the following information to priv...@nifi.apache.org and the PMC will create an account for you and add you as a Contributor to the NiFi Jira projects: email address preferred username (N.B. hyphens not allowed) alternate username (in c

Re: nifi proxy issue + jira account

2022-12-15 Thread Matt Burgess
Kyle, Per the new Apache Jira instructions [1], please provide the following information to priv...@nifi.apache.org and I can create an account for you and add you as a contributor: - email address - preferred username (N.B. hyphens not allowed) - alternate username (in case the preferred one is

Re: Log Cleansing

2022-12-14 Thread Matt Burgess
Bryan, Do you have an example of the kind of cleansing you'd like to do? NiFi has things like GrokReader [1], syslog parsers and such to parse logs, and the corresponding writers to format the log text the way you want. Regards, Matt [1] https://nifi.apache.org/docs/nifi-docs/components/org.apa

Re: [VOTE] Adopt NiFi 2.0 Proposed Release Goals

2022-12-12 Thread Matt Burgess
+1 (binding) On Mon, Dec 12, 2022 at 12:02 PM David Handermann wrote: > > Team, > > Following positive feedback on NiFi 2.0 Proposed Release Goals [1] on the > recent discussion thread [2], I am calling this vote to adopt the following > as Release Goals for NiFi 2.0: > > 1. Remove Java 8 support

Re: is cdc db2

2022-12-08 Thread Matt Burgess
Sagar, There is an open Jira for SQL Server CDC [1] which may have been abandoned waiting for review. I can take a look at the Pull Request but it would clearly need changes to match the current version of NiFi at the very least. I couldn't find a Jira for a CDC processor for DB2, but feel free to

Re: [VOTE] Release Apache NiFi 1.19.1 (RC2)

2022-12-06 Thread Matt Burgess
+1 (binding) Ran through release helper, tested a handful of flows including one with the new UpdateDatabaseTable processor. I verified the existence of NIFI-10956 in this release; however it has been in multiple releases and I put a PR up, IMO it is not a blocker and can wait until the next relea

Re: [VOTE] Release Apache NiFi 1.19.0 (RC1)

2022-11-27 Thread Matt Burgess
+1 (binding) Ran through release helper, tested a couple of flows including one with the new UpdateDatabaseTable processor. Thanks for RM'ing Joe! On Wed, Nov 23, 2022 at 10:58 AM Joe Witt wrote: > > Hello, > > I am pleased to be calling this vote for the source release of Apache NiFi > 1.19.0. >

Re: [discuss] NiFi 1.19.0

2022-11-15 Thread Matt Burgess
Awesome, thanks all! I'd like to wait for Github actions to be happy and then merge? On Tue, Nov 15, 2022 at 7:11 PM Mark Bean wrote: > > NIFI-10703 has been worked through. I think it's good to go now. > > On Tue, Nov 15, 2022 at 4:43 PM Matt Burgess wrote: > > >

Re: [discuss] NiFi 1.19.0

2022-11-15 Thread Matt Burgess
I can take that review too, if your suggestions are incorporated. I started looking the other day but didn't get to dig in. I'll try to reproduce in the meantime. Regards, Matt On Tue, Nov 15, 2022 at 4:42 PM Nathan Gough wrote: > > We might also want https://issues.apache.org/jira/browse/NIFI-1

Re: [discuss] NiFi 1.19.0

2022-11-15 Thread Matt Burgess
Mark, I can review any incoming changes when they land, if that helps. Regards, Matt On Tue, Nov 15, 2022 at 4:05 PM Mark Bean wrote: > > I will be on it in about 2 hours, if not addressed sooner. > > On Tue, Nov 15, 2022 at 3:46 PM Joe Witt wrote: > > > NIFI-10703 would be great to get in. J

Re: Adding Kotlin support to the scripting bundle

2022-11-07 Thread Matt Burgess
Mike, As an alternative it might be worth looking into a processor specifically for Kotlin like we have with ExecuteGroovyScript (EGS). ExecuteGroovyScript doesn't use JSR-223 and instead goes right for the GroovyShell. This allows for cool things like Groovy-specific idioms and helpful "meta-capa

Re: Jira contributor access

2022-10-03 Thread Matt Burgess
Mathew, I have added you as a Contributor to the NiFi projects in Jira. Looking forward to your contributions! Regards, Matt On Mon, Oct 3, 2022 at 2:25 AM Mathew Kiprop wrote: > > My Jira username is '*mkapkiai*' > > On Mon, Oct 3, 2022 at 9:21 AM Mathew Kiprop > wrote: > > > Hello Apache NiF

Re: [VOTE] Release Apache NiFi 1.18.0 (RC4)

2022-10-03 Thread Matt Burgess
+1 binding Ran through release helper, verified NIFI-10568 (statefulness added to runtime component manifests) and NIFI-9042 (DatabaseParameterProvider) and ran a couple of flows exercising various components such as record-based processors and controller services. Thanks for RM'ing Joe! On Mon

Re: [VOTE] Release Apache NiFi 1.18.0 (RC3)

2022-10-01 Thread Matt Burgess
+1 (binding) Ran through release helper, tested DatabaseParameterProvider and various flows. Thanks for RM'ing Joe! On Thu, Sep 29, 2022 at 3:22 PM Joe Witt wrote: > > Hello, > > I am pleased to be calling this vote for the source release of Apache > NiFi 1.18.0. > > The source zip, including si

Re: New NiFi PMC Member Nathan Gough

2022-09-07 Thread Matt Burgess
Congratulations Nathan! Well deserved. On Wed, Sep 7, 2022 at 12:10 PM David Handermann wrote: > > Apache NiFi Community, > > On behalf of the Apache NiFi Project Management Committee, I am very > pleased to announce that Nathan Gough has accepted the invitation to join > the PMC. > > Nathan has

Re: [VOTE] Release Apache NiFi 1.17.0 (RC2)

2022-07-29 Thread Matt Burgess
+1 (binding) Ran through release helper and tested various flows. Thanks for RM'ing Joe! On Thu, Jul 28, 2022 at 10:29 AM Joe Witt wrote: > > Hello, > > I am pleased to be calling this vote for the source release of Apache NiFi > 1.17.0. > > The source zip, including signatures, digests, etc. ca

Re: [VOTE] Release Apache NiFi NAR Maven Plugin 1.3.5

2022-07-21 Thread Matt Burgess
+1 (binding) Built on OSX 10.15.7 with OpenJDK Runtime Environment Zulu11.54+23-CA (build 11.0.14+9-LTS), ran NiFi build and verified manifest changes. Thanks for RM'ing Bryan! On Wed, Jul 20, 2022 at 4:37 PM Bryan Bende wrote: > > Hello Apache NiFi Community, > > I am pleased to be calling this

Re: Jira contributor access

2022-07-13 Thread Matt Burgess
Kirill, I have added you as a contributor to the Apache NiFi Jira projects. Looking forward to your contributions! Regards, Matt On Mon, Jul 11, 2022 at 2:11 AM Kirill Morozov wrote: > > Hi, > could anyone help me with this ? > > Thank you! > > чт, 7 июл. 2022 г. в 16:05, Kirill Morozov : > > >

Re: [VOTE] Release Apache NiFi NAR Maven Plugin 1.3.4

2022-06-17 Thread Matt Burgess
+1 (binding) Checked commit and SHAs, built with Java 1.8 on Mac, built NiFi from main with the RC Maven plugin. Thanks for RM'ing Kevin! On Thu, Jun 16, 2022 at 5:11 PM Kevin Doran wrote: > > Hello Apache NiFi Community, > > I am pleased to be calling this vote for the source release of Apache >

  1   2   3   4   5   6   >