RE: [EXT] Re: [VOTE] Create NiFi Standard Libraries sub-project
+1, binding -Original Message- From: Kevin Doran Sent: Tuesday, September 3, 2019 7:12 PM To: dev@nifi.apache.org; dev@nifi.apache.org Subject: [EXT] Re: [VOTE] Create NiFi Standard Libraries sub-project +1, binding From: Tony Kurc Sent: Tuesday, September 3, 2019 8:33 PM To: dev@nifi.apache.org Subject: Re: [VOTE] Create NiFi Standard Libraries sub-project +1 (binding) On Wed, Sep 4, 2019 at 12:29 AM Aldrin Piri wrote: > +1, binding > > On Tue, Sep 3, 2019 at 19:46 Yolanda Davis > wrote: > > > +1 Create NiFi Standard Libraries (binding) > > > > On Tue, Sep 3, 2019 at 7:03 PM Koji Kawamura > > > > wrote: > > > > > +1 Create NiFi Standard Libraries (binding) > > > > > > On Wed, Sep 4, 2019 at 7:25 AM Mike Thomsen > > > > > > wrote: > > > > > > > > +1 binding > > > > > > > > On Tue, Sep 3, 2019 at 5:33 PM Andy LoPresto > > > > > > > wrote: > > > > > > > > > +1, create NiFi Standard Libraries (binding) > > > > > > > > > > Andy LoPresto > > > > > alopre...@apache.org > > > > > alopresto.apa...@gmail.com > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D > > > > > EF69 > > > > > > > > > > > On Sep 3, 2019, at 2:16 PM, Bryan Bende > wrote: > > > > > > > > > > > > All, > > > > > > > > > > > > In a previous thread there was a plan discussed to > > > > > > restructure > some > > > of > > > > > > the repositories in order to address several different > > > > > > issues, > such > > > as > > > > > > build time, reusability of code, and eventually separating > > > > > > how > the > > > > > > framework and extensions are released [1][2]. > > > > > > > > > > > > The overall plan requires many steps to get there, so I'd > > > > > > like to propose starting with a small actionable step - the > > > > > > creation of a > > new > > > > > > sub-project called NiFi Standard Libraries (formerly > > > > > > referred to > as > > > > > > nifi-commons). > > > > > > > > > > > > Project Name: Apache NiFi Standard Libraries Git Repository: > > > > > > nifi-standard-libraries > > > > > > JIRA: NIFILIBS > > > > > > > > > > > > Description: > > > > > > > > > > > > A collection of standard implementations used across the > > > > > > NiFi > > > ecosystem. > > > > > > > > > > > > Candidate Libraries: > > > > > > > > > > > > In general, each library may consist of multiple Maven > > > > > > modules, > and > > > > > > should be independent from the rest of the ecosystem, and > > > > > > from > > other > > > > > > libraries within NiFi Standard Libraries. > > > > > > > > > > > > In addition, each library may make it's own decision about > whether > > it > > > > > > is considered a public facing extension point/API, or an > > > > > > internal library that may be changed at any time. This > > > > > > should be > documented > > in > > > > > > a README at the root of each library, such as > > > > > > nifi-standard-libraries/nifi-xyz/README. > > > > > > > > > > > > An initial library that has been discussed was referred to > > > > > > as 'nifi-security' and would centralize much of the security > > > > > > related > > > code > > > > > > shared by NiFi and NiFi Registry, such as shared security > > > > > > APIs, > and > > > > > > implementations for various providers, such as LDAP/Kerberos/etc. > > > > > > > > > > > > A second candidate library would be an optimistic-locking > > > > > > library based on NiFi's revision concept. Currently this has > > > > > > been created inside nifi-registry for now [3], but could be > > > > > > moved as soon as nifi-standard-libraries exists. > > > > > > > > > > > > (This list does not have to be final in order to decide if > > > > > > we are creating NiFi Standard Libraries or not) > > > > > > > > > > > > Integration & Usage: > > > > > > > > > > > > Once NiFi Standard Libraries is created, the community can > > > > > > start creating and/or moving code there and perform releases > > > > > > as > > necessary. > > > A > > > > > > release will consist of the standard Apache source release, > > > > > > plus artifacts released to Maven central. The community can > > > > > > then > decide > > > > > > when it is appropriate to integrate these released libraries > > > > > > into > > one > > > > > > of our downstream projects. > > > > > > > > > > > > For example, if we create a nifi-security library in > > > > > > nifi-standard-libraries, we can release that whenever we > > > > > > decide, > > but > > > > > > we may not integrate it into NiFi or NiFi Registry until it > > > > > > makes sense for a given release of those projects. > > > > > > > > > > > > This vote will be open for 48 hours, please vote: > > > > > > > > > > > > [ ] +1 Create NiFi Standard Libraries [ ] +0 no opinion [ ] > > > > > > -1 Do not create NiFi Standard Libraries because... > > > > > > > > > > > > [1] > > > > > > > > > > > https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapach >
Re: [VOTE] Create NiFi Standard Libraries sub-project
+1, binding From: Tony Kurc Sent: Tuesday, September 3, 2019 8:33 PM To: dev@nifi.apache.org Subject: Re: [VOTE] Create NiFi Standard Libraries sub-project +1 (binding) On Wed, Sep 4, 2019 at 12:29 AM Aldrin Piri wrote: > +1, binding > > On Tue, Sep 3, 2019 at 19:46 Yolanda Davis > wrote: > > > +1 Create NiFi Standard Libraries (binding) > > > > On Tue, Sep 3, 2019 at 7:03 PM Koji Kawamura > > wrote: > > > > > +1 Create NiFi Standard Libraries (binding) > > > > > > On Wed, Sep 4, 2019 at 7:25 AM Mike Thomsen > > > wrote: > > > > > > > > +1 binding > > > > > > > > On Tue, Sep 3, 2019 at 5:33 PM Andy LoPresto > > > wrote: > > > > > > > > > +1, create NiFi Standard Libraries (binding) > > > > > > > > > > Andy LoPresto > > > > > alopre...@apache.org > > > > > alopresto.apa...@gmail.com > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > > > > > > > > On Sep 3, 2019, at 2:16 PM, Bryan Bende > wrote: > > > > > > > > > > > > All, > > > > > > > > > > > > In a previous thread there was a plan discussed to restructure > some > > > of > > > > > > the repositories in order to address several different issues, > such > > > as > > > > > > build time, reusability of code, and eventually separating how > the > > > > > > framework and extensions are released [1][2]. > > > > > > > > > > > > The overall plan requires many steps to get there, so I'd like to > > > > > > propose starting with a small actionable step - the creation of a > > new > > > > > > sub-project called NiFi Standard Libraries (formerly referred to > as > > > > > > nifi-commons). > > > > > > > > > > > > Project Name: Apache NiFi Standard Libraries > > > > > > Git Repository: nifi-standard-libraries > > > > > > JIRA: NIFILIBS > > > > > > > > > > > > Description: > > > > > > > > > > > > A collection of standard implementations used across the NiFi > > > ecosystem. > > > > > > > > > > > > Candidate Libraries: > > > > > > > > > > > > In general, each library may consist of multiple Maven modules, > and > > > > > > should be independent from the rest of the ecosystem, and from > > other > > > > > > libraries within NiFi Standard Libraries. > > > > > > > > > > > > In addition, each library may make it's own decision about > whether > > it > > > > > > is considered a public facing extension point/API, or an internal > > > > > > library that may be changed at any time. This should be > documented > > in > > > > > > a README at the root of each library, such as > > > > > > nifi-standard-libraries/nifi-xyz/README. > > > > > > > > > > > > An initial library that has been discussed was referred to as > > > > > > 'nifi-security' and would centralize much of the security related > > > code > > > > > > shared by NiFi and NiFi Registry, such as shared security APIs, > and > > > > > > implementations for various providers, such as LDAP/Kerberos/etc. > > > > > > > > > > > > A second candidate library would be an optimistic-locking library > > > > > > based on NiFi's revision concept. Currently this has been created > > > > > > inside nifi-registry for now [3], but could be moved as soon as > > > > > > nifi-standard-libraries exists. > > > > > > > > > > > > (This list does not have to be final in order to decide if we are > > > > > > creating NiFi Standard Libraries or not) > > > > > > > > > > > > Integration & Usage: > > > > > > > > > > > > Once NiFi Standard Libraries is created, the community can start > > > > > > creating and/or moving code there and perform releases as > > necessary. > > > A > > > > > > release will consist of the standard Apache source release, plus > > > > > > artifacts released to Maven central. The community can then > decide > > > > > > when it is appropriate to integrate these released libraries into > > one > > > > > > of our downstream projects. > > > > > > > > > > > > For example, if we create a nifi-security library in > > > > > > nifi-standard-libraries, we can release that whenever we decide, > > but > > > > > > we may not integrate it into NiFi or NiFi Registry until it makes > > > > > > sense for a given release of those projects. > > > > > > > > > > > > This vote will be open for 48 hours, please vote: > > > > > > > > > > > > [ ] +1 Create NiFi Standard Libraries > > > > > > [ ] +0 no opinion > > > > > > [ ] -1 Do not create NiFi Standard Libraries because... > > > > > > > > > > > > [1] > > > > > > > > > > > http://apache-nifi.1125220.n5.nabble.com/discuss-Splitting-NiFi-framework-and-extension-repos-and-releases-td27499.html > > > > > > [2] > > > > > > > > > > > https://cwiki.apache.org/confluence/display/NIFIREG/NiFi+Project+and+Repository+Restructuring > > > > > > [3] > > > > > > > > > > > https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-revision > > > > > > > > > > > > > > > > > > > -- > > -- > > yolanda.m.da...@gmail.com > > @YolandaMDavis > > >
Re: File-based Versioned Flow Representation
I will throw a vote for JSON-based representation moving forward as well, with the understanding that we now have three different codebases which convert from a serialized, human-readable format (XML, YAML, JSON) into the hydrated flow state object and back, and we should ensure through unit, integration, and regression tests that all three function identically (or at least with expected and documented differences). In addition, we should organize those codebases intentionally so that changes to one are applied to the others if relevant, and perhaps organize and expose them in such a way that easy tooling is available around them to convert the various serialized forms to another. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Sep 3, 2019, at 8:16 AM, Kevin Doran wrote: > > I agree I think that the JSON-serialized version flow snapshot in > Registry, which is also supported by CLI and MiNiFi Toolkit, should > become the new "de facto" for portable/file flow format, including > used in places that templates used to be used. > > +1 for adding import/export capabilities for this format in NiFi > similar to the way templates work today. It would be nice if the "Add > Process Group..." dialog had additional options other than import from > NiFi Registry, such as import from file or even just a text box the > JSON representation could be pasted into. > > On Fri, Aug 30, 2019 at 9:05 PM Bryan Bende wrote: >> >> I’d like to see us move towards the versioned flow representation from >> registry and add features to NiFi like right click on PG and export to >> versioned flow, import from a file etc. >> >> The Minifi java toolkit already has a transform from versioned flow JSON to >> YAML, just like the template transform. >> >> >> On Fri, Aug 30, 2019 at 4:05 PM Samuel Hjelmfelt >> wrote: >> >>> Hello everyone,NIFI-6539 brings up the topic of deployment scenarios when >>> a NiFi registry is not available. MiNiFi allows configuration via a YML >>> file-based representation of a flow, often created from an exported NiFi >>> template. However, there has been a push to focus on versioned flows in the >>> registry and to move away from templates. >>> Long term, what is the plan for file-based flow representations? Is the >>> intent to continue using templates even in NiFi 2.x, or is there some other >>> solution planned? Both deployments without a registry and registry >>> import/export will require some kind of file-based flow representation. >>> -Sam >>> >> -- >> Sent from Gmail Mobile
Re: [VOTE] Create NiFi Standard Libraries sub-project
+1 (binding) On Wed, Sep 4, 2019 at 12:29 AM Aldrin Piri wrote: > +1, binding > > On Tue, Sep 3, 2019 at 19:46 Yolanda Davis > wrote: > > > +1 Create NiFi Standard Libraries (binding) > > > > On Tue, Sep 3, 2019 at 7:03 PM Koji Kawamura > > wrote: > > > > > +1 Create NiFi Standard Libraries (binding) > > > > > > On Wed, Sep 4, 2019 at 7:25 AM Mike Thomsen > > > wrote: > > > > > > > > +1 binding > > > > > > > > On Tue, Sep 3, 2019 at 5:33 PM Andy LoPresto > > > wrote: > > > > > > > > > +1, create NiFi Standard Libraries (binding) > > > > > > > > > > Andy LoPresto > > > > > alopre...@apache.org > > > > > alopresto.apa...@gmail.com > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > > > > > > > > On Sep 3, 2019, at 2:16 PM, Bryan Bende > wrote: > > > > > > > > > > > > All, > > > > > > > > > > > > In a previous thread there was a plan discussed to restructure > some > > > of > > > > > > the repositories in order to address several different issues, > such > > > as > > > > > > build time, reusability of code, and eventually separating how > the > > > > > > framework and extensions are released [1][2]. > > > > > > > > > > > > The overall plan requires many steps to get there, so I'd like to > > > > > > propose starting with a small actionable step - the creation of a > > new > > > > > > sub-project called NiFi Standard Libraries (formerly referred to > as > > > > > > nifi-commons). > > > > > > > > > > > > Project Name: Apache NiFi Standard Libraries > > > > > > Git Repository: nifi-standard-libraries > > > > > > JIRA: NIFILIBS > > > > > > > > > > > > Description: > > > > > > > > > > > > A collection of standard implementations used across the NiFi > > > ecosystem. > > > > > > > > > > > > Candidate Libraries: > > > > > > > > > > > > In general, each library may consist of multiple Maven modules, > and > > > > > > should be independent from the rest of the ecosystem, and from > > other > > > > > > libraries within NiFi Standard Libraries. > > > > > > > > > > > > In addition, each library may make it's own decision about > whether > > it > > > > > > is considered a public facing extension point/API, or an internal > > > > > > library that may be changed at any time. This should be > documented > > in > > > > > > a README at the root of each library, such as > > > > > > nifi-standard-libraries/nifi-xyz/README. > > > > > > > > > > > > An initial library that has been discussed was referred to as > > > > > > 'nifi-security' and would centralize much of the security related > > > code > > > > > > shared by NiFi and NiFi Registry, such as shared security APIs, > and > > > > > > implementations for various providers, such as LDAP/Kerberos/etc. > > > > > > > > > > > > A second candidate library would be an optimistic-locking library > > > > > > based on NiFi's revision concept. Currently this has been created > > > > > > inside nifi-registry for now [3], but could be moved as soon as > > > > > > nifi-standard-libraries exists. > > > > > > > > > > > > (This list does not have to be final in order to decide if we are > > > > > > creating NiFi Standard Libraries or not) > > > > > > > > > > > > Integration & Usage: > > > > > > > > > > > > Once NiFi Standard Libraries is created, the community can start > > > > > > creating and/or moving code there and perform releases as > > necessary. > > > A > > > > > > release will consist of the standard Apache source release, plus > > > > > > artifacts released to Maven central. The community can then > decide > > > > > > when it is appropriate to integrate these released libraries into > > one > > > > > > of our downstream projects. > > > > > > > > > > > > For example, if we create a nifi-security library in > > > > > > nifi-standard-libraries, we can release that whenever we decide, > > but > > > > > > we may not integrate it into NiFi or NiFi Registry until it makes > > > > > > sense for a given release of those projects. > > > > > > > > > > > > This vote will be open for 48 hours, please vote: > > > > > > > > > > > > [ ] +1 Create NiFi Standard Libraries > > > > > > [ ] +0 no opinion > > > > > > [ ] -1 Do not create NiFi Standard Libraries because... > > > > > > > > > > > > [1] > > > > > > > > > > > http://apache-nifi.1125220.n5.nabble.com/discuss-Splitting-NiFi-framework-and-extension-repos-and-releases-td27499.html > > > > > > [2] > > > > > > > > > > > https://cwiki.apache.org/confluence/display/NIFIREG/NiFi+Project+and+Repository+Restructuring > > > > > > [3] > > > > > > > > > > > https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-revision > > > > > > > > > > > > > > > > > > > -- > > -- > > yolanda.m.da...@gmail.com > > @YolandaMDavis > > >
Re: [VOTE] Create NiFi Standard Libraries sub-project
+1, binding On Tue, Sep 3, 2019 at 19:46 Yolanda Davis wrote: > +1 Create NiFi Standard Libraries (binding) > > On Tue, Sep 3, 2019 at 7:03 PM Koji Kawamura > wrote: > > > +1 Create NiFi Standard Libraries (binding) > > > > On Wed, Sep 4, 2019 at 7:25 AM Mike Thomsen > > wrote: > > > > > > +1 binding > > > > > > On Tue, Sep 3, 2019 at 5:33 PM Andy LoPresto > > wrote: > > > > > > > +1, create NiFi Standard Libraries (binding) > > > > > > > > Andy LoPresto > > > > alopre...@apache.org > > > > alopresto.apa...@gmail.com > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > > > > > > On Sep 3, 2019, at 2:16 PM, Bryan Bende wrote: > > > > > > > > > > All, > > > > > > > > > > In a previous thread there was a plan discussed to restructure some > > of > > > > > the repositories in order to address several different issues, such > > as > > > > > build time, reusability of code, and eventually separating how the > > > > > framework and extensions are released [1][2]. > > > > > > > > > > The overall plan requires many steps to get there, so I'd like to > > > > > propose starting with a small actionable step - the creation of a > new > > > > > sub-project called NiFi Standard Libraries (formerly referred to as > > > > > nifi-commons). > > > > > > > > > > Project Name: Apache NiFi Standard Libraries > > > > > Git Repository: nifi-standard-libraries > > > > > JIRA: NIFILIBS > > > > > > > > > > Description: > > > > > > > > > > A collection of standard implementations used across the NiFi > > ecosystem. > > > > > > > > > > Candidate Libraries: > > > > > > > > > > In general, each library may consist of multiple Maven modules, and > > > > > should be independent from the rest of the ecosystem, and from > other > > > > > libraries within NiFi Standard Libraries. > > > > > > > > > > In addition, each library may make it's own decision about whether > it > > > > > is considered a public facing extension point/API, or an internal > > > > > library that may be changed at any time. This should be documented > in > > > > > a README at the root of each library, such as > > > > > nifi-standard-libraries/nifi-xyz/README. > > > > > > > > > > An initial library that has been discussed was referred to as > > > > > 'nifi-security' and would centralize much of the security related > > code > > > > > shared by NiFi and NiFi Registry, such as shared security APIs, and > > > > > implementations for various providers, such as LDAP/Kerberos/etc. > > > > > > > > > > A second candidate library would be an optimistic-locking library > > > > > based on NiFi's revision concept. Currently this has been created > > > > > inside nifi-registry for now [3], but could be moved as soon as > > > > > nifi-standard-libraries exists. > > > > > > > > > > (This list does not have to be final in order to decide if we are > > > > > creating NiFi Standard Libraries or not) > > > > > > > > > > Integration & Usage: > > > > > > > > > > Once NiFi Standard Libraries is created, the community can start > > > > > creating and/or moving code there and perform releases as > necessary. > > A > > > > > release will consist of the standard Apache source release, plus > > > > > artifacts released to Maven central. The community can then decide > > > > > when it is appropriate to integrate these released libraries into > one > > > > > of our downstream projects. > > > > > > > > > > For example, if we create a nifi-security library in > > > > > nifi-standard-libraries, we can release that whenever we decide, > but > > > > > we may not integrate it into NiFi or NiFi Registry until it makes > > > > > sense for a given release of those projects. > > > > > > > > > > This vote will be open for 48 hours, please vote: > > > > > > > > > > [ ] +1 Create NiFi Standard Libraries > > > > > [ ] +0 no opinion > > > > > [ ] -1 Do not create NiFi Standard Libraries because... > > > > > > > > > > [1] > > > > > > > http://apache-nifi.1125220.n5.nabble.com/discuss-Splitting-NiFi-framework-and-extension-repos-and-releases-td27499.html > > > > > [2] > > > > > > > https://cwiki.apache.org/confluence/display/NIFIREG/NiFi+Project+and+Repository+Restructuring > > > > > [3] > > > > > > > https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-revision > > > > > > > > > > > > > -- > -- > yolanda.m.da...@gmail.com > @YolandaMDavis >
Re: [VOTE] Create NiFi Standard Libraries sub-project
+1 Create NiFi Standard Libraries (binding) On Tue, Sep 3, 2019 at 7:03 PM Koji Kawamura wrote: > +1 Create NiFi Standard Libraries (binding) > > On Wed, Sep 4, 2019 at 7:25 AM Mike Thomsen > wrote: > > > > +1 binding > > > > On Tue, Sep 3, 2019 at 5:33 PM Andy LoPresto > wrote: > > > > > +1, create NiFi Standard Libraries (binding) > > > > > > Andy LoPresto > > > alopre...@apache.org > > > alopresto.apa...@gmail.com > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > > > > On Sep 3, 2019, at 2:16 PM, Bryan Bende wrote: > > > > > > > > All, > > > > > > > > In a previous thread there was a plan discussed to restructure some > of > > > > the repositories in order to address several different issues, such > as > > > > build time, reusability of code, and eventually separating how the > > > > framework and extensions are released [1][2]. > > > > > > > > The overall plan requires many steps to get there, so I'd like to > > > > propose starting with a small actionable step - the creation of a new > > > > sub-project called NiFi Standard Libraries (formerly referred to as > > > > nifi-commons). > > > > > > > > Project Name: Apache NiFi Standard Libraries > > > > Git Repository: nifi-standard-libraries > > > > JIRA: NIFILIBS > > > > > > > > Description: > > > > > > > > A collection of standard implementations used across the NiFi > ecosystem. > > > > > > > > Candidate Libraries: > > > > > > > > In general, each library may consist of multiple Maven modules, and > > > > should be independent from the rest of the ecosystem, and from other > > > > libraries within NiFi Standard Libraries. > > > > > > > > In addition, each library may make it's own decision about whether it > > > > is considered a public facing extension point/API, or an internal > > > > library that may be changed at any time. This should be documented in > > > > a README at the root of each library, such as > > > > nifi-standard-libraries/nifi-xyz/README. > > > > > > > > An initial library that has been discussed was referred to as > > > > 'nifi-security' and would centralize much of the security related > code > > > > shared by NiFi and NiFi Registry, such as shared security APIs, and > > > > implementations for various providers, such as LDAP/Kerberos/etc. > > > > > > > > A second candidate library would be an optimistic-locking library > > > > based on NiFi's revision concept. Currently this has been created > > > > inside nifi-registry for now [3], but could be moved as soon as > > > > nifi-standard-libraries exists. > > > > > > > > (This list does not have to be final in order to decide if we are > > > > creating NiFi Standard Libraries or not) > > > > > > > > Integration & Usage: > > > > > > > > Once NiFi Standard Libraries is created, the community can start > > > > creating and/or moving code there and perform releases as necessary. > A > > > > release will consist of the standard Apache source release, plus > > > > artifacts released to Maven central. The community can then decide > > > > when it is appropriate to integrate these released libraries into one > > > > of our downstream projects. > > > > > > > > For example, if we create a nifi-security library in > > > > nifi-standard-libraries, we can release that whenever we decide, but > > > > we may not integrate it into NiFi or NiFi Registry until it makes > > > > sense for a given release of those projects. > > > > > > > > This vote will be open for 48 hours, please vote: > > > > > > > > [ ] +1 Create NiFi Standard Libraries > > > > [ ] +0 no opinion > > > > [ ] -1 Do not create NiFi Standard Libraries because... > > > > > > > > [1] > > > > http://apache-nifi.1125220.n5.nabble.com/discuss-Splitting-NiFi-framework-and-extension-repos-and-releases-td27499.html > > > > [2] > > > > https://cwiki.apache.org/confluence/display/NIFIREG/NiFi+Project+and+Repository+Restructuring > > > > [3] > > > > https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-revision > > > > > > > -- -- yolanda.m.da...@gmail.com @YolandaMDavis
Re: [VOTE] Create NiFi Standard Libraries sub-project
+1 Create NiFi Standard Libraries (binding) On Wed, Sep 4, 2019 at 7:25 AM Mike Thomsen wrote: > > +1 binding > > On Tue, Sep 3, 2019 at 5:33 PM Andy LoPresto wrote: > > > +1, create NiFi Standard Libraries (binding) > > > > Andy LoPresto > > alopre...@apache.org > > alopresto.apa...@gmail.com > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > > On Sep 3, 2019, at 2:16 PM, Bryan Bende wrote: > > > > > > All, > > > > > > In a previous thread there was a plan discussed to restructure some of > > > the repositories in order to address several different issues, such as > > > build time, reusability of code, and eventually separating how the > > > framework and extensions are released [1][2]. > > > > > > The overall plan requires many steps to get there, so I'd like to > > > propose starting with a small actionable step - the creation of a new > > > sub-project called NiFi Standard Libraries (formerly referred to as > > > nifi-commons). > > > > > > Project Name: Apache NiFi Standard Libraries > > > Git Repository: nifi-standard-libraries > > > JIRA: NIFILIBS > > > > > > Description: > > > > > > A collection of standard implementations used across the NiFi ecosystem. > > > > > > Candidate Libraries: > > > > > > In general, each library may consist of multiple Maven modules, and > > > should be independent from the rest of the ecosystem, and from other > > > libraries within NiFi Standard Libraries. > > > > > > In addition, each library may make it's own decision about whether it > > > is considered a public facing extension point/API, or an internal > > > library that may be changed at any time. This should be documented in > > > a README at the root of each library, such as > > > nifi-standard-libraries/nifi-xyz/README. > > > > > > An initial library that has been discussed was referred to as > > > 'nifi-security' and would centralize much of the security related code > > > shared by NiFi and NiFi Registry, such as shared security APIs, and > > > implementations for various providers, such as LDAP/Kerberos/etc. > > > > > > A second candidate library would be an optimistic-locking library > > > based on NiFi's revision concept. Currently this has been created > > > inside nifi-registry for now [3], but could be moved as soon as > > > nifi-standard-libraries exists. > > > > > > (This list does not have to be final in order to decide if we are > > > creating NiFi Standard Libraries or not) > > > > > > Integration & Usage: > > > > > > Once NiFi Standard Libraries is created, the community can start > > > creating and/or moving code there and perform releases as necessary. A > > > release will consist of the standard Apache source release, plus > > > artifacts released to Maven central. The community can then decide > > > when it is appropriate to integrate these released libraries into one > > > of our downstream projects. > > > > > > For example, if we create a nifi-security library in > > > nifi-standard-libraries, we can release that whenever we decide, but > > > we may not integrate it into NiFi or NiFi Registry until it makes > > > sense for a given release of those projects. > > > > > > This vote will be open for 48 hours, please vote: > > > > > > [ ] +1 Create NiFi Standard Libraries > > > [ ] +0 no opinion > > > [ ] -1 Do not create NiFi Standard Libraries because... > > > > > > [1] > > http://apache-nifi.1125220.n5.nabble.com/discuss-Splitting-NiFi-framework-and-extension-repos-and-releases-td27499.html > > > [2] > > https://cwiki.apache.org/confluence/display/NIFIREG/NiFi+Project+and+Repository+Restructuring > > > [3] > > https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-revision > > > >
Re: [VOTE] Create NiFi Standard Libraries sub-project
+1 binding On Tue, Sep 3, 2019 at 5:33 PM Andy LoPresto wrote: > +1, create NiFi Standard Libraries (binding) > > Andy LoPresto > alopre...@apache.org > alopresto.apa...@gmail.com > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Sep 3, 2019, at 2:16 PM, Bryan Bende wrote: > > > > All, > > > > In a previous thread there was a plan discussed to restructure some of > > the repositories in order to address several different issues, such as > > build time, reusability of code, and eventually separating how the > > framework and extensions are released [1][2]. > > > > The overall plan requires many steps to get there, so I'd like to > > propose starting with a small actionable step - the creation of a new > > sub-project called NiFi Standard Libraries (formerly referred to as > > nifi-commons). > > > > Project Name: Apache NiFi Standard Libraries > > Git Repository: nifi-standard-libraries > > JIRA: NIFILIBS > > > > Description: > > > > A collection of standard implementations used across the NiFi ecosystem. > > > > Candidate Libraries: > > > > In general, each library may consist of multiple Maven modules, and > > should be independent from the rest of the ecosystem, and from other > > libraries within NiFi Standard Libraries. > > > > In addition, each library may make it's own decision about whether it > > is considered a public facing extension point/API, or an internal > > library that may be changed at any time. This should be documented in > > a README at the root of each library, such as > > nifi-standard-libraries/nifi-xyz/README. > > > > An initial library that has been discussed was referred to as > > 'nifi-security' and would centralize much of the security related code > > shared by NiFi and NiFi Registry, such as shared security APIs, and > > implementations for various providers, such as LDAP/Kerberos/etc. > > > > A second candidate library would be an optimistic-locking library > > based on NiFi's revision concept. Currently this has been created > > inside nifi-registry for now [3], but could be moved as soon as > > nifi-standard-libraries exists. > > > > (This list does not have to be final in order to decide if we are > > creating NiFi Standard Libraries or not) > > > > Integration & Usage: > > > > Once NiFi Standard Libraries is created, the community can start > > creating and/or moving code there and perform releases as necessary. A > > release will consist of the standard Apache source release, plus > > artifacts released to Maven central. The community can then decide > > when it is appropriate to integrate these released libraries into one > > of our downstream projects. > > > > For example, if we create a nifi-security library in > > nifi-standard-libraries, we can release that whenever we decide, but > > we may not integrate it into NiFi or NiFi Registry until it makes > > sense for a given release of those projects. > > > > This vote will be open for 48 hours, please vote: > > > > [ ] +1 Create NiFi Standard Libraries > > [ ] +0 no opinion > > [ ] -1 Do not create NiFi Standard Libraries because... > > > > [1] > http://apache-nifi.1125220.n5.nabble.com/discuss-Splitting-NiFi-framework-and-extension-repos-and-releases-td27499.html > > [2] > https://cwiki.apache.org/confluence/display/NIFIREG/NiFi+Project+and+Repository+Restructuring > > [3] > https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-revision > >
Re: [VOTE] Create NiFi Standard Libraries sub-project
+1, create NiFi Standard Libraries (binding) Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Sep 3, 2019, at 2:16 PM, Bryan Bende wrote: > > All, > > In a previous thread there was a plan discussed to restructure some of > the repositories in order to address several different issues, such as > build time, reusability of code, and eventually separating how the > framework and extensions are released [1][2]. > > The overall plan requires many steps to get there, so I'd like to > propose starting with a small actionable step - the creation of a new > sub-project called NiFi Standard Libraries (formerly referred to as > nifi-commons). > > Project Name: Apache NiFi Standard Libraries > Git Repository: nifi-standard-libraries > JIRA: NIFILIBS > > Description: > > A collection of standard implementations used across the NiFi ecosystem. > > Candidate Libraries: > > In general, each library may consist of multiple Maven modules, and > should be independent from the rest of the ecosystem, and from other > libraries within NiFi Standard Libraries. > > In addition, each library may make it's own decision about whether it > is considered a public facing extension point/API, or an internal > library that may be changed at any time. This should be documented in > a README at the root of each library, such as > nifi-standard-libraries/nifi-xyz/README. > > An initial library that has been discussed was referred to as > 'nifi-security' and would centralize much of the security related code > shared by NiFi and NiFi Registry, such as shared security APIs, and > implementations for various providers, such as LDAP/Kerberos/etc. > > A second candidate library would be an optimistic-locking library > based on NiFi's revision concept. Currently this has been created > inside nifi-registry for now [3], but could be moved as soon as > nifi-standard-libraries exists. > > (This list does not have to be final in order to decide if we are > creating NiFi Standard Libraries or not) > > Integration & Usage: > > Once NiFi Standard Libraries is created, the community can start > creating and/or moving code there and perform releases as necessary. A > release will consist of the standard Apache source release, plus > artifacts released to Maven central. The community can then decide > when it is appropriate to integrate these released libraries into one > of our downstream projects. > > For example, if we create a nifi-security library in > nifi-standard-libraries, we can release that whenever we decide, but > we may not integrate it into NiFi or NiFi Registry until it makes > sense for a given release of those projects. > > This vote will be open for 48 hours, please vote: > > [ ] +1 Create NiFi Standard Libraries > [ ] +0 no opinion > [ ] -1 Do not create NiFi Standard Libraries because... > > [1] > http://apache-nifi.1125220.n5.nabble.com/discuss-Splitting-NiFi-framework-and-extension-repos-and-releases-td27499.html > [2] > https://cwiki.apache.org/confluence/display/NIFIREG/NiFi+Project+and+Repository+Restructuring > [3] > https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-revision
[VOTE] Create NiFi Standard Libraries sub-project
All, In a previous thread there was a plan discussed to restructure some of the repositories in order to address several different issues, such as build time, reusability of code, and eventually separating how the framework and extensions are released [1][2]. The overall plan requires many steps to get there, so I'd like to propose starting with a small actionable step - the creation of a new sub-project called NiFi Standard Libraries (formerly referred to as nifi-commons). Project Name: Apache NiFi Standard Libraries Git Repository: nifi-standard-libraries JIRA: NIFILIBS Description: A collection of standard implementations used across the NiFi ecosystem. Candidate Libraries: In general, each library may consist of multiple Maven modules, and should be independent from the rest of the ecosystem, and from other libraries within NiFi Standard Libraries. In addition, each library may make it's own decision about whether it is considered a public facing extension point/API, or an internal library that may be changed at any time. This should be documented in a README at the root of each library, such as nifi-standard-libraries/nifi-xyz/README. An initial library that has been discussed was referred to as 'nifi-security' and would centralize much of the security related code shared by NiFi and NiFi Registry, such as shared security APIs, and implementations for various providers, such as LDAP/Kerberos/etc. A second candidate library would be an optimistic-locking library based on NiFi's revision concept. Currently this has been created inside nifi-registry for now [3], but could be moved as soon as nifi-standard-libraries exists. (This list does not have to be final in order to decide if we are creating NiFi Standard Libraries or not) Integration & Usage: Once NiFi Standard Libraries is created, the community can start creating and/or moving code there and perform releases as necessary. A release will consist of the standard Apache source release, plus artifacts released to Maven central. The community can then decide when it is appropriate to integrate these released libraries into one of our downstream projects. For example, if we create a nifi-security library in nifi-standard-libraries, we can release that whenever we decide, but we may not integrate it into NiFi or NiFi Registry until it makes sense for a given release of those projects. This vote will be open for 48 hours, please vote: [ ] +1 Create NiFi Standard Libraries [ ] +0 no opinion [ ] -1 Do not create NiFi Standard Libraries because... [1] http://apache-nifi.1125220.n5.nabble.com/discuss-Splitting-NiFi-framework-and-extension-repos-and-releases-td27499.html [2] https://cwiki.apache.org/confluence/display/NIFIREG/NiFi+Project+and+Repository+Restructuring [3] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-revision
Re: nifi deploy automation - variables pass through (incl controller services)
The short answer is - please revisit all of this after the 1.10.0 release because all of this will be solved with the new parameters feature :) Longer answer... The CLI has two commands: nifi pg-get-vars nifi pg-set-var These can be used to get and set process group level variables, which can be referenced in any property that supports EL with at least VARIABLE_REGISTRY scope. Sensitive property values are never sent to registry and thus have to be set one time after first importing the flow from registry. This is what the discussion of using the REST API was for, since the CLI does not have a 'set-property' command to set a value of a single property on a component. On Tue, Sep 3, 2019 at 4:13 PM Evan Reynolds wrote: > > There is a CLI command to change variables ... ? I've moved most of my > processor variables to extra-args.property, so it's easy to change them. But > there are a few I can't change - for example, > DistributedMapCacheClientService controller service has a "Server Hostname" > property that doesn't support the expression language, so we have to > essentially search and replace the string in the flow.xml file before > deploying it. Similarly, the DBCPConnectionPool has a password that is a > sensitive values - so even if we set it to an argument, that argument gets > wiped when we go through the registry to deploy, so we are currently > deploying and then going and manually filling in that password. > > I just looked again at the CLI but I didn't see a command that would set > them, but I may have very easily missed it! Is there? > > --- > > Evan Reynolds > e...@usermind.com > > > On 9/2/19, 7:59 PM, "Andrew Grande" wrote: > > I think there is confusion here. Changing *properties* is considered a > tracked change, the behavior is per design. > > There are *variables* on the PG level which are considered non-trackable > environment settings and won't trigger a dirty flag. > > There is a dedicated command in the cli to list and set these *variables*. > > Hope it helps, > Andrew > > On Mon, Sep 2, 2019, 4:07 PM sivasankar Reddy > wrote: > > > Hi Bryan, > > > > Thanks for the options. After pg-import or pg-change-version, if we use > > nifi rest api and update properties, looks like nifi instance detects > > changes(compares with bucket flow in registry) and creates * mark. > > > > We can change pg-change-version however just for properties setting , > may > > be its not advisable to change version in nifi instance. > > > > Any ideas around this will be helpful. > > > > Thanks > > Siva > > > > On 2019/08/23 16:23:38, Bryan Bende wrote: > > > Well there are two different encryption parts here - > > > encryption-at-rest and encryption-in-transit... > > > > > > Encryption-at-rest would have to be part of your script that is > > > calling the REST API. Somehow you create a config file with encrypted > > > values, and your script needs to read those in an decrypt them in > > > memory and then put them into the content that will be sent in the > > > HTTP PUT or POST request. > > > > > > Encryption-in-transit would be handled by the fact that you would be > > > making an HTTPS request to NiFi, so the contents of the POST or PUT > > > request are encrypted in transit. This is the same thing that is > > > happening if you were in the NiFi UI and typed in a password into the > > > sensitive property field. > > > > > > > > > On Fri, Aug 23, 2019 at 11:35 AM sivasankar Reddy > > > wrote: > > > > > > > > Hi Bryan, > > > > > > > > Thanks for the reply. > > > > Able to get the json object and set the property for Password. > However > > value of password can be either referred from config file (its plain > string > > visible in config). Is there any way that password is encrypted and this > > set property can take decrypted value. > > > > > > > > this way we ensure that password is sensitive. Any other ideas to > > achieve this > > > > > > > > > > > > Regards, > > > > Siva > > > > > > > > > > > > > > > > On 2019/08/22 17:38:37, Bryan Bende wrote: > > > > > The parameters work is on-going and will be in the next release > which > > > > > is 1.10.0. Releases don't really have specific timelines, but > > > > > generally they happen every few months. Most likely 1.10.0 is a > > couple > > > > > of weeks away, but depends on when active work is completed and > when > > > > > someone volunteers to make a release. > > > > > > > > > > There is no timeline for the "set-property" command since it was > just > > > > > suggested as a new feature in this email thread :) It requires > > someone > > > > > creating a JIRA and deciding to work on it. >
Re: nifi deploy automation - variables pass through (incl controller services)
There is a CLI command to change variables ... ? I've moved most of my processor variables to extra-args.property, so it's easy to change them. But there are a few I can't change - for example, DistributedMapCacheClientService controller service has a "Server Hostname" property that doesn't support the expression language, so we have to essentially search and replace the string in the flow.xml file before deploying it. Similarly, the DBCPConnectionPool has a password that is a sensitive values - so even if we set it to an argument, that argument gets wiped when we go through the registry to deploy, so we are currently deploying and then going and manually filling in that password. I just looked again at the CLI but I didn't see a command that would set them, but I may have very easily missed it! Is there? --- Evan Reynolds e...@usermind.com On 9/2/19, 7:59 PM, "Andrew Grande" wrote: I think there is confusion here. Changing *properties* is considered a tracked change, the behavior is per design. There are *variables* on the PG level which are considered non-trackable environment settings and won't trigger a dirty flag. There is a dedicated command in the cli to list and set these *variables*. Hope it helps, Andrew On Mon, Sep 2, 2019, 4:07 PM sivasankar Reddy wrote: > Hi Bryan, > > Thanks for the options. After pg-import or pg-change-version, if we use > nifi rest api and update properties, looks like nifi instance detects > changes(compares with bucket flow in registry) and creates * mark. > > We can change pg-change-version however just for properties setting , may > be its not advisable to change version in nifi instance. > > Any ideas around this will be helpful. > > Thanks > Siva > > On 2019/08/23 16:23:38, Bryan Bende wrote: > > Well there are two different encryption parts here - > > encryption-at-rest and encryption-in-transit... > > > > Encryption-at-rest would have to be part of your script that is > > calling the REST API. Somehow you create a config file with encrypted > > values, and your script needs to read those in an decrypt them in > > memory and then put them into the content that will be sent in the > > HTTP PUT or POST request. > > > > Encryption-in-transit would be handled by the fact that you would be > > making an HTTPS request to NiFi, so the contents of the POST or PUT > > request are encrypted in transit. This is the same thing that is > > happening if you were in the NiFi UI and typed in a password into the > > sensitive property field. > > > > > > On Fri, Aug 23, 2019 at 11:35 AM sivasankar Reddy > wrote: > > > > > > Hi Bryan, > > > > > > Thanks for the reply. > > > Able to get the json object and set the property for Password. However > value of password can be either referred from config file (its plain string > visible in config). Is there any way that password is encrypted and this > set property can take decrypted value. > > > > > > this way we ensure that password is sensitive. Any other ideas to > achieve this > > > > > > > > > Regards, > > > Siva > > > > > > > > > > > > On 2019/08/22 17:38:37, Bryan Bende wrote: > > > > The parameters work is on-going and will be in the next release which > > > > is 1.10.0. Releases don't really have specific timelines, but > > > > generally they happen every few months. Most likely 1.10.0 is a > couple > > > > of weeks away, but depends on when active work is completed and when > > > > someone volunteers to make a release. > > > > > > > > There is no timeline for the "set-property" command since it was just > > > > suggested as a new feature in this email thread :) It requires > someone > > > > creating a JIRA and deciding to work on it. > > > > > > > > All of the functionality in the CLI and NiPyAPI, and even NiFi's own > > > > UI, is based on the REST API. So you can still perform a "set > > > > property" by using the REST API to modify the configuration of a > > > > processor, the CLI would just make it easier so that you wouldn't > have > > > > to understand the lower level API details. The best way to understand > > > > the API calls is to us the UI while you have Chrome Dev Tools open, > > > > and then perform the action you are interested in and look at the > > > > requests made on the Network tab. You'll be able to see what URLs are > > > > called and what the request and response bodies look like. > > > > > > > > On Thu, Aug 22, 2019 at 11:46 AM sivasankar Reddy < > mail2ms...@gmail.com> wrote: > > > > > > > > > > Hi Bryan, > > > > > > > > > > Thanks for the reply.
Re: ValidateXML Schema File EL scope
It was likely just a decision by the developer of the processor, but in the current implementation you can parse the schema once when the processor is started which will be a lot more efficient then parsing every time the processor executes. If we do support flow file attributes then it could probably be made to cache parsed schemas. On Tue, Sep 3, 2019 at 7:26 AM wrote: > > We are using Apache NiFi 1.9.2 for one of our flows and had a question below. > > Why does the ValidateXML processor Schema File property supports only > Variable registry in Expression Language scope? > We are trying to devise a solution for our flow so that based on the incoming > file name and filetype ,appropriate XSD (camt 53,camt 52 etc) will be > selected for validation and updated in the flowfile attribute. > This attribute will then be used in the ValidateXML processor as Schema file > name so that it becomes easily configurable for future for other files as > well and we do not have to hardcode > > Thanks. > Chinmay > This e-mail and any files transmitted with it are for the sole use of the > intended recipient(s) and may contain confidential and privileged > information. If you are not the intended recipient(s), please reply to the > sender and destroy all copies of the original message. Any unauthorized > review, use, disclosure, dissemination, forwarding, printing or copying of > this email, and/or any action taken in reliance on the contents of this > e-mail is strictly prohibited and may be unlawful. Where permitted by > applicable law, this e-mail and other e-mail communications sent to and from > Cognizant e-mail addresses may be monitored. This e-mail and any files > transmitted with it are for the sole use of the intended recipient(s) and may > contain confidential and privileged information. If you are not the intended > recipient(s), please reply to the sender and destroy all copies of the > original message. Any unauthorized review, use, disclosure, dissemination, > forwarding, printing or copying of this email, and/or any action taken in > reliance on the contents of this e-mail is strictly prohibited and may be > unlawful. Where permitted by applicable law, this e-mail and other e-mail > communications sent to and from Cognizant e-mail addresses may be monitored.
IPV6 support
Hi , May I know when (which release)is ipv6 support planned for the Nifi . Thanks & Regards Ganesh.B
ValidateXML Schema File EL scope
We are using Apache NiFi 1.9.2 for one of our flows and had a question below. Why does the ValidateXML processor Schema File property supports only Variable registry in Expression Language scope? We are trying to devise a solution for our flow so that based on the incoming file name and filetype ,appropriate XSD (camt 53,camt 52 etc) will be selected for validation and updated in the flowfile attribute. This attribute will then be used in the ValidateXML processor as Schema file name so that it becomes easily configurable for future for other files as well and we do not have to hardcode Thanks. Chinmay This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored. This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
回复: CaptureChangeMySQL bit(1) type field is parsed to '{}'
Anybody has any insight on this?
wangl...@geekplus.com.cn
发件人： wangl...@geekplus.com.cn
发送时间： 2019-08-22 19:31
收件人： users
主题： CaptureChangeMySQL bit(1) type field is parsed to '{}'
CaptureChangeMySQL -> ConvertJSONToSQL -> ExecuteSQL to replicate my database.
But when the field type is bit, there will be error. Because teh value is
parsed to '{}'
`is_cancel` bit(1) DEFAULT b'0' COMMENT 'canceled',
[ {
"id" : 6173148,
"task_id" : 6173148,
"charger_id" : 1,
"is_cancel" : "{}",
"priority" : 0,
"business_sequence" : 0
} ]
Is this a bug？
Or there‘s some method i can avoid this?
Thanks,
Lei
wangl...@geekplus.com.cn
Re: nifi deploy automation - variables pass through (incl controller services)
Using the REST API to set a property was meant only for sensitive properties since they cant use variables. Changes to sensitive properties do not produce a change that is tracked by version control. All of this will be unnecessary with the new parameters functionality in 1.10.0. On Mon, Sep 2, 2019 at 10:59 PM Andrew Grande wrote: > I think there is confusion here. Changing *properties* is considered a > tracked change, the behavior is per design. > > There are *variables* on the PG level which are considered non-trackable > environment settings and won't trigger a dirty flag. > > There is a dedicated command in the cli to list and set these *variables*. > > Hope it helps, > Andrew > > On Mon, Sep 2, 2019, 4:07 PM sivasankar Reddy > wrote: > > > Hi Bryan, > > > > Thanks for the options. After pg-import or pg-change-version, if we use > > nifi rest api and update properties, looks like nifi instance detects > > changes(compares with bucket flow in registry) and creates * mark. > > > > We can change pg-change-version however just for properties setting , may > > be its not advisable to change version in nifi instance. > > > > Any ideas around this will be helpful. > > > > Thanks > > Siva > > > > On 2019/08/23 16:23:38, Bryan Bende wrote: > > > Well there are two different encryption parts here - > > > encryption-at-rest and encryption-in-transit... > > > > > > Encryption-at-rest would have to be part of your script that is > > > calling the REST API. Somehow you create a config file with encrypted > > > values, and your script needs to read those in an decrypt them in > > > memory and then put them into the content that will be sent in the > > > HTTP PUT or POST request. > > > > > > Encryption-in-transit would be handled by the fact that you would be > > > making an HTTPS request to NiFi, so the contents of the POST or PUT > > > request are encrypted in transit. This is the same thing that is > > > happening if you were in the NiFi UI and typed in a password into the > > > sensitive property field. > > > > > > > > > On Fri, Aug 23, 2019 at 11:35 AM sivasankar Reddy < > mail2ms...@gmail.com> > > wrote: > > > > > > > > Hi Bryan, > > > > > > > > Thanks for the reply. > > > > Able to get the json object and set the property for Password. > However > > value of password can be either referred from config file (its plain > string > > visible in config). Is there any way that password is encrypted and this > > set property can take decrypted value. > > > > > > > > this way we ensure that password is sensitive. Any other ideas to > > achieve this > > > > > > > > > > > > Regards, > > > > Siva > > > > > > > > > > > > > > > > On 2019/08/22 17:38:37, Bryan Bende wrote: > > > > > The parameters work is on-going and will be in the next release > which > > > > > is 1.10.0. Releases don't really have specific timelines, but > > > > > generally they happen every few months. Most likely 1.10.0 is a > > couple > > > > > of weeks away, but depends on when active work is completed and > when > > > > > someone volunteers to make a release. > > > > > > > > > > There is no timeline for the "set-property" command since it was > just > > > > > suggested as a new feature in this email thread :) It requires > > someone > > > > > creating a JIRA and deciding to work on it. > > > > > > > > > > All of the functionality in the CLI and NiPyAPI, and even NiFi's > own > > > > > UI, is based on the REST API. So you can still perform a "set > > > > > property" by using the REST API to modify the configuration of a > > > > > processor, the CLI would just make it easier so that you wouldn't > > have > > > > > to understand the lower level API details. The best way to > understand > > > > > the API calls is to us the UI while you have Chrome Dev Tools open, > > > > > and then perform the action you are interested in and look at the > > > > > requests made on the Network tab. You'll be able to see what URLs > are > > > > > called and what the request and response bodies look like. > > > > > > > > > > On Thu, Aug 22, 2019 at 11:46 AM sivasankar Reddy < > > mail2ms...@gmail.com> wrote: > > > > > > > > > > > > Hi Bryan, > > > > > > > > > > > > Thanks for the reply. looks like even set-property will be a new > > feature, as current CLI doesn't have that. > > > > > > > > > > > > Could you please share timelines of these features if its in > > roadmap. > > > > > > 1. "set-property" > > > > > > 2. “parameters” > > > > > > > > > > > > The only option currently is set to set sensitive parameters > > manually? or any other option through CLI > > > > > > > > > > > > Regards, > > > > > > Siva > > > > > > > > > > > > On 2019/08/22 04:10:46, Bryan Bende wrote: > > > > > > > Currently there isn’t a good way with the CLI, we would need to > > add a > > > > > > > command like set-property that took the id of a component, and > > the name and > > > > > > > value for a property. > > > > > > > > > > > > > > The next release will have a new feature called
Apache NiFi contributor access
Hi, I would like to get contributor access to Apache Nifi on JIRA. Can you please add me to contributor list? My username is 'b.ganesh' . Thanks & Regards, Ganesh.B
