Hi Bryan, Thanks for the options. After pg-import or pg-change-version, if we use nifi rest api and update properties, looks like nifi instance detects changes(compares with bucket flow in registry) and creates * mark.
We can change pg-change-version however just for properties setting , may be its not advisable to change version in nifi instance. Any ideas around this will be helpful. Thanks Siva On 2019/08/23 16:23:38, Bryan Bende <[email protected]> wrote: > Well there are two different encryption parts here - > encryption-at-rest and encryption-in-transit... > > Encryption-at-rest would have to be part of your script that is > calling the REST API. Somehow you create a config file with encrypted > values, and your script needs to read those in an decrypt them in > memory and then put them into the content that will be sent in the > HTTP PUT or POST request. > > Encryption-in-transit would be handled by the fact that you would be > making an HTTPS request to NiFi, so the contents of the POST or PUT > request are encrypted in transit. This is the same thing that is > happening if you were in the NiFi UI and typed in a password into the > sensitive property field. > > > On Fri, Aug 23, 2019 at 11:35 AM sivasankar Reddy <[email protected]> > wrote: > > > > Hi Bryan, > > > > Thanks for the reply. > > Able to get the json object and set the property for Password. However > > value of password can be either referred from config file (its plain string > > visible in config). Is there any way that password is encrypted and this > > set property can take decrypted value. > > > > this way we ensure that password is sensitive. Any other ideas to achieve > > this > > > > > > Regards, > > Siva > > > > > > > > On 2019/08/22 17:38:37, Bryan Bende <[email protected]> wrote: > > > The parameters work is on-going and will be in the next release which > > > is 1.10.0. Releases don't really have specific timelines, but > > > generally they happen every few months. Most likely 1.10.0 is a couple > > > of weeks away, but depends on when active work is completed and when > > > someone volunteers to make a release. > > > > > > There is no timeline for the "set-property" command since it was just > > > suggested as a new feature in this email thread :) It requires someone > > > creating a JIRA and deciding to work on it. > > > > > > All of the functionality in the CLI and NiPyAPI, and even NiFi's own > > > UI, is based on the REST API. So you can still perform a "set > > > property" by using the REST API to modify the configuration of a > > > processor, the CLI would just make it easier so that you wouldn't have > > > to understand the lower level API details. The best way to understand > > > the API calls is to us the UI while you have Chrome Dev Tools open, > > > and then perform the action you are interested in and look at the > > > requests made on the Network tab. You'll be able to see what URLs are > > > called and what the request and response bodies look like. > > > > > > On Thu, Aug 22, 2019 at 11:46 AM sivasankar Reddy <[email protected]> > > > wrote: > > > > > > > > Hi Bryan, > > > > > > > > Thanks for the reply. looks like even set-property will be a new > > > > feature, as current CLI doesn't have that. > > > > > > > > Could you please share timelines of these features if its in roadmap. > > > > 1. "set-property" > > > > 2. “parameters” > > > > > > > > The only option currently is set to set sensitive parameters manually? > > > > or any other option through CLI > > > > > > > > Regards, > > > > Siva > > > > > > > > On 2019/08/22 04:10:46, Bryan Bende <[email protected]> wrote: > > > > > Currently there isn’t a good way with the CLI, we would need to add a > > > > > command like set-property that took the id of a component, and the > > > > > name and > > > > > value for a property. > > > > > > > > > > The next release will have a new feature called “parameters” which > > > > > will > > > > > solve this problem. You’ll be able to use a parameter in a sensitive > > > > > property and then use CLI to set parameter values. > > > > > > > > > > On Wed, Aug 21, 2019 at 12:29 PM sivasankar Reddy > > > > > <[email protected]> > > > > > wrote: > > > > > > > > > > > Hi Bryan, > > > > > > > > > > > > Thanks for the update. Could you please mention about > > > > > > sensitive parameters aspect from deployment point of view using CLI. > > > > > > > > > > > > How to set the sensitive parameters values after pg-import. > > > > > > > > > > > > Regards, > > > > > > Siva > > > > > > > > > > > > On 2019/08/21 11:47:49, Bryan Bende <[email protected]> wrote: > > > > > > > It does not, I was saying we could easily improve the command to > > > > > > > add > > > > > > those > > > > > > > parameters. > > > > > > > > > > > > > > On Wed, Aug 21, 2019 at 6:36 AM sivasankar Reddy > > > > > > > <[email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Bryan, > > > > > > > > > > > > > > > > Thanks for the update. Does the existing pg-import takes > > > > > > > > optional > > > > > > params > > > > > > > > for x/y coordinates. If yes can you share the sample command as > > > > > > pg-import > > > > > > > > --help doesn't has this info. > > > > > > > > > > > > > > > > Regards, > > > > > > > > siva > > > > > > > > > > > > > > > > > > > > > > > > On 2019/08/20 22:19:23, Bryan Bende <[email protected]> wrote: > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > 1) pg-import will keeping finding a location that does not > > > > > > > > > overlap > > > > > > with > > > > > > > > any > > > > > > > > > existing process groups, so it is more of auto layout. We > > > > > > > > > could > > > > > > easily > > > > > > > > add > > > > > > > > > optional params for x/y coordinates. > > > > > > > > > > > > > > > > > > 2) Both are fine choices.. CLI is more geared towards simple > > > > > > commands, > > > > > > > > > where as with NiPyApi you could more easily write complex > > > > > > > > > logic in > > > > > > > > Python. > > > > > > > > > With CLI you would write a wrapper shell script that made > > > > > > > > > individual > > > > > > > > calls > > > > > > > > > to the CLI. I think it is really just a matter of preference. > > > > > > > > > > > > > > > > > > -Bryan > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Aug 20, 2019 at 5:11 PM sivasankar Reddy < > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Bryan, > > > > > > > > > > > > > > > > > > > > Thanks for your input. I would like to know more > > > > > > > > > > > > > > > > > > > > 1. how does CLI pg-import works if already canvas has 100 > > > > > > > > > > to 150 > > > > > > > > flows. > > > > > > > > > > Will CLI pg-import always place the flow in canvas which > > > > > > > > > > can be > > > > > > easily > > > > > > > > > > located. As nipyapi has location(x,y) during deploy flow. > > > > > > > > > > > > > > > > > > > > 2. From enterprise usage, do you recommend toolkit CLI vs > > > > > > > > > > nipyapi, > > > > > > it > > > > > > > > > > seems toolkit CLI is part of official Nifi. > > > > > > > > > > > > > > > > > > > > Any other points regarding this will be helpful. > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > Siva > > > > > > > > > > > > > > > > > > > > On 2019/08/08 22:41:43, Bryan Bende <[email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > > > A lot depends on whether you are doing the promotion with > > > > > > > > > > > the UI > > > > > > or > > > > > > > > > > > with scripts using something like NiFi CLI or NiPyApi. > > > > > > > > > > > > > > > > > > > > > > 1) Currently references to controller services outside the > > > > > > versioned > > > > > > > > > > > process group need to be set after initial import. In the > > > > > > > > > > > next > > > > > > > > release > > > > > > > > > > > it will auto-select by name as long as there is only one > > > > > > > > > > > service > > > > > > with > > > > > > > > > > > the given name and type in a parent group (name is not > > > > > > > > > > > unique). > > > > > > > > > > > > > > > > > > > > > > 2) The snapshot of the variables are part of the > > > > > > > > > > > versioned flow, > > > > > > so > > > > > > > > > > > after importing you would have to set the new values in > > > > > > > > > > > the > > > > > > target > > > > > > > > > > > environment. This can be done in the UI or with scripts. > > > > > > > > > > > > > > > > > > > > > > 3) The REST API allows for positioning of any component > > > > > > > > > > > that is > > > > > > > > > > > created. i can't speak for how to do it with NiPyApi, but > > > > > > > > > > > I know > > > > > > the > > > > > > > > > > > CLI pg-import command attempts to automatically organize > > > > > > > > > > > the > > > > > > imported > > > > > > > > > > > process groups on the root canvas so that they don't > > > > > > > > > > > overlap. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > Bryan > > > > > > > > > > > > > > > > > > > > > > On Thu, Aug 8, 2019 at 4:32 PM sivasankar Reddy < > > > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > We are looking for nifi deploy automation from regsitry > > > > > > > > > > > > -> > > > > > > higher > > > > > > > > nifi > > > > > > > > > > instance. Quick observations during intial evaluation. > > > > > > > > > > > > > > > > > > > > > > > > 1. After process group import, controller services are > > > > > > > > > > > > not set > > > > > > in > > > > > > > > nifi > > > > > > > > > > instance > > > > > > > > > > > > 2. How to set variables from source process group to > > > > > > destination > > > > > > > > > > process group. > > > > > > > > > > > > 3. Is there any possibility of definining the > > > > > > > > > > > > positioning of > > > > > > > > process > > > > > > > > > > group to be imported in Nifi Canvas. > > > > > > > > > > > > > > > > > > > > > > > > Any other best practices regarding NiFi deployment > > > > > > > > > > > > automation > > > > > > > > will be > > > > > > > > > > helpful. > > > > > > > > > > > > > > > > > > > > > > > > Thanks in Advance! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Sent from Gmail Mobile > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Sent from Gmail Mobile > > > > > > > > > > > > > > > > > > -- > > > > > Sent from Gmail Mobile > > > > > > > > >
