There is a CLI command to change variables ... ? I've moved most of my processor variables to extra-args.property, so it's easy to change them. But there are a few I can't change - for example, DistributedMapCacheClientService controller service has a "Server Hostname" property that doesn't support the expression language, so we have to essentially search and replace the string in the flow.xml file before deploying it. Similarly, the DBCPConnectionPool has a password that is a sensitive values - so even if we set it to an argument, that argument gets wiped when we go through the registry to deploy, so we are currently deploying and then going and manually filling in that password.
I just looked again at the CLI but I didn't see a command that would set them, but I may have very easily missed it! Is there? --------------------------------------------------------------- Evan Reynolds [email protected] On 9/2/19, 7:59 PM, "Andrew Grande" <[email protected]> wrote: I think there is confusion here. Changing *properties* is considered a tracked change, the behavior is per design. There are *variables* on the PG level which are considered non-trackable environment settings and won't trigger a dirty flag. There is a dedicated command in the cli to list and set these *variables*. Hope it helps, Andrew On Mon, Sep 2, 2019, 4:07 PM sivasankar Reddy <[email protected]> wrote: > Hi Bryan, > > Thanks for the options. After pg-import or pg-change-version, if we use > nifi rest api and update properties, looks like nifi instance detects > changes(compares with bucket flow in registry) and creates * mark. > > We can change pg-change-version however just for properties setting , may > be its not advisable to change version in nifi instance. > > Any ideas around this will be helpful. > > Thanks > Siva > > On 2019/08/23 16:23:38, Bryan Bende <[email protected]> wrote: > > Well there are two different encryption parts here - > > encryption-at-rest and encryption-in-transit... > > > > Encryption-at-rest would have to be part of your script that is > > calling the REST API. Somehow you create a config file with encrypted > > values, and your script needs to read those in an decrypt them in > > memory and then put them into the content that will be sent in the > > HTTP PUT or POST request. > > > > Encryption-in-transit would be handled by the fact that you would be > > making an HTTPS request to NiFi, so the contents of the POST or PUT > > request are encrypted in transit. This is the same thing that is > > happening if you were in the NiFi UI and typed in a password into the > > sensitive property field. > > > > > > On Fri, Aug 23, 2019 at 11:35 AM sivasankar Reddy <[email protected]> > wrote: > > > > > > Hi Bryan, > > > > > > Thanks for the reply. > > > Able to get the json object and set the property for Password. However > value of password can be either referred from config file (its plain string > visible in config). Is there any way that password is encrypted and this > set property can take decrypted value. > > > > > > this way we ensure that password is sensitive. Any other ideas to > achieve this > > > > > > > > > Regards, > > > Siva > > > > > > > > > > > > On 2019/08/22 17:38:37, Bryan Bende <[email protected]> wrote: > > > > The parameters work is on-going and will be in the next release which > > > > is 1.10.0. Releases don't really have specific timelines, but > > > > generally they happen every few months. Most likely 1.10.0 is a > couple > > > > of weeks away, but depends on when active work is completed and when > > > > someone volunteers to make a release. > > > > > > > > There is no timeline for the "set-property" command since it was just > > > > suggested as a new feature in this email thread :) It requires > someone > > > > creating a JIRA and deciding to work on it. > > > > > > > > All of the functionality in the CLI and NiPyAPI, and even NiFi's own > > > > UI, is based on the REST API. So you can still perform a "set > > > > property" by using the REST API to modify the configuration of a > > > > processor, the CLI would just make it easier so that you wouldn't > have > > > > to understand the lower level API details. The best way to understand > > > > the API calls is to us the UI while you have Chrome Dev Tools open, > > > > and then perform the action you are interested in and look at the > > > > requests made on the Network tab. You'll be able to see what URLs are > > > > called and what the request and response bodies look like. > > > > > > > > On Thu, Aug 22, 2019 at 11:46 AM sivasankar Reddy < > [email protected]> wrote: > > > > > > > > > > Hi Bryan, > > > > > > > > > > Thanks for the reply. looks like even set-property will be a new > feature, as current CLI doesn't have that. > > > > > > > > > > Could you please share timelines of these features if its in > roadmap. > > > > > 1. "set-property" > > > > > 2. “parameters” > > > > > > > > > > The only option currently is set to set sensitive parameters > manually? or any other option through CLI > > > > > > > > > > Regards, > > > > > Siva > > > > > > > > > > On 2019/08/22 04:10:46, Bryan Bende <[email protected]> wrote: > > > > > > Currently there isn’t a good way with the CLI, we would need to > add a > > > > > > command like set-property that took the id of a component, and > the name and > > > > > > value for a property. > > > > > > > > > > > > The next release will have a new feature called “parameters” > which will > > > > > > solve this problem. You’ll be able to use a parameter in a > sensitive > > > > > > property and then use CLI to set parameter values. > > > > > > > > > > > > On Wed, Aug 21, 2019 at 12:29 PM sivasankar Reddy < > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi Bryan, > > > > > > > > > > > > > > Thanks for the update. Could you please mention about > > > > > > > sensitive parameters aspect from deployment point of view > using CLI. > > > > > > > > > > > > > > How to set the sensitive parameters values after pg-import. > > > > > > > > > > > > > > Regards, > > > > > > > Siva > > > > > > > > > > > > > > On 2019/08/21 11:47:49, Bryan Bende <[email protected]> wrote: > > > > > > > > It does not, I was saying we could easily improve the > command to add > > > > > > > those > > > > > > > > parameters. > > > > > > > > > > > > > > > > On Wed, Aug 21, 2019 at 6:36 AM sivasankar Reddy < > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi Bryan, > > > > > > > > > > > > > > > > > > Thanks for the update. Does the existing pg-import takes > optional > > > > > > > params > > > > > > > > > for x/y coordinates. If yes can you share the sample > command as > > > > > > > pg-import > > > > > > > > > --help doesn't has this info. > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > siva > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2019/08/20 22:19:23, Bryan Bende <[email protected]> > wrote: > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > 1) pg-import will keeping finding a location that does > not overlap > > > > > > > with > > > > > > > > > any > > > > > > > > > > existing process groups, so it is more of auto layout. > We could > > > > > > > easily > > > > > > > > > add > > > > > > > > > > optional params for x/y coordinates. > > > > > > > > > > > > > > > > > > > > 2) Both are fine choices.. CLI is more geared towards > simple > > > > > > > commands, > > > > > > > > > > where as with NiPyApi you could more easily write > complex logic in > > > > > > > > > Python. > > > > > > > > > > With CLI you would write a wrapper shell script that > made individual > > > > > > > > > calls > > > > > > > > > > to the CLI. I think it is really just a matter of > preference. > > > > > > > > > > > > > > > > > > > > -Bryan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Aug 20, 2019 at 5:11 PM sivasankar Reddy < > > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Bryan, > > > > > > > > > > > > > > > > > > > > > > Thanks for your input. I would like to know more > > > > > > > > > > > > > > > > > > > > > > 1. how does CLI pg-import works if already canvas has > 100 to 150 > > > > > > > > > flows. > > > > > > > > > > > Will CLI pg-import always place the flow in canvas > which can be > > > > > > > easily > > > > > > > > > > > located. As nipyapi has location(x,y) during deploy > flow. > > > > > > > > > > > > > > > > > > > > > > 2. From enterprise usage, do you recommend toolkit CLI > vs nipyapi, > > > > > > > it > > > > > > > > > > > seems toolkit CLI is part of official Nifi. > > > > > > > > > > > > > > > > > > > > > > Any other points regarding this will be helpful. > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > Siva > > > > > > > > > > > > > > > > > > > > > > On 2019/08/08 22:41:43, Bryan Bende <[email protected]> > wrote: > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > > > > > A lot depends on whether you are doing the promotion > with the UI > > > > > > > or > > > > > > > > > > > > with scripts using something like NiFi CLI or > NiPyApi. > > > > > > > > > > > > > > > > > > > > > > > > 1) Currently references to controller services > outside the > > > > > > > versioned > > > > > > > > > > > > process group need to be set after initial import. > In the next > > > > > > > > > release > > > > > > > > > > > > it will auto-select by name as long as there is only > one service > > > > > > > with > > > > > > > > > > > > the given name and type in a parent group (name is > not unique). > > > > > > > > > > > > > > > > > > > > > > > > 2) The snapshot of the variables are part of the > versioned flow, > > > > > > > so > > > > > > > > > > > > after importing you would have to set the new values > in the > > > > > > > target > > > > > > > > > > > > environment. This can be done in the UI or with > scripts. > > > > > > > > > > > > > > > > > > > > > > > > 3) The REST API allows for positioning of any > component that is > > > > > > > > > > > > created. i can't speak for how to do it with > NiPyApi, but I know > > > > > > > the > > > > > > > > > > > > CLI pg-import command attempts to automatically > organize the > > > > > > > imported > > > > > > > > > > > > process groups on the root canvas so that they don't > overlap. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > Bryan > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Aug 8, 2019 at 4:32 PM sivasankar Reddy < > > > > > > > > > [email protected]> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > We are looking for nifi deploy automation from > regsitry -> > > > > > > > higher > > > > > > > > > nifi > > > > > > > > > > > instance. Quick observations during intial evaluation. > > > > > > > > > > > > > > > > > > > > > > > > > > 1. After process group import, controller services > are not set > > > > > > > in > > > > > > > > > nifi > > > > > > > > > > > instance > > > > > > > > > > > > > 2. How to set variables from source process group > to > > > > > > > destination > > > > > > > > > > > process group. > > > > > > > > > > > > > 3. Is there any possibility of definining the > positioning of > > > > > > > > > process > > > > > > > > > > > group to be imported in Nifi Canvas. > > > > > > > > > > > > > > > > > > > > > > > > > > Any other best practices regarding NiFi > deployment automation > > > > > > > > > will be > > > > > > > > > > > helpful. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks in Advance! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Sent from Gmail Mobile > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Sent from Gmail Mobile > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Sent from Gmail Mobile > > > > > > > > > > > > >
