[jira] [Commented] (NIFIREG-139) Nifi registry should pick up the controller service definitions while deploying a flow
[ https://issues.apache.org/jira/browse/NIFIREG-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16605735#comment-16605735 ] Bryan Bende commented on NIFIREG-139: - Since controller services are extensions we can't really ensure that all controller services would have the new property, or that they would support dyanmic properties, because anyone can implement their own CS that doesn't provide those. For the "Mapping Name" idea it could be built into all controller services at the framework level, but I'm not sure that I like the idea of modifying core NiFi to support this concept, since the primary use case is someone using the UI where this is not an issue. It is only an issue when trying to script deployments which core NiFi has no concept of whether that is being done or not, so it feels more appropriate for the CLI to handle this in some way. > Nifi registry should pick up the controller service definitions while > deploying a flow > -- > > Key: NIFIREG-139 > URL: https://issues.apache.org/jira/browse/NIFIREG-139 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 >Reporter: ohad israeli >Priority: Critical > > I am trying to use the registry to copy a simple flow for example of > httprequest->joltconversion->httpresponse. > This flow is using the internal http context service which is installed and > configured on the source (dev env). While deploying the flow to the target > (prod env) the flow cant find the service on the target (prod end) > to summarize, on deployment of a flow if the flow is using a service > controller its setting should also be deployed on the target machine -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-139) Nifi registry should pick up the controller service definitions while deploying a flow
[ https://issues.apache.org/jira/browse/NIFIREG-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16605506#comment-16605506 ] Julian Gimbel commented on NIFIREG-139: --- Two other possibilities come into my mind: # is to add a new property to the CS that names something like "Mapping Name". That one can be used to check for equality within the different environments. That would mean changes in Nifi as well as in Nifi Registry. If that property is evaluated with the expression language it might give quite powerful deployment possibilities and much more flexibility. # As some of the CS already allow for Dynamic Attributes it would also be possible to use a specifically named dynamic attribute. The name of that specific attribute can be provided to the CLI as an optional parameter that would be used to check for name equality. The disadvantage is, that Dynamic Attributes do not allow > Nifi registry should pick up the controller service definitions while > deploying a flow > -- > > Key: NIFIREG-139 > URL: https://issues.apache.org/jira/browse/NIFIREG-139 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 >Reporter: ohad israeli >Priority: Critical > > I am trying to use the registry to copy a simple flow for example of > httprequest->joltconversion->httpresponse. > This flow is using the internal http context service which is installed and > configured on the source (dev env). While deploying the flow to the target > (prod env) the flow cant find the service on the target (prod end) > to summarize, on deployment of a flow if the flow is using a service > controller its setting should also be deployed on the target machine -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-139) Nifi registry should pick up the controller service definitions while deploying a flow
[
https://issues.apache.org/jira/browse/NIFIREG-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604920#comment-16604920
]
Bryan Bende commented on NIFIREG-139:
-
[~jugi] I like this idea in theory, but I'm not sure we have the information to
correctly implement it...
When saving a flow to registry that references a CS from a parent group, the
versioned flow only has the id that was referenced.
For example, here is the properties section from PutHDFS when referencing a
keytab service from the parent group:
{code:java}
"properties" : {
"Compression codec" : "NONE",
"kerberos-credentials-service" : "8ac840fe-c730-392e-b1e2-a42dce0cde1d",
"Kerberos Relogin Period" : "4 hours",
"Conflict Resolution Strategy" : "fail"
}{code}
At the time of importing this versioned flow to a new instance, we don't know
what the name of the keytab service was in the previous instance that created
the flow, so I'm not sure how to line it up in the target environment.
The only thing I could think of would be to have a naming convention that could
be used to link components. So for example, maybe if the process group name is
"FOO## My Process Group" then we parse out whatever is before the ## which is
FOO in this case, and then look for controller services in parent groups where
the name also starts with "FOO##", and then if they match the type needed by
the processor then we can auto-assign it.
Keeping in mind that names are not unique so the best we could do is take the
first one found that matches the pattern.
> Nifi registry should pick up the controller service definitions while
> deploying a flow
> --
>
> Key: NIFIREG-139
> URL: https://issues.apache.org/jira/browse/NIFIREG-139
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: ohad israeli
>Priority: Critical
>
> I am trying to use the registry to copy a simple flow for example of
> httprequest->joltconversion->httpresponse.
> This flow is using the internal http context service which is installed and
> configured on the source (dev env). While deploying the flow to the target
> (prod env) the flow cant find the service on the target (prod end)
> to summarize, on deployment of a flow if the flow is using a service
> controller its setting should also be deployed on the target machine
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFIREG-139) Nifi registry should pick up the controller service definitions while deploying a flow
[ https://issues.apache.org/jira/browse/NIFIREG-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602697#comment-16602697 ] Julian Gimbel commented on NIFIREG-139: --- Hi, we are versioning and deploying some process groups that reference controller services (CS) that are defined outside of the process group e.g. KeytabCredentialsServices for QA and Production. Now if we want to deploy the process group we still need to manually select the correct CS in the processors on production like for example in PutHDFS. Would it be possible to include a “map cs by name” parameter to the CLI so that it walks through the processors and if it finds a CS with the same name as in the source it will use that one? I know that currently only the ID is stored, but that one differs between the two Nifi instances. Best regards Julian > Nifi registry should pick up the controller service definitions while > deploying a flow > -- > > Key: NIFIREG-139 > URL: https://issues.apache.org/jira/browse/NIFIREG-139 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 >Reporter: ohad israeli >Priority: Critical > > I am trying to use the registry to copy a simple flow for example of > httprequest->joltconversion->httpresponse. > This flow is using the internal http context service which is installed and > configured on the source (dev env). While deploying the flow to the target > (prod env) the flow cant find the service on the target (prod end) > to summarize, on deployment of a flow if the flow is using a service > controller its setting should also be deployed on the target machine -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-139) Nifi registry should pick up the controller service definitions while deploying a flow
[
https://issues.apache.org/jira/browse/NIFIREG-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358918#comment-16358918
]
Kevin Doran commented on NIFIREG-139:
-
{quote}If it is at a higher level then it is an external reference and will not
be copied but rather an attempt to bind to a similar one in the other
environment will be made.{quote}
I'm not sure this is true for controller services. [~markap14] could verify.
> Nifi registry should pick up the controller service definitions while
> deploying a flow
> --
>
> Key: NIFIREG-139
> URL: https://issues.apache.org/jira/browse/NIFIREG-139
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: ohad israeli
>Priority: Critical
>
> I am trying to use the registry to copy a simple flow for example of
> httprequest->joltconversion->httpresponse.
> This flow is using the internal http context service which is installed and
> configured on the source (dev env). While deploying the flow to the target
> (prod env) the flow cant find the service on the target (prod end)
> to summarize, on deployment of a flow if the flow is using a service
> controller its setting should also be deployed on the target machine
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFIREG-139) Nifi registry should pick up the controller service definitions while deploying a flow
[ https://issues.apache.org/jira/browse/NIFIREG-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352620#comment-16352620 ] Joseph Witt commented on NIFIREG-139: - [~israelio] If the controller service is scoped/configured within the item being version controlled it will be included/copied. If it is at a higher level then it is an external reference and will not be copied but rather an attempt to bind to a similar one in the other environment will be made. If you create the service in the other environment just as you did in the original environment it will likely do what you want. If you change to have the CS in the same scope as the versioned item it will too. Please confirm how you have it setup and let us know if that new understanding helps you. Others can confirm if I've stated anything inaccurate here. > Nifi registry should pick up the controller service definitions while > deploying a flow > -- > > Key: NIFIREG-139 > URL: https://issues.apache.org/jira/browse/NIFIREG-139 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 >Reporter: ohad israeli >Priority: Critical > > I am trying to use the registry to copy a simple flow for example of > httprequest->joltconversion->httpresponse. > This flow is using the internal http context service which is installed and > configured on the source (dev env). While deploying the flow to the target > (prod env) the flow cant find the service on the target (prod end) > to summarize, on deployment of a flow if the flow is using a service > controller its setting should also be deployed on the target machine -- This message was sent by Atlassian JIRA (v7.6.3#76005)
