[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147500#comment-16147500
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2116
  
@mcgilman good catch! Changes look good. +1 merged to master.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Matt Gilman
> Fix For: 1.4.0
>
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147499#comment-16147499
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2116


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Matt Gilman
> Fix For: 1.4.0
>
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147497#comment-16147497
 ] 

ASF subversion and git services commented on NIFI-4224:
---

Commit 3de0b8edff22616165f6fb0c5beebe31ac27ad17 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=3de0b8e ]

NIFI-4224:
- Ensuring the parent permissions are used when building the variable registry 
for a process group ancestry. This closes #2116.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Matt Gilman
> Fix For: 1.4.0
>
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147391#comment-16147391
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/2116

NIFI-4224: Reopened Variable Registry at a Process Group Level

NIFI-4224:
- Ensuring the parent permissions are used when building the variable 
registry for a process group ancestry.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-4224

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2116.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2116


commit ad53a8c0471db0f6f75ae38f1f4356e5a4cec13b
Author: Matt Gilman 
Date:   2017-08-30T15:02:48Z

NIFI-4224:
- Ensuring the parent permissions are used when building the variable 
registry for a process group ancestry.




> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Matt Gilman
> Fix For: 1.4.0
>
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130804#comment-16130804
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2051


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.4.0
>
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130503#comment-16130503
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2051
  
Thanks @markap14! This has been merged to master.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122297#comment-16122297
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2051
  
@mcgilman thanks for the thorough review! I have pushed a new commit, which 
I believe addresses all of the feedback above.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121861#comment-16121861
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132502384
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121762#comment-16121762
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132477480
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121741#comment-16121741
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132472005
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -314,7 +430,7 @@ public Response updateProcessGroup(
 Authorizable authorizable = 
lookup.getProcessGroup(id).getAuthorizable();
 authorizable.authorize(authorizer, 
RequestAction.WRITE, NiFiUserUtils.getNiFiUser());
 },
-null,
+() -> 
serviceFacade.verifyUpdateProcessGroup(requestProcessGroupDTO),
--- End diff --

Good call. Originally, I had it updating the variables here but then 
refactored quite a bit. I think it's best to go ahead and leave in the addition 
of the verifyUpdateProcessGroup method, but make the verification a NOP. We do 
this in a few other places, as well, so I feel it's best to leave the 
'plumbing' there, since it's already been built. It will make it easier to 
update later.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121651#comment-16121651
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2051
  
@mcgilman thanks for the review. Will update and push a new commit.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120360#comment-16120360
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132257748
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -238,6 +313,47 @@ public Response getProcessGroup(
 return generateOkResponse(entity).build();
 }
 
+
+/**
+ * Retrieves the Variable Registry for the group with the given ID
+ *
+ * @param groupId the ID of the Process Group
+ * @return the Variable Registry for the group
+ */
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{id}/variable-registry")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistry(
--- End diff --

Can we add a flag (probably enabled by default) that will include all 
variables available in this scope (defined in this Process Group and any 
ancestors)? Also, the response should indicate where each variable is defined.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120361#comment-16120361
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132257197
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessGroupDTO.java
 ---
@@ -200,4 +204,13 @@ public void setInactiveRemotePortCount(Integer 
inactiveRemotePortCount) {
 this.inactiveRemotePortCount = inactiveRemotePortCount;
 }
 
+
+@ApiModelProperty("The variables that are configured for the Process 
Group")
+public Map getVariables() {
+return variables;
+}
+
+public void setVariables(final Map variables) {
--- End diff --

Also, can we update the documentation to indicate that this only includes 
the variables defined at this Process Group?


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118891#comment-16118891
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131979744
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -140,6 +185,22 @@
 private TemplateResource templateResource;
 private ControllerServiceResource controllerServiceResource;
 
+private final DtoFactory dtoFactory = new DtoFactory();
--- End diff --

If using the `DtoFactory` here, it can be injected in the Spring context.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118904#comment-16118904
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131996939
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -314,7 +430,7 @@ public Response updateProcessGroup(
 Authorizable authorizable = 
lookup.getProcessGroup(id).getAuthorizable();
 authorizable.authorize(authorizer, 
RequestAction.WRITE, NiFiUserUtils.getNiFiUser());
 },
-null,
+() -> 
serviceFacade.verifyUpdateProcessGroup(requestProcessGroupDTO),
--- End diff --

This is verifying that the variables can be updated but they are ignored in 
the corresponding `updateProcessGroup` call below. Since variable updates are 
not supported in this endpoint, can this check be removed?


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118901#comment-16118901
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131997934
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118893#comment-16118893
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131993623
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
 ---
@@ -640,7 +643,128 @@ public Response scheduleComponents(
 
componentsToSchedule.entrySet().stream().collect(Collectors.toMap(Map.Entry::getKey,
 e -> getRevision(e.getValue(), e.getKey(;
 
 // update the process group
-final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+return generateOkResponse(entity).build();
+}
+);
+}
+
+
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}/controller-services")
+@ApiOperation(value = "Enable or disable Controller Services in the 
specified Process Group.",
+response = ActivateControllerServicesEntity.class,
+authorizations = {
+@Authorization(value = "Read - /flow", type = ""),
+@Authorization(value = "Write - /{component-type}/{uuid} - For 
every service being enabled/disabled", type = "")
+})
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to 
complete the request because it was invalid. The request should not be retried 
without modification."),
+@ApiResponse(code = 401, message = "Client could not 
be authenticated."),
+@ApiResponse(code = 403, message = "Client is not 
authorized to make this request."),
+@ApiResponse(code = 404, message = "The specified 
resource could not be found."),
+@ApiResponse(code = 409, message = "The request was 
valid but NiFi was not in the appropriate state to process it. Retrying the 
same request later may be successful.")
+}
+)
+public Response activateControllerServices(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(value = "The process group id.", required = true)
+@PathParam("id") String id,
+@ApiParam(value = "The request to schedule or unschedule. If 
the comopnents in the request are not specified, all authorized components will 
be considered.", required = true)
+final ActivateControllerServicesEntity requestEntity) {
+
+// ensure the same id is being used
+if (!id.equals(requestEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process 
group id (%s) in the request body does "
++ "not equal the process group id of the requested 
resource (%s).", requestEntity.getId(), id));
+}
+
+final ControllerServiceState state;
+if (requestEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must 
be specified.");
--- End diff --

Copy/paste issue in error message... ControllerServiceState is not a 
'scheduled state'.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118887#comment-16118887
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131983043
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -774,6 +791,80 @@ public RemoteProcessGroupPortEntity 
updateRemoteProcessGroupOutputPort(
 }
 
 @Override
+public Set 
identifyComponentsAffectedByVariableRegistryUpdate(final VariableRegistryDTO 
variableRegistryDto) {
--- End diff --

`NiFiServiceFacadeLock` handles read/write locking based on the method 
name. Assuming this method needs that thread safety, the name of this method 
needs to be accounted for. Either by changing it here or adding another advice 
to intercept this method call. 


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118903#comment-16118903
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132005935
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118898#comment-16118898
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131985592
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardProcessGroupDAO.java
 ---
@@ -145,6 +202,21 @@ public ProcessGroup updateProcessGroup(ProcessGroupDTO 
processGroupDTO) {
 }
 
 @Override
+public ProcessGroup updateVariableRegistry(final VariableRegistryDTO 
variableRegistry) {
--- End diff --

Assuming this action needs to be recorded in the Flow History, the 
`ProcessGroupAuditor` will need to account for this method invocation.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1611#comment-1611
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131983144
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -821,6 +925,46 @@ public ScheduleComponentsEntity 
scheduleComponents(final String processGroupId,
 }
 
 @Override
+public ActivateControllerServicesEntity 
activateControllerServices(final String processGroupId, final 
ControllerServiceState state, final Map serviceRevisions) {
--- End diff --

`NiFiServiceFacadeLock` handles read/write locking based on the method 
name. Assuming this method needs that thread safety, the name of this method 
needs to be accounted for. Either by changing it here or adding another advice 
to intercept this method call.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118899#comment-16118899
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131998712
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118890#comment-16118890
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131985519
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardProcessGroupDAO.java
 ---
@@ -99,29 +116,69 @@ public void verifyScheduleComponents(final String 
groupId, final ScheduledState
 }
 
 @Override
-public void scheduleComponents(final String groupId, final 
ScheduledState state, final Set componentIds) {
+public void verifyActivateControllerServices(final String groupId, 
final ControllerServiceState state, final Set serviceIds) {
+final ProcessGroup group = locateProcessGroup(flowController, 
groupId);
+
+group.findAllControllerServices().stream()
+.filter(service -> 
serviceIds.contains(service.getIdentifier()))
+.forEach(service -> {
+if (state == ControllerServiceState.ENABLED) {
+service.verifyCanEnable();
+} else {
+service.verifyCanDisable();
+}
+});
+}
+
+@Override
+public CompletableFuture scheduleComponents(final String 
groupId, final ScheduledState state, final Set componentIds) {
 final ProcessGroup group = locateProcessGroup(flowController, 
groupId);
 
+CompletableFuture future = 
CompletableFuture.completedFuture(null);
+
 for (final String componentId : componentIds) {
 final Connectable connectable = 
group.findLocalConnectable(componentId);
 if (ScheduledState.RUNNING.equals(state)) {
 if 
(ConnectableType.PROCESSOR.equals(connectable.getConnectableType())) {
-
connectable.getProcessGroup().startProcessor((ProcessorNode) connectable);
+final CompletableFuture processorFuture = 
connectable.getProcessGroup().startProcessor((ProcessorNode) connectable);
+future = CompletableFuture.allOf(future, 
processorFuture);
 } else if 
(ConnectableType.INPUT_PORT.equals(connectable.getConnectableType())) {
 connectable.getProcessGroup().startInputPort((Port) 
connectable);
 } else if 
(ConnectableType.OUTPUT_PORT.equals(connectable.getConnectableType())) {
 connectable.getProcessGroup().startOutputPort((Port) 
connectable);
 }
 } else {
 if 
(ConnectableType.PROCESSOR.equals(connectable.getConnectableType())) {
-
connectable.getProcessGroup().stopProcessor((ProcessorNode) connectable);
+final CompletableFuture processorFuture = 
connectable.getProcessGroup().stopProcessor((ProcessorNode) connectable);
+future = CompletableFuture.allOf(future, 
processorFuture);
 } else if 
(ConnectableType.INPUT_PORT.equals(connectable.getConnectableType())) {
 connectable.getProcessGroup().stopInputPort((Port) 
connectable);
 } else if 
(ConnectableType.OUTPUT_PORT.equals(connectable.getConnectableType())) {
 connectable.getProcessGroup().stopOutputPort((Port) 
connectable);
 }
 }
 }
+
+return future;
+}
+
+@Override
+public Future activateControllerServices(final String groupId, 
final ControllerServiceState state, final Set serviceIds) {
--- End diff --

Assuming this action needs to be recorded in the Flow History, the 
`ProcessGroupAuditor` will need to account for this method invocation.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118892#comment-16118892
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131978526
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/VariableRegistryDTO.java
 ---
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.web.api.dto;
+
+import java.util.Set;
+
+import javax.xml.bind.annotation.XmlType;
+
+import com.wordnik.swagger.annotations.ApiModelProperty;
+
+@XmlType(name = "variableRegistry")
+public class VariableRegistryDTO {
+private Set variables;
--- End diff --

Rather than maintaining a `Set` of `VariableDTO` with a `canUpdate` flag, 
these should probably contain a `Set` of `VariableEntity` which has the 
corresponding permissions set. This would be more consistent with existing DTOs 
for Controller Service references. 


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118896#comment-16118896
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131994112
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
 ---
@@ -640,7 +643,128 @@ public Response scheduleComponents(
 
componentsToSchedule.entrySet().stream().collect(Collectors.toMap(Map.Entry::getKey,
 e -> getRevision(e.getValue(), e.getKey(;
 
 // update the process group
-final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+return generateOkResponse(entity).build();
+}
+);
+}
+
+
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}/controller-services")
+@ApiOperation(value = "Enable or disable Controller Services in the 
specified Process Group.",
+response = ActivateControllerServicesEntity.class,
+authorizations = {
+@Authorization(value = "Read - /flow", type = ""),
+@Authorization(value = "Write - /{component-type}/{uuid} - For 
every service being enabled/disabled", type = "")
+})
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to 
complete the request because it was invalid. The request should not be retried 
without modification."),
+@ApiResponse(code = 401, message = "Client could not 
be authenticated."),
+@ApiResponse(code = 403, message = "Client is not 
authorized to make this request."),
+@ApiResponse(code = 404, message = "The specified 
resource could not be found."),
+@ApiResponse(code = 409, message = "The request was 
valid but NiFi was not in the appropriate state to process it. Retrying the 
same request later may be successful.")
+}
+)
+public Response activateControllerServices(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(value = "The process group id.", required = true)
+@PathParam("id") String id,
+@ApiParam(value = "The request to schedule or unschedule. If 
the comopnents in the request are not specified, all authorized components will 
be considered.", required = true)
+final ActivateControllerServicesEntity requestEntity) {
+
+// ensure the same id is being used
+if (!id.equals(requestEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process 
group id (%s) in the request body does "
++ "not equal the process group id of the requested 
resource (%s).", requestEntity.getId(), id));
+}
+
+final ControllerServiceState state;
+if (requestEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must 
be specified.");
+} else {
+try {
+state = 
ControllerServiceState.valueOf(requestEntity.getState());
+} catch (final IllegalArgumentException iae) {
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [%s].",
--- End diff --

Looks like another copy/paste in this error message. Also, appears to be 
missing 'state' in where the message was copied from.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118897#comment-16118897
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131998273
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118902#comment-16118902
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131997868
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
--- End diff --

Can we set the permissions on this entity for consistency with all of the 
other endpoints?


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118895#comment-16118895
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131983295
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -3062,34 +3206,70 @@ public ProcessGroupEntity getProcessGroup(final 
String groupId) {
 return createProcessGroupEntity(processGroup);
 }
 
-private ControllerServiceEntity createControllerServiceEntity(final 
ControllerServiceNode serviceNode, final Set serviceIds) {
+private ControllerServiceEntity createControllerServiceEntity(final 
ControllerServiceNode serviceNode, final Set serviceIds, final NiFiUser 
user) {
 final ControllerServiceDTO dto = 
dtoFactory.createControllerServiceDto(serviceNode);
 
 final ControllerServiceReference ref = serviceNode.getReferences();
 final ControllerServiceReferencingComponentsEntity 
referencingComponentsEntity = 
createControllerServiceReferencingComponentsEntity(ref, serviceIds);
 
dto.setReferencingComponents(referencingComponentsEntity.getControllerServiceReferencingComponents());
 
 final RevisionDTO revision = 
dtoFactory.createRevisionDTO(revisionManager.getRevision(serviceNode.getIdentifier()));
-final PermissionsDTO permissions = 
dtoFactory.createPermissionsDto(serviceNode);
+final PermissionsDTO permissions = 
dtoFactory.createPermissionsDto(serviceNode, user);
 final List bulletins = 
dtoFactory.createBulletinDtos(bulletinRepository.findBulletinsForSource(serviceNode.getIdentifier()));
 final List bulletinEntities = 
bulletins.stream().map(bulletin -> entityFactory.createBulletinEntity(bulletin, 
permissions.getCanRead())).collect(Collectors.toList());
 return entityFactory.createControllerServiceEntity(dto, revision, 
permissions, bulletinEntities);
 }
 
 @Override
-public Set getControllerServices(final String 
groupId) {
-final Set serviceNodes = 
controllerServiceDAO.getControllerServices(groupId);
+public VariableRegistryEntity getVariableRegistry(final String 
groupId) {
+final ProcessGroup processGroup = 
processGroupDAO.getProcessGroup(groupId);
+if (processGroup == null) {
+throw new ResourceNotFoundException("Could not find group with 
ID " + groupId);
+}
+
+return createVariableRegistryEntity(processGroup);
+}
+
+private VariableRegistryEntity createVariableRegistryEntity(final 
ProcessGroup processGroup) {
+final VariableRegistryDTO registryDto = 
dtoFactory.createVariableRegistryDto(processGroup);
+final RevisionDTO revision = 
dtoFactory.createRevisionDTO(revisionManager.getRevision(processGroup.getIdentifier()));
+final PermissionsDTO permissions = 
dtoFactory.createPermissionsDto(processGroup);
+return entityFactory.createVariableRegistryEntity(registryDto, 
revision, permissions);
+}
+
+@Override
+public VariableRegistryEntity populateAffectedComponents(final 
VariableRegistryDTO variableRegistryDto) {
--- End diff --

`NiFiServiceFacadeLock` handles read/write locking based on the method 
name. Assuming this method needs that thread safety, the name of this method 
needs to be accounted for. Either by changing it here or adding another advice 
to intercept this method call.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118900#comment-16118900
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131999511
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118894#comment-16118894
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131995022
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
 ---
@@ -640,7 +643,128 @@ public Response scheduleComponents(
 
componentsToSchedule.entrySet().stream().collect(Collectors.toMap(Map.Entry::getKey,
 e -> getRevision(e.getValue(), e.getKey(;
 
 // update the process group
-final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+return generateOkResponse(entity).build();
+}
+);
+}
+
+
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}/controller-services")
+@ApiOperation(value = "Enable or disable Controller Services in the 
specified Process Group.",
+response = ActivateControllerServicesEntity.class,
+authorizations = {
+@Authorization(value = "Read - /flow", type = ""),
+@Authorization(value = "Write - /{component-type}/{uuid} - For 
every service being enabled/disabled", type = "")
+})
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to 
complete the request because it was invalid. The request should not be retried 
without modification."),
+@ApiResponse(code = 401, message = "Client could not 
be authenticated."),
+@ApiResponse(code = 403, message = "Client is not 
authorized to make this request."),
+@ApiResponse(code = 404, message = "The specified 
resource could not be found."),
+@ApiResponse(code = 409, message = "The request was 
valid but NiFi was not in the appropriate state to process it. Retrying the 
same request later may be successful.")
+}
+)
+public Response activateControllerServices(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(value = "The process group id.", required = true)
+@PathParam("id") String id,
+@ApiParam(value = "The request to schedule or unschedule. If 
the comopnents in the request are not specified, all authorized components will 
be considered.", required = true)
+final ActivateControllerServicesEntity requestEntity) {
+
+// ensure the same id is being used
+if (!id.equals(requestEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process 
group id (%s) in the request body does "
++ "not equal the process group id of the requested 
resource (%s).", requestEntity.getId(), id));
+}
+
+final ControllerServiceState state;
+if (requestEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must 
be specified.");
+} else {
+try {
+state = 
ControllerServiceState.valueOf(requestEntity.getState());
+} catch (final IllegalArgumentException iae) {
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [%s].",
+
StringUtils.join(EnumSet.of(ControllerServiceState.ENABLED, 
ControllerServiceState.DISABLED), ", ")));
+}
+}
+
+// ensure its a supported scheduled state
+if (ControllerServiceState.DISABLING.equals(state) || 
ControllerServiceState.ENABLING.equals(state)) {
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [%s].",
+
StringUtils.join(EnumSet.of(ControllerServiceState.ENABLED, 
ControllerServiceState.DISABLED), ", ")));
+}
+
+// if the components are not specified, gather all components and 
their current revision
+if (requestEntity.getComponents() == null) {
+// get the current revisions for the components being updated
+final Set revisions = 
serviceFacade.getRevisionsFromGroup(id, group -> {
+final Set componentIds = new HashSet<>();
+
+final Predicate filter;
 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118886#comment-16118886
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131983689
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -821,6 +925,46 @@ public ScheduleComponentsEntity 
scheduleComponents(final String processGroupId,
 }
 
 @Override
+public ActivateControllerServicesEntity 
activateControllerServices(final String processGroupId, final 
ControllerServiceState state, final Map serviceRevisions) {
+
+final NiFiUser user = NiFiUserUtils.getNiFiUser();
+return activateControllerServices(user, processGroupId, state, 
serviceRevisions);
+}
+
+@Override
+public ActivateControllerServicesEntity 
activateControllerServices(final NiFiUser user, final String processGroupId, 
final ControllerServiceState state,
--- End diff --

`NiFiServiceFacadeLock` handles read/write locking based on the method 
name. Assuming this method needs that thread safety, the name of this method 
needs to be accounted for. Either by changing it here or adding another advice 
to intercept this method call.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118889#comment-16118889
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131979395
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessGroupDTO.java
 ---
@@ -200,4 +204,13 @@ public void setInactiveRemotePortCount(Integer 
inactiveRemotePortCount) {
 this.inactiveRemotePortCount = inactiveRemotePortCount;
 }
 
+
+@ApiModelProperty("The variables that are configured for the Process 
Group")
+public Map getVariables() {
+return variables;
+}
+
+public void setVariables(final Map variables) {
--- End diff --

These should be set when populating the DTO in 
`DtoFactory.createConciseProcessGroupDto(...)`.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113249#comment-16113249
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2051
  
Will review...


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113154#comment-16113154
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/2051

NIFI-4224: Initial implementation of Process Group level Variable Reg…

…istry

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-4224

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2051.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2051


commit 3c43dedb2038a3c3e674c0be876c43010d672938
Author: Mark Payne 
Date:   2017-07-25T17:46:10Z

NIFI-4224: Initial implementation of Process Group level Variable Registry




> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)