[GitHub] nifi pull request: NIFI-786 AWS credential refactoring and enhance...

[jskora]

Github user jskora commented on the pull request:

https://github.com/apache/nifi/pull/244#issuecomment-221163835
  
This is really nice, it was great being able to just provide the AWS CLI 
profile and have setting automatically picked up.

Great improvement, thanks James!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-786 AWS credential refactoring and enhance...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/244


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1884 Defining API for Users, Groups, and P...

[mcgilman]

Github user mcgilman commented on the pull request:

https://github.com/apache/nifi/pull/452#issuecomment-221086451
  
@bbende The updated PR looks good to me. A number of great iterations here. 
Definitely think this is a solid foundation to continue the policy based 
authorizers managed by NiFi. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1799 Implements auto-scaling flow layout

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/442#discussion_r64277394
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/groups/StandardProcessGroup.java
 ---
@@ -2168,6 +2170,16 @@ private boolean isDisconnected(final Snippet 
snippet) {
 return true;
 }
 
+@Override
+public Set findAllPositionables() {
+Set positionables = Sets.newHashSet();
+positionables.addAll(findAllConnectables(this, true));
+positionables.addAll(findAllProcessGroups());
--- End diff --

We should check to see if this recurses within nested process groups when 
scaling top-level process group. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1660 - Enhance the expression language wit...

[ckmcd]

Github user ckmcd commented on the pull request:

https://github.com/apache/nifi/pull/303#issuecomment-221073806
  
I've rebase on master.  Other suggested changes are in the works.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1913 Properly removing flowfile in HandleH...

[pvillard31]

Github user pvillard31 commented on the pull request:

https://github.com/apache/nifi/pull/462#issuecomment-221058590
  
There is a double space on line 160 in test class. Travis seems not to be 
run on 0.x branch so I don't know if contrib check would fail because of that 
(cannot personally check right now).
Otherwise, +1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/461


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[mcgilman]

Github user mcgilman commented on the pull request:

https://github.com/apache/nifi/pull/461#issuecomment-221051044
  
Thanks @markap14 

All changes have been applied.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[mcgilman]

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64262070
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -2024,6 +1843,24 @@ public Response instantiateTemplate(
 // templates
 // -
 
+private void authorizeSnippetUsage(final AuthorizableLookup lookup, 
final String groupId, final String snippetId) {
+// ensure write access to the target process group
+lookup.getProcessGroup(groupId).authorize(authorizer, 
RequestAction.WRITE);
+
+// ensure read permission to every component in the snippet
+final Snippet snippet = lookup.getSnippet(snippetId);
+final Set authorizables = new HashSet<>();
+
authorizables.addAll(snippet.getProcessGroups().keySet().stream().map(id -> 
lookup.getProcessGroup(id)).collect(Collectors.toSet()));
--- End diff --

Awesome. Will do!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request: MINIFI-6: Basic C++ implementation f...

[simonellistonball]

Github user simonellistonball commented on the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/4#issuecomment-221049133
  
Tried building this with the default settings (-O0), which yields 1.3MB 
binary. Is the intention to provide a different build script for more 
constrained environments, (-Os yields 459K binary).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64259242
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/SnippetResource.java
 ---
@@ -0,0 +1,397 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api;
+
+import com.wordnik.swagger.annotations.Api;
+import com.wordnik.swagger.annotations.ApiOperation;
+import com.wordnik.swagger.annotations.ApiParam;
+import com.wordnik.swagger.annotations.ApiResponse;
+import com.wordnik.swagger.annotations.ApiResponses;
+import com.wordnik.swagger.annotations.Authorization;
+import org.apache.nifi.authorization.Authorizer;
+import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
+import org.apache.nifi.authorization.user.NiFiUser;
+import org.apache.nifi.authorization.user.NiFiUserUtils;
+import org.apache.nifi.cluster.manager.impl.WebClusterManager;
+import org.apache.nifi.controller.Snippet;
+import org.apache.nifi.util.NiFiProperties;
+import org.apache.nifi.web.NiFiServiceFacade;
+import org.apache.nifi.web.Revision;
+import org.apache.nifi.web.api.dto.SnippetDTO;
+import org.apache.nifi.web.api.entity.SnippetEntity;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.Consumes;
+import javax.ws.rs.DELETE;
+import javax.ws.rs.HttpMethod;
+import javax.ws.rs.POST;
+import javax.ws.rs.PUT;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import java.net.URI;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * RESTful endpoint for querying dataflow snippets.
+ */
+@Path("/snippets")
+@Api(
+value = "/snippets",
+description = "Endpoint for accessing dataflow snippets."
+)
+public class SnippetResource extends ApplicationResource {
+
+private NiFiServiceFacade serviceFacade;
+private WebClusterManager clusterManager;
+private NiFiProperties properties;
+private Authorizer authorizer;
+
+/**
+ * Populate the uri's for the specified snippet.
+ *
+ * @param entity processors
+ * @return dtos
+ */
+private SnippetEntity 
populateRemainingSnippetEntityContent(SnippetEntity entity) {
+if (entity.getSnippet() != null) {
+populateRemainingSnippetContent(entity.getSnippet());
+}
+return entity;
+}
+
+/**
+ * Populates the uri for the specified snippet.
+ */
+private SnippetDTO populateRemainingSnippetContent(SnippetDTO snippet) 
{
+String snippetGroupId = snippet.getParentGroupId();
+
+// populate the snippet href
+snippet.setUri(generateResourceUri("process-groups", 
snippetGroupId, "snippets", snippet.getId()));
+
+return snippet;
+}
+
+// 
+// snippets
+// 
+
+/**
+ * Creates a snippet based off the specified configuration.
+ *
+ * @param httpServletRequest request
+ * @param snippetEntity A snippetEntity
+ * @return A snippetEntity
+ */
+@POST
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Creates a snippet",
+response = SnippetEntity.class,
+authorizations = {
+@Authorization(value = "Read Only", type = "ROLE_MONITOR"),
+@Authorization(value = "Data Flow Manager", type = "ROLE_DFM"),
+@Authorization(value = "Administrator", type = "ROLE_ADMIN")

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[mcgilman]

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64259949
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
 ---
@@ -321,6 +320,157 @@ public Response getControllerServices(
 return clusterContext(generateOkResponse(entity)).build();
 }
 
+/**
+ * Updates the specified process group.
+ *
+ * @param httpServletRequest request
+ * @param id The id of the process group.
+ * @param scheduleComponentsEntity A scheduleComponentsEntity.
+ * @return A processGroupEntity.
+ */
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}")
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Updates a process group",
+response = ScheduleComponentsEntity.class,
+authorizations = {
+@Authorization(value = "Data Flow Manager", type = "ROLE_DFM")
+}
+)
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to 
complete the request because it was invalid. The request should not be retried 
without modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized 
to make this request."),
+@ApiResponse(code = 404, message = "The specified resource 
could not be found."),
+@ApiResponse(code = 409, message = "The request was valid but 
NiFi was not in the appropriate state to process it. Retrying the same request 
later may be successful.")
+}
+)
+public Response scheduleComponents(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(
+value = "The process group id.",
+required = true
+)
+@PathParam("id") String id,
+ScheduleComponentsEntity scheduleComponentsEntity) {
+
+authorizeFlow();
+
+// ensure the same id is being used
+if (!id.equals(scheduleComponentsEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process 
group id (%s) in the request body does "
++ "not equal the process group id of the requested 
resource (%s).", scheduleComponentsEntity.getId(), id));
+}
+
+final ScheduledState state;
+if (scheduleComponentsEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must 
be specified.");
+} else {
+try {
+state = 
ScheduledState.valueOf(scheduleComponentsEntity.getState());
+} catch (final IllegalArgumentException iae) {
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [].", 
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ", 
")));
+}
+}
+
+// ensure its a supported scheduled state
+if (ScheduledState.DISABLED.equals(state) || 
ScheduledState.STARTING.equals(state) || ScheduledState.STOPPING.equals(state)) 
{
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [].", 
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ", 
")));
+}
+
+// if the components are not specified, gather all components and 
their current revision
+if (scheduleComponentsEntity.getComponents() == null) {
+// TODO - this will break while clustered until nodes are able 
to process/replicate requests
+// get the current revisions for the components being updated
+final Set revisions = 
serviceFacade.getRevisionsFromGroup(id, group -> {
+final Set componentIds = new HashSet<>();
+
+// ensure authorized for each processor we will attempt to 
schedule
+
group.findAllProcessors().stream().filter(ScheduledState.RUNNING.equals(state) 
? ProcessGroup.SCHEDULABLE_PROCESSORS : 
ProcessGroup.UNSCHEDULABLE_PROCESSORS).forEach(processor -> {
+if (processor.isAuthorized(authorizer, 
RequestAction.WRITE)) {
--- End diff --

Will do!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket

[GitHub] nifi pull request: NIFI-1884 Defining API for Users, Groups, and P...

[bbende]

Github user bbende commented on the pull request:

https://github.com/apache/nifi/pull/452#issuecomment-221046683
  
@mcgilman @jtstorck @alopresto pushed two new commits, the first contains 
minor changes to address some of Matt's comments, the second is an attempt at 
what Matt suggested about how the MutableAuthorizer could be an abstract class 
that provides an implementation of authorize(), let me know what you think


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64259092
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/SnippetResource.java
 ---
@@ -0,0 +1,397 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api;
+
+import com.wordnik.swagger.annotations.Api;
+import com.wordnik.swagger.annotations.ApiOperation;
+import com.wordnik.swagger.annotations.ApiParam;
+import com.wordnik.swagger.annotations.ApiResponse;
+import com.wordnik.swagger.annotations.ApiResponses;
+import com.wordnik.swagger.annotations.Authorization;
+import org.apache.nifi.authorization.Authorizer;
+import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
+import org.apache.nifi.authorization.user.NiFiUser;
+import org.apache.nifi.authorization.user.NiFiUserUtils;
+import org.apache.nifi.cluster.manager.impl.WebClusterManager;
+import org.apache.nifi.controller.Snippet;
+import org.apache.nifi.util.NiFiProperties;
+import org.apache.nifi.web.NiFiServiceFacade;
+import org.apache.nifi.web.Revision;
+import org.apache.nifi.web.api.dto.SnippetDTO;
+import org.apache.nifi.web.api.entity.SnippetEntity;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.Consumes;
+import javax.ws.rs.DELETE;
+import javax.ws.rs.HttpMethod;
+import javax.ws.rs.POST;
+import javax.ws.rs.PUT;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import java.net.URI;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * RESTful endpoint for querying dataflow snippets.
+ */
+@Path("/snippets")
+@Api(
+value = "/snippets",
+description = "Endpoint for accessing dataflow snippets."
+)
+public class SnippetResource extends ApplicationResource {
+
+private NiFiServiceFacade serviceFacade;
+private WebClusterManager clusterManager;
+private NiFiProperties properties;
+private Authorizer authorizer;
+
+/**
+ * Populate the uri's for the specified snippet.
+ *
+ * @param entity processors
+ * @return dtos
+ */
+private SnippetEntity 
populateRemainingSnippetEntityContent(SnippetEntity entity) {
+if (entity.getSnippet() != null) {
+populateRemainingSnippetContent(entity.getSnippet());
+}
+return entity;
+}
+
+/**
+ * Populates the uri for the specified snippet.
+ */
+private SnippetDTO populateRemainingSnippetContent(SnippetDTO snippet) 
{
+String snippetGroupId = snippet.getParentGroupId();
+
+// populate the snippet href
+snippet.setUri(generateResourceUri("process-groups", 
snippetGroupId, "snippets", snippet.getId()));
+
+return snippet;
+}
+
+// 
+// snippets
+// 
+
+/**
+ * Creates a snippet based off the specified configuration.
+ *
+ * @param httpServletRequest request
+ * @param snippetEntity A snippetEntity
+ * @return A snippetEntity
+ */
+@POST
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Creates a snippet",
+response = SnippetEntity.class,
+authorizations = {
+@Authorization(value = "Read Only", type = "ROLE_MONITOR"),
+@Authorization(value = "Data Flow Manager", type = "ROLE_DFM"),
+@Authorization(value = "Administrator", type = "ROLE_ADMIN")

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64256472
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
 ---
@@ -321,6 +320,157 @@ public Response getControllerServices(
 return clusterContext(generateOkResponse(entity)).build();
 }
 
+/**
+ * Updates the specified process group.
+ *
+ * @param httpServletRequest request
+ * @param id The id of the process group.
+ * @param scheduleComponentsEntity A scheduleComponentsEntity.
+ * @return A processGroupEntity.
+ */
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}")
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Updates a process group",
+response = ScheduleComponentsEntity.class,
+authorizations = {
+@Authorization(value = "Data Flow Manager", type = "ROLE_DFM")
+}
+)
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to 
complete the request because it was invalid. The request should not be retried 
without modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized 
to make this request."),
+@ApiResponse(code = 404, message = "The specified resource 
could not be found."),
+@ApiResponse(code = 409, message = "The request was valid but 
NiFi was not in the appropriate state to process it. Retrying the same request 
later may be successful.")
+}
+)
+public Response scheduleComponents(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(
+value = "The process group id.",
+required = true
+)
+@PathParam("id") String id,
+ScheduleComponentsEntity scheduleComponentsEntity) {
+
+authorizeFlow();
+
+// ensure the same id is being used
+if (!id.equals(scheduleComponentsEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process 
group id (%s) in the request body does "
++ "not equal the process group id of the requested 
resource (%s).", scheduleComponentsEntity.getId(), id));
+}
+
+final ScheduledState state;
+if (scheduleComponentsEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must 
be specified.");
+} else {
+try {
+state = 
ScheduledState.valueOf(scheduleComponentsEntity.getState());
+} catch (final IllegalArgumentException iae) {
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [].", 
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ", 
")));
+}
+}
+
+// ensure its a supported scheduled state
+if (ScheduledState.DISABLED.equals(state) || 
ScheduledState.STARTING.equals(state) || ScheduledState.STOPPING.equals(state)) 
{
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [].", 
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ", 
")));
+}
+
+// if the components are not specified, gather all components and 
their current revision
+if (scheduleComponentsEntity.getComponents() == null) {
+// TODO - this will break while clustered until nodes are able 
to process/replicate requests
+// get the current revisions for the components being updated
+final Set revisions = 
serviceFacade.getRevisionsFromGroup(id, group -> {
+final Set componentIds = new HashSet<>();
+
+// ensure authorized for each processor we will attempt to 
schedule
+
group.findAllProcessors().stream().filter(ScheduledState.RUNNING.equals(state) 
? ProcessGroup.SCHEDULABLE_PROCESSORS : 
ProcessGroup.UNSCHEDULABLE_PROCESSORS).forEach(processor -> {
+if (processor.isAuthorized(authorizer, 
RequestAction.WRITE)) {
+componentIds.add(processor.getIdentifier());
+}
+});
+
+// ensure authorized for each input port we will attempt 
to schedule
+
group.findAllInputPorts().stream().filter(ScheduledState.RUNNING.equals(state) 
? 

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64256311
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
 ---
@@ -321,6 +320,157 @@ public Response getControllerServices(
 return clusterContext(generateOkResponse(entity)).build();
 }
 
+/**
+ * Updates the specified process group.
+ *
+ * @param httpServletRequest request
+ * @param id The id of the process group.
+ * @param scheduleComponentsEntity A scheduleComponentsEntity.
+ * @return A processGroupEntity.
+ */
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}")
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Updates a process group",
+response = ScheduleComponentsEntity.class,
+authorizations = {
+@Authorization(value = "Data Flow Manager", type = "ROLE_DFM")
+}
+)
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to 
complete the request because it was invalid. The request should not be retried 
without modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized 
to make this request."),
+@ApiResponse(code = 404, message = "The specified resource 
could not be found."),
+@ApiResponse(code = 409, message = "The request was valid but 
NiFi was not in the appropriate state to process it. Retrying the same request 
later may be successful.")
+}
+)
+public Response scheduleComponents(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(
+value = "The process group id.",
+required = true
+)
+@PathParam("id") String id,
+ScheduleComponentsEntity scheduleComponentsEntity) {
+
+authorizeFlow();
+
+// ensure the same id is being used
+if (!id.equals(scheduleComponentsEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process 
group id (%s) in the request body does "
++ "not equal the process group id of the requested 
resource (%s).", scheduleComponentsEntity.getId(), id));
+}
+
+final ScheduledState state;
+if (scheduleComponentsEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must 
be specified.");
+} else {
+try {
+state = 
ScheduledState.valueOf(scheduleComponentsEntity.getState());
+} catch (final IllegalArgumentException iae) {
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [].", 
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ", 
")));
+}
+}
+
+// ensure its a supported scheduled state
+if (ScheduledState.DISABLED.equals(state) || 
ScheduledState.STARTING.equals(state) || ScheduledState.STOPPING.equals(state)) 
{
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [].", 
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ", 
")));
+}
+
+// if the components are not specified, gather all components and 
their current revision
+if (scheduleComponentsEntity.getComponents() == null) {
+// TODO - this will break while clustered until nodes are able 
to process/replicate requests
+// get the current revisions for the components being updated
+final Set revisions = 
serviceFacade.getRevisionsFromGroup(id, group -> {
+final Set componentIds = new HashSet<>();
+
+// ensure authorized for each processor we will attempt to 
schedule
+
group.findAllProcessors().stream().filter(ScheduledState.RUNNING.equals(state) 
? ProcessGroup.SCHEDULABLE_PROCESSORS : 
ProcessGroup.UNSCHEDULABLE_PROCESSORS).forEach(processor -> {
+if (processor.isAuthorized(authorizer, 
RequestAction.WRITE)) {
--- End diff --

It seems cleaner to me to use a filter here to check if the processor is 
authorized, rather than mixing stream filters with a forEach and then 
performing additional if-checks. I.e., instead of "if 
(processor.isAuthorized...) {...}" we could just do ".filter(processor -> 
processor.isAuthorized(...).forEach(...)" 

[GitHub] nifi pull request: NIFI-1913 Properly removing flowfile in HandleH...

[JPercivall]

GitHub user JPercivall opened a pull request:

https://github.com/apache/nifi/pull/462

NIFI-1913 Properly removing flowfile in HandleHttpRequest when the Co…

…ntext Map fails to register the request

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JPercivall/nifi NIFI-1913

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/462.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #462


commit 44013976749d377706999fc49a1a4d3014909a69
Author: jpercivall 
Date:   2016-05-23T16:24:43Z

NIFI-1913 Properly removing flowfile in HandleHttpRequest when the Context 
Map fails to register the request




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[mcgilman]

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64251051
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/groups/ProcessGroup.java
 ---
@@ -46,6 +48,26 @@
 public interface ProcessGroup extends Authorizable {
 
 /**
+ * Predicate for filtering schedulable Processors.
+ */
+Predicate SCHEDULABLE_PROCESSORS = node -> 
!node.isRunning() && node.getScheduledState() != ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering unschedulable Processors.
+ */
+Predicate UNSCHEDULABLE_PROCESSORS = node -> 
node.isRunning();
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate SCHEDULABLE_PORTS = port -> port.getScheduledState() 
!= ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate UNSCHEDULABLE_PORTS = port -> port.getScheduledState() 
== ScheduledState.RUNNING;
--- End diff --

Happy to update the logic to whatever it needs to be. However, this is how 
it was previously.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[mcgilman]

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64250889
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -568,164 +699,50 @@ public void verifyDeleteReportingTask(String 
reportingTaskId) {
 
 
 @Override
-public void verifyUpdateSnippet(SnippetDTO snippetDto) {
+public void verifyUpdateSnippet(SnippetDTO snippetDto, final 
Set affectedComponentIds) {
 try {
 // if snippet does not exist, then the update request is 
likely creating it
 // so we don't verify since it will fail
 if (snippetDAO.hasSnippet(snippetDto.getId())) {
 snippetDAO.verifyUpdate(snippetDto);
 }
 } catch (final Exception e) {
-revisionManager.cancelClaim(snippetDto.getId());
+affectedComponentIds.forEach(id -> 
revisionManager.cancelClaim(snippetDto.getId()));
 throw e;
 }
 }
 
-private Set getRevisionsForGroup(final String groupId) {
-final Set revisions = new HashSet<>();
-
-revisions.add(revisionManager.getRevision(groupId));
-final ProcessGroup processGroup = 
processGroupDAO.getProcessGroup(groupId);
-if (processGroup == null) {
-throw new IllegalArgumentException("Snippet contains a 
reference to Process Group with ID " + groupId + " but no Process Group exists 
with that ID");
-}
-
-processGroup.getConnections().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getFunnels().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getInputPorts().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getOutputPorts().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getLabels().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getProcessors().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getRemoteProcessGroups().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getProcessGroups().stream().map(c -> 
c.getIdentifier()).forEach(id -> revisions.addAll(getRevisionsForGroup(id)));
-
-return revisions;
-}
-
-private Set getRevisionsForSnippet(final SnippetDTO 
snippetDto) {
-final Set requiredRevisions = new HashSet<>();
-
requiredRevisions.add(revisionManager.getRevision(snippetDto.getId()));
-snippetDto.getConnections().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getFunnels().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getInputPorts().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getOutputPorts().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getLabels().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getProcessors().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getRemoteProcessGroups().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-  

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[mcgilman]

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64250267
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/groups/StandardProcessGroup.java
 ---
@@ -283,38 +283,21 @@ public boolean isRootGroup() {
 public void startProcessing() {
 readLock.lock();
 try {
-for (final ProcessorNode node : processors.values()) {
+
findAllProcessors().stream().filter(SCHEDULABLE_PROCESSORS).forEach(node -> {
--- End diff --

Totally agree. There are a number of improvements that we can make with 
Java 8 available. Let's create a separate JIRA to identify those.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[mcgilman]

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64250769
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/groups/ProcessGroup.java
 ---
@@ -46,6 +48,26 @@
 public interface ProcessGroup extends Authorizable {
 
 /**
+ * Predicate for filtering schedulable Processors.
+ */
+Predicate SCHEDULABLE_PROCESSORS = node -> 
!node.isRunning() && node.getScheduledState() != ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering unschedulable Processors.
+ */
+Predicate UNSCHEDULABLE_PROCESSORS = node -> 
node.isRunning();
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate SCHEDULABLE_PORTS = port -> port.getScheduledState() 
!= ScheduledState.DISABLED;
--- End diff --

I did not modify any logic here. Simply relocated to not repeat. If that 
logic is incorrect, I can certainly update it, but it hasn't changed here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64249465
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -568,164 +699,50 @@ public void verifyDeleteReportingTask(String 
reportingTaskId) {
 
 
 @Override
-public void verifyUpdateSnippet(SnippetDTO snippetDto) {
+public void verifyUpdateSnippet(SnippetDTO snippetDto, final 
Set affectedComponentIds) {
 try {
 // if snippet does not exist, then the update request is 
likely creating it
 // so we don't verify since it will fail
 if (snippetDAO.hasSnippet(snippetDto.getId())) {
 snippetDAO.verifyUpdate(snippetDto);
 }
 } catch (final Exception e) {
-revisionManager.cancelClaim(snippetDto.getId());
+affectedComponentIds.forEach(id -> 
revisionManager.cancelClaim(snippetDto.getId()));
 throw e;
 }
 }
 
-private Set getRevisionsForGroup(final String groupId) {
-final Set revisions = new HashSet<>();
-
-revisions.add(revisionManager.getRevision(groupId));
-final ProcessGroup processGroup = 
processGroupDAO.getProcessGroup(groupId);
-if (processGroup == null) {
-throw new IllegalArgumentException("Snippet contains a 
reference to Process Group with ID " + groupId + " but no Process Group exists 
with that ID");
-}
-
-processGroup.getConnections().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getFunnels().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getInputPorts().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getOutputPorts().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getLabels().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getProcessors().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getRemoteProcessGroups().stream().map(c -> 
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev -> 
revisions.add(rev));
-processGroup.getProcessGroups().stream().map(c -> 
c.getIdentifier()).forEach(id -> revisions.addAll(getRevisionsForGroup(id)));
-
-return revisions;
-}
-
-private Set getRevisionsForSnippet(final SnippetDTO 
snippetDto) {
-final Set requiredRevisions = new HashSet<>();
-
requiredRevisions.add(revisionManager.getRevision(snippetDto.getId()));
-snippetDto.getConnections().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getFunnels().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getInputPorts().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getOutputPorts().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getLabels().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getProcessors().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getRemoteProcessGroups().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(), 
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-  

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64248276
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/groups/StandardProcessGroup.java
 ---
@@ -283,38 +283,21 @@ public boolean isRootGroup() {
 public void startProcessing() {
 readLock.lock();
 try {
-for (final ProcessorNode node : processors.values()) {
+
findAllProcessors().stream().filter(SCHEDULABLE_PROCESSORS).forEach(node -> {
--- End diff --

It seems like here, rather than finding all processors and putting them 
into a set, then converting to a stream and filtering, we should allow a 
Predicate to be passed into the findAllProcessors() method? Same for 
input/output ports


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64248023
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/groups/ProcessGroup.java
 ---
@@ -46,6 +48,26 @@
 public interface ProcessGroup extends Authorizable {
 
 /**
+ * Predicate for filtering schedulable Processors.
+ */
+Predicate SCHEDULABLE_PROCESSORS = node -> 
!node.isRunning() && node.getScheduledState() != ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering unschedulable Processors.
+ */
+Predicate UNSCHEDULABLE_PROCESSORS = node -> 
node.isRunning();
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate SCHEDULABLE_PORTS = port -> port.getScheduledState() 
!= ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate UNSCHEDULABLE_PORTS = port -> port.getScheduledState() 
== ScheduledState.RUNNING;
--- End diff --

Shouldn't this be "port.isRunning()" instead, like we do for Processor? It 
seems like we could actually just have a SCHEDULABLE_TRIGGERABLE and an 
UNSCHEDULABLE_TRIGGERABLE rather than separate ones for ports & processors?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/461#discussion_r64247888
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/groups/ProcessGroup.java
 ---
@@ -46,6 +48,26 @@
 public interface ProcessGroup extends Authorizable {
 
 /**
+ * Predicate for filtering schedulable Processors.
+ */
+Predicate SCHEDULABLE_PROCESSORS = node -> 
!node.isRunning() && node.getScheduledState() != ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering unschedulable Processors.
+ */
+Predicate UNSCHEDULABLE_PROCESSORS = node -> 
node.isRunning();
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate SCHEDULABLE_PORTS = port -> port.getScheduledState() 
!= ScheduledState.DISABLED;
--- End diff --

It seems odd to me that we are checking "!node.isRunning()" for 
ProcessorsNode but not checking "!port.isRunning()" for ports.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1660 - Enhance the expression language wit...

[ckmcd]

Github user ckmcd commented on the pull request:

https://github.com/apache/nifi/pull/303#issuecomment-221019823
  
@mattyb149, @markap14  I am working on your suggestions.  Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1781: UI authorization updates

[mcgilman]

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/461

NIFI-1781: UI authorization updates

NIFI-1781:
- Including access policies in the breadcrumb's trail.
- Updating toolbox according to group access policies.
- Updating actions in palette based on selection access policies.
NIFI-1554:
- Introducing authorization during two phase commit.
- Introducing snippet authorization according to the encapsulated 
components and the action performed.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-1781

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/461.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #461






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1660 - Enhance the expression language wit...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/303#discussion_r64235176
  
--- Diff: 
nifi-commons/nifi-expression-language/src/test/java/org/apache/nifi/attribute/expression/language/TestQuery.java
 ---
@@ -233,6 +233,24 @@ public void testEmbeddedExpressionsAndQuotes() {
 }
 
 @Test
+public void testJsonPath() {
+final Map attributes = new HashMap<>();
+attributes.put("json",
--- End diff --

Would recommend we pull the JSON out into a file in src/test/resources. 
Having this live in the codebase makes it very difficult to read & edit


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1660 - Enhance the expression language wit...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/303#discussion_r64234770
  
--- Diff: 
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/JsonPathEvaluator.java
 ---
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.functions;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import 
org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+import 
org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
+import 
org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
+
+import com.jayway.jsonpath.Configuration;
+import com.jayway.jsonpath.DocumentContext;
+import com.jayway.jsonpath.InvalidJsonException;
+import com.jayway.jsonpath.JsonPath;
+import com.jayway.jsonpath.spi.json.JacksonJsonProvider;
+import com.jayway.jsonpath.spi.json.JsonProvider;
+
+
+public class JsonPathEvaluator extends StringEvaluator {
+
+private static final StringQueryResult NULL_RESULT = new 
StringQueryResult("");
+private static final Configuration STRICT_PROVIDER_CONFIGURATION = 
Configuration.builder().jsonProvider(new JacksonJsonProvider()).build();
+private static final JsonProvider JSON_PROVIDER = 
STRICT_PROVIDER_CONFIGURATION.jsonProvider();
+
+private final Evaluator subject;
+private final Evaluator jsonPathExp;
+
+public JsonPathEvaluator(final Evaluator subject, final 
Evaluator jsonPathExp) {
+this.subject = subject;
+this.jsonPathExp = jsonPathExp;
+}
+
+@Override
+public QueryResult evaluate(final Map 
attributes) {
+final String subjectValue = 
subject.evaluate(attributes).getValue();
+if (subjectValue == null || subjectValue.length() == 0) {
+return NULL_RESULT;
+}
+DocumentContext documentContext = null;
+try {
+documentContext = 
validateAndEstablishJsonContext(subjectValue);
+} catch (InvalidJsonException e) {
+return NULL_RESULT;
+}
+
+final JsonPath compiledJsonPath = 
JsonPath.compile(jsonPathExp.evaluate(attributes).getValue());
--- End diff --

In the vast majority of use cases, I would expect that the JSON Path will 
be a literal string that does not reference any attributes. In this case, I 
would want to avoid compiling the JSON Path every time. Would recommend that if 
the jsonPathExp is an instance of the StringLiteralEvaluator, we pre-compile 
the JSON Path. We follow a similar pattern with the MatchesEvaluator for 
pre-compiling regular expressions, and it can make a pretty huge difference on 
performance.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1907 Moving lazy init of SSLContext...

[joewitt]

Github user joewitt commented on the pull request:

https://github.com/apache/nifi/pull/457#issuecomment-220988156
  
@bbende i think the move makes sense to improve the lifecycle of when such 
issues are detected.  However, it does seem like throwing RuntimeExceptions for 
those could be improved.  Could they instead be considered 
IllegalStateExceptions and be a documented part of that interface?  Also for 
the last runtime exception should we include the underlying exception when we 
create a new exception or do we intentionally bury it here?

That said these questions/concerns are not 'because of this change' and i 
do agree the context should be constructed during the building phase of the 
client so I'm a +1 anyway.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

RE: EXTERNAL: Re: Provenance disabled

[Gresock, Joseph]

Ah, so obvious.  That was it, thanks =)

Joe Gresock
AMPERE TTO
Software Engineer Stf
Lockheed Martin MST
joseph.gres...@lmco.com

-Original Message-
From: Simon Elliston Ball [mailto:si...@simonellistonball.com] 
Sent: Monday, May 23, 2016 9:06 AM
To: dev@nifi.apache.org
Subject: EXTERNAL: Re: Provenance disabled

This will happen if you are authenticated as a user who does have the 
provenance role. That’s certainly the most common cause. There are other 
configurations which can do this, but they are unlikely on a clean install. Can 
you share anything about the authentication / provenance config?

Simon

> On 23 May 2016, at 14:03, Gresock, Joseph  wrote:
> 
> Hi folks,
> 
> What would cause Nifi (0.5.1) to start up with the Data Provenance icon 
> disabled (grayed out) and no Provenance context menu on processors?  I don't 
> see any errors or warnings in the logs related to provenance, and it appears 
> to be writing files to the provenance_repository on disk.  This is happening 
> on a brand new instance.  Any help would be appreciated.
> 
> Thanks,
> 
> Joe Gresock
> AMPERE TTO
> Software Engineer Stf
> Lockheed Martin MST
> joseph.gres...@lmco.com
> 

Re: Provenance disabled

[Simon Elliston Ball]

This will happen if you are authenticated as a user who does have the 
provenance role. That’s certainly the most common cause. There are other 
configurations which can do this, but they are unlikely on a clean install. Can 
you share anything about the authentication / provenance config?

Simon

> On 23 May 2016, at 14:03, Gresock, Joseph  wrote:
> 
> Hi folks,
> 
> What would cause Nifi (0.5.1) to start up with the Data Provenance icon 
> disabled (grayed out) and no Provenance context menu on processors?  I don't 
> see any errors or warnings in the logs related to provenance, and it appears 
> to be writing files to the provenance_repository on disk.  This is happening 
> on a brand new instance.  Any help would be appreciated.
> 
> Thanks,
> 
> Joe Gresock
> AMPERE TTO
> Software Engineer Stf
> Lockheed Martin MST
> joseph.gres...@lmco.com
> 

Provenance disabled

[Gresock, Joseph]

Hi folks,

What would cause Nifi (0.5.1) to start up with the Data Provenance icon 
disabled (grayed out) and no Provenance context menu on processors?  I don't 
see any errors or warnings in the logs related to provenance, and it appears to 
be writing files to the provenance_repository on disk.  This is happening on a 
brand new instance.  Any help would be appreciated.

Thanks,

Joe Gresock
AMPERE TTO
Software Engineer Stf
Lockheed Martin MST
joseph.gres...@lmco.com