[jira] [Created] (NIFI-2185) Implement two-phase read for GET requests when replicated across the cluster
Mark Payne created NIFI-2185: Summary: Implement two-phase read for GET requests when replicated across the cluster Key: NIFI-2185 URL: https://issues.apache.org/jira/browse/NIFI-2185 Project: Apache NiFi Issue Type: Task Components: Core Framework Affects Versions: 1.0.0 Reporter: Mark Payne Assignee: Mark Payne Priority: Blocker Fix For: 1.0.0 With the new zero-master clustering paradigm, we can have a scenario where a GET request and a POST request occur 'at the same time' but the requests are replicated to the nodes in different orders. For instance, if I have a Process Group with ID 1234, it's possible that the group could be read via a GET and modified via a POST request simultaneously. If we have a 3 node cluster, this can result in Nodes 1 and 2 servicing the GET request before the POST request, while Node 3 services the POST request before the GET request. As a result, Nodes 1 and 2 return one view of the Process Group while Node 3 returns a different view of the Process Group. This wasn't an issue with the NCM because the NCM provided a locking mechanism that prevented any GET requests from being replicated while a POST/PUT/DELETE was also being replicated. In the new clustering model, in order to implement the same type of logic, we need a two-phase read, where the first phase simply obtains a Read Lock, and the second phase handles the logic of creating the response for the client to consume and then unlocking the Read Lock. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2159) Fingerprint not detecting flow.xml differences
[
https://issues.apache.org/jira/browse/NIFI-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364852#comment-15364852
]
Mark Payne commented on NIFI-2159:
--
I implemented the code that you specified here and verified that the unit test
passes and all looks good. Verified that nodes are able to then reconnect to
the cluster as appropriate. Created a PR to pull this in. Thanks for catching
this, [~devriesb]!
> Fingerprint not detecting flow.xml differences
> --
>
> Key: NIFI-2159
> URL: https://issues.apache.org/jira/browse/NIFI-2159
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Brandon DeVries
>Priority: Blocker
> Fix For: 1.0.0, 0.7.0
>
>
> We have very occasionally observed situations where the flow.xml across a
> cluster was able to become inconsistent, resulting in unexpected behavior.
> While investigating, the following issue was found. While we are not 100%
> that this problem is the one we were looking for, it is definitely an issue...
> Within the FingerprintFactory, the text values from tags were
> always being appended to the fingerprint as a default value of "NO_VALUE"
> regardless of whether or not they actually contained a value. This can
> manifest as two different flow.xml files being considered the same when they
> have the same number of relationship tags, even though those tags contain
> different values. Here is a suggested fix:
> Change the following in FingerprintFactory.java\[1\]:
> {code}
> final List sortedRelationshipElems = sortElements(relationshipElems,
> getConnectionRelationshipsComparator());
> for (final Element relationshipElem : sortedRelationshipElems) {
> addConnectionRelationshipFingerprint(builder, relationshipElem);
> }
> {code}
> To:
> {code}
> for (int i = 0;ibuilder.append(getValue(relationshipElems.item(i),"NO_VALUE"));
> }
> {code}
> The following unit test will show that the value of the relationship is now
> being used in the fingerprint:
> {code}
> @Test
> public void testResourceValueInFingerprint() throws IOException{
>final String fp1 =
> fingerprinter.createFingerprint(getResourceBytes("/nifi/fingerprint/flow1a.xml"),null);
>assertEquals(3,StringUtils.countMatches(fp1,"success"));
> }
> {code}
> Additionally, the tag isn't being included in the fingerprint either.
> While the following solution has not been tested, it is possible that this
> could be fixed by adding the following line to the addConnectionFingerprint()
> method \[2\]:
> {code}
> appendFirstValue(builder, DomUtils.getChildNodesByTagName(connectionElem,
> "name"));
> {code}
> \[1\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L861-864
> \[2\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L857
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2182) Processor's Active Thread Count no longer shown in UI
Mark Payne created NIFI-2182: Summary: Processor's Active Thread Count no longer shown in UI Key: NIFI-2182 URL: https://issues.apache.org/jira/browse/NIFI-2182 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Affects Versions: 1.0.0 Reporter: Mark Payne Priority: Blocker Fix For: 1.0.0 The UI no longer shows the number of active threads that a processor is using. To replicate, create a GenerateFlowFile processor and configure it to create 1 MB flowfiles. Then route to a CompressContent processor. Configure CompressContent to use GZIP compression with a compression level of 9. This will ensure that Compress Content is not able to keep up with GenerateFlowFile, so it should always have active threads. Start both processors and notice that neither shows any active threads. I have not tried testing this in standalone mode, only in clustered mode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2159) Fingerprint not detecting flow.xml differences
[
https://issues.apache.org/jira/browse/NIFI-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362878#comment-15362878
]
Mark Payne commented on NIFI-2159:
--
[~devriesb] - agreed, and i am ok with marking this as a blocker. I have
updated the priority as such.
> Fingerprint not detecting flow.xml differences
> --
>
> Key: NIFI-2159
> URL: https://issues.apache.org/jira/browse/NIFI-2159
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Brandon DeVries
>Priority: Blocker
> Fix For: 1.0.0, 0.7.0
>
>
> We have very occasionally observed situations where the flow.xml across a
> cluster was able to become inconsistent, resulting in unexpected behavior.
> While investigating, the following issue was found. While we are not 100%
> that this problem is the one we were looking for, it is definitely an issue...
> Within the FingerprintFactory, the text values from tags were
> always being appended to the fingerprint as a default value of "NO_VALUE"
> regardless of whether or not they actually contained a value. This can
> manifest as two different flow.xml files being considered the same when they
> have the same number of relationship tags, even though those tags contain
> different values. Here is a suggested fix:
> Change the following in FingerprintFactory.java\[1\]:
> {code}
> final List sortedRelationshipElems = sortElements(relationshipElems,
> getConnectionRelationshipsComparator());
> for (final Element relationshipElem : sortedRelationshipElems) {
> addConnectionRelationshipFingerprint(builder, relationshipElem);
> }
> {code}
> To:
> {code}
> for (int i = 0;ibuilder.append(getValue(relationshipElems.item(i),"NO_VALUE"));
> }
> {code}
> The following unit test will show that the value of the relationship is now
> being used in the fingerprint:
> {code}
> @Test
> public void testResourceValueInFingerprint() throws IOException{
>final String fp1 =
> fingerprinter.createFingerprint(getResourceBytes("/nifi/fingerprint/flow1a.xml"),null);
>assertEquals(3,StringUtils.countMatches(fp1,"success"));
> }
> {code}
> Additionally, the tag isn't being included in the fingerprint either.
> While the following solution has not been tested, it is possible that this
> could be fixed by adding the following line to the addConnectionFingerprint()
> method \[2\]:
> {code}
> appendFirstValue(builder, DomUtils.getChildNodesByTagName(connectionElem,
> "name"));
> {code}
> \[1\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L861-864
> \[2\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L857
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2159) Fingerprint not detecting flow.xml differences
[
https://issues.apache.org/jira/browse/NIFI-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-2159:
-
Priority: Blocker (was: Critical)
> Fingerprint not detecting flow.xml differences
> --
>
> Key: NIFI-2159
> URL: https://issues.apache.org/jira/browse/NIFI-2159
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Brandon DeVries
>Priority: Blocker
> Fix For: 1.0.0, 0.7.0
>
>
> We have very occasionally observed situations where the flow.xml across a
> cluster was able to become inconsistent, resulting in unexpected behavior.
> While investigating, the following issue was found. While we are not 100%
> that this problem is the one we were looking for, it is definitely an issue...
> Within the FingerprintFactory, the text values from tags were
> always being appended to the fingerprint as a default value of "NO_VALUE"
> regardless of whether or not they actually contained a value. This can
> manifest as two different flow.xml files being considered the same when they
> have the same number of relationship tags, even though those tags contain
> different values. Here is a suggested fix:
> Change the following in FingerprintFactory.java\[1\]:
> {code}
> final List sortedRelationshipElems = sortElements(relationshipElems,
> getConnectionRelationshipsComparator());
> for (final Element relationshipElem : sortedRelationshipElems) {
> addConnectionRelationshipFingerprint(builder, relationshipElem);
> }
> {code}
> To:
> {code}
> for (int i = 0;ibuilder.append(getValue(relationshipElems.item(i),"NO_VALUE"));
> }
> {code}
> The following unit test will show that the value of the relationship is now
> being used in the fingerprint:
> {code}
> @Test
> public void testResourceValueInFingerprint() throws IOException{
>final String fp1 =
> fingerprinter.createFingerprint(getResourceBytes("/nifi/fingerprint/flow1a.xml"),null);
>assertEquals(3,StringUtils.countMatches(fp1,"success"));
> }
> {code}
> Additionally, the tag isn't being included in the fingerprint either.
> While the following solution has not been tested, it is possible that this
> could be fixed by adding the following line to the addConnectionFingerprint()
> method \[2\]:
> {code}
> appendFirstValue(builder, DomUtils.getChildNodesByTagName(connectionElem,
> "name"));
> {code}
> \[1\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L861-864
> \[2\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L857
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2159) Fingerprint not detecting flow.xml differences
[
https://issues.apache.org/jira/browse/NIFI-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362738#comment-15362738
]
Mark Payne commented on NIFI-2159:
--
[~devriesb] I agree this is a pretty important issue. It looks like it has been
around for quite a long time, though, and you created this ticket as 'Critical'
rather than 'Blocker'. Given that we are in Release Candidate state for 0.7.0,
do you think this should be moved off of 0.7.0? Or do you think this needs to
be a blocker?
> Fingerprint not detecting flow.xml differences
> --
>
> Key: NIFI-2159
> URL: https://issues.apache.org/jira/browse/NIFI-2159
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Brandon DeVries
>Priority: Critical
> Fix For: 1.0.0, 0.7.0
>
>
> We have very occasionally observed situations where the flow.xml across a
> cluster was able to become inconsistent, resulting in unexpected behavior.
> While investigating, the following issue was found. While we are not 100%
> that this problem is the one we were looking for, it is definitely an issue...
> Within the FingerprintFactory, the text values from tags were
> always being appended to the fingerprint as a default value of "NO_VALUE"
> regardless of whether or not they actually contained a value. This can
> manifest as two different flow.xml files being considered the same when they
> have the same number of relationship tags, even though those tags contain
> different values. Here is a suggested fix:
> Change the following in FingerprintFactory.java\[1\]:
> {code}
> final List sortedRelationshipElems = sortElements(relationshipElems,
> getConnectionRelationshipsComparator());
> for (final Element relationshipElem : sortedRelationshipElems) {
> addConnectionRelationshipFingerprint(builder, relationshipElem);
> }
> {code}
> To:
> {code}
> for (int i = 0;ibuilder.append(getValue(relationshipElems.item(i),"NO_VALUE"));
> }
> {code}
> The following unit test will show that the value of the relationship is now
> being used in the fingerprint:
> {code}
> @Test
> public void testResourceValueInFingerprint() throws IOException{
>final String fp1 =
> fingerprinter.createFingerprint(getResourceBytes("/nifi/fingerprint/flow1a.xml"),null);
>assertEquals(3,StringUtils.countMatches(fp1,"success"));
> }
> {code}
> Additionally, the tag isn't being included in the fingerprint either.
> While the following solution has not been tested, it is possible that this
> could be fixed by adding the following line to the addConnectionFingerprint()
> method \[2\]:
> {code}
> appendFirstValue(builder, DomUtils.getChildNodesByTagName(connectionElem,
> "name"));
> {code}
> \[1\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L861-864
> \[2\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L857
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2170) Separate RevisionManager into a DistributedLockingManager and Revision Manager
Mark Payne created NIFI-2170: Summary: Separate RevisionManager into a DistributedLockingManager and Revision Manager Key: NIFI-2170 URL: https://issues.apache.org/jira/browse/NIFI-2170 Project: Apache NiFi Issue Type: Task Components: Core Framework Affects Versions: 1.0.0 Reporter: Mark Payne Assignee: Mark Payne Priority: Critical Fix For: 1.0.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1781) Update UI to reflect component level authorization
[ https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-1781. -- Resolution: Fixed > Update UI to reflect component level authorization > -- > > Key: NIFI-1781 > URL: https://issues.apache.org/jira/browse/NIFI-1781 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png, > nifi-sample-flow.png > > > - Update to UI to visual access level per component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2159) Fingerprint not detecting flow.xml differences
[
https://issues.apache.org/jira/browse/NIFI-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-2159:
-
Fix Version/s: 1.0.0
> Fingerprint not detecting flow.xml differences
> --
>
> Key: NIFI-2159
> URL: https://issues.apache.org/jira/browse/NIFI-2159
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Brandon DeVries
>Priority: Critical
> Fix For: 1.0.0, 0.7.0
>
>
> We have very occasionally observed situations where the flow.xml across a
> cluster was able to become inconsistent, resulting in unexpected behavior.
> While investigating, the following issue was found. While we are not 100%
> that this problem is the one we were looking for, it is definitely an issue...
> Within the FingerprintFactory, the text values from tags were
> always being appended to the fingerprint as a default value of "NO_VALUE"
> regardless of whether or not they actually contained a value. This can
> manifest as two different flow.xml files being considered the same when they
> have the same number of relationship tags, even though those tags contain
> different values. Here is a suggested fix:
> Change the following in FingerprintFactory.java\[1\]:
> {code}
> final List sortedRelationshipElems = sortElements(relationshipElems,
> getConnectionRelationshipsComparator());
> for (final Element relationshipElem : sortedRelationshipElems) {
> addConnectionRelationshipFingerprint(builder, relationshipElem);
> }
> {code}
> To:
> {code}
> for (int i = 0;ibuilder.append(getValue(relationshipElems.item(i),"NO_VALUE"));
> }
> {code}
> The following unit test will show that the value of the relationship is now
> being used in the fingerprint:
> {code}
> @Test
> public void testResourceValueInFingerprint() throws IOException{
>final String fp1 =
> fingerprinter.createFingerprint(getResourceBytes("/nifi/fingerprint/flow1a.xml"),null);
>assertEquals(3,StringUtils.countMatches(fp1,"success"));
> }
> {code}
> Additionally, the tag isn't being included in the fingerprint either.
> While the following solution has not been tested, it is possible that this
> could be fixed by adding the following line to the addConnectionFingerprint()
> method \[2\]:
> {code}
> appendFirstValue(builder, DomUtils.getChildNodesByTagName(connectionElem,
> "name"));
> {code}
> \[1\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L861-864
> \[2\]
> https://github.com/apache/nifi/blob/270944ec692e12c221cdff202bdab56309dfcfd7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/fingerprint/FingerprintFactory.java#L857
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2164) ConsumeJMS should allow user to configure trade-off between 'best effort' and 'guaranteed receipt' of data
Mark Payne created NIFI-2164: Summary: ConsumeJMS should allow user to configure trade-off between 'best effort' and 'guaranteed receipt' of data Key: NIFI-2164 URL: https://issues.apache.org/jira/browse/NIFI-2164 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Mark Payne Fix For: 1.0.0 Currently the ConsumeJMS Processor uses auto-acknowledge acknowledgement. This is beneficial for many use cases but could result in data loss if NiFi is shut down. We should expose a 'Delivery Guarantee' property that allows the user to choose between 'Best Effort', which will provide better performance or 'Guaranteed Receipt', which will guarantee that data has been committed to NiFi's Content & FlowFile Repositories before acknowledging the message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (NIFI-2146) When saving flow, NiFi should allow the flow to automatically be backed up
[
https://issues.apache.org/jira/browse/NIFI-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne closed NIFI-2146.
> When saving flow, NiFi should allow the flow to automatically be backed up
> --
>
> Key: NIFI-2146
> URL: https://issues.apache.org/jira/browse/NIFI-2146
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration, Core Framework, Documentation & Website
>Reporter: Mark Payne
> Fix For: 1.0.0
>
>
> Each time that a user updates the NiFi flow, the new flow is written to the
> flow.xml.gz on each node. When this occurs, we should provide the option of
> also writing the flow.xml.gz to an 'archive' folder, using a filename based
> on the current time. We should also ensure that we keep old flows for X
> (configurable) amount of time and only use a max of some configurable amount
> of disk space.
> StandardXMLFlowConfigurationDAO (in nifi-framework-core) already has a method
> for `save(FlowController controller, boolean archive)` but 'false' is always
> passed as the 'archive' field. We should instead pass in `true` if configured
> to archive.
> Additionally, we should change the archive filename so that it is based on a
> user-readable date/time, such as flow-2016-06-29-11-10-50am.xml.gz, for
> instance, for a flow that was saved at 11:10:50 AM on June 29, 2016. This
> allows an admin to easily revert a flow back to how it was at some point in
> history.
> Changes required to implement this include:
> 1. Update nifi.properties to include new properties (with the suggested
> defaults that may need to be adjusted):
> - nifi.flow.configuration.archive.enabled=true
> - nifi.flow.configuration.archive.max.time=30 days
> - nifi.flow.configuration.archive.max.storage=500 MB
> There already is a property named `nifi.flow.configuration.archive.dir` that
> should be used to specify where to save the archived flow. This is used
> currently only when the user clicks the 'archive' button in the UI.
> 2. Update StandardFlowService.saveFlowChanges(TimeUnit delayUnit, long delay)
> so that it passes a value of `true` for the 'archive' flag when calling the
> overridden saveFlowChanges method in the case that the
> `nifi.flow.configuration.archive.enabled` property is set to true (default).
> 3. Update StandardXMLFlowConfigurationDAO.createArchiveFile() to use a more
> user-friendly filename, as described above.
> 4. Update StandardXMLFlowConfigurationDAO.save() so that the `if (archive) {`
> branch first cleans up old/expired flows so that the `archive` directory is
> smaller than the configured max storage space and so that any archived flow
> that is older than the configured `nifi.flow.configuration.archive.max.time`
> property is removed.
> 5. Update Admin Guide to explain the new properties
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-2149) When saving flow, NiFi should allow the flow to automatically be backed up
[
https://issues.apache.org/jira/browse/NIFI-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-2149.
--
Resolution: Duplicate
Ticket was created when JIRA was having problems, so I kept receiving errors
when Submitting. It appears that the ticket was created 3 times.
> When saving flow, NiFi should allow the flow to automatically be backed up
> --
>
> Key: NIFI-2149
> URL: https://issues.apache.org/jira/browse/NIFI-2149
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration, Core Framework, Documentation & Website
>Reporter: Mark Payne
> Fix For: 1.0.0
>
>
> Each time that a user updates the NiFi flow, the new flow is written to the
> flow.xml.gz on each node. When this occurs, we should provide the option of
> also writing the flow.xml.gz to an 'archive' folder, using a filename based
> on the current time. We should also ensure that we keep old flows for X
> (configurable) amount of time and only use a max of some configurable amount
> of disk space.
> StandardXMLFlowConfigurationDAO (in nifi-framework-core) already has a method
> for `save(FlowController controller, boolean archive)` but 'false' is always
> passed as the 'archive' field. We should instead pass in `true` if configured
> to archive.
> Additionally, we should change the archive filename so that it is based on a
> user-readable date/time, such as flow-2016-06-29-11-10-50am.xml.gz, for
> instance, for a flow that was saved at 11:10:50 AM on June 29, 2016. This
> allows an admin to easily revert a flow back to how it was at some point in
> history.
> Changes required to implement this include:
> 1. Update nifi.properties to include new properties (with the suggested
> defaults that may need to be adjusted):
> - nifi.flow.configuration.archive.enabled=true
> - nifi.flow.configuration.archive.max.time=30 days
> - nifi.flow.configuration.archive.max.storage=500 MB
> There already is a property named `nifi.flow.configuration.archive.dir` that
> should be used to specify where to save the archived flow. This is used
> currently only when the user clicks the 'archive' button in the UI.
> 2. Update StandardFlowService.saveFlowChanges(TimeUnit delayUnit, long delay)
> so that it passes a value of `true` for the 'archive' flag when calling the
> overridden saveFlowChanges method in the case that the
> `nifi.flow.configuration.archive.enabled` property is set to true (default).
> 3. Update StandardXMLFlowConfigurationDAO.createArchiveFile() to use a more
> user-friendly filename, as described above.
> 4. Update StandardXMLFlowConfigurationDAO.save() so that the `if (archive) {`
> branch first cleans up old/expired flows so that the `archive` directory is
> smaller than the configured max storage space and so that any archived flow
> that is older than the configured `nifi.flow.configuration.archive.max.time`
> property is removed.
> 5. Update Admin Guide to explain the new properties
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2158) Bulletins ignored if SLF4J logging threshold is too high
Mark Payne created NIFI-2158: Summary: Bulletins ignored if SLF4J logging threshold is too high Key: NIFI-2158 URL: https://issues.apache.org/jira/browse/NIFI-2158 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne Fix For: 1.0.0 If a Processor logs something at the INFO level, and the Bulletin Level for the processor is set to INFO, the bulletin will not be emitted if the Processor's log level in the SLF4J configuration is set to WARN or ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1780) Investigate how we can tone down the ZooKeeper logs
[ https://issues.apache.org/jira/browse/NIFI-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15357432#comment-15357432 ] Mark Payne commented on NIFI-1780: -- This was address in 1.0.0 as part of a different ticket, where significant logging was reduced. > Investigate how we can tone down the ZooKeeper logs > --- > > Key: NIFI-1780 > URL: https://issues.apache.org/jira/browse/NIFI-1780 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne > Fix For: 1.0.0 > > > We've had a few people mention seeing logs like the following in the nifi app > log: > 2016-04-14 19:45:29,310 INFO [Framework Task Thread > Thread-2-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Opening > socket connection to server localhost/127.0.0.1:2181. Will not attempt to > authenticate using SASL (unknown error) > 2016-04-14 19:45:29,313 WARN [Framework Task Thread > Thread-2-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Session > 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_45] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) > ~[na:1.8.0_45] > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > ~[na:na] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) > ~[na:na] > The first indicates that kerberos is not being used, but it sounds much > scarier when it indicates "unknown error". The second is a legitimate concern > that it is unable to connect, but it can be pretty concerning when it is > repeated over and over in the logs. We should see if we can reduce the amount > of logs generated here. > There also some other places where Zookeeper logging is quite verbose, such > as when a node running an embedded ZK server is disconnected from the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1780) Investigate how we can tone down the ZooKeeper logs
[ https://issues.apache.org/jira/browse/NIFI-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-1780. -- Resolution: Fixed > Investigate how we can tone down the ZooKeeper logs > --- > > Key: NIFI-1780 > URL: https://issues.apache.org/jira/browse/NIFI-1780 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne > Fix For: 1.0.0 > > > We've had a few people mention seeing logs like the following in the nifi app > log: > 2016-04-14 19:45:29,310 INFO [Framework Task Thread > Thread-2-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Opening > socket connection to server localhost/127.0.0.1:2181. Will not attempt to > authenticate using SASL (unknown error) > 2016-04-14 19:45:29,313 WARN [Framework Task Thread > Thread-2-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Session > 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_45] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) > ~[na:1.8.0_45] > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > ~[na:na] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) > ~[na:na] > The first indicates that kerberos is not being used, but it sounds much > scarier when it indicates "unknown error". The second is a legitimate concern > that it is unable to connect, but it can be pretty concerning when it is > repeated over and over in the logs. We should see if we can reduce the amount > of logs generated here. > There also some other places where Zookeeper logging is quite verbose, such > as when a node running an embedded ZK server is disconnected from the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1780) Investigate how we can tone down the ZooKeeper logs
[ https://issues.apache.org/jira/browse/NIFI-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1780: - Fix Version/s: 1.0.0 > Investigate how we can tone down the ZooKeeper logs > --- > > Key: NIFI-1780 > URL: https://issues.apache.org/jira/browse/NIFI-1780 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne > Fix For: 1.0.0 > > > We've had a few people mention seeing logs like the following in the nifi app > log: > 2016-04-14 19:45:29,310 INFO [Framework Task Thread > Thread-2-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Opening > socket connection to server localhost/127.0.0.1:2181. Will not attempt to > authenticate using SASL (unknown error) > 2016-04-14 19:45:29,313 WARN [Framework Task Thread > Thread-2-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Session > 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_45] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) > ~[na:1.8.0_45] > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > ~[na:na] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) > ~[na:na] > The first indicates that kerberos is not being used, but it sounds much > scarier when it indicates "unknown error". The second is a legitimate concern > that it is unable to connect, but it can be pretty concerning when it is > repeated over and over in the logs. We should see if we can reduce the amount > of logs generated here. > There also some other places where Zookeeper logging is quite verbose, such > as when a node running an embedded ZK server is disconnected from the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1563) Remove storage of components' stats and bulletins from NCM
[ https://issues.apache.org/jira/browse/NIFI-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15357295#comment-15357295 ] Mark Payne commented on NIFI-1563: -- [~mcgilman] - I believe we can close this ticket now, as the bulletins are working now. Do you agree? > Remove storage of components' stats and bulletins from NCM > -- > > Key: NIFI-1563 > URL: https://issues.apache.org/jira/browse/NIFI-1563 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, each node in a cluster sends a period heartbeat that contains the > stats for all components in the node. This happens every 5 seconds by > default. This results in quite a lot of chatter between the NCM and nodes. It > made sense to take this approach when the clustering concept was designed > because there were no Process Groups, and we had no notion of merging > responses from nodes for a web request. > However, now we should replicate the request to all nodes and then merge the > responses on demand, rather than storing this information on the NCM. This > requires far less bandwidth because we only need to pull the stats for a > particular Process Group and only on demand instead of every 5 seconds. > Additionally, this is laying groundwork for the Zero-Master clustering that > we want to have in place for 1.0.0. > In order to remove the stats from the NCM, we will also need to remove the > bulletins and stats history. These requests will all need to be federated and > the responses merged on-demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1897) Allow Nodes to Federate Requests to other nodes
[ https://issues.apache.org/jira/browse/NIFI-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-1897. -- Resolution: Fixed > Allow Nodes to Federate Requests to other nodes > --- > > Key: NIFI-1897 > URL: https://issues.apache.org/jira/browse/NIFI-1897 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > > When a user makes a web request to any node, it should be capable of > replicating the request to all nodes in the cluster, handling the logic that > the NCM currently handles for replicating requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1136) Provide user the ability to request that provenance events that are returned are the latest events
[ https://issues.apache.org/jira/browse/NIFI-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1136: - Fix Version/s: (was: 1.0.0) > Provide user the ability to request that provenance events that are returned > are the latest events > -- > > Key: NIFI-1136 > URL: https://issues.apache.org/jira/browse/NIFI-1136 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Payne > > Currently, the Provenance Events that are retrieved and returned to the user > are not ordered in any particular way. If the user searches for events for a > given Component ID, for instance, they will receive back up to 1000 events, > but those events could have happened just now or 5 days ago. > Changing to always return the most recent events that match the query becomes > quite expensive and is not worth the trade-off in many cases. > We should allow the user to choose whether or not they need the most recent > events and only then concern ourselves with the most recent events. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1006) Change format for configuring repositories in nifi.properties file
[ https://issues.apache.org/jira/browse/NIFI-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1006: - Fix Version/s: (was: 1.0.0) > Change format for configuring repositories in nifi.properties file > -- > > Key: NIFI-1006 > URL: https://issues.apache.org/jira/browse/NIFI-1006 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Core Framework >Reporter: Mark Payne > > Currently, the Content Repository, FlowFile Repository, and Provenance > Repository are all configured within the nifi.properties file. This includes > the repository implementation to use and all properties for those > repositories. This becomes quite confusing to configure, and it makes it very > difficult to provide examples of each of these repositories. > I would like to see this changed to a format like is used to configure the > Authority Providers. This way, in the nifi.properties file, we would > configure just two things: the .xml file that includes the repository > configurations (with IDs) and the ID of the repository to use. > This also makes it far easier to create a repository that may wrap one or > more other repositories, by defining them all in the .xml file and then > specifying the ID of the 'wrapping' repository as the one to use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1413) NiFi nodes should not fail to start just because its templates are not in-sync with the Cluster.
[ https://issues.apache.org/jira/browse/NIFI-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15357205#comment-15357205 ] Mark Payne commented on NIFI-1413: -- [~joewitt] with this PR we implement the following logic: - If a node joins a cluster and has templates not available to the cluster, those templates are preserved but not shown in the list of templates when dragging onto the canvas. If the node is restarted as a standalone node, the template will be made available again. - If a node joins a cluster and is missing 1 or more templates, it will inherit the template from the cluster. - If a node joins a cluster and has a conflicting view of what a template should look like (this should never happen, as templates are created and are immutable, but just in case...) then the node assumes the template definition provided by the cluster. Does this all seem reasonable to you? > NiFi nodes should not fail to start just because its templates are not > in-sync with the Cluster. > > > Key: NIFI-1413 > URL: https://issues.apache.org/jira/browse/NIFI-1413 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > If a node tries to join a cluster, the node will fail to join and a > notification will indicate that the flow on the node is different than the > cluster's flow, just because the templates are different. > Rather, we should rename the old template to ".standalone" or > something of that nature and then write the template to a new file if it > conflicts. If the template is simply missing, it should just be downloaded. > We should make sure that NCM lists only those templates that are available to > it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1966) Address Issue of Nodes Disagreeing About Flow Contents on Startup
[ https://issues.apache.org/jira/browse/NIFI-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne reassigned NIFI-1966: Assignee: Mark Payne > Address Issue of Nodes Disagreeing About Flow Contents on Startup > - > > Key: NIFI-1966 > URL: https://issues.apache.org/jira/browse/NIFI-1966 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > During startup a coordinator is selected at random to act as the manager of > data flow and distributing it to other nodes. This works fine unless that > selected coordinator happens to have a uninheritable data flow. When this > occurs, the other nodes either are > - Able to connect to Coordinator and then unable to join due to an > uninheritable flow exception due to being presented with a bad data flow. In > this case, the node shut down. > - Unable to connect to the Coordinator due to lack of quorem and never > present with the bad data flow. In this case, the node ran its local flow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1966) Address Issue of Nodes Disagreeing About Flow Contents on Startup
[ https://issues.apache.org/jira/browse/NIFI-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1966: - Summary: Address Issue of Nodes Disagreeing About Flow Contents on Startup (was: Cluster - Coordinator Determination Issues) > Address Issue of Nodes Disagreeing About Flow Contents on Startup > - > > Key: NIFI-1966 > URL: https://issues.apache.org/jira/browse/NIFI-1966 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Priority: Blocker > Fix For: 1.0.0 > > > During startup a coordinator is selected at random to act as the manager of > data flow and distributing it to other nodes. This works fine unless that > selected coordinator happens to have a uninheritable data flow. When this > occurs, the other nodes either are > - Able to connect to Coordinator and then unable to join due to an > uninheritable flow exception due to being presented with a bad data flow. In > this case, the node shut down. > - Unable to connect to the Coordinator due to lack of quorem and never > present with the bad data flow. In this case, the node ran its local flow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2150) Templates contain a great deal of unused information
Mark Payne created NIFI-2150: Summary: Templates contain a great deal of unused information Key: NIFI-2150 URL: https://issues.apache.org/jira/browse/NIFI-2150 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne Fix For: 1.0.0 Templates currently contain a great deal of unused information, such as the description of each property, the allowable values, the default values, etc. These values are ignored when the template is instantiated, because the Processor class itself drives these values. As a result, these should be cleansed from templates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2149) When saving flow, NiFi should allow the flow to automatically be backed up
Mark Payne created NIFI-2149:
Summary: When saving flow, NiFi should allow the flow to
automatically be backed up
Key: NIFI-2149
URL: https://issues.apache.org/jira/browse/NIFI-2149
Project: Apache NiFi
Issue Type: Improvement
Components: Configuration, Core Framework, Documentation & Website
Reporter: Mark Payne
Fix For: 1.0.0
Each time that a user updates the NiFi flow, the new flow is written to the
flow.xml.gz on each node. When this occurs, we should provide the option of
also writing the flow.xml.gz to an 'archive' folder, using a filename based on
the current time. We should also ensure that we keep old flows for X
(configurable) amount of time and only use a max of some configurable amount of
disk space.
StandardXMLFlowConfigurationDAO (in nifi-framework-core) already has a method
for `save(FlowController controller, boolean archive)` but 'false' is always
passed as the 'archive' field. We should instead pass in `true` if configured
to archive.
Additionally, we should change the archive filename so that it is based on a
user-readable date/time, such as flow-2016-06-29-11-10-50am.xml.gz, for
instance, for a flow that was saved at 11:10:50 AM on June 29, 2016. This
allows an admin to easily revert a flow back to how it was at some point in
history.
Changes required to implement this include:
1. Update nifi.properties to include new properties (with the suggested
defaults that may need to be adjusted):
- nifi.flow.configuration.archive.enabled=true
- nifi.flow.configuration.archive.max.time=30 days
- nifi.flow.configuration.archive.max.storage=500 MB
There already is a property named `nifi.flow.configuration.archive.dir` that
should be used to specify where to save the archived flow. This is used
currently only when the user clicks the 'archive' button in the UI.
2. Update StandardFlowService.saveFlowChanges(TimeUnit delayUnit, long delay)
so that it passes a value of `true` for the 'archive' flag when calling the
overridden saveFlowChanges method in the case that the
`nifi.flow.configuration.archive.enabled` property is set to true (default).
3. Update StandardXMLFlowConfigurationDAO.createArchiveFile() to use a more
user-friendly filename, as described above.
4. Update StandardXMLFlowConfigurationDAO.save() so that the `if (archive) {`
branch first cleans up old/expired flows so that the `archive` directory is
smaller than the configured max storage space and so that any archived flow
that is older than the configured `nifi.flow.configuration.archive.max.time`
property is removed.
5. Update Admin Guide to explain the new properties
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2145) When saving flow, NiFi should allow the flow to automatically be backed up
Mark Payne created NIFI-2145:
Summary: When saving flow, NiFi should allow the flow to
automatically be backed up
Key: NIFI-2145
URL: https://issues.apache.org/jira/browse/NIFI-2145
Project: Apache NiFi
Issue Type: Improvement
Components: Configuration, Core Framework
Reporter: Mark Payne
Fix For: 1.0.0
Each time that a user updates the NiFi flow, the new flow is written to the
flow.xml.gz on each node. When this occurs, we should provide the option of
also writing the flow.xml.gz to an 'archive' folder, using a filename based on
the current time. We should also ensure that we keep old flows for X
(configurable) amount of time and only use a max of some configurable amount of
disk space.
StandardXMLFlowConfigurationDAO (in nifi-framework-core) already has a method
for `save(FlowController controller, boolean archive)` but 'false' is always
passed as the 'archive' field. We should instead pass in `true` if configured
to archive. Additionally, we should change the archive filename so that it is
based on a user-readable date/time, such as flow-2016-06-29-11-10-50am.xml.gz,
for instance, for a flow that was saved at 11:10:50 AM on June 29, 2016. This
allows an admin to easily revert a flow back to how it was at some point in
history.
Changes required to implement this include:
1. Update nifi.properties to include new properties (with the suggested
defaults that may need to be adjusted):
- nifi.flow.configuration.archive.enabled=true
- nifi.flow.configuration.archive.max.time=30 days
- nifi.flow.configuration.archive.max.storage=500 MB
There already is a property named `nifi.flow.configuration.archive.dir` that
should be used to specify where to save the archived flow. This is used
currently only when the user clicks the 'archive' button in the UI.
2. Update StandardFlowService.saveFlowChanges(TimeUnit delayUnit, long delay)
so that it passes a value of `true` for the 'archive' flag when calling the
overridden saveFlowChanges method in the case that the
`nifi.flow.configuration.archive.enabled` property is set to true (default).
3. Update StandardXMLFlowConfigurationDAO.createArchiveFile() to use a more
user-friendly filename, as described above.
4. Update StandardXMLFlowConfigurationDAO.save() so that the `if (archive) {`
branch first cleans up old/expired flows so that the `archive` directory is
smaller than the configured max storage space and so that any archived flow
that is older than the configured `nifi.flow.configuration.archive.max.time`
property is removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2143) Provenance "Filter By" drop-down truncates selected value
Mark Payne created NIFI-2143: Summary: Provenance "Filter By" drop-down truncates selected value Key: NIFI-2143 URL: https://issues.apache.org/jira/browse/NIFI-2143 Project: Apache NiFi Issue Type: Bug Components: Core UI Reporter: Mark Payne Priority: Minor Fix For: 1.0.0 When I open the Provenance page, on the left-hand side where I have the option to filter the results, the selected value is "by component n..." instead of "by component name". When I click the drop-down, I do see the full "by component name" but it is truncated when selected. Also, I would recommend we change "by type" to "by event type" in order to disambiguate between "by component type" and "by type". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (NIFI-2123) Authorize Individual Provenance Events
[ https://issues.apache.org/jira/browse/NIFI-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-2123: - Comment: was deleted (was: As part of the new authorization paradigm, we need to authorize each provenance event individually. We do this by finding the component that the event belongs to and then authorizing using that component's authorizations. If the component no longer exists, no user will have authorization to the event.) > Authorize Individual Provenance Events > -- > > Key: NIFI-2123 > URL: https://issues.apache.org/jira/browse/NIFI-2123 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > As part of the new authorization paradigm, we need to authorize each > provenance event individually. We do this by finding the component that the > event belongs to and then authorizing using that component's authorizations. > If the component no longer exists, no user will have authorization to the > event. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2123) Authorize Individual Provenance Events
[ https://issues.apache.org/jira/browse/NIFI-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350961#comment-15350961 ] Mark Payne commented on NIFI-2123: -- As part of the new authorization paradigm, we need to authorize each provenance event individually. We do this by finding the component that the event belongs to and then authorizing using that component's authorizations. If the component no longer exists, no user will have authorization to the event. > Authorize Individual Provenance Events > -- > > Key: NIFI-2123 > URL: https://issues.apache.org/jira/browse/NIFI-2123 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2123) Authorize Individual Provenance Events
[ https://issues.apache.org/jira/browse/NIFI-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-2123: - Description: As part of the new authorization paradigm, we need to authorize each provenance event individually. We do this by finding the component that the event belongs to and then authorizing using that component's authorizations. If the component no longer exists, no user will have authorization to the event. > Authorize Individual Provenance Events > -- > > Key: NIFI-2123 > URL: https://issues.apache.org/jira/browse/NIFI-2123 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > As part of the new authorization paradigm, we need to authorize each > provenance event individually. We do this by finding the component that the > event belongs to and then authorizing using that component's authorizations. > If the component no longer exists, no user will have authorization to the > event. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2123) Authorize Individual Provenance Events
Mark Payne created NIFI-2123: Summary: Authorize Individual Provenance Events Key: NIFI-2123 URL: https://issues.apache.org/jira/browse/NIFI-2123 Project: Apache NiFi Issue Type: Task Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne Priority: Blocker Fix For: 1.0.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2089) NIFI-1824 causes failures on Windows
[ https://issues.apache.org/jira/browse/NIFI-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-2089: - Fix Version/s: 1.0.0 > NIFI-1824 causes failures on Windows > > > Key: NIFI-2089 > URL: https://issues.apache.org/jira/browse/NIFI-2089 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Percivall >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0, 0.7.0 > > > Below is the output of a maven build of the current 0.x branch on Windows 8. > After reverting NIFI-1824 (commit 1ff55244ceca8220dbfdc3ca58eb42218536579f) > the test succeeds. > Output: > --- > T E S T S > --- > Running org.apache.nifi.cluster.HeartbeatPayloadTest > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec - > in org.apache.nifi.cluster.HeartbeatPayloadTest > Running org.apache.nifi.controller.repository.io.TestLimitedInputStream > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in > org.apache.nifi.controller.repository.io.TestLimitedInputStream > Running org.apache.nifi.controller.repository.TestFileSystemRepository > Tests run: 28, Failures: 1, Errors: 2, Skipped: 0, Time elapsed: 33.81 sec > <<< FAILURE! - in > org.apache.nifi.controller.repository.TestFileSystemRepository > testRemoveDeletesFileIfNoClaimants(org.apache.nifi.controller.repository.TestFileSystemRepository) > Time elapsed: 0.86 sec <<< FAILURE! > java.lang.AssertionError: null > at org.junit.Assert.fail(Assert.java:86) > at org.junit.Assert.assertTrue(Assert.java:41) > at org.junit.Assert.assertFalse(Assert.java:64) > at org.junit.Assert.assertFalse(Assert.java:74) > at > org.apache.nifi.controller.repository.TestFileSystemRepository.testRemoveDeletesFileIfNoClaimants(TestFileSystemRepository.java:274) > testExportToFile(org.apache.nifi.controller.repository.TestFileSystemRepository) > Time elapsed: 0.867 sec <<< ERROR! > java.nio.file.FileSystemException: > C:\Users\Joe\AppData\Local\Programs\Git\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-framework-core\target\content_repository\1\1466638368813-1: > The process cannot access the file because it is being used by another > process. > at > sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) > at sun.nio.fs.WindowsFileCopy.copy(WindowsFileCopy.java:165) > at > sun.nio.fs.WindowsFileSystemProvider.copy(WindowsFileSystemProvider.java:278) > at java.nio.file.Files.copy(Files.java:1274) > at > org.apache.nifi.controller.repository.TestFileSystemRepository.testExportToFile(TestFileSystemRepository.java:348) > testExportToOutputStream(org.apache.nifi.controller.repository.TestFileSystemRepository) > Time elapsed: 0.859 sec <<< ERROR! > java.nio.file.FileSystemException: > C:\Users\Joe\AppData\Local\Programs\Git\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-framework-core\target\content_repository\1\1466638376837-1: > The process cannot access the file because it is being used by another > process. > at > sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) > at sun.nio.fs.WindowsFileCopy.copy(WindowsFileCopy.java:165) > at > sun.nio.fs.WindowsFileSystemProvider.copy(WindowsFileSystemProvider.java:278) > at java.nio.file.Files.copy(Files.java:1274) > at > org.apache.nifi.controller.repository.TestFileSystemRepository.testExportToOutputStream(TestFileSystemRepository.java:335) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2089) NIFI-1824 causes failures on Windows
[ https://issues.apache.org/jira/browse/NIFI-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345459#comment-15345459 ] Mark Payne commented on NIFI-2089: -- Thanks, [~JPercivall] - I will look into this. > NIFI-1824 causes failures on Windows > > > Key: NIFI-2089 > URL: https://issues.apache.org/jira/browse/NIFI-2089 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Percivall >Assignee: Mark Payne >Priority: Blocker > Fix For: 0.7.0 > > > Below is the output of a maven build of the current 0.x branch on Windows 8. > After reverting NIFI-1824 (commit 1ff55244ceca8220dbfdc3ca58eb42218536579f) > the test succeeds. > Output: > --- > T E S T S > --- > Running org.apache.nifi.cluster.HeartbeatPayloadTest > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec - > in org.apache.nifi.cluster.HeartbeatPayloadTest > Running org.apache.nifi.controller.repository.io.TestLimitedInputStream > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in > org.apache.nifi.controller.repository.io.TestLimitedInputStream > Running org.apache.nifi.controller.repository.TestFileSystemRepository > Tests run: 28, Failures: 1, Errors: 2, Skipped: 0, Time elapsed: 33.81 sec > <<< FAILURE! - in > org.apache.nifi.controller.repository.TestFileSystemRepository > testRemoveDeletesFileIfNoClaimants(org.apache.nifi.controller.repository.TestFileSystemRepository) > Time elapsed: 0.86 sec <<< FAILURE! > java.lang.AssertionError: null > at org.junit.Assert.fail(Assert.java:86) > at org.junit.Assert.assertTrue(Assert.java:41) > at org.junit.Assert.assertFalse(Assert.java:64) > at org.junit.Assert.assertFalse(Assert.java:74) > at > org.apache.nifi.controller.repository.TestFileSystemRepository.testRemoveDeletesFileIfNoClaimants(TestFileSystemRepository.java:274) > testExportToFile(org.apache.nifi.controller.repository.TestFileSystemRepository) > Time elapsed: 0.867 sec <<< ERROR! > java.nio.file.FileSystemException: > C:\Users\Joe\AppData\Local\Programs\Git\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-framework-core\target\content_repository\1\1466638368813-1: > The process cannot access the file because it is being used by another > process. > at > sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) > at sun.nio.fs.WindowsFileCopy.copy(WindowsFileCopy.java:165) > at > sun.nio.fs.WindowsFileSystemProvider.copy(WindowsFileSystemProvider.java:278) > at java.nio.file.Files.copy(Files.java:1274) > at > org.apache.nifi.controller.repository.TestFileSystemRepository.testExportToFile(TestFileSystemRepository.java:348) > testExportToOutputStream(org.apache.nifi.controller.repository.TestFileSystemRepository) > Time elapsed: 0.859 sec <<< ERROR! > java.nio.file.FileSystemException: > C:\Users\Joe\AppData\Local\Programs\Git\nifi\nifi-nar-bundles\nifi-framework-bundle\nifi-framework\nifi-framework-core\target\content_repository\1\1466638376837-1: > The process cannot access the file because it is being used by another > process. > at > sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) > at sun.nio.fs.WindowsFileCopy.copy(WindowsFileCopy.java:165) > at > sun.nio.fs.WindowsFileSystemProvider.copy(WindowsFileSystemProvider.java:278) > at java.nio.file.Files.copy(Files.java:1274) > at > org.apache.nifi.controller.repository.TestFileSystemRepository.testExportToOutputStream(TestFileSystemRepository.java:335) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2087) Provenance Repository still removing events from Lucene Index
Mark Payne created NIFI-2087: Summary: Provenance Repository still removing events from Lucene Index Key: NIFI-2087 URL: https://issues.apache.org/jira/browse/NIFI-2087 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Mark Payne Fix For: 1.0.0 NIFI-756 was implemented in NiFi 0.3.0 to change the way in which we remove events from the Lucene Indexes. It appears, though, that we still are using the DeleteIndexAction to remove events from the Lucene Index, in addition to the code that was added. As a result, we are removing events from the lucene index, in addition to the newer mechanism of just deleting the index files when they are no longer relevant. We should be able to simply stop using the DeleteIndexAction in PersistentProvenanceRepository and then test by starting NiFi with a small index shard size (say 25 MB), create a lot of data, such that some of the data ages off. We should then be able to verify that queries no longer show the data and be able to look in the provenance repository directory to verify that the old index-* directories are getting deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2082) When a node is disconnected from cluster, it should be made extremely obvious in the UI
[ https://issues.apache.org/jira/browse/NIFI-2082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-2082: - Issue Type: Sub-task (was: Improvement) Parent: NIFI-2010 > When a node is disconnected from cluster, it should be made extremely obvious > in the UI > --- > > Key: NIFI-2082 > URL: https://issues.apache.org/jira/browse/NIFI-2082 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Mark Payne >Priority: Critical > Fix For: 1.0.0 > > > Because we now have a zero-master clustering paradigm, it's possible to open > the UI on a node that is disconnected from the cluster. In this case, changes > made to the UI will be accepted, but they will not be replicated to the > cluster. As a result, the flow will be out-of-sync with the cluster, and in > order to get the node back to the cluster, the changes would have to be > undone and the work potentially lost (user could create a template to save > the changes but may not know what all they changed). > The UI should make it very obvious that the node is disconnected and that any > changes that made will not be replicated to the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2082) When a node is disconnected from cluster, it should be made extremely obvious in the UI
Mark Payne created NIFI-2082: Summary: When a node is disconnected from cluster, it should be made extremely obvious in the UI Key: NIFI-2082 URL: https://issues.apache.org/jira/browse/NIFI-2082 Project: Apache NiFi Issue Type: Improvement Components: Core UI Affects Versions: 1.0.0 Reporter: Mark Payne Priority: Critical Fix For: 1.0.0 Because we now have a zero-master clustering paradigm, it's possible to open the UI on a node that is disconnected from the cluster. In this case, changes made to the UI will be accepted, but they will not be replicated to the cluster. As a result, the flow will be out-of-sync with the cluster, and in order to get the node back to the cluster, the changes would have to be undone and the work potentially lost (user could create a template to save the changes but may not know what all they changed). The UI should make it very obvious that the node is disconnected and that any changes that made will not be replicated to the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2080) When selecting multiple components, text is clipped in "Operate" panel
Mark Payne created NIFI-2080: Summary: When selecting multiple components, text is clipped in "Operate" panel Key: NIFI-2080 URL: https://issues.apache.org/jira/browse/NIFI-2080 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Reporter: Mark Payne Priority: Minor Fix For: 1.0.0 When I select multiple components on my canvas, the text in the "Operate" panel shows "Multiple components select..." instead of "Multiple component selected". This is on OSX when the "Show scroll bars" System Preference is set to "Always." If I change that preference to some other value, it properly shows "Multiple components selected" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2065) Limit how much we query Lucene when performing provenance query
Mark Payne created NIFI-2065: Summary: Limit how much we query Lucene when performing provenance query Key: NIFI-2065 URL: https://issues.apache.org/jira/browse/NIFI-2065 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Mark Payne Fix For: 1.0.0 The design of the Persistent Provenance Repository was such that if we read X number of events (1,000 by default) that we stop searching provenance and return these events. We search all Lucene Indices, though, and then stop reading events from the Provenance logs after we reach 1,000. We need to update the code to stop searching Lucene after we obtain 1,000 events. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-2035) Connection creation needs to verify that source and destination both exist during 1st phase of 2 phase commit
[ https://issues.apache.org/jira/browse/NIFI-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne reassigned NIFI-2035: Assignee: Mark Payne > Connection creation needs to verify that source and destination both exist > during 1st phase of 2 phase commit > - > > Key: NIFI-2035 > URL: https://issues.apache.org/jira/browse/NIFI-2035 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > > When a connection is created, we don't verify in the first step that both the > source and destination exist. In the case of a Remote Process Group, where > the ports change in the background, this can become problematic, we one node > in the cluster may not have the same ports as another and the connection > creation could end up failing on that node, resulting in the node getting > kicked out of the cluster. We should also verify that the new destination > exists when updating a connection if its destination changes, as this could > happen there as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1909) Add ability to handle schemaless records to ConvertAvroToJson
[ https://issues.apache.org/jira/browse/NIFI-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1909: - Fix Version/s: 0.7.0 1.0.0 > Add ability to handle schemaless records to ConvertAvroToJson > - > > Key: NIFI-1909 > URL: https://issues.apache.org/jira/browse/NIFI-1909 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Ryan Persaud > Fix For: 1.0.0, 0.7.0 > > > In the interest of saving space, Avro messages are not always accompanied by > their schema. Currently, ConvertAvroToJson does not handle such schemaless > records. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1306) Add a lineageStartId field to FlowFile
[ https://issues.apache.org/jira/browse/NIFI-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1306: - Fix Version/s: 1.0.0 > Add a lineageStartId field to FlowFile > -- > > Key: NIFI-1306 > URL: https://issues.apache.org/jira/browse/NIFI-1306 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Minor > Fix For: 1.0.0 > > > Currently, the FlowFile exposes a Lineage Start Date, which is used by > prioritizers to determine which FlowFile is the oldest or newest. However, if > two FlowFiles have the same lineage start date, the prioritizer is unable to > distinguish which is newer, so they are considered equal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1877) FirstInFirstOut prioritizer does not work.
[ https://issues.apache.org/jira/browse/NIFI-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1877: - Fix Version/s: 1.0.0 > FirstInFirstOut prioritizer does not work. > -- > > Key: NIFI-1877 > URL: https://issues.apache.org/jira/browse/NIFI-1877 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.6.1 >Reporter: Matthew Clarke > Fix For: 1.0.0 > > > The FirstInFirstOut (FIFO) prioritizer has no guarantee of actually working > on queued FlowFile in a FIFO order. The reliance on lastQueuedDate or time > is not sufficient to guarantee order as FlowFile processed as batched will > all have the same exact timestamp. There is also inconsistency in how > processors update lastQueuedDate. If a processor passes a FlowFile through > without any change, Date is not updated. This in turn affects the processing > order of FlowFiles. We are finding that more and more NiFI users have > specific file processing order needs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2058) Set defaults for connections' backpressure
[ https://issues.apache.org/jira/browse/NIFI-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-2058: - Assignee: Matt Gilman > Set defaults for connections' backpressure > -- > > Key: NIFI-2058 > URL: https://issues.apache.org/jira/browse/NIFI-2058 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Matt Gilman > Fix For: 1.0.0 > > > While NiFi allows users to have great flexibility over the backpressure that > is configured at each step in the flow, often times users will forget to > configure backpressure for a specific connection, and new users may not even > know about or understand this configuration option. We should set a default > value of 1 GB and 10,000 FlowFiles instead of using a default of 'unbounded'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2058) Set defaults for connections' backpressure
Mark Payne created NIFI-2058: Summary: Set defaults for connections' backpressure Key: NIFI-2058 URL: https://issues.apache.org/jira/browse/NIFI-2058 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Mark Payne Fix For: 1.0.0 While NiFi allows users to have great flexibility over the backpressure that is configured at each step in the flow, often times users will forget to configure backpressure for a specific connection, and new users may not even know about or understand this configuration option. We should set a default value of 1 GB and 10,000 FlowFiles instead of using a default of 'unbounded'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1974) Support Custom Properties in Expression Language
[ https://issues.apache.org/jira/browse/NIFI-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338303#comment-15338303 ] Mark Payne commented on NIFI-1974: -- This has been merged to 0.x but leaving ticket open because it needs to also be merged into 1.0 > Support Custom Properties in Expression Language > > > Key: NIFI-1974 > URL: https://issues.apache.org/jira/browse/NIFI-1974 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Fix For: 1.0.0, 0.7.0 > > > Add a property in "nifi.properties" config file to allows users to specify a > list of custom properties files (containing data such as environmental > specific values, or sensitive values, etc.). The key/value pairs should be > loaded upon NIFI startup and availbale to processors for use in expression > languages. > Optimally this will lay the groundwork for a UI driven Variable Registry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2057) If variable registry cannot be loaded because one file cannot be found, none of the files are loaded
Mark Payne created NIFI-2057: Summary: If variable registry cannot be loaded because one file cannot be found, none of the files are loaded Key: NIFI-2057 URL: https://issues.apache.org/jira/browse/NIFI-2057 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Mark Payne Fix For: 1.0.0 If the value specified in "nifi.variable.registry.properties" contains multiple filenames and one of the files cannot be found, none of the properties files are loaded. This should be modified so that all of the files that can be found and loaded are, and we just log errors about those that cannot be found/accessed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1974) Support Custom Properties in Expression Language
[ https://issues.apache.org/jira/browse/NIFI-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1974: - Fix Version/s: 1.0.0 > Support Custom Properties in Expression Language > > > Key: NIFI-1974 > URL: https://issues.apache.org/jira/browse/NIFI-1974 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Fix For: 1.0.0, 0.7.0 > > > Add a property in "nifi.properties" config file to allows users to specify a > list of custom properties files (containing data such as environmental > specific values, or sensitive values, etc.). The key/value pairs should be > loaded upon NIFI startup and availbale to processors for use in expression > languages. > Optimally this will lay the groundwork for a UI driven Variable Registry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2049) Bulletins do not show the 'Category'
Mark Payne created NIFI-2049: Summary: Bulletins do not show the 'Category' Key: NIFI-2049 URL: https://issues.apache.org/jira/browse/NIFI-2049 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Reporter: Mark Payne Priority: Minor Fix For: 1.0.0 Currently, when bulletins are rendered in the UI, the bulletin's category is not shown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2047) When clicking Menu button in top-right corner, a scrollbar appears on right-hand side
[ https://issues.apache.org/jira/browse/NIFI-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-2047: - Issue Type: Sub-task (was: Bug) Parent: NIFI-2010 > When clicking Menu button in top-right corner, a scrollbar appears on > right-hand side > - > > Key: NIFI-2047 > URL: https://issues.apache.org/jira/browse/NIFI-2047 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Affects Versions: 1.0.0 > Environment: OS X >Reporter: Mark Payne >Priority: Minor > Fix For: 1.0.0 > > > With OS X, under System Preferences -> General, I have "Show scroll bars" set > to "Always." When I click the menu in the top-right corner of nifi, the > right-hand side of the page is truncated as a scroll bar appears. The scroll > bar is only present when the menu is shown. > If I change "Show scroll bars" to "When scrolling," this does not happen. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2047) When clicking Menu button in top-right corner, a scrollbar appears on right-hand side
Mark Payne created NIFI-2047: Summary: When clicking Menu button in top-right corner, a scrollbar appears on right-hand side Key: NIFI-2047 URL: https://issues.apache.org/jira/browse/NIFI-2047 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 1.0.0 Environment: OS X Reporter: Mark Payne Priority: Minor Fix For: 1.0.0 With OS X, under System Preferences -> General, I have "Show scroll bars" set to "Always." When I click the menu in the top-right corner of nifi, the right-hand side of the page is truncated as a scroll bar appears. The scroll bar is only present when the menu is shown. If I change "Show scroll bars" to "When scrolling," this does not happen. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1280) Create FilterCSVColumns Processor
[ https://issues.apache.org/jira/browse/NIFI-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334585#comment-15334585 ] Mark Payne commented on NIFI-1280: -- I updated this ticket to be "Blocked By" NIFI-2039. This is needed in order to allow the data to be read more than once for those rare cases where this is needed. > Create FilterCSVColumns Processor > - > > Key: NIFI-1280 > URL: https://issues.apache.org/jira/browse/NIFI-1280 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: Mark Payne >Assignee: Toivo Adams > > We should have a Processor that allows users to easily filter out specific > columns from CSV data. For instance, a user would configure two different > properties: "Columns of Interest" (a comma-separated list of column indexes) > and "Filtering Strategy" (Keep Only These Columns, Remove Only These Columns). > We can do this today with ReplaceText, but it is far more difficult than it > would be with this Processor, as the user has to use Regular Expressions, > etc. with ReplaceText. > Eventually a Custom UI could even be built that allows a user to upload a > Sample CSV and choose which columns from there, similar to the way that Excel > works when importing CSV by dragging and selecting the desired columns? That > would certainly be a larger undertaking and would not need to be done for an > initial implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2039) Provide ability to get InputStream from FlowFile without using Process Session callback
Mark Payne created NIFI-2039: Summary: Provide ability to get InputStream from FlowFile without using Process Session callback Key: NIFI-2039 URL: https://issues.apache.org/jira/browse/NIFI-2039 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Mark Payne Fix For: 1.0.0 Provide a mechanism to obtain an InputStream for a FlowFile directly from the ProcessSession without having to provide an InputStreamCallback. Currently, we often have processors that create an AtomicReference so that the value can be set within a callback and then the value must be obtained using a get() once the callback returns. This is very un-intuitive. Additionally, we don't have the ability to return an InputStream to other code that may required to run in a callback. For instance, when using Calcite, Calcite wants a callback, essentially, that provides an Enumerator of data. We can't currently support this well because we can't create an InputStream and return it. We need to add the following method to ProcessSession: InputStream read(FlowFile flowFile) throws ProcessException; The InputStream that is returned must be tracked by ProcessSession such that when the session is committed or rolled back, the stream is closed if it hasn't been already. All of the logic that currently takes place in ProcessSession.read() after the callback returns must be done on the closing of the new InputStream that is returned. The MockProcessSession should throw an Exception if commit() is called without the InputStream being closed, ideally providing the stack trace of where the stream was obtained and where commit() was called. This is to ensure that the developer is being a good consumer, but the StandardProcessSession doesn't really need to do this, as we can simply close the InputStream so that it can no longer be consumed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2035) Connection creation needs to verify that source and destination both exist during 1st phase of 2 phase commit
Mark Payne created NIFI-2035: Summary: Connection creation needs to verify that source and destination both exist during 1st phase of 2 phase commit Key: NIFI-2035 URL: https://issues.apache.org/jira/browse/NIFI-2035 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Mark Payne When a connection is created, we don't verify in the first step that both the source and destination exist. In the case of a Remote Process Group, where the ports change in the background, this can become problematic, we one node in the cluster may not have the same ports as another and the connection creation could end up failing on that node, resulting in the node getting kicked out of the cluster. We should also verify that the new destination exists when updating a connection if its destination changes, as this could happen there as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2033) Allow Controller Services to be scoped at the Controller Level
Mark Payne created NIFI-2033: Summary: Allow Controller Services to be scoped at the Controller Level Key: NIFI-2033 URL: https://issues.apache.org/jira/browse/NIFI-2033 Project: Apache NiFi Issue Type: Sub-task Components: Core Framework Affects Versions: 1.0.0 Reporter: Mark Payne Priority: Blocker Fix For: 1.0.0 Currently, Controller Services can be scoped to any Process Group. However, Reporting Tasks should not be able to reference the services that are assigned to a process group. We need to allow Controller Services to be scoped at the Controller Level and allow them to be referenced only by Reporting Tasks and other Controller Services that are scoped at the Controller level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2032) Processors could be started before the Controller Services that they depend on
Mark Payne created NIFI-2032: Summary: Processors could be started before the Controller Services that they depend on Key: NIFI-2032 URL: https://issues.apache.org/jira/browse/NIFI-2032 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Mark Payne in the StandardControllerServiceProvider, we enable a Collection of Controller Services but do so in the background with a limited number of threads. We need to ensure that all Controller Services have at least become ENABLING before returning from this method. Otherwise, Processors that depend on them could attempt to start. If this happens, the Processor will be invalid because it references a Disabled Controller Service. As a result, the Processor will not start. In a clustered environment, we will end up with inconsistent run states across the nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1644) If unable to write to Content Repository, Process Session should automatically roll itself back
[
https://issues.apache.org/jira/browse/NIFI-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331821#comment-15331821
]
Mark Payne commented on NIFI-1644:
--
[~JPercivall] I removed 0.7.0 as a fix version. I'll have to think a bit more
about how to handle this cleanly, as rolling back the session could still
result in some weird error messages if the Exception is then caught and the
processor attempts to transfer it, remove it, or modify it some way - in that
case, they would get a "FlowFile is not known in this session" type of
message though perhaps that is preferable to sending the FlowFile along
> If unable to write to Content Repository, Process Session should
> automatically roll itself back
> ---
>
> Key: NIFI-1644
> URL: https://issues.apache.org/jira/browse/NIFI-1644
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
> Fix For: 1.0.0
>
>
> If we write to the Content Repository and get an IOException (for example,
> out of disk space), the ProcessSession catches this and then removes the
> temporary content claim and then throws a FlowFileAccessException. However,
> the entire session really should be rolled back, because the FlowFIle no
> longer has a valid Content Claim. An example of this is in the
> StandardProcessSession.write method:
> {code}
> } catch (final FlowFileAccessException ffae) {
> resetWriteClaims(); // need to reset write claim before we can
> remove the claim
> destroyContent(newClaim);
> throw ffae;
> }
> {code}
> Processors that then catch Throwable or the general Exception and route to
> failure pass along an invalid FlowFile. We end up seeing the following in the
> logs:
> {code}
> 2016-03-17 04:21:04,742 WARN [Timer-Driven Process Thread-17]
> o.a.n.c.r.WriteAheadFlowFileRepository Repository Record
> StandardRepositoryRecord[UpdateType=CONTENTMISSING,Record=StandardFlowFileRecord[uuid=01efcc28-e28f-45ab-9373-cba8933a010c,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1458188464723-45209, container=pub1,
> section=153], offset=0,
> length=1017],offset=0,name=26304456091229115,size=1017]] is marked to be
> aborted; it will be persisted in the FlowFileRepository as a DELETE record
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1644) If unable to write to Content Repository, Process Session should automatically roll itself back
[
https://issues.apache.org/jira/browse/NIFI-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-1644:
-
Fix Version/s: (was: 0.7.0)
> If unable to write to Content Repository, Process Session should
> automatically roll itself back
> ---
>
> Key: NIFI-1644
> URL: https://issues.apache.org/jira/browse/NIFI-1644
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
> Fix For: 1.0.0
>
>
> If we write to the Content Repository and get an IOException (for example,
> out of disk space), the ProcessSession catches this and then removes the
> temporary content claim and then throws a FlowFileAccessException. However,
> the entire session really should be rolled back, because the FlowFIle no
> longer has a valid Content Claim. An example of this is in the
> StandardProcessSession.write method:
> {code}
> } catch (final FlowFileAccessException ffae) {
> resetWriteClaims(); // need to reset write claim before we can
> remove the claim
> destroyContent(newClaim);
> throw ffae;
> }
> {code}
> Processors that then catch Throwable or the general Exception and route to
> failure pass along an invalid FlowFile. We end up seeing the following in the
> logs:
> {code}
> 2016-03-17 04:21:04,742 WARN [Timer-Driven Process Thread-17]
> o.a.n.c.r.WriteAheadFlowFileRepository Repository Record
> StandardRepositoryRecord[UpdateType=CONTENTMISSING,Record=StandardFlowFileRecord[uuid=01efcc28-e28f-45ab-9373-cba8933a010c,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1458188464723-45209, container=pub1,
> section=153], offset=0,
> length=1017],offset=0,name=26304456091229115,size=1017]] is marked to be
> aborted; it will be persisted in the FlowFileRepository as a DELETE record
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2030) Refactor how HTTP-based Site-to-Site ports are communicated
Mark Payne created NIFI-2030: Summary: Refactor how HTTP-based Site-to-Site ports are communicated Key: NIFI-2030 URL: https://issues.apache.org/jira/browse/NIFI-2030 Project: Apache NiFi Issue Type: Task Components: Core Framework Affects Versions: 1.0.0 Reporter: Mark Payne Fix For: 1.0.0 Currently, when clustered, the NodeIdentifier contains a field siteToSiteHttpApiPort but site-to-site should be enabled on both HTTP and HTTPS interfaces if both are made available. We should refactor NodeIdentifier so that it contains an apiPort and a secureApiPort and then when asked for the list of Peers for site-to-site, it should use the scheme of the given HTTP request to determine which port to return. I.e., if the request was made to https://myhost:443/site-to-site/peers then we should return the secure api port. If made to http://myhost:80/site-to-site/peers then we should return the non-secure api port. The fix version for this is marked as 1.0.0, but this should be backward compatible, so if it needs to slide from 1.0.0 it's not critical that it make it into this version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2021) System Diagnostics not working in Clustered mode
Mark Payne created NIFI-2021: Summary: System Diagnostics not working in Clustered mode Key: NIFI-2021 URL: https://issues.apache.org/jira/browse/NIFI-2021 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.0.0 Reporter: Mark Payne Priority: Blocker Fix For: 1.0.0 If I open the Summary panel in the UI and then click System Diagnostics, I never get a response back. Grabbing a stack trace shows that the System Diagnostics request seems to get replicated in an infinite loop (i.e., the replication causes a replication, which causes a replication, ...) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2015) Corrupted flow file leads to a wedged flow
[ https://issues.apache.org/jira/browse/NIFI-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330031#comment-15330031 ] Mark Payne commented on NIFI-2015: -- [~ozhurakousky] I do not believe that this is the same issue as NIFI-2006. NIFI-2006 has to do with a NPE on startup, whereas this is dealing with a non-existent FlowFIle UUID during normal operations > Corrupted flow file leads to a wedged flow > -- > > Key: NIFI-2015 > URL: https://issues.apache.org/jira/browse/NIFI-2015 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.6.1 >Reporter: Christopher McDermott >Assignee: Oleg Zhurakousky >Priority: Critical > Fix For: 1.0.0, 0.7.0 > > > Encountered the following error. Once encountered the processor in question > cannot make any progress. The incoming connection cannot be emptied or even > listed. Note this issue is not to address the corrupted flow file but rather > the handling of the corrupted file. > 2016-06-13 17:33:30,275 ERROR [Timer-Driven Process Thread-4] > o.a.n.p.attributes.UpdateAttribute > java.lang.IllegalStateException: Cannot create Provenance Event Record > because FlowFile UUID is not set > at > org.apache.nifi.provenance.StandardProvenanceEventRecord$Builder.assertSet(StandardProvenanceEventRecord.java:700) > ~[nifi-data-provenance-utils-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.provenance.StandardProvenanceEventRecord$Builder.build(StandardProvenanceEventRecord.java:721) > ~[nifi-data-provenance-utils-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.provenance.StandardProvenanceEventRecord$Builder.build(StandardProvenanceEventRecord.java:412) > ~[nifi-data-provenance-utils-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.StandardProcessSession.updateProvenanceRepo(StandardProcessSession.java:634) > ~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:295) > ~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:283) > ~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28) > ~[nifi-api-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1059) > ~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) > [nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > [nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123) > [nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_45] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_45] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_45] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_45] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_45] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_45] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2015) Corrupted flow file leads to a wedged flow
[ https://issues.apache.org/jira/browse/NIFI-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330028#comment-15330028 ] Mark Payne commented on NIFI-2015: -- [~ch...@mcdermott.net] which processor is being used here? Is it a custom a processor or one in the nifi codebase? The reason that I am asking is that this stack trace seems to imply that there is a FlowFile created that has no 'uuid' attribute. However, this is not supposed to be possible. Wondering if there's a validation check that is not being performed somewhere, so I would like to be able to look at what is happening here and see if there's some way to create a FlowFile with no UUID. > Corrupted flow file leads to a wedged flow > -- > > Key: NIFI-2015 > URL: https://issues.apache.org/jira/browse/NIFI-2015 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.6.1 >Reporter: Christopher McDermott >Assignee: Oleg Zhurakousky >Priority: Critical > Fix For: 1.0.0, 0.7.0 > > > Encountered the following error. Once encountered the processor in question > cannot make any progress. The incoming connection cannot be emptied or even > listed. Note this issue is not to address the corrupted flow file but rather > the handling of the corrupted file. > 2016-06-13 17:33:30,275 ERROR [Timer-Driven Process Thread-4] > o.a.n.p.attributes.UpdateAttribute > java.lang.IllegalStateException: Cannot create Provenance Event Record > because FlowFile UUID is not set > at > org.apache.nifi.provenance.StandardProvenanceEventRecord$Builder.assertSet(StandardProvenanceEventRecord.java:700) > ~[nifi-data-provenance-utils-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.provenance.StandardProvenanceEventRecord$Builder.build(StandardProvenanceEventRecord.java:721) > ~[nifi-data-provenance-utils-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.provenance.StandardProvenanceEventRecord$Builder.build(StandardProvenanceEventRecord.java:412) > ~[nifi-data-provenance-utils-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.StandardProcessSession.updateProvenanceRepo(StandardProcessSession.java:634) > ~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:295) > ~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:283) > ~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28) > ~[nifi-api-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1059) > ~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) > [nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > [nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123) > [nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_45] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_45] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_45] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_45] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_45] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_45] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2001) Introduce a 'safe' startup mode
[ https://issues.apache.org/jira/browse/NIFI-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326091#comment-15326091 ] Mark Payne commented on NIFI-2001: -- [~aperepel] - In the nifi.properties file, there is a property named "nifi.flowcontroller.autoResumeState" that does just that. By default, it is true, which means "resume the state that the flow was in when NiFi was shutdown." If that is changed to false, it will startup without any of the processors or anything running. If running in a cluster, the NCM's value for this property is propagated to the nodes in the 0.x baseline. In 1.0.0, there is no NCM, so each node will honor its own local property. I do like the idea of passing a flag to the start script, so that it will take affect only on this particular run of the application, rather than having to update the properties file for the subsequent run of the application. Taking this one step further, rather than providing a "-safe" option, I've been thinking recently about allowing users to provide a -P= option to override the value in nifi.properties. If we did that, you could do `nifi.sh start -Pnifi.flowcontroller.autoResumeState=false`. Not as concise as a `-start` option but provides a lot of power and flexibility. Specifically, I like this option for running in a docker container, etc. so that a Docker File doesn't have to change the nifi.properties file but can instead just override specific values on the command-line. Thoughts? > Introduce a 'safe' startup mode > --- > > Key: NIFI-2001 > URL: https://issues.apache.org/jira/browse/NIFI-2001 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 0.6.1 >Reporter: Andrew Grande > > *Driver*: after NiFi crashes (e.g. with OOME), one needs to go in and > manually flush queues or partially enable/disable flows to let them run and > drain data. Also, if, for some reason, NiFi was sent into a CPU or I/O death > spiral by the flow (e.g. a design mistake, or an unexpected spike in the data > without backpressure controls configured). > Today it's not possible without directly hacking the flow.xml.gz. This > becomes even more troublesome in a clustered environment. Another drastic > alternative is to wipe out all repos, but this means data loss. > *Proposal*: add a 'safe' switch/mode to NiFi, which will let it start up with > full UI and infrastructure controls, but explicitly stopping all flows and CS. > E.g. invoking *nifi.sh start -safe* on command line. > Internally, the following should happen: > * NiFi starts up as normal, with all libraries, plugins, etc. > * However, *none* of the processorts, CS & groups are started/enabled -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2000) When I click Empty queue in a cluster, only the FlowFiles on the node I'm connected to are emptied
Mark Payne created NIFI-2000: Summary: When I click Empty queue in a cluster, only the FlowFiles on the node I'm connected to are emptied Key: NIFI-2000 URL: https://issues.apache.org/jira/browse/NIFI-2000 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.0.0 Reporter: Mark Payne Assignee: Mark Payne Fix For: 1.0.0 When running NiFi in a cluster, I can queue up a bunch of data and then right-click on the connection and click Empty Queue. Whichever node I click Empty Queue on will empty the queue but not other nodes in the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1997) On restart, a node that joins cluster does not update processors' run state to match the cluster
Mark Payne created NIFI-1997: Summary: On restart, a node that joins cluster does not update processors' run state to match the cluster Key: NIFI-1997 URL: https://issues.apache.org/jira/browse/NIFI-1997 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.0.0 Reporter: Mark Payne Assignee: Mark Payne Priority: Blocker Fix For: 1.0.0 If I have a cluster, I can disconnect a node from the cluster and stop a Processor. If I then re-join the node to the cluster, it will start running the processor again, as it should. However, if I disconnect a node from the cluster and stop a Processor, and then restart that node instead of joining it back into the cluster, the problem arises. The node restarts and joins the cluster successfully but does not start the Processor that is currently stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1996) Cannot enter Process Group after Copy & Paste
Mark Payne created NIFI-1996: Summary: Cannot enter Process Group after Copy & Paste Key: NIFI-1996 URL: https://issues.apache.org/jira/browse/NIFI-1996 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.0.0 Reporter: Mark Payne Assignee: Mark Payne Priority: Blocker Fix For: 1.0.0 I copied & pasted a Process Group on the canvas. When I then try to step into the Process Group, the UI hangs for a bit and then gives me an error to check logs. Logs show that the request timed out when being replicated, and I can no longer open the UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1280) Create FilterCSVColumns Processor
[ https://issues.apache.org/jira/browse/NIFI-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325029#comment-15325029 ] Mark Payne commented on NIFI-1280: -- Also, of note, while we would like to avoid reading the dataset twice if we can, what I have found is that in cases where the data is not enormous and the node has a reasonable amount of RAM, it will end up all residing in the Operating System's disk cache most of the time, so the second pass will actually be reading from RAM, so the performance hit is not nearly as large as one would think. > Create FilterCSVColumns Processor > - > > Key: NIFI-1280 > URL: https://issues.apache.org/jira/browse/NIFI-1280 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: Mark Payne >Assignee: Toivo Adams > > We should have a Processor that allows users to easily filter out specific > columns from CSV data. For instance, a user would configure two different > properties: "Columns of Interest" (a comma-separated list of column indexes) > and "Filtering Strategy" (Keep Only These Columns, Remove Only These Columns). > We can do this today with ReplaceText, but it is far more difficult than it > would be with this Processor, as the user has to use Regular Expressions, > etc. with ReplaceText. > Eventually a Custom UI could even be built that allows a user to upload a > Sample CSV and choose which columns from there, similar to the way that Excel > works when importing CSV by dragging and selecting the desired columns? That > would certainly be a larger undertaking and would not need to be done for an > initial implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1280) Create FilterCSVColumns Processor
[ https://issues.apache.org/jira/browse/NIFI-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325013#comment-15325013 ] Mark Payne commented on NIFI-1280: -- [~Toivo Adams] correct - the data would only be read multiple times if necessary but this won't normally happen. I spent some time looking at this a few days ago, actually, looking for a way to refactor it so that we can easily enable multi-pass reading. Unfortunately, though, the only solutions that I came up with are either very hack-y or would require some changes to the NiFi API in order to allow us to obtain an InputStream and return it outside of a ProcessSession callback, which I'm not wild about. Planned to revisit again next week, but just trying to figure out a good way to make this feasible. > Create FilterCSVColumns Processor > - > > Key: NIFI-1280 > URL: https://issues.apache.org/jira/browse/NIFI-1280 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: Mark Payne >Assignee: Toivo Adams > > We should have a Processor that allows users to easily filter out specific > columns from CSV data. For instance, a user would configure two different > properties: "Columns of Interest" (a comma-separated list of column indexes) > and "Filtering Strategy" (Keep Only These Columns, Remove Only These Columns). > We can do this today with ReplaceText, but it is far more difficult than it > would be with this Processor, as the user has to use Regular Expressions, > etc. with ReplaceText. > Eventually a Custom UI could even be built that allows a user to upload a > Sample CSV and choose which columns from there, similar to the way that Excel > works when importing CSV by dragging and selecting the desired columns? That > would certainly be a larger undertaking and would not need to be done for an > initial implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1994) Templates do not handle Controller Services correctly
Mark Payne created NIFI-1994: Summary: Templates do not handle Controller Services correctly Key: NIFI-1994 URL: https://issues.apache.org/jira/browse/NIFI-1994 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.0.0 Reporter: Mark Payne Assignee: Mark Payne Priority: Blocker Fix For: 1.0.0 If we create a template that includes a Controller Service and then try to instantiate it, we get an error that indicates that the type does not exist because it references the Controller Service "simple class name" instead of the fully qualified class name. Additionally, if there is already a template with a controller service, attempting to instantiate it results in the controller services not being added to the flow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1992) Update Site-to-Site Client to support Zero-Master Clustering Paradigm
Mark Payne created NIFI-1992: Summary: Update Site-to-Site Client to support Zero-Master Clustering Paradigm Key: NIFI-1992 URL: https://issues.apache.org/jira/browse/NIFI-1992 Project: Apache NiFi Issue Type: Task Components: Core Framework, Tools and Build Affects Versions: 1.0.0 Reporter: Mark Payne Assignee: Mark Payne Priority: Blocker Fix For: 1.0.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1857) Support HTTP(S) as a transport mechanism for Site-to-Site
[ https://issues.apache.org/jira/browse/NIFI-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-1857. -- Resolution: Fixed > Support HTTP(S) as a transport mechanism for Site-to-Site > - > > Key: NIFI-1857 > URL: https://issues.apache.org/jira/browse/NIFI-1857 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Original Estimate: 480h > Remaining Estimate: 480h > > We should add support for using HTTP(S) for site-to-site to be an alternative > to the current socket based approach. > This would support the same push based or pull based approach site-to-site > offers now but it would use HTTP(S) for all interactions to include learning > about ports, learning about NCM topology, and actually exchanging data. This > mechanism should also support interaction via an HTTP proxy. > This would also require some UI work to allow the user to specify which > protocol for site-to-site to use such as 'raw' vs 'http'. We also need to > document any limitations with regard to SSL support for this mode and we'd > need to provide 'how-to' when using proxies like http_proxy or something else. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1866) Exception thrown by StandardProcessSession.exportTo implies issues reading from repository
[
https://issues.apache.org/jira/browse/NIFI-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-1866.
--
Resolution: Fixed
> Exception thrown by StandardProcessSession.exportTo implies issues reading
> from repository
> --
>
> Key: NIFI-1866
> URL: https://issues.apache.org/jira/browse/NIFI-1866
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0, 0.7.0
>
>
> When calling ProcessSession.exportTo, if unable to write to the OutputStream
> provided, we end up with a FlowFileAccessException, rather than IOException.
> This implies that we were unable to access the ContentRepository, but this is
> not accurate. We need to ensure that we throw the appropriate Exception here
> so that Processors that cause ProcessException are able to handle the
> Exception properly
> 23:23:03,338 ERROR Timer-Driven Process Thread-3
> standard.HandleHttpResponse:306 -
> org.apache.nifi.processor.exception.FlowFileAccessException: Failed to export
> StandardFlowFileRecord[uuid=366b3598-a1f7-446e-83b6-9d0404532691,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1462828983178-2, container=default,
> section=2], offset=49798,
> length=5120],offset=0,name=1095188501442536,size=5120] to
> HttpOutput@1eb25c5d{OPEN} due to org.eclipse.jetty.io.EofException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.exportTo(StandardProcessSession.java:2322)
> at
> org.apache.nifi.processors.standard.HandleHttpResponse.onTrigger(HandleHttpResponse.java:166)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1866) Exception thrown by StandardProcessSession.exportTo implies issues reading from repository
[
https://issues.apache.org/jira/browse/NIFI-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-1866:
-
Fix Version/s: 1.0.0
> Exception thrown by StandardProcessSession.exportTo implies issues reading
> from repository
> --
>
> Key: NIFI-1866
> URL: https://issues.apache.org/jira/browse/NIFI-1866
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0, 0.7.0
>
>
> When calling ProcessSession.exportTo, if unable to write to the OutputStream
> provided, we end up with a FlowFileAccessException, rather than IOException.
> This implies that we were unable to access the ContentRepository, but this is
> not accurate. We need to ensure that we throw the appropriate Exception here
> so that Processors that cause ProcessException are able to handle the
> Exception properly
> 23:23:03,338 ERROR Timer-Driven Process Thread-3
> standard.HandleHttpResponse:306 -
> org.apache.nifi.processor.exception.FlowFileAccessException: Failed to export
> StandardFlowFileRecord[uuid=366b3598-a1f7-446e-83b6-9d0404532691,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1462828983178-2, container=default,
> section=2], offset=49798,
> length=5120],offset=0,name=1095188501442536,size=5120] to
> HttpOutput@1eb25c5d{OPEN} due to org.eclipse.jetty.io.EofException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.exportTo(StandardProcessSession.java:2322)
> at
> org.apache.nifi.processors.standard.HandleHttpResponse.onTrigger(HandleHttpResponse.java:166)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1866) Exception thrown by StandardProcessSession.exportTo implies issues reading from repository
[
https://issues.apache.org/jira/browse/NIFI-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321325#comment-15321325
]
Mark Payne commented on NIFI-1866:
--
Taking a look now.
> Exception thrown by StandardProcessSession.exportTo implies issues reading
> from repository
> --
>
> Key: NIFI-1866
> URL: https://issues.apache.org/jira/browse/NIFI-1866
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 0.7.0
>
>
> When calling ProcessSession.exportTo, if unable to write to the OutputStream
> provided, we end up with a FlowFileAccessException, rather than IOException.
> This implies that we were unable to access the ContentRepository, but this is
> not accurate. We need to ensure that we throw the appropriate Exception here
> so that Processors that cause ProcessException are able to handle the
> Exception properly
> 23:23:03,338 ERROR Timer-Driven Process Thread-3
> standard.HandleHttpResponse:306 -
> org.apache.nifi.processor.exception.FlowFileAccessException: Failed to export
> StandardFlowFileRecord[uuid=366b3598-a1f7-446e-83b6-9d0404532691,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1462828983178-2, container=default,
> section=2], offset=49798,
> length=5120],offset=0,name=1095188501442536,size=5120] to
> HttpOutput@1eb25c5d{OPEN} due to org.eclipse.jetty.io.EofException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.exportTo(StandardProcessSession.java:2322)
> at
> org.apache.nifi.processors.standard.HandleHttpResponse.onTrigger(HandleHttpResponse.java:166)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1984) Deleting processor with incoming flow causes NiFi to stop responding
[ https://issues.apache.org/jira/browse/NIFI-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1984: - Component/s: Core Framework > Deleting processor with incoming flow causes NiFi to stop responding > > > Key: NIFI-1984 > URL: https://issues.apache.org/jira/browse/NIFI-1984 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Cole Mackenzie >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > Attempting to delete a processor with an incoming flow produces a warning > stating it is not possible, then causes the webui to hang, requiring restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1984) Deleting processor with incoming flow causes NiFi to stop responding
[ https://issues.apache.org/jira/browse/NIFI-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320450#comment-15320450 ] Mark Payne commented on NIFI-1984: -- I logged into JIRA just to create this ticket and then saw you beat me to it! Looking at this now. > Deleting processor with incoming flow causes NiFi to stop responding > > > Key: NIFI-1984 > URL: https://issues.apache.org/jira/browse/NIFI-1984 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Cole Mackenzie >Assignee: Mark Payne > Fix For: 1.0.0 > > > Attempting to delete a processor with an incoming flow produces a warning > stating it is not possible, then causes the webui to hang, requiring restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1984) Deleting processor with incoming flow causes NiFi to stop responding
[ https://issues.apache.org/jira/browse/NIFI-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1984: - Priority: Blocker (was: Major) > Deleting processor with incoming flow causes NiFi to stop responding > > > Key: NIFI-1984 > URL: https://issues.apache.org/jira/browse/NIFI-1984 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Cole Mackenzie >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > Attempting to delete a processor with an incoming flow produces a warning > stating it is not possible, then causes the webui to hang, requiring restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1984) Deleting processor with incoming flow causes NiFi to stop responding
[ https://issues.apache.org/jira/browse/NIFI-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1984: - Fix Version/s: 1.0.0 > Deleting processor with incoming flow causes NiFi to stop responding > > > Key: NIFI-1984 > URL: https://issues.apache.org/jira/browse/NIFI-1984 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Cole Mackenzie >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > Attempting to delete a processor with an incoming flow produces a warning > stating it is not possible, then causes the webui to hang, requiring restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1984) Deleting processor with incoming flow causes NiFi to stop responding
[ https://issues.apache.org/jira/browse/NIFI-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne reassigned NIFI-1984: Assignee: Mark Payne > Deleting processor with incoming flow causes NiFi to stop responding > > > Key: NIFI-1984 > URL: https://issues.apache.org/jira/browse/NIFI-1984 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Cole Mackenzie >Assignee: Mark Payne > > Attempting to delete a processor with an incoming flow produces a warning > stating it is not possible, then causes the webui to hang, requiring restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1974) Support Custom Properties in Expression Language
[ https://issues.apache.org/jira/browse/NIFI-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1974: - Fix Version/s: 1.0.0 > Support Custom Properties in Expression Language > > > Key: NIFI-1974 > URL: https://issues.apache.org/jira/browse/NIFI-1974 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > Fix For: 1.0.0 > > > Add a property in "nifi.properties" config file to allows users to specify a > list of custom properties files (containing data such as environmental > specific values, or sensitive values, etc.). The key/value pairs should be > loaded upon NIFI startup and availbale to processors for use in expression > languages. > Optimally this will lay the groundwork for a UI driven Variable Registry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1974) Support Custom Properties in Expression Language
[ https://issues.apache.org/jira/browse/NIFI-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319269#comment-15319269 ] Mark Payne commented on NIFI-1974: -- Reviewing now... > Support Custom Properties in Expression Language > > > Key: NIFI-1974 > URL: https://issues.apache.org/jira/browse/NIFI-1974 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis > > Add a property in "nifi.properties" config file to allows users to specify a > list of custom properties files (containing data such as environmental > specific values, or sensitive values, etc.). The key/value pairs should be > loaded upon NIFI startup and availbale to processors for use in expression > languages. > Optimally this will lay the groundwork for a UI driven Variable Registry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1373) Annotate all classes to indicate their API stability and intended audience
[ https://issues.apache.org/jira/browse/NIFI-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318965#comment-15318965 ] Mark Payne commented on NIFI-1373: -- I have created a JIRA in the Yetus JIRA tracker and linked to it from here. Thanks, [~busbey]! > Annotate all classes to indicate their API stability and intended audience > -- > > Key: NIFI-1373 > URL: https://issues.apache.org/jira/browse/NIFI-1373 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework, Core UI, Extensions >Reporter: Joseph Witt >Assignee: Joseph Witt > Fix For: 1.0.0 > > > Sean Busbey and Tony Kurc have pointed out the need to document the intended > audience and stability of the APIs. This is very necessary as we need to be > able to make changes and be fluid where intended and extremely strict to > compatibility where intended. Presently with things being unmarked it can be > ambiguous for those not deeply familiar with the api and codebase. > By leveraging the annotations from Apache Yetus we can resolve this > discrepancy across the codebase: > http://yetus.apache.org/documentation/0.1.0/audience-annotations-apidocs/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1713) Timestamps on bulletins are reported using different timezone when reported from NCM vs. Node
[ https://issues.apache.org/jira/browse/NIFI-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318647#comment-15318647 ] Mark Payne commented on NIFI-1713: -- The nodes are not reporting different timezones for events. The issue was that the timezone being used was the same on all nodes but the timezone was different on the NCM (I believe the nodes were using the correct timezone and the NCM was not). So now that we have no NCM, this won't affect us. > Timestamps on bulletins are reported using different timezone when reported > from NCM vs. Node > - > > Key: NIFI-1713 > URL: https://issues.apache.org/jira/browse/NIFI-1713 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > > When we replicate requests to gather bulletins from all nodes and the NCM, > when I have NCM and 3 nodes running on localhost, I see timestamp reported > from nodes as EST and timestamp reported from NCM as EDT, even though all are > running on the same box. This results in not only confusing timestamps but > the bulletins are not in the proper order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1052) Improve flow behavior on refactored processor names/packages
[ https://issues.apache.org/jira/browse/NIFI-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1052: - Fix Version/s: (was: 0.7.0) 1.0.0 > Improve flow behavior on refactored processor names/packages > > > Key: NIFI-1052 > URL: https://issues.apache.org/jira/browse/NIFI-1052 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: sumanth chinthagunta >Assignee: Joseph Witt > Fix For: 1.0.0 > > > My case is simple and easy to reproduce: > 1. Create a flow with custom processor (e.g. abc.MyProcesser ) > 2. Make sure flow is working. Then stop the server. > 3. Replace nar in lib with new nar after refactoring processor's package ( > e.g., xzy.MyProcesser) > 4. When you try to start nifi , it fails with error in the log saying class > not found ( obviously coz deployed flow is still looking for old class) > My expectation is system should at least start and only throw error when > user try to start a flow that that had reference to class(processor) which > is no more available in the newly deployed nar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1960) Update Administrator Guide for new Clustering Properties
[ https://issues.apache.org/jira/browse/NIFI-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne reassigned NIFI-1960: Assignee: Mark Payne > Update Administrator Guide for new Clustering Properties > > > Key: NIFI-1960 > URL: https://issues.apache.org/jira/browse/NIFI-1960 > Project: Apache NiFi > Issue Type: Task > Components: Documentation & Website >Reporter: Matt Gilman >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > Several parts of the application are fundamentally different in 1.x. > Specifically, documentation around clustering and authorization need to be > completely revisited. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1960) Update Administrator Guide for new Clustering Properties
[ https://issues.apache.org/jira/browse/NIFI-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1960: - Summary: Update Administrator Guide for new Clustering Properties (was: Update Administrator Guide for new Clusterin Properties) > Update Administrator Guide for new Clustering Properties > > > Key: NIFI-1960 > URL: https://issues.apache.org/jira/browse/NIFI-1960 > Project: Apache NiFi > Issue Type: Task > Components: Documentation & Website >Reporter: Matt Gilman >Priority: Blocker > Fix For: 1.0.0 > > > Several parts of the application are fundamentally different in 1.x. > Specifically, documentation around clustering and authorization need to be > completely revisited. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1960) Update Administrator Guide for new Clustering and Authorization Properties
[ https://issues.apache.org/jira/browse/NIFI-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-1960. -- Resolution: Fixed > Update Administrator Guide for new Clustering and Authorization Properties > -- > > Key: NIFI-1960 > URL: https://issues.apache.org/jira/browse/NIFI-1960 > Project: Apache NiFi > Issue Type: Task > Components: Documentation & Website >Reporter: Matt Gilman >Priority: Blocker > Fix For: 1.0.0 > > > Several parts of the application are fundamentally different in 1.x. > Specifically, documentation around clustering and authorization need to be > completely revisited. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1958) FormatUtils does not handle time units larger than "day"
[
https://issues.apache.org/jira/browse/NIFI-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314540#comment-15314540
]
Mark Payne commented on NIFI-1958:
--
[~alopresto] I can understand adding the "week" time duration, but I don't
think month and year really are good candidates, as you can't really convert a
month into a smaller unit like seconds because the month may have 28, 29, 30,
or 31 days in it. And because of leap years you can't really convert years into
seconds, etc. either. Is there a particular use case where you see a need for
months or years?
> FormatUtils does not handle time units larger than "day"
>
>
> Key: NIFI-1958
> URL: https://issues.apache.org/jira/browse/NIFI-1958
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.6.1
>Reporter: Andy LoPresto
> Labels: beginner, time, unit
> Fix For: 1.0.0, 0.7.0
>
>
> {{FormatUtils}}, the class used to handle unit conversions from user-provided
> configuration values, does not accept values using "week", "month", "year",
> etc. These units should be added.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-1713) Timestamps on bulletins are reported using different timezone when reported from NCM vs. Node
[ https://issues.apache.org/jira/browse/NIFI-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-1713. -- Resolution: Not A Problem This is no longer an issue, as the NCM has been removed. > Timestamps on bulletins are reported using different timezone when reported > from NCM vs. Node > - > > Key: NIFI-1713 > URL: https://issues.apache.org/jira/browse/NIFI-1713 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > > When we replicate requests to gather bulletins from all nodes and the NCM, > when I have NCM and 3 nodes running on localhost, I see timestamp reported > from nodes as EST and timestamp reported from NCM as EDT, even though all are > running on the same box. This results in not only confusing timestamps but > the bulletins are not in the proper order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1745) Refactor how NCM federates requests to nodes
[ https://issues.apache.org/jira/browse/NIFI-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-1745. -- Resolution: Fixed > Refactor how NCM federates requests to nodes > > > Key: NIFI-1745 > URL: https://issues.apache.org/jira/browse/NIFI-1745 > Project: Apache NiFi > Issue Type: Task >Affects Versions: 1.0.0 >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > > Currently, all of the logic to federate requests exists within the > WebClusterManager class. This needs to be refactored into better encapsulated > code so that the logic is more easily moved to the nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1373) Annotate all classes to indicate their API stability and intended audience
[ https://issues.apache.org/jira/browse/NIFI-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314523#comment-15314523 ] Mark Payne commented on NIFI-1373: -- [~joewitt] [~tkurc] [~busbey] Do these annotations provide any sort of notion of "Implementation Stability" vs. "Dependency Stability"? For example, our ProcessContext class is very stable in terms of any existing method will not have its signature or functionality change, so you can feel at ease knowing that the code you've written that makes use of ProcessContext will continue to work between minor versions. However, we may well add new methods to this interface between minor versions, so others should not be trying to implement this interface. Are these two notions taken into consideration when using the InterfaceStability annotation? > Annotate all classes to indicate their API stability and intended audience > -- > > Key: NIFI-1373 > URL: https://issues.apache.org/jira/browse/NIFI-1373 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework, Core UI, Extensions >Reporter: Joseph Witt >Assignee: Joseph Witt > Fix For: 1.0.0 > > > Sean Busbey and Tony Kurc have pointed out the need to document the intended > audience and stability of the APIs. This is very necessary as we need to be > able to make changes and be fluid where intended and extremely strict to > compatibility where intended. Presently with things being unmarked it can be > ambiguous for those not deeply familiar with the api and codebase. > By leveraging the annotations from Apache Yetus we can resolve this > discrepancy across the codebase: > http://yetus.apache.org/documentation/0.1.0/audience-annotations-apidocs/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1135) For Provenance Query, bring back Event Summaries instead of the Events themselves
[ https://issues.apache.org/jira/browse/NIFI-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne reassigned NIFI-1135: Assignee: Mark Payne > For Provenance Query, bring back Event Summaries instead of the Events > themselves > - > > Key: NIFI-1135 > URL: https://issues.apache.org/jira/browse/NIFI-1135 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, when we query Provenance, we pull back up to 1000 events. These > are full Provenance Events with attributes, etc. If the query takes a long > time, we will request those objects that already have matched the query many > times. This amounts to a great deal of heap being used and sending back very > large JSON objects (10+ MB is not uncommon and it could potentially be far > worse). > We should instead use a ProvenanceEventSummary object. This object should > contain just the info shown in the results table and the pointer to the > actual event in the Provenance Store. This allows us to return the queries > much faster, store less data in the heap, and provide less data back to the > end user with virtually the same experience. > The one place that this would differ in UX is when the user clicks the "info" > button to view the entire provenance event, we would have to pull the event > back from the server, rather than already having that in memory. > We should consider storing all of the fields in the results table in Lucene > to provide faster results. Otherwise, we could still get potentially better > results with the current approach if we just ensure that the first fields > that we store are those in the results table. This allows us to read just a > small portion of the event from file and deserializing just a small amount of > data before moving on to the next event. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1135) For Provenance Query, bring back Event Summaries instead of the Events themselves
[ https://issues.apache.org/jira/browse/NIFI-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1135: - Fix Version/s: (was: 1.0.0) > For Provenance Query, bring back Event Summaries instead of the Events > themselves > - > > Key: NIFI-1135 > URL: https://issues.apache.org/jira/browse/NIFI-1135 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Mark Payne > > Currently, when we query Provenance, we pull back up to 1000 events. These > are full Provenance Events with attributes, etc. If the query takes a long > time, we will request those objects that already have matched the query many > times. This amounts to a great deal of heap being used and sending back very > large JSON objects (10+ MB is not uncommon and it could potentially be far > worse). > We should instead use a ProvenanceEventSummary object. This object should > contain just the info shown in the results table and the pointer to the > actual event in the Provenance Store. This allows us to return the queries > much faster, store less data in the heap, and provide less data back to the > end user with virtually the same experience. > The one place that this would differ in UX is when the user clicks the "info" > button to view the entire provenance event, we would have to pull the event > back from the server, rather than already having that in memory. > We should consider storing all of the fields in the results table in Lucene > to provide faster results. Otherwise, we could still get potentially better > results with the current approach if we just ensure that the first fields > that we store are those in the results table. This allows us to read just a > small portion of the event from file and deserializing just a small amount of > data before moving on to the next event. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1960) Update Administrator Guide for new Clustering and Authorization Properties
[ https://issues.apache.org/jira/browse/NIFI-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1960: - Summary: Update Administrator Guide for new Clustering and Authorization Properties (was: Documentation) > Update Administrator Guide for new Clustering and Authorization Properties > -- > > Key: NIFI-1960 > URL: https://issues.apache.org/jira/browse/NIFI-1960 > Project: Apache NiFi > Issue Type: Task > Components: Documentation & Website >Reporter: Matt Gilman >Priority: Blocker > Fix For: 1.0.0 > > > Several parts of the application are fundamentally different in 1.x. > Specifically, documentation around clustering and authorization need to be > completely revisited. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1413) NiFi nodes should not fail to start just because its templates are not in-sync with the Cluster.
[ https://issues.apache.org/jira/browse/NIFI-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1413: - Priority: Blocker (was: Major) > NiFi nodes should not fail to start just because its templates are not > in-sync with the Cluster. > > > Key: NIFI-1413 > URL: https://issues.apache.org/jira/browse/NIFI-1413 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > If a node tries to join a cluster, the node will fail to join and a > notification will indicate that the flow on the node is different than the > cluster's flow, just because the templates are different. > Rather, we should rename the old template to ".standalone" or > something of that nature and then write the template to a new file if it > conflicts. If the template is simply missing, it should just be downloaded. > We should make sure that NCM lists only those templates that are available to > it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1413) NiFi nodes should not fail to start just because its templates are not in-sync with the Cluster.
[ https://issues.apache.org/jira/browse/NIFI-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314064#comment-15314064 ] Mark Payne commented on NIFI-1413: -- Setting Priority to Blocker because otherwise in 1.0.0 we could get into cases where people lose their templates when changing from NCM-based cluster to masterless clustering. > NiFi nodes should not fail to start just because its templates are not > in-sync with the Cluster. > > > Key: NIFI-1413 > URL: https://issues.apache.org/jira/browse/NIFI-1413 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Priority: Blocker > Fix For: 1.0.0 > > > If a node tries to join a cluster, the node will fail to join and a > notification will indicate that the flow on the node is different than the > cluster's flow, just because the templates are different. > Rather, we should rename the old template to ".standalone" or > something of that nature and then write the template to a new file if it > conflicts. If the template is simply missing, it should just be downloaded. > We should make sure that NCM lists only those templates that are available to > it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1413) NiFi nodes should not fail to start just because its templates are not in-sync with the Cluster.
[ https://issues.apache.org/jira/browse/NIFI-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1413: - Fix Version/s: 1.0.0 > NiFi nodes should not fail to start just because its templates are not > in-sync with the Cluster. > > > Key: NIFI-1413 > URL: https://issues.apache.org/jira/browse/NIFI-1413 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne > Fix For: 1.0.0 > > > If a node tries to join a cluster, the node will fail to join and a > notification will indicate that the flow on the node is different than the > cluster's flow, just because the templates are different. > Rather, we should rename the old template to ".standalone" or > something of that nature and then write the template to a new file if it > conflicts. If the template is simply missing, it should just be downloaded. > We should make sure that NCM lists only those templates that are available to > it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1926) Create a general purpose Aggregating processor
[ https://issues.apache.org/jira/browse/NIFI-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314039#comment-15314039 ] Mark Payne commented on NIFI-1926: -- [~ozhurakousky] - can you expand more on the differences between this processor and Merge Content? Can you explain both how they are different and when you would use one vs. the other? I worry that having both of these may end up being very confusing to users, including myself. Thanks -Mark > Create a general purpose Aggregating processor > -- > > Key: NIFI-1926 > URL: https://issues.apache.org/jira/browse/NIFI-1926 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Oleg Zhurakousky >Assignee: Oleg Zhurakousky > > Based on multiple question on mailing list it appears that we need to have a > general purpose processor to aggregate data (FlowFiles or content of > FlowFiles etc). Basically provide an implementation of this > http://www.enterpriseintegrationpatterns.com/patterns/messaging/Aggregator.html > Keep in mind that while MergeContent processor could be looked at as one of > the variation of aggregation, the pattern and the problem has much more > moving parts that need to be addressed (release, correlation, and aggregation > strategies). So the new processor should be flexible enough to handle variety > of aggregation use cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312329#comment-15312329
]
Mark Payne commented on NIFI-1265:
--
This all looks good. Was able to test that application starts up and runs
properly. Tested ListenHTTP, HandleHttpRequest, HandleHttpResponse processors
as well, including using a SSL and non-SSL configurations. All works as
expected. Merged to master.
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-1265.
--
Resolution: Fixed
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1947) Update Cluster Awareness
Mark Payne created NIFI-1947: Summary: Update Cluster Awareness Key: NIFI-1947 URL: https://issues.apache.org/jira/browse/NIFI-1947 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 1.0.0 Reporter: Mark Payne Fix For: 1.0.0 With the NCM removed for version 1.0.0, we now have a situation where a user can navigate to one node in a cluster while the node is disconnected. If the node is then reconnected, the UI does not update to indicate to the user that the node is now part of a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
