[GitHub] nifi pull request #704: NIFI-2253 flexibly cleaning zookeeper connect string
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/704#discussion_r71884405
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/cluster/ZooKeeperClientConfig.java
---
@@ -70,7 +75,10 @@ public static ZooKeeperClientConfig createConfig(final
Properties properties) {
if (connectString == null || connectString.trim().isEmpty()) {
throw new IllegalStateException("The '" +
NiFiProperties.ZOOKEEPER_CONNECT_STRING + "' property is not set in
nifi.properties");
}
-
+final String cleanedConnectString =
cleanConnectString(connectString);
+if (cleanedConnectString.isEmpty()) {
+throw new IllegalStateException("The '" +
NiFiProperties.ZOOKEEPER_CONNECT_STRING + "' property is set but contains
unrecognized data in nifi.properties");
--- End diff --
This message could probably be made a little more clear - what is
"unrecognized data"? Perhaps spell out that the expected format is a
comma-separated list of : pairs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #720: Introduce data resource for authorizing Provenance and Queu...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/720 Code looks good now. Unit tests are good. Tested functionality and all appears to work as expected. +1 merged to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #649: Nifi 1413
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/649 Nifi 1413 You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-1413 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/649.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #649 commit ef1aeb7f5e232dc54d40b2b4ad7300d5092c1596 Author: Mark Payne <marka...@hotmail.com> Date: 2016-06-30T14:48:46Z NIFI-1413: Ensure that if a node's templates don't match the clusters that we take the following actions: -Local templates remain but aren't shown in the cluster's templates. -Any templates from the cluster that don't exist on the node are added to the node. -Any conflicting template definitions are replaced by those in the cluster This closes #596 commit 83f9b7a036ce7875995bfd339f02a9c189220f59 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-14T14:06:46Z NIFI-1413: Rebased against master and updated templates endpoint merge to use template entities instead of template dto's --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #614: NIFI-2145: Auto flow.xml archive
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/614 Oh no! As always, I forgot to add the magical words to close the PR. Can you close this PR manually? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #614: NIFI-2145: Auto flow.xml archive
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/614 @ijokarumawak looked over the code, and all looks good. I updated the docs to fix a few misspellings / typos. Tested and verified behavior, including turning archive on & off, aging off based on size, and based on time. Used md5 hash to verify contents. All look great! Put together very well. Thanks for jumping in on this - I think it will be a huge benefit to the community! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #647: NIFI-1157 NIFI-1307 Resolve deprecated classes and methods
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/647 @joewitt the code changes look good. Was able to build and verify that all still compiles and that all unit tests still pass. Tested a lot of different scenarios that I could think of to ensure that application still functions as intended, and everything looks good! +1 merged to master! Thanks for going through all of this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #652: NIFI-2087: Do not update the lucene index to delete ...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/652 NIFI-2087: Do not update the lucene index to delete expired events... Instead, just wait for the entire index to expire and remove the files for that index. This is much more efficient. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2087 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/652.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #652 commit 4e62ba28797cd58d2b4ab116908c4d2b53c0916b Author: Mark Payne <marka...@hotmail.com> Date: 2016-06-30T20:06:24Z NIFI-2087: Do not update the lucene index to delete expired events. Instead, just wait for the entire index to expire and remove the files for that index. This is much more efficient. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #654: Addressing issue with enabling/disabling Controller Service...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/654 These changes all look good, and they do address the issues seen. Thanks! Will merge to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #652: NIFI-2087: Do not update the lucene index to delete expired...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/652 @JPercivall no, that should be it. We have one other place where we use the DeleteFileAction but it still needs to stay there. This should take care of it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #701: NIFI-2292: Funnel all cluster node status changes th...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/701 NIFI-2292: Funnel all cluster node status changes through the cluster coordinator⦠⦠instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2292 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/701.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #701 commit 03942d0091e6adc213dc42e3c81725ab5ca257fa Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-21T18:30:08Z NIFI-2292: Funnel all cluster node status changes through the cluster coordinator instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #717: NIFI-2292: Funnel all cluster node status changes th...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/717 NIFI-2292: Funnel all cluster node status changes through the cluster coordinator⦠You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2292-rebased Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/717.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #717 commit 15449162f55acc110bcf67d409738b68e3a37801 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-25T17:41:48Z NIFI-2292: Funnel all cluster node status changes through the cluster coordinator instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions. commit 0edda2187d0055ebd5a52401f86e1773e37d802c Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-21T18:30:08Z NIFI-2292: Funnel all cluster node status changes through the cluster coordinator instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #701: NIFI-2292: Funnel all cluster node status changes th...
Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/701 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #729: NIFI-2406: Ensure that hearbeat monitor continues to...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/729 NIFI-2406: Ensure that hearbeat monitor continues to run while instance ⦠â¦is running. This way if a node sends heartbeat to this node as elected coordinator changes, we notify the node accordingly. Handle Exceptions more gracefully in leader election code. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2406 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/729.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #729 commit e9f5a2182362f32e837d4eabd47e6d3604a37e55 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-27T15:55:02Z NIFI-2406: Ensure that hearbeat monitor continues to run while instance is running. This way if a node sends heartbeat to this node as elected coordinator changes, we notify the node accordingly. Handle Exceptions more gracefully in leader election code. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #728: NIFI-2388: Use null for group id if no group specifi...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/728 NIFI-2388: Use null for group id if no group specified instead of throwing NPE You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2388 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/728.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #728 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #529: NIFI-2028: Fixed Site-to-Site Transit URI
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/529#discussion_r72617680
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site/src/main/java/org/apache/nifi/remote/StandardRemoteGroupPort.java
---
@@ -366,6 +371,14 @@ private int receiveFlowFiles(final Transaction
transaction, final ProcessContext
return flowFilesReceived.size();
}
+private String createTransitUri(Transaction transaction, String
sourceFlowFileIdentifier) {
+if (transaction instanceof CustomTransitUriTransaction) {
--- End diff --
@ijokarumawak I think I like this a little better, but it still feels off
to me. Transaction is an interface, so I don't like inspecting the type of the
interface in order to determine how to interact with it. What if instead of
taking this approach, we update the Communicant interface to have a
`createTransitUri(String sourceFlowFileIdentifier)` method? Then, we can just
call `transaction.getCommunicant().createTransitUri(sourceFlowFileIdentifier)`.
The raw socket transaction and the HTTP Transaction can then wrap the `Peer`
class as necessary to ensure that the appropriate URI is created. This allows
us to get away from having to have special interfaces that extend Transaction
as well as having to inspect the Transaction to find out how to handle it. It
also avoids modifying the `Transaction` interface, which is sort of the 'main'
interface for Site-to-Site communications. Thoughts?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #688: NIFI-2304: If node address is not set, default to lo...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/688#discussion_r72613879
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/heartbeat/ClusterProtocolHeartbeatMonitor.java
---
@@ -91,12 +91,8 @@ public ClusterProtocolHeartbeatMonitor(final
ClusterCoordinator clusterCoordinat
this.clusterNodesPath =
zkClientConfig.resolvePath("cluster/nodes");
String hostname =
properties.getProperty(NiFiProperties.CLUSTER_NODE_ADDRESS);
-if (hostname == null) {
-try {
-hostname = InetAddress.getLocalHost().getHostName();
-} catch (UnknownHostException e) {
-throw new RuntimeException("Unable to determine local
hostname and the '" + NiFiProperties.CLUSTER_NODE_ADDRESS + "' property is not
set");
-}
+if (hostname == null || hostname.trim().isEmpty()) {
+hostname = "localhost";
--- End diff --
@ijokarumawak the reason that I explicitly changed it to 'localhost' is
because this is what the documentation says will be used. This is how it was
done in 0.x, as well. I think the reason for this is that we want to urge
admins to configure it, because getLocalHost().getHostName() may not
necessarily return the correct result if there are multiple network interfaces,
which is quite common in production systems. For example, the hostname on a
private network may be nifi-01 while the hostname on the public lan will be
r00s80-hq.mycompany.com. So getLocalHost() i believe will return whatever the
'hostname' command returns (in linux at least). But there's a good chance that
we want to use the private LAN for cluster comms. So this just sort of urges
the admin to configure it... at least, I think that's why the decision was made
to use "localhost." Not sure since I didn't make the decision :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #688: NIFI-2304: If node address is not set, default to lo...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/688#discussion_r72614705
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/heartbeat/ClusterProtocolHeartbeatMonitor.java
---
@@ -91,12 +91,8 @@ public ClusterProtocolHeartbeatMonitor(final
ClusterCoordinator clusterCoordinat
this.clusterNodesPath =
zkClientConfig.resolvePath("cluster/nodes");
String hostname =
properties.getProperty(NiFiProperties.CLUSTER_NODE_ADDRESS);
-if (hostname == null) {
-try {
-hostname = InetAddress.getLocalHost().getHostName();
-} catch (UnknownHostException e) {
-throw new RuntimeException("Unable to determine local
hostname and the '" + NiFiProperties.CLUSTER_NODE_ADDRESS + "' property is not
set");
-}
+if (hostname == null || hostname.trim().isEmpty()) {
--- End diff --
@ijokarumawak I just updated the PR to check if empty as well, since I do
think the error message is cleaner in that case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #688: NIFI-2304: If node address is not set, default to lo...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/688#discussion_r72614233
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/heartbeat/ClusterProtocolHeartbeatMonitor.java
---
@@ -91,12 +91,8 @@ public ClusterProtocolHeartbeatMonitor(final
ClusterCoordinator clusterCoordinat
this.clusterNodesPath =
zkClientConfig.resolvePath("cluster/nodes");
String hostname =
properties.getProperty(NiFiProperties.CLUSTER_NODE_ADDRESS);
-if (hostname == null) {
-try {
-hostname = InetAddress.getLocalHost().getHostName();
-} catch (UnknownHostException e) {
-throw new RuntimeException("Unable to determine local
hostname and the '" + NiFiProperties.CLUSTER_NODE_ADDRESS + "' property is not
set");
-}
+if (hostname == null || hostname.trim().isEmpty()) {
--- End diff --
For port, we do validate that the port can be parsed via Integer.parseInt
-- if empty string, that will fail. I guess the error message may be more clear
if we explicitly check for empty string though and indicate that the property
is not specified.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #734: NIFI-2395: Ensure that if we fail to index provenanc...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/734#discussion_r72636644
--- Diff:
nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/PersistentProvenanceRepository.java
---
@@ -1668,24 +1669,32 @@ public Thread newThread(final Runnable r) {
}
});
+final AtomicInteger indexingFailureCount = new
AtomicInteger(0);
try {
for (int i = 0; i <
configuration.getIndexThreadPoolSize(); i++) {
final Callable callable = new
Callable() {
@Override
public Object call() throws IOException {
while (!eventQueue.isEmpty() ||
!finishedAdding.get()) {
-final
Tuple<StandardProvenanceEventRecord, Integer> tuple;
try {
-tuple = eventQueue.poll(10,
TimeUnit.MILLISECONDS);
-} catch (final
InterruptedException ie) {
--- End diff --
yeah you're right - we need to ensure that we re-set the interrupt flag.
Pushed an update to the PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #731: NIFI-2337: Turn off logging for org.apache.curator.f...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/731 NIFI-2337: Turn off logging for org.apache.curator.framework.recipes.⦠â¦leader.LeaderSelector since it logs only a single error, and that error is a but in Curator You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2337 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/731.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #731 commit 9e9fce91eef9d3f8aa627407aa5d3677f7691c26 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-27T20:30:44Z NIFI-2337: Turn off logging for org.apache.curator.framework.recipes.leader.LeaderSelector since it logs only a single error, and that error is a but in Curator --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #705: NIFI-2360: Leave ZooKeeper running when a node is disconnec...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/705 @JPercivall - interestingly enough i didn't see this in any of my testing but when i switched back to the branch i saw it immediately when i started up the cluster. A restart fixed it, but then after restarting the cluster another 10 times or so, i was able to replicate the issue again. So it seems a bit sporadic. I was able to address the issue, though. I have updated the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #742: NIFI-2431: Before registering for the Cluster Coordi...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/742 NIFI-2431: Before registering for the Cluster Coordinator role, check⦠⦠if another node already has this role. If so, do not register for this role until after the node has joined the cluster and inherited the flow. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2431 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/742.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #742 commit d0333bb499daafbeb3580be5992949a93414e6c2 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-29T13:14:17Z NIFI-2431: Before registering for the Cluster Coordinator role, check if another node already has this role. If so, do not register for this role until after the node has joined the cluster and inherited the flow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #734: NIFI-2395: Ensure that if we fail to index provenanc...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/734#discussion_r72788219
--- Diff:
nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/PersistentProvenanceRepository.java
---
@@ -1668,24 +1669,32 @@ public Thread newThread(final Runnable r) {
}
});
+final AtomicInteger indexingFailureCount = new
AtomicInteger(0);
try {
for (int i = 0; i <
configuration.getIndexThreadPoolSize(); i++) {
final Callable callable = new
Callable() {
@Override
public Object call() throws IOException {
while (!eventQueue.isEmpty() ||
!finishedAdding.get()) {
-final
Tuple<StandardProvenanceEventRecord, Integer> tuple;
try {
-tuple = eventQueue.poll(10,
TimeUnit.MILLISECONDS);
-} catch (final
InterruptedException ie) {
-continue;
+final
Tuple<StandardProvenanceEventRecord, Integer> tuple;
+try {
+tuple =
eventQueue.poll(10, TimeUnit.MILLISECONDS);
+} catch (final
InterruptedException ie) {
+continue;
--- End diff --
D'oh! You caught another one! Updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #718: NIFI-2366 - Fixed ID generation semantics in cluster...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/718#discussion_r72813537
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/web/util/SnippetUtilsTest.java
---
@@ -0,0 +1,162 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.util;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertTrue;
+
+import java.lang.reflect.Method;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.UUID;
+
+import org.apache.nifi.util.ComponentIdGenerator;
+import org.junit.Test;
+
+public class SnippetUtilsTest {
+
+/*
+ * This test validates condition where component is being replicated
across
+ * the cluster
+ */
+@Test
+public void validateWithSameSeedSameInceptionIdSameInstanceId() throws
Exception {
+Method generateIdMethod =
SnippetUtils.class.getDeclaredMethod("generateId", String.class, String.class,
--- End diff --
@olegz I do not agree with the assessment, but at this point I don't have a
strong enough opposition to it, to avoid merging so I will accept it and merge
:)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #746: NIFI-2388: Addressed a spot that assumes that Proces...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/746 NIFI-2388: Addressed a spot that assumes that Process Group is always⦠⦠set on StandardControllerServiceNode You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2388-Part2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/746.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #746 commit 27649c4c32f49f632ffdd33418698484ce8df4f5 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-29T14:57:33Z NIFI-2388: Addressed a spot that assumes that Process Group is always set on StandardControllerServiceNode --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #746: NIFI-2388: Addressed a spot that assumes that Proces...
Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/746 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #752: NIFI-2291: Correct the Content URI for 1.0.0 REST AP...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/752 NIFI-2291: Correct the Content URI for 1.0.0 REST API; added cluster ⦠â¦node identifier & whether or not clustered to ReportingContext so that the Reporting Task could make use of it You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2291 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/752.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #752 commit 998a5be3b1fef77f1cb189e9b88e158656433d11 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-30T01:17:47Z NIFI-2291: Correct the Content URI for 1.0.0 REST API; added cluster node identifier & whether or not clustered to ReportingContext so that the Reporting Task could make use of it --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #758: NIFI-2444 NIFI-2445 fixed PublishKafka
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/758 @olegz the KafkaPublisher class is now logging everything twice. I think we need to get rid of the internal logger and use only the Processor Logger. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #702: NIFI-1950 Updating FileAuthorizer to convert access control...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/702 @bbende I was able to verify functionality and code review looks good. Thanks for updating this! +1 pushed to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #720: Introduce data resource for authorizing Provenance and Queu...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/720 Things look good on here, but I noticed that the seeding of the authorizations.xml file does not include the DN's of the hosts in my authorizers.xml file, but it did include my 'admin dn' from the authorizers.xml file (for the 'view data' and 'modify data' policies of the root group). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #718: NIFI-2366 - Fixed ID generation semantics in clustered envi...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/718 @olegz looks good! Did a full build and created several process groups, processors, input & output ports, funnels, remote process groups, and labels. Copied & pasted them all over the place, multiple times. Moved in & out of process groups. Created templates and instantiated them. all appear to work as expected. Thanks for jumping in and knocking this out - i know it gets tricky! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #529: NIFI-2028: Fixed Site-to-Site Transit URI
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/529#discussion_r72096781
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site/src/main/java/org/apache/nifi/remote/StandardRemoteGroupPort.java
---
@@ -366,6 +371,17 @@ private int receiveFlowFiles(final Transaction
transaction, final ProcessContext
return flowFilesReceived.size();
}
+private String createTransitUri(final Transaction transaction, String
sourceFlowFileIdentifier) {
+// Use actual uri as transitUri if this transport protocol has
meaningful uri.
+switch (remoteGroup.getTransportProtocol()) {
--- End diff --
This feels wrong to me. We should be using the URI that the communicant
gives to us, regardless of which implementation is being used. If the URI
returned by the HTTP implementation is not meaningful, then we should update
the code so that we pass in a more meaningful URI
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #828: NIFI-2540: Exclude from templates the parts of prope...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/828 NIFI-2540: Exclude from templates the parts of property descriptors t⦠â¦hat are not necessary. Also ensure that Property Descriptors are not completely removed from Controller Services. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2540 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/828.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #828 commit e3ebf1b67cced7c2fe89f73511ff9aba6f2f814e Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-10T15:36:21Z NIFI-2540: Exclude from templates the parts of property descriptors that are not necessary. Also ensure that Property Descriptors are not completely removed from Controller Services. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #771: NIFI-2452
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/771 NIFI-2452 Ensure that we keep track of how many references we have to each lucene searcher and only close the underlying index reader if there are no references to the searcher. Also updated to prefer newer provenance events over older provenance events, and calculate FlowFile lineage based on an event id instead of a FlowFile UUID, as it's much more efficient You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2452 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/771.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #771 commit 3595b8bbc590d98ff1ad939ca2bd022b257307b6 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-01T18:51:02Z NIFI-2452: Ensure that we do not close Index Readers that are still in use commit 87100428902db677a014a3936f2dda4aceee5dd9 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-02T13:56:05Z NIFI-2452: Ensure that we keep track of how many references we have to each lucene searcher and only close the underlying index reader if there are no references to the searcher. Also updated to prefer newer provenance events over older provenance events, and calculate FlowFile lineage based on an event id instead of a FlowFile UUID, as it's much more efficient --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #761: NIFI-2444 NIFI-2445 fixed PublishKafka
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/761 Thanks @olegz. All looks good. +1 merged to 0.x --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #836: NIFI-2545: Ensure that when @OnUnscheduled and @OnSt...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/836 NIFI-2545: Ensure that when @OnUnscheduled and @OnStopped methods are⦠⦠called that the active thread count takes that thread into account You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2545 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/836.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #836 commit d6417ef43db2e0697b35ff1c67722abcfccf66c2 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-11T15:18:50Z NIFI-2545: Ensure that when @OnUnscheduled and @OnStopped methods are called that the active thread count takes that thread into account --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #829: NIFI-2535: Do not include properties that are unset ...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/829 NIFI-2535: Do not include properties that are unset in flow fingerpri⦠â¦nt. This allows a new property to be added to a processor without affecting the fingerprint, if the value is never set You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2535 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/829.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #829 commit f676eaaee2ba98361fac2d10ef524eb9d9335bdf Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-10T17:57:45Z NIFI-2535: Do not include properties that are unset in flow fingerprint. This allows a new property to be added to a processor without affecting the fingerprint, if the value is never set --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #832: NIFI-2544: Created integration tests for clustering ...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/832 NIFI-2544: Created integration tests for clustering and addressed a f⦠â¦ew minor bugs that were found in doing so You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2544 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/832.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #832 commit 944543ed3bfb17166648d0db85283ca262c450de Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-09T20:50:04Z NIFI-2544: Created integration tests for clustering and addressed a few minor bugs that were found in doing so --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #776: NIFI-2446: Add option to specify key password when d...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/776 NIFI-2446: Add option to specify key password when different than keystore password NIFI-2446: Add option to specify key password when different than keystore password You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2466 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/776.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #776 commit aa4d4189c40380554295af730746957438cf2616 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-03T14:49:17Z NIFI-2446: Add option to specify key password when different than keystore password --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #782: NIFI-2474: Remove VariableRegistry from outward faci...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/782 NIFI-2474: Remove VariableRegistry from outward facing API so that th⦠â¦e it is more flexible to evolve You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2474 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/782.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #782 commit 2004ed2fdeaf1190cc8ac8ccb137909b17f07ba2 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-04T00:44:04Z NIFI-2474: Remove VariableRegistry from outward facing API so that the it is more flexible to evolve --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #776: NIFI-2466: Add option to specify key password when differen...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/776 @alopresto wow i really messed up my testing. Sorry about that, and great catch. I replicated your steps above and added the keystore to src/test/resources for the StandardSSLContextService and then added a unit test to verify that things work properly. Pushed as a separate commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #760: Reporting appropriate error message when not authorized to ...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/760 Verified behavior before & after patch. Change looks good. +1, merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #778: NIFI-2407 Implements EL support on some properties o...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/778#discussion_r73411049
--- Diff:
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/test/java/org/apache/nifi/processors/hadoop/PutHDFSTest.java
---
@@ -199,7 +202,39 @@ public void testPutFile() throws IOException {
TestRunner runner = TestRunners.newTestRunner(proc);
runner.setProperty(PutHDFS.DIRECTORY, "target/test-classes");
runner.setProperty(PutHDFS.CONFLICT_RESOLUTION, "replace");
-runner.setValidateExpressionUsage(false);
+try (FileInputStream fis = new
FileInputStream("src/test/resources/testdata/randombytes-1");) {
+Map<String, String> attributes = new HashMap<String, String>();
+attributes.put(CoreAttributes.FILENAME.key(), "randombytes-1");
+runner.enqueue(fis, attributes);
+runner.run();
+}
+
+Configuration config = new Configuration();
+FileSystem fs = FileSystem.get(config);
+
+List failedFlowFiles = runner
+.getFlowFilesForRelationship(new
Relationship.Builder().name("failure").build());
+assertTrue(failedFlowFiles.isEmpty());
+
+List flowFiles =
runner.getFlowFilesForRelationship(PutHDFS.REL_SUCCESS);
+assertEquals(1, flowFiles.size());
--- End diff --
The above assertion that failed flowfiles is empty and that this has size 1
can be simplified to simply using:
runner.assertAllFlowFilesTransferred(PutHDFS.REL_SUCCESS, 1);
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #778: NIFI-2407 Implements EL support on some properties o...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/778#discussion_r73416186
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-dbcp-service/src/test/java/org/apache/nifi/dbcp/DBCPServiceTest.java
---
@@ -258,6 +264,28 @@ public void testURLClassLoaderGetConnection() throws
ClassNotFoundException, Mal
DriverManager.deregisterDriver(shim);
}
+@Test
+public void testPropertiesUseVariableRegistry() throws
InitializationException, SQLException {
+final VariableRegistry variableRegistry =
mock(VariableRegistry.class);
+
given(variableRegistry.getVariableValue("databaseurl")).willReturn("jdbc:derby:"
+ DB_LOCATION + ";create=true");
+
given(variableRegistry.getVariableValue("drivername")).willReturn("org.apache.derby.jdbc.EmbeddedDriver");
+
given(variableRegistry.getVariableValue("username")).willReturn("tester");
+
given(variableRegistry.getVariableValue("password")).willReturn("testerp");
+
+final TestRunner runner =
TestRunners.newTestRunner(mock(org.apache.nifi.processor.Processor.class),
variableRegistry);
+final DBCPConnectionPool service = new DBCPConnectionPool();
+runner.addControllerService("dbcpService", service);
+
+runner.setProperty(service, DBCPConnectionPool.DATABASE_URL,
"${databaseurl}");
+runner.setProperty(service, DBCPConnectionPool.DB_DRIVERNAME,
"${drivername}");
+runner.setProperty(service, DBCPConnectionPool.DB_USER,
"${username}");
+runner.setProperty(service, DBCPConnectionPool.DB_PASSWORD,
"${password}");
+
+runner.enableControllerService(service);
+
+
then(variableRegistry).should(times(4)).getVariableValue(anyString());
--- End diff --
This feels brittle. The controller service being tested does not call into
the variable registry - it simply calls evaluateAttributeExpressions(). So
verifying that the mock framework calls into the variable registry 4 times is
really testing the mock framework moreso than the Controller Service. The
Controller Service (or mock framework) could certainly be changed so that they
call getVariableValue() many more times without any change in functionality, as
well.
I think a better test would be to ensure that each time that the property's
value is obtained, the EL is first evaluated. This is already done by the mock
framework - if the PropertyDescriptor indicates that it supports expression
language but getValue() is called on the PropertyValue without first calling
evaluateAttributeExpressions(), the mock framework calls Assert.fail()
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #752: NIFI-2291: Correct the Content URI for 1.0.0 REST API; adde...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/752 Excellent, thanks, @YolandaMDavis ! Congratulations of your first merge :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #778: NIFI-2407 Implements EL support on some properties o...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/778#discussion_r73406263
--- Diff:
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
---
@@ -191,9 +191,19 @@ public final void abstractOnScheduled(ProcessContext
context) throws IOException
HdfsResources resources = hdfsResources.get();
if (resources.getConfiguration() == null) {
String configResources =
context.getProperty(HADOOP_CONFIGURATION_RESOURCES).getValue();
-String dir =
context.getProperty(DIRECTORY_PROP_NAME).getValue();
-dir = dir == null ? "/" : dir;
-resources = resetHDFSResources(configResources, dir,
context);
+final String dir;
+final PropertyDescriptor directoryPropDescriptor =
context.getProperties().keySet().stream()
--- End diff --
@jtstorck I think this can be made a lot simpler by simply calling
getPropertyDescriptor(DIRECTORY_PROP_NAME) of the AbstractProcessor super
class, no?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #776: NIFI-2466: Add option to specify key password when differen...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/776 @alopresto sorry about that NPE - i had a typo where i referenced the configContext instead of validation context. The config context isn't guaranteed to be set when customValidate is called. Pushed a new commit for that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #741: NIFI-2322, NIFI-2423, NIFI-2412 Kafka improvements
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/741#discussion_r73512333 --- Diff: nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-pubsub-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/AbstractKafkaProcessor.java --- @@ -141,6 +166,10 @@ SHARED_DESCRIPTORS.add(CLIENT_ID); SHARED_DESCRIPTORS.add(SECURITY_PROTOCOL); SHARED_DESCRIPTORS.add(KERBEROS_PRINCIPLE); +SHARED_DESCRIPTORS.add(SSL_KEY_PASSWORD); --- End diff -- @olegz I do agree that it would be nice to allow users to explicitly mark dynamic properties as sensitive. However, even with that capability, providing the option of using an SSLContextService is still preferable, I believe, as it is typically much easier to configure, since you can configure the service once and then use anywhere. I have attached a patch to NIFI-2423 that incorporates the SSL Context Service. Can you please review & if good either merge into your PR or give it a +1 on the JIRA? Then I should be able to merge in this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #690: NIFI-2035: Verify existence of source and destination when ...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/690 Hey @YolandaMDavis - the changes that were made in the later commit will not completely prevent the IllegalArgumentException from being thrown in verifyCreate... it's still possible because after the authorization is performed, it's still possible for the background thread to change the ports. But you'd have to have *REALLY* good timing to hit that :) Very unlikely you'd be able to trigger it intentionally. So if you think the change looks reasonable and I didn't break anything, I think it should be good to go --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #783: NIFI-2478 Fixed Zk leaders path to include root.
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/783 @ijokarumawak I verified the fix works & contrib-check looks good. +1 merged to master. Thanks for catching that! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #789: NIFI-2483: Removed 'passthrough' relationship
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/789 NIFI-2483: Removed 'passthrough' relationship You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2483 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/789.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #789 commit 186f5727c921afe69221b8453d23f3e1fcdf2619 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-04T14:41:18Z NIFI-2483: Removed 'passthrough' relationship --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #729: NIFI-2406: Ensure that hearbeat monitor continues to run wh...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/729 @YolandaMDavis thanks - PR i was rebased & pushed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #786: NIFI-2472: Normalize coordinates for all components ...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/786 NIFI-2472: Normalize coordinates for all components in a snippet, not⦠⦠just processors You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/786.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #786 commit ee948076adba809c3ad8ccadf62d6a99735d7771 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-04T14:31:58Z NIFI-2472: Normalize coordinates for all components in a snippet, not just processors --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #779: NIFI-2471 fix Hadoop configuration resources when ta...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/779#discussion_r73546113
--- Diff:
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java
---
@@ -177,9 +178,17 @@ protected KerberosProperties getKerberosProperties() {
return results;
}
+@Override
+public void onPropertyModified(PropertyDescriptor descriptor, String
oldValue, String newValue) {
--- End diff --
Looks like you're right - it is not calling @OnStopped methods when
@OnScheduled throws an Exception. I created a ticket for that -
https://issues.apache.org/jira/browse/NIFI-2484. I think we should avoid
working around the bug in this PR though and just address it in the framework.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #848: NIFI-2551: Addressed a threading issue in the Standa...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/848 NIFI-2551: Addressed a threading issue in the StandardResourceClaimManager⦠⦠and performed some refactoring so that we can ensure thread-safety across different components of the application, such as ProcessSession and WriteAheadFlowFile Repository when interacting with the StandardResourceClaimManager. Update DebugFlow to allow it to write to a FlowFile multiple times, which exposes the concurrency bug. Also avoided calling ContentRepository.remove() from ProcessSession whenever the content is no longer needed, as that can cause problems now that the Resource Claim is backing the content claim. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2551-1.x Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/848.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #848 commit 2c2b4e9a5ddf030b60631f8ef24fca3b8b5324d6 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-12T17:56:03Z NIFI-2551: Addressed a threading issue in the StandardResourceClaimManager and performed some refactoring so that we can ensure thread-safety across different components of the application, such as ProcessSession and WriteAheadFlowFile Repository when interacting with the StandardResourceClaimManager. Update DebugFlow to allow it to write to a FlowFile multiple times, which exposes the concurrency bug. Also avoided calling ContentRepository.remove() from ProcessSession whenever the content is no longer needed, as that can cause problems now that the Resource Claim is backing the content claim. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #847: NIFI-2551: Addressed a threading issue in the Standa...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/847 NIFI-2551: Addressed a threading issue in the StandardResourceClaimManager⦠⦠and performed some refactoring so that we can ensure thread-safety across different components of the application, such as ProcessSession and WriteAheadFlowFile Repository when interacting with the StandardResourceClaimManager. Update DebugFlow to allow it to write to a FlowFile multiple times, which exposes the concurrency bug. Also avoided calling ContentRepository.remove() from ProcessSession whenever the content is no longer needed, as that can cause problems now that the Resource Claim is backing the content claim. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2551 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/847.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #847 commit ec79ff0d6a595a969cf65ca9cd65d51caab08620 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-12T17:56:03Z NIFI-2551: Addressed a threading issue in the StandardResourceClaimManager and performed some refactoring so that we can ensure thread-safety across different components of the application, such as ProcessSession and WriteAheadFlowFile Repository when interacting with the StandardResourceClaimManager. Update DebugFlow to allow it to write to a FlowFile multiple times, which exposes the concurrency bug. Also avoided calling ContentRepository.remove() from ProcessSession whenever the content is no longer needed, as that can cause problems now that the Resource Claim is backing the content claim. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #693: NIFI-2333: Consider reporting tasks and controller s...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/693 NIFI-2333: Consider reporting tasks and controller services when determining whether or not flow is empty You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2333 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/693.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #693 commit 99e92e1569173c9512926c06ce0b30aadc2d73df Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-20T19:25:32Z NIFI-2333: Consider reporting tasks and controller services when determining whether or not flow is empty --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #696: NIFI-2336: Ensure that we save changes to the flow w...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/696 NIFI-2336: Ensure that we save changes to the flow when we start or stop the current process group NIFI-2336: Ensure that we save changes to the flow when we start or stop the current process group You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2336 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/696.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #696 commit ff0222e689cc622cb311ac8dae204587b1a68e7c Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-20T23:06:34Z NIFI-2336: Ensure that we save changes to the flow when we start or stop the current process group --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #697: NIFI-2334: Delegate requests for cluster info to the...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/697 NIFI-2334: Delegate requests for cluster info to the cluster coordinator You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2334 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/697.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #697 commit 439db3a2fb54c1f7a0bcb07b61163b2fc1e86d2f Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-21T00:34:10Z NIFI-2334: Delegate requests for cluster info to the cluster coordinator --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #678: NIFI-2316, NIFI-2318: Ensure that we do not save the flow b...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/678 @JPercivall I think the issue that you are seeing now is related to NIFI-2292. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #667: NIFI-2305: Do not run processors that are marked as ...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/667 NIFI-2305: Do not run processors that are marked as Primary Node Only⦠⦠if disconnected from cluster You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2305 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/667.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #667 commit c333950665adcf7197a5bb6acd54b1f6ef8f599e Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-18T14:50:39Z NIFI-2305: Do not run processors that are marked as Primary Node Only if disconnected from cluster --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #666: Fixing issues loading the Policy Management UI
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/666 LGTM. The verbage for the access policies is definitely better imho. +1. Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #671: NIFI-826 (part deux)
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/671 +1 all looks good. I've pushed to master. Thanks for knocking this out! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #682: NiFi 2324 - Log number of requests per URI if we hav...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/682 NiFi 2324 - Log number of requests per URI if we have too many outstanding requests to replicate You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2324 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/682.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #682 commit f3ee623c1e68cf45ee39a7ed58b14ad6e275c014 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-19T19:57:05Z NIFI-2289: Ensure that we include Node ID's of all nodes when finding cluster coordinator, even if the node is currently still connecting or has not yet joined the cluster, which ccan be the case if all nodes in the cluster are restarting at the same time. commit 4bcfba1ccc2405f0be4f1b3f405e4f33425687f1 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-19T20:55:55Z NIFI-2324: Long number of requests per URI if we have too many outstanding requests to replicate --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #683: NIFI-2324: Log number of requests per URI if we have...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/683 NIFI-2324: Log number of requests per URI if we have too many outstan⦠â¦ding requests to replicate You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2324 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/683.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #683 commit 4e8e204da8abc0d37922c2ad3ac2f3aa52715f1a Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-19T20:59:14Z NIFI-2324: Log number of requests per URI if we have too many outstanding requests to replicate --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #682: NiFi 2324 - Log number of requests per URI if we hav...
Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/682 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #681: NIFI-2289: Ensure that we include Node ID's of all n...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/681 NIFI-2289: Ensure that we include Node ID's of all nodes when finding⦠⦠cluster coordinator, even if the node is currently still connecting or has not yet joined the cluster, which ccan be the case if all nodes in the cluster are restarting at the same time. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2289-PART2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/681.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #681 commit f3ee623c1e68cf45ee39a7ed58b14ad6e275c014 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-19T19:57:05Z NIFI-2289: Ensure that we include Node ID's of all nodes when finding cluster coordinator, even if the node is currently still connecting or has not yet joined the cluster, which ccan be the case if all nodes in the cluster are restarting at the same time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #690: NIFI-2035: Verify existence of source and destinatio...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/690 NIFI-2035: Verify existence of source and destination when creating a⦠⦠connection You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2035 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/690.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #690 commit 07cc1748864a69ded3a62e6cb0debe22e4183586 Author: Mark Payne <marka...@hotmail.com> Date: 2016-06-20T18:55:08Z NIFI-2035: Verify existence of source and destination when creating a connection --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #686: NIFI-2330: Ensure that we set ProcessGroupDTO's cont...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/686 NIFI-2330: Ensure that we set ProcessGroupDTO's contents after popula⦠â¦ting sets of components, as the contents dto now copies the sets You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2330 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/686.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #686 commit 4b22d8358b73126f7f062fcba3617df2198a6385 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-20T15:57:41Z NIFI-2330: Ensure that we set ProcessGroupDTO's contents after populating sets of components, as the contents dto now copies the sets --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #688: NIFI-2304: If node address is not set, default to lo...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/688 NIFI-2304: If node address is not set, default to localhost, as the a⦠â¦dmin guide says it should You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2304 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/688.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #688 commit c7415177426d7a18a7bdb54af0b57066dc5d5dbb Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-20T16:21:58Z NIFI-2304: If node address is not set, default to localhost, as the admin guide says it should --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #689: NIFI-2305: Do not run processors that are marked as ...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/689 NIFI-2305: Do not run processors that are marked as Primary Node Only⦠⦠if disconnected from cluster You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2305-0.x Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/689.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #689 commit 05c9cc6b790ef53c6cf1e5f6ceaba85122e8ccf1 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-20T16:39:26Z NIFI-2305: Do not run processors that are marked as Primary Node Only if disconnected from cluster --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #658: Addressing issue searching Provenance while clustered and R...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/658 Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #658: Addressing issue searching Provenance while clustered and R...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/658 Looks good, Matt! Thanks for fixing this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #636: NIFI-2249: Move URI out of component so it is accessible wi...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/636 @mcgilman looks good - merged to master. Thanks for knocking this one out! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #625: NIFI-2214 updated both of the Azure Event Hub processors to...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/625 @apsaltis great, thanks! The MIT licenses are handled a little differently and are placed in the LICENSE file instead of the NOTICE file. I updated accordingly and pushed to master. Thanks for jumping in and fixing this issue! Unfortunately, I'm noticing now that i forgot the magical incantation to close the PR. Can you close it manually? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #663: NIFI-1157 removed method that is no longer necessary post d...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/663 @joewitt I do agree that these methods are now superfluous and should be removed. Verified that all builds and all still looks okay. Contrib-check is good, as well. +1 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #621: NIFI-2185: Proxy requests through the cluster coordi...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/621 NIFI-2185: Proxy requests through the cluster coordinator rather than⦠⦠making use of distributed read/write locks You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2185 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/621.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #621 commit b8bfb286fc54dfae9b09fe010209942f04fb2065 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-08T16:58:06Z NIFI-2185: Proxy requests through the cluster coordinator rather than making use of distributed read/write locks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #625: NIFI-2214 updated both of the Azure Event Hub processors to...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/625 @apsaltis very cool that you were able to jump in and knock this out! The community very much appreciates it! Looking through the code, it all looks good. I did notice though that the client library was changed, and the licensing information was not updated. Need to be sure that the NOTICE file in both the nar's src/main/resources directory as well as the nifi-assembly's NOTICE file. The previous client used an ASF 2.0 license, and the new one uses an MIT license. Also, I did notice that the Put could be more efficient by sending several events asynchronously and then waiting on the ACK's to come back. I created a new JIRA for this though (https://issues.apache.org/jira/browse/NIFI-2220) because it's not necessarily critical to get that into this PR. If you can update the LICENSE info, then I should be able to get this merged in. Thanks again for knocking this out! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #791: NIFI-2484: Ensure that if a Processor throws an Exce...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/791 NIFI-2484: Ensure that if a Processor throws an Exception from an @On⦠â¦Scheduled method that @OnStopped methods get called You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2484 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/791.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #791 commit d3cff21dc464e76c6736109c7c26c12cf418d5a1 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-04T17:05:03Z NIFI-2484: Ensure that if a Processor throws an Exception from an @OnScheduled method that @OnStopped methods get called --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #778: NIFI-2407 Implements EL support on some properties of proce...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/778 @jtstorck all looks good now. Thanks for updating those! +1, merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #779: NIFI-2471 fix Hadoop configuration resources when talking t...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/779 OK thanks @mosermw. Looks good. Merged to master & 0.x branches. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #729: NIFI-2406: Ensure that hearbeat monitor continues to run wh...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/729 @YolandaMDavis is this cluster running an embedded zookeeper? If so, which node(s)? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #808: NIFI-2495: Ensure that we always close Index Searche...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/808 NIFI-2495: Ensure that we always close Index Searchers when we're fin⦠â¦ished with them You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2495 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/808.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #808 commit 8c3ccbaabff78910a684db2f1fb0627de27c6b94 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-08T15:01:32Z NIFI-2495: Ensure that we always close Index Searchers when we're finished with them --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #820: NIFI-2406: Addressed regression introduced in NIFI-2...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/820 NIFI-2406: Addressed regression introduced in NIFI-2406 where the clu⦠â¦ster does not recognize a new Cluster Coordinator when the coordinator is shutdown You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2406-PART2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/820.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #820 commit 3c9289e6c16b915e2a6dbe2e2d82d89a8d3321b8 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-09T15:11:08Z NIFI-2406: Addressed regression introduced in NIFI-2406 where the cluster does not recognize a new Cluster Coordinator when the coordinator is shutdown --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #771: NIFI-2452
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/771#discussion_r73220319 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java --- @@ -16,7 +16,38 @@ */ package org.apache.nifi.controller; -import com.sun.jersey.api.client.ClientHandlerException; --- End diff -- @joewitt this change was due to a bug that I ran into when testing this PR. The bug was preventing me from moving forward, and was a rather trivial fix, so I just included it in the PR. I could have created a separate JIRA and PR but since it would have blocked this PR it seemed more prudent to just roll it in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #771: NIFI-2452
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/771#discussion_r73220870 --- Diff: nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java --- @@ -481,6 +485,165 @@ public void testIndexAndCompressOnRolloverAndSubsequentSearch() throws IOExcepti assertTrue(newRecordSet.getMatchingEvents().isEmpty()); } +// TODO: Switch to 10,000. --- End diff -- Whoops! Overlooked that. Was just a comment that I added in so that if i forgot to change the timeout back to 10,000 millis, the reviewer would catch it - glad it worked! :) Made it much longer so that i could set a breakpoint. Will change back to 10,000 millis and update the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #771: NIFI-2452
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/771#discussion_r73220717
--- Diff:
nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/PersistentProvenanceRepository.java
---
@@ -2166,6 +2171,28 @@ private Lineage computeLineage(final
Collection flowFileUuids, final NiF
}
@Override
+public ComputeLineageSubmission submitLineageComputation(final long
eventId, final NiFiUser user) {
+final ProvenanceEventRecord event;
+try {
+event = getEvent(eventId);
+} catch (final Exception e) {
+logger.error("Failed to retrieve Provenance Event with ID " +
eventId + " to calculate data lineage due to: " + e, e);
+final AsyncLineageSubmission result = new
AsyncLineageSubmission(LineageComputationType.FLOWFILE_LINEAGE, eventId,
Collections. emptySet(), 1, user.getIdentity());
+result.getResult().setError("Failed to retrieve Provenance
Event with ID " + eventId + ". See logs for more information.");
+return result;
+}
+
+if (event == null) {
+final AsyncLineageSubmission result = new
AsyncLineageSubmission(LineageComputationType.FLOWFILE_LINEAGE, eventId,
Collections. emptySet(), 1, user.getIdentity());
+result.getResult().setError("Could not find Provenance Event
with ID " + eventId);
+lineageSubmissionMap.put(result.getLineageIdentifier(),
result);
+return result;
+}
+
+return
submitLineageComputation(Collections.singleton(event.getFlowFileUuid()), user,
LineageComputationType.FLOWFILE_LINEAGE, eventId, event.getLineageStartDate(),
Long.MAX_VALUE);
--- End diff --
When we obtain the event by ID above, we are not querying anything - we are
simply looking up the event directly by ID. We then formulate a query based on
the UUID of the FlowFile. This is different than passing in the UUID of the
FlowFile directly, though, because when we look up the Event directly, we also
have the Lineage Start Date. Knowing that Lineage Start Date can dramatically
reduce the amount of searching required if the start date is fairly recent.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #705: NIFI-2360: Leave ZooKeeper running when a node is di...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/705 NIFI-2360: Leave ZooKeeper running when a node is disconnected. Do no⦠â¦t allow the last node in the cluster to be disconnected. Change ClusterProtocoLHeartbeater to use RetryNTime retry strategy instead of RetryForever because web requests could block on this You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2360 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/705.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #705 commit 8ad138d02201d2b593ebb579b51b0c853829b568 Author: Mark Payne <marka...@hotmail.com> Date: 2016-07-22T13:21:12Z NIFI-2360: Leave ZooKeeper running when a node is disconnected. Do not allow the last node in the cluster to be disconnected. Change ClusterProtocoLHeartbeater to use RetryNTime retry strategy instead of RetryForever because web requests could block on this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #741: NIFI-2322, NIFI-2423, NIFI-2412 Kafka improvements
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/741#discussion_r7456 --- Diff: nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-pubsub-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/AbstractKafkaProcessor.java --- @@ -141,6 +166,10 @@ SHARED_DESCRIPTORS.add(CLIENT_ID); SHARED_DESCRIPTORS.add(SECURITY_PROTOCOL); SHARED_DESCRIPTORS.add(KERBEROS_PRINCIPLE); +SHARED_DESCRIPTORS.add(SSL_KEY_PASSWORD); --- End diff -- This feels very odd to me to provide properties for the passwords but not the keystore and truststore. I do realize that the keystore and truststore can be specified via user-defined properties, but it adds to the complexity of configuring this, as the user has to lookup the appropriate property names. This is a usability miss. Why do we not simply expose a property for an SSLContextService? We could then pull the appropriate values from the SSLContextService via getKeyStorePassword(), getKeyStoreFile(), etc. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #690: NIFI-2035: Verify existence of source and destination when ...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/690 @YolandaMDavis I will look into the NPE. The test case that you described will not cause the issue that you are expecting, though. There is no way to test it without doing some funky stuff with breakpoints, etc. It is a timing issue. Would recommend that a code review and testing that the change doesn't break anything is sufficient. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #752: NIFI-2291: Correct the Content URI for 1.0.0 REST API; adde...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/752 @YolandaMDavis wow that's a good catch and is very... odd. I think this is actually a JDK bug. I'm about 99% sure that the anonymous inner class *should* be able to access that method. And if it can't, it should be a compile-time error not a runtime error. In fact, the Javadocs for that error even explicitly say: > Normally, this error is caught by the compiler; this error can only occur at run time if the definition of a class has incompatibly changed. I'm quite certain that the definition of the class has not changed. In any event, I believe the proposed solution is a reasonable one, so I implemented it and pushed a new commit. I also addressed your other feedback by making the URI path a constant. Good call. Thanks for taking the time to review this. Let me know if you run into anything else. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #761: NIFI-2444 NIFI-2445 fixed PublishKafka
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/761#discussion_r73238512
--- Diff:
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/main/java/org/apache/nifi/processors/kafka/KafkaPublisher.java
---
@@ -233,12 +230,10 @@ void setProcessLog(ProcessorLog processLog) {
*/
private void warnOrError(String message, Exception e) {
if (e == null) {
-logger.warn(message);
if (this.processLog != null) {
--- End diff --
This feels a little off to me, @olegz - why not just pass the ComponentLog
in to the constructor? It should always be present.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #761: NIFI-2444 NIFI-2445 fixed PublishKafka
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/761#discussion_r73238649
--- Diff:
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-pubsub-processors/src/test/java/org/apache/nifi/processors/kafka/pubsub/StubPublishKafka.java
---
@@ -85,21 +95,31 @@ protected KafkaPublisher
buildKafkaResource(ProcessContext context, ProcessSessi
@SuppressWarnings("unchecked")
private void instrumentProducer(Producer<byte[], byte[]> producer,
boolean failRandomly) {
+
when(producer.send(Mockito.any(ProducerRecord.class))).then(new
Answer<Future>() {
-@SuppressWarnings("rawtypes")
@Override
public Future answer(InvocationOnMock
invocation) throws Throwable {
ProducerRecord<byte[], byte[]> record =
(ProducerRecord<byte[], byte[]>) invocation.getArguments()[0];
-String value = new String(record.value(),
StandardCharsets.UTF_8);
+final String value = new String(record.value(),
StandardCharsets.UTF_8);
if ("fail".equals(value) && !StubPublishKafka.this.failed)
{
StubPublishKafka.this.failed = true;
throw new RuntimeException("intentional");
}
-Future future = mock(Future.class);
-if ("futurefail".equals(value) &&
!StubPublishKafka.this.failed) {
-StubPublishKafka.this.failed = true;
-when(future.get(Mockito.anyLong(),
Mockito.any(TimeUnit.class))).thenThrow(ExecutionException.class);
-}
+
+Future future = executor.submit(new
Callable() {
+@Override
+public RecordMetadata call() throws Exception {
+if ("futurefail".equals(value) &&
!StubPublishKafka.this.failed) {
+// System.out.println("FAIL");
--- End diff --
Can remove this line?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #690: NIFI-2035: Verify existence of source and destination when ...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/690 @YolandaMDavis I updated the PR with a new commit so that you can see what changed. It looks like the way to cause it to hit is to drag a connection from a Processor to the Remote Process Group, and then once the dialog is up to choose which port, remove the port from a different browser tab, and then wait 30 seconds for the RPG to refresh ports. Then click the "Apply" button on the "Add Connection" dialog --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #776: NIFI-2466: Add option to specify key password when differen...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/776 @alopresto yes i was able to test with ListenHTTP / PostHTTP so that the SSL Context was used on both the receiving & sending ends, using both keystore and truststore. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #847: NIFI-2551: Addressed a threading issue in the StandardResou...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/847 Great, thanks, @mosermw ! Will remove those outdated comments and push. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #866: NIFI-2566: Refactoring to improve robustness of clus...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/866 NIFI-2566: Refactoring to improve robustness of cluster You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2566 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/866.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #866 commit 90dc74627157296b4084ed65c1b752bfe25db1e8 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-13T23:38:07Z NIFI-2566: Refactored to allow just the Leader Election Manager to be responsible for determining who is the Cluster Coordinator commit 3fd121b2ff57c5fc51171791351580c29c58e553 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-14T23:40:31Z NIFI-2566: Removed storage of cluster roles from heartbeats and NodeConnectionStatus; use LeaderElectionManager to determine roles instead commit 457a2df5c3296013ce896ca7900807d9bdb69a71 Author: Mark Payne <marka...@hotmail.com> Date: 2016-08-15T20:35:56Z NIFI-2566: Updated Heartbeats so that if a node is out-of-sync with cluster topology, cluster coordinator will provide updated information back to the nodes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #848: NIFI-2551: Addressed a threading issue in the Standa...
Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/848 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #847: NIFI-2551: Addressed a threading issue in the Standa...
Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/847 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #872: NIFI-2574 checkpoint commit to allow 1831 progress
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/872 @joewitt it appears that the queue swap threshold isn't set when the Connection Builder is created/populated --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
