[jira] [Commented] (MINIFI-115) Upgrade NiFi dependencies to 1.0
[ https://issues.apache.org/jira/browse/MINIFI-115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596265#comment-15596265 ] ASF GitHub Bot commented on MINIFI-115: --- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/43#discussion_r84537920 --- Diff: minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-framework-core/src/main/java/org/apache/nifi/minifi/status/StatusRequestParser.java --- @@ -122,15 +122,13 @@ static RemoteProcessGroupStatusBean parseRemoteProcessGroupStatusRequest(RemoteP new BulletinQuery.Builder() .sourceIdMatches(inputRemoteProcessGroupStatus.getId()) .build()); -List authorizationIssues = inputRemoteProcessGroupStatus.getAuthorizationIssues(); --- End diff -- Why was this removed? Aren't Auth Issues part of S2S? > Upgrade NiFi dependencies to 1.0 > > > Key: MINIFI-115 > URL: https://issues.apache.org/jira/browse/MINIFI-115 > Project: Apache NiFi MiNiFi > Issue Type: Task > Components: Core Framework >Affects Versions: 0.0.1 >Reporter: Aldrin Piri >Assignee: Aldrin Piri > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: [NIFI-2929] remove canvas element names when zoomed out and view permissions are removed. This closes #1152
Repository: nifi Updated Branches: refs/heads/master 220b76803 -> 6676e94a7 [NIFI-2929] remove canvas element names when zoomed out and view permissions are removed. This closes #1152 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/6676e94a Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/6676e94a Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/6676e94a Branch: refs/heads/master Commit: 6676e94a7cce68501dc5c010e42c036a2d92a472 Parents: 220b768 Author: Scott AslanAuthored: Fri Oct 21 15:45:46 2016 -0400 Committer: Matt Gilman Committed: Fri Oct 21 16:11:27 2016 -0400 -- .../src/main/webapp/js/nf/canvas/nf-port.js | 3 +++ .../webapp/js/nf/canvas/nf-process-group.js | 20 +++- .../main/webapp/js/nf/canvas/nf-processor.js| 20 +++- .../js/nf/canvas/nf-remote-process-group.js | 20 +++- 4 files changed, 12 insertions(+), 51 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/6676e94a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-port.js -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-port.js b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-port.js index 5b58d44..d5e4ac7 100644 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-port.js +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-port.js @@ -309,6 +309,9 @@ nf.Port = (function () { return name; } }); +} else { +// clear the port name +port.select('text.port-name').text(null); } // remove tooltips if necessary http://git-wip-us.apache.org/repos/asf/nifi/blob/6676e94a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-process-group.js -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-process-group.js b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-process-group.js index 80c7eec..89c1c85 100644 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-process-group.js +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-process-group.js @@ -144,17 +144,6 @@ nf.ProcessGroup = (function () { 'class': 'process-group-name' }); -// process group preview -processGroup.append('image') -.call(nf.CanvasUtils.disableImageHref) -.attr({ -'width': 352, -'height': 113, -'x': 6, -'y': 22, -'class': 'process-group-preview' -}); - // always support selecting and navigation processGroup.on('dblclick', function (d) { // enter this group on double click @@ -813,9 +802,6 @@ nf.ProcessGroup = (function () { processGroup.select('text.process-group-name').text(null); } -// hide the preview - processGroup.select('image.process-group-preview').style('display', 'none'); - // populate the stats processGroup.call(updateProcessGroupStatus); } else { @@ -830,11 +816,11 @@ nf.ProcessGroup = (function () { return name; } }); +} else { +// clear the process group name +processGroup.select('text.process-group-name').text(null); } -// show the preview - processGroup.select('image.process-group-preview').style('display', 'block'); - // remove the tooltips processGroup.call(removeTooltips); http://git-wip-us.apache.org/repos/asf/nifi/blob/6676e94a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-processor.js -- diff --git
[23/27] nifi git commit: Adjusting test to use a temporary folder to alleviate Windows concerns with provenance testing.
Adjusting test to use a temporary folder to alleviate Windows concerns with provenance testing. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/df61cde3 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/df61cde3 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/df61cde3 Branch: refs/heads/appveyor-improvement Commit: df61cde3784b40ba7d1dd0b74e049db547f8af9a Parents: af10118 Author: Aldrin PiriAuthored: Fri Oct 14 11:09:25 2016 -0400 Committer: Aldrin Piri Committed: Fri Oct 21 15:54:33 2016 -0400 -- .../TestPersistentProvenanceRepository.java | 32 +++- 1 file changed, 17 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/df61cde3/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java -- diff --git a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java b/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java index 80a67eb..7c8a6dc 100644 --- a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java +++ b/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java @@ -85,6 +85,7 @@ import org.junit.BeforeClass; import org.junit.Ignore; import org.junit.Rule; import org.junit.Test; +import org.junit.rules.TemporaryFolder; import org.junit.rules.TestName; import org.mockito.Mockito; import org.mockito.invocation.InvocationOnMock; @@ -97,6 +98,9 @@ public class TestPersistentProvenanceRepository { @Rule public TestName name = new TestName(); +@Rule +TemporaryFolder testFolder = new TemporaryFolder(); + private PersistentProvenanceRepository repo; private RepositoryConfiguration config; @@ -104,9 +108,9 @@ public class TestPersistentProvenanceRepository { private EventReporter eventReporter; private List reportedEvents = Collections.synchronizedList(new ArrayList()); -private RepositoryConfiguration createConfiguration() { +private RepositoryConfiguration createConfiguration() throws IOException { config = new RepositoryConfiguration(); -config.addStorageDirectory(new File("target/storage/" + UUID.randomUUID().toString())); +config.addStorageDirectory(testFolder.newFolder()); config.setCompressOnRollover(true); config.setMaxEventFileLife(2000L, TimeUnit.SECONDS); config.setCompressionBlockBytes(100); @@ -169,7 +173,6 @@ public class TestPersistentProvenanceRepository { } - private EventReporter getEventReporter() { return eventReporter; } @@ -360,7 +363,7 @@ public class TestPersistentProvenanceRepository { int immenseAttrSize = 33000; // must be greater than 32766 for a meaningful test StringBuilder immenseBldr = new StringBuilder(immenseAttrSize); -for (int i=0; i < immenseAttrSize; i++) { +for (int i = 0; i < immenseAttrSize; i++) { immenseBldr.append('0'); } final String uuid = "----"; @@ -1491,9 +1494,9 @@ public class TestPersistentProvenanceRepository { assertEquals(5, lineageNodes.stream().map(node -> node.getNodeType()).filter(t -> t == LineageNodeType.PROVENANCE_EVENT_NODE).count()); final Set eventNodes = lineageNodes.stream() -.filter(node -> node.getNodeType() == LineageNodeType.PROVENANCE_EVENT_NODE) -.map(node -> (EventNode) node) -.collect(Collectors.toSet()); +.filter(node -> node.getNodeType() == LineageNodeType.PROVENANCE_EVENT_NODE) +.map(node -> (EventNode) node) +.collect(Collectors.toSet()); final Map nodesByType = eventNodes.stream().collect(Collectors.groupingBy(EventNode::getEventType)); assertEquals(1, nodesByType.get(ProvenanceEventType.RECEIVE).size()); @@ -1517,9 +1520,9 @@ public class TestPersistentProvenanceRepository { assertEquals(3, expandChildNodes.stream().map(node -> node.getNodeType()).filter(t -> t == LineageNodeType.PROVENANCE_EVENT_NODE).count()); final Set childEventNodes = expandChildNodes.stream() -
[12/27] nifi git commit: NIFI-1710 - Resolve path name to nifi.sh script
NIFI-1710 - Resolve path name to nifi.sh script This closes #1137. Signed-off-by: Andy LoPrestoProject: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/3914141c Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/3914141c Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/3914141c Branch: refs/heads/appveyor-improvement Commit: 3914141c45c574fb8d22a72af99165cb535f5f17 Parents: 27dba60 Author: Andre F de Miranda Authored: Fri Oct 14 14:36:48 2016 +1100 Committer: Andy LoPresto Committed: Tue Oct 18 16:43:03 2016 -0700 -- .../src/main/resources/bin/nifi.sh | 28 ++-- 1 file changed, 26 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/3914141c/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh index 79850ee..ff8addd 100755 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh @@ -17,12 +17,36 @@ # Script structure inspired from Apache Karaf and other Apache projects with similar startup approaches -SCRIPT_DIR=$(dirname "$0") -SCRIPT_NAME=$(basename "$0") +# Discover the path of the file + + +# Since MacOS X, FreeBSD and some other systems lack gnu readlink, we use a more portable +# approach based on following StackOverflow comment http://stackoverflow.com/a/1116890/76 + +TARGET_FILE=$0 + +cd $(dirname $TARGET_FILE) +TARGET_FILE=$(basename $TARGET_FILE) + +# Iterate down a (possible) chain of symlinks +while [ -L "$TARGET_FILE" ] +do +TARGET_FILE=$(readlink $TARGET_FILE) +cd $(dirname $TARGET_FILE) +TARGET_FILE=$(basename $TARGET_FILE) +done + +# Compute the canonicalized name by finding the physical path +# for the directory we're in and appending the target file. +PHYS_DIR=`pwd -P` + +SCRIPT_DIR=$PHYS_DIR PROGNAME=$(basename "$0") . "$SCRIPT_DIR"/nifi-env.sh + + warn() { echo "${PROGNAME}: $*" }
[01/27] nifi git commit: NIFI-2873: Nifi throws UnknownHostException with HA NameNode [Forced Update!]
Repository: nifi Updated Branches: refs/heads/appveyor-improvement cde49cc6f -> 81313dda6 (forced update) NIFI-2873: Nifi throws UnknownHostException with HA NameNode Signed-off-by: Matt BurgessNIFI-2873: Changed test hive-site.xml to use local FS, fixed Checkstyle violations This closes #1113 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/e969a5ff Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/e969a5ff Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/e969a5ff Branch: refs/heads/appveyor-improvement Commit: e969a5ffe3a1e03b0b6a0f0a11e954dc396bf25a Parents: 7107616 Author: d810146 Authored: Fri Oct 7 13:27:30 2016 +1100 Committer: Matt Burgess Committed: Fri Oct 14 09:23:19 2016 -0400 -- .../org/apache/nifi/processors/hive/PutHiveStreaming.java | 2 ++ .../java/org/apache/nifi/util/hive/HiveConfigurator.java| 9 + .../nifi-hive-processors/src/test/resources/hive-site.xml | 2 +- 3 files changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/e969a5ff/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveStreaming.java -- diff --git a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveStreaming.java b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveStreaming.java index 84eda83..e53ddc9 100644 --- a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveStreaming.java +++ b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveStreaming.java @@ -323,6 +323,8 @@ public class PutHiveStreaming extends AbstractProcessor { .withMaxOpenConnections(maxConnections) .withHeartBeatInterval(heartbeatInterval); +hiveConfigurator.preload(hiveConfig); + if (SecurityUtil.isSecurityEnabled(hiveConfig)) { final String principal = context.getProperty(kerberosProperties.getKerberosPrincipal()).getValue(); final String keyTab = context.getProperty(kerberosProperties.getKerberosKeytab()).getValue(); http://git-wip-us.apache.org/repos/asf/nifi/blob/e969a5ff/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveConfigurator.java -- diff --git a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveConfigurator.java b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveConfigurator.java index 748847d..ad22177 100644 --- a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveConfigurator.java +++ b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveConfigurator.java @@ -18,6 +18,7 @@ package org.apache.nifi.util.hive; import org.apache.commons.lang3.StringUtils; import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.hive.conf.HiveConf; import org.apache.hadoop.security.UserGroupInformation; @@ -71,6 +72,14 @@ public class HiveConfigurator { return hiveConfig; } +public void preload(Configuration configuration) { +try { +FileSystem.get(configuration); +} catch (IOException ioe) { +// Suppress exception as future uses of this configuration will fail +} +} + public UserGroupInformation authenticate(final Configuration hiveConfig, String principal, String keyTab, long ticketRenewalPeriod, ComponentLog log) throws AuthenticationFailedException { UserGroupInformation ugi; http://git-wip-us.apache.org/repos/asf/nifi/blob/e969a5ff/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/resources/hive-site.xml -- diff --git a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/resources/hive-site.xml b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/resources/hive-site.xml index 52e32a4..7e7f86c 100644 --- a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/resources/hive-site.xml +++ b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/resources/hive-site.xml @@ -17,6 +17,6 @@ fs.default.name -hdfs://hive +file:///
[04/27] nifi git commit: NIFI-2901: Comment out hadoop.version overrides for vendor profiles
NIFI-2901: Comment out hadoop.version overrides for vendor profiles This closes #1139 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/d63e675a Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/d63e675a Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/d63e675a Branch: refs/heads/appveyor-improvement Commit: d63e675a7d2905ae8408adc8114c01c2ea4604c8 Parents: b52b839 Author: Matt BurgessAuthored: Fri Oct 14 12:05:52 2016 -0400 Committer: Oleg Zhurakousky Committed: Fri Oct 14 12:40:31 2016 -0400 -- pom.xml | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/d63e675a/pom.xml -- diff --git a/pom.xml b/pom.xml index 497823b..735a9ec 100644 --- a/pom.xml +++ b/pom.xml @@ -1869,8 +1869,9 @@ language governing permissions and limitations under the License. --> + command-line 2.7.1.2.4.0.0-169 +--> @@ -1892,8 +1893,9 @@ language governing permissions and limitations under the License. --> + command-line 2.7.0-mapr-1602 +--> @@ -1915,8 +1917,9 @@ language governing permissions and limitations under the License. --> + command-line 2.6.0-cdh5.8.1 +-->
[15/27] nifi git commit: Add jsk...@apache.org to KEYS.
Add jsk...@apache.org to KEYS. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/ae503f5d Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/ae503f5d Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/ae503f5d Branch: refs/heads/appveyor-improvement Commit: ae503f5d456a6170bd67cc9ce26c4eb6ed5c7673 Parents: 611cadd Author: Joe SkoraAuthored: Thu Oct 20 00:59:16 2016 -0400 Committer: Joe Skora Committed: Thu Oct 20 00:59:16 2016 -0400 -- KEYS | 58 ++ 1 file changed, 58 insertions(+) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/ae503f5d/KEYS -- diff --git a/KEYS b/KEYS index e7b0e08..c2246d1 100644 --- a/KEYS +++ b/KEYS @@ -402,3 +402,61 @@ BzezmX5TWerF5OgeZCh5CtD/XRwubt7esaUGKcaQrMqV6YUU5CfgLMIk3GFHuvWR XSQLfHE6JkMV7M6Y5ObosaDPZ0UAKl8XbZjwkc2Nf+fZoITI =Gsga -END PGP PUBLIC KEY BLOCK- +pub 4096R/7D84EBD6 2016-10-13 +uid Joseph Skora (CODE SIGNING KEY) +sig 37D84EBD6 2016-10-13 Joseph Skora (CODE SIGNING KEY) +sub 4096R/CCFB335F 2016-10-13 +sig 7D84EBD6 2016-10-13 Joseph Skora (CODE SIGNING KEY) + +-BEGIN PGP PUBLIC KEY BLOCK- +Version: GnuPG v1 + +mQINBFf+3HgBEADYhGAjqDg76ns9SinxM6GBWMbqkW0sFD77F9TqDEujajXtD7w2 +mAykzpxYpOBpCEQKzL8VDcpx/XaWD83/WAqZiP6K6LDjZWuY0HjCw03d5Cn5Vmzl +92CAy5KLv0UqvF/xynT8yToW5OFC0FWdcQkc4z4pmyvPHMd2BNhpK3OfaRIr2zGN +yBhsKm4guCv4A9cYtcba6AayJpggmJTowvgFee5mCsExVP9i7VogicS+1azfaPsO +PVoJ4rvacbJ25qnBpblTkWwp3NFytmaaekORj6d3QK11ARfLs4AhviiZgW55BeU6 +k651wEnlsFju/vN1eaXQisS61lg1krVsLQKPd/cg/R4tRJNZti1lW744Zxnzstw0 +oMEa2ICVtDLLHNHM2TTHVODNGR3QwK2i+MQ1u+CSHyJZ2g5EybKnk19hkKQRjTdY +2GU2c004kBCz3afmj5eVDo6z/1/Q5ehfGdfRcYIY/Qu0rbbOUxC5m4SmJUFQE0ud +Hyijq0PRRUCr4vWyLZBDqf5Bh0SXHqAvoR8EMT2KECuQxoIFhogCOKpaEXF05xn8 +EFt+XqaE2puMnD9zIE2tYqTy/mbw2CPCRXFe5u+1GkqOQF7J6Lwx8v7o7L2bnwOe +W05ZHur33aAYlQv4jd0cWMpvqbhGIjnv6EnyzH23xADPyvkIB8cJqGiXZwARAQAB +tDNKb3NlcGggU2tvcmEgKENPREUgU0lHTklORyBLRVkpIDxqc2tvcmFAYXBhY2hl +Lm9yZz6JAjcEEwECACECGwMCHgECF4AFAlf+5aQFCwkIBwMFFQoJCAsFFgIDAQAA +CgkQlBwUQ32E69bZvhAAswWlZNptTgtlQo2AkRGB4junJsFPueuKR4D8FsNPio1A +iegkd/gR9//Ld7NCxQaqHxf3lQ0lY6CS6Q/+Th7S/+/wNiCP7uPDa1uthrFbx3wg +dz5aDakDOzwviliPlD2MFjehQIHHf0WwllzcTZXtHFVjsS45KABCNoT8F0tuFoqx +nxHHMOhnFHMOsi4hFtJzD0tNvoZBsE9zZKZoobkWWki4YuLlR5gUOlyS91Is7kRg +/YxODziWxxoNLp8MXCNVvokcSJTEnhZQMGqlpkoHpg9FcKdA4HVVHgtfeQXHVkud +XHdD943zOUOYFZQMoFti8fe4W+8KxFeyByicWETUz5En98z0FlaK/wfYMCkWcZm4 +HwK8Xu4cJQrCl4bO9fJEqUBeCPW2BI2DArR7eAODUxXCJ/7Cd9m1zEsB7MY8X8cK +mIeix/IjfgNyXlMQW8HLyDU521PaQQq9E0WLkhuQ2KbqSTDC70VaiILH7rpf6Hiu +BMull3Nay/WMlK/sGc+/QXz9rA+vuMVvWTMX9uyBvMNI1bZDEUOHMiUFOKphWinR +YJ9Qr9AX/w5Ew8y+Kwp7OZW+QSxZKAiWtjgBZqe8KytuYMUCWrBRjQTjb2SRCu+W +aAV4+0D+xrc07LDqC9G2vG90OS6wYLjkvKq3siweawKjNss/avQY7zEpalrcpZG5 +Ag0EV/7ceAEQAKAsmYHVSR0q2FC/7TZNUB1HKB08c+mRBBb1rrkiB6WwobzJHmur +8M0hSaOAHQsPVe9VMHkZfKLW3ohcBcn2IiRbwMaRcq4o/vACUzKzGL3BPpESksMG +Rm5pSI7UPbZovLLSs1wN43FYjfz5OkN6DWbwXqQPj1vIEIlr+hu4XCJ+FitpA5P9 +FEGOgIHupllyfx3dJePatHvKnmkl9jQaqnIwSWvz1DOrG9RCBOLe174BePiVMaTt +osEnLxsSsR/9PqbDvoH/jIrRWsfiZVd8kFRx5VmRlrHFUBxlQDH/zhrjVsH9TRFI +PKOo4XpeB/QS9TNdBe6nHEwYjIa0Bg/Ckb7BtCOs5Pcz5xY6Q3JRWc7PtuQAG3mj +tC8cc5wvK+yWPQFQHUTgB7McNmTg2CP6DttcZGnt+5+WUm+4kT4KnCYUMK301qBT +7cltRWKHuPrnDZQU0qp8pjGQxiRWWh2EQa5Jf4+Sww6Wn8YKLRhEsyDZg/DpvtrH +CSAU2XV5Ma8MjGNohz7zzY003ket0ywG401u8IQ482L4PwF/Ya/kRPLWqTr1jYQ/ +laGGfVBgIBxqgqmAdL0y31QHsWZvsg8a+QHuFQXFaRZXXhjyDZsdwcoCqfHJEGuF +murGfiIBVwTgEn1llnJm1XyYfJ/VHzbarTOSE5FGN2pIgNX6wlzCuJiXABEBAAGJ +Ah8EGAECAAkFAlf+3HgCGwwACgkQlBwUQ32E69Yo5Q/+O6Rh8w8fDZp6Ix87lhW7 +AyAeAnKJmKIpXHSAfDwznzrsptjMPsYWIdKZh/Oj7qkR/hx/dErK9m6qKZm5rGiy +b8JhvxYrgyozqqYC1ueYow/POaUeVUQNYGOc0tf+FFVBh89ml7+hOBzZjauEdXlj +0QTwjR39KYpWo6fudusZWRVv8thOlV/DQFMBa0uPQsivlb0P7yBqrkzQi7aHKfLo +rAhRpKtqsN3Uv3dpGrcBIwwMfobxwyTlYTPqQckv8nsTIVSGqYRZMSPRUDoNQgkt +6PW9Zn73wdqmQtJ2PekKz8jQMCxXxrbASF0UUfVaTt1ezmjqP7BiThGX3VQqcZM8 +qUb3l2nMSEhG26qln9pxHH5UMfVGVa0euPuKq/7eZZoKDwm1kw0gxByEV/g9Bm/6 +FY3WOK3fa3Z7otcZ/yt/EbU2/6qCMFactRXbJqH1gb1sdWIfK9PIysLJld3K2nIw +QrSl3kGaEBuL2gXBQdMTwliBb56gI8P+K3W9kgQCOR7m4w1UpT+OaJ9J0FriNz+b +pPDMQJgdQkaoj80wsIi7kWCy6lt2GuHbOLKT8CX0MfVCTYlCb4ubgCNPjru3eVWo +52ujPZY/uzwR2STGx3wf2n/dNSHXBsWybLo6agvSEhjCuT1dh/lmPCg1pZvlmKmq +Ry2jgybWqDPTomJ0jsgp7GA= +=h5+w +-END PGP PUBLIC KEY BLOCK-
[16/27] nifi git commit: Dummy commit to clean stalled gh PRs
Dummy commit to clean stalled gh PRs This closes: #315 (superseded) This closes: #1024 (superseded) Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/231f5143 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/231f5143 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/231f5143 Branch: refs/heads/appveyor-improvement Commit: 231f5143ab8100d6a5d4543a5476694378fdcf6d Parents: ae503f5 Author: Andre F de MirandaAuthored: Thu Oct 20 23:02:33 2016 +1100 Committer: Andre F de Miranda Committed: Thu Oct 20 23:02:33 2016 +1100 -- --
[25/27] nifi git commit: Making the TemporaryFolder public.
Making the TemporaryFolder public. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/c49243ee Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/c49243ee Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/c49243ee Branch: refs/heads/appveyor-improvement Commit: c49243eec970781244eb0555b1395a128c9af0d4 Parents: df61cde Author: Aldrin PiriAuthored: Fri Oct 14 12:09:31 2016 -0400 Committer: Aldrin Piri Committed: Fri Oct 21 15:54:33 2016 -0400 -- .../apache/nifi/provenance/TestPersistentProvenanceRepository.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/c49243ee/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java -- diff --git a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java b/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java index 7c8a6dc..4a0cd84 100644 --- a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java +++ b/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java @@ -99,7 +99,7 @@ public class TestPersistentProvenanceRepository { public TestName name = new TestName(); @Rule -TemporaryFolder testFolder = new TemporaryFolder(); +public TemporaryFolder testFolder = new TemporaryFolder(); private PersistentProvenanceRepository repo; private RepositoryConfiguration config;
[09/27] nifi git commit: NIFI-2500 This closes #1131. made container queue configurable
NIFI-2500 This closes #1131. made container queue configurable Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/5afbc433 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/5afbc433 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/5afbc433 Branch: refs/heads/appveyor-improvement Commit: 5afbc433642a51a40c22a2048cfa7d0e9cf0a86a Parents: 9fb3039 Author: Oleg ZhurakouskyAuthored: Thu Oct 13 12:08:29 2016 -0400 Committer: joewitt Committed: Tue Oct 18 14:47:49 2016 -0400 -- .../processors/standard/HandleHttpRequest.java | 26 ++-- 1 file changed, 18 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/5afbc433/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java -- diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java index 9f8edfc..c1acc85 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java @@ -219,19 +219,20 @@ public class HandleHttpRequest extends AbstractProcessor { .allowableValues(CLIENT_NONE, CLIENT_WANT, CLIENT_NEED) .defaultValue(CLIENT_NONE.getValue()) .build(); +public static final PropertyDescriptor CONTAINER_QUEUE_SIZE = new PropertyDescriptor.Builder() +.name("container-queue-size").displayName("Container Queue Size") +.description("The size of the queue for Http Request Containers").required(true) + .addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR).defaultValue("50").build(); public static final Relationship REL_SUCCESS = new Relationship.Builder() .name("success") .description("All content that is received is routed to the 'success' relationship") .build(); -private volatile Server server; -private AtomicBoolean initialized = new AtomicBoolean(false); -private final BlockingQueue containerQueue = new LinkedBlockingQueue<>(50); +private static final List propertyDescriptors; -@Override -protected List getSupportedPropertyDescriptors() { -final List descriptors = new ArrayList<>(); +static { +List descriptors = new ArrayList<>(); descriptors.add(PORT); descriptors.add(HOSTNAME); descriptors.add(SSL_CONTEXT); @@ -246,8 +247,17 @@ public class HandleHttpRequest extends AbstractProcessor { descriptors.add(ALLOW_OPTIONS); descriptors.add(ADDITIONAL_METHODS); descriptors.add(CLIENT_AUTH); +descriptors.add(CONTAINER_QUEUE_SIZE); +propertyDescriptors = Collections.unmodifiableList(descriptors); +} -return descriptors; +private volatile Server server; +private AtomicBoolean initialized = new AtomicBoolean(false); +private volatile BlockingQueue containerQueue; + +@Override +protected List getSupportedPropertyDescriptors() { +return propertyDescriptors; } @Override @@ -264,7 +274,7 @@ public class HandleHttpRequest extends AbstractProcessor { if(initialized.get()){ return; } - +this.containerQueue = new LinkedBlockingQueue<>(context.getProperty(CONTAINER_QUEUE_SIZE).asInteger()); final String host = context.getProperty(HOSTNAME).getValue(); final int port = context.getProperty(PORT).asInteger(); final SSLContextService sslService = context.getProperty(SSL_CONTEXT).asControllerService(SSLContextService.class);
[10/27] nifi git commit: [NIFI-2913] update z-index of codemirror resize handle to match the z-index of the codemirror vert. and horz. scrollbars
[NIFI-2913] update z-index of codemirror resize handle to match the z-index of the codemirror vert. and horz. scrollbars This closes #1145. Signed-off-by: Andy LoPrestoProject: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/f287bc91 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/f287bc91 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/f287bc91 Branch: refs/heads/appveyor-improvement Commit: f287bc9119c2f0d018329f53c85fb502b5c96f18 Parents: 5afbc43 Author: Scott Aslan Authored: Tue Oct 18 14:43:48 2016 -0400 Committer: Andy LoPresto Committed: Tue Oct 18 15:24:18 2016 -0700 -- .../src/main/webapp/js/jquery/nfeditor/jquery.nfeditor.css | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/f287bc91/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/nfeditor/jquery.nfeditor.css -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/nfeditor/jquery.nfeditor.css b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/nfeditor/jquery.nfeditor.css index 3c0a6d9..dcf1457 100644 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/nfeditor/jquery.nfeditor.css +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/nfeditor/jquery.nfeditor.css @@ -38,6 +38,7 @@ height: 16px; background-image: url(../../../images/iconResize.png); background-color: #fff; +z-index: 6; } /*
[11/27] nifi git commit: NIFI-2905: Log error stream of ExecuteProcess cmd
NIFI-2905: Log error stream of ExecuteProcess cmd ExecuteProcess ignores error stream when Redirect Error Stream is false, this commit let it to be logged instead so that user can see it on bulletin. This closes #1143. Signed-off-by: Andy LoPrestoProject: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/27dba60f Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/27dba60f Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/27dba60f Branch: refs/heads/appveyor-improvement Commit: 27dba60f27579ca9f33aa1afbf4bb46182d57df4 Parents: f287bc9 Author: Koji Kawamura Authored: Mon Oct 17 17:07:32 2016 +0900 Committer: Andy LoPresto Committed: Tue Oct 18 15:56:27 2016 -0700 -- .../processors/standard/ExecuteProcess.java | 3 +- .../processors/standard/TestExecuteProcess.java | 46 2 files changed, 47 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/27dba60f/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java -- diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java index 62860ed..c0d3f2b 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java @@ -316,8 +316,7 @@ public class ExecuteProcess extends AbstractProcessor { @Override public void run() { try (final BufferedReader reader = new BufferedReader(new InputStreamReader(externalProcess.getErrorStream( { -while (reader.read() >= 0) { -} +reader.lines().filter(line -> line != null && line.length() > 0).forEach(getLogger()::warn); } catch (final IOException ioe) { } } http://git-wip-us.apache.org/repos/asf/nifi/blob/27dba60f/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java -- diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java index 160bbdb..8af82f1 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java @@ -30,6 +30,7 @@ import java.util.concurrent.ExecutorService; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processors.standard.util.ArgumentUtils; +import org.apache.nifi.util.LogMessage; import org.apache.nifi.util.MockFlowFile; import org.apache.nifi.util.TestRunner; import org.apache.nifi.util.TestRunners; @@ -205,4 +206,49 @@ public class TestExecuteProcess { // assertEquals(inFile.length(), totalFlowFilesSize); } + +@Test +public void testNotRedirectErrorStream() { +final TestRunner runner = TestRunners.newTestRunner(ExecuteProcess.class); +runner.setProperty(ExecuteProcess.COMMAND, "cd"); +runner.setProperty(ExecuteProcess.COMMAND_ARGUMENTS, "does-not-exist"); + +ProcessContext processContext = runner.getProcessContext(); + +ExecuteProcess processor = (ExecuteProcess) runner.getProcessor(); +processor.updateScheduledTrue(); +processor.setupExecutor(processContext); + +processor.onTrigger(processContext, runner.getProcessSessionFactory()); + +final List warnMessages = runner.getLogger().getWarnMessages(); +assertEquals("If redirect error stream is false, " + +"the output should be logged as a warning so that user can notice on bulletin.", 1, warnMessages.size()); +final List succeeded = runner.getFlowFilesForRelationship(ExecuteProcess.REL_SUCCESS); +assertEquals(0, succeeded.size()); +} + + +@Test +public void
[27/27] nifi git commit: Removing MaxPermSize option.
Removing MaxPermSize option. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/af101186 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/af101186 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/af101186 Branch: refs/heads/appveyor-improvement Commit: af101186f55cabb458677c813032a0aa00fb3a23 Parents: 00d6fea Author: Aldrin PiriAuthored: Fri Oct 14 09:39:53 2016 -0400 Committer: Aldrin Piri Committed: Fri Oct 21 15:54:33 2016 -0400 -- appveyor.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/af101186/appveyor.yml -- diff --git a/appveyor.yml b/appveyor.yml index d54984f..b486557 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -29,8 +29,8 @@ install: } - cmd: SET M2_HOME=C:\maven\apache-maven-3.1.1 - cmd: SET PATH=%M2_HOME%\bin;%JAVA_HOME%\bin;%PATH%% - - cmd: SET MAVEN_OPTS=-XX:MaxPermSize=2g -Xmx4g - - cmd: SET JAVA_OPTS=-XX:MaxPermSize=2g -Xmx4g + - cmd: SET MAVEN_OPTS=-Xmx4g + - cmd: SET JAVA_OPTS=-Xmx4g build_script: - mvn -q -T 2C -DskipTests clean install test_script:
[18/27] nifi git commit: NIFI-2843: - Removing the View Configuration menu item from the context menu on Process Groups.
NIFI-2843: - Removing the View Configuration menu item from the context menu on Process Groups. This closes #1087 Signed-off-by: jpercivallProject: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/af27e407 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/af27e407 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/af27e407 Branch: refs/heads/appveyor-improvement Commit: af27e4070b6ec2fb4bb7acfe4c2fd573849c802c Parents: 99d3c39 Author: Matt Gilman Authored: Fri Sep 30 10:34:43 2016 -0400 Committer: jpercivall Committed: Thu Oct 20 12:45:37 2016 -0400 -- .../nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js | 3 --- 1 file changed, 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/af27e407/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js index 56b6a71..1fb262e 100644 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js @@ -967,9 +967,6 @@ nf.CanvasUtils = (function () { return false; } -if (nf.CanvasUtils.isProcessGroup(selection)) { -return true; -} if (nf.CanvasUtils.canRead(selection) === false) { return false; }
[19/27] nifi git commit: NIFI-2828: Fixed issue with transitive Hadoop dependencies in Hive NAR
NIFI-2828: Fixed issue with transitive Hadoop dependencies in Hive NAR This closes #1146 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/f5f132b1 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/f5f132b1 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/f5f132b1 Branch: refs/heads/appveyor-improvement Commit: f5f132b1abc63a0b3612e4efe6a3faa129c337e2 Parents: af27e40 Author: Matt BurgessAuthored: Tue Oct 18 16:22:15 2016 -0400 Committer: Oleg Zhurakousky Committed: Thu Oct 20 14:56:01 2016 -0400 -- .../nifi-hive-bundle/nifi-hive-processors/pom.xml | 7 +++ nifi-nar-bundles/nifi-hive-bundle/pom.xml | 5 + pom.xml | 1 + 3 files changed, 13 insertions(+) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/f5f132b1/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/pom.xml -- diff --git a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/pom.xml b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/pom.xml index 3d07b86..eebdcbf 100644 --- a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/pom.xml +++ b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/pom.xml @@ -58,6 +58,7 @@ org.apache.hadoop hadoop-client +${hive.hadoop.version} com.google.code.findbugs @@ -68,6 +69,12 @@ org.apache.nifi nifi-hadoop-utils + + +org.apache.hadoop +hadoop-client + + com.github.stephenc.findbugs http://git-wip-us.apache.org/repos/asf/nifi/blob/f5f132b1/nifi-nar-bundles/nifi-hive-bundle/pom.xml -- diff --git a/nifi-nar-bundles/nifi-hive-bundle/pom.xml b/nifi-nar-bundles/nifi-hive-bundle/pom.xml index 5ab10e8..0186740 100644 --- a/nifi-nar-bundles/nifi-hive-bundle/pom.xml +++ b/nifi-nar-bundles/nifi-hive-bundle/pom.xml @@ -26,6 +26,11 @@ 1.1.0-SNAPSHOT pom + + +${hive.hadoop.version} + + nifi-hive-processors nifi-hive-nar http://git-wip-us.apache.org/repos/asf/nifi/blob/f5f132b1/pom.xml -- diff --git a/pom.xml b/pom.xml index 735a9ec..7cf41b4 100644 --- a/pom.xml +++ b/pom.xml @@ -101,6 +101,7 @@ language governing permissions and limitations under the License. --> 2.2.0 0.6.0 1.2.1 +2.6.2 1.1.2 1.0.1
[03/27] nifi git commit: NIFI-2897: Fixed SelectHiveQL for CSV output of complex types
NIFI-2897: Fixed SelectHiveQL for CSV output of complex types This closes #1132 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/b52b8398 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/b52b8398 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/b52b8398 Branch: refs/heads/appveyor-improvement Commit: b52b8398957dcb60b93ea5f697f45d3e0e212082 Parents: 88d1251 Author: Matt BurgessAuthored: Thu Oct 13 13:27:49 2016 -0400 Committer: Oleg Zhurakousky Committed: Fri Oct 14 12:35:38 2016 -0400 -- .../java/org/apache/nifi/util/hive/HiveJdbcCommon.java| 10 ++ 1 file changed, 10 insertions(+) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/b52b8398/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveJdbcCommon.java -- diff --git a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveJdbcCommon.java b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveJdbcCommon.java index d4b2945..689baf9 100644 --- a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveJdbcCommon.java +++ b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/util/hive/HiveJdbcCommon.java @@ -339,6 +339,16 @@ public class HiveJdbcCommon { rowValues.add(""); } break; +case ARRAY: +case STRUCT: +case JAVA_OBJECT: +String complexValueString = rs.getString(i); +if (complexValueString != null) { + rowValues.add(StringEscapeUtils.escapeCsv(complexValueString)); +} else { +rowValues.add(""); +} +break; default: if (value != null) { rowValues.add(value.toString());
[17/27] nifi git commit: NIFI-1069 - improve init script exit codes so that results are LSB compliant
NIFI-1069 - improve init script exit codes so that results are LSB compliant This closes: #1093 Signed-off-by: Andre F de MirandaProject: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/99d3c397 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/99d3c397 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/99d3c397 Branch: refs/heads/appveyor-improvement Commit: 99d3c397481a98a5b3d2d4e44b84516fc0e3675e Parents: 231f514 Author: Michal Klempa Authored: Tue Oct 4 14:25:49 2016 +0200 Committer: Andre F de Miranda Committed: Thu Oct 20 23:12:23 2016 +1100 -- .../main/java/org/apache/nifi/bootstrap/RunNiFi.java | 15 ++- .../nifi-resources/src/main/resources/bin/nifi.sh| 2 ++ 2 files changed, 12 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/99d3c397/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java -- diff --git a/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java b/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java index 04e7ba3..8d92c44 100644 --- a/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java +++ b/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java @@ -204,6 +204,7 @@ public class RunNiFi { final File configFile = getBootstrapConfFile(); final RunNiFi runNiFi = new RunNiFi(configFile, verbose); +Integer exitStatus = null; switch (cmd.toLowerCase()) { case "start": runNiFi.start(); @@ -215,7 +216,7 @@ public class RunNiFi { runNiFi.stop(); break; case "status": -runNiFi.status(); +exitStatus = runNiFi.status(); break; case "restart": runNiFi.stop(); @@ -228,6 +229,9 @@ public class RunNiFi { runNiFi.env(); break; } +if (exitStatus != null) { +System.exit(exitStatus); +} } private static File getBootstrapConfFile() { @@ -581,23 +585,23 @@ public class RunNiFi { return new Status(port, pid, pingSuccess, alive); } -public void status() throws IOException { +public int status() throws IOException { final Logger logger = cmdLogger; final Status status = getStatus(logger); if (status.isRespondingToPing()) { logger.info("Apache NiFi is currently running, listening to Bootstrap on port {}, PID={}", new Object[]{status.getPort(), status.getPid() == null ? "unknown" : status.getPid()}); -return; +return 0; } if (status.isProcessRunning()) { logger.info("Apache NiFi is running at PID {} but is not responding to ping requests", status.getPid()); -return; +return 4; } if (status.getPort() == null) { logger.info("Apache NiFi is not running"); -return; +return 3; } if (status.getPid() == null) { @@ -605,6 +609,7 @@ public class RunNiFi { } else { logger.info("Apache NiFi is not running"); } +return 3; } public void env() { http://git-wip-us.apache.org/repos/asf/nifi/blob/99d3c397/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh index ff8addd..7b471d7 100755 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh @@ -295,12 +295,14 @@ run() { else (eval $RUN_NIFI_CMD $@) fi +EXIT_STATUS=$? # Wait just a bit (3 secs) to wait for the logging to finish and then echo a new-line. # We do this to avoid having logs spewed on the console after running the command and then not giving # control back to the user sleep 3 echo +exit $EXIT_STATUS } main() {
[14/27] nifi git commit: NIFI-2919 Improved GetFile processor to fail (and provide bulletins) if target directory is inaccessible.
NIFI-2919 Improved GetFile processor to fail (and provide bulletins) if target directory is inaccessible. This closes #1147. Signed-off-by: Andy LoPrestoFixed typos in error messages, renamed variables in test, and cleaned up unnecessary imports. (+1 squashed commit) Squashed commits: [e755cbd] NIFI-2919 improved GetFile to fail if target directory is inaccessible Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/611cadd2 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/611cadd2 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/611cadd2 Branch: refs/heads/appveyor-improvement Commit: 611cadd231309126a0a7737391f6e07730bb3864 Parents: a8e1c77 Author: Oleg Zhurakousky Authored: Wed Oct 19 09:30:12 2016 -0400 Committer: Andy LoPresto Committed: Wed Oct 19 16:33:33 2016 -0400 -- .../nifi/processors/standard/GetFile.java | 9 +- .../nifi/processors/standard/TestGetFile.java | 101 +++ 2 files changed, 87 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/611cadd2/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetFile.java -- diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetFile.java b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetFile.java index 45c95eb..cdfb857 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetFile.java +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetFile.java @@ -48,7 +48,6 @@ import java.util.concurrent.atomic.AtomicReference; import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; import java.util.regex.Pattern; - import org.apache.nifi.annotation.behavior.InputRequirement; import org.apache.nifi.annotation.behavior.InputRequirement.Requirement; import org.apache.nifi.annotation.behavior.TriggerWhenEmpty; @@ -295,14 +294,14 @@ public class GetFile extends AbstractProcessor { } private Set performListing(final File directory, final FileFilter filter, final boolean recurseSubdirectories) { +Path p = directory.toPath(); +if (!Files.isWritable(p) || !Files.isReadable(p)) { +throw new IllegalStateException("Directory '" + directory + "' does not have sufficient permissions (i.e., not writable and readable)"); +} final Set queue = new HashSet<>(); if (!directory.exists()) { return queue; } -// this check doesn't work on Windows -if (!directory.canRead()) { -getLogger().warn("No read permission on directory {}", new Object[]{directory.toString()}); -} final File[] children = directory.listFiles(); if (children == null) { http://git-wip-us.apache.org/repos/asf/nifi/blob/611cadd2/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestGetFile.java -- diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestGetFile.java b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestGetFile.java index eb8a764..daf807a 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestGetFile.java +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestGetFile.java @@ -16,8 +16,6 @@ */ package org.apache.nifi.processors.standard; -import org.apache.nifi.processors.standard.GetFile; - import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; @@ -32,10 +30,10 @@ import java.text.DateFormat; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date; +import java.util.HashSet; import java.util.List; import java.util.Locale; import java.util.Set; - import org.apache.nifi.flowfile.attributes.CoreAttributes; import org.apache.nifi.util.MockFlowFile; import org.apache.nifi.util.TestRunner; @@ -45,6 +43,72 @@ import org.junit.Test; public class TestGetFile { @Test +public void testWithInaccessibleDir() throws IOException { +File
[21/27] nifi git commit: NIFI-2341 - Introduce ParseCEF processor
NIFI-2341 - Introduce ParseCEF processor This closes #785 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/b864d49f Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/b864d49f Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/b864d49f Branch: refs/heads/appveyor-improvement Commit: b864d49f66692dabeb805965a08e83e14d27fd58 Parents: 9f6fb61 Author: Andre F de MirandaAuthored: Thu Aug 4 23:30:37 2016 +1000 Committer: Matt Burgess Committed: Thu Oct 20 17:16:57 2016 -0400 -- nifi-assembly/NOTICE| 5 + .../src/main/resources/META-INF/NOTICE | 6 + .../nifi-standard-processors/pom.xml| 11 + .../nifi/processors/standard/ParseCEF.java | 327 +++ .../org.apache.nifi.processor.Processor | 1 + .../nifi/processors/standard/TestParseCEF.java | 244 ++ 6 files changed, 594 insertions(+) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/b864d49f/nifi-assembly/NOTICE -- diff --git a/nifi-assembly/NOTICE b/nifi-assembly/NOTICE index 794a9d5..9d54885 100644 --- a/nifi-assembly/NOTICE +++ b/nifi-assembly/NOTICE @@ -1013,6 +1013,11 @@ The following binary components are provided under the Apache Software License v Expert Group and released to the public domain, as explained at http://creativecommons.org/publicdomain/zero/1.0/ + (ASLv2) ParCEFone +The following NOTICE information applies: + ParCEFone + Copyright 2016 Fluenda + This includes derived works from the Apache Software License V2 library python-evtx (https://github.com/williballenthin/python-evtx) Copyright 2012, 2013 Willi Ballenthin william.ballent...@mandiant.com while at Mandiant http://www.mandiant.com http://git-wip-us.apache.org/repos/asf/nifi/blob/b864d49f/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-nar/src/main/resources/META-INF/NOTICE -- diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-nar/src/main/resources/META-INF/NOTICE b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-nar/src/main/resources/META-INF/NOTICE index 205d9ec..0887920 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-nar/src/main/resources/META-INF/NOTICE +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-nar/src/main/resources/META-INF/NOTICE @@ -164,6 +164,11 @@ The following binary components are provided under the Apache Software License v "GCC RUntime Library Exception" http://gcc.gnu.org/onlinedocs/libstdc++/manual/license.html + (ASLv2) ParCEFone +The following NOTICE information applies: + ParCEFone + Copyright 2016 Fluenda + Common Development and Distribution License 1.1 @@ -175,6 +180,7 @@ The following binary components are provided under the Common Development and Di (CDDL 1.1) (GPL2 w/ CPE) jersey-server (com.sun.jersey:jersey-server:jar:1.19 - https://jersey.java.net/jersey-server/) (CDDL 1.1) (GPL2 w/ CPE) JavaMail API (compat) (javax.mail:mail:jar:1.4.7 - http://kenai.com/projects/javamail/mail) (CDDL 1.1) (GPL2 w/ CPE) Javax JMS Api (javax.jms:javax.jms-api:jar:2.0.1 - http://java.net/projects/jms-spec/pages/Home) +(CDDL 1.1) (GPL2 w/ CPE) Expression Language 3.0 API (javax.el:javax.el-api:jar:3.0.0 - http://uel-spec.java.net) * Common Development and Distribution License v1.0: http://git-wip-us.apache.org/repos/asf/nifi/blob/b864d49f/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml -- diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml index b3bea5f..cce9c57 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml @@ -254,6 +254,17 @@ language governing permissions and limitations under the License. --> org.everit.json.schema 1.4.0 + +com.fluenda +ParCEFone +1.2.0 + + +org.slf4j +slf4j-log4j12 + + + http://git-wip-us.apache.org/repos/asf/nifi/blob/b864d49f/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ParseCEF.java
[05/27] nifi git commit: NIFI-2894: Fixed typo in PutSQL documentation
NIFI-2894: Fixed typo in PutSQL documentation This closes #1129 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/596b9886 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/596b9886 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/596b9886 Branch: refs/heads/appveyor-improvement Commit: 596b98865b1c3526072ea120f86f0ff9627f82a8 Parents: d63e675 Author: Matt BurgessAuthored: Thu Oct 13 09:19:08 2016 -0400 Committer: Oleg Zhurakousky Committed: Fri Oct 14 12:45:34 2016 -0400 -- .../src/main/java/org/apache/nifi/processors/standard/PutSQL.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/596b9886/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutSQL.java -- diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutSQL.java b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutSQL.java index eb27d40..adfac05 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutSQL.java +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutSQL.java @@ -116,7 +116,7 @@ public class PutSQL extends AbstractProcessor { static final PropertyDescriptor SUPPORT_TRANSACTIONS = new PropertyDescriptor.Builder() .name("Support Fragmented Transactions") .description("If true, when a FlowFile is consumed by this Processor, the Processor will first check the fragment.identifier and fragment.count attributes of that FlowFile. " -+ "If the fragment.count value is greater than 1, the Processor will not process any FlowFile will that fragment.identifier until all are available; " ++ "If the fragment.count value is greater than 1, the Processor will not process any FlowFile with that fragment.identifier until all are available; " + "at that point, it will process all FlowFiles with that fragment.identifier as a single transaction, in the order specified by the FlowFiles' fragment.index attributes. " + "This Provides atomicity of those SQL statements. If this value is false, these attributes will be ignored and the updates will occur independent of one another.") .allowableValues("true", "false")
[22/27] nifi git commit: Dummy commit to clean stalled gh PRs
Dummy commit to clean stalled gh PRs This closes: #513 (superseded) Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/220b7680 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/220b7680 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/220b7680 Branch: refs/heads/appveyor-improvement Commit: 220b76803b2f70ddd6dcc366478d722599349983 Parents: b864d49 Author: Andre F de MirandaAuthored: Thu Oct 20 23:58:39 2016 +1100 Committer: Andre F de Miranda Committed: Fri Oct 21 23:17:44 2016 +1100 -- --
[20/27] nifi git commit: NIFI-2685 - Configure surefire to use JAVA Headless mode
NIFI-2685 - Configure surefire to use JAVA Headless mode This closes #961 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/9f6fb617 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/9f6fb617 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/9f6fb617 Branch: refs/heads/appveyor-improvement Commit: 9f6fb617ba440a77cc0e44dcf38ab0ce09a7a15c Parents: f5f132b Author: Andre F de MirandaAuthored: Fri Aug 26 17:04:17 2016 +1000 Committer: Matt Burgess Committed: Thu Oct 20 16:46:54 2016 -0400 -- pom.xml | 3 +++ 1 file changed, 3 insertions(+) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/9f6fb617/pom.xml -- diff --git a/pom.xml b/pom.xml index 7cf41b4..11bcd36 100644 --- a/pom.xml +++ b/pom.xml @@ -1435,6 +1435,9 @@ language governing permissions and limitations under the License. --> maven-surefire-plugin 2.18 + +true + **/*Test.class **/Test*.class
[26/27] nifi git commit: Updating appveyor to mimic Travis CI build.
Updating appveyor to mimic Travis CI build. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/00d6feac Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/00d6feac Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/00d6feac Branch: refs/heads/appveyor-improvement Commit: 00d6feacd1e8f479e395263b1f1ee7abdfdee683 Parents: 220b768 Author: Aldrin PiriAuthored: Fri Oct 14 09:22:06 2016 -0400 Committer: Aldrin Piri Committed: Fri Oct 21 15:54:33 2016 -0400 -- appveyor.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/00d6feac/appveyor.yml -- diff --git a/appveyor.yml b/appveyor.yml index c7aa2f2..d54984f 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -32,9 +32,9 @@ install: - cmd: SET MAVEN_OPTS=-XX:MaxPermSize=2g -Xmx4g - cmd: SET JAVA_OPTS=-XX:MaxPermSize=2g -Xmx4g build_script: - - mvn -q clean package --batch-mode -DskipTests + - mvn -q -T 2C -DskipTests clean install test_script: - - mvn -q clean install --batch-mode -Pcontrib-check + - mvn -P contrib-check verify cache: - C:\maven\ - C:\Users\appveyor\.m2
[13/27] nifi git commit: [NIFI-1459] add css translate3d to properties table editors in order to move the element along the z-axis of the 3D space and allow the scrollbars to properly be poistioned. T
[NIFI-1459] add css translate3d to properties table editors in order to move the element along the z-axis of the 3D space and allow the scrollbars to properly be poistioned. This closes #1070 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/a8e1c775 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/a8e1c775 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/a8e1c775 Branch: refs/heads/appveyor-improvement Commit: a8e1c775fd95cfb73ab5453a8e7ec7c1d3e03b98 Parents: 3914141 Author: Scott AslanAuthored: Wed Oct 19 11:57:29 2016 -0400 Committer: Matt Gilman Committed: Wed Oct 19 15:39:58 2016 -0400 -- .../js/jquery/propertytable/jquery.propertytable.js | 12 +--- .../src/main/webapp/js/application.js | 6 -- 2 files changed, 13 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/a8e1c775/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/propertytable/jquery.propertytable.js -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/propertytable/jquery.propertytable.js b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/propertytable/jquery.propertytable.js index 47231f0..246129c 100644 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/propertytable/jquery.propertytable.js +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/propertytable/jquery.propertytable.js @@ -77,7 +77,8 @@ 'background-color': 'rgb(255, 255, 255)', 'overflow': 'hidden', 'padding': '10px 20px', -'cursor': 'move' +'cursor': 'move', +'transform': 'translate3d(0px, 0px, 0px)' }).appendTo(container); // create the input field @@ -301,7 +302,8 @@ 'border-radius': '2px', 'box-shadow': 'rgba(0, 0, 0, 0.247059) 0px 2px 5px', 'background-color': 'rgb(255, 255, 255)', -'cursor': 'move' +'cursor': 'move', +'transform': 'translate3d(0px, 0px, 0px)' }).draggable({ cancel: 'input, textarea, pre, .nf-checkbox, .button, .' + editorClass, containment: 'parent' @@ -506,7 +508,8 @@ 'border-radius': '2px', 'box-shadow': 'rgba(0, 0, 0, 0.247059) 0px 2px 5px', 'background-color': 'rgb(255, 255, 255)', -'cursor': 'move' +'cursor': 'move', +'transform': 'translate3d(0px, 0px, 0px)' }).draggable({ cancel: '.button, .combo', containment: 'parent' @@ -723,6 +726,7 @@ 'box-shadow': 'rgba(0, 0, 0, 0.247059) 0px 2px 5px', 'background-color': 'rgb(255, 255, 255)', 'cursor': 'move', +'transform': 'translate3d(0px, 0px, 0px)', 'top': offset.top - 24, 'left': offset.left - 20 }).appendTo('body'); @@ -805,6 +809,7 @@ 'box-shadow': 'rgba(0, 0, 0, 0.247059) 0px 2px 5px', 'background-color': 'rgb(255, 255, 255)', 'cursor': 'move', +'transform': 'translate3d(0px, 0px, 0px)', 'top': offset.top - 22, 'left': offset.left - 43 }).draggable({ @@ -835,6 +840,7 @@ 'box-shadow': 'rgba(0, 0, 0, 0.247059) 0px 2px 5px', 'background-color': 'rgb(255, 255, 255)', 'cursor': 'move', +'transform': 'translate3d(0px, 0px, 0px)', 'top': offset.top - 26, 'left': offset.left - 20 }); http://git-wip-us.apache.org/repos/asf/nifi/blob/a8e1c775/nifi-nar-bundles/nifi-update-attribute-bundle/nifi-update-attribute-ui/src/main/webapp/js/application.js -- diff --git a/nifi-nar-bundles/nifi-update-attribute-bundle/nifi-update-attribute-ui/src/main/webapp/js/application.js b/nifi-nar-bundles/nifi-update-attribute-bundle/nifi-update-attribute-ui/src/main/webapp/js/application.js index 1963f3d..a7984d7 100644 ---
[07/27] nifi git commit: NIFI-2902 Fix for S3 Signer v4 Override
NIFI-2902 Fix for S3 Signer v4 Override This closes #1140. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/b9a940bb Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/b9a940bb Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/b9a940bb Branch: refs/heads/appveyor-improvement Commit: b9a940bbd1c7afede946e2e7db948cc38582208f Parents: c883f98 Author: James WingAuthored: Sat Oct 15 12:49:15 2016 -0700 Committer: Pierre Villard Committed: Mon Oct 17 22:51:45 2016 +0200 -- .../processors/aws/s3/AbstractS3Processor.java | 2 +- .../nifi/processors/aws/s3/TestPutS3Object.java | 64 2 files changed, 65 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/b9a940bb/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java -- diff --git a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java index 2f3fde8..b96c05e 100644 --- a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java +++ b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java @@ -121,7 +121,7 @@ public abstract class AbstractS3Processor extends AbstractAWSCredentialsProvider .required(false) .allowableValues( new AllowableValue("Default Signature", "Default Signature"), -new AllowableValue("AWSS3V4Signer", "Signature v4"), +new AllowableValue("AWSS3V4SignerType", "Signature v4"), new AllowableValue("S3SignerType", "Signature v2")) .defaultValue("Default Signature") .build(); http://git-wip-us.apache.org/repos/asf/nifi/blob/b9a940bb/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/TestPutS3Object.java -- diff --git a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/TestPutS3Object.java b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/TestPutS3Object.java new file mode 100644 index 000..00f4423 --- /dev/null +++ b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/s3/TestPutS3Object.java @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.aws.s3; + +import java.util.List; + +import org.apache.nifi.components.AllowableValue; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.util.TestRunner; +import org.apache.nifi.util.TestRunners; + +import com.amazonaws.ClientConfiguration; +import com.amazonaws.auth.AWSCredentialsProvider; +import com.amazonaws.auth.DefaultAWSCredentialsProviderChain; +import com.amazonaws.services.s3.AmazonS3Client; + +import org.junit.Assert; +import org.junit.Test; + + +/** + * Unit tests for {@link PutS3Object}, without interaction with S3. + */ +public class TestPutS3Object { + +@Test +public void testSignerOverrideOptions() { +final AWSCredentialsProvider credentialsProvider = new DefaultAWSCredentialsProviderChain(); +final ClientConfiguration config = new ClientConfiguration(); +final PutS3Object processor = new PutS3Object(); +final TestRunner runner = TestRunners.newTestRunner(processor); + +final List allowableSignerValues = PutS3Object.SIGNER_OVERRIDE.getAllowableValues(); +final String defaultSignerValue =
[06/27] nifi git commit: NIFI-2906 removed unused memebr variables
NIFI-2906 removed unused memebr variables This closes #1144. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/c883f98c Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/c883f98c Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/c883f98c Branch: refs/heads/appveyor-improvement Commit: c883f98cbf803e5d8982a2a811c3ca46c5c249ca Parents: 596b988 Author: Oleg ZhurakouskyAuthored: Mon Oct 17 11:54:58 2016 -0400 Committer: Pierre Villard Committed: Mon Oct 17 22:32:23 2016 +0200 -- .../main/java/org/apache/nifi/processors/standard/ReplaceText.java | 2 -- 1 file changed, 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/c883f98c/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java -- diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java index 9b20ec6..59721f0 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java @@ -210,8 +210,6 @@ public class ReplaceText extends AbstractProcessor { final ComponentLog logger = getLogger(); -final String unsubstitutedRegex = context.getProperty(SEARCH_VALUE).getValue(); -String unsubstitutedReplacement = context.getProperty(REPLACEMENT_VALUE).getValue(); final String replacementStrategy = context.getProperty(REPLACEMENT_STRATEGY).getValue(); final Charset charset = Charset.forName(context.getProperty(CHARACTER_SET).getValue());
[02/27] nifi git commit: NIFI-2199 - allows nifi.sh restart through ssh
NIFI-2199 - allows nifi.sh restart through ssh Closes: #1092 Signed-off-by: Andre F de MirandaProject: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/88d12513 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/88d12513 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/88d12513 Branch: refs/heads/appveyor-improvement Commit: 88d125137e3bf3faf80b522edf5cf14f99419d60 Parents: e969a5f Author: Michal Klempa Authored: Tue Oct 4 12:18:46 2016 +0200 Committer: Andre F de Miranda Committed: Sat Oct 15 00:48:04 2016 +1100 -- .../nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/88d12513/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh index b58a622..79850ee 100755 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh @@ -267,7 +267,7 @@ run() { RUN_NIFI_CMD="cd "\""${NIFI_HOME}"\"" && ${sudo_cmd_prefix} "\""${JAVA}"\"" -cp "\""${BOOTSTRAP_CLASSPATH}"\"" -Xms12m -Xmx24m ${BOOTSTRAP_DIR_PARAMS} org.apache.nifi.bootstrap.RunNiFi" if [ "$1" = "start" ]; then -(eval $RUN_NIFI_CMD $@ &) +(eval $RUN_NIFI_CMD $@ &)> /dev/null 2>&1 < /dev/null else (eval $RUN_NIFI_CMD $@) fi
[24/27] nifi git commit: Remove repo cleanup in favor of delegation to JUnit TemporaryFolder.
Remove repo cleanup in favor of delegation to JUnit TemporaryFolder. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/81313dda Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/81313dda Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/81313dda Branch: refs/heads/appveyor-improvement Commit: 81313dda6cfa9708aad70c653fc96646eeeb89fd Parents: c49243e Author: Aldrin PiriAuthored: Fri Oct 14 13:37:26 2016 -0400 Committer: Aldrin Piri Committed: Fri Oct 21 15:54:33 2016 -0400 -- .../TestPersistentProvenanceRepository.java | 24 1 file changed, 24 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/81313dda/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java -- diff --git a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java b/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java index 4a0cd84..0fcbacd 100644 --- a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java +++ b/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/test/java/org/apache/nifi/provenance/TestPersistentProvenanceRepository.java @@ -146,30 +146,6 @@ public class TestPersistentProvenanceRepository { } catch (final IOException ioe) { } } - -// Delete all of the storage files. We do this in order to clean up the tons of files that -// we create but also to ensure that we have closed all of the file handles. If we leave any -// streams open, for instance, this will throw an IOException, causing our unit test to fail. -for (final File storageDir : config.getStorageDirectories()) { -int i; -for (i = 0; i < 3; i++) { -try { -FileUtils.deleteFile(storageDir, true); -break; -} catch (final IOException ioe) { -// if there is a virus scanner, etc. running in the background we may not be able to -// delete the file. Wait a sec and try again. -if (i == 2) { -throw ioe; -} else { -try { -Thread.sleep(1000L); -} catch (final InterruptedException ie) { -} -} -} -} -} }
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596099#comment-15596099 ] ASF GitHub Bot commented on MINIFI-117: --- Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84531003 --- Diff: minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/ConfigSchema.java --- @@ -142,128 +125,10 @@ public ConfigSchema(Map map) { } } -protected List getProcessorSchemas(List processorMaps) { -if (processorMaps == null) { -return null; -} -List processors = convertListToType(processorMaps, "processor", ProcessorSchema.class, PROCESSORS_KEY); - -MapidMap = processors.stream().map(ProcessorSchema::getId).filter( -s -> !StringUtil.isNullOrEmpty(s)).collect(Collectors.toMap(Function.identity(), s -> 2, Integer::compareTo)); - -// Set unset ids -processors.stream().filter(connection -> StringUtil.isNullOrEmpty(connection.getId())).forEachOrdered(processor -> processor.setId(getUniqueId(idMap, processor.getName(; - -return processors; -} - -protected List getConnectionSchemas(List connectionMaps) { -if (connectionMaps == null) { -return null; -} -List connections = convertListToType(connectionMaps, "connection", ConnectionSchema.class, CONNECTIONS_KEY); -Map idMap = connections.stream().map(ConnectionSchema::getId).filter( -s -> !StringUtil.isNullOrEmpty(s)).collect(Collectors.toMap(Function.identity(), s -> 2, Integer::compareTo)); - -Map processorNameToIdMap = new HashMap<>(); - -// We can't look up id by name for names that appear more than once -Set duplicateProcessorNames = new HashSet<>(); - -List processors = getProcessors(); -if (processors != null) { -processors.stream().forEachOrdered(p -> processorNameToIdMap.put(p.getName(), p.getId())); - -Set processorNames = new HashSet<>(); - processors.stream().map(ProcessorSchema::getName).forEachOrdered(n -> { -if (!processorNames.add(n)) { -duplicateProcessorNames.add(n); -} -}); -} - -Set remoteInputPortIds = new HashSet<>(); -List remoteProcessingGroups = getRemoteProcessingGroups(); -if (remoteProcessingGroups != null) { - remoteInputPortIds.addAll(remoteProcessingGroups.stream().filter(r -> r.getInputPorts() != null) -.flatMap(r -> r.getInputPorts().stream()).map(RemoteInputPortSchema::getId).collect(Collectors.toSet())); -} - -Set problematicDuplicateNames = new HashSet<>(); -Set missingProcessorNames = new HashSet<>(); -// Set unset ids -connections.stream().filter(connection -> StringUtil.isNullOrEmpty(connection.getId())).forEachOrdered(connection -> connection.setId(getUniqueId(idMap, connection.getName(; - -connections.stream().filter(connection -> StringUtil.isNullOrEmpty(connection.getSourceId())).forEach(connection -> { -String sourceName = connection.getSourceName(); -if (remoteInputPortIds.contains(sourceName)) { -connection.setSourceId(sourceName); -} else { -if (duplicateProcessorNames.contains(sourceName)) { -problematicDuplicateNames.add(sourceName); -} -String sourceId = processorNameToIdMap.get(sourceName); -if (StringUtil.isNullOrEmpty(sourceId)) { -missingProcessorNames.add(sourceName); -} else { -connection.setSourceId(sourceId); -} -} -}); - -connections.stream().filter(connection -> StringUtil.isNullOrEmpty(connection.getDestinationId())) -.forEach(connection -> { -String destinationName = connection.getDestinationName(); -if (remoteInputPortIds.contains(destinationName)) { -connection.setDestinationId(destinationName); -} else { -if (duplicateProcessorNames.contains(destinationName)) { -problematicDuplicateNames.add(destinationName); -} -String destinationId =
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596088#comment-15596088 ] ASF GitHub Bot commented on MINIFI-117: --- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84530471 --- Diff: minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/ConnectionSchema.java --- @@ -60,30 +53,11 @@ public ConnectionSchema(Map map) { super(map, CONNECTIONS_KEY); sourceId = getOptionalKeyAsType(map, SOURCE_ID_KEY, String.class, CONNECTIONS_KEY, ""); --- End diff -- Ah I see, could you just add a comment above these two "optional" calls explaining that? > Maintainable Configuration Versioning > - > > Key: MINIFI-117 > URL: https://issues.apache.org/jira/browse/MINIFI-117 > Project: Apache NiFi MiNiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander > > In order to avoid a tangled web of if/else statements around every possible > permutation of config.yml and an equally complicated validation routine, we > need to utilize a versioning strategy that allows for specific parsing and > validation code for each version of the ConfigSchema. > This will allow us to determine if a schema will be compatible with an older > MiNiFi instance more easily by validating the structure of the file at a > given version while still supporting an upconvert on read to the latest > ConfigSchema while parsing at runtime. > It should also facilitate a deprecation strategy of removing older versions' > classes without needing to touch current implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596074#comment-15596074 ] ASF GitHub Bot commented on MINIFI-117: --- Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84529673 --- Diff: minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/ConnectionSchema.java --- @@ -60,30 +53,11 @@ public ConnectionSchema(Map map) { super(map, CONNECTIONS_KEY); sourceId = getOptionalKeyAsType(map, SOURCE_ID_KEY, String.class, CONNECTIONS_KEY, ""); --- End diff -- It is treated as required in the getValidationIssues implementation. I needed a chance to set it when upconverting before the validation occurs. I could change that to map manipulation before instantiating the ConnectionSchema but that felt worse. > Maintainable Configuration Versioning > - > > Key: MINIFI-117 > URL: https://issues.apache.org/jira/browse/MINIFI-117 > Project: Apache NiFi MiNiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander > > In order to avoid a tangled web of if/else statements around every possible > permutation of config.yml and an equally complicated validation routine, we > need to utilize a versioning strategy that allows for specific parsing and > validation code for each version of the ConfigSchema. > This will allow us to determine if a schema will be compatible with an older > MiNiFi instance more easily by validating the structure of the file at a > given version while still supporting an upconvert on read to the latest > ConfigSchema while parsing at runtime. > It should also facilitate a deprecation strategy of removing older versions' > classes without needing to touch current implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596075#comment-15596075 ] ASF GitHub Bot commented on MINIFI-117: --- Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84529741 --- Diff: minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/serialization/SchemaLoader.java --- @@ -30,13 +32,13 @@ import java.util.stream.Collectors; public class SchemaLoader { -private static final Map> configSchemaFactories = initConfigSchemaFactories(); +private static final Map > configSchemaFactories = initConfigSchemaFactories(); -private static Map > initConfigSchemaFactories() { -Map > result = new HashMap<>(); -result.put(String.valueOf((Object)null), ConfigSchema::new); -result.put("", ConfigSchema::new); -result.put("1", ConfigSchema::new); --- End diff -- Good point, I'll take a stab at updating the doc > Maintainable Configuration Versioning > - > > Key: MINIFI-117 > URL: https://issues.apache.org/jira/browse/MINIFI-117 > Project: Apache NiFi MiNiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander > > In order to avoid a tangled web of if/else statements around every possible > permutation of config.yml and an equally complicated validation routine, we > need to utilize a versioning strategy that allows for specific parsing and > validation code for each version of the ConfigSchema. > This will allow us to determine if a schema will be compatible with an older > MiNiFi instance more easily by validating the structure of the file at a > given version while still supporting an upconvert on read to the latest > ConfigSchema while parsing at runtime. > It should also facilitate a deprecation strategy of removing older versions' > classes without needing to touch current implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596069#comment-15596069 ] ASF GitHub Bot commented on MINIFI-117: --- Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84529455 --- Diff: minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/ConfigMain.java --- @@ -96,13 +99,21 @@ public int validate(String[] args) { } try (InputStream inputStream = pathInputStreamFactory.create(args[1])) { try { -ConfigSchema configSchema = SchemaLoader.loadConfigSchemaFromYaml(inputStream); +ConvertableSchema configSchema = SchemaLoader.loadConvertableSchemaFromYaml(inputStream); if (!configSchema.isValid()) { configSchema.getValidationIssues().forEach(s -> System.out.println(s)); System.out.println(); return ERR_INVALID_CONFIG; } else { - System.out.println(NO_VALIDATION_ERRORS_FOUND_IN_TEMPLATE); +ConfigSchema currentSchema = configSchema.convert(); --- End diff -- I think it would, good call :) > Maintainable Configuration Versioning > - > > Key: MINIFI-117 > URL: https://issues.apache.org/jira/browse/MINIFI-117 > Project: Apache NiFi MiNiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander > > In order to avoid a tangled web of if/else statements around every possible > permutation of config.yml and an equally complicated validation routine, we > need to utilize a versioning strategy that allows for specific parsing and > validation code for each version of the ConfigSchema. > This will allow us to determine if a schema will be compatible with an older > MiNiFi instance more easily by validating the structure of the file at a > given version while still supporting an upconvert on read to the latest > ConfigSchema while parsing at runtime. > It should also facilitate a deprecation strategy of removing older versions' > classes without needing to touch current implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596058#comment-15596058 ] ASF GitHub Bot commented on MINIFI-117: --- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84488138 --- Diff: minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/ConfigMain.java --- @@ -96,13 +99,21 @@ public int validate(String[] args) { } try (InputStream inputStream = pathInputStreamFactory.create(args[1])) { try { -ConfigSchema configSchema = SchemaLoader.loadConfigSchemaFromYaml(inputStream); +ConvertableSchema configSchema = SchemaLoader.loadConvertableSchemaFromYaml(inputStream); if (!configSchema.isValid()) { configSchema.getValidationIssues().forEach(s -> System.out.println(s)); System.out.println(); return ERR_INVALID_CONFIG; } else { - System.out.println(NO_VALIDATION_ERRORS_FOUND_IN_TEMPLATE); +ConfigSchema currentSchema = configSchema.convert(); --- End diff -- Would it make sense to convert and show any validation errors even if it's not valid in the old version? > Maintainable Configuration Versioning > - > > Key: MINIFI-117 > URL: https://issues.apache.org/jira/browse/MINIFI-117 > Project: Apache NiFi MiNiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander > > In order to avoid a tangled web of if/else statements around every possible > permutation of config.yml and an equally complicated validation routine, we > need to utilize a versioning strategy that allows for specific parsing and > validation code for each version of the ConfigSchema. > This will allow us to determine if a schema will be compatible with an older > MiNiFi instance more easily by validating the structure of the file at a > given version while still supporting an upconvert on read to the latest > ConfigSchema while parsing at runtime. > It should also facilitate a deprecation strategy of removing older versions' > classes without needing to touch current implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596059#comment-15596059 ] ASF GitHub Bot commented on MINIFI-117: --- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84528235 --- Diff: minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/serialization/SchemaLoader.java --- @@ -30,13 +32,13 @@ import java.util.stream.Collectors; public class SchemaLoader { -private static final Map> configSchemaFactories = initConfigSchemaFactories(); +private static final Map > configSchemaFactories = initConfigSchemaFactories(); -private static Map > initConfigSchemaFactories() { -Map > result = new HashMap<>(); -result.put(String.valueOf((Object)null), ConfigSchema::new); -result.put("", ConfigSchema::new); -result.put("1", ConfigSchema::new); --- End diff -- The changes in the Config schemas need to be documented. Could you add a section to the admin guide (the Config Schema portion) that explains the versioning scheme as well as what changed in v2? I am suggesting to put it in the docs instead of a migration guidance for a release because working with multiple different version of MiNiFi and configs will be an everyday task (esp when C is added). > Maintainable Configuration Versioning > - > > Key: MINIFI-117 > URL: https://issues.apache.org/jira/browse/MINIFI-117 > Project: Apache NiFi MiNiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander > > In order to avoid a tangled web of if/else statements around every possible > permutation of config.yml and an equally complicated validation routine, we > need to utilize a versioning strategy that allows for specific parsing and > validation code for each version of the ConfigSchema. > This will allow us to determine if a schema will be compatible with an older > MiNiFi instance more easily by validating the structure of the file at a > given version while still supporting an upconvert on read to the latest > ConfigSchema while parsing at runtime. > It should also facilitate a deprecation strategy of removing older versions' > classes without needing to touch current implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596057#comment-15596057 ] ASF GitHub Bot commented on MINIFI-117: --- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84527193 --- Diff: minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/ConfigSchema.java --- @@ -142,128 +125,10 @@ public ConfigSchema(Map map) { } } -protected List getProcessorSchemas(List processorMaps) { -if (processorMaps == null) { -return null; -} -List processors = convertListToType(processorMaps, "processor", ProcessorSchema.class, PROCESSORS_KEY); - -MapidMap = processors.stream().map(ProcessorSchema::getId).filter( -s -> !StringUtil.isNullOrEmpty(s)).collect(Collectors.toMap(Function.identity(), s -> 2, Integer::compareTo)); - -// Set unset ids -processors.stream().filter(connection -> StringUtil.isNullOrEmpty(connection.getId())).forEachOrdered(processor -> processor.setId(getUniqueId(idMap, processor.getName(; - -return processors; -} - -protected List getConnectionSchemas(List connectionMaps) { -if (connectionMaps == null) { -return null; -} -List connections = convertListToType(connectionMaps, "connection", ConnectionSchema.class, CONNECTIONS_KEY); -Map idMap = connections.stream().map(ConnectionSchema::getId).filter( -s -> !StringUtil.isNullOrEmpty(s)).collect(Collectors.toMap(Function.identity(), s -> 2, Integer::compareTo)); - -Map processorNameToIdMap = new HashMap<>(); - -// We can't look up id by name for names that appear more than once -Set duplicateProcessorNames = new HashSet<>(); - -List processors = getProcessors(); -if (processors != null) { -processors.stream().forEachOrdered(p -> processorNameToIdMap.put(p.getName(), p.getId())); - -Set processorNames = new HashSet<>(); - processors.stream().map(ProcessorSchema::getName).forEachOrdered(n -> { -if (!processorNames.add(n)) { -duplicateProcessorNames.add(n); -} -}); -} - -Set remoteInputPortIds = new HashSet<>(); -List remoteProcessingGroups = getRemoteProcessingGroups(); -if (remoteProcessingGroups != null) { - remoteInputPortIds.addAll(remoteProcessingGroups.stream().filter(r -> r.getInputPorts() != null) -.flatMap(r -> r.getInputPorts().stream()).map(RemoteInputPortSchema::getId).collect(Collectors.toSet())); -} - -Set problematicDuplicateNames = new HashSet<>(); -Set missingProcessorNames = new HashSet<>(); -// Set unset ids -connections.stream().filter(connection -> StringUtil.isNullOrEmpty(connection.getId())).forEachOrdered(connection -> connection.setId(getUniqueId(idMap, connection.getName(; - -connections.stream().filter(connection -> StringUtil.isNullOrEmpty(connection.getSourceId())).forEach(connection -> { -String sourceName = connection.getSourceName(); -if (remoteInputPortIds.contains(sourceName)) { -connection.setSourceId(sourceName); -} else { -if (duplicateProcessorNames.contains(sourceName)) { -problematicDuplicateNames.add(sourceName); -} -String sourceId = processorNameToIdMap.get(sourceName); -if (StringUtil.isNullOrEmpty(sourceId)) { -missingProcessorNames.add(sourceName); -} else { -connection.setSourceId(sourceId); -} -} -}); - -connections.stream().filter(connection -> StringUtil.isNullOrEmpty(connection.getDestinationId())) -.forEach(connection -> { -String destinationName = connection.getDestinationName(); -if (remoteInputPortIds.contains(destinationName)) { -connection.setDestinationId(destinationName); -} else { -if (duplicateProcessorNames.contains(destinationName)) { -problematicDuplicateNames.add(destinationName); -} -String destinationId =
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596056#comment-15596056 ] ASF GitHub Bot commented on MINIFI-117: --- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84527495 --- Diff: minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/ConnectionSchema.java --- @@ -60,30 +53,11 @@ public ConnectionSchema(Map map) { super(map, CONNECTIONS_KEY); sourceId = getOptionalKeyAsType(map, SOURCE_ID_KEY, String.class, CONNECTIONS_KEY, ""); -if (StringUtil.isNullOrEmpty(sourceId)) { -sourceName = getRequiredKeyAsType(map, SOURCE_NAME_KEY, String.class, CONNECTIONS_KEY); -} - -String sourceRelationshipName = getOptionalKeyAsType(map, SOURCE_RELATIONSHIP_NAME_KEY, String.class, CONNECTIONS_KEY, null); -if (StringUtil.isNullOrEmpty(sourceRelationshipName)) { -sourceRelationshipNames = getOptionalKeyAsType(map, SOURCE_RELATIONSHIP_NAMES_KEY, List.class, CONNECTIONS_KEY, new ArrayList()); -if (sourceRelationshipNames.isEmpty()) { - addValidationIssue(getIssueText(SOURCE_RELATIONSHIP_NAMES_KEY, CONNECTIONS_KEY, "expected at least one relationship to be specified")); -} -} else { -if (map.containsKey(SOURCE_RELATIONSHIP_NAMES_KEY)) { -addValidationIssue("Only one of " + SOURCE_RELATIONSHIP_NAME_KEY + ", " + SOURCE_RELATIONSHIP_NAMES_KEY + " should be set per connection. Found both on " -+ (StringUtil.isNullOrEmpty(getName()) ? getId() : getName())); -sourceRelationshipNames = getRequiredKeyAsType(map, SOURCE_RELATIONSHIP_NAMES_KEY, List.class, CONNECTIONS_KEY); -} else { -sourceRelationshipNames = new ArrayList<>(Arrays.asList(sourceRelationshipName)); -} +sourceRelationshipNames = getOptionalKeyAsType(map, SOURCE_RELATIONSHIP_NAMES_KEY, List.class, CONNECTIONS_KEY, new ArrayList<>()); +if (sourceRelationshipNames.isEmpty()) { +addValidationIssue("Expected at least one value in " + SOURCE_RELATIONSHIP_NAMES_KEY + " for " + CONNECTIONS_KEY + " " + getName()); } - destinationId = getOptionalKeyAsType(map, DESTINATION_ID_KEY, String.class, CONNECTIONS_KEY, ""); --- End diff -- Same comment as on source, I think this should be "required". > Maintainable Configuration Versioning > - > > Key: MINIFI-117 > URL: https://issues.apache.org/jira/browse/MINIFI-117 > Project: Apache NiFi MiNiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander > > In order to avoid a tangled web of if/else statements around every possible > permutation of config.yml and an equally complicated validation routine, we > need to utilize a versioning strategy that allows for specific parsing and > validation code for each version of the ConfigSchema. > This will allow us to determine if a schema will be compatible with an older > MiNiFi instance more easily by validating the structure of the file at a > given version while still supporting an upconvert on read to the latest > ConfigSchema while parsing at runtime. > It should also facilitate a deprecation strategy of removing older versions' > classes without needing to touch current implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-117) Maintainable Configuration Versioning
[ https://issues.apache.org/jira/browse/MINIFI-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596060#comment-15596060 ] ASF GitHub Bot commented on MINIFI-117: --- Github user JPercivall commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/45#discussion_r84527469 --- Diff: minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/ConnectionSchema.java --- @@ -60,30 +53,11 @@ public ConnectionSchema(Map map) { super(map, CONNECTIONS_KEY); sourceId = getOptionalKeyAsType(map, SOURCE_ID_KEY, String.class, CONNECTIONS_KEY, ""); --- End diff -- I believe this should now be required instead of optional (since there is no id vs name confusion) > Maintainable Configuration Versioning > - > > Key: MINIFI-117 > URL: https://issues.apache.org/jira/browse/MINIFI-117 > Project: Apache NiFi MiNiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander > > In order to avoid a tangled web of if/else statements around every possible > permutation of config.yml and an equally complicated validation routine, we > need to utilize a versioning strategy that allows for specific parsing and > validation code for each version of the ConfigSchema. > This will allow us to determine if a schema will be compatible with an older > MiNiFi instance more easily by validating the structure of the file at a > given version while still supporting an upconvert on read to the latest > ConfigSchema while parsing at runtime. > It should also facilitate a deprecation strategy of removing older versions' > classes without needing to touch current implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MINIFI-57) WARN in logs from startup
[ https://issues.apache.org/jira/browse/MINIFI-57?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595861#comment-15595861 ] ASF GitHub Bot commented on MINIFI-57: -- GitHub user apiri opened a pull request: https://github.com/apache/nifi-minifi/pull/47 MINIFI-57 Adjusting minifi-framework-nar dependencies MINIFI-57 Adjusting minifi-framework-nar dependencies and migrating shared item to the framework NAR. This is an initial solution to the extraneous WARN generated on startup. Some of this will change pending the results of #43. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apiri/nifi-minifi MINIFI-57 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi/pull/47.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #47 > WARN in logs from startup > - > > Key: MINIFI-57 > URL: https://issues.apache.org/jira/browse/MINIFI-57 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Aldrin Piri > > WARN in logs. Probably can be pulled in differently to avoid this. > 2016-07-08 09:04:30,300 WARN [main] org.apache.nifi.nar.NarClassLoader > ./work/nar/framework/minifi-framework-nar-0.0.1.nar-unpacked does not > contain META-INF/bundled-dependencies! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: Dummy commit to clean stalled gh PRs
Repository: nifi Updated Branches: refs/heads/master b864d49f6 -> 220b76803 Dummy commit to clean stalled gh PRs This closes: #513 (superseded) Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/220b7680 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/220b7680 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/220b7680 Branch: refs/heads/master Commit: 220b76803b2f70ddd6dcc366478d722599349983 Parents: b864d49 Author: Andre F de MirandaAuthored: Thu Oct 20 23:58:39 2016 +1100 Committer: Andre F de Miranda Committed: Fri Oct 21 23:17:44 2016 +1100 -- --