[GitHub] nifi pull request #534: Fix for NIFI-1838, NIFI-1152, NIFI-2117
Github user PuspenduBanerjee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/534#discussion_r68490863
--- Diff:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/InvokeScriptedProcessor.java
---
@@ -92,11 +92,10 @@
logger.error(message, t);
}
}
-} else {
-// Return defaults for now
-relationships.add(REL_SUCCESS);
-relationships.add(REL_FAILURE);
}
+// Add defaults
+relationships.add(REL_SUCCESS);
+relationships.add(REL_FAILURE);
--- End diff --
Even in original state of `InvokeScriptedProcesor`, the documentation says
> SUCCESS and FAILURE are always returned, and if the script
processor has defined additional relationships, those will be added as
well
```
/**
* Returns the valid relationships for this processor. SUCCESS and
FAILURE are always returned, and if the script
* processor has defined additional relationships, those will be added
as well.
*
* @return a Set of Relationships supported by this processor
*/
@Override
public Set getRelationships()
```
So, ideally those test cases does not require modification, they are there
as per contract stated in the java-doc and this piece is valid.
I strongly believe we should not break that contract as that may create
implication for scripted-processor in production.
Please correct me otherwise.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #534: Fix for NIFI-1838 & NIFI-1152
Github user PuspenduBanerjee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/534#discussion_r68351775
--- Diff:
nifi-mock/src/test/java/org/apache/nifi/util/TestMockProcessSession.java ---
@@ -36,7 +36,7 @@ public void testPenalizeFlowFileFromProcessor() {
protected static class PoorlyBehavedProcessor extends
AbstractProcessor {
-private static final Relationship REL_FAILURE = new
Relationship.Builder()
+protected static final Relationship REL_FAILURE = new
Relationship.Builder()
--- End diff --
reverted back.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #534: Fix for NIFI-1838 & NIFI-1152
Github user PuspenduBanerjee commented on the issue: https://github.com/apache/nifi/pull/534 @pvillard31 @mattyb149 Do we still need more discussion on this or will you be able to check it in? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #534: Fix for NIFI-1838 & NIFI-1152
Github user PuspenduBanerjee commented on the issue: https://github.com/apache/nifi/pull/534 @pvillard31 Thanks for your effort to review this. I have incorporated your comments as much as I could. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #534: Fix for NIFI-1838 & NIFI-1152
Github user PuspenduBanerjee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/534#discussion_r67446868
--- Diff:
nifi-mock/src/test/java/org/apache/nifi/util/TestMockProcessSession.java ---
@@ -36,7 +36,7 @@ public void testPenalizeFlowFileFromProcessor() {
protected static class PoorlyBehavedProcessor extends
AbstractProcessor {
-private static final Relationship REL_FAILURE = new
Relationship.Builder()
+protected static final Relationship REL_FAILURE = new
Relationship.Builder()
--- End diff --
No not needed anymore. :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #534: Fix for NIFI-1838 & NIFI-1152
Github user PuspenduBanerjee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/534#discussion_r67446830
--- Diff:
nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java ---
@@ -793,13 +798,21 @@ private void validateState(final FlowFile flowFile) {
throw new FlowFileHandlingException(flowFile + " is not the
most recent version of this flow file within this session");
}
+
for (final List flowFiles : transferMap.values()) {
if (flowFiles.contains(flowFile)) {
throw new IllegalStateException(flowFile + " has already
been transferred");
}
}
}
+private void validateDestinationRelation(final Relationship
relationship) {
+if (!relationships.contains(relationship)) {
+rollback();
--- End diff --
I think, though it's for testing purpose , we should emulate as real as we
can and rollback is the best possible way to revert back to a stable state.
And, in a real context we may face such issue in case of orchestration, where
an attribute in a flowfile may finally dictate where it should be transferred.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #534: Fix for NIFI-1838 & NIFI-1152
Github user PuspenduBanerjee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/534#discussion_r67439195
--- Diff:
nifi-mock/src/test/java/org/apache/nifi/util/TestMockProcessSession.java ---
@@ -55,4 +55,35 @@ public void onTrigger(final ProcessContext ctx, final
ProcessSession session) th
}
}
+
+
+@Test(expected = AssertionError.class)
+public void testNonExistentRelationFromProcessor() {
+
TestRunners.newTestRunner(NonExistentRelationTestProcessor.class).run();
+}
+
+protected static class NonExistentRelationTestProcessor extends
PoorlyBehavedProcessor {
+
+static final Relationship REL_NON_EXISTENT = new
Relationship.Builder()
+.name("NonExistent Relation")
+.build();
+
+public Set relationships =
Collections.singleton(REL_FAILURE);
+
+@Override
+public Set getRelationships() {
+return relationships;
--- End diff --
Good finding. Can't really remember why did I add that. removed now and
checking in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #534: Fix for NIFI-1838 & NIFI-1152
Github user PuspenduBanerjee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/534#discussion_r67394594
--- Diff:
nifi-mock/src/test/java/org/apache/nifi/util/TestMockProcessSession.java ---
@@ -36,7 +36,7 @@ public void testPenalizeFlowFileFromProcessor() {
protected static class PoorlyBehavedProcessor extends
AbstractProcessor {
-private static final Relationship REL_FAILURE = new
Relationship.Builder()
+protected static final Relationship REL_FAILURE = new
Relationship.Builder()
--- End diff --
Directly using it in Test NonExistentRelationTestProcessor public
Set relationships = Collections.singleton(REL_FAILURE);
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #534: Fix for NIFI-1838 & NIFI-1152
Github user PuspenduBanerjee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/534#discussion_r67394024
--- Diff:
nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java ---
@@ -793,13 +798,21 @@ private void validateState(final FlowFile flowFile) {
throw new FlowFileHandlingException(flowFile + " is not the
most recent version of this flow file within this session");
}
+
for (final List flowFiles : transferMap.values()) {
if (flowFiles.contains(flowFile)) {
throw new IllegalStateException(flowFile + " has already
been transferred");
}
}
}
+private void validateDestinationRelation(final Relationship
relationship) {
+if (!relationships.contains(relationship)) {
+rollback();
--- End diff --
So, if a relationship does not exist what should we do?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #534: Fix for NIFI-1838 & NIFI-1152
Github user PuspenduBanerjee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/534#discussion_r67393509
--- Diff:
nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java ---
@@ -587,13 +589,15 @@ public void transfer(final FlowFile flowFile, final
Relationship relationship) {
}
validateState(flowFile);
+validateDestinationRelation(relationship);
List list = transferMap.get(relationship);
if (list == null) {
list = new ArrayList<>();
transferMap.put(relationship, list);
}
beingProcessed.remove(flowFile.getId());
+ // if(sharedState.)
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #534: Fix for NIFI-1838 & NIFI-1152
GitHub user PuspenduBanerjee opened a pull request: https://github.com/apache/nifi/pull/534 Fix for NIFI-1838 & NIFI-1152 Fix for NIFI-1838 & NIFI-1152 You can merge this pull request into a Git repository by running: $ git pull https://github.com/PuspenduBanerjee/nifi NIFI-1152 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/534.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #534 commit 1ee24388bb54a82d008d4ab7e32dd39c2bb0ff4d Author: Puspendu Banerjee <puspendu.baner...@gmail.com> Date: 2016-05-03T01:36:50Z Fix for NIFI-1838 & NIFI-1152 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #514: fixes NIFI-1989
GitHub user PuspenduBanerjee opened a pull request: https://github.com/apache/nifi/pull/514 fixes NIFI-1989 Fixes : https://issues.sonatype.org/browse/MVNCENTRAL-244 > Taking a look at the POM at http://repo1.maven.org/maven2/org/apache/commons/commons-io/1.3.2/commons-io-1.3.2.pom, the groupId and artifactId are different from the deploy path and it seems the identical artifacts are available at http://repo1.maven.org/maven2/commons-io/commons-io/1.3.2/ Having the same classes available under two different GAV coordinates may lead to a lot of problems should one decide to update the component to a new version. I see a lot of old builds could break when we delete from the old coordinates, but what about a (correct) relocation POM? Builds would continue to work and the mishap would appear on the screen. You can merge this pull request into a Git repository by running: $ git pull https://github.com/PuspenduBanerjee/nifi NIFI-1989 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/514.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #514 commit e25bdd7b2b455728fb14ab7d0cd9a2c76c271aad Author: Puspendu Banerjee <puspendu.baner...@gmail.com> Date: 2016-06-09T18:00:57Z fixes NIFI-1989 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #400: Fix for NIFI-1838 & NIFI-1152 & Code modification fo...
Github user PuspenduBanerjee closed the pull request at: https://github.com/apache/nifi/pull/400 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Fix for NIFI-1838 & NIFI-1152 & Code modificati...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/400#issuecomment-216557197 @markap14 If it looks good, will you be able to check in? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Fix for NIFI-1838 & NIFI-1152 & Code modificati...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/400#issuecomment-216414833 Updated to address Fix for NIFI-1838 & NIFI-1152 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Code modification for typeSafety
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/400#issuecomment-216302926 @apiri Please incorporate PR #370 as soon as you can. It will be a great help. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Enhancement for NIFI-1045 : Add "backup suffix"...
Github user PuspenduBanerjee closed the pull request at: https://github.com/apache/nifi/pull/230 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NiFi-924: Camel integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/219#issuecomment-213595825 @joewitt @olegz Thank you all and sorry for the delay in reply, was dead busy. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NiFi-924: Camel integration
Github user PuspenduBanerjee closed the pull request at: https://github.com/apache/nifi/pull/219 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-1571 initial commit of SpringContext suppo...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/271#issuecomment-198245248 @olegz Sorry for the delay in response, I shall look into that during next week. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/218#issuecomment-195862566 @trkurc Oh Man! Thanks for your guidance. Any chance of using power-shell in future? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/218#issuecomment-195441150 @trkurc Appreciate the Spirit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/218#issuecomment-195231594 @apiri For now I have no way to check Windows or Cygwin as I do not have any MS windows installation, total linux guy and on vacation to my homeland, so can't even borrow a windows PC from a colleague. . Please help to find out someone with Windows. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Enhancement for NIFI-1045 : Add "backup suffix"...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/230#issuecomment-187767254 @trixpan any chance to review again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/218#issuecomment-186010907 @trkurc Please review once you get a chance. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Enhancement for NIFI-1045 : Add "backup suffix"...
GitHub user PuspenduBanerjee opened a pull request: https://github.com/apache/nifi/pull/230 Enhancement for NIFI-1045 : Add "backup suffix" Conflict Resolution Enhancement for NIFI-1045 : Add "backup suffix" Conflict Resolution Strategy to PutHDFS / PutFile You can merge this pull request into a Git repository by running: $ git pull https://github.com/PuspenduBanerjee/nifi ext-resolver Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/230.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #230 commit 925ea04c26b4eb59bcf06e3ea2ab2bb4d55b7b6d Author: puspendu.baner...@gmail.com <puspendu.baner...@gmail.com> Date: 2016-02-17T07:33:29Z Enhancement for NIFI-1045 : Add "backup suffix" Conflict Resolution Strategy to PutHDFS / PutFile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Fix NiFi-1501 : Test Monitor Activity has spuri...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/225#issuecomment-185025159 BTW.. is there any way to comment or ask question on Extention Registry page at cwiki, seems I don't have access to comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Fix Nifi-1501 : Test Monitor Activity has spuri...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/225#issuecomment-184942046 Hi Team, Did anyone get a chance to review yet? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Fix Nifi-1501 : Test Monitor Activity has spuri...
GitHub user PuspenduBanerjee opened a pull request: https://github.com/apache/nifi/pull/225 Fix Nifi-1501 : Test Monitor Activity has spurious failures. Added test support with adaptive threshold to cater the assumption that MonitorActivity shouldn't generate inactivity, b/c run() will reset the lastSuccessfulTransfer. But that is only true if @OnSchedule & onTrigger does not get called more than MonitorActivity.THRESHOLD apart. You can merge this pull request into a Git repository by running: $ git pull https://github.com/PuspenduBanerjee/nifi NiFi-1501 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/225.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #225 commit 4660ea1ddbee75ff7ad8df9b383f13a08f03b288 Author: puspendu.baner...@gmail.com <puspendu.baner...@gmail.com> Date: 2016-02-16T08:00:08Z Fix Nifi-1501 : Test Monitor Activity has spurious failures. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NiFi-1481 Enhancement[ nifi.sh env]
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/218#issuecomment-183624627 @apiri Can you please review it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Enhancement for nifi-1481 [ nifi.sh env]
GitHub user PuspenduBanerjee opened a pull request: https://github.com/apache/nifi/pull/218 Enhancement for nifi-1481 [ nifi.sh env] Initial working version of nifi-1481 , with plenty of room for enhancement. You can merge this pull request into a Git repository by running: $ git pull https://github.com/PuspenduBanerjee/nifi pb_nifi-1481 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/218.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #218 commit c68664c327f4f5163610343165d28667b6bbcb28 Author: puspendu.baner...@gmail.com <puspendu.baner...@gmail.com> Date: 2016-02-10T07:17:00Z Fix-1481 support for ./nifi.sh env commit ec6dc8632e5d72739bd1741079993ca662b4306c Author: puspendu.baner...@gmail.com <puspendu.baner...@gmail.com> Date: 2016-02-13T03:25:22Z Removed compile time Dependency on tools.jar/classes.jar. Added better detection of toos.jar/classes.jar with more meaningful message. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-924:Nifi-Camel Integration
Github user PuspenduBanerjee closed the pull request at: https://github.com/apache/nifi/pull/197 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Fix NiFi-1461
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/204#issuecomment-180234634 I understand people are busy with 0.5 rel , still does anyone have time & interest to start reviewing, this major [ as per issue tracker] issue? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Fix NiFi-1461
GitHub user PuspenduBanerjee opened a pull request: https://github.com/apache/nifi/pull/204 Fix NiFi-1461 fix for [NiFi-1461](https://issues.apache.org/jira/browse/NIFI-1461) You can merge this pull request into a Git repository by running: $ git pull https://github.com/PuspenduBanerjee/nifi nifi-head Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/204.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #204 commit dc120bfdd8578b2f47e6f0f591be78b07762caec Author: puspendu.baner...@gmail.com <puspendu.baner...@gmail.com> Date: 2016-02-04T08:40:26Z Fix NiFi-1461 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-1460 Test Performance improvement. test Ti...
Github user PuspenduBanerjee commented on the pull request:
https://github.com/apache/nifi/pull/202#issuecomment-179166551
@trkurc I think, if UUID format is of concern the following piece will run
more than 2 times faster compared to `UUID(0,long)`, could you please re-verify:
```java
private static String createFakeUUIDString(final AtomicLong atomicLong){
final String s=Long.toHexString(atomicLong.incrementAndGet());
return new StringBuffer("---"
.substring(0,(35-s.length()))+s)
.insert(23, '-').toString();
}
```
I hope, everyone wins that way too along with more performance benefit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1460 Test Performance improvement. test Ti...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/202#issuecomment-179265990 Probably a basic question, but am I missing any process to request a PR to merge? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Test Performance improvement. test Timeout Miti...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/202#issuecomment-178847399 @joewitt Another chance of improving test performance & less IO by replacing `UUID.randomUUID()` with` AtomicLong#incrementAndGet()`, which is sufficient for our test cases at org.apache.nifi.controller#TestStandardFlowFileQueue. UUID.randomUUID() affects linux/unix systems heavily with Blocking IO from /dev/(u)random, entropy starvation etc., so let's not use that unless necessary. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Performance improvement. test Timeout Mitigatio...
GitHub user PuspenduBanerjee opened a pull request: https://github.com/apache/nifi/pull/202 Performance improvement. test Timeout Mitigation. less IO, less depen⦠Performance improvement. test Timeout Mitigation. less IO, less dependency on /dev/(u)random. You can merge this pull request into a Git repository by running: $ git pull https://github.com/PuspenduBanerjee/nifi orgin_master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/202.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #202 commit 0657b3a908910241cbf709a7b1387766d1e12e8b Author: puspendu.baner...@gmail.com <puspendu.baner...@gmail.com> Date: 2016-02-02T21:04:37Z Performance improvement. test Timeout Mitigation. less IO, less dependency on /dev/(u)random. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-1460 Test Performance improvement. test Ti...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/202#issuecomment-178875032 @trkurc Thanks for reviewing. let me know in case can be merged or if I need to create a patch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-1460 Test Performance improvement. test Ti...
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/202#issuecomment-178874441 Raised issue : https://issues.apache.org/jira/browse/NIFI-1460 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-178634682 Please find _mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V_ near line 170 in all build at [my PR](https://travis-ci.org/apache/nifi/builds/106415259) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-178626208 @joewitt Seeing that Travis build has started to skip tests. Is that a intended decision? I am asking as I am not seeing such configuration in .travis.yml. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-924:Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request:
https://github.com/apache/nifi/pull/197#issuecomment-178252892
@joewitt For Non-Uniform charset attached piece should work.
```java
/**
* @param rawBytes
* @param patternToMatch
* @return
*/
private static boolean matchPattern(byte[] rawBytes , final String
patternToMatch){
out.println("Pattern= "+patternToMatch);
final Pattern pattern = Pattern.compile(patternToMatch);
final int patternByteLength=patternToMatch.getBytes().length;
out.println("PatternByteLength= "+patternByteLength);
final int minSliceByteLength=2;
final int sliceByteLength = patternByteLength > minSliceByteLength
? patternByteLength : minSliceByteLength;
int offset = 0;
while (offset <= rawBytes.length) {
String s=new String(Arrays.copyOfRange(rawBytes,
offset>patternByteLength? offset-(patternByteLength+1):0,
(offset +
sliceByteLength)));
out.println("Retrieved byte[]= "+s);
if (pattern.matcher(s).find()) {
return true;
}
offset += sliceByteLength;
};
return false;
}
```
I tried with \:
```java
matchPattern("Iâ¥NYâ¥Yâ¥NABCDdkk;lskiepdmclod;die;'cnjywkl;sfdsXRSHSYâ¥â¥Y".getBytes(),
"â¥â¥Y");
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-177439597 Moved to : https://github.com/apache/nifi/pull/197 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: NIFI-924:Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/197#issuecomment-177442040 @olegz Please review once you get a chance. @joewitt please find NiFi Template attached with PR at [CamelProcessorTestingTemplate.xml.zip](https://github.com/apache/nifi/files/81/CamelProcessorTestingTemplate.xml.zip) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee closed the pull request at: https://github.com/apache/nifi/pull/186 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-176814383 Closing PR, as found some issue in Integration test. Shall raise a new PR later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-176439666 Hello @joewitt PFA sample camel contexts. [sample-application-contexts.zip](https://github.com/apache/nifi/files/108925/sample-application-contexts.zip) - You can copy the content & directly paste at **Camel Spring Context Definition **property. - In case you require some extra camel module add their maven groupId:artifactId:version in groupId/artifactId/version form [GAV]. Multiple GAVs can be separated by ',' at **Extra Libraries **property.. I shall attach a presentation or some other visual representation, as we progress with the PR. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-176232391 @apiri Again OOM in travis: testExecuteIngestAndUpdate(org.apache.nifi.processors.standard.TestExecuteStreamCommand) Time elapsed: 1.717 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:149) at java.lang.StringCoding.decode(StringCoding.java:193) at java.lang.StringCoding.decode(StringCoding.java:254) at java.lang.String.(String.java:536) at java.lang.String.(String.java:556) at org.apache.nifi.processors.standard.TestExecuteStreamCommand.testExecuteIngestAndUpdate(TestExecuteStreamCommand.java:128) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-175688480 Hello, Now my build is failing in Travis CI, where as it's running fine locally. Can anyone please help [INFO] BUILD FAILURE [INFO] [INFO] Total time: 01:07 min [INFO] Finished at: 2016-01-27T06:50:28+00:00 [INFO] Final Memory: 150M/420M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) on project nifi-kite-processors: Failed to resolve dependencies for one or more projects in the reactor. Reason: Missing: [ERROR] -- [ERROR] 1) org.apache.hive:hive-serde:jar:0.12.0-cdh5.0.0 [ERROR] [ERROR] Try downloading the file manually from the project website. [ERROR] [ERROR] Then, install it using the command: [ERROR] mvn install:install-file -DgroupId=org.apache.hive -DartifactId=hive-serde -Dversion=0.12.0-cdh5.0.0 -Dpackaging=jar -Dfile=/path/to/file [ERROR] [ERROR] Alternatively, if you host your own repository you can deploy the file there: [ERROR] mvn deploy:deploy-file -DgroupId=org.apache.hive -DartifactId=hive-serde -Dversion=0.12.0-cdh5.0.0 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id] [ERROR] [ERROR] Path to dependency: [ERROR] 1) org.apache.nifi:nifi-kite-processors:jar:0.4.2-SNAPSHOT [ERROR] 2) org.apache.hive.hcatalog:hive-hcatalog-core:jar:1.2.0 [ERROR] 3) org.apache.hive:hive-metastore:jar:1.2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/186#discussion_r51009342
--- Diff:
nifi-nar-bundles/nifi-camel-bundle/nifi-camel-processors/src/main/java/org/apache/nifi/processors/camel/CamelProcessor.java
---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.camel;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+import groovy.lang.GroovyClassLoader;
+
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.camel.CamelContext;
+import org.apache.camel.Exchange;
+import org.apache.camel.ProducerTemplate;
+import org.apache.camel.ServiceStatus;
+import org.apache.camel.impl.DefaultExchange;
+import org.apache.camel.impl.DefaultShutdownStrategy;
+import org.apache.camel.spi.ShutdownStrategy;
+import org.apache.camel.spring.SpringCamelContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.commons.lang3.math.NumberUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.springframework.context.ApplicationContext;
+import org.springframework.context.support.AbstractApplicationContext;
+import org.springframework.context.support.ClassPathXmlApplicationContext;
+import org.springframework.context.support.FileSystemXmlApplicationContext;
+import org.springframework.context.support.GenericXmlApplicationContext;
+import org.springframework.core.io.ByteArrayResource;
+
+/**
+ * This processor runs a Camel Route.
+ */
+@Tags({"camel", "route", "put"})
+@InputRequirement(Requirement.INPUT_ALLOWED)
+@CapabilityDescription("Runs a Camel Route. Each input FlowFile is
converted into a Camel Exchange "
+ + "for processing by configured Route. It exports
ProcessSession to camel exchange header 'nifiSession'")
+public class CamelProcessor extends AbstractProcessor {
+
+protected static final Relationship SUCCESS = new
Relationship.Builder().name("success")
+.description("Camel Route has Executed Successfully").build();
+
+protected static final Relationship FAILURE = new
Relationship.Builder().name("failure")
+.description("Camel Route has Failed to Execute").build();
+
+public static final PropertyDescriptor CAMEL_SPRING_CONTEXT_FILE_PATH
= new PropertyDescriptor.Builder()
+.name("Camel Spring Config File Path")
+.description("The Classpath where NiFi can find Spring Application
context file"
+ + " Ex:
classpath:/META-INF/camel-application-context.xml")
+
.defaultValue("classpath:/META-INF/camel-application-context.xml").required(true).addValidator(Validator.VALID)
+.build();
+
+public static final PropertyDescriptor CAMEL_SPRING_
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-175781324 @apiri Thanks. It has built correctly with intermittent Heap issue. :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request: Nifi-Camel Integration
Github user PuspenduBanerjee commented on the pull request: https://github.com/apache/nifi/pull/186#issuecomment-174913731 Hello Joe, Thanks for the review. About Licensing, I am trying to add some maven plugin to automate that process and strongly believe that it should be centralized [ as rat plugin is checking License ]. Any help in this regard will be very much appreciated. About documentation, will it be a good idea to add a README file or do you have any other suggestion? And yes , there are some WIP code containing e.printStackStrack() , ThreadLocal etc. Working to get those removed. Thanks again. Puspendu Banerjee --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
