[GitHub] nifi issue #583: NIFI-2115 Detailed Version Info for About Box

[jvwing]

Github user jvwing commented on the issue:

https://github.com/apache/nifi/pull/583
  
Here's what the result looks like while the details section is expanded. I 
plead guilty to being a bad UI designer, any tips would be appreciated.


![nifi-2115-about-box-info](https://cloud.githubusercontent.com/assets/3151078/16352007/0c6735ac-3a1f-11e6-8471-c56ef57641cb.png)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #583: NIFI-2115 Detailed Version Info for About Box

[jvwing]

GitHub user jvwing opened a pull request:

https://github.com/apache/nifi/pull/583

NIFI-2115 Detailed Version Info for About Box

* Java version and vendor
* OS name and version
* Build number(commit SHA), branch, and timestamp

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jvwing/nifi NIFI-2115-about-box-info-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/583.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #583


commit 3aefbc27a352d99a239211df692911912f9deb05
Author: James Wing 
Date:   2016-06-23T20:53:23Z

NIFI-2115 Detailed Version Info for About Box

* Java version and vendor
* OS name and version
* Build number(commit SHA), branch, and timestamp




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #582: NIFI-1952 Updates to REST resources for user/group m...

[jtstorck]

GitHub user jtstorck opened a pull request:

https://github.com/apache/nifi/pull/582

NIFI-1952 Updates to REST resources for user/group management

Updated StandardPolicyBasedAuthorizerDAO to throw 
ResourceNotFoundExceptions when user/group/policy not found
Added spec for StandardPolicyBasedAuthorizerDAO
Added exception mapper for AuthorizationAccessException, added mapper to 
nifi-web-api-context.xml
Added rest endpoints to get all users and user groups
Merged UsersResource and UserGroupsResource into TenantsResource

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jtstorck/nifi NIFI-1952

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/582.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #582


commit b9c047eecf5acd6bcca661341597762b25252daa
Author: Jeff Storck 
Date:   2016-06-23T00:47:53Z

NIFI-1952 Updated StandardPolicyBasedAuthorizerDAO to throw 
ResourceNotFoundExceptions when user/group/policy not found
Added spec for StandardPolicyBasedAuthorizerDAO
Added exception mapper for AuthorizationAccessException, added mapper to 
nifi-web-api-context.xml
Added rest endpoints to get all users and user groups
Merged UsersResource and UserGroupsResource into TenantsResource




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #581: NIFI-1916 Improvements to FileAuthorizer to not parse flow ...

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/581
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #581: NIFI-1916 Improvements to FileAuthorizer to not pars...

[bbende]

GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/581

NIFI-1916 Improvements to FileAuthorizer to not parse flow when unnce…

…ssary and to recreate missing authorizations.xml

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-1916-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/581.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #581


commit 9aa1e71cdd8e93d4acc3a04e6f62c1aaf52ac22e
Author: Bryan Bende 
Date:   2016-06-24T21:04:44Z

NIFI-1916 Improvements to FileAuthorizer to not parse flow when unncessary 
and to recreate missing authorizations.xml




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #579: NIFI-1554: Changing process for cluster detection

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/579


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #579: NIFI-1554: Changing process for cluster detection

[bbende]

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/579
  
+1 verified build passes contrib-check and was able to get into a secured 
NiFi instance


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #580: 1.0 documentation updates - overview section

[Haimo-Liu]

GitHub user Haimo-Liu opened a pull request:

https://github.com/apache/nifi/pull/580

1.0 documentation updates - overview section

updated overview section that covers zero-master clustering, multi-tenancy, 
plus some other features, and adjusted formatting

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Haimo-Liu/nifi 1.0-docs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/580.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #580


commit e5f05ab1801b0072da1fe365caffc3180eea81a9
Author: Haimo Liu 
Date:   2016-06-23T21:08:25Z

update overview

clustering paragraph

commit d00557fceca1265697bb13db1c91da539916e079
Author: Haimo Liu 
Date:   2016-06-24T03:10:30Z

Update overview.adoc

commit c256ddcdff2d055eb68d4741ca02e8dce7d0b073
Author: Haimo Liu 
Date:   2016-06-24T15:19:23Z

Update overview.adoc

commit d58c8ebb1fda40f6e50f20ddaaca55cb883b9749
Author: Haimo Liu 
Date:   2016-06-24T15:55:39Z

Update overview.adoc

commit c7f7a74ea47848cdf5f5b66701c03aa58176a282
Author: Haimo Liu 
Date:   2016-06-24T19:32:14Z

Update overview.adoc

commit ff1e8a3e7114233c5a655034fcd813f26f1307c1
Author: Haimo Liu 
Date:   2016-06-24T19:58:54Z

Update overview.adoc

commit c01d96312e942cc933bed2e56112095ff8bd3157
Author: Haimo Liu 
Date:   2016-06-24T20:10:49Z

Add files via upload

commit bce6c7932dfea529528e59c727e7326483e2cf82
Author: Haimo Liu 
Date:   2016-06-24T20:12:17Z

Update overview.adoc




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #579: NIFI-1554: Changing process for cluster detection

[bbende]

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/579
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #579: NIFI-1554: Changing process for cluster detection

[mcgilman]

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/579

NIFI-1554: Changing process for cluster detection

- Updating cluster detection to support access through the flow resource.
- Using the correct resource when authorizing a root group port during site 
to site.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-1554

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/579.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #579


commit 8ed4523ed4c807576d2adb56cbf3879086d65666
Author: Matt Gilman 
Date:   2016-06-24T20:29:00Z

NIFI-1554:
- Updating cluster detection to support access through the flow resource.
- Using the correct resource when authorizing a root group port during site 
to site.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Fwd: Problem with EvaluationJsonPath

[Anuj Handa]

Hi ,

I am trying to user the Evaluate Json path processor.
It works fine except when the Value is evaluated to 0, then it throws  an
error


2016-06-24 15:00:56,917 ERROR [Timer-Driven Process Thread-9]
o.a.n.p.standard.EvaluateJsonPath
EvaluateJsonPath[id=475d43ec-549b-4378-81d5-778041558781] Unable to return
a scalar value for the expression
$..['SubwayPOSTransaction']['OrderSale']['SatelliteNumber'] for FlowFile
1519640. Evaluated value was [0]. Transferring to failure.

when i change the ReturnType to Json then it works but it makes the value
[0]

Any thoughts if its an issue or configuration of the processor that is
incorrect ?

[image: Inline image 1]

Apache NiFi 0.7.0 RC1 Release Helper Guide

[Joe Percivall]

Hello Apache NiFi community,
Please find the associated guidance to help those interested in 
validating/verifying the release so they can vote.
# Download latest KEYS file:  https://dist.apache.org/repos/dist/dev/nifi/KEYS
# Import keys file:  gpg --import KEYS
# [optional] Clear out local maven artifact repository
# Pull down nifi-0.7.0 source release artifacts for review:
  wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.0/nifi-0.7.0-source-release.zip
  wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.0/nifi-0.7.0-source-release.zip.asc
  wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.0/nifi-0.7.0-source-release.zip.md5
  wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.0/nifi-0.7.0-source-release.zip.sha1
  wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.0/nifi-0.7.0-source-release.zip.sha256
# Verify the signature  gpg --verify nifi-0.7.0-source-release.zip.asc
# Verify the hashes (md5, sha1, sha256) match the source and what was provided 
in the vote email thread  md5sum nifi-0.7.0-source-release.zip  sha1sum 
nifi-0.7.0-source-release.zip  sha256sum nifi-0.7.0-source-release.zip
# Unzip nifi-0.7.0-source-release.zip
# Verify the build works including release audit tool (RAT) checks  cd 
nifi-0.7.0  mvn clean install -Pcontrib-check
# Verify the contents contain a good README, NOTICE, and LICENSE.
# Verify the git commit ID is correct
# Verify the RC was branched off the correct git commit ID
# Look at the resulting convenience binary as found in nifi-assembly/target
# Make sure the README, NOTICE, and LICENSE are present and correct
# Run the resulting convenience binary and make sure it works as expected
# Send a response to the vote thread indicating a +1, 0, -1 based on your 
findings.
Thank you for your time and effort to validate the release!Joe 

[GitHub] nifi issue #362: NIFI-1769: added support for SSE-KMS and signature s3v4 aut...

[jvwing]

Github user jvwing commented on the issue:

https://github.com/apache/nifi/pull/362
  
@miquillo I'm happy to review and test the contributions you have made, and 
I would be happy to put some code where my comments with an implementation of 
the signature version controls.  But I am hoping you will continue to work us 
on this PR.  I would very much like your help in either writing the code or 
reviewing and testing the changes, as you have both experience with SSE-KMS in 
NiFi and knowledge of the driving use case.  What would you feel most 
comfortable with?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #577: [NIFI-2027] Responsive Tables

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/577
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #577: [NIFI-2027] Responsive Tables

[scottyaslan]

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/577

[NIFI-2027] Responsive Tables



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi responsiveDevBranch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/577.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #577


commit 7fcf5ee36cceb14455cb99136212b62a29571a84
Author: Scott Aslan 
Date:   2016-06-24T18:23:41Z

[NIFI-2027] Responsive Tables




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: escape functions in Expression Language

[Bryan Bende]

Hi Devin,

I'm not aware of a built in escape function in NiFi's EL, although you
might be able to do some of it with the replace functions.

NiFi's expression language is in this module:
https://github.com/apache/nifi/tree/master/nifi-commons/nifi-expression-language

The README has some good details about how to make changes. Let us know if
it isn't clear.

Thanks,

Bryan


On Fri, Jun 24, 2016 at 1:15 PM, Devin Fisher <
devin.fis...@perfectsearchcorp.com> wrote:

> I'm looking to create a bit of JSON using Nifi Expression Language so that
> I can send it to a web service. But It does not look like Nifi Expression
> Language have a function for escaping text to put into JSON.  I've looked
> around in the wiki and I don't see any documentation now how to extend
> the Expression Language (not even sure if it is possible).
>
> Looks like nifi makes use of Apache commons-lang which has some functions
> that would do the heavy lifting. [1]
>
> So if someone can help me understand the Expression Language end points I
> could do the work fairly easily I think. And add functions for the other
> formats supported by StringEscapeUtils (XML, csv, html).
>
> I think this could be useful.
>
> Devin
>
> [1]
>
> https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/StringEscapeUtils.html
>

[GitHub] nifi pull request #534: Fix for NIFI-1838 & NIFI-1152

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/534#discussion_r68430574
  
--- Diff: 
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/InvokeScriptedProcessor.java
 ---
@@ -92,11 +92,10 @@
 logger.error(message, t);
 }
 }
-} else {
-// Return defaults for now
-relationships.add(REL_SUCCESS);
-relationships.add(REL_FAILURE);
 }
+// Add defaults
+relationships.add(REL_SUCCESS);
+relationships.add(REL_FAILURE);
--- End diff --

In ExecuteScript, the two relationships success and failure are defined.

In this case, the original code is adding, by default, the two 
relationships if and only if the processor didn't add its own relationships. 
With the change you propose, success and failure will be added in any case even 
if we don't want success/failure, but rather relationships "a", "b", "c". I 
think that the failing tests should be modified to add the relationships 
success/failure. At least if I am not missing something here...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

escape functions in Expression Language

[Devin Fisher]

I'm looking to create a bit of JSON using Nifi Expression Language so that
I can send it to a web service. But It does not look like Nifi Expression
Language have a function for escaping text to put into JSON.  I've looked
around in the wiki and I don't see any documentation now how to extend
the Expression Language (not even sure if it is possible).

Looks like nifi makes use of Apache commons-lang which has some functions
that would do the heavy lifting. [1]

So if someone can help me understand the Expression Language end points I
could do the work fairly easily I think. And add functions for the other
formats supported by StringEscapeUtils (XML, csv, html).

I think this could be useful.

Devin

[1]
https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/StringEscapeUtils.html

Re: Build error with processor project

[Ben Meng]

Thanks Andrew. I don’t have any proxy settings, don’t even have settings.xml. I 
just tried again and it worked fine this time. Not sure what caused it, but I’m 
able to work on my customized processor now.

Thanks,
Ben




On 6/24/16, 4:32 AM, "Andrew Psaltis"  wrote:

>Hi Ben,
>I followed your steps and was able to build the testprocessor just fine.
>From looking at your output, specifically these lines:
>
>Downloading:
>https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.pom
>
>Downloaded:
>https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.pom
>(0
>B at 0.0 KB/sec)
>
>Downloading:
>https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.jar
>
>Downloaded:
>https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.jar
>(0
>B at 0.0 KB/sec)
>
>
>I wonder if it is a more general maven or system configuration issue that
>is causing the problem here. Seems like maven cannot pulldown plugins it
>needs. Do you have any proxy settings by chance in a ~/m2/settings.xml
>file? What version of maven are you using? If it helps here are the
>particulars of my environment:
>
>
>HW13095:testprocessor apsaltis$ mvn --version
>Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
>2015-11-10T11:41:47-05:00)
>Maven home: /usr/local/Cellar/maven/3.3.9/libexec
>Java version: 1.8.0_77, vendor: Oracle Corporation
>Java home:
>/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/jre
>Default locale: en_US, platform encoding: UTF-8
>OS name: "mac os x", version: "10.11.5", arch: "x86_64", family: "mac"
>
>
>Thanks,
>Andrew
>
>On Fri, Jun 24, 2016 at 2:48 AM, Ben Meng 
>wrote:
>
>> Hi,
>>
>> I created a project with Maven processor archetype (followed the
>> instruction on NiFi Wiki). When I tried to build it, I got a
>> NoClassDefFoundError. What am I missing?
>>
>> Thanks,
>> Ben
>>
>>
>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.nifi
>> -DarchetypeArtifactId=nifi-processor-bundle-archetype
>> -DarchetypeVersion=0.6.1 -DnifiVersion=0.6.1
>>
>> [INFO] Scanning for projects...
>>
>> [INFO]
>>
>> [INFO]
>> 
>>
>> [INFO] Building Maven Stub Project (No POM) 1
>>
>> [INFO]
>> 
>>
>> [INFO]
>>
>> [INFO] >>> maven-archetype-plugin:2.4:generate (default-cli) >
>> generate-sources @ standalone-pom >>>
>>
>> [INFO]
>>
>> [INFO] <<< maven-archetype-plugin:2.4:generate (default-cli) <
>> generate-sources @ standalone-pom <<<
>>
>> [INFO]
>>
>> [INFO] --- maven-archetype-plugin:2.4:generate (default-cli) @
>> standalone-pom ---
>>
>> [INFO] Generating project in Interactive mode
>>
>> [INFO] Archetype repository not defined. Using the one from
>> [org.apache.nifi:nifi-processor-bundle-archetype:0.6.1] found in catalog
>> remote
>>
>> Define value for property 'groupId': : com.mydomain
>>
>> Define value for property 'artifactId': : testprocessor
>>
>> Define value for property 'version':  1.0-SNAPSHOT: :
>>
>> Define value for property 'artifactBaseName': : testprocessor
>>
>> Define value for property 'package':
>> com.mydomain.processors.testprocessor: :
>>
>> [INFO] Using property: nifiVersion = 0.6.1
>>
>> Confirm properties configuration:
>>
>> groupId: com.mydomain
>>
>> artifactId: testprocessor
>>
>> version: 1.0-SNAPSHOT
>>
>> artifactBaseName: testprocessor
>>
>> package: com.mydomain.processors.testprocessor
>>
>> nifiVersion: 0.6.1
>>
>>  Y: :
>>
>> [INFO]
>> 
>>
>> [INFO] Using following parameters for creating project from Archetype:
>> nifi-processor-bundle-archetype:0.6.1
>>
>> [INFO]
>> 
>>
>> [INFO] Parameter: groupId, Value: com.mydomain
>>
>> [INFO] Parameter: artifactId, Value: testprocessor
>>
>> [INFO] Parameter: version, Value: 1.0-SNAPSHOT
>>
>> [INFO] Parameter: package, Value: com.mydomain.processors.testprocessor
>>
>> [INFO] Parameter: packageInPathFormat, Value:
>> com/mydomain/processors/testprocessor
>>
>> [INFO] Parameter: package, Value: com.mydomain.processors.testprocessor
>>
>> [INFO] Parameter: artifactBaseName, Value: testprocessor
>>
>> [INFO] Parameter: version, Value: 1.0-SNAPSHOT
>>
>> [INFO] Parameter: groupId, Value: com.mydomain
>>
>> [INFO] Parameter: artifactId, Value: testprocessor
>>
>> [INFO] Parameter: nifiVersion, Value: 0.6.1
>>
>> [INFO] project created from Archetype in dir:
>> /Users/ben.meng/IdeaProjects/test/testprocessor
>>
>> [INFO]
>> 
>>
>> [INFO] BUILD SUCCESS
>>
>> [INFO]
>> 

[GitHub] nifi issue #525: NIFI-1976 - Windows Event Log native processor

[brosander]

Github user brosander commented on the issue:

https://github.com/apache/nifi/pull/525
  
@JPercivall just pushed change to update duplicate logging in the same way 
as 
https://github.com/apache/nifi/pull/548/files#diff-a88d41c9ef1b881c177706d4cc4b0bf1R294


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: UI can take a very long time to become available

[ddewaele]

Thx for the tip ! Flow is up and running again. 




--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/UI-can-take-a-very-long-time-to-become-available-tp12201p12209.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Build error with processor project

[Andrew Psaltis]

Hi Ben,
I followed your steps and was able to build the testprocessor just fine.
>From looking at your output, specifically these lines:

Downloading:
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.pom

Downloaded:
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.pom
(0
B at 0.0 KB/sec)

Downloading:
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.jar

Downloaded:
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.jar
(0
B at 0.0 KB/sec)


I wonder if it is a more general maven or system configuration issue that
is causing the problem here. Seems like maven cannot pulldown plugins it
needs. Do you have any proxy settings by chance in a ~/m2/settings.xml
file? What version of maven are you using? If it helps here are the
particulars of my environment:


HW13095:testprocessor apsaltis$ mvn --version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
2015-11-10T11:41:47-05:00)
Maven home: /usr/local/Cellar/maven/3.3.9/libexec
Java version: 1.8.0_77, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.11.5", arch: "x86_64", family: "mac"


Thanks,
Andrew

On Fri, Jun 24, 2016 at 2:48 AM, Ben Meng 
wrote:

> Hi,
>
> I created a project with Maven processor archetype (followed the
> instruction on NiFi Wiki). When I tried to build it, I got a
> NoClassDefFoundError. What am I missing?
>
> Thanks,
> Ben
>
>
> $ mvn archetype:generate -DarchetypeGroupId=org.apache.nifi
> -DarchetypeArtifactId=nifi-processor-bundle-archetype
> -DarchetypeVersion=0.6.1 -DnifiVersion=0.6.1
>
> [INFO] Scanning for projects...
>
> [INFO]
>
> [INFO]
> 
>
> [INFO] Building Maven Stub Project (No POM) 1
>
> [INFO]
> 
>
> [INFO]
>
> [INFO] >>> maven-archetype-plugin:2.4:generate (default-cli) >
> generate-sources @ standalone-pom >>>
>
> [INFO]
>
> [INFO] <<< maven-archetype-plugin:2.4:generate (default-cli) <
> generate-sources @ standalone-pom <<<
>
> [INFO]
>
> [INFO] --- maven-archetype-plugin:2.4:generate (default-cli) @
> standalone-pom ---
>
> [INFO] Generating project in Interactive mode
>
> [INFO] Archetype repository not defined. Using the one from
> [org.apache.nifi:nifi-processor-bundle-archetype:0.6.1] found in catalog
> remote
>
> Define value for property 'groupId': : com.mydomain
>
> Define value for property 'artifactId': : testprocessor
>
> Define value for property 'version':  1.0-SNAPSHOT: :
>
> Define value for property 'artifactBaseName': : testprocessor
>
> Define value for property 'package':
> com.mydomain.processors.testprocessor: :
>
> [INFO] Using property: nifiVersion = 0.6.1
>
> Confirm properties configuration:
>
> groupId: com.mydomain
>
> artifactId: testprocessor
>
> version: 1.0-SNAPSHOT
>
> artifactBaseName: testprocessor
>
> package: com.mydomain.processors.testprocessor
>
> nifiVersion: 0.6.1
>
>  Y: :
>
> [INFO]
> 
>
> [INFO] Using following parameters for creating project from Archetype:
> nifi-processor-bundle-archetype:0.6.1
>
> [INFO]
> 
>
> [INFO] Parameter: groupId, Value: com.mydomain
>
> [INFO] Parameter: artifactId, Value: testprocessor
>
> [INFO] Parameter: version, Value: 1.0-SNAPSHOT
>
> [INFO] Parameter: package, Value: com.mydomain.processors.testprocessor
>
> [INFO] Parameter: packageInPathFormat, Value:
> com/mydomain/processors/testprocessor
>
> [INFO] Parameter: package, Value: com.mydomain.processors.testprocessor
>
> [INFO] Parameter: artifactBaseName, Value: testprocessor
>
> [INFO] Parameter: version, Value: 1.0-SNAPSHOT
>
> [INFO] Parameter: groupId, Value: com.mydomain
>
> [INFO] Parameter: artifactId, Value: testprocessor
>
> [INFO] Parameter: nifiVersion, Value: 0.6.1
>
> [INFO] project created from Archetype in dir:
> /Users/ben.meng/IdeaProjects/test/testprocessor
>
> [INFO]
> 
>
> [INFO] BUILD SUCCESS
>
> [INFO]
> 
>
> [INFO] Total time: 38.897 s
>
> [INFO] Finished at: 2016-06-23T23:02:47-07:00
>
> [INFO] Final Memory: 15M/245M
>
> [INFO]
> 
>
>
>
> $ cd testprocessor/
>
> $ mvn clean install
>
> [INFO] Scanning for projects...
>
> [INFO] Inspecting build with total of 3 modules...
>
> [INFO] Installing Nexus Staging features:
>
> 

Re: UI can take a very long time to become available

[Matt Gilman]

Hi,

Thanks for checking out Apache NiFi! In order to prevent components from
automatically resuming after startup you can disable this flag in your
nifi.properties:

nifi.flowcontroller.autoResumeState=true

Matt

On Fri, Jun 24, 2016 at 4:20 AM, ddewaele  wrote:

> I've  added a gist
>    to
> show you some logging threaddumps of when the issue occurs. In this
> particular case I was unable to bring up Nifi 
>
> I had to kill my mqtt broker so that the ConsumeMQTT processor wouldn't be
> picking up any new messages.
> Then I was able to start Nifi and enter the UI. However, I was unable to
> stop the ConsumeMQTT processor. (spinning refresh wheel of death).
>
> There is some activity in the logs but no msgs are processed and ui / api
> is
> unresponsive.
>
> nifi.sh dump also doesn't work due to a socket timeout (probably tries to
> contact the api).
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/UI-can-take-a-very-long-time-to-become-available-tp12201p12204.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: UI can take a very long time to become available

[Matthew Clarke]

The long startup times are the result of the number of queued FlowFile that
are being recovered/reloaded in to your flow on startup. There is an
existing JIRA ticket to address this process. nifi-32. Your out of memory
issues are related to the very large number of FlowFile you are loading in
to memory for your mergeContent processor. Try splitting that merge process
between two different mergeContent processors. First mergeContent merges
bundles of 10,000 and the second merges bundles of 100. The end result will
still be bundles of 1,000,000; however, the memory impact will be far less.

Matt
On Jun 24, 2016 5:29 AM, "ddewaele"  wrote:

> I've  added a gist
>    to
> show you some logging threaddumps of when the issue occurs. In this
> particular case I was unable to bring up Nifi 
>
> I had to kill my mqtt broker so that the ConsumeMQTT processor wouldn't be
> picking up any new messages.
> Then I was able to start Nifi and enter the UI. However, I was unable to
> stop the ConsumeMQTT processor. (spinning refresh wheel of death).
>
> There is some activity in the logs but no msgs are processed and ui / api
> is
> unresponsive.
>
> nifi.sh dump also doesn't work due to a socket timeout (probably tries to
> contact the api).
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/UI-can-take-a-very-long-time-to-become-available-tp12201p12204.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: UI can take a very long time to become available

[ddewaele]

I've  added a gist
   to
show you some logging threaddumps of when the issue occurs. In this
particular case I was unable to bring up Nifi 

I had to kill my mqtt broker so that the ConsumeMQTT processor wouldn't be
picking up any new messages.
Then I was able to start Nifi and enter the UI. However, I was unable to
stop the ConsumeMQTT processor. (spinning refresh wheel of death).

There is some activity in the logs but no msgs are processed and ui / api is
unresponsive.

nifi.sh dump also doesn't work due to a socket timeout (probably tries to
contact the api).






--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/UI-can-take-a-very-long-time-to-become-available-tp12201p12204.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

UI can take a very long time to become available

[ddewaele]

I have a simple dataflow running on 1.0.0-SNAPSHOT that looks like this

ConsumeMQTT -> MergeContent (batch them up by 100 items) -> PutHDFS

The ConsumeMQTT is receiving about 100 msgs per 5 minutes.

>From time to time I need to restart Nifi because it seems to hang / starts
processing very slowly. I believe this is due to memory related issues
(sometimes I see OutOfMemory errors, but I already have it running on a 6GB
heap).

The problem is that when restarting the flow, it sometimes takes a very very
long time before the UI comes up. 

I had to restart the flow when there were 100 msgs in the queue (50MB)
between ConsumeMQTT / MergeContent. (For some reason ConsumeMQTT was
grinding to a halt).

After restarting it took over 7 minutes for the UI to come up.



Sometimes the UI doesn't come up at all, even after waiting for 20 minutes.
At that point you see similar messages in the logs. Sometimes if fails with
an out of memory. At that point I never get the opportunity to stop the flow
as I cannot access the UI / API.

Any tips on optimizing memory usage and are there any plans to make the UI /
API available immediately upon startup so that corrective actions can take
place before nifi starts grinding to a halt ?







--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/UI-can-take-a-very-long-time-to-become-available-tp12201.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Build error with processor project

[Ben Meng]

Hi,

I created a project with Maven processor archetype (followed the instruction on 
NiFi Wiki). When I tried to build it, I got a NoClassDefFoundError. What am I 
missing?

Thanks,
Ben


$ mvn archetype:generate -DarchetypeGroupId=org.apache.nifi 
-DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=0.6.1 
-DnifiVersion=0.6.1

[INFO] Scanning for projects...

[INFO]

[INFO] 

[INFO] Building Maven Stub Project (No POM) 1

[INFO] 

[INFO]

[INFO] >>> maven-archetype-plugin:2.4:generate (default-cli) > generate-sources 
@ standalone-pom >>>

[INFO]

[INFO] <<< maven-archetype-plugin:2.4:generate (default-cli) < generate-sources 
@ standalone-pom <<<

[INFO]

[INFO] --- maven-archetype-plugin:2.4:generate (default-cli) @ standalone-pom 
---

[INFO] Generating project in Interactive mode

[INFO] Archetype repository not defined. Using the one from 
[org.apache.nifi:nifi-processor-bundle-archetype:0.6.1] found in catalog remote

Define value for property 'groupId': : com.mydomain

Define value for property 'artifactId': : testprocessor

Define value for property 'version':  1.0-SNAPSHOT: :

Define value for property 'artifactBaseName': : testprocessor

Define value for property 'package':  com.mydomain.processors.testprocessor: :

[INFO] Using property: nifiVersion = 0.6.1

Confirm properties configuration:

groupId: com.mydomain

artifactId: testprocessor

version: 1.0-SNAPSHOT

artifactBaseName: testprocessor

package: com.mydomain.processors.testprocessor

nifiVersion: 0.6.1

 Y: :

[INFO] 


[INFO] Using following parameters for creating project from Archetype: 
nifi-processor-bundle-archetype:0.6.1

[INFO] 


[INFO] Parameter: groupId, Value: com.mydomain

[INFO] Parameter: artifactId, Value: testprocessor

[INFO] Parameter: version, Value: 1.0-SNAPSHOT

[INFO] Parameter: package, Value: com.mydomain.processors.testprocessor

[INFO] Parameter: packageInPathFormat, Value: 
com/mydomain/processors/testprocessor

[INFO] Parameter: package, Value: com.mydomain.processors.testprocessor

[INFO] Parameter: artifactBaseName, Value: testprocessor

[INFO] Parameter: version, Value: 1.0-SNAPSHOT

[INFO] Parameter: groupId, Value: com.mydomain

[INFO] Parameter: artifactId, Value: testprocessor

[INFO] Parameter: nifiVersion, Value: 0.6.1

[INFO] project created from Archetype in dir: 
/Users/ben.meng/IdeaProjects/test/testprocessor

[INFO] 

[INFO] BUILD SUCCESS

[INFO] 

[INFO] Total time: 38.897 s

[INFO] Finished at: 2016-06-23T23:02:47-07:00

[INFO] Final Memory: 15M/245M

[INFO] 



$ cd testprocessor/

$ mvn clean install

[INFO] Scanning for projects...

[INFO] Inspecting build with total of 3 modules...

[INFO] Installing Nexus Staging features:

[INFO]   ... total of 3 executions of maven-deploy-plugin replaced with 
nexus-staging-maven-plugin

[INFO] 

[INFO] Reactor Build Order:

[INFO]

[INFO] testprocessor

[INFO] nifi-testprocessor-processors

[INFO] nifi-testprocessor-nar

[INFO]

[INFO] 

[INFO] Building testprocessor 1.0-SNAPSHOT

[INFO] 

Downloading: 
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.pom

Downloaded: 
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.pom
 (0 B at 0.0 KB/sec)

Downloading: 
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.jar

Downloaded: 
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.5.2/maven-install-plugin-2.5.2.jar
 (0 B at 0.0 KB/sec)

[INFO]

[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ testprocessor ---

[INFO]

[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-maven) @ testprocessor 
---

[INFO]

[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ testprocessor 
---

[INFO]

[INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ testprocessor 
---

[INFO] No sources to compile

[INFO]

[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
testprocessor ---

[WARNING] Error injecting: 
org.apache.maven.plugins.site.SiteDescriptorAttachMojo

java.lang.NoClassDefFoundError: Lorg/apache/maven/doxia/tools/SiteTool;

at