[GitHub] nifi pull request: NIFI-1180: Modify PutS3Object to enable encrypt...

[alopresto]

Github user alopresto commented on the pull request:

https://github.com/apache/nifi/pull/246#issuecomment-206705777
  
@adamonduty Will do in the morning. Don't want to miss merging into both 
branches now that `1.0` and `0.x` have split. (Provided docs on that on [the 
wiki](https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-Stepstomerge/closepullrequestswithtwomainbranches).
 ) Those commits will also be **GPG-signed** now that [GitHub supports 
it](https://help.github.com/articles/signing-commits-using-gpg/). 

Thanks for reminding me. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1578: Create PutSlack processor

[adamonduty]

Github user adamonduty commented on the pull request:

https://github.com/apache/nifi/pull/256#issuecomment-206686328
  
@joewitt I'm only now just realizing that my reply to your comment was in 
the wrong place. Here's my reply which I originally made on the JIRA:

==
My use case around PutSlack at this point is pretty simple. Basically the 
same use cases around PutEmail, but sending to a Slack channel instead of an 
email distribution list. This allows teams to keep an eye on their dataflows in 
a natural place, especially since many other Slack integrations exist (Jenkins, 
github, etc).

To configure PutSlack, a Slack user (or maybe admin) has to add the 
Incoming Webhooks integration to the Slack team, then add a webhook to a 
specific channel. Once the user has the URL associated with this webhook, they 
drop the URL into PutSlack along with a webhook text template.

Does that help? The use cases would probably also increase with a GetSlack 
processor, which when paired with PutSlack could allow one to manually trigger 
flows in NiFi via a chat bot.

==

Also, just to be clear, the Travis build above failed on JDK8 due to an out 
of memory exception unrelated to PutSlack. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1180: Modify PutS3Object to enable encrypt...

[adamonduty]

Github user adamonduty commented on the pull request:

https://github.com/apache/nifi/pull/246#issuecomment-206685104
  
@alopresto What do you think about merging this one with your new 
committing superpowers? :)  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1630 Making TestPutUDP select an open port...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/332


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DRAFT][REPORT] Apache NiFi - April 2016

[Matt Burgess]

+1, impressed with community pace, productivity, engagement, and enthusiasm, 
the report/numbers definitely reflect this.

> On Apr 6, 2016, at 11:20 PM, Aldrin Piri  wrote:
> 
> Joe, looks great overall.  Minor point is that the vote for 0.6.0 wrapped
> up Saturday, 26 March.
> 
> Awesome growth for the community.  Certainly felt like that was the case,
> nice to see actual numbers.
> 
>> On Wed, Apr 6, 2016 at 10:55 PM, Joe Witt  wrote:
>> 
>> NiFi Community,
>> 
>> Please review the included April 2016 report for the Apache board.
>> If anything seems inaccurate or missing please advise.
>> 
>> Thanks
>> Joe
>> 
>> --
>> 
>> Report from the Apache NiFi committee [Joe Witt]
>> 
>> ## Description:
>> - Apache NiFi is an easy to use, powerful, and reliable system to process
>> and distribute data.
>> 
>> ## Issues:
>> - There are no issues requiring board attention at this time.
>> 
>> ## Activity:
>> - Released NiFi 0.5 with considerable new core features and extensions.
>> - Released NiFi 0.6 adding features to set stage for 1.0 release.
>> - Substantial progress made toward the 1.0 release line which is now
>> being built on master.
>> - Meetups featuring NiFi talks have occurred in the reporting period in
>> Santa Clara, Chicago, WashingtonDC, New York City, Baltimore,
>> Toronto, Dublin, Denmark, Montreal
>> - Established child project known as 'MiNiFi' to provide an
>> alternative command and control and implementation model tailored
>> toward the first mile challenges of data acquisition. JIRA and Git
>> infrastructure setup and contributions underway.
>> 
>> ## Health report:
>> - Activity on mailing lists, JIRA, Git continues trending well.
>> - During the last review cycle several new committers have joined.
>> - Pipeline for PMC progression is very strong.
>> 
>> ## PMC changes:
>> 
>> - Currently 13 PMC members.
>> - No new PMC members in the last three months.
>> - Last PMC addition was Sean Busbey on Wed Nov 25 2015
>> 
>> ## Committer base changes:
>> 
>> - Currently 21 committers.
>> - New commmitters:
>>- Andy LoPresto was added as a committer on Sat Apr 02 2016
>>- Oleg Zhurakousky was added as a committer on Wed Mar 16 2016
>>- Matt Burgess was added as a committer on Fri Mar 04 2016
>>- Joe Skora was added as a committer on Mon Feb 15 2016
>> 
>> ## Releases:
>> 
>> - nifi-0.6.0 was released on Fri Mar 25 2016
>> - nifi-0.5.1 was released on Thu Feb 25 2016
>> - nifi-0.5.0 was released on Mon Feb 15 2016
>> 
>> ## Mailing list activity:
>> 
>> - us...@nifi.apache.org:
>>- 262 subscribers (up 56 in the last 3 months):
>>- 832 emails sent to list (709 in previous quarter)
>> 
>> - dev@nifi.apache.org:
>>- 231 subscribers (up 23 in the last 3 months):
>>- 2610 emails sent to list (1389 in previous quarter)
>> 
>> ## JIRA activity:
>> 
>> - 388 JIRA tickets created in the last 3 months
>> - 237 JIRA tickets closed/resolved in the last 3 months
>> 

Re: [DRAFT][REPORT] Apache NiFi - April 2016

[Aldrin Piri]

Joe, looks great overall.  Minor point is that the vote for 0.6.0 wrapped
up Saturday, 26 March.

Awesome growth for the community.  Certainly felt like that was the case,
nice to see actual numbers.

On Wed, Apr 6, 2016 at 10:55 PM, Joe Witt  wrote:

> NiFi Community,
>
> Please review the included April 2016 report for the Apache board.
> If anything seems inaccurate or missing please advise.
>
> Thanks
> Joe
>
> --
>
> Report from the Apache NiFi committee [Joe Witt]
>
> ## Description:
>  - Apache NiFi is an easy to use, powerful, and reliable system to process
>  and distribute data.
>
> ## Issues:
>  - There are no issues requiring board attention at this time.
>
> ## Activity:
>  - Released NiFi 0.5 with considerable new core features and extensions.
>  - Released NiFi 0.6 adding features to set stage for 1.0 release.
>  - Substantial progress made toward the 1.0 release line which is now
>  being built on master.
>  - Meetups featuring NiFi talks have occurred in the reporting period in
>  Santa Clara, Chicago, WashingtonDC, New York City, Baltimore,
>  Toronto, Dublin, Denmark, Montreal
>  - Established child project known as 'MiNiFi' to provide an
>  alternative command and control and implementation model tailored
>  toward the first mile challenges of data acquisition. JIRA and Git
>  infrastructure setup and contributions underway.
>
> ## Health report:
>  - Activity on mailing lists, JIRA, Git continues trending well.
>  - During the last review cycle several new committers have joined.
>  - Pipeline for PMC progression is very strong.
>
> ## PMC changes:
>
>  - Currently 13 PMC members.
>  - No new PMC members in the last three months.
>  - Last PMC addition was Sean Busbey on Wed Nov 25 2015
>
> ## Committer base changes:
>
>  - Currently 21 committers.
>  - New commmitters:
> - Andy LoPresto was added as a committer on Sat Apr 02 2016
> - Oleg Zhurakousky was added as a committer on Wed Mar 16 2016
> - Matt Burgess was added as a committer on Fri Mar 04 2016
> - Joe Skora was added as a committer on Mon Feb 15 2016
>
> ## Releases:
>
>  - nifi-0.6.0 was released on Fri Mar 25 2016
>  - nifi-0.5.1 was released on Thu Feb 25 2016
>  - nifi-0.5.0 was released on Mon Feb 15 2016
>
> ## Mailing list activity:
>
>  - us...@nifi.apache.org:
> - 262 subscribers (up 56 in the last 3 months):
> - 832 emails sent to list (709 in previous quarter)
>
>  - dev@nifi.apache.org:
> - 231 subscribers (up 23 in the last 3 months):
> - 2610 emails sent to list (1389 in previous quarter)
>
> ## JIRA activity:
>
>  - 388 JIRA tickets created in the last 3 months
>  - 237 JIRA tickets closed/resolved in the last 3 months
>

Re: [VOTE] Release Apache NiFi 0.6.1 (RC1)

[Aldrin Piri]

+1, binding

Build and tests worked on OS X and Windows 7.  Branch looked good on our
Travis build and ran some flow templates from reviews from some of the
associated issues with anticipated results.

Verified hashes and signatures.

On Wed, Apr 6, 2016 at 5:33 PM, Joe Witt  wrote:

> Hello Apache NiFi Community,
>
> I am pleased to be calling this vote for the source release of Apache
> NiFi 0.6.1.
>
> The source zip, including signatures, digests, etc. can be found at:
>  https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.1/
>
> The Git tag is nifi-0.6.1-RC1
> The Git commit hash is d51b24e18356b76ce649c8285af0806acd9071d0
> *
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=d51b24e18356b76ce649c8285af0806acd9071d0
> *
> https://github.com/apache/nifi/commit/d51b24e18356b76ce649c8285af0806acd9071d0
>
> Checksums of nifi-0.6.1-source-release.zip:
> MD5: b82a7fe60e03b679d7c06f57319fa396
> SHA1: 32c45d51d1e1858eaba1df889711c69c17b44781
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/joewitt.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 11 issues were closed/resolved for this release:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12335496
> Release note highlights can be found here:
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.1
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build from source, and test. Then
> please vote:
>
> [ ] +1 Release this package as nifi-0.6.1
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
> Thanks!
>

[DRAFT][REPORT] Apache NiFi - April 2016

[Joe Witt]

NiFi Community,

Please review the included April 2016 report for the Apache board.
If anything seems inaccurate or missing please advise.

Thanks
Joe

--

Report from the Apache NiFi committee [Joe Witt]

## Description:
 - Apache NiFi is an easy to use, powerful, and reliable system to process
 and distribute data.

## Issues:
 - There are no issues requiring board attention at this time.

## Activity:
 - Released NiFi 0.5 with considerable new core features and extensions.
 - Released NiFi 0.6 adding features to set stage for 1.0 release.
 - Substantial progress made toward the 1.0 release line which is now
 being built on master.
 - Meetups featuring NiFi talks have occurred in the reporting period in
 Santa Clara, Chicago, WashingtonDC, New York City, Baltimore,
 Toronto, Dublin, Denmark, Montreal
 - Established child project known as 'MiNiFi' to provide an
 alternative command and control and implementation model tailored
 toward the first mile challenges of data acquisition. JIRA and Git
 infrastructure setup and contributions underway.

## Health report:
 - Activity on mailing lists, JIRA, Git continues trending well.
 - During the last review cycle several new committers have joined.
 - Pipeline for PMC progression is very strong.

## PMC changes:

 - Currently 13 PMC members.
 - No new PMC members in the last three months.
 - Last PMC addition was Sean Busbey on Wed Nov 25 2015

## Committer base changes:

 - Currently 21 committers.
 - New commmitters:
- Andy LoPresto was added as a committer on Sat Apr 02 2016
- Oleg Zhurakousky was added as a committer on Wed Mar 16 2016
- Matt Burgess was added as a committer on Fri Mar 04 2016
- Joe Skora was added as a committer on Mon Feb 15 2016

## Releases:

 - nifi-0.6.0 was released on Fri Mar 25 2016
 - nifi-0.5.1 was released on Thu Feb 25 2016
 - nifi-0.5.0 was released on Mon Feb 15 2016

## Mailing list activity:

 - us...@nifi.apache.org:
- 262 subscribers (up 56 in the last 3 months):
- 832 emails sent to list (709 in previous quarter)

 - dev@nifi.apache.org:
- 231 subscribers (up 23 in the last 3 months):
- 2610 emails sent to list (1389 in previous quarter)

## JIRA activity:

 - 388 JIRA tickets created in the last 3 months
 - 237 JIRA tickets closed/resolved in the last 3 months

[GitHub] nifi pull request: NIFI-1630 Making TestPutUDP select an open port...

[bbende]

GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/332

NIFI-1630 Making TestPutUDP select an open port for testing to avoid 
conflicts



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-1630

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/332.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #332


commit 13523bd08a07534d130987cbe5cd612a55be7c9a
Author: Bryan Bende 
Date:   2016-04-06T19:09:39Z

NIFI-1630 Making TestPutUDP select an open port for testing to avoid 
conflicts




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Adding requirejs, angularjs and its dependencie...

[joewitt]

Github user joewitt commented on the pull request:

https://github.com/apache/nifi/pull/331#issuecomment-206589245
  
The author email appears tied to a local system name.  Probably not what 
you wanted.

There doesn't appear to be updates to the LICENSE/NOTICE of NiFi which will 
be needed given this is a source dependency.  It appears to include a lot of 
cruft like README files and gitignore files and so on which seem avoidable.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Apache NiFi 0.6.1 RC1 Release Helper Guide

[Joe Witt]

Hello Apache NiFi community,

Please find the associated guidance to help those interested in
validating/verifying the release so they can vote.

# Download latest KEYS file:
  https://dist.apache.org/repos/dist/dev/nifi/KEYS

# Import keys file:
  gpg --import KEYS

# [optional] Clear out local maven artifact repository

# Pull down nifi-0.6.1 source release artifacts for review:

  wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.1/nifi-0.6.1-source-release.zip
  wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.1/nifi-0.6.1-source-release.zip.asc
  wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.1/nifi-0.6.1-source-release.zip.md5
  wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.1/nifi-0.6.1-source-release.zip.sha1

# Verify the signature
  gpg --verify nifi-0.6.1-source-release.zip.asc

# Verify the hashes (md5, sha1) match the source and what was provided
in the vote email thread
  md5sum nifi-0.6.1-source-release.zip
  sha1sum nifi-0.6.1-source-release.zip

# Unzip nifi-0.6.1-source-release.zip

# Verify the build works including release audit tool (RAT) checks
  cd nifi-0.6.1
  mvn clean install -Pcontrib-check

# Verify the contents contain a good README, NOTICE, and LICENSE.

# Verify the git commit ID is correct

# Verify the RC was branched off the correct git commit ID

# Look at the resulting convenience binary as found in nifi-assembly/target

# Make sure the README, NOTICE, and LICENSE are present and correct

# Run the resulting convenience binary and make sure it works as expected

# Send a response to the vote thread indicating a +1, 0, -1 based on
your findings.

Thank you for your time and effort to validate the release!

[GitHub] nifi pull request: NIFI-1583 Added ConvertJSONtoCSV processor

[joewitt]

Github user joewitt commented on the pull request:

https://github.com/apache/nifi/pull/254#issuecomment-206581942
  
I am not sure we can use org.json:json dependency.  Its license as found in 
the readme of https://github.com/stleary/JSON-java/blob/master/README says  
"The license includes this restriction: "The software shall be used for good,
not evil." If your conscience cannot live with that, then choose a different
package.""

I'm not sure this is acceptable in that it doesn't claim to be public 
domain or be part of any approved ASF friendly license.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[VOTE] Release Apache NiFi 0.6.1 (RC1)

[Joe Witt]

Hello Apache NiFi Community,

I am pleased to be calling this vote for the source release of Apache
NiFi 0.6.1.

The source zip, including signatures, digests, etc. can be found at:
 https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.1/

The Git tag is nifi-0.6.1-RC1
The Git commit hash is d51b24e18356b76ce649c8285af0806acd9071d0
* 
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=d51b24e18356b76ce649c8285af0806acd9071d0
* https://github.com/apache/nifi/commit/d51b24e18356b76ce649c8285af0806acd9071d0

Checksums of nifi-0.6.1-source-release.zip:
MD5: b82a7fe60e03b679d7c06f57319fa396
SHA1: 32c45d51d1e1858eaba1df889711c69c17b44781

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/joewitt.asc

KEYS file available here:
https://dist.apache.org/repos/dist/release/nifi/KEYS

11 issues were closed/resolved for this release:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12335496
Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.1

The vote will be open for 72 hours.
Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[ ] +1 Release this package as nifi-0.6.1
[ ] +0 no opinion
[ ] -1 Do not release this package because...

Thanks!

[GitHub] nifi pull request: Adding requirejs, angularjs and its dependencie...

[scottyaslan]

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/331

Adding requirejs, angularjs and its dependencies, and manually boostr…

…apping an angular app to the body

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi responsiveDevBranch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/331.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #331


commit e8499212fc2aeaa735fe26ff8febd84be8663c47
Author: James Aslan 
Date:   2016-04-06T20:37:35Z

Adding requirejs, angularjs and its dependencies, and manually boostrapping 
an angular app to the body




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1678

[markap14]

Github user markap14 commented on the pull request:

https://github.com/apache/nifi/pull/323#issuecomment-206513343
  
I updated the PR with a new commit to address an issue that I ran into. 
When we create 'container' nodes in ZooKeeper whose paths are not sensitive, we 
will mark the node as wide-open, rather than applying ACLs. This allows us to 
not have to worry about if we have state management and heartbeats using 
different ACLs.

Also removed use of username/passwords from ZooKeeper configuration. This 
was done because ZooKeeper indicates that these should be used only for local 
testing and not for production. In production, security should be administered 
via Kerberos. Username and password were always sent over the wire unencrypted. 
Additionally, we are now using Apache Curator to handle Leader Election, and 
Curator does not appear to support using digest auth via username/password.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1678

[markap14]

Github user markap14 commented on the pull request:

https://github.com/apache/nifi/pull/323#issuecomment-206513196
  
I updated the PR with a new commit to address an issue that I ran into. 
When we create 'container' nodes in ZooKeeper whose paths are not sensitive, we 
will mark the node as wide-open, rather than applying ACLs. This allows us to 
not have to worry about if we have state management and heartbeats using 
different ACLs.

Also removed use of username/passwords from ZooKeeper configuration. This 
was done because ZooKeeper indicates that these should be used only for local 
testing and not for production. In production, security should be administered 
via Kerberos. Username and password were always sent over the wire unencrypted. 
Additionally, we are now using Apache Curator to handle Leader Election, and 
Curator does not appear to support using digest auth via username/password.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi/pull/5


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1678

[markap14]

Github user markap14 commented on the pull request:

https://github.com/apache/nifi/pull/317#issuecomment-206489781
  
I updated the PR with a new commit to address an issue that I ran into. 
When we create 'container' nodes in ZooKeeper whose paths are not sensitive, we 
will mark the node as wide-open, rather than applying ACLs. This allows us to 
not have to worry about if we have state management and heartbeats using 
different ACLs.

Also removed use of username/passwords from ZooKeeper configuration. This 
was done because ZooKeeper indicates that these should be used only for local 
testing and not for production. In production, security should be administered 
via Kerberos. Username and password were always sent over the wire unencrypted. 
Additionally, we are now using Apache Curator to handle Leader Election, and 
Curator does not appear to support using digest auth via username/password.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 added state to UpdateAttribute as wel...

[JPercivall]

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/319#issuecomment-206472004
  
@mattyb149 I did what you suggested, did a soft reset, re-committed (making 
a new hash like @apiri pointed out) and did a force push. It's rebuilding now


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58744217
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework-nar/pom.xml ---
@@ -0,0 +1,67 @@
+
+
+http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+
+minifi-framework-bundle
+org.apache.nifi.minifi
+0.0.1-SNAPSHOT
+
+4.0.0
+
+org.apache.nifi.minifi
+minifi-framework-nar
--- End diff --

That works for me


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58744060
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/MiNiFi.java
 ---
@@ -0,0 +1,236 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.Thread.UncaughtExceptionHandler;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.util.Timer;
+import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.apache.nifi.NiFiServer;
+import org.apache.nifi.documentation.DocGenerator;
+import org.apache.nifi.nar.ExtensionManager;
+import org.apache.nifi.nar.ExtensionMapping;
+import org.apache.nifi.nar.NarClassLoaders;
+import org.apache.nifi.nar.NarUnpacker;
+import org.apache.nifi.util.FileUtils;
+import org.apache.nifi.util.NiFiProperties;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.slf4j.bridge.SLF4JBridgeHandler;
+
+public class MiNiFi {
--- End diff --

Forgot to point out that the BootstrapListener's log and variable names 
also refer to nifi instead of minifi. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[apiri]

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58742228
  
--- Diff: minifi-assembly/pom.xml ---
@@ -96,160 +85,250 @@ limitations under the License.
 slf4j-api
 compile
 
+
+
 
-ch.qos.logback
-logback-classic
-compile
+org.apache.nifi.minifi
+minifi-resources
+resources
+runtime
+zip
+0.0.1-SNAPSHOT
+
+
+org.apache.nifi.minifi
+minifi-bootstrap
+0.0.1-SNAPSHOT
+
+
+org.apache.nifi.minifi
+minifi-runtime
+0.0.1-SNAPSHOT
+
+
+
+
+org.apache.nifi
+nifi-processor-utils
 
-
 
+
+
+org.apache.nifi
+nifi-api
+
+
+org.apache.nifi
+nifi-runtime
+
+
+org.apache.nifi
+nifi-bootstrap
--- End diff --

It is not, and I have taken some time to additionally defeat some other 
dependencies.  These were likely remnants of a failed experiment.  Good find!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[apiri]

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58742147
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/MiNiFi.java
 ---
@@ -0,0 +1,236 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.Thread.UncaughtExceptionHandler;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.util.Timer;
+import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.apache.nifi.NiFiServer;
+import org.apache.nifi.documentation.DocGenerator;
+import org.apache.nifi.nar.ExtensionManager;
+import org.apache.nifi.nar.ExtensionMapping;
+import org.apache.nifi.nar.NarClassLoaders;
+import org.apache.nifi.nar.NarUnpacker;
+import org.apache.nifi.util.FileUtils;
+import org.apache.nifi.util.NiFiProperties;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.slf4j.bridge.SLF4JBridgeHandler;
+
+public class MiNiFi {
--- End diff --

Certainly will adjust.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[apiri]

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58742154
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework-nar/pom.xml ---
@@ -0,0 +1,67 @@
+
+
+http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+
+minifi-framework-bundle
+org.apache.nifi.minifi
+0.0.1-SNAPSHOT
+
+4.0.0
+
+org.apache.nifi.minifi
+minifi-framework-nar
--- End diff --

Was modeling after the nifi-framework-nar that is currently bundled through 
some of iterations.  In the end, given the reuse approach, just a few of these 
libraries are utilized in lieu of the full NAR.  I think that transition will 
happen later and would like to keep this available moving forward as other 
folks may wish to contribute to the core framework and things are shifted from 
NiFi to MiNiFi specific efforts.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58732157
  
--- Diff: minifi-assembly/pom.xml ---
@@ -96,160 +85,250 @@ limitations under the License.
 slf4j-api
 compile
 
+
+
 
-ch.qos.logback
-logback-classic
-compile
+org.apache.nifi.minifi
+minifi-resources
+resources
+runtime
+zip
+0.0.1-SNAPSHOT
+
+
+org.apache.nifi.minifi
+minifi-bootstrap
+0.0.1-SNAPSHOT
+
+
+org.apache.nifi.minifi
+minifi-runtime
+0.0.1-SNAPSHOT
+
+
+
+
+org.apache.nifi
+nifi-processor-utils
 
-
 
+
+
+org.apache.nifi
+nifi-api
+
+
+org.apache.nifi
+nifi-runtime
+
+
+org.apache.nifi
+nifi-bootstrap
--- End diff --

Why is nifi-bootstrap needed? Isn't minifi-bootstrap replacing that? 

Also nifi-bootstrap is explicitly included in the bootstrap lib in 
minifi-assembly's dependencies.xml.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58728826
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework-nar/pom.xml ---
@@ -0,0 +1,67 @@
+
+
+http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+
+minifi-framework-bundle
+org.apache.nifi.minifi
+0.0.1-SNAPSHOT
+
+4.0.0
+
+org.apache.nifi.minifi
+minifi-framework-nar
--- End diff --

This package is effectively not used. Running a "find" for 
"minifi-framework-nar" on the project only turns up two places, here and and in 
minifi-framework-bundle pom declaring minifi-framwork-nar as a module.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/5#discussion_r58712548
  
--- Diff: 
minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/MiNiFi.java
 ---
@@ -0,0 +1,236 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.Thread.UncaughtExceptionHandler;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.util.Timer;
+import java.util.TimerTask;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.apache.nifi.NiFiServer;
+import org.apache.nifi.documentation.DocGenerator;
+import org.apache.nifi.nar.ExtensionManager;
+import org.apache.nifi.nar.ExtensionMapping;
+import org.apache.nifi.nar.NarClassLoaders;
+import org.apache.nifi.nar.NarUnpacker;
+import org.apache.nifi.util.FileUtils;
+import org.apache.nifi.util.NiFiProperties;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.slf4j.bridge.SLF4JBridgeHandler;
+
+public class MiNiFi {
--- End diff --

All of the log statements and exception messages refer to "NiFi" instead of 
"MiNiFi".


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Compression of Data in HDFS

[Bryan Bende]

Ok one more question...

On GetHDFS are you setting the Directory to
"\landing\databasename\prodeiw_arc\"
and then setting Recurse Sub-Directories to true to have it go into each
table's directory?

The reason I ask is because the FlowFiles coming out of GetHDFS have an
attribute on them called Path, the documentation says:

The path is set to the relative path of the file's directory on HDFS. For
example, if the Directory property is set to /tmp, then files picked up
from /tmp will have the path attribute set to "./". If the Recurse
Subdirectories property is set to true and a file is picked up from
/tmp/abc/1/2/3, then the path attribute will be set to "abc/1/2/3"

So theoretically if you were pointing to "\landing\databasename\prodeiw_arc\"
and then it recursed into "\landing\databasename\prodeiw_arc\tablename",
the path attribute would end up being "tablename".

You could then reference this in your PutHDFS processor by setting the
Directory to "/landing/teradata/compressed/prodeiw_arc/$[path}"



On Wed, Apr 6, 2016 at 8:46 AM, jamesgreen 
wrote:

> Hi Brian, Thanks for the help!
>
> I have tried two ways
> a.
> 1.  I use GetHDFS to retrieve data from the HDFS , I then use putHDFS
> and set
> the compression to GZIP.
> 2.  In the Directory I am putting the complete path i.e
> /landing/teradata/compressed/prodeiw_arc
> b.
> 1.   I use GetHDFS to retrieve data from the HDFS, I then use Compress
> Content to apply the compression and then use PutHDFS
> 2.  In the Directory I am putting the complete path i.e
> /landing/teradata/compressed/prodeiw_arc
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/Compression-of-Data-in-HDFS-tp8821p8825.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

[GitHub] nifi pull request: NIFI-1582 added state to UpdateAttribute as wel...

[apiri]

Github user apiri commented on the pull request:

https://github.com/apache/nifi/pull/319#issuecomment-206384283
  
There are no facilities to manually start a Travis job.  You could likely 
squash your two commits and push those.  The core thing is that a new hash is 
seen for the Travis webhooks, I believe.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 added state to UpdateAttribute as wel...

[mattyb149]

Github user mattyb149 commented on the pull request:

https://github.com/apache/nifi/pull/319#issuecomment-206383566
  
There was something that could be done, I can't remember. Maybe reset 
--soft HEAD^ and re-commit and push -f. Will give the same code a new commit 
SHA and should kick it off again. I thought it was simpler to do but maybe not. 
Also maybe we can re-run the Travis job manually?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1582 added state to UpdateAttribute as wel...

[JPercivall]

Github user JPercivall commented on the pull request:

https://github.com/apache/nifi/pull/319#issuecomment-206383057
  
@mattyb149 just tried pushing with no new commits and nothing happened. I 
believe I could push an empty commit but I don't think it's worth cluttering 
this PR with unnecessary commits. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Compression of Data in HDFS

[jamesgreen]

Hi Brian, Thanks for the help!

I have tried two ways 
a.
1.  I use GetHDFS to retrieve data from the HDFS , I then use putHDFS and 
set
the compression to GZIP.
2.  In the Directory I am putting the complete path i.e
/landing/teradata/compressed/prodeiw_arc
b.
1.   I use GetHDFS to retrieve data from the HDFS, I then use Compress
Content to apply the compression and then use PutHDFS
2.  In the Directory I am putting the complete path i.e
/landing/teradata/compressed/prodeiw_arc




--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Compression-of-Data-in-HDFS-tp8821p8825.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

[GitHub] nifi pull request: NIFI-1690 Changed MonitorMemory to use allowabl...

[joewitt]

Github user joewitt commented on a diff in the pull request:

https://github.com/apache/nifi/pull/328#discussion_r58706787
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-reporting-tasks/src/main/java/org/apache/nifi/controller/MonitorMemory.java
 ---
@@ -91,12 +92,22 @@
 + " that the memory pool is exceeding this threshold.")
 public class MonitorMemory extends AbstractReportingTask {
 
+private static final AllowableValue[] memPoolAllowableValues;
+
+static {
+List memoryPoolBeans = 
ManagementFactory.getMemoryPoolMXBeans();
+memPoolAllowableValues = new 
AllowableValue[memoryPoolBeans.size()];
+for (int i = 0; i < memPoolAllowableValues.length; i++) {
+memPoolAllowableValues[i] = new 
AllowableValue(memoryPoolBeans.get(i).getName());
+}
+}
+
 public static final PropertyDescriptor MEMORY_POOL_PROPERTY = new 
PropertyDescriptor.Builder()
 .name("Memory Pool")
 .description("The name of the JVM Memory Pool to monitor")
 .required(true)
-.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
-.defaultValue(null)
+.allowableValues(memPoolAllowableValues)
+.defaultValue(memPoolAllowableValues.length == 0 ? null : 
memPoolAllowableValues[0].getValue())
--- End diff --

I think we're better off simply not having a default value.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Compression of Data in HDFS

[Bryan Bende]

Hello,

Can you describe your flow a bit more?

Are you using ListHDFS + FetchHDFS to retrieve the data from HDFS?

What value do you have for the Directory property in PutHDFS?

Thanks,

Bryan

On Wed, Apr 6, 2016 at 7:12 AM, jamesgreen 
wrote:

> I am trying to compress a whole lot of files from my HDFS and write to
> another folder on the HDFS
> My Folder Structure is as follows:
> \landing\databasename\prodeiw_arc\tablename\_SUCCESS
> \landing\databasename\prodeiw_arc\tablename\part-m-0
>
> \landing\databasename\prodeiw_arc\tablename2\_SUCCESS
> \landing\databasename\prodeiw_arc\tablename2\part-m-0
>
> I am trying to compress to the following
> \landing\compressed\prodeiw_arc\tablename\_SUCCESS
> \landing\compressed\prodeiw_arc\tablename\part-m-0
>
> \landing\compressed\prodeiw_arc\tablename2\_SUCCESS
> \landing\compressed\prodeiw_arc\tablename2\part-m-0
>
> I have found that it compresses to
> \landing\compressed\prodeiw_arc\_SUCCESS
> \landing\compressed\prodeiw_arc\tablename\part-m-0
>
> it will then continue to overwrite. Is there anyway I can keep the
> directory
> structure when doing a PutHDFS?
>
> Thanks and Regards
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/Compression-of-Data-in-HDFS-tp8821.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: How to pass variables to nifi Processor

[Bryan Bende]

Hello,

For properties that support expression language, they can reference
properties passed in through the bootstrap.conf file.
So you could define a property in bootstrap.conf like -Dfoo=abc and in a
processor property reference that as $[foo}.

There are plans for this be improved in the future through the idea of a
variable registry:
https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry

You may also want to take a look at this project that provides a way to
deploy templates from one instance to another:
https://github.com/aperepel/nifi-api-deploy

Thanks,

Bryan


On Tue, Apr 5, 2016 at 1:32 PM, Rajeswari Raghunathan - Contractor <
rajeswari.raghunathan.contrac...@8451.com> wrote:

> Hi Team,
>
> We are using NIFI in our company to migrate all data from SFTP server to
> HDFS.
> We have different environment like dev, test, prod and  want to accomplish
> Continuous integration with NIFI. In order to do so, we need to separate
> all sensitive data like server hostname,username,password from hardcoding
> in Processor (eg:-GETSFTP).
> Can anyone help me by providing best solution for this problem?
>
> Regards,
> Rajeswari
>

Compression of Data in HDFS

[jamesgreen]

I am trying to compress a whole lot of files from my HDFS and write to
another folder on the HDFS
My Folder Structure is as follows:
\landing\databasename\prodeiw_arc\tablename\_SUCCESS
\landing\databasename\prodeiw_arc\tablename\part-m-0

\landing\databasename\prodeiw_arc\tablename2\_SUCCESS
\landing\databasename\prodeiw_arc\tablename2\part-m-0

I am trying to compress to the following
\landing\compressed\prodeiw_arc\tablename\_SUCCESS
\landing\compressed\prodeiw_arc\tablename\part-m-0

\landing\compressed\prodeiw_arc\tablename2\_SUCCESS
\landing\compressed\prodeiw_arc\tablename2\part-m-0

I have found that it compresses to 
\landing\compressed\prodeiw_arc\_SUCCESS
\landing\compressed\prodeiw_arc\tablename\part-m-0

it will then continue to overwrite. Is there anyway I can keep the directory
structure when doing a PutHDFS?

Thanks and Regards




--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Compression-of-Data-in-HDFS-tp8821.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

[GitHub] nifi-minifi pull request: MINIFI-5: Base MiNiFi executable

[apiri]

GitHub user apiri opened a pull request:

https://github.com/apache/nifi-minifi/pull/5

MINIFI-5: Base MiNiFi executable

MINIFI-5:  Creating a base MiNiFi project to serve as a basis for further 
extension and design reusing NiFi libraries

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apiri/nifi-minifi minifi-5

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi/pull/5.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5


commit e94b6dae3dcb18bd60c9234de62543c897e83146
Author: Aldrin Piri 
Date:   2016-04-06T01:19:57Z

MINIFI-5:  Creating a base MiNiFi project to serve as a basis for further 
extension and design reusing NiFi libraries




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---