date:20160321

Re: [VOTE] Release Apache NiFi 0.6.0 (RC1)

2016-03-21 Thread Andy LoPresto

Thanks to Aldrin and Matt Burgess, we were able to push a new signature for 
each artifact to the repository and verify it. Please resume release 
verification.

hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 0s @ 20:01:08 $ rmf nifi-0.6.0-source-release.zip.asc
nifi-0.6.0-source-release.zip.asc
hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 0s @ 20:01:14 $ wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip.asc
--2016-03-21 20:01:16--  
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip.asc
Resolving dist.apache.org... 209.188.14.144
Connecting to dist.apache.org|209.188.14.144|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 801 [text/plain]
Saving to: 'nifi-0.6.0-source-release.zip.asc'

nifi-0.6.0-source-release.zip.asc 
100%[>] 801  
--.-KB/sin 0s

2016-03-21 20:01:16 (34.7 MB/s) - 'nifi-0.6.0-source-release.zip.asc' saved 
[801/801]

hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 0s @ 20:01:17 $ diff nifi-0.6.0-source-release.zip.asc aldrin-new.asc
hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 0s @ 20:01:29 $ gpg --verify -v aldrin-new.asc nifi-0.6.0-source-release.zip
gpg: Signature made Mon Mar 21 19:39:05 2016 PDT using RSA key ID 4CFE5D00
gpg: using PGP trust model
gpg: Good signature from "Aldrin Piri (Code Signing Key) " 
[full]
gpg: binary signature, digest algorithm SHA512
hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 0s @ 20:01:38 $ gpg --verify -v nifi-0.6.0-source-release.zip.asc 
nifi-0.6.0-source-release.zip
gpg: Signature made Mon Mar 21 19:39:05 2016 PDT using RSA key ID 4CFE5D00
gpg: using PGP trust model
gpg: Good signature from "Aldrin Piri (Code Signing Key) " 
[full]
gpg: binary signature, digest algorithm SHA512
hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 0s @ 20:01:50 $

hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 0s @ 20:01:50 $ wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-bin.zip.asc
--2016-03-21 20:03:43--  
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-bin.zip.asc
Resolving dist.apache.org... 209.188.14.144
Connecting to dist.apache.org|209.188.14.144|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 801 [text/plain]
Saving to: 'nifi-0.6.0-bin.zip.asc'

nifi-0.6.0-bin.zip.asc
100%[>] 801  
--.-KB/sin 0s

2016-03-21 20:03:43 (27.3 MB/s) - 'nifi-0.6.0-bin.zip.asc' saved [801/801]

hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 0s @ 20:03:44 $ wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-bin.zip
--2016-03-21 20:03:47--  
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-bin.zip
Resolving dist.apache.org... 209.188.14.144
Connecting to dist.apache.org|209.188.14.144|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 441687095 (421M) [application/octet-stream]
Saving to: 'nifi-0.6.0-bin.zip'

nifi-0.6.0-bin.zip
100%[>] 421.23M  
10.7MB/sin 44s

2016-03-21 20:04:31 (9.59 MB/s) - 'nifi-0.6.0-bin.zip' saved 
[441687095/441687095]

hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 44s @ 20:04:32 $ gpg --verify -v nifi-0.6.0-bin.zip.asc
gpg: assuming signed data in 'nifi-0.6.0-bin.zip'
gpg: Signature made Mon Mar 21 19:49:21 2016 PDT using RSA key ID 4CFE5D00
gpg: using PGP trust model
gpg: Good signature from "Aldrin Piri (Code Signing Key) " 
[full]
gpg: binary signature, digest algorithm SHA512
hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 2s @ 20:05:04 $

hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 2s @ 20:05:04 $ wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-bin.tar.gz
--2016-03-21 20:06:34--  
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-bin.tar.gz
Resolving dist.apache.org... 209.188.14.144
Connecting to dist.apache.org|209.188.14.144|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 441632655 (421M) [application/octet-stream]
Saving to: 'nifi-0.6.0-bin.tar.gz'

nifi-0.6.0-bin.tar.gz 
100%[>] 421.17M  
10.3MB/sin 40s

2016-03-21 20:07:15 (10.4 MB/s) - 'nifi-0.6.0-bin.tar.gz' saved 
[441632655/441632655]

hw12203:/Users/alopresto/Workspace/scratch/release_verification/0.6.0 alopresto
 41s @ 20:07:16 $ wget

Re: [VOTE] Release Apache NiFi 0.6.0 (RC1)

2016-03-21 Thread Aldrin Piri

Hey Matt,

Thanks for the heads up.  Something is definitely awry.  While verifying
seemingly checks out in my environment:

$ gpg --verify nifi-0.6.0-source-release.zip.asc
> gpg: assuming signed data in `nifi-0.6.0-source-release.zip'
> gpg: Signature made Mon Mar 21 14:38:50 2016 EDT using RSA key ID 4CFE5D00
> gpg: Good signature from "Aldrin Piri (Code Signing Key) <
> ald...@apache.org>"


I receive similar errors on another system I did not perform/verify the
release process on.  For now, I ask that folks likely hold up until I can
resolve the issue and determine if it was a matter of publishing or the
entire build and another RC is needed.


On Mon, Mar 21, 2016 at 9:44 PM, Matt Burgess  wrote:

> I'm getting an error on Aldrin's signature during gpg --verify:
>
> gpg: BAD signature from "Aldrin Piri (Code Signing Key)  >"
> [unknown]
>
> Is anyone else seeing this?
>
> Thanks,
> Matt
>
> On Mon, Mar 21, 2016 at 4:37 PM, Aldrin Piri  wrote:
>
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > nifi-0.6.0.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1077
> >
> > The Git tag is NIFI-1634-RC1
> > The Git commit hash is 736896246cf021dbed31d4eb1e22e0755e4705f0
> > *
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=736896246cf021dbed31d4eb1e22e0755e4705f0
> > *
> >
> >
> https://github.com/apache/nifi/commit/736896246cf021dbed31d4eb1e22e0755e4705f0
> >
> > Checksums of nifi-0.6.0-source-release.zip:
> > MD5: 1559157db000d53221aeabc5dd607cfc
> > SHA1: feed12016d7f2d450fb1e3a238634757cc17b0f1
> >
> > Release artifacts are signed with the following key:
> > *https://people.apache.org/keys/committer/aldrin.asc
> > *
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 71 issues were closed/resolved for this release:
> > *
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12334372=Html=12316020
> > <
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12334372=Html=12316020
> > >*
> >
> > Release note highlights can be found here:
> > *
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.0
> > <
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.0
> > >*
> >
> > The vote will be open for 72 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [ ] +1 Release this package as nifi-0.6.0
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
>

[GitHub] nifi pull request: NIFI-1620 Allow empty Content-Type in InvokeHTT...

2016-03-21 Thread taftster

Github user taftster commented on a diff in the pull request:

https://github.com/apache/nifi/pull/272#discussion_r56928549
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
 ---
@@ -444,14 +455,27 @@ public void onPropertyModified(final 
PropertyDescriptor descriptor, final String
 
 @Override
 protected Collection customValidate(final 
ValidationContext validationContext) {
-final List results = new ArrayList<>(1);
+final List results = new 
ArrayList(3);
+
+// proxy host and port validation
 final boolean proxyHostSet = 
validationContext.getProperty(PROP_PROXY_HOST).isSet();
 final boolean proxyPortSet = 
validationContext.getProperty(PROP_PROXY_PORT).isSet();
-
 if ((proxyHostSet && !proxyPortSet) || (!proxyHostSet && 
proxyPortSet)) {
 results.add(new ValidationResult.Builder().subject("Proxy Host 
and Port").valid(false).explanation("If Proxy Host or Proxy Port is set, both 
must be set").build());
 }
 
+// body and content-type validation
+final boolean contentTypeSet = 
validationContext.getProperty(PROP_CONTENT_TYPE).isSet();
+final boolean sendBody = 
validationContext.getProperty(PROP_SEND_BODY).asBoolean();
+final String contentType = 
validationContext.getProperty(PROP_CONTENT_TYPE).getValue();
+if(contentTypeSet && contentType.isEmpty() && sendBody) {
+results.add(new 
ValidationResult.Builder().subject("Content-Type and Send body")
+.valid(false).explanation("If Content-Type is set to 
empty, Send body property must be set to false").build());
+}
+if(!sendBody && !contentType.isEmpty()) {
+results.add(new 
ValidationResult.Builder().subject("Content-Type and Send 
body").valid(false).explanation("If body is not sent, Content-Type must be set 
to empty").build());
+}
+
 return results;
--- End diff --

This custom validation rule can be removed if the content-type property is 
left alone.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1620 Allow empty Content-Type in InvokeHTT...

2016-03-21 Thread taftster

Github user taftster commented on a diff in the pull request:

https://github.com/apache/nifi/pull/272#discussion_r56928543
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
 ---
@@ -215,14 +215,24 @@
 .build();
 
 public static final PropertyDescriptor PROP_CONTENT_TYPE = new 
PropertyDescriptor.Builder()
-.name("Content-Type")
-.description("The Content-Type to specify for when content is 
being transmitted through a PUT or POST. "
-+ "In the case of an empty value after evaluating an 
expression language expression, Content-Type defaults to " + 
DEFAULT_CONTENT_TYPE)
-.required(true)
-.expressionLanguageSupported(true)
-.defaultValue("${" + CoreAttributes.MIME_TYPE.key() + "}")
-.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
-.build();
+.name("Content-Type")
+.description("The Content-Type to specify for when content is 
being transmitted through a PUT or POST. "
++ "In the case of an empty value after evaluating an 
expression language expression, Content-Type defaults to " + 
DEFAULT_CONTENT_TYPE + "."
++ "If and only if body is not sent, Content-Type must 
be set to empty")
+.required(false)
+.expressionLanguageSupported(true)
+.defaultValue("${" + CoreAttributes.MIME_TYPE.key() + "}")
+
.addValidator(StandardValidators.createAttributeExpressionLanguageValidator(AttributeExpression.ResultType.STRING))
+.build();
+
--- End diff --

I'm not sure if we even need this change.  In the case that the 
PROP_SEND_BODY is false, we should always suppress setting the Content-Type 
header.  Therefore, I would leave this as-is a required field, with 
documentation in the PROP_SEND_FIELD property describing the behavior for empty 
message bodies.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1620 Allow empty Content-Type in InvokeHTT...

2016-03-21 Thread taftster

Github user taftster commented on a diff in the pull request:

https://github.com/apache/nifi/pull/272#discussion_r56928547
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
 ---
@@ -215,14 +215,24 @@
 .build();
 
 public static final PropertyDescriptor PROP_CONTENT_TYPE = new 
PropertyDescriptor.Builder()
-.name("Content-Type")
-.description("The Content-Type to specify for when content is 
being transmitted through a PUT or POST. "
-+ "In the case of an empty value after evaluating an 
expression language expression, Content-Type defaults to " + 
DEFAULT_CONTENT_TYPE)
-.required(true)
-.expressionLanguageSupported(true)
-.defaultValue("${" + CoreAttributes.MIME_TYPE.key() + "}")
-.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
-.build();
+.name("Content-Type")
+.description("The Content-Type to specify for when content is 
being transmitted through a PUT or POST. "
++ "In the case of an empty value after evaluating an 
expression language expression, Content-Type defaults to " + 
DEFAULT_CONTENT_TYPE + "."
++ "If and only if body is not sent, Content-Type must 
be set to empty")
+.required(false)
+.expressionLanguageSupported(true)
+.defaultValue("${" + CoreAttributes.MIME_TYPE.key() + "}")
+
.addValidator(StandardValidators.createAttributeExpressionLanguageValidator(AttributeExpression.ResultType.STRING))
+.build();
+
+public static final PropertyDescriptor PROP_SEND_BODY = new 
PropertyDescriptor.Builder()
+.name("Send body")
+.description("True if the body content should be sent, false 
otherwise")
+.defaultValue("true")
--- End diff --

We should refer to this as the HTTP spec refers to it.  It's called the 
"message-body" in the spec, so I think we should use that here as well.  i.e.

```
.name("send-message-body")
.displayName("Send Message Body")
.description("If true, sends the HTTP message body on POST/PUT requests 
(default).  If false, suppresses the message body and content-type header for 
these requests.")
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [VOTE] Release Apache NiFi 0.6.0 (RC1)

2016-03-21 Thread Matt Burgess

I'm getting an error on Aldrin's signature during gpg --verify:

gpg: BAD signature from "Aldrin Piri (Code Signing Key) "
[unknown]

Is anyone else seeing this?

Thanks,
Matt

On Mon, Mar 21, 2016 at 4:37 PM, Aldrin Piri  wrote:

> Hello,
>
> I am pleased to be calling this vote for the source release of Apache NiFi
> nifi-0.6.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1077
>
> The Git tag is NIFI-1634-RC1
> The Git commit hash is 736896246cf021dbed31d4eb1e22e0755e4705f0
> *
>
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=736896246cf021dbed31d4eb1e22e0755e4705f0
> *
>
> https://github.com/apache/nifi/commit/736896246cf021dbed31d4eb1e22e0755e4705f0
>
> Checksums of nifi-0.6.0-source-release.zip:
> MD5: 1559157db000d53221aeabc5dd607cfc
> SHA1: feed12016d7f2d450fb1e3a238634757cc17b0f1
>
> Release artifacts are signed with the following key:
> *https://people.apache.org/keys/committer/aldrin.asc
> *
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 71 issues were closed/resolved for this release:
> *
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12334372=Html=12316020
> <
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12334372=Html=12316020
> >*
>
> Release note highlights can be found here:
> *
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.0
> <
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.0
> >*
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build from source, and test. Then
> please vote:
>
> [ ] +1 Release this package as nifi-0.6.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
> Thanks!
>

Re: Apache NiFi 0.6.0 RC1 Release Helper Guide

2016-03-21 Thread Andy LoPresto

There was a copy/paste error in the previous message. The fourth URI in the 
“source release artifacts” section should be: 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip.sha1

> # Pull down nifi-0.6.0 source release artifacts for review:
> 
>  wget
> *https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip
> *
>  wget
> *https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip.asc
> *
>  wget
> *https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip.md5
> *
>  wget
> *https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip.sha1
> *

Thanks everyone.

Andy LoPresto
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Mar 21, 2016, at 1:38 PM, Aldrin Piri  wrote:
> 
> Hello Apache NiFi community,
> 
> Please find the associated guidance to help those interested in
> validating/verifying the
> release so they can vote.
> 
> # Download latest KEYS file:
>  https://dist.apache.org/repos/dist/dev/nifi/KEYS
> 
> # Import keys file:
>  gpg --import KEYS
> 
> # [optional] Clear out local maven artifact repository
> 
> # Pull down nifi-0.6.0 source release artifacts for review:
> 
>  wget
> *https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip
> *
>  wget
> *https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip.asc
> *
>  wget
> *https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip.md5
> *
>  wget
> *https://dist.apache.org/repos/dist/dev/nifi/nifi-0.6.0/nifi-0.6.0-source-release.zip.md5
> *
> 
> # Verify the signature
>  gpg --verify nifi-0.6.0-source-release.zip.asc
> 
> # Verify the hashes (md5 and sha1) match the source and what was provided
> in the vote email thread
>  md5sum nifi-0.6.0-source-release.zip
>  sha1sum nifi-0.6.0-source-release.zip
> 
> # Unzip nifi-0.6.0-source-release.zip
> 
> # Verify the build works including release audit tool (RAT) checks
>  cd nifi-0.6.0
>  mvn clean install -Pcontrib-check
> 
> # Verify the contents contain a good README, NOTICE, and LICENSE.
> 
> # Verify the git commit ID is correct
> 
> # Verify the RC was branched off the correct git commit ID
> 
> # Look at the resulting convenience binary as found in nifi-assembly/target
> 
> # Make sure the README, NOTICE, and LICENSE are present and correct
> 
> # Run the resulting convenience binary and make sure it works as expected
> 
> # Send a response to the vote thread indicating a +1, 0, -1 based on your
> findings.
> 
> Thank you for your time and effort to validate the release!



signature.asc
Description: Message signed with OpenPGP using GPGMail

Processor additional documentation

2016-03-21 Thread Bryan Bende

I think we opted for the reverse which was to document how to remove the
parent when you don't want it.

It should be on this page somewhere:

https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions

On Monday, March 21, 2016, Matt Burgess > wrote:

> Oops, missed that for Uwe's PR because I have the snapshot installed. But
> the parent specification is in the maven archetype:
>
>
> https://github.com/apache/nifi/blob/master/nifi-maven-archetypes/nifi-processor-bundle-archetype/src/main/resources/archetype-resources/pom.xml#L21
>
> Should it be commented out with some inline doc about when to use it?
>
> On Mon, Mar 21, 2016 at 5:11 PM, Matt Gilman 
> wrote:
>
> > This is happening because your bundle pom is referencing the
> > nifi-nar-bundles as its parent. This is unresolvable because SNAPSHOT
> > artifacts are not released to any artifact repositories. So this would
> have
> > to be built locally...
> >
> > That said, you shouldn't need to use this as your parent pom. All of
> these
> > examples do because they exist within the NiFi source tree. Your bundle
> > will likely live in your own source tree with your own parent pom. Unless
> > it's some that will be contributed back in which case you should probably
> > clone the NiFi repo and build within there.
> >
> > Typically, parent poms will be used for configuring plugin/dependency
> > management and build configuration/infrastructure. Let us know if that
> > helps.
> >
> > Matt
> >
> > On Mon, Mar 21, 2016 at 4:50 PM, Uwe Geercken 
> wrote:
> >
> > > Matt,
> > >
> > > sorry to bug you again. I guess I am not good at the maven stuff. Will
> > > still have to learn a lot...
> > >
> > > when I import the top level into eclipse, I get following error:
> > >
> > > [INFO] Scanning for projects...
> > > [ERROR] [ERROR] Some problems were encountered while processing the
> POMs:
> > > [FATAL] Non-resolvable parent POM for
> > > org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT: Could not find
> > > artifact org.apache.nifi:nifi-nar-bundles:pom:0.6.0-SNAPSHOT and
> > > 'parent.relativePath' points at wrong local POM @ line 19, column 13
> > >  @
> > > [ERROR] The build could not read 1 project -> [Help 1]
> > > [ERROR]
> > > [ERROR]   The project
> org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT
> > >
> (/home/uwe/development/git/nifi_processors/nifi-datamelt-bundle/pom.xml)
> > > has 1 error
> > > [ERROR] Non-resolvable parent POM for
> > > org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT: Could not find
> > > artifact org.apache.nifi:nifi-nar-bundles:pom:0.6.0-SNAPSHOT and
> > > 'parent.relativePath' points at wrong local POM @ line 19, column 13 ->
> > > [Help 2]
> > >
> > >
> > > Any quick advice before I start diggin in?
> > >
> > > Greetings Uwe
> > >
> > >
> > > > Gesendet: Montag, 21. März 2016 um 18:25 Uhr
> > > > Von: "Matt Burgess" 
> > > > An: dev@nifi.apache.org
> > > > Betreff: Re: Aw: Re: Re: Re: Re: Processor additional documentation
> > > >
> > > > For that pull request I added POMs at each level, you can run mvn
> > > install from the top bundle and it will build the NAR under the
> > > nifi-datamelt-nar/target folder.
> > > >
> > > > You can import the top level POM into your IDE of choice :)
> > > >
> > > > Regards,
> > > > Matt
> > > >
> > > > > On Mar 21, 2016, at 1:21 PM, Uwe Geercken 
> > wrote:
> > > > >
> > > > > I am sorry for that. I have completely overlooked that.
> > > > >
> > > > > So tell me Matt how is it: I create an automatic package with
> eclipse
> > > and maven and then manually put it in an archive? or is there a
> complete
> > > automatic approach?
> > > > >
> > > > > Anything I have to specifically do with maven? And is there
> > > documentation available?
> > > > >
> > > > > greetings and thanks for help.
> > > > >
> > > > > Uwe
> > > > >
> > > > >> Gesendet: Montag, 21. März 2016 um 14:04 Uhr
> > > > >> Von: "Matt Burgess" 
> > > > >> An: dev@nifi.apache.org
> > > > >> Betreff: Re: Re: Re: Re: Processor additional documentation
> > > > >>
> > > > >> Uwe,
> > > > >>
> > > > >> The additional details piece appears to be a result of your ".nar"
> > > file
> > > > >> actually being more like a ".jar", rather than a bundle that
> > includes
> > > a JAR
> > > > >> which in turn includes your source code and docs.  Since you did
> all
> > > the
> > > > >> hard work with creating some useful processors, I took the liberty
> > of
> > > > >> moving some of your project stuff around into the NAR structure
> the
> > > folks
> > > > >> have been referring to:
> > > > >>
> > > > >> https://github.com/uwegeercken/nifi_processors/pull/1
> > > > >>
> > > > >> This will build a NAR that contains (among other things) a JAR
> with
> > > the
> > > > >> classes, docs, and other processor resources, and is bundled such
>

Re: Re: Re: Re: Re: Re: Processor additional documentation

2016-03-21 Thread Matt Burgess

Oops, missed that for Uwe's PR because I have the snapshot installed. But
the parent specification is in the maven archetype:

https://github.com/apache/nifi/blob/master/nifi-maven-archetypes/nifi-processor-bundle-archetype/src/main/resources/archetype-resources/pom.xml#L21

Should it be commented out with some inline doc about when to use it?

On Mon, Mar 21, 2016 at 5:11 PM, Matt Gilman 
wrote:

> This is happening because your bundle pom is referencing the
> nifi-nar-bundles as its parent. This is unresolvable because SNAPSHOT
> artifacts are not released to any artifact repositories. So this would have
> to be built locally...
>
> That said, you shouldn't need to use this as your parent pom. All of these
> examples do because they exist within the NiFi source tree. Your bundle
> will likely live in your own source tree with your own parent pom. Unless
> it's some that will be contributed back in which case you should probably
> clone the NiFi repo and build within there.
>
> Typically, parent poms will be used for configuring plugin/dependency
> management and build configuration/infrastructure. Let us know if that
> helps.
>
> Matt
>
> On Mon, Mar 21, 2016 at 4:50 PM, Uwe Geercken  wrote:
>
> > Matt,
> >
> > sorry to bug you again. I guess I am not good at the maven stuff. Will
> > still have to learn a lot...
> >
> > when I import the top level into eclipse, I get following error:
> >
> > [INFO] Scanning for projects...
> > [ERROR] [ERROR] Some problems were encountered while processing the POMs:
> > [FATAL] Non-resolvable parent POM for
> > org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT: Could not find
> > artifact org.apache.nifi:nifi-nar-bundles:pom:0.6.0-SNAPSHOT and
> > 'parent.relativePath' points at wrong local POM @ line 19, column 13
> >  @
> > [ERROR] The build could not read 1 project -> [Help 1]
> > [ERROR]
> > [ERROR]   The project org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT
> > (/home/uwe/development/git/nifi_processors/nifi-datamelt-bundle/pom.xml)
> > has 1 error
> > [ERROR] Non-resolvable parent POM for
> > org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT: Could not find
> > artifact org.apache.nifi:nifi-nar-bundles:pom:0.6.0-SNAPSHOT and
> > 'parent.relativePath' points at wrong local POM @ line 19, column 13 ->
> > [Help 2]
> >
> >
> > Any quick advice before I start diggin in?
> >
> > Greetings Uwe
> >
> >
> > > Gesendet: Montag, 21. März 2016 um 18:25 Uhr
> > > Von: "Matt Burgess" 
> > > An: dev@nifi.apache.org
> > > Betreff: Re: Aw: Re: Re: Re: Re: Processor additional documentation
> > >
> > > For that pull request I added POMs at each level, you can run mvn
> > install from the top bundle and it will build the NAR under the
> > nifi-datamelt-nar/target folder.
> > >
> > > You can import the top level POM into your IDE of choice :)
> > >
> > > Regards,
> > > Matt
> > >
> > > > On Mar 21, 2016, at 1:21 PM, Uwe Geercken 
> wrote:
> > > >
> > > > I am sorry for that. I have completely overlooked that.
> > > >
> > > > So tell me Matt how is it: I create an automatic package with eclipse
> > and maven and then manually put it in an archive? or is there a complete
> > automatic approach?
> > > >
> > > > Anything I have to specifically do with maven? And is there
> > documentation available?
> > > >
> > > > greetings and thanks for help.
> > > >
> > > > Uwe
> > > >
> > > >> Gesendet: Montag, 21. März 2016 um 14:04 Uhr
> > > >> Von: "Matt Burgess" 
> > > >> An: dev@nifi.apache.org
> > > >> Betreff: Re: Re: Re: Re: Processor additional documentation
> > > >>
> > > >> Uwe,
> > > >>
> > > >> The additional details piece appears to be a result of your ".nar"
> > file
> > > >> actually being more like a ".jar", rather than a bundle that
> includes
> > a JAR
> > > >> which in turn includes your source code and docs.  Since you did all
> > the
> > > >> hard work with creating some useful processors, I took the liberty
> of
> > > >> moving some of your project stuff around into the NAR structure the
> > folks
> > > >> have been referring to:
> > > >>
> > > >> https://github.com/uwegeercken/nifi_processors/pull/1
> > > >>
> > > >> This will build a NAR that contains (among other things) a JAR with
> > the
> > > >> classes, docs, and other processor resources, and is bundled such
> > that the
> > > >> framework can find everything it needs. I tested this and the
> > Additional
> > > >> Details links work correctly.  Cheers!
> > > >>
> > > >> Regards,
> > > >> Matt
> > > >>
> > > >>> On Sun, Mar 20, 2016 at 1:31 PM, Joe Witt 
> > wrote:
> > > >>>
> > > >>> Uwe
> > > >>>
> > > >>> Noticed your other threads on great progress.  That is awesome.
> > > >>>
> > > >>> Really want to help you get to the bottom of the additional details
> > > >>> piece though.  We clearly have to do a better job with documenting
> > (or
> > > >>> implementing) how to do

Re: Re: Re: Re: Re: Re: Processor additional documentation

2016-03-21 Thread Matt Gilman

This is happening because your bundle pom is referencing the
nifi-nar-bundles as its parent. This is unresolvable because SNAPSHOT
artifacts are not released to any artifact repositories. So this would have
to be built locally...

That said, you shouldn't need to use this as your parent pom. All of these
examples do because they exist within the NiFi source tree. Your bundle
will likely live in your own source tree with your own parent pom. Unless
it's some that will be contributed back in which case you should probably
clone the NiFi repo and build within there.

Typically, parent poms will be used for configuring plugin/dependency
management and build configuration/infrastructure. Let us know if that
helps.

Matt

On Mon, Mar 21, 2016 at 4:50 PM, Uwe Geercken  wrote:

> Matt,
>
> sorry to bug you again. I guess I am not good at the maven stuff. Will
> still have to learn a lot...
>
> when I import the top level into eclipse, I get following error:
>
> [INFO] Scanning for projects...
> [ERROR] [ERROR] Some problems were encountered while processing the POMs:
> [FATAL] Non-resolvable parent POM for
> org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT: Could not find
> artifact org.apache.nifi:nifi-nar-bundles:pom:0.6.0-SNAPSHOT and
> 'parent.relativePath' points at wrong local POM @ line 19, column 13
>  @
> [ERROR] The build could not read 1 project -> [Help 1]
> [ERROR]
> [ERROR]   The project org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT
> (/home/uwe/development/git/nifi_processors/nifi-datamelt-bundle/pom.xml)
> has 1 error
> [ERROR] Non-resolvable parent POM for
> org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT: Could not find
> artifact org.apache.nifi:nifi-nar-bundles:pom:0.6.0-SNAPSHOT and
> 'parent.relativePath' points at wrong local POM @ line 19, column 13 ->
> [Help 2]
>
>
> Any quick advice before I start diggin in?
>
> Greetings Uwe
>
>
> > Gesendet: Montag, 21. März 2016 um 18:25 Uhr
> > Von: "Matt Burgess" 
> > An: dev@nifi.apache.org
> > Betreff: Re: Aw: Re: Re: Re: Re: Processor additional documentation
> >
> > For that pull request I added POMs at each level, you can run mvn
> install from the top bundle and it will build the NAR under the
> nifi-datamelt-nar/target folder.
> >
> > You can import the top level POM into your IDE of choice :)
> >
> > Regards,
> > Matt
> >
> > > On Mar 21, 2016, at 1:21 PM, Uwe Geercken  wrote:
> > >
> > > I am sorry for that. I have completely overlooked that.
> > >
> > > So tell me Matt how is it: I create an automatic package with eclipse
> and maven and then manually put it in an archive? or is there a complete
> automatic approach?
> > >
> > > Anything I have to specifically do with maven? And is there
> documentation available?
> > >
> > > greetings and thanks for help.
> > >
> > > Uwe
> > >
> > >> Gesendet: Montag, 21. März 2016 um 14:04 Uhr
> > >> Von: "Matt Burgess" 
> > >> An: dev@nifi.apache.org
> > >> Betreff: Re: Re: Re: Re: Processor additional documentation
> > >>
> > >> Uwe,
> > >>
> > >> The additional details piece appears to be a result of your ".nar"
> file
> > >> actually being more like a ".jar", rather than a bundle that includes
> a JAR
> > >> which in turn includes your source code and docs.  Since you did all
> the
> > >> hard work with creating some useful processors, I took the liberty of
> > >> moving some of your project stuff around into the NAR structure the
> folks
> > >> have been referring to:
> > >>
> > >> https://github.com/uwegeercken/nifi_processors/pull/1
> > >>
> > >> This will build a NAR that contains (among other things) a JAR with
> the
> > >> classes, docs, and other processor resources, and is bundled such
> that the
> > >> framework can find everything it needs. I tested this and the
> Additional
> > >> Details links work correctly.  Cheers!
> > >>
> > >> Regards,
> > >> Matt
> > >>
> > >>> On Sun, Mar 20, 2016 at 1:31 PM, Joe Witt 
> wrote:
> > >>>
> > >>> Uwe
> > >>>
> > >>> Noticed your other threads on great progress.  That is awesome.
> > >>>
> > >>> Really want to help you get to the bottom of the additional details
> > >>> piece though.  We clearly have to do a better job with documenting
> (or
> > >>> implementing) how to do this.  Do you have any more details to share
> > >>> on symptoms you're seeing?
> > >>>
> > >>> Thanks
> > >>> Joe
> > >>>
> >  On Fri, Mar 18, 2016 at 5:35 PM, Uwe Geercken 
> wrote:
> >  Dan,
> > 
> >  ok. I was wrong. The index file is created - it's my
> > >>> additionalDetails.html file that is missing. I have no idea what is
> wrong.
> > 
> >  I will try it tomorrow - maybe I will find something with a clear
> head.
> > 
> >  Rgds,
> > 
> >  Uwe
> > 
> > > Gesendet: Freitag, 18. März 2016 um 19:14 Uhr
> > > Von: "dan bress" 
> > > An: dev@nifi.apache.org
> >

Aw: Re: Re: Re: Re: Re: Processor additional documentation

2016-03-21 Thread Uwe Geercken

Matt,

sorry to bug you again. I guess I am not good at the maven stuff. Will still 
have to learn a lot...

when I import the top level into eclipse, I get following error:

[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for 
org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT: Could not find artifact 
org.apache.nifi:nifi-nar-bundles:pom:0.6.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 19, column 13
 @ 
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT 
(/home/uwe/development/git/nifi_processors/nifi-datamelt-bundle/pom.xml) has 1 
error
[ERROR] Non-resolvable parent POM for 
org.apache.nifi:nifi-datamelt-bundle:0.6.0-SNAPSHOT: Could not find artifact 
org.apache.nifi:nifi-nar-bundles:pom:0.6.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 19, column 13 -> [Help 2]


Any quick advice before I start diggin in?

Greetings Uwe


> Gesendet: Montag, 21. März 2016 um 18:25 Uhr
> Von: "Matt Burgess" 
> An: dev@nifi.apache.org
> Betreff: Re: Aw: Re: Re: Re: Re: Processor additional documentation
>
> For that pull request I added POMs at each level, you can run mvn install 
> from the top bundle and it will build the NAR under the 
> nifi-datamelt-nar/target folder. 
> 
> You can import the top level POM into your IDE of choice :)
> 
> Regards,
> Matt
> 
> > On Mar 21, 2016, at 1:21 PM, Uwe Geercken  wrote:
> > 
> > I am sorry for that. I have completely overlooked that.
> > 
> > So tell me Matt how is it: I create an automatic package with eclipse and 
> > maven and then manually put it in an archive? or is there a complete 
> > automatic approach?
> > 
> > Anything I have to specifically do with maven? And is there documentation 
> > available?
> > 
> > greetings and thanks for help.
> > 
> > Uwe
> > 
> >> Gesendet: Montag, 21. März 2016 um 14:04 Uhr
> >> Von: "Matt Burgess" 
> >> An: dev@nifi.apache.org
> >> Betreff: Re: Re: Re: Re: Processor additional documentation
> >> 
> >> Uwe,
> >> 
> >> The additional details piece appears to be a result of your ".nar" file
> >> actually being more like a ".jar", rather than a bundle that includes a JAR
> >> which in turn includes your source code and docs.  Since you did all the
> >> hard work with creating some useful processors, I took the liberty of
> >> moving some of your project stuff around into the NAR structure the folks
> >> have been referring to:
> >> 
> >> https://github.com/uwegeercken/nifi_processors/pull/1
> >> 
> >> This will build a NAR that contains (among other things) a JAR with the
> >> classes, docs, and other processor resources, and is bundled such that the
> >> framework can find everything it needs. I tested this and the Additional
> >> Details links work correctly.  Cheers!
> >> 
> >> Regards,
> >> Matt
> >> 
> >>> On Sun, Mar 20, 2016 at 1:31 PM, Joe Witt  wrote:
> >>> 
> >>> Uwe
> >>> 
> >>> Noticed your other threads on great progress.  That is awesome.
> >>> 
> >>> Really want to help you get to the bottom of the additional details
> >>> piece though.  We clearly have to do a better job with documenting (or
> >>> implementing) how to do this.  Do you have any more details to share
> >>> on symptoms you're seeing?
> >>> 
> >>> Thanks
> >>> Joe
> >>> 
>  On Fri, Mar 18, 2016 at 5:35 PM, Uwe Geercken  
>  wrote:
>  Dan,
>  
>  ok. I was wrong. The index file is created - it's my
> >>> additionalDetails.html file that is missing. I have no idea what is wrong.
>  
>  I will try it tomorrow - maybe I will find something with a clear head.
>  
>  Rgds,
>  
>  Uwe
>  
> > Gesendet: Freitag, 18. März 2016 um 19:14 Uhr
> > Von: "dan bress" 
> > An: dev@nifi.apache.org
> > Betreff: Re: Re: Re: Processor additional documentation
> > 
> > Uwe,
> >   No, the index.html is generated for you.  additionalDetails.html is
> >>> your
> > responsibility only if you feel like the generated index.html doesn't
> >>> fully
> > describe your processor.
> > 
> >   I would guess 80% of the included processors do not have
> > additionalDetails.html.  If you haven't browsed here [1] at examples of
> >>> the
> > generated index.html and user supplied additionalDetails.html, it might
> > clear things up.
> > 
> > [1] https://nifi.apache.org/docs.html
> > 
> > Dan
> > 
> > On Fri, Mar 18, 2016 at 11:08 AM Uwe Geercken 
> >>> wrote:
> > 
> >> Dan,
> >> 
> >> but maybe I have a wrong understanding: do I have to create an
> >>> index.html
> >> file? Currently I have only created an additionalDetails.html file.
> >> 
> >> I will also try to

[VOTE] Release Apache NiFi 0.6.0 (RC1)

2016-03-21 Thread Aldrin Piri

Hello,

I am pleased to be calling this vote for the source release of Apache NiFi
nifi-0.6.0.

The source zip, including signatures, digests, etc. can be found at:
https://repository.apache.org/content/repositories/orgapachenifi-1077

The Git tag is NIFI-1634-RC1
The Git commit hash is 736896246cf021dbed31d4eb1e22e0755e4705f0
*
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=736896246cf021dbed31d4eb1e22e0755e4705f0
*
https://github.com/apache/nifi/commit/736896246cf021dbed31d4eb1e22e0755e4705f0

Checksums of nifi-0.6.0-source-release.zip:
MD5: 1559157db000d53221aeabc5dd607cfc
SHA1: feed12016d7f2d450fb1e3a238634757cc17b0f1

Release artifacts are signed with the following key:
*https://people.apache.org/keys/committer/aldrin.asc
*

KEYS file available here:
https://dist.apache.org/repos/dist/release/nifi/KEYS

71 issues were closed/resolved for this release:
*https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12334372=Html=12316020
*

Release note highlights can be found here:
*https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.6.0
*

The vote will be open for 72 hours.
Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[ ] +1 Release this package as nifi-0.6.0
[ ] +0 no opinion
[ ] -1 Do not release this package because...

Thanks!

[GitHub] nifi pull request: NIFI-1563: Federate requests and merge response...

2016-03-21 Thread mcgilman

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/294#discussion_r56890522
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/pom.xml 
---
@@ -26,5 +26,13 @@
 com.wordnik
 swagger-annotations
 
+
+org.apache.nifi
+nifi-utils
+
+
+org.apache.nifi
+nifi-api
+
--- End diff --

Are these dependencies necessary? Would like to have this artifact as 
'light' and 'portability' as possible. Should just be definitions of transport 
objects for using the REST API.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1563: Federate requests and merge response...

2016-03-21 Thread mcgilman

Github user mcgilman commented on the pull request:

https://github.com/apache/nifi/pull/294#issuecomment-199458161
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1563: Federate requests and merge response...

2016-03-21 Thread markap14

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/294

NIFI-1563: Federate requests and merge responses from nodes instead oâ¦

â¦f storing bulletins and stats at NCM

- Updating UI to support restructured status history DTO.

Return 'Insufficient History' message if aggregate stats don't have enough 
data points, even if all nodes do (which can be the case if the node performing 
the aggregation has a different value for the 
'nifi.components.status.snapshot.frequency' property than the other nodes)

Bug fixes; code cleanup; replicate requests to bulletin board endpoint

Refactored the StatusDTO objects into StatusDTO, 
StatusSnapshotDTO, NodeStatusSnapshotDTO objects

- Introducing endpoints for accessing individual component status.

- Wiring up new endpoints and updated core.
- Code clean up.

- Starting to handling status merging of individual components.
- Nodewise breakdown has been added to Processors but the remaining 
components still need to be updated.

Refactor so that System Diagnostics requests are replicated to nodes 
instead of the information being pulled from Heartbeats

Replicate request for counters instead of pulling them from heartbeats

Removed the getCounters / setCounters method from HeartbeatPayload

- Implementing component specific endpoints.

- Removing unused endpoints.

- Supporting nodewise breakdown for system diagnostics and counters.
- Updating DTOs to use more consistent naming.
- Code clean up.
- Addressing contrib issues.

Removed ProcessGroupStatus from HeartbeatPayload

- Removing nodewise from the system diagnostics endpoint. Had included it 
for testing that option but did not intend for it to be committed.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-1563

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/294.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #294


commit 1bbe0aa7f7c80d13a67178261b88551389a92f17
Author: Mark Payne 
Date:   2016-03-03T18:29:34Z

NIFI-1563: Federate requests and merge responses from nodes instead of 
storing bulletins and stats at NCM

- Updating UI to support restructured status history DTO.

Return 'Insufficient History' message if aggregate stats don't have enough 
data points, even if all nodes do (which can be the case if the node performing 
the aggregation has a different value for the 
'nifi.components.status.snapshot.frequency' property than the other nodes)

Bug fixes; code cleanup; replicate requests to bulletin board endpoint

Refactored the StatusDTO objects into StatusDTO, 
StatusSnapshotDTO, NodeStatusSnapshotDTO objects

- Introducing endpoints for accessing individual component status.

- Wiring up new endpoints and updated core.
- Code clean up.

- Starting to handling status merging of individual components.
- Nodewise breakdown has been added to Processors but the remaining 
components still need to be updated.

Refactor so that System Diagnostics requests are replicated to nodes 
instead of the information being pulled from Heartbeats

Replicate request for counters instead of pulling them from heartbeats

Removed the getCounters / setCounters method from HeartbeatPayload

- Implementing component specific endpoints.

- Removing unused endpoints.

- Supporting nodewise breakdown for system diagnostics and counters.
- Updating DTOs to use more consistent naming.
- Code clean up.
- Addressing contrib issues.

Removed ProcessGroupStatus from HeartbeatPayload

- Removing nodewise from the system diagnostics endpoint. Had included it 
for testing that option but did not intend for it to be committed.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1613 Initial version, try to improve conve...

2016-03-21 Thread ToivoAdams

GitHub user ToivoAdams opened a pull request:

https://github.com/apache/nifi/pull/293

NIFI-1613 Initial version, try to improve conversion for different SQâ¦

Initial version.
1. New method createSqlStringValue(), placeholder for future logic
2. New test testCreateSqlStringValue()
3. Refactored TestConvertJSONToSQL. 
  Setting up Connection pooling is expensive operation.
  So let's do this only once and reuse MockDBCPService in each test.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ToivoAdams/nifi nifi-1613

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/293.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #293


commit 7c2eea7902f81456a880853d782615c8874a38a8
Author: Toivo Adams 
Date:   2016-03-20T19:13:15Z

NIFI-1613 Initial version, try to improve conversion for different SQL 
types. New test and refactored existing test to reuse DBCP service.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Closing in on the Apache NiFi 0.6.0 release

2016-03-21 Thread Aldrin Piri

All,

It looks like the last ticket for 0.6.0 has been merged and resolved.

I will begin the RC process shortly working off of commit
736896246cf021dbed31d4eb1e22e0755e4705f0 [1] [2].

[1]
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=736896246cf021dbed31d4eb1e22e0755e4705f0
[2]
https://github.com/apache/nifi/commit/736896246cf021dbed31d4eb1e22e0755e4705f0

On Mon, Mar 21, 2016 at 1:48 AM, Tony Kurc  wrote:

> The Locale issue was reviewed, confirmed as fixed by reporter and merged
> in.
>
> On Sun, Mar 20, 2016 at 10:35 PM, Joe Witt  wrote:
>
> > Team,
> >
> > There are a couple finishing touches PRs to fix a big defect in
> > SplitText for certain input types, improve locale handling and test
> > behavior for Kit bundle, and to clean up content viewing from
> > connections.
> >
> > Getting good input on findings folks have so please keep it coming as
> > that helps ensure a solid/healthy RC.
> >
> > Thanks
> > Joe
> >
> > On Sat, Mar 19, 2016 at 6:21 PM, Tony Kurc  wrote:
> > > Recommend https://issues.apache.org/jira/browse/NIFI-1651 be included
> in
> > > 0.6.0
> > >
> > > On Wed, Mar 16, 2016 at 4:08 PM, Joe Witt  wrote:
> > >
> > >> Team,
> > >>
> > >> Ok sooo close.  We have 5 tickets remaining.
> > >>
> > >> - Additional functionality/cleanup for SplitText [1]
> > >> [status] Still in discussions. Recommend we move this change to 0.7.0.
> > >> Solid effort on both code contributor and reviewer side but this is a
> > >> tricky one.
> > >>
> > >> - Support Kerberos based authentication to REST API [2]
> > >> [status] PR is in. Reviewing and PR tweaking appears active.  Looks
> > >> quite close and comments indicate great results.
> > >>
> > >> - Add Kerberos support to HBase processors [3]
> > >> [status] Patch in. Under review.  Running on live test system with
> > >> great results.
> > >>
> > >> - Add support for Spring Context loaded processors (Spring
> > >> Integrations, Camel, ...) [4]
> > >> [status] Appears ready. Getting review feedback.
> > >>
> > >> - Zookeeper interaction for NiFI state management should limit state
> to
> > >> 1MB [6]
> > >> [status] Patch is in and review under way.  Looks close.
> > >>
> > >> [1] https://issues.apache.org/jira/browse/NIFI-1118
> > >> [2] https://issues.apache.org/jira/browse/NIFI-1274
> > >> [3] https://issues.apache.org/jira/browse/NIFI-1488
> > >> [4] https://issues.apache.org/jira/browse/NIFI-1571
> > >> [5] https://issues.apache.org/jira/browse/NIFI-1626
> > >>
> > >> Thanks
> > >>
> > >> On Wed, Mar 16, 2016 at 4:04 PM, Joe Witt  wrote:
> > >> > Team,
> > >> >
> > >> > Ok sooo close.  We have 6 tickets remaining.
> > >> >
> > >> > - Additional functionality/cleanup for SplitText [1]
> > >> > [status] Still in discussions. Recommend we move this change to
> 0.7.0.
> > >> > Solid effort on both code contributor and reviewer side but this is
> a
> > >> > tricky one.
> > >> >
> > >> > - Support Kerberos based authentication to REST API [2]
> > >> > [status] PR is in. Reviewing and PR tweaking appears active.  Looks
> > >> > quite close and comments indicate great results.
> > >> >
> > >> > - Add Kerberos support to HBase processors [3]
> > >> > [status] Patch in. Under review.  Running on live test system with
> > >> > great results.
> > >> >
> > >> > - Add support for Spring Context loaded processors (Spring
> > >> > Integrations, Camel, ...) [4]
> > >> > [status] Appears ready. Getting review feedback.
> > >> >
> > >> > - Support based database snapshot/query/change capture [5]
> > >> > [status] Appears ready. Needs review.
> > >> >
> > >> > - Zookeeper interaction for NiFI state management should limit state
> > to
> > >> 1MB [6]
> > >> > [status] Patch is in and review under way.  Looks close.
> > >> >
> > >> > [1] https://issues.apache.org/jira/browse/NIFI-1118
> > >> > [2] https://issues.apache.org/jira/browse/NIFI-1274
> > >> > [3] https://issues.apache.org/jira/browse/NIFI-1488
> > >> > [4] https://issues.apache.org/jira/browse/NIFI-1571
> > >> > [5] https://issues.apache.org/jira/browse/NIFI-1575
> > >> > [6] https://issues.apache.org/jira/browse/NIFI-1626
> > >> >
> > >> > Thanks
> > >> > Joe
> > >> >
> > >> > On Tue, Mar 15, 2016 at 11:56 AM, Aldrin Piri  >
> > >> wrote:
> > >> >> Sounds great.  Will scope those out in the process.  Thanks, Tony.
> > >> >>
> > >> >> On Tue, Mar 15, 2016 at 11:37 AM, Tony Kurc 
> > wrote:
> > >> >>
> > >> >>> Aldrin,
> > >> >>> I did some crappy shell scripts to help, I'll try to put those up
> on
> > >> the
> > >> >>> wiki.
> > >> >>>
> > >> >>> Tony
> > >> >>>
> > >> >>> On Tue, Mar 15, 2016 at 11:29 AM, Aldrin Piri <
> aldrinp...@gmail.com
> > >
> > >> >>> wrote:
> > >> >>>
> > >> >>> > In an effort to avoid me missing helpers and spamming the list,
> I
> > >> will
> > >> >>> > throw my hat into the ring to do this one.
> > >> >>> >
> > >> >>> > On Tue,

Re: Re: Re: Re: Re: Processor additional documentation

2016-03-21 Thread Matt Gilman

Uwe,

Here's the documentation we have for NARs which provides an example
breakdown for a Processor that utilizes a Controller Service [1]. I would
suggest looking at any of the NAR bundle that exist in the source tree
today [2]. The UpdateAttribute NAR bundle is an example of a Processor and
a Custom UI being bundled in the same NAR [3].

Matt

[1] https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#nars
[2] https://github.com/apache/nifi/tree/master/nifi-nar-bundles
[3]
https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-update-attribute-bundle

On Mon, Mar 21, 2016 at 1:21 PM, Uwe Geercken  wrote:

> I am sorry for that. I have completely overlooked that.
>
> So tell me Matt how is it: I create an automatic package with eclipse and
> maven and then manually put it in an archive? or is there a complete
> automatic approach?
>
> Anything I have to specifically do with maven? And is there documentation
> available?
>
> greetings and thanks for help.
>
> Uwe
>
> > Gesendet: Montag, 21. März 2016 um 14:04 Uhr
> > Von: "Matt Burgess" 
> > An: dev@nifi.apache.org
> > Betreff: Re: Re: Re: Re: Processor additional documentation
> >
> > Uwe,
> >
> > The additional details piece appears to be a result of your ".nar" file
> > actually being more like a ".jar", rather than a bundle that includes a
> JAR
> > which in turn includes your source code and docs.  Since you did all the
> > hard work with creating some useful processors, I took the liberty of
> > moving some of your project stuff around into the NAR structure the folks
> > have been referring to:
> >
> > https://github.com/uwegeercken/nifi_processors/pull/1
> >
> > This will build a NAR that contains (among other things) a JAR with the
> > classes, docs, and other processor resources, and is bundled such that
> the
> > framework can find everything it needs. I tested this and the Additional
> > Details links work correctly.  Cheers!
> >
> > Regards,
> > Matt
> >
> > On Sun, Mar 20, 2016 at 1:31 PM, Joe Witt  wrote:
> >
> > > Uwe
> > >
> > > Noticed your other threads on great progress.  That is awesome.
> > >
> > > Really want to help you get to the bottom of the additional details
> > > piece though.  We clearly have to do a better job with documenting (or
> > > implementing) how to do this.  Do you have any more details to share
> > > on symptoms you're seeing?
> > >
> > > Thanks
> > > Joe
> > >
> > > On Fri, Mar 18, 2016 at 5:35 PM, Uwe Geercken 
> wrote:
> > > > Dan,
> > > >
> > > > ok. I was wrong. The index file is created - it's my
> > > additionalDetails.html file that is missing. I have no idea what is
> wrong.
> > > >
> > > > I will try it tomorrow - maybe I will find something with a clear
> head.
> > > >
> > > > Rgds,
> > > >
> > > > Uwe
> > > >
> > > >> Gesendet: Freitag, 18. März 2016 um 19:14 Uhr
> > > >> Von: "dan bress" 
> > > >> An: dev@nifi.apache.org
> > > >> Betreff: Re: Re: Re: Processor additional documentation
> > > >>
> > > >> Uwe,
> > > >>No, the index.html is generated for you.  additionalDetails.html
> is
> > > your
> > > >> responsibility only if you feel like the generated index.html
> doesn't
> > > fully
> > > >> describe your processor.
> > > >>
> > > >>I would guess 80% of the included processors do not have
> > > >> additionalDetails.html.  If you haven't browsed here [1] at
> examples of
> > > the
> > > >> generated index.html and user supplied additionalDetails.html, it
> might
> > > >> clear things up.
> > > >>
> > > >> [1] https://nifi.apache.org/docs.html
> > > >>
> > > >> Dan
> > > >>
> > > >> On Fri, Mar 18, 2016 at 11:08 AM Uwe Geercken 
> > > wrote:
> > > >>
> > > >> > Dan,
> > > >> >
> > > >> > but maybe I have a wrong understanding: do I have to create an
> > > index.html
> > > >> > file? Currently I have only created an additionalDetails.html
> file.
> > > >> >
> > > >> > I will also try to reduce the html code to a minimum and see if
> it is
> > > a
> > > >> > problem with my code.
> > > >> >
> > > >> > Bye,
> > > >> >
> > > >> > Uwe
> > > >> >
> > > >> > > Gesendet: Freitag, 18. März 2016 um 19:03 Uhr
> > > >> > > Von: "dan bress" 
> > > >> > > An: dev@nifi.apache.org
> > > >> > > Betreff: Re: Re: Processor additional documentation
> > > >> > >
> > > >> > > Uwe,
> > > >> > >No its not a problem to have both index.html and
> > > >> > additionalDetails.html
> > > >> > >  The NiFi framework generates nearly all of the documentation
> for
> > > your
> > > >> > > processor for you.  It will generate information about the
> > > properties and
> > > >> > > relationships your processor exposes to its users.  If you need
> to
> > > >> > express
> > > >> > > more about your processor, then that is where
> additionalDetails.html
> > > >> > comes
> > > >> > > into play.  For example, if your processor uses a custom query
> > > language.
> > >

Re: Aw: Re: Re: Re: Re: Processor additional documentation

2016-03-21 Thread Matt Burgess

For that pull request I added POMs at each level, you can run mvn install from 
the top bundle and it will build the NAR under the nifi-datamelt-nar/target 
folder. 

You can import the top level POM into your IDE of choice :)

Regards,
Matt

> On Mar 21, 2016, at 1:21 PM, Uwe Geercken  wrote:
> 
> I am sorry for that. I have completely overlooked that.
> 
> So tell me Matt how is it: I create an automatic package with eclipse and 
> maven and then manually put it in an archive? or is there a complete 
> automatic approach?
> 
> Anything I have to specifically do with maven? And is there documentation 
> available?
> 
> greetings and thanks for help.
> 
> Uwe
> 
>> Gesendet: Montag, 21. März 2016 um 14:04 Uhr
>> Von: "Matt Burgess" 
>> An: dev@nifi.apache.org
>> Betreff: Re: Re: Re: Re: Processor additional documentation
>> 
>> Uwe,
>> 
>> The additional details piece appears to be a result of your ".nar" file
>> actually being more like a ".jar", rather than a bundle that includes a JAR
>> which in turn includes your source code and docs.  Since you did all the
>> hard work with creating some useful processors, I took the liberty of
>> moving some of your project stuff around into the NAR structure the folks
>> have been referring to:
>> 
>> https://github.com/uwegeercken/nifi_processors/pull/1
>> 
>> This will build a NAR that contains (among other things) a JAR with the
>> classes, docs, and other processor resources, and is bundled such that the
>> framework can find everything it needs. I tested this and the Additional
>> Details links work correctly.  Cheers!
>> 
>> Regards,
>> Matt
>> 
>>> On Sun, Mar 20, 2016 at 1:31 PM, Joe Witt  wrote:
>>> 
>>> Uwe
>>> 
>>> Noticed your other threads on great progress.  That is awesome.
>>> 
>>> Really want to help you get to the bottom of the additional details
>>> piece though.  We clearly have to do a better job with documenting (or
>>> implementing) how to do this.  Do you have any more details to share
>>> on symptoms you're seeing?
>>> 
>>> Thanks
>>> Joe
>>> 
 On Fri, Mar 18, 2016 at 5:35 PM, Uwe Geercken  wrote:
 Dan,
 
 ok. I was wrong. The index file is created - it's my
>>> additionalDetails.html file that is missing. I have no idea what is wrong.
 
 I will try it tomorrow - maybe I will find something with a clear head.
 
 Rgds,
 
 Uwe
 
> Gesendet: Freitag, 18. März 2016 um 19:14 Uhr
> Von: "dan bress" 
> An: dev@nifi.apache.org
> Betreff: Re: Re: Re: Processor additional documentation
> 
> Uwe,
>   No, the index.html is generated for you.  additionalDetails.html is
>>> your
> responsibility only if you feel like the generated index.html doesn't
>>> fully
> describe your processor.
> 
>   I would guess 80% of the included processors do not have
> additionalDetails.html.  If you haven't browsed here [1] at examples of
>>> the
> generated index.html and user supplied additionalDetails.html, it might
> clear things up.
> 
> [1] https://nifi.apache.org/docs.html
> 
> Dan
> 
> On Fri, Mar 18, 2016 at 11:08 AM Uwe Geercken 
>>> wrote:
> 
>> Dan,
>> 
>> but maybe I have a wrong understanding: do I have to create an
>>> index.html
>> file? Currently I have only created an additionalDetails.html file.
>> 
>> I will also try to reduce the html code to a minimum and see if it is
>>> a
>> problem with my code.
>> 
>> Bye,
>> 
>> Uwe
>> 
>>> Gesendet: Freitag, 18. März 2016 um 19:03 Uhr
>>> Von: "dan bress" 
>>> An: dev@nifi.apache.org
>>> Betreff: Re: Re: Processor additional documentation
>>> 
>>> Uwe,
>>>   No its not a problem to have both index.html and
>> additionalDetails.html
>>> The NiFi framework generates nearly all of the documentation for
>>> your
>>> processor for you.  It will generate information about the
>>> properties and
>>> relationships your processor exposes to its users.  If you need to
>> express
>>> more about your processor, then that is where additionalDetails.html
>> comes
>>> into play.  For example, if your processor uses a custom query
>>> language.
>>> 
>>> Generated index.html example:
>>> 
>> 
>>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.attributes.UpdateAttribute/index.html
>>> 
>>> additionalDetails.html example:
>>> 
>> 
>>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.attributes.UpdateAttribute/additionalDetails.html
>>> 
>>> On Fri, Mar 18, 2016 at 10:54 AM Uwe Geercken 
>> wrote:
>>> 
 Bryan,
 
 all looks ok. I looked into the nifi-home/work/docs folder. There
>>> is
 nothing but a

Aw: Re: Re: Re: Re: Processor additional documentation

2016-03-21 Thread Uwe Geercken

I am sorry for that. I have completely overlooked that.

So tell me Matt how is it: I create an automatic package with eclipse and maven 
and then manually put it in an archive? or is there a complete automatic 
approach?

Anything I have to specifically do with maven? And is there documentation 
available?

greetings and thanks for help.

Uwe

> Gesendet: Montag, 21. März 2016 um 14:04 Uhr
> Von: "Matt Burgess" 
> An: dev@nifi.apache.org
> Betreff: Re: Re: Re: Re: Processor additional documentation
>
> Uwe,
> 
> The additional details piece appears to be a result of your ".nar" file
> actually being more like a ".jar", rather than a bundle that includes a JAR
> which in turn includes your source code and docs.  Since you did all the
> hard work with creating some useful processors, I took the liberty of
> moving some of your project stuff around into the NAR structure the folks
> have been referring to:
> 
> https://github.com/uwegeercken/nifi_processors/pull/1
> 
> This will build a NAR that contains (among other things) a JAR with the
> classes, docs, and other processor resources, and is bundled such that the
> framework can find everything it needs. I tested this and the Additional
> Details links work correctly.  Cheers!
> 
> Regards,
> Matt
> 
> On Sun, Mar 20, 2016 at 1:31 PM, Joe Witt  wrote:
> 
> > Uwe
> >
> > Noticed your other threads on great progress.  That is awesome.
> >
> > Really want to help you get to the bottom of the additional details
> > piece though.  We clearly have to do a better job with documenting (or
> > implementing) how to do this.  Do you have any more details to share
> > on symptoms you're seeing?
> >
> > Thanks
> > Joe
> >
> > On Fri, Mar 18, 2016 at 5:35 PM, Uwe Geercken  wrote:
> > > Dan,
> > >
> > > ok. I was wrong. The index file is created - it's my
> > additionalDetails.html file that is missing. I have no idea what is wrong.
> > >
> > > I will try it tomorrow - maybe I will find something with a clear head.
> > >
> > > Rgds,
> > >
> > > Uwe
> > >
> > >> Gesendet: Freitag, 18. März 2016 um 19:14 Uhr
> > >> Von: "dan bress" 
> > >> An: dev@nifi.apache.org
> > >> Betreff: Re: Re: Re: Processor additional documentation
> > >>
> > >> Uwe,
> > >>No, the index.html is generated for you.  additionalDetails.html is
> > your
> > >> responsibility only if you feel like the generated index.html doesn't
> > fully
> > >> describe your processor.
> > >>
> > >>I would guess 80% of the included processors do not have
> > >> additionalDetails.html.  If you haven't browsed here [1] at examples of
> > the
> > >> generated index.html and user supplied additionalDetails.html, it might
> > >> clear things up.
> > >>
> > >> [1] https://nifi.apache.org/docs.html
> > >>
> > >> Dan
> > >>
> > >> On Fri, Mar 18, 2016 at 11:08 AM Uwe Geercken 
> > wrote:
> > >>
> > >> > Dan,
> > >> >
> > >> > but maybe I have a wrong understanding: do I have to create an
> > index.html
> > >> > file? Currently I have only created an additionalDetails.html file.
> > >> >
> > >> > I will also try to reduce the html code to a minimum and see if it is
> > a
> > >> > problem with my code.
> > >> >
> > >> > Bye,
> > >> >
> > >> > Uwe
> > >> >
> > >> > > Gesendet: Freitag, 18. März 2016 um 19:03 Uhr
> > >> > > Von: "dan bress" 
> > >> > > An: dev@nifi.apache.org
> > >> > > Betreff: Re: Re: Processor additional documentation
> > >> > >
> > >> > > Uwe,
> > >> > >No its not a problem to have both index.html and
> > >> > additionalDetails.html
> > >> > >  The NiFi framework generates nearly all of the documentation for
> > your
> > >> > > processor for you.  It will generate information about the
> > properties and
> > >> > > relationships your processor exposes to its users.  If you need to
> > >> > express
> > >> > > more about your processor, then that is where additionalDetails.html
> > >> > comes
> > >> > > into play.  For example, if your processor uses a custom query
> > language.
> > >> > >
> > >> > > Generated index.html example:
> > >> > >
> > >> >
> > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.attributes.UpdateAttribute/index.html
> > >> > >
> > >> > > additionalDetails.html example:
> > >> > >
> > >> >
> > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.attributes.UpdateAttribute/additionalDetails.html
> > >> > >
> > >> > > On Fri, Mar 18, 2016 at 10:54 AM Uwe Geercken 
> > >> > wrote:
> > >> > >
> > >> > > > Bryan,
> > >> > > >
> > >> > > > all looks ok. I looked into the nifi-home/work/docs folder. There
> > is
> > >> > > > nothing but a components folder. Inside there is a folder for my
> > >> > processor:
> > >> > > > com.datamelt.nifi.test.TemplateProcessor and inside the folder
> > there
> > >> > is a
> > >> > > > file index.html and it contains the code of my
> >

[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi

2016-03-21 Thread lordjc

Github user lordjc closed the pull request at:

https://github.com/apache/nifi/pull/253


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Re: Re: Re: Processor additional documentation

2016-03-21 Thread Matt Burgess

Uwe,

The additional details piece appears to be a result of your ".nar" file
actually being more like a ".jar", rather than a bundle that includes a JAR
which in turn includes your source code and docs.  Since you did all the
hard work with creating some useful processors, I took the liberty of
moving some of your project stuff around into the NAR structure the folks
have been referring to:

https://github.com/uwegeercken/nifi_processors/pull/1

This will build a NAR that contains (among other things) a JAR with the
classes, docs, and other processor resources, and is bundled such that the
framework can find everything it needs. I tested this and the Additional
Details links work correctly.  Cheers!

Regards,
Matt

On Sun, Mar 20, 2016 at 1:31 PM, Joe Witt  wrote:

> Uwe
>
> Noticed your other threads on great progress.  That is awesome.
>
> Really want to help you get to the bottom of the additional details
> piece though.  We clearly have to do a better job with documenting (or
> implementing) how to do this.  Do you have any more details to share
> on symptoms you're seeing?
>
> Thanks
> Joe
>
> On Fri, Mar 18, 2016 at 5:35 PM, Uwe Geercken  wrote:
> > Dan,
> >
> > ok. I was wrong. The index file is created - it's my
> additionalDetails.html file that is missing. I have no idea what is wrong.
> >
> > I will try it tomorrow - maybe I will find something with a clear head.
> >
> > Rgds,
> >
> > Uwe
> >
> >> Gesendet: Freitag, 18. März 2016 um 19:14 Uhr
> >> Von: "dan bress" 
> >> An: dev@nifi.apache.org
> >> Betreff: Re: Re: Re: Processor additional documentation
> >>
> >> Uwe,
> >>No, the index.html is generated for you.  additionalDetails.html is
> your
> >> responsibility only if you feel like the generated index.html doesn't
> fully
> >> describe your processor.
> >>
> >>I would guess 80% of the included processors do not have
> >> additionalDetails.html.  If you haven't browsed here [1] at examples of
> the
> >> generated index.html and user supplied additionalDetails.html, it might
> >> clear things up.
> >>
> >> [1] https://nifi.apache.org/docs.html
> >>
> >> Dan
> >>
> >> On Fri, Mar 18, 2016 at 11:08 AM Uwe Geercken 
> wrote:
> >>
> >> > Dan,
> >> >
> >> > but maybe I have a wrong understanding: do I have to create an
> index.html
> >> > file? Currently I have only created an additionalDetails.html file.
> >> >
> >> > I will also try to reduce the html code to a minimum and see if it is
> a
> >> > problem with my code.
> >> >
> >> > Bye,
> >> >
> >> > Uwe
> >> >
> >> > > Gesendet: Freitag, 18. März 2016 um 19:03 Uhr
> >> > > Von: "dan bress" 
> >> > > An: dev@nifi.apache.org
> >> > > Betreff: Re: Re: Processor additional documentation
> >> > >
> >> > > Uwe,
> >> > >No its not a problem to have both index.html and
> >> > additionalDetails.html
> >> > >  The NiFi framework generates nearly all of the documentation for
> your
> >> > > processor for you.  It will generate information about the
> properties and
> >> > > relationships your processor exposes to its users.  If you need to
> >> > express
> >> > > more about your processor, then that is where additionalDetails.html
> >> > comes
> >> > > into play.  For example, if your processor uses a custom query
> language.
> >> > >
> >> > > Generated index.html example:
> >> > >
> >> >
> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.attributes.UpdateAttribute/index.html
> >> > >
> >> > > additionalDetails.html example:
> >> > >
> >> >
> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.attributes.UpdateAttribute/additionalDetails.html
> >> > >
> >> > > On Fri, Mar 18, 2016 at 10:54 AM Uwe Geercken 
> >> > wrote:
> >> > >
> >> > > > Bryan,
> >> > > >
> >> > > > all looks ok. I looked into the nifi-home/work/docs folder. There
> is
> >> > > > nothing but a components folder. Inside there is a folder for my
> >> > processor:
> >> > > > com.datamelt.nifi.test.TemplateProcessor and inside the folder
> there
> >> > is a
> >> > > > file index.html and it contains the code of my
> additionalDetails.html
> >> > file.
> >> > > >
> >> > > > when I open the file in the web browser it looks good. I looked at
> >> > other
> >> > > > index.html files and they look similar.
> >> > > >
> >> > > > but I noted that some folders have an inde.html file AND an
> >> > > > additionalDetails.html file. maybe that is the problem?
> >> > > >
> >> > > > greetings,
> >> > > >
> >> > > > Uwe
> >> > > >
> >> > > >
> >> > > >
> >> > > > Gesendet: Freitag, 18. März 2016 um 16:18 Uhr
> >> > > > Von: "Bryan Bende" 
> >> > > > An: dev@nifi.apache.org
> >> > > > Betreff: Re: Processor additional documentation
> >> > > > Hi Uwe,
> >> > > >
> >> > > > Do you have the additionalDetails.html file in your processors jar
> >> > project,
> >> > > > under src/main/resources?
> >> > > >
>

[GitHub] nifi pull request: NIFI-1656 Added locale support to ConvertAvroSc...

2016-03-21 Thread apiri

Github user apiri commented on the pull request:

https://github.com/apache/nifi/pull/292#issuecomment-199261919
  
Yeah, can certainly figure something out with the build matrix.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Aw: Re: Re: Re: Processor: User friendly vs system friendly design

2016-03-21 Thread Uwe Geercken

Adam,

thanks for your feedback. You have some viable points here - let me answer you 
quickly from my train trip to work and I will have a deeper look this evening.

I would be happy to add the processors to the Nifi standard distribution. It 
there documentation from the project of how you specifically work with git? I 
know git but how does your workflow look like?

CSV Processor: 
- opencsv looks great that will be easy to do. If the processor handles a 
multiline file then my question would be if it should also output 
(additionally) a multiline flowfile vs many flowfiles?
- I agree on the header feature that makes the correlation of csv header to 
template placeholders easier for users
- DecimalFormat: I have a reference somewhere in the code but a more precise 
wording and a link would be appropriate I think
- Naming: As discussed, at this stage I am not sure how the final result will 
look like. E.g. if another template engine like freemaker can be used as well. 
The final scope will also have an influence on the naming of the pieces. I 
think we can discuss the details further down the road.

Velocity Processor:
- Map: if I send the map to velocity then it needs a name in the velocity 
context. like e.g. "mymap". in the template one would have to reference a field 
(e.g. column_001) like this: ${mymap.column_001}. So that is different for the 
user. Without the map  it would be ${column_001}.
- Regex filter: A nicety. But I thought there might be a use case where you 
would not want all attributes to be passed to velocity.


General:
I am still somewhat unhappy of duplicating the data into attribute values. As 
said this way I mix the data/content with the attributes. And by keeping the 
original data in the relationship "original" it's actually trippled. That could 
be a performance issue. WHat do you think.

My train arrives. Talk to you later.

Regards,

Uwe



> Gesendet: Montag, 21. März 2016 um 04:14 Uhr
> Von: "Adam Taft" 
> An: dev@nifi.apache.org
> Betreff: Re: Re: Re: Processor: User friendly vs system friendly design
>
> Uwe,
> 
> Great progress.
> 
> Do you intend that your processors are part of the Apache NiFi standard
> distribution?  If so, I think there is enough here to warrant switching the
> discussion over to JIRA tickets with accompanying Github activity.
> 
> Since you've split the functionality into two processors, I'd probably
> recommend creating two feature enhancement JIRA tickets, one for the CSV
> related processor and another for the Velocity processor.  If you agree,
> please open JIRA tickets at [1].
> 
> Here are some thoughts on your processors:
> 
> :: CSV Processor Comments ::
> 
> -  For the CSV processor, it's doing a very naive String#split and doesn't
> deal with quoted CSV files.  What are your thoughts of providing more
> robust CSV handling support?  Perhaps using the opencsv project? [2]
> 
> -  I think I like the ability for the CSV processor to deal with multiple
> rows.  I know I recommend you try using SplitText for this.  But it
> probably wouldn't hurt for this processor to accommodate multi-line CSV
> input files.  This way, you can throw a single line file or multi-line
> file, it will handle both.  For multiple lines, it would export a new
> flowfile for each row in the CSV input.
> 
> -  If we support multi-line CSV input files, I think it makes sense that it
> could also deal with a header line as well.  And optionally, it could
> provide prefix values extracted from the header line.  i.e. if the header
> line names columns "name", "email", "dob", the processor would export
> flowfile attributes with prefix.name, prefix.email, and prefix.dob (instead
> of using numbers).
> 
> -  I like using DecimalFormat in your solution.  Maybe we should reference
> the DecimalFormat API in your documentation.
> 
> -  We should probably consider a name with the phrase "CSV" in it, which I
> think is appropriate for the NIFI naming style.  Something like
> ConvertCSVToAttributes.  I think the NIFI community will probably help
> shape the best name.
> 
> :: Velocity Processor Comments ::
> 
> -  Is there any harm sending the entire Map of flowfile
> attributes to Velocity?  Why did you decide to include a regex to filter
> the attributes send to the engine?  I don't disagree with this feature, but
> wondering if it's required or just a nicety.
> 
> -  There are likely some code review and naming items to talk about.  For
> example, the reference to the VelocityEngine or Template may not be safely
> published.  And the name of the processor should probably reference
> Velocity.
> 
> Thanks,
> 
> Adam
> 
> [1] https://issues.apache.org/jira/browse/NIFI
> [1] http://opencsv.sourceforge.net
> 
> 
> 
> On Sat, Mar 19, 2016 at 2:09 PM, Uwe Geercken  wrote:
> 
> > Adam,
> >
> > ok. I got it. This way one can use any flowfile attributes and merge them
> > with the template.
> >
> > I have worked all day on the

[GitHub] nifi pull request: NIFI-1620 Allow empty Content-Type in InvokeHTT...

2016-03-21 Thread pvillard31

Github user pvillard31 commented on the pull request:

https://github.com/apache/nifi/pull/272#issuecomment-199144952
  
I updated the PR with a true/false "send body" property, should be in line 
with last comments. Let me know if not.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [VOTE] Release Apache NiFi 0.6.0 (RC1)

Re: [VOTE] Release Apache NiFi 0.6.0 (RC1)

[GitHub] nifi pull request: NIFI-1620 Allow empty Content-Type in InvokeHTT...

[GitHub] nifi pull request: NIFI-1620 Allow empty Content-Type in InvokeHTT...

[GitHub] nifi pull request: NIFI-1620 Allow empty Content-Type in InvokeHTT...

Re: [VOTE] Release Apache NiFi 0.6.0 (RC1)

Re: Apache NiFi 0.6.0 RC1 Release Helper Guide

Processor additional documentation

Re: Re: Re: Re: Re: Re: Processor additional documentation

Re: Re: Re: Re: Re: Re: Processor additional documentation

Aw: Re: Re: Re: Re: Re: Processor additional documentation

[VOTE] Release Apache NiFi 0.6.0 (RC1)

[GitHub] nifi pull request: NIFI-1563: Federate requests and merge response...

[GitHub] nifi pull request: NIFI-1563: Federate requests and merge response...

[GitHub] nifi pull request: NIFI-1563: Federate requests and merge response...

[GitHub] nifi pull request: NIFI-1613 Initial version, try to improve conve...

Re: Closing in on the Apache NiFi 0.6.0 release

Re: Re: Re: Re: Re: Processor additional documentation

Re: Aw: Re: Re: Re: Re: Processor additional documentation

Aw: Re: Re: Re: Re: Processor additional documentation

[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi

Re: Re: Re: Re: Processor additional documentation

[GitHub] nifi pull request: NIFI-1656 Added locale support to ConvertAvroSc...

Aw: Re: Re: Re: Processor: User friendly vs system friendly design

[GitHub] nifi pull request: NIFI-1620 Allow empty Content-Type in InvokeHTT...

25 matches

Site Navigation

Mail list logo

Footer information