Re: PutSlack processor

[Andy LoPresto]

Adam,

Yes, this would be excellent. I was actually thinking of doing the same thing 
today after reading this article: 
https://slack.engineering/distributed-security-alerting-c89414c992d6#.7ppxbvtpt

Andy LoPresto
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Feb 29, 2016, at 22:08, Adam Lamar  wrote:
> 
> Hi All,
> 
> Recently I've been working on a PutSlack processor, which allows one to post 
> messages to the online messaging service at www.slack.com. Would you be 
> willing to integrate this processor into the NiFi distribution? If so, do I 
> just create a JIRA and subsequent PR?
> 
> Cheers,
> Adam

[GitHub] nifi pull request: NIFI-1180: Modify PutS3Object to enable encrypt...

[alopresto]

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/246#discussion_r54505449
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/PutS3Object.java
 ---
@@ -407,6 +420,12 @@ public void process(final InputStream rawIn) throws 
IOException {
 }
 }
 
+final String Sse = 
context.getProperty(SERVER_SIDE_ENCRYPTION).getValue();
--- End diff --

Minor note -- naming convention would be `sse`, or preferably, 
`serverSideEncryption`.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1180: Modify PutS3Object to enable encrypt...

[adamonduty]

Github user adamonduty commented on the pull request:

https://github.com/apache/nifi/pull/246#issuecomment-190465236
  
For what its worth, I already made the change to `server-side-encryption` 
in 33805e2e7d849ec3d1cd48b4e0d63c5c7bfc0792. Sorry if that wasn't clear.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1180: Modify PutS3Object to enable encrypt...

[mosermw]

Github user mosermw commented on the pull request:

https://github.com/apache/nifi/pull/246#issuecomment-190377532
  
When a PropertyDescriptor does not have a .displayName() set, then the 
.name() is used in the UI.  There was a time when .displayName() did not exist 
and .name() displayed in the UI, so that's why you see many standard processors 
that do not use displayName.  displayName was added so that you could change 
the name used in the UI without breaking backward compatibility in the flow.xml.

So the original comment from @alopresto represents the latest best practice 
for NiFi processor development.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1180: Modify PutS3Object to enable encrypt...

[alopresto]

Github user alopresto commented on the pull request:

https://github.com/apache/nifi/pull/246#issuecomment-190332181
  
@adamonduty You are correct that the `.name`/`.displayName` convention is 
not documented well. I pinged @joewitt to look for a canonical source for the 
convention and we need to improve the documentation. 

In this case, I am ok with not changing the existing `PropertyDescriptors`, 
as it would break backwards compatibility as you noted. Unfortunately, this is 
a side-effect of the attributes not being set "properly" in the first place -- 
the `displayName` value is visible to the end user through the UI and may need 
to be changed, while the `name` value is used for object resolution in the 
application and so should not need to change if the messaging changes (imagine 
future internationalization where the `displayName` value is 
language-dependent). However, I do think in this case, the new 
`PropertyDescriptor` should follow the correct convention and a comment added 
noting the reason for the difference with other `PropertyDescriptor`s in the 
class. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: OnStopped annotation

[Oleg Zhurakousky]

I meant could you share the test code (via github)

> On Feb 29, 2016, at 12:18 PM, Oleg Zhurakousky  
> wrote:
> 
> Ok, so you are invoking ‘quietlyInvokeMethodsWithAnnotations’ in your test 
> code?
> If so could you your test code where you invoke it? I have a hunch, but want 
> to look before I speculate.
> 
> Cheers
> Oleg
>> On Feb 29, 2016, at 11:58 AM, Pierre Villard  
>> wrote:
>> 
>> I just wanted to test the processors with local SNMP set-up and I noticed
>> that modification of properties in my processor didn't have any effect.
>> So I switched to debug, added a processor, started it, and stopped it just
>> after. Conclusion: my close() method is never called.
>> I correctly go through quietlyInvokeMethodsWithAnnotations() in
>> ReflectionUtils but since the method is not seen as annotated, the close
>> method is not called.
>> 
>> Thanks,
>> Pierre
>> 
>> 2016-02-28 22:24 GMT+01:00 Oleg Zhurakousky :
>> 
>>> I am puzzled as I can’t see how can it not work.
>>> Are there steps to reproduce it? I am trying to read into your initial
>>> email and suspecting you were doing some sort of testing, so want to make
>>> sure I am doing the same thing. . . .
>>> 
>>> Oleg
 On Feb 28, 2016, at 2:46 PM, Pierre Villard 
>>> wrote:
 
 No I am not under testing framework, all my unit tests are OK. I wanted
>>> to
 perform some additional tests and deployed Nifi with the new processors.
 
 You can find the method here [1] if you want to have a look.
 Thanks for your help.
 
 [1]
 
>>> https://github.com/pvillard31/nifi/blob/NIFI-1537/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/processors/AbstractSNMPProcessor.java#L212-L243
 
 2016-02-28 17:11 GMT+01:00 Oleg Zhurakousky <
>>> ozhurakou...@hortonworks.com>:
 
> Also, reading Aldrin’s response and assuming you are using Test mocks I
> would probably recommend to not use them for tests that require full
> lifecycle test of the component until we actually improve it.
> Instead you can code straight agains FlowController essentially
>>> executing
> as a full blown NiFi minus UI. Here is an example:
> 
>>> https://github.com/apache/nifi/pull/210/files#diff-7be646c38c5447f7824e444343633829R92
> 
> Cheers
> Oleg
> 
> On Feb 28, 2016, at 11:07 AM, Oleg Zhurakousky <
> ozhurakou...@hortonworks.com>
>>> wrote:
> 
> Pierre
> Can you paste the method definition? Just want to look at the signature
> and see if there is something obvious
> 
> Sent from my iPhone
> 
> On Feb 28, 2016, at 10:26, Pierre Villard  > wrote:
> 
> Hi,
> 
> I am working on SNMP processors [1] and I'm almost ready for a PR...
>>> but I
> have an issue I can't explain. In my processors, I have implemented a
> method close() with the @OnStopped annotation but it seems the
>>> annotation
> is not seen. When debugging and stopping my processor, I correctly go
> through quietlyInvokeMethodsWithAnnotations() in ReflectionUtils and my
> method close() appears without any annotation. I guess I am missing
> something simple. Any idea?
> 
> [1] https://issues.apache.org/jira/browse/NIFI-1537
> 
> 
> 
>>> 
>>> 
> 

Re: OnStopped annotation

[Oleg Zhurakousky]

Ok, so you are invoking ‘quietlyInvokeMethodsWithAnnotations’ in your test code?
If so could you your test code where you invoke it? I have a hunch, but want to 
look before I speculate.

Cheers
Oleg
> On Feb 29, 2016, at 11:58 AM, Pierre Villard  
> wrote:
> 
> I just wanted to test the processors with local SNMP set-up and I noticed
> that modification of properties in my processor didn't have any effect.
> So I switched to debug, added a processor, started it, and stopped it just
> after. Conclusion: my close() method is never called.
> I correctly go through quietlyInvokeMethodsWithAnnotations() in
> ReflectionUtils but since the method is not seen as annotated, the close
> method is not called.
> 
> Thanks,
> Pierre
> 
> 2016-02-28 22:24 GMT+01:00 Oleg Zhurakousky :
> 
>> I am puzzled as I can’t see how can it not work.
>> Are there steps to reproduce it? I am trying to read into your initial
>> email and suspecting you were doing some sort of testing, so want to make
>> sure I am doing the same thing. . . .
>> 
>> Oleg
>>> On Feb 28, 2016, at 2:46 PM, Pierre Villard 
>> wrote:
>>> 
>>> No I am not under testing framework, all my unit tests are OK. I wanted
>> to
>>> perform some additional tests and deployed Nifi with the new processors.
>>> 
>>> You can find the method here [1] if you want to have a look.
>>> Thanks for your help.
>>> 
>>> [1]
>>> 
>> https://github.com/pvillard31/nifi/blob/NIFI-1537/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/processors/AbstractSNMPProcessor.java#L212-L243
>>> 
>>> 2016-02-28 17:11 GMT+01:00 Oleg Zhurakousky <
>> ozhurakou...@hortonworks.com>:
>>> 
 Also, reading Aldrin’s response and assuming you are using Test mocks I
 would probably recommend to not use them for tests that require full
 lifecycle test of the component until we actually improve it.
 Instead you can code straight agains FlowController essentially
>> executing
 as a full blown NiFi minus UI. Here is an example:
 
>> https://github.com/apache/nifi/pull/210/files#diff-7be646c38c5447f7824e444343633829R92
 
 Cheers
 Oleg
 
 On Feb 28, 2016, at 11:07 AM, Oleg Zhurakousky <
 ozhurakou...@hortonworks.com>
>> wrote:
 
 Pierre
 Can you paste the method definition? Just want to look at the signature
 and see if there is something obvious
 
 Sent from my iPhone
 
 On Feb 28, 2016, at 10:26, Pierre Villard > wrote:
 
 Hi,
 
 I am working on SNMP processors [1] and I'm almost ready for a PR...
>> but I
 have an issue I can't explain. In my processors, I have implemented a
 method close() with the @OnStopped annotation but it seems the
>> annotation
 is not seen. When debugging and stopping my processor, I correctly go
 through quietlyInvokeMethodsWithAnnotations() in ReflectionUtils and my
 method close() appears without any annotation. I guess I am missing
 something simple. Any idea?
 
 [1] https://issues.apache.org/jira/browse/NIFI-1537
 
 
 
>> 
>> 

Re: OnStopped annotation

[Pierre Villard]

I just wanted to test the processors with local SNMP set-up and I noticed
that modification of properties in my processor didn't have any effect.
So I switched to debug, added a processor, started it, and stopped it just
after. Conclusion: my close() method is never called.
I correctly go through quietlyInvokeMethodsWithAnnotations() in
ReflectionUtils but since the method is not seen as annotated, the close
method is not called.

Thanks,
Pierre

2016-02-28 22:24 GMT+01:00 Oleg Zhurakousky :

> I am puzzled as I can’t see how can it not work.
> Are there steps to reproduce it? I am trying to read into your initial
> email and suspecting you were doing some sort of testing, so want to make
> sure I am doing the same thing. . . .
>
> Oleg
> > On Feb 28, 2016, at 2:46 PM, Pierre Villard 
> wrote:
> >
> > No I am not under testing framework, all my unit tests are OK. I wanted
> to
> > perform some additional tests and deployed Nifi with the new processors.
> >
> > You can find the method here [1] if you want to have a look.
> > Thanks for your help.
> >
> > [1]
> >
> https://github.com/pvillard31/nifi/blob/NIFI-1537/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/processors/AbstractSNMPProcessor.java#L212-L243
> >
> > 2016-02-28 17:11 GMT+01:00 Oleg Zhurakousky <
> ozhurakou...@hortonworks.com>:
> >
> >> Also, reading Aldrin’s response and assuming you are using Test mocks I
> >> would probably recommend to not use them for tests that require full
> >> lifecycle test of the component until we actually improve it.
> >> Instead you can code straight agains FlowController essentially
> executing
> >> as a full blown NiFi minus UI. Here is an example:
> >>
> https://github.com/apache/nifi/pull/210/files#diff-7be646c38c5447f7824e444343633829R92
> >>
> >> Cheers
> >> Oleg
> >>
> >> On Feb 28, 2016, at 11:07 AM, Oleg Zhurakousky <
> >> ozhurakou...@hortonworks.com>
> wrote:
> >>
> >> Pierre
> >> Can you paste the method definition? Just want to look at the signature
> >> and see if there is something obvious
> >>
> >> Sent from my iPhone
> >>
> >> On Feb 28, 2016, at 10:26, Pierre Villard  >> > wrote:
> >>
> >> Hi,
> >>
> >> I am working on SNMP processors [1] and I'm almost ready for a PR...
> but I
> >> have an issue I can't explain. In my processors, I have implemented a
> >> method close() with the @OnStopped annotation but it seems the
> annotation
> >> is not seen. When debugging and stopping my processor, I correctly go
> >> through quietlyInvokeMethodsWithAnnotations() in ReflectionUtils and my
> >> method close() appears without any annotation. I guess I am missing
> >> something simple. Any idea?
> >>
> >> [1] https://issues.apache.org/jira/browse/NIFI-1537
> >>
> >>
> >>
>
>

[GitHub] nifi pull request: NIFI-615 - Create a processor to extract WAV fi...

[jskora]

GitHub user jskora opened a pull request:

https://github.com/apache/nifi/pull/252

NIFI-615 - Create a processor to extract WAV file characteristics.

* Created new ExtractMediaMetadata processor using Apache Tika Detector and 
Parser.
* Refactored nifi-image-bundle, nifi-image-nar, and nifi-image-processors 
to nifi-media-bundle, nifi-media-nar, and nifi-media-processors to reflect 
broader media related purpose.
* Preserved existing ExtractImageMetadata and ResizeImage processors as 
well as existing ImageViewerController components to prevent impact on existing 
uses.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jskora/nifi NIFI-615

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/252.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #252


commit 319d4e4643a9958e196c7226155c25c74371137c
Author: Joe Skora 
Date:   2016-02-26T20:33:40Z

NIFI-615 - Create a processor to extract WAV file characteristics.
* Created new ExtractMediaMetadata processor using Apache Tika Detector and 
Parser.
* Refactored nifi-image-bundle, nifi-image-nar, and nifi-image-processors 
to nifi-media-bundle, nifi-media-nar, and nifi-media-processors to reflect 
broader media related purpose.
* Preserved existing ExtractImageMetadata and ResizeImage processors as 
well as existing ImageViewerController components to prevent impact on existing 
uses.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi-1509 - Support aws lambda put processor

[mans2singh]

Github user mans2singh commented on the pull request:

https://github.com/apache/nifi/pull/216#issuecomment-190237652
  
@aldrin @jvwing - Please let me know if there is anything else required for 
this processor.  Thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1516 - AWS DynamoDB Get/Put/Delete Process...

[mans2singh]

Github user mans2singh commented on the pull request:

https://github.com/apache/nifi/pull/224#issuecomment-190236625
  
@olegz @jvwing - Please let me know if you have any other recommendations 
for me.  Thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---