Re: NIFI Secure Access ( Site to Site )

[Vinay]

Andy,

 When  i ran the command as suggested i get below for Verify Return Code

SSL-Session:
Protocol  : TLSv1.2
Cipher: DHE-RSA-AES256-GCM-SHA384
Session-ID:
57898563A942ADD1A8F3AB420FED9C68D38C05B9457CEA0B82B57E2CE0DDAA9A
Session-ID-ctx: 
Master-Key:
1432AS9BA2A69843B54D1FF7E8D872937A1CBA1D6F1C23AA7053161BF27DD8DB99DDCB5F6CC309C71E05C4A5413B2534F
Key-Arg   : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1468630371
Timeout   : 300 (sec)
Verify return code: 19 (self signed certificate in certificate chain)




Regards,
Vinay









--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/NIFI-Secure-Access-Site-to-Site-tp12735p12844.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: DistributedMapCacheClient Groovy example

[Sumanth Chinthagunta]

Added Jira https://issues.apache.org/jira/browse/NIFI-2299 


Thanks 
-Sumo

> On Jul 17, 2016, at 4:56 AM, Matt Burgess  wrote:
> 
> Adding API JARs to the scripting NAR is a good idea since it extends the 
> capabilities as you have shown. Mind writing an improvement Jira to capture 
> this?
> 
> Thanks,
> Matt
> 
> 
>> On Jul 17, 2016, at 4:51 AM, Sumanth Chinthagunta  wrote:
>> 
>> I had to custom build NiFi to add distributed-cache support for scripting 
>> processors as described here: 
>> https://github.com/xmlking/mapr-nifi-hadoop-libraries-bundle/blob/master/nifi-mapr-build.md#add-optional
>>  
>> 
>> Please consider adding this support in next release. 
>> 
>> -Sumo 
>> 
>>> On Jul 16, 2016, at 10:17 PM, Sumanth Chinthagunta  
>>> wrote:
>>> 
>>> Hi Matt,
>>> 
>>> Did you find any solution to use DistributedMapCacheClientService from 
>>> Scripting processors ?
>>> 
>>> I tried to use module directory with 
>>> nifi-sumo-common-0.7.0-SNAPSHOT-all.jar bundling with 
>>> org.apache.nifi:nifi-distributed-cache-client-service-api dependency here: 
>>> https://github.com/xmlking/nifi-scripting/releases/tag/0.7.0 
>>> 
>>> 
>>> but getting weird error :(
>>> 
>>> Would be nice if this dependency is bundled with scripting processor Nar in 
>>> NiFi 0.7.1 :)
>>> 
>>>  
>>>   org.apache.nifi
>>>   nifi-distributed-cache-client-service-api
>>>   
>>> 
>>> My Groovy script:
>>> 
>>> import org.apache.nifi.controller.ControllerService
>>> import com.crossbusiness.nifi.processors.StringSerDe
>>> 
>>> final StringSerDe stringSerDe = new StringSerDe();
>>> 
>>> def lookup = context.controllerServiceLookup
>>> def cacheServiceName = DistributedMapCacheClientServiceName.value
>>> 
>>> log.error  "cacheServiceName: ${cacheServiceName}"
>>> 
>>> def cacheServiceId = 
>>> lookup.getControllerServiceIdentifiers(ControllerService).find {
>>>   cs -> lookup.getControllerServiceName(cs) == cacheServiceName
>>> }
>>> 
>>> log.error  "cacheServiceId:  ${cacheServiceId}"
>>> 
>>> def cache = lookup.getControllerService(cacheServiceId)
>>> log.error cache.get("aaa", stringSerDe, stringSerDe )
>>> Error: 
>>> 
>>> 00:02:04 CDT
>>> ERROR
>>> 3886ddbc-1ccd-437e-8e34-f5a98602264b
>>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] cacheServiceName: 
>>> DistributedMapCacheClientService
>>> 00:02:04 CDT
>>> ERROR
>>> 3886ddbc-1ccd-437e-8e34-f5a98602264b
>>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] cacheServiceId:  
>>> 8971c8e8-6bc5-4e07-8e30-7189fa8a5252
>>> 00:02:04 CDT
>>> ERROR
>>> 3886ddbc-1ccd-437e-8e34-f5a98602264b
>>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] 
>>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] failed to process 
>>> due to org.apache.nifi.processor.exception.ProcessException: 
>>> javax.script.ScriptException: javax.script.ScriptException: 
>>> groovy.lang.MissingMethodException: No signature of method: 
>>> com.sun.proxy.$Proxy123.get() is applicable for argument types: 
>>> (java.lang.String, com.crossbusiness.nifi.processors.StringSerDe, 
>>> com.crossbusiness.nifi.processors.StringSerDe) values: [aaa, 
>>> com.crossbusiness.nifi.processors.StringSerDe@5b1ef80b 
>>> , ...]
>>> Possible solutions: grep(), get(java.lang.Object, 
>>> org.apache.nifi.distributed.cache.client.Serializer, 
>>> org.apache.nifi.distributed.cache.client.Deserializer), wait(), any(), 
>>> getAt(java.lang.String), every(); rolling back session: 
>>> org.apache.nifi.processor.exception.ProcessException: 
>>> javax.script.ScriptException: javax.script.ScriptException: 
>>> groovy.lang.MissingMethodException: No signature of method: 
>>> com.sun.proxy.$Proxy123.get() is applicable for argument types: 
>>> (java.lang.String, com.crossbusiness.nifi.processors.StringSerDe, 
>>> com.crossbusiness.nifi.processors.StringSerDe) values: [aaa, 
>>> com.crossbusiness.nifi.processors.StringSerDe@5b1ef80b 
>>> , ...]
>>> Possible solutions: grep(), get(java.lang.Object, 
>>> org.apache.nifi.distributed.cache.client.Serializer, 
>>> org.apache.nifi.distributed.cache.client.Deserializer), wait(), any(), 
>>> getAt(java.lang.String), every()
>>> 00:02:04 CDT
>>> ERROR
>>> 3886ddbc-1ccd-437e-8e34-f5a98602264b
>>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] Failed to process 
>>> session due to org.apache.nifi.processor.exception.ProcessException: 
>>> javax.script.ScriptException: javax.script.ScriptException: 
>>> groovy.lang.MissingMethodException: No signature of method: 
>>> com.sun.proxy.$Proxy123.get() is applicable for argument types: 
>>> (java.lang.String, 

Re: ConsumeKafka vs getKafka

[Sumanth Chinthagunta]

Done.

https://issues.apache.org/jira/browse/NIFI-2298 


Thanks 
-Sumo

> On Jul 17, 2016, at 1:37 PM, Sumanth Chinthagunta  wrote:
> 
> Env: Linux , Kafka 9.0.x, NiFi: 0.7 
> 
> We like to use new ConsumeKafka processor  but found critical limitations 
> compared to old getKafka processor. 
> New ConsumeKafka is not writing critical Kafka attributes  i.e., kafka.key, 
> kafka.offset, kafka.partition etc into flowFile attributes. 
> 
> 
> Old getKafka processor: 
> 
> Standard FlowFile Attributes
> Key: 'entryDate'
>Value: 'Sun Jul 17 15:17:00 CDT 2016'
> Key: 'lineageStartDate'
>Value: 'Sun Jul 17 15:17:00 CDT 2016'
> Key: 'fileSize'
>Value: '183'
> FlowFile Attribute Map Content
> Key: 'filename'
>Value: '19709945781167274'
> Key: 'kafka.key'
>Value: '{"database":"test","table":"sc_job","pk.systemid":1}'
> Key: 'kafka.offset'
>Value: '1184010261'
> Key: 'kafka.partition'
>Value: '0'
> Key: 'kafka.topic'
>Value: ‘data'
> Key: 'path'
>Value: './'
> Key: 'uuid'
>Value: '244059bb-9ad9-4d74-b1fb-312eee72124a'
>  
> —
> new ConsumeKafka processor : 
>  
> Standard FlowFile Attributes
> Key: 'entryDate'
>Value: 'Sun Jul 17 15:18:41 CDT 2016'
> Key: 'lineageStartDate'
>Value: 'Sun Jul 17 15:18:41 CDT 2016'
> Key: 'fileSize'
>Value: '183'
> FlowFile Attribute Map Content
> Key: 'filename'
>Value: '19710046870478139'
> Key: 'path'
>Value: './'
> Key: 'uuid'
>Value: '349fbeb3-e342-4533-be4c-424793fa5c59’
> 
> Thanks 
> Sumo 

Re: Missing Emails - Re: [apache/nifi] Nifi-1972 Apache Ignite Put Cache Processor (#502)

[Joe Witt]

Hello Mans

We ended up creating an 'iss...@nifi.apache.org' mailing list so
please subscribe to that.  It is not meant for discussion threads but
is meant for automated e-mails from github, jira.  See more about this
here https://nifi.apache.org/mailing_lists.html

Thanks
Joe

On Sun, Jul 17, 2016 at 9:15 PM, M Singh  wrote:
> Hi Folks:
> I am no longer getting any emails for my replies to github pull request.  I 
> have subscribed to the comments/etc on the pull request and was getting all 
> the emails till the first week of July.
>
> If there is any setting I am missing, please let me know.
> Thanks
>
> On Sunday, July 17, 2016 11:28 AM, Joe Percivall 
>  wrote:
>
>
>  In 
> nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/logback.xml:>
>  @@ -89,7 +89,8 @@
>>  
>>  > level="INFO"/>
>>  > name="org.apache.nifi.controller.repository.StandardProcessSession" 
>> level="WARN" />
>> -
>> +
>> +
> Instead of adding this to the default logback, which would get used for every 
> NiFi instance, why not add documentation to the description or Additional 
> Details section stating that periodic stats are logged and this statement, 
> added to the logback.xml, will suppress them. This will get around adding a 
> logger definition here that's specific to the processor but also give the 
> user the ability to change it if needed. Besides, some users may like having 
> the stats logged by default.—
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub, or mute the thread.
>
>

Missing Emails - Re: [apache/nifi] Nifi-1972 Apache Ignite Put Cache Processor (#502)

[M Singh]

Hi Folks:
I am no longer getting any emails for my replies to github pull request.  I 
have subscribed to the comments/etc on the pull request and was getting all the 
emails till the first week of July.  

If there is any setting I am missing, please let me know.
Thanks 

On Sunday, July 17, 2016 11:28 AM, Joe Percivall  
wrote:
 

 In 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/logback.xml:>
 @@ -89,7 +89,8 @@
>  
>   level="INFO"/>
>   name="org.apache.nifi.controller.repository.StandardProcessSession" 
> level="WARN" />
> -
> +
> +
Instead of adding this to the default logback, which would get used for every 
NiFi instance, why not add documentation to the description or Additional 
Details section stating that periodic stats are logged and this statement, 
added to the logback.xml, will suppress them. This will get around adding a 
logger definition here that's specific to the processor but also give the user 
the ability to change it if needed. Besides, some users may like having the 
stats logged by default.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.  

  

Re: ConsumeKafka vs getKafka

[Oleg Zhurakousky]

Sumo

Thank you for pointing this out. Very valid comments and just in time for the 
upcoming 1.0 release. Would you mind to raise a JIRA and we'll address it 
promptly? 

Cheers 
Oleg

> On Jul 17, 2016, at 16:38, Sumanth Chinthagunta  wrote:
> 
> Env: Linux , Kafka 9.0.x, NiFi: 0.7 
> 
> We like to use new ConsumeKafka processor  but found critical limitations 
> compared to old getKafka processor. 
> New ConsumeKafka is not writing critical Kafka attributes  i.e., kafka.key, 
> kafka.offset, kafka.partition etc into flowFile attributes. 
> 
> 
> Old getKafka processor: 
> 
> Standard FlowFile Attributes
> Key: 'entryDate'
>   Value: 'Sun Jul 17 15:17:00 CDT 2016'
> Key: 'lineageStartDate'
>   Value: 'Sun Jul 17 15:17:00 CDT 2016'
> Key: 'fileSize'
>   Value: '183'
> FlowFile Attribute Map Content
> Key: 'filename'
>   Value: '19709945781167274'
> Key: 'kafka.key'
>   Value: '{"database":"test","table":"sc_job","pk.systemid":1}'
> Key: 'kafka.offset'
>   Value: '1184010261'
> Key: 'kafka.partition'
>   Value: '0'
> Key: 'kafka.topic'
>   Value: ‘data'
> Key: 'path'
>   Value: './'
> Key: 'uuid'
>   Value: '244059bb-9ad9-4d74-b1fb-312eee72124a'
> 
> —
> new ConsumeKafka processor : 
> 
> Standard FlowFile Attributes
> Key: 'entryDate'
>   Value: 'Sun Jul 17 15:18:41 CDT 2016'
> Key: 'lineageStartDate'
>   Value: 'Sun Jul 17 15:18:41 CDT 2016'
> Key: 'fileSize'
>   Value: '183'
> FlowFile Attribute Map Content
> Key: 'filename'
>   Value: '19710046870478139'
> Key: 'path'
>   Value: './'
> Key: 'uuid'
>   Value: '349fbeb3-e342-4533-be4c-424793fa5c59’
> 
> Thanks 
> Sumo

Re: External Repositories (aka Extension Registry) for Dynamically-loaded Extensions

[Joe Witt]

Andre

Seems like a fair point and also I added you to the wiki permissions.

Thanks
Joe

On Sun, Jul 17, 2016 at 9:01 AM, Andre  wrote:
> All,
>
> I don't have write access to the wiki so I ended up not being able to write
> the comment over there but I wanted to make a very basic comment.
>
>
> The current package security mechanism proposes using MD5 and SHA1 hashing
> algo's for pom's and adoc's.
>
> Be mindful that for ages the general guidance is that both algorithms
> should not be used in regulated systems (i.e. government, banking, health,
> etc) across multiple jurisdictions out of security concerns:
>
> http://csrc.nist.gov/groups/ST/hash/policy.html
> https://web.archive.org/web/20131113032317/http://blogs.technet.com/b/pki/archive/2013/11/12/sha1-deprecation-policy.aspx
> https://security.googleblog.com/2014/09/gradually-sunsetting-sha-1.html
> http://blog.pcisecuritystandards.org/migrating-from-ssl-and-early-tls
> https://threatpost.com/sloth-attacks-up-ante-on-sha-1-md5-deprecation/115807/
>
> I understand most of discussion around the ciphers still floats around its
> use in PKI and TLS comms and that the scenario is well far fetched for
> NiFi's (unless the previous holders of the copyright decided to use the
> open source repo... 8D )
>
> But considering how easy it is to use SHA2 or SHA3, may I suggest we
> revisit the choice of algorithms?
>
> Cheers
>
> On Thu, Jul 14, 2016 at 7:43 AM, Matt Foley  wrote:
>
>> Okay, that's why I sent the email first :-)
>>
>> I talked with Oleg offline, and clarified that his concern is that there
>> are many user requirements expressed in the first four sections, and that
>> any decreases in scope from what's expressed there should be done with full
>> community consensus, and there hasn't been enough discussion with enough
>> participants yet to achieve that consensus.
>>
>> So for now, I'll just edit the material I added.  I'll move Section 5 into
>> Section 3 without removing anything except redundancy, and expand the
>> current Section 6 to make it easier to discuss the individual points that
>> obviously need more discussion, like the Browse/Discovery experience and
>> Dependency Management.
>>
>> Any further thoughts or concerns are welcome.
>>
>> Thanks,
>> --Matt
>> 
>> From: Oleg Zhurakousky
>> Sent: Wednesday, July 13, 2016 10:53 AM
>> To: Matt Foley
>> Cc: dev@nifi.apache.org; Bryan Bende; Mark Payne; Brandon DeVries; Joe
>> Witt; Aldrin Piri
>> Subject: Re: External Repositories (aka Extension Registry) for
>> Dynamically-loaded Extensions
>>
>> Matt,
>>
>> Due to “other tasks” I personally haven’t been able yet to follow
>> everything that’s going on, nor had I have a chance to look at the POC you
>> and Puspendu have created, so I think global update of the page would be
>> too early at the moment as it doesn’t really capture all of the concerns
>> and there are certainly quite a few vey common concerns that need to be
>> addressed outside of what I was able to extract from recent interaction.
>> I will do my best to go through all the comments by the end of next week.
>>
>> Cheers
>> Oleg
>>
>> > On Jul 13, 2016, at 1:46 PM, Matt Foley  wrote:
>> >
>> > Hi all,
>> > I'm about to do a global edit of the design page at
>> https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
>> > based on comments by Joe Witt and Aldrin Piri.
>> >
>> > This will include editing text that was written by the folks on the CC:
>> line.  If any of you object up front, please say so, otherwise after the
>> fact there's always Page History :-)
>> >
>> > Among other edits, I intend to unify Sections 3 (Proposed Features) and
>> 5 (Use Cases), removing contradictions and reducing the discussion to what
>> we actually intend to implement.  I will add a section on "Possible Future
>> Features", so as to not lose the other ideas.
>> >
>> > I will also break out sections for "Dependency Management and
>> Auto-loading", and "Discovery - Browsing Extensions (and their docs)
>> Without Loading Them".  These are two areas that are least well defined,
>> and have already generated the most discussion.
>> >
>> > I will start this editing tomorrow morning, unless there are '-1's.
>> > Thanks,
>> > --Matt
>> > 
>> > From: Matt Foley
>> > Sent: Monday, July 11, 2016 4:20 PM
>> > To: dev@nifi.apache.org
>> > Subject: Re: External Repositories (aka Extension Registry) for
>> Dynamically-loaded Extensions
>> >
>> > Thanks, Joe.  I've responded to the issues you raised in Sections 5 and
>> 6.
>> > Sections 2 and 3 were written by others and I am reticent to edit their
>> text;
>> > I hope they will accept or respond to your comments there.
>> >
>> > Thanks everyone,
>> > --Matt
>> > 
>> > From: Joe Witt 

Re: Master build failing?

[Joe Witt]

very strange - thanks for flagging.  Will look into it

On Sun, Jul 17, 2016 at 10:34 AM, Andre  wrote:
> All,
>
> I've noticed the travis build is failing again.
>
> Last working commit seems to be:
>
> https://github.com/apache/nifi/commit/4d4c525d9c74e81a50d582bd6b7f7045f1d322a6
>
> After that all travis builds fail. From what I could gather the builds are
> experiencing the following issue:
>
> java.lang.IllegalStateException: Extensions class loaders have already
> been loaded.
> at org.apache.nifi.nar.NarClassLoaders.load(NarClassLoaders.java:67)
> at 
> org.apache.nifi.controller.service.StandardControllerServiceProviderTest.setupSuite(StandardControllerServiceProviderTest.java:41)
>
> Running 
> org.apache.nifi.controller.service.TestStandardControllerServiceProvider
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.901
> sec - in 
> org.apache.nifi.controller.service.TestStandardControllerServiceProvider
> Running org.apache.nifi.fingerprint.FingerprintFactoryTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046
> sec - in org.apache.nifi.fingerprint.FingerprintFactoryTest
> Running org.apache.nifi.cluster.HeartbeatPayloadTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014
> sec - in org.apache.nifi.cluster.HeartbeatPayloadTest
>
> Results :
>
>
> Tests in error:
>   StandardControllerServiceProviderTest.setupSuite:41 » IllegalState
> Extensions ...

Master build failing?

[Andre]

All,

I've noticed the travis build is failing again.

Last working commit seems to be:

https://github.com/apache/nifi/commit/4d4c525d9c74e81a50d582bd6b7f7045f1d322a6

After that all travis builds fail. From what I could gather the builds are
experiencing the following issue:

java.lang.IllegalStateException: Extensions class loaders have already
been loaded.
at org.apache.nifi.nar.NarClassLoaders.load(NarClassLoaders.java:67)
at 
org.apache.nifi.controller.service.StandardControllerServiceProviderTest.setupSuite(StandardControllerServiceProviderTest.java:41)

Running org.apache.nifi.controller.service.TestStandardControllerServiceProvider
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.901
sec - in 
org.apache.nifi.controller.service.TestStandardControllerServiceProvider
Running org.apache.nifi.fingerprint.FingerprintFactoryTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046
sec - in org.apache.nifi.fingerprint.FingerprintFactoryTest
Running org.apache.nifi.cluster.HeartbeatPayloadTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014
sec - in org.apache.nifi.cluster.HeartbeatPayloadTest

Results :


Tests in error:
  StandardControllerServiceProviderTest.setupSuite:41 » IllegalState
Extensions ...

Re: DistributedMapCacheClient Groovy example

[Matt Burgess]

Adding API JARs to the scripting NAR is a good idea since it extends the 
capabilities as you have shown. Mind writing an improvement Jira to capture 
this?

Thanks,
Matt


> On Jul 17, 2016, at 4:51 AM, Sumanth Chinthagunta  wrote:
> 
> I had to custom build NiFi to add distributed-cache support for scripting 
> processors as described here: 
> https://github.com/xmlking/mapr-nifi-hadoop-libraries-bundle/blob/master/nifi-mapr-build.md#add-optional
>  
> 
> Please consider adding this support in next release. 
> 
> -Sumo 
> 
>> On Jul 16, 2016, at 10:17 PM, Sumanth Chinthagunta  wrote:
>> 
>> Hi Matt,
>> 
>> Did you find any solution to use DistributedMapCacheClientService from 
>> Scripting processors ?
>> 
>> I tried to use module directory with nifi-sumo-common-0.7.0-SNAPSHOT-all.jar 
>> bundling with org.apache.nifi:nifi-distributed-cache-client-service-api 
>> dependency here: 
>> https://github.com/xmlking/nifi-scripting/releases/tag/0.7.0 
>> 
>> 
>> but getting weird error :(
>> 
>> Would be nice if this dependency is bundled with scripting processor Nar in 
>> NiFi 0.7.1 :)
>> 
>>   
>>org.apache.nifi
>>nifi-distributed-cache-client-service-api
>>
>> 
>> My Groovy script:
>> 
>> import org.apache.nifi.controller.ControllerService
>> import com.crossbusiness.nifi.processors.StringSerDe
>> 
>> final StringSerDe stringSerDe = new StringSerDe();
>> 
>> def lookup = context.controllerServiceLookup
>> def cacheServiceName = DistributedMapCacheClientServiceName.value
>> 
>> log.error  "cacheServiceName: ${cacheServiceName}"
>> 
>> def cacheServiceId = 
>> lookup.getControllerServiceIdentifiers(ControllerService).find {
>>cs -> lookup.getControllerServiceName(cs) == cacheServiceName
>> }
>> 
>> log.error  "cacheServiceId:  ${cacheServiceId}"
>> 
>> def cache = lookup.getControllerService(cacheServiceId)
>> log.error cache.get("aaa", stringSerDe, stringSerDe )
>> Error: 
>> 
>> 00:02:04 CDT
>> ERROR
>> 3886ddbc-1ccd-437e-8e34-f5a98602264b
>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] cacheServiceName: 
>> DistributedMapCacheClientService
>> 00:02:04 CDT
>> ERROR
>> 3886ddbc-1ccd-437e-8e34-f5a98602264b
>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] cacheServiceId:  
>> 8971c8e8-6bc5-4e07-8e30-7189fa8a5252
>> 00:02:04 CDT
>> ERROR
>> 3886ddbc-1ccd-437e-8e34-f5a98602264b
>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] 
>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] failed to process due 
>> to org.apache.nifi.processor.exception.ProcessException: 
>> javax.script.ScriptException: javax.script.ScriptException: 
>> groovy.lang.MissingMethodException: No signature of method: 
>> com.sun.proxy.$Proxy123.get() is applicable for argument types: 
>> (java.lang.String, com.crossbusiness.nifi.processors.StringSerDe, 
>> com.crossbusiness.nifi.processors.StringSerDe) values: [aaa, 
>> com.crossbusiness.nifi.processors.StringSerDe@5b1ef80b 
>> , ...]
>> Possible solutions: grep(), get(java.lang.Object, 
>> org.apache.nifi.distributed.cache.client.Serializer, 
>> org.apache.nifi.distributed.cache.client.Deserializer), wait(), any(), 
>> getAt(java.lang.String), every(); rolling back session: 
>> org.apache.nifi.processor.exception.ProcessException: 
>> javax.script.ScriptException: javax.script.ScriptException: 
>> groovy.lang.MissingMethodException: No signature of method: 
>> com.sun.proxy.$Proxy123.get() is applicable for argument types: 
>> (java.lang.String, com.crossbusiness.nifi.processors.StringSerDe, 
>> com.crossbusiness.nifi.processors.StringSerDe) values: [aaa, 
>> com.crossbusiness.nifi.processors.StringSerDe@5b1ef80b 
>> , ...]
>> Possible solutions: grep(), get(java.lang.Object, 
>> org.apache.nifi.distributed.cache.client.Serializer, 
>> org.apache.nifi.distributed.cache.client.Deserializer), wait(), any(), 
>> getAt(java.lang.String), every()
>> 00:02:04 CDT
>> ERROR
>> 3886ddbc-1ccd-437e-8e34-f5a98602264b
>> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] Failed to process 
>> session due to org.apache.nifi.processor.exception.ProcessException: 
>> javax.script.ScriptException: javax.script.ScriptException: 
>> groovy.lang.MissingMethodException: No signature of method: 
>> com.sun.proxy.$Proxy123.get() is applicable for argument types: 
>> (java.lang.String, com.crossbusiness.nifi.processors.StringSerDe, 
>> com.crossbusiness.nifi.processors.StringSerDe) values: [aaa, 
>> com.crossbusiness.nifi.processors.StringSerDe@5b1ef80b 
>> , ...]
>> Possible solutions: grep(), get(java.lang.Object, 
>> 

Re: DistributedMapCacheClient Groovy example

[Sumanth Chinthagunta]

I had to custom build NiFi to add distributed-cache support for scripting 
processors as described here: 
https://github.com/xmlking/mapr-nifi-hadoop-libraries-bundle/blob/master/nifi-mapr-build.md#add-optional
 

Please consider adding this support in next release. 

-Sumo 

> On Jul 16, 2016, at 10:17 PM, Sumanth Chinthagunta  wrote:
> 
> Hi Matt,
> 
> Did you find any solution to use DistributedMapCacheClientService from 
> Scripting processors ?
> 
> I tried to use module directory with nifi-sumo-common-0.7.0-SNAPSHOT-all.jar 
> bundling with org.apache.nifi:nifi-distributed-cache-client-service-api 
> dependency here: 
> https://github.com/xmlking/nifi-scripting/releases/tag/0.7.0 
> 
> 
> but getting weird error :(
> 
> Would be nice if this dependency is bundled with scripting processor Nar in 
> NiFi 0.7.1 :)
> 
>
> org.apache.nifi
> nifi-distributed-cache-client-service-api
> 
> 
> My Groovy script:
> 
> import org.apache.nifi.controller.ControllerService
> import com.crossbusiness.nifi.processors.StringSerDe
> 
> final StringSerDe stringSerDe = new StringSerDe();
> 
> def lookup = context.controllerServiceLookup
> def cacheServiceName = DistributedMapCacheClientServiceName.value
> 
> log.error  "cacheServiceName: ${cacheServiceName}"
> 
> def cacheServiceId = 
> lookup.getControllerServiceIdentifiers(ControllerService).find {
> cs -> lookup.getControllerServiceName(cs) == cacheServiceName
> }
> 
> log.error  "cacheServiceId:  ${cacheServiceId}"
> 
> def cache = lookup.getControllerService(cacheServiceId)
> log.error cache.get("aaa", stringSerDe, stringSerDe )
>  Error: 
>  
> 00:02:04 CDT
> ERROR
> 3886ddbc-1ccd-437e-8e34-f5a98602264b
> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] cacheServiceName: 
> DistributedMapCacheClientService
> 00:02:04 CDT
> ERROR
> 3886ddbc-1ccd-437e-8e34-f5a98602264b
> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] cacheServiceId:  
> 8971c8e8-6bc5-4e07-8e30-7189fa8a5252
> 00:02:04 CDT
> ERROR
> 3886ddbc-1ccd-437e-8e34-f5a98602264b
> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] 
> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] failed to process due 
> to org.apache.nifi.processor.exception.ProcessException: 
> javax.script.ScriptException: javax.script.ScriptException: 
> groovy.lang.MissingMethodException: No signature of method: 
> com.sun.proxy.$Proxy123.get() is applicable for argument types: 
> (java.lang.String, com.crossbusiness.nifi.processors.StringSerDe, 
> com.crossbusiness.nifi.processors.StringSerDe) values: [aaa, 
> com.crossbusiness.nifi.processors.StringSerDe@5b1ef80b 
> , ...]
> Possible solutions: grep(), get(java.lang.Object, 
> org.apache.nifi.distributed.cache.client.Serializer, 
> org.apache.nifi.distributed.cache.client.Deserializer), wait(), any(), 
> getAt(java.lang.String), every(); rolling back session: 
> org.apache.nifi.processor.exception.ProcessException: 
> javax.script.ScriptException: javax.script.ScriptException: 
> groovy.lang.MissingMethodException: No signature of method: 
> com.sun.proxy.$Proxy123.get() is applicable for argument types: 
> (java.lang.String, com.crossbusiness.nifi.processors.StringSerDe, 
> com.crossbusiness.nifi.processors.StringSerDe) values: [aaa, 
> com.crossbusiness.nifi.processors.StringSerDe@5b1ef80b 
> , ...]
> Possible solutions: grep(), get(java.lang.Object, 
> org.apache.nifi.distributed.cache.client.Serializer, 
> org.apache.nifi.distributed.cache.client.Deserializer), wait(), any(), 
> getAt(java.lang.String), every()
> 00:02:04 CDT
> ERROR
> 3886ddbc-1ccd-437e-8e34-f5a98602264b
> ExecuteScript[id=3886ddbc-1ccd-437e-8e34-f5a98602264b] Failed to process 
> session due to org.apache.nifi.processor.exception.ProcessException: 
> javax.script.ScriptException: javax.script.ScriptException: 
> groovy.lang.MissingMethodException: No signature of method: 
> com.sun.proxy.$Proxy123.get() is applicable for argument types: 
> (java.lang.String, com.crossbusiness.nifi.processors.StringSerDe, 
> com.crossbusiness.nifi.processors.StringSerDe) values: [aaa, 
> com.crossbusiness.nifi.processors.StringSerDe@5b1ef80b 
> , ...]
> Possible solutions: grep(), get(java.lang.Object, 
> org.apache.nifi.distributed.cache.client.Serializer, 
> org.apache.nifi.distributed.cache.client.Deserializer), wait(), any(), 
> getAt(java.lang.String), every(): 
> org.apache.nifi.processor.exception.ProcessException: 
> javax.script.ScriptException: javax.script.ScriptException: 
> groovy.lang.MissingMethodException: No signature of method: 
> com.sun.proxy.$Proxy123.get() is applicable for 

Re: ListenSMTP processor

[Andre]

Dan,

Ingesting Microsoft Journals seem like a great suggestion for a new
processor ( ParseExchangeJounal ?).

Regarding TNEF: As far as I know, Apache Commons - Mail does not pase
"winmail.dat"
type attachments. As far as I understand the only ASL compatible
implementation of a TNEF extractor is Apache's POI and even that
implementation is not part of POI's main release.

If TNEF support is required we will ether have to code from scratch or
perhaps use https://github.com/koodaamo/tnefparse together with
ExecuteScript (although since tnefparse  is LGPL, this solution cannot be
packaged as part of NiFi).

Cheers

On Sun, Jul 17, 2016 at 10:53 AM, djmdata  wrote:

> What is the JIRA #?
>
> I have a production system that reads email from a custom SMTP listener and
> places the SMTP payload into Kafka. A Storm topology reads messages from
> Kafka and parses the emails (Java code using JavaMail API) into useful info
> (subject, text, attachments, body, etc...).
>
> I'm looking at plugging NiFi into this to replace the custom SMTP listener.
> If you had a processor that could act as a reliable (we can't lose emails)
> and performant SMTP listener alternative we would use it.
>
> Your "email parser processor" is an interesting idea - but beware of the
> mess you'll find in the wild with email. In our case, we try to parse
> Exchange (full of non-standard wonders like "TNEF" attachments") as well as
> email from virtually anywhere (GMail, Yahoo, Joe's email client...). If you
> can crack that you'll be on to something. We have even more complexity in
> that we read "Microsoft Journals" which wrap the standard SMTP layout in a
> Microsoft layer (you'll see this at large Exchange shops doing this kind of
> thing for use cases like compliance).
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/ListenSMTP-processor-tp10510p12827.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>