Strange Timeout error with PutKafka

I say strange because the timeout (63ms) is so very short. The communication timeout I’ve set is 30 sec. Has anyone overseen this? 2016-03-16 14:41:38,227 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=852c8d42-a2fa-3478-b06b-84ceb6\ 6f8b0b] Failed t

Re: Strange Timeout error with PutKafka

Chris, I have seen that when the diskspace kafka relies on is full. We've seen a number of interesting exceptions recently in testing various configurations. But recommend checking that. Also, what version of Kafka broker are you using? With Apache NiFi 0.5.x we moved to the kafka client 0.9. I

Re: ListHDFS cache (again)

Conrad, If you are using a version that used DistributedMapCacheClientService, it also was saving state in a local file, $NIFI_HOME/conf/state/ You would need to delete that file, as well. I am not sure though if it would notice that the file was deleted without restarting NiFi. That processor

Re: ListenSyslog & scheduling strategy

Conrad, The Processor will only be scheduled to run on the Primary Node. However, it will be initialized on all nodes. It looks like the initialization step is where the Processor is opening the listening port. We should probably reconsider this in the future. It may make sense to instead run

Re: NPE in ExtractText

Conrad, I have never seen this personally. However, given your description, I was able to create a unit test that replicates the issue. I created a JIRA for this [1]. It was easily replicated. We should have a fix available for the 0.6.0 release. Thanks! -Mark [1] https://issues.apache.org/jir

Re: NPE in ExtractText

Thanks for the quick response. Conrad From: Mark Payne mailto:marka...@hotmail.com>> Reply-To: "users@nifi.apache.org" mailto:users@nifi.apache.org>> Date: Wednesday, 16 March 2016 at 15:18 To: "users@nifi.apache.org" mailto:users@nifi.

DetectDuplicate : java.net.ConnectException

Hi, Iam new to Apache Nifi and Iam trying it out on my Windows machine. When I use the DetectDuplicate processor, I get java.net.ConnectException. How can I start DistributedMapCacheClientService? Thanks, Arathi

Re: DetectDuplicate : java.net.ConnectException

Arathi, You'll need to add another Controller Service, one of type DistributedMapCacheServer, set up on port 4557 (to match your DistributedMapCacheClientService), and enable/start it. Then you should be able to connect successfully. Regards, Matt On Thu, Mar 17, 2016 at 4:15 PM, Arathi Maddula

Re: NPE in ExtractText

Conrad, What version of NiFi are you running? There was a bug that was introduced to the latest 'master' branch a few days ago that caused this, but it has since been fixed. If you are using an already released version, then we will obviously need to dig a little deeper. Thanks -Mark > On Mar

RE: DetectDuplicate : java.net.ConnectException

Hi Matt, Thanks a lot! DistributedMapCacheClientService is working now. Thanks, Arathi From: Matt Burgess [mailto:mattyb...@gmail.com] Sent: Thursday, March 17, 2016 4:18 PM To: users@nifi.apache.org Subject: Re: DetectDuplicate : java.net.ConnectException Arathi, You'll need to add another Co

Aw: Re: User Authentication with username and password

Hello everybody,   Iwas very busy - sorry for responding so late.   Thanks for all the feedback. I will probably go the LDAP way. I was looking for something quick and dirty, but I see that the architecture and implementation is not as quick as dirty and it's better that you take your time to

Re: Strange Timeout error with PutKafka

Chris, Assuming the client can validate #2 i am with you. Please do feel free to fire up a JIRA for this. Thanks Joe On Wed, Mar 16, 2016 at 1:24 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) wrote: > It turns out the root cause of the problem was an invalid topic name. > Strang

Re: DetectDuplicate : java.net.ConnectException

Hello, DistributedMapCacheClientService requires a DistributedMapCacheServer to be setup first. You then point the client service at the host (most likely local host in your scenario) and port of the server. Let us know if that doesn't help. -Bryan On Thu, Mar 17, 2016 at 3:50 PM, Arathi Maddu

NPE in ExtractText

Hi, I’m getting repeated NullPointerException reported for an ExtractText processor (processing the resultant splits from a ListenSyslog) thus: datanode2-cm1.mis-cds.local:9092ExtractText[id=4372efbf-efcf-3065-acd4-b8fdb91d64fb] ExtractText[id=4372efbf-efcf-3065-acd4-b8fdb91d64fb] failed to proc

Re: Can't use PutS3Object with user names containing commas

Joe, This is very likely possible. You can file a feature request/issue at [1] and someone will probably pick it up quickly. Have you tried wrapping the username in single quotes to see if that changes the behavior? [1] https://issues.apache.org/jira/secure/CreateIssue!default.jspa

Re: NPE in ExtractText

Full stop if you ever see a NullPointerException coming out of a processor it is 1000% a bug every time and please ping us any time. Thanks Joe On Wed, Mar 16, 2016 at 11:20 AM, Conrad Crampton wrote: > Hi Joe, > Great (not that it’s a bug) that I’m not going mad. > Thanks for looking into it. >

Can't use PutS3Object with user names containing commas

I'm trying to use a PutS3Object processor and I've run into a rather annoying problem. My company's user names include commas, so the various permission-granting attributes like "Read Permission User List" fail because they are hard-coded to split on commas. Would it be possible to add an att

RE: DetectDuplicate : java.net.ConnectException

Hi Adrin, I already created an instance of the DistributedMapCacheClientService with the server hostname pointing to localhost and port 4557. I use this service in the “Configure Processor” dialog of “DetectDuplicate” processor. But when I execute the workflow, I get java.net.ConnectException in

RE: DetectDuplicate : java.net.ConnectException

Hi Bryan, How can I setup a DistributedMapCacheServer? Thanks, Arathi From: Bryan Bende [mailto:bbe...@gmail.com] Sent: Thursday, March 17, 2016 4:09 PM To: users@nifi.apache.org Subject: Re: DetectDuplicate : java.net.ConnectException Hello, DistributedMapCacheClientService requires a Distrib

Re: Strange Timeout error with PutKafka

Joe, I’ll checkout the disk-space. We are running 0.9. If disk space is not the issue we’ll give 0.8 a try. Thanks very much for your quick reply. Cheers, Chris On 3/16/16, 11:04 AM, "Joe Witt" wrote: >Chris, >I have seen that when the diskspace kafka relies on is full. We've >seen a nu

Re: ListHDFS cache (again)

Hi Mark, I’m using 0.5.1 and yes I am using DistributedMapCacheClientService (that is the only option isn’t it?). Deleting the local file I guess is the piece that I’m missing – recreating the processor therefore would have a different id and I suppose this is the reason for it starting after th

Re: NPE in ExtractText

Conrad, Ahhh i *think* you have found a bug that is present. It appears to be this line. We should be doing a null check there before checking length of value. https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi

Re: NPE in ExtractText

Hi, I don’t know if this is expected behaviour but I think I understand why this is happening now. I have a regexp in the ExtractText processors viz: (?s:^.+: (\d\d?)(\w\w\w)(\d{4}) ([\d ]\d:\d\d:\d\d) Product=(.+?) OriginIP=(.+?) Origin=(.+?) Action=(.+?) SIP=(.+?) Source=(.+?) SPort=(\d+?) DI