Re: [jira] [Commented] (NIFI-994) Processor to tail files

I think we are on the same page, but I left out some details. The key is that the processor always starts at the beginning when it finds a file but discards content it thinks was previously committed downstream. One approach could be storing a checksum of processed content with the other state

Re: PutHDFS Configuration Issue

Glad that first issue was resolved! I am by no means a kerberos expert, but having set this up once before, the setup should be something like the following: nifi.kerberos.krb5.file=/etc/krb5.conf (or wherever your conf file is) This is the file that would have your realms defined. Then on

Re: PutHDFS Configuration Issue

Hey Domenic - Since it looks like you're going to be using Kerberos, you should probably be aware of NIFI-997 , which I recently posted a patch to fix. Please let us know if any other issues come up. Ricky On Wed, Sep 30, 2015 at 2:01 PM, Bryan

PutHDFS Configuration Issue

Hello, I am trying to set up the PutHDFS processor on NiFi, and I am running into an issue. I have the Hadoop Configuration Resources set and pointing to my core-site.xml and hdfs-site.xml, and I have a Kerberos Principal and Keytab file. However, I am getting the error below. 'Kerberos

Re: [jira] [Commented] (NIFI-994) Processor to tail files

Joe can you please reply as a comment in the ticket instead of an email thread? It makes it a lot easier to keep track of the discussion. Thanks, Joe- - - - - - Joseph Percivall linkedin.com/in/Percivall e: joeperciv...@yahoo.com On Wednesday, September 30, 2015 1:27 PM, Joe Skora

Re: PutHDFS Configuration Issue

Hi Domenic, It sounds like you are on the right path... just to confirm, did you restart NiFi after setting nifi.kerberos.krb5.file in nifi.properties? It will only pick up changes to nifi.properties on a restart. Also, On Wed, Sep 30, 2015 at 12:38 PM, DomenicPuzio

[GitHub] nifi pull request: NIFI-988: PutDistributedMapCache processor

Github user joemeszaros commented on the pull request: https://github.com/apache/nifi/pull/92#issuecomment-144332289 @markap14 Did you get a proper answer for your cache related questions? If you do not have any concern with this new processor, it would be reasonable to implement the

[GitHub] nifi pull request: NIFI-988: PutDistributedMapCache processor

Github user markap14 commented on the pull request: https://github.com/apache/nifi/pull/92#issuecomment-144378199 @joemeszaros the concern that I have with the notion of the ExtendedDistributedMapCacheClient is that once that is released, it will have the same caveats as the

Re: need help with Async code

Sumo, I did some digging around on your Github Repo and see that you've migrated your ControllerService lookup to your @OnScheduled method, making use of the ProcessContext. This approach is certainly more preferred in terms of allowing configuration of the Processor than the prior method you

Re: need help with Async code

thanks for clarifying on getControllerServiceIdentifiers API. I have an other question: If I have a processor that is designed to have on side effect on FlowFile, what is the best/clean way to read content of the flowfile? e.g., my processor’s only job is to log content of FlowFile, is there a

Re: [jira] [Commented] (NIFI-994) Processor to tail files

Joe, The problem with "tail -F" is that if NiFi is restarted and then we do essentially "tail -F" we may have missed a lot of data that was written to the log file while NiFi was down. The idea behind this Processor is to be able to recover that data, even if it was written to a log file (or