Processor disabled state not maintained across template download/import

If I create a template from a flow that has some disabled components, when I download and import that template into a different NiFi instance, the disabled state of those components is lost (they are no longer disabled.) I’m not sure when this information is being lost (is it saved in the

Re: Double-click action

I like this, Matt. It has often find myself double clicking to try to configure, or view configuration. It just seems natural. Chris McDermott Remote Business Analytics STaTS/StoreFront Remote HPE Storage Hewlett Packard Enterprise Mobile: +1 978-697-5315 On 10/24/16, 9:50 AM, "Matt

Re: [ANNOUNCE] Apache NiFi 0.7.1 release

Great job folks! Chris McDermott Remote Business Analytics STaTS/StoreFront Remote HPE Storage Hewlett Packard Enterprise Mobile: +1 978-697-5315 On 10/21/16, 12:02 AM, "Joe Skora" wrote: Hello, The Apache NiFi team would like to announce the release of Apache

Fetch file - false file not founds

I think I have found a bug in FetchFile. I have a flow that reads files from an NFS volume. To simulate the NFS server being offline/unreachable I am using IP tables to block all TCP traffic between the NFS server and the host running NiFi. Trying to read files using FetchFile, I am seeing

Re: ConsumeJMS and Acknowledgement Mode

you agree since adding ACK mode options would be a simply change, although at the moment I am reluctant to do so (see my explanation in JIRA) Cheers Oleg > On Sep 14, 2016, at 10:13 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wr

Re: Binary Data over the PutTCP processor

Hello friend, I’m not really a developer but I thought I might chime in while you wait for a more “official” answer. Reading the original JIRA for the PutTCP processor and glancing at the code, it seems to send the binary data unmolested. The Character Set setting (e.g. UTF-8) is only to

Re: ConsumeJMS and Acknowledgement Mode

(actual value 1) is the default. So although it is not being set explicitly it is set. We can definitely expose the other two (client ack and dups ok). Please raise JIRA. Cheers Oleg On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote

Re: ConsumeJMS and Acknowledgement Mode

st files on NiFi restart. Do you concur? Chris McDermott Remote Business Analytics STaTS/StoreFront Remote HPE Storage Hewlett Packard Enterprise Mobile: +1 978-697-5315 On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <chris.mcderm...@hpe.com> wr

Re: ConsumeJMS and Acknowledgement Mode

4, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: > > Hi Oleg, > > Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. S

Re: ConsumeJMS and Acknowledgement Mode

ecommend Publish/ConsumeJMS pair. As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using? Cheers Oleg On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevi

ConsumeJMS and Acknowledgement Mode

Hey folks, The GetJMSQueue processor allows configuration of the Acknowledgment Mode. Acknowledgement Mode Client Acknowledge · Client Acknowledge · Auto Acknowledge The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may

Re: NIFI-2680

IFI-2680 indicates that the problem affects 0.7.0, >0.8.0, and 1.0.0. > >From your stack trace here it appears that you're using Kafka Processors from an 0.7.x release. Have you verified that this issue still exists >with a newer version of NiFi? > &

Re: NIFI-2680

trace here it appears that you're using Kafka Processors from an 0.7.x release. Have you verified that this issue still exists with a newer version of NiFi? Thanks! -Mark > On Sep 8, 2016, at 8:00 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)

NIFI-2680

Hello dev folks. Would it be possible for someone to take a look at NIFI-2680? https://issues.apache.org/jira/browse/NIFI-2680?jql=project%20%3D%20NIFI%20AND%20reporter%20%3D%20currentUser() I believe I have a similar situation where writing to Kafka fails when the Kafka broker is restarted.

Re: PostHTTP Penalize file on HTTP 5xx response

fferent behavior please advise. Thanks Joe On Thu, Sep 1, 2016 at 1:11 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: > Thanks, everyone for the feedback. I’ll file a JIRA for this and see if I can

Re: PostHTTP Penalize file on HTTP 5xx response

nt input). So penalizing the FlowFile so that it >> > can be retried after a little bit is reasonable >> > IMO. >> > >> > When using the prioritizers, any FlowFile that is penalized will not hold >> > up other FlowFiles. They are alwa

PostHTTP Penalize file on HTTP 5xx response

I wanted to ask if it would be at all sane to have the PostHTTP processor penalize a flowfile on 5xx response. 5xx indicates that the request may be good but it cannot be handle by the server Currently it seems the processor routes files eliciting this response to the failure output but does

Re: PutKakfa can send a file to its success output although the Kafka put fails

for a bug report. Thanks Joe On Thu, Aug 25, 2016 at 4:45 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: > I just wanted to sanity check this before opening a JIRA. > > > > I have PutK

PutKakfa can send a file to its success output although the Kafka put fails

I just wanted to sanity check this before opening a JIRA. I have PutKafka processor where I’ve set the Max Record Size property to 1MB. If a transfer larger file to it, I see a bulletin and the following message in the log file, but the flowfile is transferred to the success output. Caused by:

Re: Max Kafka message size

exactly what I had in mind. As you mentioned earlier, that is where the default value came from in the first place. So, yes that is what we’re going to link together. Cheers Oleg > On Aug 21, 2016, at 6:15 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris

Re: Max Kafka message size

0, 2016, at 7:03 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: > > Jira is https://issues.apache.org/jira/browse/NIFI-2614. > > Thanks, > > Chris McDermott > > Remote Business Analytic

Re: Max Kafka message size

Jira is https://issues.apache.org/jira/browse/NIFI-2614. Thanks, Chris McDermott Remote Business Analytics STaTS/StoreFront Remote HPE Storage Hewlett Packard Enterprise Mobile: +1 978-697-5315 On 8/20/16, 6:57 PM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <

Re: Max Kafka message size

ain/java/org/apache/nifi/processors/kafka/ > PublishingContext.java#L36-L41 > [2] > https://github.com/apache/nifi/blob/rel/nifi-0.6.1/nifi- > nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/ > main/java/org/apache/nifi/processors/kafka/PutKafka.java#

Re: Max Kafka message size

he/nifi/blob/rel/nifi-0.7.0/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/main/java/org/apache/nifi/processors/kafka/PutKafka.java#L495 [3] https://kafka.apache.org/08/configuration.html [4] http://ingest.tips/2015/01/21/handling-large-messages-kafka/ On Sat, Aug 20, 201

Max Kafka message size

Hi folks, From experimentation and looking at the code it seems that the max message size that can be sent via the PublishKafka and PutKafka processors in 0.7.0 is 1MB. Can someone please confirm my read on this? Thanks, Chris McDermott Remote Business Analytics STaTS/StoreFront Remote HPE

Re: Connection not honoring flow file expiration

seems like >the ones you are noticing are just about to get expired. Please >confirm? > >Thanks >Joe > >On Tue, Aug 2, 2016 at 2:06 PM, McDermott, Chris Kevin (MSDU - >STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: >> I have a connection that has a Flo

Re: GetJMSQueue does not detect dead connections

eg > > >> On Jun 16, 2016, at 1:34 PM, McDermott, Chris Kevin (MSDU - >> STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: >> >> Folks, >> >> I’ve been trying to test my GetJMSQueue configuration so that it detects a >> dead broker c

Re: connection fileQueue size

Pradeep, We are using the the Ganglia reporting task with the ELK stack to monitor our NiFis. The reporting task sends metrics to a LogStash running a ganglia plugin. We use Kibana to visualize the metrics and Watcher to create notifications. It was fairly easy to setup and works quite

Re: Need help on better usage of Email Processor

Hi Sachin, We are using logstash/elastic-search (the ELK stack) and Watcher to do this sort of thing. We find the separation of concerns more reliable and flexible rather than trying to have NiFi monitor itself. Cheers, Chris McDermott Remote Business Analytics STaTS/StoreFront Remote HPE

Re: GetJMSQueue does not detect dead connections

rify? >>> >>> Also, yeh tests I was referring to are >>> https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/test/java/org/apache/nifi/jms/processors >>> >>> Let me know if you need more help >>&g

Re: GetJMSQueue does not detect dead connections

On 6/16/16, 2:53 PM, "Oleg Zhurakousky" <ozhurakou...@hortonworks.com> wrote: >Yes, you can probably look at the test case for it since it uses embedded AMQP. > >Let m know if you need more help with it. > >Cheers >Oleg >> On Jun 16, 2016, at 2:50 PM,

Re: GetJMSQueue does not detect dead connections

ting Get/PutJMS* in favor of Publish/SubscribeJMS, >I’d suggest start using those once. > >Cheers >Oleg > > >> On Jun 16, 2016, at 1:34 PM, McDermott, Chris Kevin (MSDU - >> STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: >> >> Folks, >&g

GetJMSQueue does not detect dead connections

Folks, I’ve been trying to test my GetJMSQueue configuration so that it detects a dead broker connection and fails over to an alternate broker. When I say dead connection I mean TCP connection that has not been closed but is no longer passing traffic. In the real world this typically happens

Re: unwedgeable flow caused by "Cannot create Provenance Event Record because FlowFile UUID is not set"

Thanks, Oleg. I’ve created NIFI-2015. Regards, Chris McDermott Remote Business Analytics STaTS/StoreFront Remote HPE Storage Hewlett Packard Enterprise Mobile: +1 978-697-5315 https://www.storefrontremote.com On 6/13/16, 3:10 PM, "Oleg Zhurakousky" wrote:

unwedgeable flow caused by "Cannot create Provenance Event Record because FlowFile UUID is not set"

2016-06-13 17:33:30,275 ERROR [Timer-Driven Process Thread-4] o.a.n.p.attributes.UpdateAttribute java.lang.IllegalStateException: Cannot create Provenance Event Record because FlowFile UUID is not set at

Re: GetKafka blowing up with assertion error in Kafka client code

gate further and let you know > >Thanks again for looking into this > >Oleg > >Sent from my iPhone > >> On Apr 28, 2016, at 18:41, McDermott, Chris Kevin (MSDU - >> STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: >> >> Oleg, >> >&

Re: GetKafka blowing up with assertion error in Kafka client code

s >Oleg > >> On Apr 14, 2016, at 8:25 PM, McDermott, Chris Kevin (MSDU - >> STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: >> >> I looked at the Kafka client code and it seemed to me to be a bug in the >> caller. There is a map passed that ma

Re: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)

er logging to avoid these 'false >alarm' log messages. > >Thanks >-Mark > >[1] https://issues.apache.org/jira/browse/NIFI-1780 ><https://issues.apache.org/jira/browse/NIFI-1780> > > >> On Apr 14, 2016, at 3:48 PM, McDermott, Chris Kevin (MSDU - >> STaTS/Storefron

RE: GetKafka blowing up with assertion error in Kafka client code

means in the context of what you are trying to do but its clearly a problem originated in Kafka. Could you share your config or whatever other details? Cheers Oleg > On Apr 14, 2016, at 4:00 PM, McDermott, Chris Kevin (MSDU - > STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: >

GetKafka blowing up with assertion error in Kafka client code

I’m running based of of 0.7.0 Snapshot. The GetKafka config is pretty generic. Batch size 1, 1 concurrent task. 2016-04-14 19:27:23,204 ERROR [Timer-Driven Process Thread-9] o.apache.nifi.processors.kafka.GetKafka java.lang.IllegalStateException: java.util.concurrent.ExecutionException:

Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)

I’m seeing this a lot in my logs. Does anyone have any idea what it is about? The cluster view in the UI shows all nodes connected. 2016-04-14 19:45:29,310 INFO [Framework Task Thread Thread-2-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Opening socket connection to server

Re: NPE in PutKafka processors

String(this.delimiterBytes, StandardCharsets.UTF_8) : null; +} + +/** * Returns the key bytes as String */ String getKeyBytesAsString() { On 4/13/16, 2:54 PM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <chris.mcderm...@hpe.com> wro

Re: Small issue with FetchFile

. > >Thanks >Joe > >On Fri, Apr 1, 2016 at 12:38 PM, McDermott, Chris Kevin (MSDU - >STaTS/StorefrontRemote) <chris.mcderm...@hpe.com> wrote: >> When FetchFile encounters a FileNotFound it logs an ERROR. My problem with >> that is that FileNotFound may actual

Small issue with FetchFile

When FetchFile encounters a FileNotFound it logs an ERROR. My problem with that is that FileNotFound may actually be expected or a at least tolerable, and I don’t want to see a bulletin for that. Does anyone think it would be appropriate to reduce FileNotFound to be logged as WARN, or

Re: Error using PutKafka with round robin partitioning

th record: " + record.partition() > + " is not in the range > [0..." > + numPartitions > + "]."); > >so, when partition count doesn’t match the partition size it

Error using PutKafka with round robin partitioning

I’m running off of the HEAD of master (or at least close to it - e4b7e4.) It looks like the processor has lost its mind with the respect to the number of partitions. Granted my cluster has been in rough shape mostly through my own abuse and I probably caused the problem myself. Has anyone

Re: Kafka error running latest 0.6.0.SNAPSHOT

? On 3/25/16, 2:47 PM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <chris.mcderm...@hpe.com> wrote: >I’m running the latest from master which I built myself. I’m encountering the >follow PutKafka error. Any suggestions? > > >Caused by: java.lan

Kafka error running latest 0.6.0.SNAPSHOT

I’m running the latest from master which I built myself. I’m encountering the follow PutKafka error. Any suggestions? Caused by: java.lang.IllegalStateException: Failed to create partitioner at org.apache.nifi.processors.kafka.KafkaPublisher.(KafkaPublisher.java:75) ~[na:na]

Question about code formatting for pull requests

I’m getting a pull request ready and discovered that my IDE did a bunch of code “cleanup” on the files I modified. The cleanup is mostly reordering imports and removing leading and trailing whitespace. Are these changes desirable/acceptable or do I need to go back and undo them? The commit

Re: NiFi-1660

nifi/blob/master/nifi-commons/nifi-expression-language/pom.xml >> . >> If a parent POM has declared a version already (such as the NiFi parent POM >> declaring jayway 2.0.0 at present), you may want to keep that unless you >> need to override for some reason. >> >