Re: PublishKafka setting record timestamp

2018-02-01 Thread Bryan Bende
Yes that is what I was saying for PublishKafka, but it seems like it only makes sense if you had one message per flow file, which is not always the case. If you have a flow file with 10k messages in it, and you are streaming it to Kafka based on some delimiter, would you want to use the same

Re: PublishKafka setting record timestamp

2018-01-31 Thread Mika Borner
Thanks Bryan For PublishKafka, couldn't this be done by using an attribute that contains the timestamp? For PublishKafkaRecord this sounds reasonable. I guess, I will open an enhancement request then. Mika> On 01/30/2018 02:58 PM, Bryan Bende wrote: Hello, The timestamp in Kafka is

Re: PublishKafka setting record timestamp

2018-01-30 Thread Bryan Bende
Hello, The timestamp in Kafka is separate from the headers, currently there isn't a way to specify the timestamp from NiFi. For PublishKafkaRecord, I could see having an option to take the value of a specified field from each record and make that the timestamp, assuming it can be converted to a

PublishKafka setting record timestamp

2018-01-29 Thread Mika Borner
Hi, Is it possible to set the producer record timestamp within the PublishKafka_1_0 / PublishKafkaRecord_1_0 processor? I tried to use the "Attributes to Send as Headers" option with a timestamp attribute, but this did not work. Not sure if the timestamp producer record's timestamp is in