custom ProducerRecord timestamp and compact+delete cleanup

2017-11-27 Thread Michal Hariš
Hi, I've read on the message timestamp(KIP-32) that for time-rolling deletes use LogAppendTime, but this only applies to delete policy. What timestamp is used when the cleanup policy is _both_ compact+delete? It would be very useful if it was the timestamp from the record metadata rather than

Re: Kafka Connect ++ Kafka Streams

2016-03-25 Thread Michal Hariš
a custom internal schema format for the in-memory runtime instead of just using Avro as the internal - the systems that talk in Avro would have a performance gain and non-Avro guys would have converters the same way they have them now. On Thu, Mar 24, 2016 at 11:46 AM, Michal Hariš <michal.ha

Kafka Connect ++ Kafka Streams

2016-03-24 Thread Michal Hariš
Hello Kafka people! Great to see Kafka Streams coming along, the design validates (and in many way supersedes) my own findings from working with various stream processing systems/frameworks and eventually ending-up using just a small custom library built directly around Kafka. I have set out