Re: Kafka 0.9.0.1 plan

2016-02-05 Thread Pierre-Yves Ritschard
Hi, Since it's still early in 0.9.0.0's life, if KAFKA-3006 has a chance of making the cut (provided a resolution is attained on the KIP-45) it would be great to avoid leaving too much time for code relying on Arrays to become common place. On Sat, Feb 6, 2016 at 12:05 AM, Ismael Juma

Re: [DISCUSS] KIP-45 Standardize all client sequence interaction on j.u.Collection.

2016-02-03 Thread Pierre-Yves Ritschard
A good compromise would be to add an arity with a single TopicPartition. Jason Gustafson writes: > Most of the use cases of pause/resume that I've seen work only on single > partitions (e.g in Kafka Streams), so the current varargs method is kind of > nice. It would also be nice to be able to

Re: [DISCUSS] KIP-45 Standardize all client sequence interaction on j.u.Collection.

2016-01-27 Thread Pierre-Yves Ritschard
that most users > haven't finished (or perhaps haven't even started) migrating their code to > use the new consumer. > > What do you think? > > Thanks, > Jason > > > On Tue, Jan 26, 2016 at 11:52 AM, Pierre-Yves Ritschard <p...@spootnik.org> > wrote: > >> >> I updated the KIP accordingly. >> >> Cheers, >> - pyr >>

Re: [DISCUSS] KIP-45 Standardize all client sequence interaction on j.u.Collection.

2016-01-26 Thread Pierre-Yves Ritschard
turn types). > > Hopefully it should be straightforward to address these points. > > Thanks, > Ismael > > On Tue, Jan 26, 2016 at 9:00 AM, Pierre-Yves Ritschard <p...@spootnik.org> > wrote: > >> >> KAFKA-3006 is under review, and would change some commonly use

[ANN] kinsky: clojure 0.9.0 client

2016-01-07 Thread Pierre-Yves Ritschard
Hi list, While the 0.9.0.0 client lib is great to work with, I extracted some of the facade code I use internally into a library which smooths some aspects of interacting with Kafka from Clojure. The library provides a simple way to build rebalance listeners, serializers and deserializers. It

Unifying kafka-clients call signatures

2015-12-21 Thread Pierre-Yves Ritschard
Hi list, I've been working on an issue at https://issues.apache.org/jira/browse/KAFKA-3006 and it is now a good time to ask for feedback. The attached PR moves all signatures which accepted either arrays or java.util.List to accept java.util.Collection. The aim is to provide consumers of

article: hands-on kafka: dynamic DNS

2015-04-24 Thread Pierre-Yves Ritschard
Hi list! I just wanted to mention a small article I put together to describe an approach to leverage log compaction when you have compound types and messages are operations on that compound type with an example use-case: http://spootnik.org/entries/2015/04/23_hands-on-kafka-dynamic-dns.html

[ANN] Apache Cloudstack 4.5 kafka-event-bus plugin

2015-04-24 Thread Pierre-Yves Ritschard
Hi list, I thought I'd also mention that the next release of Apache Cloudstack adds the ability to publish all events happening on throughout the environment to kafka. Events are published as JSON. http://cloudstack-administration.readthedocs.org/en/latest/events.html#kafka-configuration

[ANN] sqlstream: Simple MySQL binlog to Kafka stream

2015-03-16 Thread Pierre-Yves Ritschard
Hi kafka, I just wanted to mention I published a very simple project which can connect as MySQL replication client and stream replication events to kafka: https://github.com/pyr/sqlstream When you don't have control over an application, it can provide a simple way of consolidating SQL data in

Re: [ANN] sqlstream: Simple MySQL binlog to Kafka stream

2015-03-16 Thread Pierre-Yves Ritschard
you anticipate that there will be duplicate events in the output stream, or are you going to go for exactly-once? -James On Mar 16, 2015, at 7:18 AM, Pierre-Yves Ritschard p...@spootnik.org wrote: Hi kafka, I just wanted to mention I published a very simple project which can connect

zookeeper-less offset management

2015-03-12 Thread Pierre-Yves Ritschard
Hi list, I was under the impression that consumers still needed to interact with zookeeper to track their offset. Going through recent Jiras to track the progress I see that https://issues.apache.org/jira/browse/KAFKA-1000 and https://issues.apache.org/jira/browse/KAFKA-1012 seem to indicate that

Log compaction recover strategy

2015-03-10 Thread Pierre-Yves Ritschard
Hi kafka, I've started implementing simple materialized views with the log compaction feature to test it out, and it works great. I'll share the code and an accompanying article shortly but first wanted to discuss some of the production implications my sandbox has. I've separated the project in

Re: powered by kafka

2014-11-10 Thread Pierre-Yves Ritschard
I guess I should mention that exoscale (https://exoscale.ch) is powered by kafka as well. Cheers, - pyr On Sun, Nov 9, 2014 at 7:36 PM, Gwen Shapira gshap...@cloudera.com wrote: I'm not Jay, but fixed it anyways ;) Gwen On Sun, Nov 9, 2014 at 10:34 AM, vipul jhawar

Re: [ANNOUNCEMENT] Apache Kafka 0.8.2-beta Released

2014-10-30 Thread Pierre-Yves Ritschard
Hi Joe et al. Congrats on the beta release! Do I read correctly that libraries can now rely on org.apache.kafka/kafka-clients which does not pull in scala anymore ? If so, awesome! - pyr On Tue, Oct 28, 2014 at 2:01 AM, Libo Yu yu_l...@hotmail.com wrote: Congrats! When do you think the