When ia created topic visible?

2017-01-24 Thread F21
Hi all, I working on a Go program that needs to create topics and consume messages from kafka. Currently, I am creating a topic by setting up the appropriate nodes in zookeeper. Once the changes are committed to zookeeper, how long does it take for kafka to see the topic? I am noticing in

Does kafka support reloading TLS certificates without downtime?

2016-12-19 Thread F21
I want to use Vault[1] to manage my TLS certificates. The certificates would be issued by Vault and have a short lifetime of around 72 hours. There would be a co-process to request a new certificate before expiry. Does kafka provide any method to reload the TLS certificate without any down

Re: any update on this?

2016-09-20 Thread F21
One possible solution might be to use parkeeper, which uses consul as the backend and exposes a facade that looks like zookeeper: https://github.com/glerchundi/parkeeper The project doesn't seem very active though and it is unclear whether it supports all the features that are used by kafka.

Re: Consumer api 0.10 backwards compatible with Kafka 0.8

2016-09-07 Thread F21
I believe there were some incompatibilities between 0.8.x and 0.9.x for the clients. If you are upgrading from 0.8.x to 0.10.x, you should check out the upgrade guide available here: http://kafka.apache.org/documentation.html#upgrade_10 Cheers, Francis On 7/09/2016 6:19 PM, Vadim Keylis

Re: Using kafka as a "message bus" for an event store

2016-09-06 Thread F21
wrote: inline On Mon, Sep 5, 2016 at 11:58 PM, F21 <f21.gro...@gmail.com> wrote: Hi Tom, Thank you so much for your response. I had a feeling that approach would run into scalability problems, so thank you for confirming that. Another approach would be to have each service request a subscr

Re: Using kafka as a "message bus" for an event store

2016-09-05 Thread F21
a way to check when the topic was last read from? Does this sound like a saner way? Cheers, Francis On 5/09/2016 11:00 PM, Tom Crayford wrote: inline On Mon, Sep 5, 2016 at 12:00 AM, F21 <f21.gro...@gmail.com> wrote: Hi all, I am currently looking at using Kafka as a "

Using kafka as a "message bus" for an event store

2016-09-04 Thread F21
Hi all, I am currently looking at using Kafka as a "message bus" for an event store. I plan to have all my events written into HBase for permanent storage and then have a reader/writer that reads from HBase to push them into kafka. In terms of kafka, I plan to set it to keep all messages