Thanks , for your answer srikanth. Current KafkaSpout doesn't support this new feature available in Kafka. I was looking for something similar a few weeks back.
Srikanth On Fri, May 15, 2015 at 3:04 PM, Cristian Makoto Sandiga < [email protected]> wrote: > Thank you Jeffery, yes. > > 2015-05-15 16:01 GMT-03:00 Jeffery Maass <[email protected]>: > > As I read your email and the link within it, I believe that your question >> is: >> >> * Does there exist or is anyone working on a KafkaSpout which stores its >> offsets in a highly durable Kafka Topic, instead of within ZooKeeper? >> >> Is that a correct interpretation of your question? >> >> Thank you for your time! >> >> +++++++++++++++++++++ >> Jeff Maass <[email protected]> >> linkedin.com/in/jeffmaass >> stackoverflow.com/users/373418/maassql >> +++++++++++++++++++++ >> >> >> On Fri, May 15, 2015 at 1:34 PM, Cristian Makoto Sandiga < >> [email protected]> wrote: >> >>> Hi, >>> >>> I want to replace my RedisSpout and i was developing a POC, with >>> KafkaSpout, but i'm worried for consumer offset manager with zookeeper and >>> HighLoad write, maybe loose offset, failover, etc. >>> >>> I was thinking about a zookeeper cluster for any data loose, but i think >>> is still better save in kafka server offset at least is a kafka >>> recommendation >>> https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka >>> . >>> >>> That has any documentation about offset.storage in KafkaSpout i read all >>> mailing list but i didn't find nothing talking about it, has anybody >>> working in a possible feature? or maybe somebody has a good recommendation >>> for me?. >>> >>> I really will appreciate your help. >>> >>> Application with normal traffic: 100 000 msgs/seg High 500 000 msgs/seg >>> >>> replication factor-> 2 >>> >>> 1 topic -> 10 partitions >>> 1 topic -> 10 partitions >>> 1 topic -> 5 partition >>> >>> storm-kafka -> 0.9.4, >>> kafka_2.10 -> 0.8.2.1 >>> >> >> >
