Re: creating a single kafka producer object for all partitions

2015-01-12 Thread Hafiz Mujadid
message will be added to the discussion below: http://apache-spark-user-list.1001560.n3.nabble.com/creating-a-single-kafka-producer-object-for-all-partitions-tp21097.html To unsubscribe from Apache Spark User List, click here. NAML http://apache-spark-user-list.1001560.n3.nabble.com/template

Re: creating a single kafka producer object for all partitions

2015-01-12 Thread Cody Koeninger
You should take a look at https://issues.apache.org/jira/browse/SPARK-4122 which is implementing writing to kafka in a pretty similar way (make a new producer inside foreachPartition) On Mon, Jan 12, 2015 at 5:24 AM, Sean Owen so...@cloudera.com wrote: Leader-not-found suggests a problem with

creating a single kafka producer object for all partitions

2015-01-12 Thread Hafiz Mujadid
this? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/creating-a-single-kafka-producer-object-for-all-partitions-tp21097.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Re: creating a single kafka producer object for all partitions

2015-01-12 Thread Sean Owen
Leader-not-found suggests a problem with zookeeper config. It depends a lot on the specifics of your error. But this is really a Kafka question, better for the Kafka list. On Mon, Jan 12, 2015 at 11:10 AM, Hafiz Mujadid hafizmujadi...@gmail.com wrote: Actually this code is producing error leader