Re: Is there a Kafka sink for Spark Structured Streaming

2017-05-22 Thread Michael Armbrust
option("kafka.bootstrap.servers", "host1:port1,host2:port2") \ >> .option("topic", "topic1") \ >> .option("checkpointLocation", "/path/to/HDFS/dir") \ >> .start() >> >> Described here: >> >> https://databricks.com/blog/2017/04/26/processing-data-in-apache-kafka-with-structured-streaming-in-apache-spark-2-2.html >> >> >> >> On 19 May 2017 at 10:45, <kanth...@gmail.com> wrote: >> >>> Is there a Kafka sink for Spark Structured Streaming ? >>> >>> Sent from my iPhone >>> >> >> >

Re: Is there a Kafka sink for Spark Structured Streaming

2017-05-19 Thread kant kodali
ot;to_json(struct(*)) AS >>> value") \ >>> .writeStream \ >>> .format("kafka") \ >>> .option("kafka.bootstrap.servers", "host1:port1,host2:port2") \ >>> .option("topic", "topic1"

Re: Is there a Kafka sink for Spark Structured Streaming

2017-05-19 Thread Tathagata Das
.option("topic", "topic1") \ >> .option("checkpointLocation", "/path/to/HDFS/dir") \ >> .start() >> >> Described here: >> >> https://databricks.com/blog/2017/04/26/processing-data-in-apache-kafka-with-structured-streaming-in-apache-spark-2-2.html >> >> >> >> On 19 May 2017 at 10:45, <kanth...@gmail.com> wrote: >> >>> Is there a Kafka sink for Spark Structured Streaming ? >>> >>> Sent from my iPhone >>> >> >> >

Re: Is there a Kafka sink for Spark Structured Streaming

2017-05-19 Thread kant kodali
quot;host1:port1,host2:port2") \ > .option("topic", "topic1") \ > .option("checkpointLocation", "/path/to/HDFS/dir") \ > .start() > > Described here: > > https://databricks.com/blog/2017/04/26/processing-data-in-apache-kafka-with-s

Re: Is there a Kafka sink for Spark Structured Streaming

2017-05-19 Thread kanth909
> .option("checkpointLocation", "/path/to/HDFS/dir") \ > .start() > Described here: > https://databricks.com/blog/2017/04/26/processing-data-in-apache-kafka-with-structured-streaming-in-apache-spark-2-2.html > > >> On 19 May 2017 at 10:45, <kanth...@gmail.com> wrote: >> Is there a Kafka sink for Spark Structured Streaming ? >> >> Sent from my iPhone >

Re: Is there a Kafka sink for Spark Structured Streaming

2017-05-19 Thread Patrick McGloin
-2-2.html On 19 May 2017 at 10:45, <kanth...@gmail.com> wrote: > Is there a Kafka sink for Spark Structured Streaming ? > > Sent from my iPhone >

Is there a Kafka sink for Spark Structured Streaming

2017-05-19 Thread kanth909
Is there a Kafka sink for Spark Structured Streaming ? Sent from my iPhone