He's just suggesting that since the DataStreamWriter start() method can
fill in an option named "path", we should make that a synonym for "topic".
Then you could do something like.
df.writeStream.format("kafka").start("topic")
Seems reasonable if people don't think that is confusing.
On Mon, May 1, 2017 at 8:43 AM, Cody Koeninger <c...@koeninger.org> wrote:
> I'm confused about what you're suggesting. Are you saying that a
> Kafka sink should take a filesystem path as an option?
>
> On Mon, May 1, 2017 at 8:52 AM, Jacek Laskowski <ja...@japila.pl> wrote:
> > Hi,
> >
> > I've just found out that KafkaSourceProvider supports topic option
> > that sets the Kafka topic to save a DataFrame to.
> >
> > You can also use topic column to assign rows to topics.
> >
> > Given the features, I've been wondering why "path" option is not
> > supported (even of least precedence) so when no topic column or option
> > are defined, save(path: String) would be the least priority.
> >
> > WDYT?
> >
> > It looks pretty trivial to support --> see KafkaSourceProvider at
> > lines [1] and [2] if I'm not mistaken.
> >
> > [1] https://github.com/apache/spark/blob/master/external/
> kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/
> KafkaSourceProvider.scala#L145
> > [2] https://github.com/apache/spark/blob/master/external/
> kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/
> KafkaSourceProvider.scala#L163
> >
> > Pozdrawiam,
> > Jacek Laskowski
> > ----
> > https://medium.com/@jaceklaskowski/
> > Mastering Apache Spark 2 https://bit.ly/mastering-apache-spark
> > Follow me at https://twitter.com/jaceklaskowski
> >
> > ---------------------------------------------------------------------
> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>
