Re: [Beginner] Kafka 0.11 header support in Spark Structured Streaming
I made a JIRA for it - https://issues.apache.org/jira/browse/SPARK-23539 Unfortunately it is blocked by Kafka version upgrade, which has a few nasty issues related to Kafka bugs - https://issues.apache.org/jira/browse/SPARK-18057 On Wed, Feb 28, 2018 at 3:17 PM, karthikus wrote: > TD, > > Thanks for your response. > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
Re: [Beginner] Kafka 0.11 header support in Spark Structured Streaming
TD, Thanks for your response. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Re: [Beginner] Kafka 0.11 header support in Spark Structured Streaming
Unfortunately, exposing Kafka headers is not yet supported in Structured Streaming. The community is more than welcome to add support for it :) On Tue, Feb 27, 2018 at 2:51 PM, Karthik Jayaraman wrote: > Hi all, > > I am using Spark 2.2.1 Structured Streaming to read messages from Kafka. I > would like to know how to access the Kafka headers programmatically ? Since > the Kafka message header support is introduced in Kafka 0.11 ( > https://issues.apache.org/jira/browse/KAFKA-4208), is it supported in > Spark. ? If yes, can anyone point me to an example ? > > - Karthik >