Alexey created SPARK-23832:
------------------------------
Summary: Adding possibility to set timestamp into KafkaRowWriter
Key: SPARK-23832
URL: https://issues.apache.org/jira/browse/SPARK-23832
Project: Spark
Issue Type: Improvement
Components: Input/Output, Structured Streaming
Affects Versions: 2.3.0
Reporter: Alexey
At the moment we can't set timestamp for Structured Streaming into Kafka sink.
This timestamp can be used for log.message.timestamp.type=CreateTime processing
semantics.
We would add timestamp column for this purposes as done for topic, key and
value columns at the moment.
The problem code line is this:
[https://github.com/apache/spark/blob/3e778f5a91b0553b09fe0e0ee84d771a71504960/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala#L91]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
