Alexey created SPARK-23832: ------------------------------ Summary: Adding possibility to set timestamp into KafkaRowWriter Key: SPARK-23832 URL: https://issues.apache.org/jira/browse/SPARK-23832 Project: Spark Issue Type: Improvement Components: Input/Output, Structured Streaming Affects Versions: 2.3.0 Reporter: Alexey
At the moment we can't set timestamp for Structured Streaming into Kafka sink. This timestamp can be used for log.message.timestamp.type=CreateTime processing semantics. We would add timestamp column for this purposes as done for topic, key and value columns at the moment. The problem code line is this: [https://github.com/apache/spark/blob/3e778f5a91b0553b09fe0e0ee84d771a71504960/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala#L91] -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org