Peter Schulz created SPARK-50160:
------------------------------------
Summary: KafkaWriteTask: support timestamp customisation
Key: SPARK-50160
URL: https://issues.apache.org/jira/browse/SPARK-50160
Project: Spark
Issue Type: Improvement
Components: SQL, Structured Streaming
Affects Versions: 3.5.3
Reporter: Peter Schulz
Currently, there is no way to customise the timestamp of a {{ProducerRecord}}
produced by the {{{}KafkaWriteTask{}}}. Here at Wikimedia we often use
event-time semantics, so it would be helpful if kafka records produced via
spark could resemble that.
I already checked out the code and was able to adapted it to fulfil our need.
Since I couldn't find any (closed) ticket concerned with this topic, I assume
no such feature has been denied until now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
