[
https://issues.apache.org/jira/browse/SPARK-55300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18055574#comment-18055574
]
Kavpreet Grewal commented on SPARK-55300:
-----------------------------------------
PR: https://github.com/apache/spark/pull/54080
> Visualize the behavior of startingOffset/endingOffset option in Kafka
> connector
> -------------------------------------------------------------------------------
>
> Key: SPARK-55300
> URL: https://issues.apache.org/jira/browse/SPARK-55300
> Project: Spark
> Issue Type: Improvement
> Components: Documentation
> Affects Versions: 4.2
> Reporter: Kavpreet Grewal
> Priority: Major
> Labels: pull-request-available
>
> The guide doc of Kafka connector describes the behavior of
> startingTimestamp/endingTimestamp and related timestamp offset options as
> follows:
> [Structured Streaming + Kafka Integration Guide (Kafka broker version 0.10.0
> or higher) - Spark 4.1.0
> Documentation|https://spark.apache.org/docs/latest/structured-streaming-kafka-integration.html]
> {quote}The start point of timestamp when a query is started, a string
> specifying a starting timestamp for all partitions in topics being
> subscribed. Please refer the details on timestamp offset options below. If
> Kafka doesn't return the matched offset, the behavior will follow to the
> value of the option {{startingOffsetsByTimestampStrategy}}
> {quote}
> which can be illustrated in (ASCII style).
> We got some feedback from users that the text explanation in the guide doc
> may not be sufficient to come up with an illustration in their mind. Their
> feedback is something like “A picture is worth a thousand words”. So it would
> be great to visualize this and update the guide doc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
