gaborgsomogyi commented on a change in pull request #32609: URL: https://github.com/apache/spark/pull/32609#discussion_r638530241
########## File path: docs/structured-streaming-kafka-integration.md ########## @@ -512,6 +526,17 @@ The following configurations are optional: </tr> </table> +### Details on timestamp offset options + +The returned offset for each partition is the earliest offset whose timestamp is greater than or equal to the given timestamp in the corresponding partition. +The behavior varies across options if the matched offset doesn't exist - check the description of each option. + +Spark simply passes the timestamp information to <code>KafkaConsumer.offsetsForTimes</code>, and doesn't interpret or reason about the value. +For more details on <code>KafkaConsumer.offsetsForTimes</code>, please refer <a href="https://kafka.apache.org/21/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#offsetsForTimes-java.util.Map-";>javadoc</a> for details. Review comment: I've double checked and no "latest" found. I think from maintenance perspective it would be the best to lower the version to the minimum. Such case the API must remain the same and we don't have to touch it time to time. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
