viirya commented on a change in pull request #32747:
URL: https://github.com/apache/spark/pull/32747#discussion_r655123411
##########
File path:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala
##########
@@ -66,30 +67,40 @@ private[kafka010] trait KafkaOffsetReader {
/**
* Resolves the specific offsets based on timestamp per topic-partition.
* The returned offset for each partition is the earliest offset whose
timestamp is greater
- * than or equal to the given timestamp in the corresponding partition. If
the matched offset
- * doesn't exist, depending on `failsOnNoMatchingOffset` parameter, the
offset will be set to
- * latest or this method throws an error.
+ * than or equal to the given timestamp in the corresponding partition.
+ *
+ * If the matched offset doesn't exist, the behavior depends on the
destination and the option:
+ *
+ * - isStartingOffsets = false => implementation should provide the offset
same as 'latest'
+ * - isStartingOffsets = true => implementation should follow the strategy
on non-matching
+ * starting offset, passed as
`strategyOnNoMatchStartingOffset`
*
* @param partitionTimestamps the timestamp per topic-partition.
- * @param failsOnNoMatchingOffset whether to fail the query when no matched
offset can be found.
*/
def fetchSpecificTimestampBasedOffsets(
partitionTimestamps: Map[TopicPartition, Long],
- failsOnNoMatchingOffset: Boolean): KafkaSourceOffset
+ isStartingOffsets: Boolean,
Review comment:
`isStartingOffsets` here actually means the starting offset of the
query, right? We have another concept that is the start offset for current
batch (e.g. `isStartingOffsets` in `fetchPartitionOffsets`), do you think it is
better to rename this as `isQueryStartingOffsets`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
