HyukjinKwon commented on code in PR #59219:
URL: https://github.com/apache/spark/pull/59219#discussion_r4170925979
##########
connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala:
##########
@@ -374,13 +374,16 @@ private[kafka010] class KafkaSourceProvider extends
DataSourceRegister
STARTING_OFFSETS_OPTION_KEY, EarliestOffsetRangeLimit) match {
case EarliestOffsetRangeLimit => // good to go
case LatestOffsetRangeLimit =>
- throw new IllegalArgumentException("starting offset can't be latest " +
- "for batch queries on Kafka")
+ throw KafkaExceptions.invalidBatchOffset(
+ offsetName = STARTING_OFFSETS_OPTION_KEY,
Review Comment:
**Non-blocking (P2):** `STARTING_OFFSETS_OPTION_KEY` /
`ENDING_OFFSETS_OPTION_KEY` are the lowercased `CaseInsensitiveMap` lookup keys
(`"startingoffsets"` / `"endingoffsets"`, see
`KafkaSourceProvider.scala:581-582`). Passing them here makes the structured
`offsetName` parameter, and the rendered message, lowercase: `startingoffsets
can't be latest for batch queries on Kafka`.
That contradicts the PR description ("Messages now consistently identify the
relevant option as `startingOffsets` or `endingOffsets`"). It also fails the
new `testInvalidBatchOffset` assertions, which expect `offsetName ->
"startingOffsets"` and the camelCase message on both the V1 and V2 suites. The
per-partition message also regresses from the existing `startingOffsets for t-0
...` wording. Because `offsetName` becomes part of a stable error contract,
it's worth getting right before release.
Please pass the user-facing names (`"startingOffsets"` / `"endingOffsets"`)
at all four call sites here and at lines 385, 398, and 405, while keeping the
lowercase constants for option lookup.
**Verification:**
- **Behavior:** Structured-error assertions for global, topic-level, and
partition-level startingOffsets=latest and endingOffsets=earliest on both the
V1 and V2 Kafka batch read paths check the condition, SQLSTATE 22023, camelCase
offsetName parameter, and exact message.
- **Compatibility:** Batch reads with valid starting/ending offsets,
including mixed-case option keys, still resolve and validate as before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
