HeartSaVioR commented on issue #25237: [SPARK-28489][SS] Fix a bug that KafkaOffsetRangeCalculator.getRanges may drop offsets URL: https://github.com/apache/spark/pull/25237#issuecomment-514465712 Hmm... I'm now reading comment on getRanges. I'm not sure `numPartitions` is actually `minPartitions` (so some typos on javadoc), but if they're same, below comment would say it's OK to have more or less partitions than `minPartitions`. > The number of Spark tasks will be *approximately* `numPartitions`. It can be less or more depending on rounding errors or Kafka partitions that didn't receive any new data. https://github.com/apache/spark/blob/d67b98ea016e9b714bef68feaac108edd08159c9/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala#L32-L46 Please ignore my review comments if the javadoc meant it. Looks great.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
