Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211803267
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -337,6 +338,7 @@ private[kafka010] case class
KafkaMicroBatchInputPartitionReader(
val record = consumer.get(nextOffset, rangeToRead.untilOffset,
pollTimeoutMs, failOnDataLoss)
if (record != null) {
nextRow = converter.toUnsafeRow(record)
+ nextOffset = record.offset + 1
--- End diff --
why this change?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
