Github user QuentinAmbard commented on a diff in the pull request:
https://github.com/apache/spark/pull/21917#discussion_r207802444
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala
---
@@ -223,17 +240,46 @@
Github user koeninger commented on a diff in the pull request:
https://github.com/apache/spark/pull/21917#discussion_r207721681
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/OffsetWithRecordScannerSuite.scala
---
@@ -0,0 +1,110 @@
+/*
+
Github user koeninger commented on a diff in the pull request:
https://github.com/apache/spark/pull/21917#discussion_r207721657
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/OffsetRange.scala
---
@@ -90,21 +90,23 @@ final class OffsetRange
Github user koeninger commented on a diff in the pull request:
https://github.com/apache/spark/pull/21917#discussion_r207721492
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaDataConsumer.scala
---
@@ -191,6 +211,11 @@ private[kafka010]
Github user koeninger commented on a diff in the pull request:
https://github.com/apache/spark/pull/21917#discussion_r207721482
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaDataConsumer.scala
---
@@ -183,6 +187,22 @@ private[kafka010]
Github user koeninger commented on a diff in the pull request:
https://github.com/apache/spark/pull/21917#discussion_r207721435
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala
---
@@ -223,17 +240,46 @@ private[spark]
Github user koeninger commented on a diff in the pull request:
https://github.com/apache/spark/pull/21917#discussion_r207437645
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala
---
@@ -223,17 +240,46 @@ private[spark]
GitHub user QuentinAmbard opened a pull request:
https://github.com/apache/spark/pull/21917
[SPARK-24720][STREAMING-KAFKA] add option to align ranges with offset
having records to support kafka transaction
## What changes were proposed in this pull request?
This fix adds
