Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22042
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r212507190
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -160,6 +160,23 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r212522664
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -161,6 +161,22 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r212521083
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -239,56 +335,74 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r212522432
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -161,6 +161,22 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r212504622
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReadSupport.scala
---
@@ -331,6 +331,7 @@ private[kafka010]
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r212033844
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -337,6 +338,7 @@ private[kafka010] case
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r212032759
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -597,6 +614,254 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211805733
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -597,6 +614,254 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211801632
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -346,11 +437,40 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211802112
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -250,33 +341,39 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211801676
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -250,33 +341,39 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211801968
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -250,33 +341,39 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211795985
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -80,6 +90,72 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211805275
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -597,6 +614,254 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211801549
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -250,33 +341,39 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211804454
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -597,6 +614,254 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211802489
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -288,7 +385,7 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211801254
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -250,33 +341,39 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211804879
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -597,6 +614,254 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211803267
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -337,6 +338,7 @@ private[kafka010] case
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211805821
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -597,6 +614,254 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211805409
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -597,6 +614,254 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211804704
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -160,6 +160,23 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211803763
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
---
@@ -160,6 +160,23 @@ abstract class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211805993
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala
---
@@ -327,6 +332,14 @@ class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211786471
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -250,33 +294,42 @@ private[kafka010] case
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211786183
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -31,22 +31,21 @@ import
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r211786163
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -31,22 +31,21 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r210422755
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -31,22 +31,21 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r210423180
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -91,6 +90,17 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r210985375
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -346,11 +385,40 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r210422521
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -31,22 +31,21 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209479417
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -251,32 +274,53 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209479551
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceRDD.scala
---
@@ -77,44 +77,6 @@ private[kafka010] class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209475048
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -251,32 +274,53 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209476712
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -347,9 +391,12 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209473392
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -31,6 +31,17 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209477156
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -251,32 +274,53 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209476548
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -251,32 +274,53 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209478033
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -251,32 +274,53 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209473432
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -31,6 +31,17 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209474755
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -251,32 +274,53 @@ private[kafka010] case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r209473316
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -95,6 +106,10 @@ private[kafka010] case class
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/22042
[SPARK-25005][SS]Support non-consecutive offsets for Kafka
## What changes were proposed in this pull request?
As the user uses Kafka transactions to write data, the offsets in Kafka
will
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22042#discussion_r208676022
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceRDD.scala
---
@@ -77,44 +77,6 @@ private[kafka010] class
46 matches
Mail list logo
