Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20097
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r160251566
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -392,6 +443,21 @@ class MicroBatchExecution(
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159985442
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala
---
@@ -240,31 +240,35 @@ class StreamingQueryManager
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159983634
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -392,6 +443,21 @@ class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159978428
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -392,6 +443,21 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159980348
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala
---
@@ -166,19 +167,31 @@ final class DataStreamReader
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159981422
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala
---
@@ -240,31 +240,35 @@ class StreamingQueryManager
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159980158
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -33,15 +40,21 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159978319
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -392,6 +443,21 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159961186
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -357,31 +400,39 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159960120
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -236,14 +264,27 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159980863
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
---
@@ -35,6 +35,16 @@ case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159981274
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala
---
@@ -28,17 +28,38 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159958467
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -33,15 +40,21 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159558199
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/RateSourceProvider.scala
---
@@ -236,6 +236,21 @@ class RateStreamSource(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159556678
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala
---
@@ -32,13 +32,17 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159555278
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -236,14 +257,31 @@ class MicroBatchExecution(
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159555174
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala
---
@@ -240,31 +240,35 @@ class StreamingQueryManager
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159554508
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -317,6 +355,8 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159554071
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala
---
@@ -240,31 +240,35 @@ class StreamingQueryManager
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159553087
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -317,6 +355,8 @@ class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159552400
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -409,7 +475,12 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159552212
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -357,7 +397,16 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159551588
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -317,6 +355,8 @@ class MicroBatchExecution(
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159497524
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala
---
@@ -167,6 +167,24 @@ final class DataStreamReader
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159493843
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala
---
@@ -167,6 +167,24 @@ final class DataStreamReader
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159492891
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala
---
@@ -111,7 +112,7 @@ class
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159481242
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -357,7 +397,16 @@ class
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159480662
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -317,6 +355,8 @@ class
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159480191
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -236,14 +257,31 @@ class
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r15947
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -236,14 +257,31 @@ class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159365598
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala
---
@@ -32,16 +32,17 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159365501
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -226,7 +226,7 @@ trait ProgressReporter extends
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159365350
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -357,7 +397,16 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159102835
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
---
@@ -35,6 +35,12 @@ case class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159114315
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -236,14 +257,31 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159364715
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -367,21 +416,20 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159112097
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala
---
@@ -167,6 +167,24 @@ final class DataStreamReader
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159364509
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -317,6 +355,8 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159111979
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala
---
@@ -167,6 +167,24 @@ final class DataStreamReader
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159112055
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala
---
@@ -167,6 +167,24 @@ final class DataStreamReader
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159114536
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -367,21 +416,20 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159110696
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -68,8 +76,20 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159356689
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala
---
@@ -32,16 +32,17 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159356698
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala
---
@@ -32,16 +32,17 @@ import
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159114244
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -192,7 +212,8 @@ class MicroBatchExecution(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159356713
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala
---
@@ -111,7 +112,7 @@ class
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159364969
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -236,14 +257,31 @@ class MicroBatchExecution(
GitHub user jose-torres opened a pull request:
https://github.com/apache/spark/pull/20097
[SPARK-22912] v2 data source support in MicroBatchExecution
## What changes were proposed in this pull request?
Support for v2 data sources in microbatch streaming.
## How was
49 matches
Mail list logo
