Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/22674#discussion_r224755348
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala ---
@@ -39,7 +39,22 @@ case class SparkListenerSQLExecutionStart
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/22674
I would just up the timeout in that suite. Now that we're pushing a bunch
more stuff to the LiveListenerBus, it may not be draining quickly enough. On
slow jenkins' it could likely cause flakiness
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/22674#discussion_r224013755
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -3356,21 +3356,11 @@ class Dataset[T] private[sql](
* user-registered
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/22674#discussion_r224006828
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala
---
@@ -75,95 +76,74 @@ trait QueryExecutionListener
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/22674#discussion_r224000145
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -3356,21 +3356,11 @@ class Dataset[T] private[sql](
* user-registered
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/22674#discussion_r224000809
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala ---
@@ -71,14 +72,35 @@ object SQLExecution {
val callSite
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/22478
thanks! merging to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/22478
[SPARK-25472] Don't have legitimate stops of streams cause stream exceptions
## What changes were proposed in this pull request?
Legitimate stops of streams may actually cause an exception
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/22478
cc @zsxwing @jose-torres
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/20673
After benchmarking, observed that this didn't provide much benefit :(
Closing the PR
---
-
To unsubscribe, e-mail: reviews
Github user brkyvz closed the pull request at:
https://github.com/apache/spark/pull/20673
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/22210
LGTM! Good catches
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/22106#discussion_r211031056
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaTestUtils.scala
---
@@ -120,61 +120,56 @@ private[kafka010] class
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/22106#discussion_r211030720
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala
---
@@ -130,6 +130,13 @@ class KafkaTestUtils
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/21559
Thanks! Merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195798990
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -221,26 +222,72 @@ class MemoryStreamInputPartition(records
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195797571
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -221,26 +222,72 @@ class MemoryStreamInputPartition(records
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195268299
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -228,19 +229,45 @@ trait MemorySinkBase extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195268861
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -228,19 +229,45 @@ trait MemorySinkBase extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195269434
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/memoryV2.scala
---
@@ -110,40 +126,61 @@ class MemorySinkV2 extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195268999
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/memoryV2.scala
---
@@ -81,22 +84,35 @@ class MemorySinkV2 extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195268218
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -228,19 +229,45 @@ trait MemorySinkBase extends
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/21559
Jenkins add to whitelist
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/21559
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/21275
LGTM! Pending tests
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185886437
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -128,40 +130,49 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185887294
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -266,93 +276,62 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185893229
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/WatermarkTracker.scala
---
@@ -0,0 +1,75 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185892675
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -384,22 +363,21 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185896682
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSinkSuite.scala
---
@@ -279,13 +279,10 @@ class FileStreamSinkSuite extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185678739
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -266,93 +276,62 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185890809
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -266,93 +276,62 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185893837
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/WatermarkTracker.scala
---
@@ -0,0 +1,75 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185887201
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -266,93 +276,62 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185891436
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -373,7 +352,7 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185885198
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -128,40 +130,49 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185677898
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -128,40 +130,49 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21220#discussion_r185883611
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -128,40 +130,49 @@ class MicroBatchExecution
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21126#discussion_r183912999
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -207,62 +209,126 @@ trait ProgressReporter extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21134#discussion_r183860834
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala
---
@@ -53,7 +53,7 @@ private[kafka010] class
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21126#discussion_r183542367
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala
---
@@ -492,6 +492,77 @@ class StreamingQuerySuite extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21126#discussion_r183533297
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -207,62 +209,92 @@ trait ProgressReporter extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21126#discussion_r183542307
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala
---
@@ -492,6 +492,77 @@ class StreamingQuerySuite extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21126#discussion_r183542025
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala
---
@@ -733,6 +804,11 @@ class StreamingQuerySuite extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21126#discussion_r183541978
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala
---
@@ -492,6 +492,77 @@ class StreamingQuerySuite extends
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/21124
LGTM!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21048#discussion_r180936744
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala
---
@@ -0,0 +1,344 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21048#discussion_r180938118
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala
---
@@ -0,0 +1,344 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21048#discussion_r180935752
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala
---
@@ -0,0 +1,344 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21048#discussion_r180938649
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManagerSuite.scala
---
@@ -0,0 +1,188 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21048#discussion_r180938989
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreSuite.scala
---
@@ -471,6 +470,41 @@ class StateStoreSuite
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21048#discussion_r180936292
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala
---
@@ -0,0 +1,344 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21048#discussion_r180937241
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala
---
@@ -0,0 +1,344 @@
+/*
+ * Licensed
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/20937
yes, please do
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20941#discussion_r178218438
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala ---
@@ -450,8 +450,8 @@ trait StreamTest extends QueryTest
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20941#discussion_r178208005
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala ---
@@ -444,6 +445,26 @@ trait StreamTest extends QueryTest
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20767#discussion_r173543942
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -342,80 +415,99 @@ private[kafka010] object
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/20698
LGTM pending tests.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171988510
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculatorSuite.scala
---
@@ -0,0 +1,136
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r17198
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala
---
@@ -0,0 +1,106
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171987888
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala
---
@@ -0,0 +1,106
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171988062
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala
---
@@ -0,0 +1,106
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171988112
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala
---
@@ -0,0 +1,106
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171983185
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala
---
@@ -0,0 +1,102
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171732779
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -370,8 +361,14 @@ private[kafka010] class
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171733516
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala
---
@@ -0,0 +1,105
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171733038
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala
---
@@ -0,0 +1,105
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171732729
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -320,28 +300,39 @@ private[kafka010
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171733181
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala
---
@@ -0,0 +1,105
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20698#discussion_r171732183
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -199,10 +179,10 @@ private[kafka010
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/20673
@HyukjinKwon Added tests
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/20673#discussion_r170646787
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -100,7 +102,18 @@ private[spark] object JsonProtocol
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/20673
[SPARK-23515] Use input/output streams for large events in
JsonProtocol.sparkEventToJson
## What changes were proposed in this pull request?
`def sparkEventToJson(event: SparkListenerEvent
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/20614
LGTM, my initial assumption that files had to be UTF-8 encoded was a wrong
one :(
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/20279
Closed in favor of #20445
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user brkyvz closed the pull request at:
https://github.com/apache/spark/pull/20279
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/20302
cc @hvanhovell @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/20302
[SPARK-23094] Fix invalid character handling in JsonDataSource
## What changes were proposed in this pull request?
There were two related fixes regarding `from_json`, `get_json_object
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/20279
[SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 APIs
## What changes were proposed in this pull request?
This PR migrates the MemoryStream to DataSourceV2 APIs.
## How
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18029
Merged to master. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r158627994
--- Diff:
external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java
---
@@ -45,18 +44,90 @@ public
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r158627640
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -56,12 +57,13 @@ import
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r158627744
--- Diff:
external/kinesis-asl/src/main/java/org/apache/spark/streaming/kinesis/KinesisInitialPositions.java
---
@@ -0,0 +1,91 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r158627941
--- Diff:
external/kinesis-asl/src/test/java/org/apache/spark/streaming/kinesis/JavaKinesisInputDStreamBuilderSuite.java
---
@@ -45,18 +44,90 @@ public
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/19975
This LGTM. @zsxwing Any other comments?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18029
Making them as singletons is unnecessary. How about this:
```java
public interface InitialPosition {
public InitialPositionInStream toKinesis();
}
public class
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18029
Actually yeah, I like your way.
On Dec 14, 2017 3:08 PM, "yashs360" <notificati...@github.com> wrote:
> *@yashs360* commented o
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19926#discussion_r157044691
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -0,0 +1,407 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r156266325
--- Diff:
external/kinesis-asl/src/main/java/org/apache/spark/streaming/kinesis/KinesisInitialPosition.java
---
@@ -0,0 +1,53 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r156265997
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisInputDStreamBuilderSuite.scala
---
@@ -101,12 +102,60 @@ class
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r156266783
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19925#discussion_r155834182
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/MicroBatchWriteSupport.java
---
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/19838
LGTM but I have very limited context on this codepath. Maybe @tdas can also
take a very quick look?
---
-
To unsubscribe, e-mail
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19838#discussion_r153910748
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingQueryListenerBus.scala
---
@@ -40,7 +40,7 @@ class
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/19495
LGTM!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r145283034
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala
---
@@ -1086,4 +1181,24 @@ object
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r145282388
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FlatMapGroupsWithStateExec.scala
---
@@ -61,6 +61,10 @@ case class
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r145282515
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/GroupStateImpl.scala
---
@@ -119,32 +116,34 @@ private[sql] class GroupStateImpl
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r145282877
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/GroupState.scala ---
@@ -205,92 +205,122 @@ trait GroupState[S] extends LogicalGroupState[S
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/19495
LGTM. Just a bunch of cosmetic nits, but fine to address them separately
---
-
To unsubscribe, e-mail: reviews-unsubscr
1 - 100 of 1220 matches
Mail list logo
