Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/11250#issuecomment-187305582
@srowen @holdenk
Thank you very much for your help!!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/11250
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/11250#issuecomment-187097084
Merged to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11250#issuecomment-186817033
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11250#issuecomment-186817034
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11250#issuecomment-186817000
**[Test build #51627 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51627/consoleFull)**
for PR 11250 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11250#issuecomment-186803513
**[Test build #51627 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51627/consoleFull)**
for PR 11250 at commit
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/11250#issuecomment-186800650
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53565173
--- Diff:
extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisStreamSuite.scala
---
@@ -268,21 +270,23 @@ abstract class
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53565164
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -613,7 +613,8 @@ class CheckpointSuite extends TestSuiteBase with
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53563513
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala
---
@@ -163,8 +166,11 @@ class FileInputDStream[K, V, F <:
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53547665
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala
---
@@ -163,8 +166,11 @@ class FileInputDStream[K, V, F <:
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53547655
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala
---
@@ -148,7 +148,10 @@ class FileInputDStream[K, V, F <:
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53547651
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala
---
@@ -148,7 +148,10 @@ class FileInputDStream[K, V, F <:
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53547645
--- Diff:
extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisStreamSuite.scala
---
@@ -268,9 +270,9 @@ abstract class
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53473423
--- Diff:
extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisStreamSuite.scala
---
@@ -241,13 +243,13 @@ abstract class
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53430913
--- Diff:
extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisStreamSuite.scala
---
@@ -241,13 +243,13 @@ abstract class
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11250#discussion_r53296925
--- Diff:
extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisStreamSuite.scala
---
@@ -241,13 +243,13 @@ abstract class
Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/11250#issuecomment-185608162
@srowen @holdenk
Could you please take a look of this PR? I ran the python streaming test
cleanly on my local before I submitted the PR.
Thanks a
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11250#issuecomment-185607707
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user huaxingao opened a pull request:
https://github.com/apache/spark/pull/11250
[SPARK-13186][Streaming]migrate away from SynchronizedMap
trait SynchronizedMap in package mutable is deprecated: Synchronization via
traits is deprecated as it is inherently unreliable. Change
Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-185604993
@srowen @holdenk
I will close this PR and submit a new one. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user huaxingao closed the pull request at:
https://github.com/apache/spark/pull/11104
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-185171737
@huaxingao OK, do you want to close this and try another PR? I can try to
take it on too.
I think this is mostly correct except we'll need more synchronization in
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-183267563
@huaxingao can you rebase this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-183434809
@srowen
Will do. I have my local branch messed up. If i can't figure out how to
fix it, I will close this PR and submit a new one. Also, one of the python
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181767095
@huaxingao one more style issue and it also needs to be updated with the
latest master since there are now merge conflicts now.
---
If your project is set up for it,
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-182169291
**[Test build #51018 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51018/consoleFull)**
for PR 11104 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-182199323
Build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-182199328
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-182198583
**[Test build #51018 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51018/consoleFull)**
for PR 11104 at commit
Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181664339
Sorry for the file line length problem. Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181666127
**[Test build #50950 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50950/consoleFull)**
for PR 11104 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181666297
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181666298
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181666295
**[Test build #50950 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50950/consoleFull)**
for PR 11104 at commit
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181541571
So I think the `putIfAbsent` API might do what your looking for there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52228031
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -65,12 +67,14 @@ class KafkaStreamSuite extends
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52215569
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -65,12 +67,14 @@ class KafkaStreamSuite extends
Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181528520
@holdenk
Could you please review one more time?
I changed to java api except the getOrElseUpdate in KafkaStreamSuite.scala.
I can't find a java equivalent
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181537776
Sure I'll take another look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52236511
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -65,12 +67,14 @@ class KafkaStreamSuite extends
Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181633831
@zsxwing
Thanks for the comments. I didn't see a PR for removing SynchronizedSet. I
will work on this.
---
If your project is set up for it, you can reply to
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52249784
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingListenerSuite.scala
---
@@ -17,7 +17,10 @@
package
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52249791
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -19,6 +19,7 @@ package org.apache.spark.streaming
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52249862
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala
---
@@ -18,7 +18,9 @@
package
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52249982
--- Diff:
extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisStreamSuite.scala
---
@@ -17,6 +17,9 @@
package
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181638140
LGTM except some nits. Thanks, @huaxingao
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52250939
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -65,12 +67,14 @@ class KafkaStreamSuite extends
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52252046
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -65,12 +67,14 @@ class KafkaStreamSuite extends
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52247757
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -65,12 +67,14 @@ class KafkaStreamSuite extends
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52247909
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -65,12 +67,14 @@ class KafkaStreamSuite extends
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181632123
By the way, is there any PR removing `SynchronizedSet`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181648992
Fixed the problems. Thank you all very much for your help!!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181650122
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181655868
**[Test build #50949 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50949/consoleFull)**
for PR 11104 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181656095
**[Test build #50949 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50949/consoleFull)**
for PR 11104 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181656096
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-181656097
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52118020
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -30,6 +30,9 @@ import org.apache.spark.{SparkConf,
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52126187
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -30,6 +30,9 @@ import org.apache.spark.{SparkConf,
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52126175
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -30,6 +30,9 @@ import org.apache.spark.{SparkConf,
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52126981
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -30,6 +30,9 @@ import org.apache.spark.{SparkConf,
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52126950
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -30,6 +30,9 @@ import org.apache.spark.{SparkConf,
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52109483
--- Diff:
extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisStreamSuite.scala
---
@@ -229,8 +231,7 @@ abstract class
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-180889389
Thanks for the PR and getting started on this :)
So the first minor thing we can update easily is import ordering in many of
the files should follow the style
GitHub user huaxingao opened a pull request:
https://github.com/apache/spark/pull/11104
[SPARK-13186][Streaming]Migrate away from SynchronizedMap
trait SynchronizedMap in package mutable is deprecated: Synchronization via
traits is deprecated as it is inherently unreliable. Change
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-180882891
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/11104#discussion_r52109403
--- Diff:
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
---
@@ -30,6 +30,9 @@ import org.apache.spark.{SparkConf,
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-180889472
@huaxingao did a first quick pass :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user huaxingao commented on the pull request:
https://github.com/apache/spark/pull/11104#issuecomment-180883943
@holdenk
Could you please review? Thanks!!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
71 matches
Mail list logo
