[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-11-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19611 @briggsw This is pretty hard to hit. I could not make it happen in a real query. So I think it's not worth to backport it. ---

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-11-06 Thread briggsw
Github user briggsw commented on the issue: https://github.com/apache/spark/pull/19611 Is there any chance of this change getting merged into 2.2.1? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19611 Thanks! Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83229/ Test PASSed. ---

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83229/testReport)** for PR 19611 at commit

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19611 LGTM pending tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83229/testReport)** for PR 19611 at commit

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83220/ Test PASSed. ---

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83220 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83220/testReport)** for PR 19611 at commit

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83217/ Test PASSed. ---

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83217/testReport)** for PR 19611 at commit

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83225/testReport)** for PR 19611 at commit

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19611 I think we can remove the unit test. It's obviously that `loadMap` is not recursive and will not cause StackOverflowError. ---

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread joseph-torres
Github user joseph-torres commented on the issue: https://github.com/apache/spark/pull/19611 One issue I want to explicitly bring up: this new unit test takes very long, almost 2 minutes on my computer. Creating 10k files isn't going to be super fast no matter what we do, but is

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83220/testReport)** for PR 19611 at commit

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83217/testReport)** for PR 19611 at commit