[GitHub] spark issue #21538: [SPARK-23754][PYTHON][FOLLOWUP][BACKPORT-2.3] Move UDF s...

2018-06-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21538 @e-dorigatti, this got merged into branch-2.3. Likewise, this also should be manually closed. Thanks for working on this. ---

[GitHub] spark issue #21544: add one supported type missing from the javadoc

2018-06-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21544 It's okay but mind double checking this file to see if there's any typo or misleading information while we are here? --- -

[GitHub] spark pull request #21082: [SPARK-22239][SQL][Python] Enable grouped aggrega...

2018-06-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21082 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21504: [SPARK-24479][SS] Added config for registering streaming...

2018-06-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21504 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21082: [SPARK-22239][SQL][Python] Enable grouped aggregate pand...

2018-06-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21082 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20641: [SPARK-23464][MESOS] Fix mesos cluster scheduler options...

2018-06-12 Thread krcz
Github user krcz commented on the issue: https://github.com/apache/spark/pull/20641 @skonto The one directly below "At the driver side in DC/OS:" in your comment. But if it's output of ps that's ok, as `ps` wouldn't be able to discern between `command "foo bar"` and `command foo

[GitHub] spark issue #21538: [SPARK-23754][PYTHON][FOLLOWUP][BACKPORT-2.3] Move UDF s...

2018-06-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21538 Merged to branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91732/ Test PASSed. ---

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21050 **[Test build #91732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91732/testReport)** for PR 21050 at commit

[GitHub] spark issue #21515: [SPARK-24372][build] Add scripts to help with preparing ...

2018-06-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21515 We could update the docs once these get in. I did not use docker for 2.3.1 but I plan to give it a shot for 2.1.3. The goal of the docker path is to always build Spark in a known

[GitHub] spark issue #21515: [SPARK-24372][build] Add scripts to help with preparing ...

2018-06-12 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21515 I guess we will need to update the release process website once these go in? Its not clear to me how the docker stuff applies here, are we recommending to create the release inside docker

[GitHub] spark pull request #21533: [SPARK-24195][Core] Bug fix for local:/ path in S...

2018-06-12 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21533#discussion_r194927085 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1517,9 +1517,12 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21545: [SPARK-23010][BUILD][FOLLOWUP] Fix java checkstyle failu...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21545 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91730/ Test PASSed. ---

[GitHub] spark issue #21545: [SPARK-23010][BUILD][FOLLOWUP] Fix java checkstyle failu...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21545 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21545: [SPARK-23010][BUILD][FOLLOWUP] Fix java checkstyle failu...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21545 **[Test build #91730 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91730/testReport)** for PR 21545 at commit

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91731/ Test PASSed. ---

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 **[Test build #91731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91731/testReport)** for PR 21366 at commit

[GitHub] spark issue #21498: [SPARK-24410][SQL][Core] Optimization for Union outputPa...

2018-06-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21498 > In theory, this should be done in a cost-based style. Changing the way how union combines data will reduce the parallelism. > For example, if we union 2 tables each has 5 partitions. Without

[GitHub] spark issue #21523: [SPARK-24506][UI] Add UI filters to tabs added after bin...

2018-06-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21523 2.1 had non-trivial conflicts. Probably not worth it in any case. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web U...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21510 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web U...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21510 **[Test build #91736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91736/testReport)** for PR 21510 at commit

[GitHub] spark issue #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web U...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21510 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/70/

[GitHub] spark issue #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web U...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21510 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21523: [SPARK-24506][UI] Add UI filters to tabs added af...

2018-06-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21523 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web U...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21510 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3959/

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91727/ Test PASSed. ---

[GitHub] spark issue #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web U...

2018-06-12 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21510 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 **[Test build #91727 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91727/testReport)** for PR 21366 at commit

[GitHub] spark issue #21523: [SPARK-24506][UI] Add UI filters to tabs added after bin...

2018-06-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21523 Merging to master / 2.3 (will try back to 2.1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91728/ Test PASSed. ---

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 **[Test build #91728 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91728/testReport)** for PR 21366 at commit

[GitHub] spark issue #20929: [SPARK-23772][SQL] Provide an option to ignore column of...

2018-06-12 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20929 sorry, I'll update tongiht --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...

2018-06-12 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21450#discussion_r194915218 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java --- @@ -138,25 +139,30 @@ case RUN_EXAMPLE:

[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...

2018-06-12 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21450#discussion_r194915296 --- Diff: launcher/src/main/java/org/apache/spark/launcher/Main.java --- @@ -101,6 +99,22 @@ public static void main(String[] argsArray) throws Exception {

[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...

2018-06-12 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21450#discussion_r194916287 --- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java --- @@ -190,6 +197,36 @@ public void testSparkRShell()

[GitHub] spark pull request #21428: [SPARK-24235][SS] Implement continuous shuffle wr...

2018-06-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21428#discussion_r194914332 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/shuffle/ContinuousShuffleSuite.scala --- @@ -288,4 +264,153 @@ class

[GitHub] spark pull request #21428: [SPARK-24235][SS] Implement continuous shuffle wr...

2018-06-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21428#discussion_r194911574 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleWriter.scala --- @@ -0,0 +1,54 @@

[GitHub] spark pull request #21428: [SPARK-24235][SS] Implement continuous shuffle wr...

2018-06-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21428#discussion_r194914390 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/shuffle/ContinuousShuffleSuite.scala --- @@ -288,4 +264,153 @@ class

[GitHub] spark pull request #21428: [SPARK-24235][SS] Implement continuous shuffle wr...

2018-06-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21428#discussion_r194907764 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleWriter.scala --- @@ -0,0 +1,54 @@

[GitHub] spark pull request #21428: [SPARK-24235][SS] Implement continuous shuffle wr...

2018-06-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21428#discussion_r194916037 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleReader.scala --- @@ -79,10 +77,10 @@

[GitHub] spark pull request #21428: [SPARK-24235][SS] Implement continuous shuffle wr...

2018-06-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21428#discussion_r194905420 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleReader.scala --- @@ -68,7 +66,7 @@

[GitHub] spark pull request #21428: [SPARK-24235][SS] Implement continuous shuffle wr...

2018-06-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21428#discussion_r194912456 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/ContinuousShuffleReadRDD.scala --- @@ -34,8 +34,10 @@ case

[GitHub] spark pull request #21545: [SPARK-23010][BUILD][FOLLOWUP] Fix java checkstyl...

2018-06-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21545 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2018-06-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17702 This approach only works if the first level glob pattern matches a lot of directories, e.g. `/my_path/*/*`. Otherwise, we can't apply it, e.g. `/my_path/{ab, cd}/*`. My proposal: think

[GitHub] spark issue #21545: [SPARK-23010][BUILD][FOLLOWUP] Fix java checkstyle failu...

2018-06-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21545 The tests are way past the point where the java style checks would have run. Merging to master (since PRs with Java changes are failing). ---

[GitHub] spark pull request #17702: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-06-12 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17702#discussion_r194911415 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -252,6 +252,18 @@ class SparkHadoopUtil extends Logging { if

[GitHub] spark issue #21523: [SPARK-24506][UI] Add UI filters to tabs added after bin...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21523 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21523: [SPARK-24506][UI] Add UI filters to tabs added after bin...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21523 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91723/ Test PASSed. ---

[GitHub] spark issue #21523: [SPARK-24506][UI] Add UI filters to tabs added after bin...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21523 **[Test build #91723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91723/testReport)** for PR 21523 at commit

[GitHub] spark issue #21508: [SPARK-24488] [SQL] Fix issue when generator is aliased ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21508 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91725/ Test PASSed. ---

[GitHub] spark issue #21508: [SPARK-24488] [SQL] Fix issue when generator is aliased ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21508 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21508: [SPARK-24488] [SQL] Fix issue when generator is aliased ...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21508 **[Test build #91725 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91725/testReport)** for PR 21508 at commit

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91726/ Test PASSed. ---

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21050 **[Test build #91726 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91726/testReport)** for PR 21050 at commit

[GitHub] spark pull request #21537: [SPARK-24505][SQL] Convert strings in codegen to ...

2018-06-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21537#discussion_r194908404 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -579,6 +579,22 @@ class CodegenContext {

[GitHub] spark issue #21501: [SPARK-15064][ML] Locale support in StopWordsRemover

2018-06-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21501 @dongjinleekr You are welcome and thanks for your contribution! --- - To unsubscribe, e-mail:

[GitHub] spark issue #21539: [SPARK-24500][SQL] Make sure streams are materialized du...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21539 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3958/

[GitHub] spark issue #21539: [SPARK-24500][SQL] Make sure streams are materialized du...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21539 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21527: [SPARK-24519] MapStatus has 2000 hardcoded

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21527 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91724/ Test FAILed. ---

[GitHub] spark issue #21527: [SPARK-24519] MapStatus has 2000 hardcoded

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21527 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21527: [SPARK-24519] MapStatus has 2000 hardcoded

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21527 **[Test build #91724 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91724/testReport)** for PR 21527 at commit

[GitHub] spark issue #21542: [WIP][SPARK-24529][Build] Add spotbugs into maven build ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21542 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21542: [WIP][SPARK-24529][Build] Add spotbugs into maven build ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21542 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91720/ Test PASSed. ---

[GitHub] spark issue #21539: [SPARK-24500][SQL] Make sure streams are materialized du...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21539 **[Test build #91735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91735/testReport)** for PR 21539 at commit

[GitHub] spark issue #21539: [SPARK-24500][SQL] Make sure streams are materialized du...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21539 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21542: [WIP][SPARK-24529][Build] Add spotbugs into maven build ...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21542 **[Test build #91720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91720/testReport)** for PR 21542 at commit

[GitHub] spark issue #21539: [SPARK-24500][SQL] Make sure streams are materialized du...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21539 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/69/

[GitHub] spark issue #21539: [SPARK-24500][SQL] Make sure streams are materialized du...

2018-06-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21539 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21543: [SPARK-24531][TESTS] Replace 2.3.0 version with 2.3.1

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21543 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21543: [SPARK-24531][TESTS] Replace 2.3.0 version with 2.3.1

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21543 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91729/ Test PASSed. ---

[GitHub] spark issue #21543: [SPARK-24531][TESTS] Replace 2.3.0 version with 2.3.1

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21543 **[Test build #91729 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91729/testReport)** for PR 21543 at commit

[GitHub] spark issue #21082: [SPARK-22239][SQL][Python] Enable grouped aggregate pand...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21082 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21082: [SPARK-22239][SQL][Python] Enable grouped aggregate pand...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21082 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91718/ Test PASSed. ---

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-12 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r194904013 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala --- @@ -183,34 +182,131 @@ private[sql] object

[GitHub] spark issue #21082: [SPARK-22239][SQL][Python] Enable grouped aggregate pand...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21082 **[Test build #91718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91718/testReport)** for PR 21082 at commit

[GitHub] spark pull request #21539: [SPARK-24500][SQL] Make sure streams are material...

2018-06-12 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21539#discussion_r194902636 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala --- @@ -679,6 +679,13 @@ class PlannerSuite extends SharedSQLContext

[GitHub] spark pull request #21539: [SPARK-24500][SQL] Make sure streams are material...

2018-06-12 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21539#discussion_r194902354 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala --- @@ -301,6 +290,37 @@ abstract class TreeNode[BaseType <:

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21319 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91722/ Test PASSed. ---

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21319 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21319 **[Test build #91722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91722/testReport)** for PR 21319 at commit

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-12 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r194898976 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowConvertersSuite.scala --- @@ -51,11 +51,11 @@ class ArrowConvertersSuite

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-12 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r194898793 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala --- @@ -183,34 +182,131 @@ private[sql] object

[GitHub] spark issue #21515: [SPARK-24372][build] Add scripts to help with preparing ...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21515 **[Test build #91734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91734/testReport)** for PR 21515 at commit

[GitHub] spark issue #21515: [SPARK-24372][build] Add scripts to help with preparing ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3957/

[GitHub] spark issue #21515: [SPARK-24372][build] Add scripts to help with preparing ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21515 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21515: [SPARK-24372][build] Add scripts to help with preparing ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21515 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21515: [SPARK-24372][build] Add scripts to help with preparing ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/68/

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/67/

[GitHub] spark pull request #21515: [SPARK-24372][build] Add scripts to help with pre...

2018-06-12 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21515#discussion_r194897476 --- Diff: dev/create-release/vote.tmpl --- @@ -0,0 +1,64 @@ +Please vote on releasing the following candidate as Apache Spark version {version}. +

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21546 **[Test build #91733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91733/testReport)** for PR 21546 at commit

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3956/

[GitHub] spark pull request #21515: [SPARK-24372][build] Add scripts to help with pre...

2018-06-12 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21515#discussion_r194896688 --- Diff: dev/create-release/spark-rm/Dockerfile --- @@ -0,0 +1,89 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-06-12 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21546 This is a WIP because I had to hack up some of the message processing code in Arrow. This should be done in Arrow, and then it can be cleaned up here. I will make these changes for version

<    1   2   3   4   5   6   7   8   >