[GitHub] spark pull request #21286: [SPARK-24194] HadoopFsRelation cannot overwrite a...

2018-05-09 Thread zheh12
GitHub user zheh12 opened a pull request: https://github.com/apache/spark/pull/21286 [SPARK-24194] HadoopFsRelation cannot overwrite a path that is also b… ## What changes were proposed in this pull request? When there are multiple tasks at the same time append a `HadoopFs

[GitHub] spark issue #21279: [SPARK-24219][k8s] Improve the docker building script to...

2018-05-09 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21279 jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21155 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90428/ Test PASSed. ---

[GitHub] spark issue #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21155 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21155 **[Test build #90428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90428/testReport)** for PR 21155 at commit [`22bde31`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21282 **[Test build #90434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90434/testReport)** for PR 21282 at commit [`8c6039c`](https://github.com/apache/spark/commit/8c

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-05-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21282 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-05-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21282 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21238 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3002/ ---

[GitHub] spark issue #21209: [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBac...

2018-05-09 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/21209 Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.ap

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21238 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3088/ Tes

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21238 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3002/ --- --

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21238 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21183: [SPARK-22210][ML] Add seed for LDA variationalTop...

2018-05-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21183#discussion_r187217859 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -622,11 +623,11 @@ object LocalLDAModel extends MLReadable[LocalLDAModel] {

[GitHub] spark pull request #21183: [SPARK-22210][ML] Add seed for LDA variationalTop...

2018-05-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21183#discussion_r187216371 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/LDASuite.scala --- @@ -252,6 +252,15 @@ class LDASuite extends SparkFunSuite with MLlibTest

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21231 **[Test build #90433 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90433/testReport)** for PR 21231 at commit [`590ba26`](https://github.com/apache/spark/commit/59

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21238 **[Test build #90432 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90432/testReport)** for PR 21238 at commit [`fa095cd`](https://github.com/apache/spark/commit/fa

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21238 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3087/ Tes

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21238 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21265: [SPARK-24146][PySpark][ML] spark.ml parity for sequentia...

2018-05-09 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/21265 Let's wait on this until we make the decision in the last thread in https://github.com/apache/spark/pull/20973 --- - To unsubs

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21238 **[Test build #90431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90431/testReport)** for PR 21238 at commit [`aebdb68`](https://github.com/apache/spark/commit/ae

[GitHub] spark pull request #20973: [SPARK-20114][ML] spark.ml parity for sequential ...

2018-05-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20973#discussion_r187216080 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/PrefixSpan.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] spark pull request #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work ...

2018-05-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21267#discussion_r187216038 --- Diff: python/pyspark/context.py --- @@ -211,9 +211,23 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize,

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-09 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21238 @rxin @liyinan926 @foxish addressed comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional c

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r187213977 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1468,3 +1472,388 @@ case class Flatten(ch

[GitHub] spark issue #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to InputPar...

2018-05-09 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21145 Thanks @jose-torres! I appreciate not blocking this commit on those changes, since it would be difficult to keep this up to date from the other paths changing, while we discussed what to call these c

[GitHub] spark issue #21183: [SPARK-22210][ML] Add seed for LDA variationalTopicInfer...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21183 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21183: [SPARK-22210][ML] Add seed for LDA variationalTopicInfer...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21183 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90429/ Test PASSed. ---

[GitHub] spark issue #21183: [SPARK-22210][ML] Add seed for LDA variationalTopicInfer...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21183 **[Test build #90429 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90429/testReport)** for PR 21183 at commit [`6c2a8aa`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to InputPar...

2018-05-09 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/21145 LGTM. I can own cleaning up the names of the streaming classes, probably wrapping that into the broader task of getting a design doc for the streaming reader API. ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90424/ Test PASSed. ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 **[Test build #90424 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90424/testReport)** for PR 21092 at commit [`306f3ed`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21274: [SPARK-24213][ML] Fix for Int id type for PowerIteration...

2018-05-09 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/21274 LGTM. ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spa

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3086/ Tes

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90430/testReport)** for PR 21193 at commit [`72faac3`](https://github.com/apache/spark/commit/72

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21193 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21221 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90422/ Test PASSed. ---

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21221 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21221 **[Test build #90422 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90422/testReport)** for PR 21221 at commit [`ad10d28`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21183: [SPARK-22210][ML] Add seed for LDA variationalTopicInfer...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21183 **[Test build #90429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90429/testReport)** for PR 21183 at commit [`6c2a8aa`](https://github.com/apache/spark/commit/6c

[GitHub] spark issue #21284: [SPARK-23852][SQL] Add test that fails if PARQUET-1217 i...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21284 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90423/ Test PASSed. ---

[GitHub] spark issue #21284: [SPARK-23852][SQL] Add test that fails if PARQUET-1217 i...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21284 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21284: [SPARK-23852][SQL] Add test that fails if PARQUET-1217 i...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21284 **[Test build #90423 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90423/testReport)** for PR 21284 at commit [`4b42ee2`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #21274: [SPARK-24213][ML] Fix for Int id type for PowerIteration...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21274 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21274: [SPARK-24213][ML] Fix for Int id type for PowerIteration...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21274 **[Test build #90427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90427/testReport)** for PR 21274 at commit [`3e40a92`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21274: [SPARK-24213][ML] Fix for Int id type for PowerIteration...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21274 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90427/ Test PASSed. ---

[GitHub] spark pull request #21209: [SPARK-24141][CORE] Fix bug in CoarseGrainedSched...

2018-05-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21209 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21209: [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBac...

2018-05-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21209 LGTM. I have a feeling the current code would heal itself later (by the fact that `ExecutorAllocationManager` calls `requestTotalExecutors`) but this looks more correct. Merging to master.

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-05-09 Thread wajda
Github user wajda commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r187195445 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -536,6 +536,15 @@ object TypeCoercion {

[GitHub] spark issue #21049: [SPARK-23957][SQL] Remove redundant sort operators from ...

2018-05-09 Thread henryr
Github user henryr commented on the issue: https://github.com/apache/spark/pull/21049 @dilipbiswal I tried hard to find something in the SQL standard that clarified the situation, but couldn't (of course that could be because the standard is pretty hard to parse... :)). So l

[GitHub] spark pull request #21280: [SPARK-19181][Core] Fixing flaky "SparkListenerSu...

2018-05-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21280#discussion_r187194900 --- Diff: core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala --- @@ -583,3 +587,23 @@ private class FirehoseListenerThatAcceptsSparkCon

[GitHub] spark issue #21255: [SPARK-24186][R][SQL]change reverse and concat to collec...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21255 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90426/ Test PASSed. ---

[GitHub] spark issue #21255: [SPARK-24186][R][SQL]change reverse and concat to collec...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21255 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21255: [SPARK-24186][R][SQL]change reverse and concat to collec...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21255 **[Test build #90426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90426/testReport)** for PR 21255 at commit [`b50d2c8`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21155 **[Test build #90428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90428/testReport)** for PR 21155 at commit [`22bde31`](https://github.com/apache/spark/commit/22

[GitHub] spark pull request #21280: [SPARK-19181][Core] Fixing flaky "SparkListenerSu...

2018-05-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21280#discussion_r187192553 --- Diff: core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala --- @@ -583,3 +587,23 @@ private class FirehoseListenerThatAcceptsSparkCon

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-09 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r187192485 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,40 @@ case class SortPrefix(child:

[GitHub] spark pull request #21280: [SPARK-19181][Core] Fixing flaky "SparkListenerSu...

2018-05-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21280#discussion_r187192271 --- Diff: core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala --- @@ -583,3 +587,23 @@ private class FirehoseListenerThatAcceptsSparkCon

[GitHub] spark issue #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to InputPar...

2018-05-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21145 LGTM to `InputPartition`. CC @marmbrus @jose-torres to check the streaming side. --- - To unsubscribe, e-mail: reviews-unsubs

[GitHub] spark issue #21203: [SPARK-24131][PySpark] Add majorMinorVersion API to PySp...

2018-05-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21203 @jkbradley A follow-up has been submitted and merged. Please see #21211. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr..

[GitHub] spark issue #21274: [SPARK-24213][ML] Fix for Int id type for PowerIteration...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21274 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21274: [SPARK-24213][ML] Fix for Int id type for PowerIteration...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21274 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3085/ Tes

[GitHub] spark issue #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression uses g...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21276 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90418/ Test PASSed. ---

[GitHub] spark issue #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression uses g...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21276 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression uses g...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21276 **[Test build #90418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90418/testReport)** for PR 21276 at commit [`a43493b`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #21265: [SPARK-24146][PySpark][ML] spark.ml parity for se...

2018-05-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21265#discussion_r187187330 --- Diff: python/pyspark/ml/fpm.py --- @@ -243,3 +244,75 @@ def setParams(self, minSupport=0.3, minConfidence=0.8, itemsCol="items", def _c

[GitHub] spark pull request #21218: [SPARK-24155][ML] Instrumentation improvements fo...

2018-05-09 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21218#discussion_r187187071 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/BisectingKMeans.scala --- @@ -278,6 +279,7 @@ class BisectingKMeans @Since("2.0.0") (

[GitHub] spark issue #21274: [SPARK-24213][ML] Fix for Int id type for PowerIteration...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21274 **[Test build #90427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90427/testReport)** for PR 21274 at commit [`3e40a92`](https://github.com/apache/spark/commit/3e

[GitHub] spark issue #21255: [SPARK-24186][R][SQL]change reverse and concat to collec...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21255 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3084/ Tes

[GitHub] spark issue #21255: [SPARK-24186][R][SQL]change reverse and concat to collec...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21255 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21255: [SPARK-24186][R][SQL]change reverse and concat to collec...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21255 **[Test build #90426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90426/testReport)** for PR 21255 at commit [`b50d2c8`](https://github.com/apache/spark/commit/b5

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90425/ Test FAILed. ---

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21278 **[Test build #90425 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90425/testReport)** for PR 21278 at commit [`a0e9b02`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21285: [SPARK-24176][SQL] LOAD DATA can't identify wildcard in ...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21285 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90416/ Test PASSed. ---

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21285: [SPARK-24176][SQL] LOAD DATA can't identify wildcard in ...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21285 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21285: [SPARK-24176][SQL] LOAD DATA can't identify wildc...

2018-05-09 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/21285 [SPARK-24176][SQL] LOAD DATA can't identify wildcard in the hdfs file path ## What changes were proposed in this pull request? When the wildcard characters (like "?") were in the LOAD DA

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21028 **[Test build #90416 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90416/testReport)** for PR 21028 at commit [`41ef6c6`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3083/ Tes

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/21278 Looking at http://r-pkgs.had.co.nz/description.html - `... the SystemRequirements field. But this is just a plain text field and is not automatically checked.` I think using `== 8` is probably fine

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187169358 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,39 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedArgs

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187169209 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,39 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedArgs

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187169097 --- Diff: R/pkg/R/sparkR.R --- @@ -163,6 +163,10 @@ sparkR.sparkContext <- function( submitOps <- getClientModeSparkSubmitOpts( Sys.ge

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187168952 --- Diff: R/pkg/R/utils.R --- @@ -756,7 +756,7 @@ launchScript <- function(script, combinedArgs, wait = FALSE) { # stdout = F means discard output

[GitHub] spark issue #21280: [SPARK-19181][Core] Fixing flaky "SparkListenerSuite.loc...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21280 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90415/ Test PASSed. ---

[GitHub] spark issue #21280: [SPARK-19181][Core] Fixing flaky "SparkListenerSuite.loc...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21280 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21278 **[Test build #90425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90425/testReport)** for PR 21278 at commit [`a0e9b02`](https://github.com/apache/spark/commit/a0

[GitHub] spark issue #21280: [SPARK-19181][Core] Fixing flaky "SparkListenerSuite.loc...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21280 **[Test build #90415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90415/testReport)** for PR 21280 at commit [`491db75`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #21274: [SPARK-24213][ML] Fix for Int id type for PowerIt...

2018-05-09 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21274#discussion_r187159250 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/PowerIterationClustering.scala --- @@ -231,8 +231,12 @@ class PowerIterationClustering priva

[GitHub] spark issue #21186: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by defau...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21186 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21186: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by defau...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21186 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90417/ Test PASSed. ---

[GitHub] spark issue #21186: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by defau...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21186 **[Test build #90417 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90417/testReport)** for PR 21186 at commit [`10e8319`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/2997/ ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/2997/ --- --

<    1   2   3   4   5   >