[GitHub] spark issue #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap, array_...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21434 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91183/ Test PASSed. ---

[GitHub] spark issue #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap, array_...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21434 **[Test build #91183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91183/testReport)** for PR 21434 at commit

[GitHub] spark issue #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap, array_...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21434 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21438: Improve SQLAppStatusListener.aggregateMetrics() t...

2018-05-26 Thread frb502
GitHub user frb502 opened a pull request: https://github.com/apache/spark/pull/21438 Improve SQLAppStatusListener.aggregateMetrics() too show ## What changes were proposed in this pull request? JIRA Issue: https://issues.apache.org/jira/browse/SPARK-24398 ## How was

[GitHub] spark issue #21438: Improve SQLAppStatusListener.aggregateMetrics() too show

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21438 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21438: Improve SQLAppStatusListener.aggregateMetrics() too show

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21438 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21436: [SPARK-24250][SQL][Follow-up] support accessing SQLConf ...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21436 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21436: [SPARK-24250][SQL][Follow-up] support accessing SQLConf ...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21436 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91180/ Test PASSed. ---

[GitHub] spark issue #21436: [SPARK-24250][SQL][Follow-up] support accessing SQLConf ...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21436 **[Test build #91180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91180/testReport)** for PR 21436 at commit

[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r191041927 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/not-in-unit-tests-multi-column-literal.sql --- @@ -0,0 +1,47 @@ +--

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21435 **[Test build #91181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91181/testReport)** for PR 21435 at commit

[GitHub] spark issue #21436: [SPARK-24250][SQL][Follow-up] support accessing SQLConf ...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21436 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21436: [SPARK-24250][SQL][Follow-up] support accessing SQLConf ...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21436 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3607/

[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-26 Thread mgyucht
Github user mgyucht commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r191042262 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/not-in-unit-tests-multi-column-literal.sql.out --- @@ -0,0 +1,71 @@ +--

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3608/

[GitHub] spark pull request #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experim...

2018-05-26 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21435#discussion_r191042359 --- Diff: python/pyspark/sql/functions.py --- @@ -2448,6 +2448,8 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21397 This seems like it should be good to me. It's a little bit different than the ArrowConverters that also have a listener, because they are iterators and the cleanup can't be put in a finally.

[GitHub] spark pull request #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experim...

2018-05-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21435#discussion_r191043047 --- Diff: python/pyspark/sql/functions.py --- @@ -2448,6 +2448,8 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark issue #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap, array_...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21434 **[Test build #91183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91183/testReport)** for PR 21434 at commit

[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r191041812 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/not-in-unit-tests-multi-column-literal.sql.out --- @@ -0,0 +1,71 @@

[GitHub] spark issue #21436: [SPARK-24250][SQL][Follow-up] support accessing SQLConf ...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21436 **[Test build #91180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91180/testReport)** for PR 21436 at commit

[GitHub] spark issue #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN subquer...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21425 **[Test build #91182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91182/testReport)** for PR 21425 at commit

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91181/ Test PASSed. ---

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21435 **[Test build #91181 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91181/testReport)** for PR 21435 at commit

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap, array_...

2018-05-26 Thread mn-mikke
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21434 Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN subquer...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21425 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN subquer...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21425 **[Test build #91182 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91182/testReport)** for PR 21425 at commit

[GitHub] spark issue #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN subquer...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21425 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91182/ Test PASSed. ---

[GitHub] spark issue #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompress...

2018-05-26 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21431 Yes. I have tested with `—conf spark.io.compression.codec=zstd`. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompress...

2018-05-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21431 Merged to master and branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortC...

2018-05-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21431 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21426 **[Test build #91184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91184/testReport)** for PR 21426 at commit

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21426 **[Test build #91185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91185/testReport)** for PR 21426 at commit

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3609/

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-26 Thread e-dorigatti
Github user e-dorigatti commented on the issue: https://github.com/apache/spark/pull/21383 I am thinking to use `fail_on_stopiteration` in the worker instead of in the `UserDefinedFunction`. I don't really like this solution since you have to fix every other place that uses an udf,

[GitHub] spark issue #21436: [SPARK-24250][SQL][Follow-up] support accessing SQLConf ...

2018-05-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21436 LGTM. Users may call `SparkSession.getActiveSession` directly instead of writing `if (isDriver) ...`. We should not break it. ---

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3611/

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21432 **[Test build #91190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91190/testReport)** for PR 21432 at commit

[GitHub] spark issue #21346: [SPARK-6237][NETWORK] Network-layer changes to allow str...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21346 **[Test build #91192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91192/testReport)** for PR 21346 at commit

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-05-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r191059478 --- Diff: common/network-common/src/test/java/org/apache/spark/network/StreamTestHelper.java --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #21439: [SPARK-24391][SQL] Support arrays of any types by...

2018-05-26 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/21439 [SPARK-24391][SQL] Support arrays of any types by from_json ## What changes were proposed in this pull request? The PR removes a restriction for element types of array type which exists

[GitHub] spark issue #21346: [SPARK-6237][NETWORK] Network-layer changes to allow str...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21346 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3612/

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21435 **[Test build #91193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91193/testReport)** for PR 21435 at commit

[GitHub] spark issue #21346: [SPARK-6237][NETWORK] Network-layer changes to allow str...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21346 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21383 **[Test build #91186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91186/testReport)** for PR 21383 at commit

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21383 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91186/ Test FAILed. ---

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21383 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21383 **[Test build #91186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91186/testReport)** for PR 21383 at commit

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3610/

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21435 **[Test build #91193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91193/testReport)** for PR 21435 at commit

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91193/ Test PASSed. ---

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r191050447 --- Diff: core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala --- @@ -414,7 +415,106 @@ class

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r191050461 --- Diff: core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala --- @@ -414,7 +415,106 @@ class

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21383 **[Test build #91187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91187/testReport)** for PR 21383 at commit

[GitHub] spark pull request #21378: [SPARK-24326][Mesos] add support for local:// sch...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21378#discussion_r191056648 --- Diff: docs/running-on-mesos.md --- @@ -753,6 +753,16 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark pull request #21378: [SPARK-24326][Mesos] add support for local:// sch...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21378#discussion_r191056976 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -418,17 +417,34 @@

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21435 I added a note in the migration guide for 2.3 to 2.4, but in branch-2.3 I don't see a section for 2.3.0 to 2.3.1. When backporting should I make that section for this or ok to just place at

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21432 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21435 I think this LGTM, should we add a note in migration doc from 2.3.0 to 2.3.1 for this change? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21426 **[Test build #91184 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91184/testReport)** for PR 21426 at commit

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91184/ Test PASSed. ---

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21435 yup Im seeing no harn to mtnetion it. It shouldn't be used in production for now. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91185/ Test PASSed. ---

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91189/ Test FAILed. ---

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21439 **[Test build #91189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91189/testReport)** for PR 21439 at commit

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21439 **[Test build #91191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91191/testReport)** for PR 21439 at commit

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3613/

[GitHub] spark issue #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experimental

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21436: [SPARK-24250][SQL][Follow-up] support accessing S...

2018-05-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21436#discussion_r191050261 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -1021,21 +1021,33 @@ object SparkSession extends Logging { /**

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21439 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21435: [SPARK-24392][PYTHON] Label pandas_udf as Experim...

2018-05-26 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21435#discussion_r191059323 --- Diff: python/pyspark/sql/functions.py --- @@ -2448,6 +2448,8 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21426 **[Test build #91185 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91185/testReport)** for PR 21426 at commit

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91188/ Test FAILed. ---

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21432 **[Test build #91188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91188/testReport)** for PR 21432 at commit

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21383 **[Test build #91187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91187/testReport)** for PR 21383 at commit

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21383 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21383 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91187/ Test PASSed. ---

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21432 **[Test build #91188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91188/testReport)** for PR 21432 at commit

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-05-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21439 **[Test build #91189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91189/testReport)** for PR 21439 at commit

[GitHub] spark pull request #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap,...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21434#discussion_r191056304 --- Diff: R/pkg/R/functions.R --- @@ -3062,6 +3077,21 @@ setMethod("array_sort", column(jc) }) +#' @details

[GitHub] spark pull request #21434: [SPARK-24331][SparkR][SQL] Adding arrays_overlap,...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21434#discussion_r191056286 --- Diff: R/pkg/R/functions.R --- @@ -3048,6 +3048,21 @@ setMethod("array_position", column(jc) }) +#'

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91190/ Test PASSed. ---

[GitHub] spark issue #21438: Improve SQLAppStatusListener.aggregateMetrics() too show

2018-05-26 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21438 please see http://spark.apache.org/contributing.html on "Pull Request" also fix the PR title to start with `[SPARK-24398]` ---

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191061877 --- Diff: pom.xml --- @@ -760,6 +760,12 @@ 1.10.19 test + --- End diff -- I think I'm a bit

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191061934 --- Diff: pom.xml --- @@ -760,6 +760,12 @@ 1.10.19 test + --- End diff -- also add

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191061892 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala --- @@ -0,0 +1,120 @@

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191061883 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -154,6 +154,24 @@ private[spark] object

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191061920 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleEventHandler.scala --- @@ -0,0

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use the Kubernetes API to popu...

2018-05-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191061890 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -154,6 +154,24 @@ private[spark] object

  1   2   >