[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1777/ Tes

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88620/testReport)** for PR 20784 at commit [`1e38840`](https://github.com/apache/spark/commit/1e

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88616/ Test PASSed. ---

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20850 **[Test build #88616 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88616/testReport)** for PR 20850 at commit [`3637a5c`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/20784 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88617/ Test FAILed. ---

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88617/testReport)** for PR 20784 at commit [`1e38840`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #20907: [WIP][SPARK-11237][ML] Add pmml export for k-mean...

2018-03-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20907#discussion_r177322828 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -185,6 +187,47 @@ class KMeansModel private[ml] ( } }

[GitHub] spark pull request #20902: [SPARK-23770][R] Exposes repartitionByRange in Sp...

2018-03-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20902#discussion_r177317996 --- Diff: R/pkg/R/DataFrame.R --- @@ -759,6 +759,67 @@ setMethod("repartition", dataFrame(sdf) }) + +#' Rep

[GitHub] spark pull request #20897: [MINOR][DOC] Fix a few markdown typos

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20897#discussion_r177318153 --- Diff: docs/mllib-feature-extraction.md --- @@ -278,8 +278,8 @@ for details on the API. multiplication. In other words, it scales each column of

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88615/ Test PASSed. ---

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88615 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88615/testReport)** for PR 20784 at commit [`5c4335b`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88613/ Test PASSed. ---

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88613 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88613/testReport)** for PR 20784 at commit [`ba6b086`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #88619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88619/testReport)** for PR 19222 at commit [`c53b6b8`](https://github.com/apache/spark/commit/c5

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1776/ Tes

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20844: [SPARK-23707][SQL] Don't need shuffle exchange wi...

2018-03-26 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/20844#discussion_r177311524 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ConfigBehaviorSuite.scala --- @@ -39,7 +39,7 @@ class ConfigBehaviorSuite extends QueryTest with Sha

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177306568 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java --- @@ -69,8 +64,9 @@ public long size() { * Fill this all with 0L.

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20906 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20906 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88611/ Test PASSed. ---

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20906 **[Test build #88611 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88611/testReport)** for PR 20906 at commit [`26c1ead`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #20756: [SPARK-23593][SQL] Add interpreted execution for ...

2018-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20756#discussion_r177302845 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ObjectExpressionsSuite.scala --- @@ -68,6 +68,32 @@ class ObjectExpressionsS

[GitHub] spark issue #20756: [SPARK-23593][SQL] Add interpreted execution for Initial...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20756 **[Test build #88618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88618/testReport)** for PR 20756 at commit [`e7640e1`](https://github.com/apache/spark/commit/e7

[GitHub] spark issue #20756: [SPARK-23593][SQL] Add interpreted execution for Initial...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20756 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20756: [SPARK-23593][SQL] Add interpreted execution for Initial...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20756 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1775/ Tes

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20906 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20906 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88614/ Test FAILed. ---

[GitHub] spark pull request #20756: [SPARK-23593][SQL] Add interpreted execution for ...

2018-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20756#discussion_r177302386 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ObjectExpressionsSuite.scala --- @@ -68,6 +68,23 @@ class ObjectExpressionsS

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20906 **[Test build #88614 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88614/testReport)** for PR 20906 at commit [`4a7a4cc`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88617/testReport)** for PR 20784 at commit [`1e38840`](https://github.com/apache/spark/commit/1e

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1774/ Tes

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20753: [SPARK-23582][SQL] StaticInvoke should support interpret...

2018-03-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20753 @hvanhovell sorry for ping again --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20850 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1773/ Tes

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20850 **[Test build #88616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88616/testReport)** for PR 20850 at commit [`3637a5c`](https://github.com/apache/spark/commit/36

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20797: [SPARK-23583][SQL] Invoke should support interpreted exe...

2018-03-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20797 @hvanhovell sorry for ping again --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177296492 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/OnHeapMemoryBlock.java --- @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20908: [SPARK-23672][PYTHON] Document support for nested...

2018-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20908#discussion_r177296271 --- Diff: python/pyspark/sql/tests.py --- @@ -3966,6 +3967,15 @@ def random_udf(v): random_udf = random_udf.asNondeterministic() ret

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177296342 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,161 @@ */ public static final int F

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177296162 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,161 @@ */ public static final int F

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177295079 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java --- @@ -89,6 +85,6 @@ public void set(int index, long value) { p

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177295038 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java --- @@ -80,7 +76,7 @@ public void zeroOut() { public void set(in

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177294950 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java --- @@ -69,8 +64,9 @@ public long size() { * Fill this all with

[GitHub] spark pull request #20908: [SPARK-23672][PYTHON] Document support for nested...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20908#discussion_r177294401 --- Diff: python/pyspark/sql/tests.py --- @@ -3966,6 +3967,15 @@ def random_udf(v): random_udf = random_udf.asNondeterministic()

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177293958 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -0,0 +1,164 @@ +/* + * Licens

[GitHub] spark pull request #20908: [SPARK-23672][PYTHON] Document support for nested...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20908#discussion_r177293289 --- Diff: python/pyspark/sql/tests.py --- @@ -3806,6 +3806,7 @@ def test_timestamp_dst(self): not _have_pandas or not _have_pyarrow, _p

[GitHub] spark pull request #20784: [SPARK-23639][SQL]Obtain token before init metast...

2018-03-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20784#discussion_r177291761 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -77,6 +80,12 @@ private[hive] object Sp

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20784 cc @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177290859 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -0,0 +1,158 @@ +/* + * Licens

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177290805 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkPod.scala --- @@ -14,17 +14,16 @@ * See the License fo

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88615/testReport)** for PR 20784 at commit [`5c4335b`](https://github.com/apache/spark/commit/5c

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1772/ Tes

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20906 **[Test build #88614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88614/testReport)** for PR 20906 at commit [`4a7a4cc`](https://github.com/apache/spark/commit/4a

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177288972 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + * L

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1771/ Tes

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20906 All that should be left before removing the inheritance is to introduce a continuous version of the logical plan WriteToDataSourceV2. I wanted to do that in a separate PR, since there's already

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88613/testReport)** for PR 20784 at commit [`ba6b086`](https://github.com/apache/spark/commit/ba

[GitHub] spark pull request #20784: [SPARK-23639][SQL]Obtain token before init metast...

2018-03-26 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/20784#discussion_r177288126 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -121,6 +134,25 @@ private[hive] objec

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88612/ Test FAILed. ---

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88612/testReport)** for PR 20784 at commit [`4063855`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88612/testReport)** for PR 20784 at commit [`4063855`](https://github.com/apache/spark/commit/40

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1747/ ---

[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88604/ Test FAILed. ---

[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20893 **[Test build #88604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88604/testReport)** for PR 20893 at commit [`4ca8a32`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177245022 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + * Licensed

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177244722 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + * Licensed

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177275489 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + * Licensed

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177243246 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + * Licensed

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20906 **[Test build #88611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88611/testReport)** for PR 20906 at commit [`26c1ead`](https://github.com/apache/spark/commit/26

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20906 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark pull request #20858: [SPARK-23736][SQL] Extending the concat function ...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20858#discussion_r177279540 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -287,3 +289,152 @@ case class ArrayCo

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88610/ Test PASSed. ---

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 **[Test build #88610 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88610/testReport)** for PR 20910 at commit [`f3540f8`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #20858: [SPARK-23736][SQL] Extending the concat function ...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20858#discussion_r177278880 --- Diff: python/pyspark/sql/functions.py --- @@ -1834,6 +1819,25 @@ def array_contains(col, value): return Column(sc._jvm.functions.array_conta

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1770/ Tes

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1747/ --- --

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20858: [SPARK-23736][SQL] Extending the concat function ...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20858#discussion_r177278671 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -699,3 +699,88 @@ abstract class TernaryExpress

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177277920 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -0,0 +1,158 @@ +/* + * Licensed

[GitHub] spark issue #20900: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_u...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20900 The issue itself here (SPARK-23645) describes kwargs arguments support in both UDF and Pandas UDF on calling side. Seems not working but the fix looks going to be quite invasive and big. So, I s

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177277876 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -0,0 +1,164 @@ +/* + * Licensed

[GitHub] spark issue #20858: [SPARK-23736][SQL] Extending the concat function to supp...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20858 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20858: [SPARK-23736][SQL] Extending the concat function to supp...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20858 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88605/ Test FAILed. ---

[GitHub] spark issue #20858: [SPARK-23736][SQL] Extending the concat function to supp...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20858 **[Test build #88605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88605/testReport)** for PR 20858 at commit [`753499d`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 **[Test build #88610 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88610/testReport)** for PR 20910 at commit [`f3540f8`](https://github.com/apache/spark/commit/f3

[GitHub] spark issue #20900: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_u...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20900 to be clear, I think both functions below ```python class F(object): def __call__(...): ... func = F() ``` ```python def naive_func(a, b)

[GitHub] spark pull request #20836: SPARK-23685 : Fix for the Spark Structured Stream...

2018-03-26 Thread sirishaSindri
Github user sirishaSindri commented on a diff in the pull request: https://github.com/apache/spark/pull/20836#discussion_r177276642 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala --- @@ -279,9 +279,8 @@ private[kafka010] case

  1   2   3   4   >