[GitHub] spark issue #20019: [SPARK-22361][SQL][TEST] Add unit test for Window Frames

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20019 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85803/ Test PASSed. ---

[GitHub] spark issue #20019: [SPARK-22361][SQL][TEST] Add unit test for Window Frames

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20019 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20019: [SPARK-22361][SQL][TEST] Add unit test for Window Frames

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20019 **[Test build #85803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85803/testReport)** for PR 20019 at commit [`d1e2454`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-08 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/19872 Ping @ueshin, do you think of the current state of the PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160232913 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * Lic

[GitHub] spark pull request #20171: [SPARK-22978] [PySpark] Register Vectorized UDFs ...

2018-01-08 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20171#discussion_r160229142 --- Diff: python/pyspark/sql/tests.py --- @@ -3616,6 +3616,34 @@ def test_vectorized_udf_basic(self): bool_f(col('bool')))

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20013 **[Test build #85806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85806/testReport)** for PR 20013 at commit [`ed59b4d`](https://github.com/apache/spark/commit/ed

[GitHub] spark issue #20187: [SPARK-22992][K8S] Remove assumption of the DNS domain

2018-01-08 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20187 @liyinan926 -- not yet, will be running them shortly. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addit

[GitHub] spark issue #20098: [SPARK-22914][DEPLOY] Register history.ui.port

2018-01-08 Thread gerashegalov
Github user gerashegalov commented on the issue: https://github.com/apache/spark/pull/20098 Thank you for review and commit @vanzin ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20187: [SPARK-22992][K8S] Remove assumption of the DNS domain

2018-01-08 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20187 @foxish have you run the integration tests against this on both minikube and GKE? --- - To unsubscribe, e-mail: reviews-unsub

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

2018-01-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20013 > I suggest running more tests(real workloads) I've tested all of these PRs attached to SPARK-18085 with lots of real and synthetic app logs. --- --

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r16052 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -119,118 +121,115 @@ private class LiveTask( import LiveEntityHelpe

[GitHub] spark issue #20187: [SPARK-22992][K8S] Remove assumption of the DNS domain

2018-01-08 Thread bowei
Github user bowei commented on the issue: https://github.com/apache/spark/pull/20187 Confirmed -- the FQDN is not going to work on clusters that have a custom cluster suffix. --- - To unsubscribe, e-mail: reviews-un

[GitHub] spark issue #20187: [SPARK-22992][K8S] Remove assumption of the DNS domain

2018-01-08 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20187 cc/ @zihongz @bowei from K8s networking - can you guys confirm that using `..svc` is strictly better than using the FQDN which made an assumption of the dns zone (cluster.local)? --- -

[GitHub] spark issue #20187: [SPARK-22992][K8S] Remove assumption of the DNS domain

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20187 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85805/ Test PASSed. ---

[GitHub] spark issue #20187: [SPARK-22992][K8S] Remove assumption of the DNS domain

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20187 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20187: [SPARK-22992][K8S] Remove assumption of the DNS domain

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20187 **[Test build #85805 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85805/testReport)** for PR 20187 at commit [`bfaf466`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-01-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20146 ok SGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spar

[GitHub] spark pull request #20174: [SPARK-22951][SQL] aggregate should not produce e...

2018-01-08 Thread liufengdb
Github user liufengdb commented on a diff in the pull request: https://github.com/apache/spark/pull/20174#discussion_r160217718 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -230,6 +236,7 @@ case class HashAggregateExec(

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19893 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19893 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85802/ Test PASSed. ---

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19893 **[Test build #85802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85802/testReport)** for PR 19893 at commit [`87c4852`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20187: [SPARK-22992][K8S] Remove assumption of the DNS domain

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20187 **[Test build #85805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85805/testReport)** for PR 20187 at commit [`bfaf466`](https://github.com/apache/spark/commit/bf

[GitHub] spark pull request #20174: [SPARK-22951][SQL] aggregate should not produce e...

2018-01-08 Thread liufengdb
Github user liufengdb commented on a diff in the pull request: https://github.com/apache/spark/pull/20174#discussion_r160216751 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -102,10 +102,12 @@ case class HashAggregateExec

[GitHub] spark pull request #20187: [SPARK-22992][K8S] Remove assumption of the DNS d...

2018-01-08 Thread foxish
GitHub user foxish opened a pull request: https://github.com/apache/spark/pull/20187 [SPARK-22992][K8S] Remove assumption of the DNS domain ## What changes were proposed in this pull request? Remove the use of FQDN to access the driver because it assumes that it's set up in

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

2018-01-08 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/11994 I see, I didn't recognize that the same registry is used to for source as well. in this case, even we have some way to eliminate MetricsRegistry from the API signature, haven't we still i

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160207776 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20186 **[Test build #85804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85804/testReport)** for PR 20186 at commit [`10b5518`](https://github.com/apache/spark/commit/10

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20186 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20186 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85800/ Test PASSed. ---

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20186 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20186 **[Test build #85800 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85800/testReport)** for PR 20186 at commit [`fb87aa6`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20186 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20186 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85801/ Test FAILed. ---

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20186 **[Test build #85801 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85801/testReport)** for PR 20186 at commit [`10b5518`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85798/ Test PASSed. ---

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #85798 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85798/testReport)** for PR 13599 at commit [`289bbf2`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #20179: [SPARK-22982] Remove unsafe asynchronous close() call fr...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20179 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85795/ Test PASSed. ---

[GitHub] spark issue #20179: [SPARK-22982] Remove unsafe asynchronous close() call fr...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20179 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20179: [SPARK-22982] Remove unsafe asynchronous close() call fr...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20179 **[Test build #85795 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85795/testReport)** for PR 20179 at commit [`8e5ffa4`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85797/ Test PASSed. ---

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #85797 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85797/testReport)** for PR 13599 at commit [`1a80150`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #20019: [SPARK-22361][SQL][TEST] Add unit test for Window Frames

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20019 **[Test build #85803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85803/testReport)** for PR 20019 at commit [`d1e2454`](https://github.com/apache/spark/commit/d1

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160190108 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * Lic

[GitHub] spark issue #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19943 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85793/ Test PASSed. ---

[GitHub] spark issue #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19943 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85794/ Test PASSed. ---

[GitHub] spark issue #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19943 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19943 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19943 **[Test build #85794 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85794/testReport)** for PR 19943 at commit [`10e5d7a`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19943 **[Test build #85793 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85793/testReport)** for PR 19943 at commit [`15cac9c`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20165 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20165 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85796/ Test PASSed. ---

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20165 **[Test build #85796 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85796/testReport)** for PR 20165 at commit [`3eafdfb`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160182013 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark issue #19274: [SPARK-22056][Streaming] Add subconcurrency for KafkaRDD...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19274 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160178850 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * Lic

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160178751 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * Lic

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160178331 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark pull request #20178: [Spark-22952][CORE] Deprecate stageAttemptId in f...

2018-01-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20178 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20178: [Spark-22952][CORE] Deprecate stageAttemptId in favour o...

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20178 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark issue #19224: [SPARK-20990][SQL] Read all JSON documents in files when...

2018-01-08 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19224 @gatorsmile I think now it is ready for review, thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For a

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160174950 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * Lic

[GitHub] spark pull request #20178: [Spark-22952][CORE] Deprecate stageAttemptId in f...

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20178#discussion_r160173491 --- Diff: core/src/main/scala/org/apache/spark/scheduler/StageInfo.scala --- @@ -56,6 +56,8 @@ class StageInfo( completionTime = Some(System.curre

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2018-01-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20080 @wangyum any comments on the follow-up here? if you've observed it gets the desired behavior, that's pretty convincing. At least we should treat both timeouts consistently. --- ---

[GitHub] spark pull request #20178: [Spark-22952][CORE] Deprecate stageAttemptId in f...

2018-01-08 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/20178#discussion_r160170753 --- Diff: core/src/main/scala/org/apache/spark/scheduler/StageInfo.scala --- @@ -56,6 +56,8 @@ class StageInfo( completionTime = Some(System.curr

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160169481 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,435 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160168115 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,435 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160167643 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,435 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160167182 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,435 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160165234 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,435 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160164794 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160164837 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160164669 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160163934 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160163140 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160162928 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160162325 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -0,0 +1,509 @@ +/* + * License

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19893 **[Test build #85802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85802/testReport)** for PR 19893 at commit [`87c4852`](https://github.com/apache/spark/commit/87

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19993 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85799/ Test PASSed. ---

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19993 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19993 **[Test build #85799 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85799/testReport)** for PR 19993 at commit [`a0c0fed`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-08 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r160158308 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -27,19 +27,55 @@ import org.apache.spark.util.AccumulatorContext /

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-08 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r160158241 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -27,19 +27,55 @@ import org.apache.spark.util.AccumulatorContext /

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-08 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r160158067 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala --- @@ -17,4 +17,22 @@ package org.apache.spark.sql.test

[GitHub] spark issue #19492: [SPARK-22228][SQL] Add support for array...

2018-01-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19492 To be honest, just to share what have been on my mind on this so far, I have been trying to think if it's worth vs the amount of changes, in particular, we now happen to have some divergence, `m

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20186 **[Test build #85801 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85801/testReport)** for PR 20186 at commit [`10b5518`](https://github.com/apache/spark/commit/10

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20186 good catch! LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20186 **[Test build #85800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85800/testReport)** for PR 20186 at commit [`fb87aa6`](https://github.com/apache/spark/commit/fb

[GitHub] spark pull request #20186: [SPARK-22990][Core] Fix method isFairScheduler in...

2018-01-08 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/20186 [SPARK-22990][Core] Fix method isFairScheduler in JobsTab and StagesTab ## What changes were proposed in this pull request? In current implementation, the function `isFairScheduler` i

[GitHub] spark issue #19492: [SPARK-22228][SQL] Add support for array...

2018-01-08 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19492 any more comments on this @viirya @HyukjinKwon? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additiona

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r160147366 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DecimalType.scala --- @@ -117,6 +117,7 @@ object DecimalType extends AbstractDataType {

[GitHub] spark issue #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-08 Thread henrify
Github user henrify commented on the issue: https://github.com/apache/spark/pull/19943 @dongjoon-hyun It is possible that the "multiple byte arrays" case happens only in write side when consumer code explicitly does it, and it is fine to use the single byte array and putArray() in rea

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19993 **[Test build #85799 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85799/testReport)** for PR 19993 at commit [`a0c0fed`](https://github.com/apache/spark/commit/a0

[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2018-01-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19993#discussion_r160143590 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -249,6 +250,27 @@ object ParamValidators { def arrayLengthGt[T](lowerBou

[GitHub] spark pull request #20019: [SPARK-22361][SQL][TEST] Add unit test for Window...

2018-01-08 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20019#discussion_r160143221 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala --- @@ -263,21 +263,60 @@ class ExpressionPars

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/13599 (re) cc @Stibbons for reviewing. I think you are another right one to review this given https://github.com/apache/spark/pull/14180. Let's start to incrementally improve this feature unless you a

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #85798 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85798/testReport)** for PR 13599 at commit [`289bbf2`](https://github.com/apache/spark/commit/28

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-08 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/13599 @holdenk @ueshin @HyukjinKwon Thanks for review the long pending PR. Will refine the PR soon. --- - To unsubscribe, e-mail: revi

<    1   2   3   4   5   6   >