[GitHub] spark issue #20199: [Spark-22967][Hive]Fix VersionSuite's unit tests by chan...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20199 Let's fix the PR title to `[SPARK-22967][TESTS]`. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addi

[GitHub] spark pull request #20199: [Spark-22967][Hive]Fix VersionSuite's unit tests ...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20199#discussion_r160396117 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala --- @@ -58,7 +58,7 @@ class VersionsSuite extends SparkFunSuite w

[GitHub] spark issue #14129: [SPARK-16280][SQL] Implement histogram_numeric SQL funct...

2018-01-09 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/14129 @cloud-fan I think this PR is to implement Hive's `histogram_numeric` function. It produces a histogram to approximate data distribution. It's different from standard equi-width or equi-height histogr

[GitHub] spark issue #20199: [Spark-22967][Hive]Fix VersionSuite's unit tests by chan...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20199 I met the similar case when I fixed some tests related with Kafka before - https://issues-test.apache.org/jira/browse/KAFKA-1194. This took me a while to debug and find this issue .. T

[GitHub] spark issue #20199: [Spark-22967][Hive]Fix VersionSuite's unit tests by chan...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20199 I can trigger a test on Windows. Let me leave the build: Build started: [SQL] `org.apache.spark.sql.hive.client.VersionsSuite` [![PR-20199](https://ci.appveyor.com/api/projects/status/gi

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2018-01-09 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20080 @srowen @jiangxb1987 I have test on my cluster with this patch. ``` bin/spark-sql --master yarn --conf spark.dynamicAllocation.enabled=true --conf spark.shuffle.service.enabled=true --conf

[GitHub] spark issue #19492: [SPARK-22228][SQL] Add support for array...

2018-01-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19492 I will be busy on relocating in next few days. I'll look into this further if I have time in maybe weekend. --- - To unsubscribe,

[GitHub] spark pull request #20199: [Spark-22967][Hive]Fix VersionSuite's unit tests ...

2018-01-09 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20199#discussion_r160404821 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala --- @@ -58,7 +58,7 @@ class VersionsSuite extends SparkFunSuite with

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20189 **[Test build #85850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85850/testReport)** for PR 20189 at commit [`7242eab`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85850/ Test FAILed. ---

[GitHub] spark issue #20199: [Spark-22967][TESTS]Fix VersionSuite's unit tests by cha...

2018-01-09 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/20199 Ok, will try. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #85849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85849/testReport)** for PR 18853 at commit [`e763330`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85849/ Test PASSed. ---

[GitHub] spark pull request #20202: [MINOR] fix a typo in BroadcastJoinSuite

2018-01-09 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20202 [MINOR] fix a typo in BroadcastJoinSuite ## What changes were proposed in this pull request? `BroadcastNestedLoopJoinExec` should be `BroadcastHashJoinExec` ## How was this patch

[GitHub] spark issue #20202: [MINOR] fix a typo in BroadcastJoinSuite

2018-01-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20202 cc @liufengdb @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark pull request #20199: [Spark-22967][TESTS]Fix VersionSuite's unit tests...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20199#discussion_r160408416 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala --- @@ -58,7 +58,7 @@ class VersionsSuite extends SparkFunSuite w

[GitHub] spark issue #20202: [MINOR] fix a typo in BroadcastJoinSuite

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20202 **[Test build #85853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85853/testReport)** for PR 20202 at commit [`c615d37`](https://github.com/apache/spark/commit/c6

[GitHub] spark issue #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19943 thanks, merging to master/2.3! Let's address the comments in follow-up. BTW @dongjoon-hyun let's keep our discussion on https://github.com/apache/spark/pull/19943#discussion_r160326383 -

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19943 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20176: [SPARK-22981][SQL] Fix incorrect results of Casting Stru...

2018-01-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20176 LGTM, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20176: [SPARK-22981][SQL] Fix incorrect results of Casti...

2018-01-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20176 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20140: [SPARK-19228][SQL] Introduce tryParseDate method to proc...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20140 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20176: [SPARK-22981][SQL] Fix incorrect results of Casting Stru...

2018-01-09 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20176 Thanks! Next, I'll fix `showString` though, one question; currently casting binary to string is different between [Cast](https://github.com/apache/spark/blob/2250cb75b99d257e698fe5418a51d8cddb4d5104/

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-09 Thread attilapiros
GitHub user attilapiros opened a pull request: https://github.com/apache/spark/pull/20203 [SPARK-22577] [core] executor page blacklist status should update with TaskSet level blacklisting ## What changes were proposed in this pull request? In this PR stage blacklisting is p

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20203 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20200: [SPARK-23005][Core] Improve RDD.take on small number of ...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20200 **[Test build #85854 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85854/testReport)** for PR 20200 at commit [`9cd590d`](https://github.com/apache/spark/commit/9c

[GitHub] spark issue #20176: [SPARK-22981][SQL] Fix incorrect results of Casting Stru...

2018-01-09 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20176 hive one is the same with mysql one; ``` hive> create table t(a BINARY); OK hive> INSERT INTO t values('abc'); OK hive> select CAST(a AS STRING) from t; OK abc ```

[GitHub] spark issue #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19943 Thank you so much, @cloud-fan , @mmccline , @viirya , @henrify , @kiszk , @HyukjinKwon ! I'll proceed to follow-ups. --- -

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160432982 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,435 @@ +/* + * Licensed to the Apache So

[GitHub] spark pull request #20171: [SPARK-22978] [PySpark] Register Vectorized UDFs ...

2018-01-09 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20171#discussion_r160441123 --- Diff: python/pyspark/sql/tests.py --- @@ -3616,6 +3616,34 @@ def test_vectorized_udf_basic(self): bool_f(col('bool')))

[GitHub] spark pull request #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20204#discussion_r160445389 --- Diff: python/run-tests.py --- @@ -175,6 +175,9 @@ def main(): task_queue = Queue.PriorityQueue() for python_exec in python_ex

[GitHub] spark pull request #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage...

2018-01-09 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20204 [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage generation script ## What changes were proposed in this pull request? Note that this PR was made based on the top of https://github.co

[GitHub] spark issue #20200: [SPARK-23005][Core] Improve RDD.take on small number of ...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20200 **[Test build #85851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85851/testReport)** for PR 20200 at commit [`93a3d84`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20204 cc @rxin, @felixcheung, @ueshin, @holdenk and @JoshRosen. Here, I made it based on #20151. Could you take a look for this one too? I can make this PR separate from #20151 by manually adding the

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20204 cc @icexelloss too. I think we had a small talk about this before. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.ap

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85855/testReport)** for PR 20204 at commit [`a3179d7`](https://github.com/apache/spark/commit/a3

[GitHub] spark issue #20200: [SPARK-23005][Core] Improve RDD.take on small number of ...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20200 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85851/ Test PASSed. ---

[GitHub] spark issue #20200: [SPARK-23005][Core] Improve RDD.take on small number of ...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20200 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20201 **[Test build #85852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85852/testReport)** for PR 20201 at commit [`be14e3b`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20201 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20201 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85852/ Test PASSed. ---

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85856/testReport)** for PR 20204 at commit [`9f2c400`](https://github.com/apache/spark/commit/9f

[GitHub] spark pull request #20205: [SPARK-16060][SQL][follow-up] add a wrapper solut...

2018-01-09 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20205 [SPARK-16060][SQL][follow-up] add a wrapper solution for vectorized orc reader ## What changes were proposed in this pull request? This is mostly from https://github.com/apache/spark/pul

[GitHub] spark issue #20205: [SPARK-16060][SQL][follow-up] add a wrapper solution for...

2018-01-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20205 cc @dongjoon-hyun @viirya @kiszk @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20163: [SPARK-22966][PySpark] Spark SQL should handle Py...

2018-01-09 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20163#discussion_r160450067 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvaluatePython.scala --- @@ -120,10 +121,18 @@ object EvaluatePython {

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20204 This is awesome. Thanks @HyukjinKwon! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comman

[GitHub] spark issue #20205: [SPARK-16060][SQL][follow-up] add a wrapper solution for...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20205 **[Test build #85857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85857/testReport)** for PR 20205 at commit [`bdf9dbf`](https://github.com/apache/spark/commit/bd

[GitHub] spark pull request #20206: [SPARK-19256][SQL] Remove ordering enforcement fr...

2018-01-09 Thread tejasapatil
GitHub user tejasapatil opened a pull request: https://github.com/apache/spark/pull/20206 [SPARK-19256][SQL] Remove ordering enforcement from `FileFormatWriter` and let planner do that ## What changes were proposed in this pull request? Thks is as per discussion in https:/

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85858/testReport)** for PR 20204 at commit [`3c3c3cb`](https://github.com/apache/spark/commit/3c

[GitHub] spark issue #20206: [SPARK-19256][SQL] Remove ordering enforcement from `Fil...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20206 **[Test build #85859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85859/testReport)** for PR 20206 at commit [`652dca2`](https://github.com/apache/spark/commit/65

[GitHub] spark pull request #20207: [SPARK-23000] [FOLLOW-UP] Fix Flaky test suite Da...

2018-01-09 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/20207 [SPARK-23000] [FOLLOW-UP] Fix Flaky test suite DataSourceWithHiveMetastoreCatalogSuite in Spark 2.3 [WIP] ## What changes were proposed in this pull request? Tried to reproduce it.

[GitHub] spark issue #20207: [SPARK-23000] [TEST-HADOOP2.6] Fix Flaky test suite Data...

2018-01-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20207 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #20207: [SPARK-23000] [TEST-HADOOP2.6] Fix Flaky test suite Data...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20207 **[Test build #85860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85860/testReport)** for PR 20207 at commit [`f239b2b`](https://github.com/apache/spark/commit/f2

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20189 **[Test build #85861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85861/testReport)** for PR 20189 at commit [`b7dc922`](https://github.com/apache/spark/commit/b7

[GitHub] spark pull request #20205: [SPARK-16060][SQL][follow-up] add a wrapper solut...

2018-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20205#discussion_r160461130 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java --- @@ -196,17 +234,26 @@ public void ini

[GitHub] spark issue #20190: [SPARK-22976][Core]: Cluster mode driver directories can...

2018-01-09 Thread RussellSpitzer
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/20190 @zsxwing I think you were the last to touch this code, could you please review? --- - To unsubscribe, e-mail: reviews-uns

[GitHub] spark issue #20202: [MINOR] fix a typo in BroadcastJoinSuite

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20202 **[Test build #85853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85853/testReport)** for PR 20202 at commit [`c615d37`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #20202: [MINOR] fix a typo in BroadcastJoinSuite

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20202 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20202: [MINOR] fix a typo in BroadcastJoinSuite

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20202 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85853/ Test PASSed. ---

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2018-01-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20080 This change alone is pretty harmless. If it seems to work as intended, seems OK? Again, just wondering if `cachedExecutorIdleTimeoutS` should be the same, and check for nonnegative values? --- ---

[GitHub] spark issue #20205: [SPARK-16060][SQL][follow-up] add a wrapper solution for...

2018-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20205 BTW, if you don't mind, could you update the followings? It's @viirya 's comment, so I made a followup, but we had better have this in your PR. To make another follow-up is overkill. :) ``

[GitHub] spark issue #20202: [MINOR] fix a typo in BroadcastJoinSuite

2018-01-09 Thread liufengdb
Github user liufengdb commented on the issue: https://github.com/apache/spark/pull/20202 thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.

[GitHub] spark issue #20158: [PySpark] Fix typo in comments in PySpark's udf() defini...

2018-01-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20158 Ping @rednaxelafx --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r160472490 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -137,4 +147,25 @@ private[sql] trait ColumnarBatchScan extend

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r160475752 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -110,107 +114,238 @@ private[spark] class AppStatusStore( if (deta

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r160475833 --- Diff: core/src/main/scala/org/apache/spark/status/storeTypes.scala --- @@ -69,51 +71,271 @@ private[spark] class JobDataWrapper( val skippedStage

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r160475957 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -110,107 +114,238 @@ private[spark] class AppStatusStore( if (deta

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20013 > the memory usage will be much larger with InMemoryStore. No it won't. The in-memory store does not use extra memory for indices. It sorts the data when the index is read, and discards that

[GitHub] spark issue #20153: [SPARK-22392][SQL] data source v2 columnar batch reader

2018-01-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20153 Is `ColumnarBatchScan` appropriate name for now? This is because if `supportBatch` is false, the class handles scan from a `row`, not `columnar` or `batch`. ---

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r160477447 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/SupportsScanColumnarBatch.java --- @@ -0,0 +1,51 @@ +/* + * Licensed to the A

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r160477594 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -17,21 +17,24 @@ package org.apache.spark.sql.exec

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r160478184 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -119,118 +121,115 @@ private class LiveTask( import LiveEntityHelpe

[GitHub] spark issue #20186: [SPARK-22990][Core] Fix method isFairScheduler in JobsTa...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20186 > I think it's fine. We usually don't test if a config can be correctly set as it's so trivial... The point is to test that the code using that config works as expected, not that it can be s

[GitHub] spark pull request #20179: [SPARK-22982] Remove unsafe asynchronous close() ...

2018-01-09 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/20179#discussion_r160481843 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -196,11 +196,24 @@ private[spark] class IndexShuffleBlockRes

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20204 Sorry for noice. I made some clean up and just rebased. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org Fo

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85862/testReport)** for PR 20204 at commit [`dadcae4`](https://github.com/apache/spark/commit/da

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #85863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85863/testReport)** for PR 20023 at commit [`6701a54`](https://github.com/apache/spark/commit/67

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #85864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85864/testReport)** for PR 20023 at commit [`20616fd`](https://github.com/apache/spark/commit/20

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2018-01-09 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 I'd just suggest trying it. Since this PR is a patch for master, please message me personally at m...@allman.ms to discuss progress and questions on a backport to 2.2. If we get it working,

[GitHub] spark issue #20205: [SPARK-16060][SQL][follow-up] add a wrapper solution for...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20205 **[Test build #85857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85857/testReport)** for PR 20205 at commit [`bdf9dbf`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20205: [SPARK-16060][SQL][follow-up] add a wrapper solution for...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20205 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20205: [SPARK-16060][SQL][follow-up] add a wrapper solution for...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20205 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85857/ Test FAILed. ---

[GitHub] spark issue #20206: [SPARK-19256][SQL] Remove ordering enforcement from `Fil...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20206 **[Test build #85859 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85859/testReport)** for PR 20206 at commit [`652dca2`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #20206: [SPARK-19256][SQL] Remove ordering enforcement from `Fil...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20206 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85859/ Test FAILed. ---

[GitHub] spark issue #20206: [SPARK-19256][SQL] Remove ordering enforcement from `Fil...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20206 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r160486870 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -119,118 +121,115 @@ private class LiveTask( import LiveEntityHelpe

[GitHub] spark pull request #20179: [SPARK-22982] Remove unsafe asynchronous close() ...

2018-01-09 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/20179#discussion_r160489460 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -196,11 +196,24 @@ private[spark] class IndexShuffleBlockRes

[GitHub] spark issue #20207: [SPARK-23000] [TEST-HADOOP2.6] Fix Flaky test suite Data...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20207 **[Test build #85860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85860/testReport)** for PR 20207 at commit [`f239b2b`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #20208: [SPARK-23007][SQL][TEST] Add schema evolution tes...

2018-01-09 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/20208 [SPARK-23007][SQL][TEST] Add schema evolution test suite for file-based data sources ## What changes were proposed in this pull request? A schema can evolve in several ways and the f

[GitHub] spark issue #20207: [SPARK-23000] [TEST-HADOOP2.6] Fix Flaky test suite Data...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20207 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85860/ Test PASSed. ---

[GitHub] spark issue #20207: [SPARK-23000] [TEST-HADOOP2.6] Fix Flaky test suite Data...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20207 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20208 Hi, @gatorsmile , @cloud-fan , @HyukjinKwon , @viirya . Could you review this PR? --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20208 **[Test build #85865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85865/testReport)** for PR 20208 at commit [`499801e`](https://github.com/apache/spark/commit/49

[GitHub] spark issue #20200: [SPARK-23005][Core] Improve RDD.take on small number of ...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20200 **[Test build #85854 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85854/testReport)** for PR 20200 at commit [`9cd590d`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20200: [SPARK-23005][Core] Improve RDD.take on small number of ...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20200 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85854/ Test PASSed. ---

[GitHub] spark issue #20200: [SPARK-23005][Core] Improve RDD.take on small number of ...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20200 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20209: [SPARK-23008][ML] OnehotEncoderEstimator python A...

2018-01-09 Thread WeichenXu123
GitHub user WeichenXu123 opened a pull request: https://github.com/apache/spark/pull/20209 [SPARK-23008][ML] OnehotEncoderEstimator python API ## What changes were proposed in this pull request? OnehotEncoderEstimator python API. ## How was this patch tested?

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20209 **[Test build #85866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85866/testReport)** for PR 20209 at commit [`f6215fc`](https://github.com/apache/spark/commit/f6

<    1   2   3   4   5   6   >