[GitHub] spark pull request #21306: [SPARK-24252][SQL] Add catalog registration and t...

2018-11-30 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21306#discussion_r237971092 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/PartitionTransforms.java --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache S

[GitHub] spark issue #23189: [SPARK-26235][Core] Change log level for ClassNotFoundEx...

2018-11-30 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23189 Sounds ok. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.

[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...

2018-11-30 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/23086 @cloud-fan, thanks for getting this done! I'll wait for the equivalent write-side PR. --- - To unsubscribe, e-mail: reviews-unsub

[GitHub] spark issue #23189: [SPARK-26235][Core] Change log level for ClassNotFoundEx...

2018-11-30 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/23189 @vanzin Ah, I see. Thanks for pointing it out! But I am now thinking overriding `logError` by calling `printMessage("Error...")`. What do you think? --- --

[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...

2018-11-30 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/23086 > I still do not think we should mix the catalog support with the data source APIs We are trying to keep these separate. `Table` is the only overlap between the two. If you prefer more separ

[GitHub] spark issue #23174: [SPARK-26194][k8s] Auto generate auth secret for k8s app...

2018-11-30 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23174 > The way it's written now Code can change after it's written... > If this change is merged into 3.x without any other changes, users will be forced to use the K8s secret based

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-11-30 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r237966508 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/FPGrowthExample.scala --- @@ -64,4 +64,3 @@ object FPGrowthExample { spark.stop()

[GitHub] spark pull request #23086: [SPARK-25528][SQL] data source v2 API refactor (b...

2018-11-30 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/23086#discussion_r237966188 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/Scan.java --- @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark issue #23174: [SPARK-26194][k8s] Auto generate auth secret for k8s app...

2018-11-30 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/23174 > There doesn't need to be a single solution. This patch going in does not preclude adding more features later, one of which might be reading this from a pre-defined secret. The way it's wr

[GitHub] spark issue #23189: [SPARK-26235][Core] Change log level for ClassNotFoundEx...

2018-11-30 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23189 > The logWarning call as the other handler below is also not overridden: It is. I even copied & pasted the code. I made the change locally and this is what happens: ``` $ ./bin/s

[GitHub] spark pull request #23166: [SPARK-26201] Fix python broadcast with encryptio...

2018-11-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23166 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23189: [SPARK-26235][Core] Change log level for ClassNotFoundEx...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23189 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23189: [SPARK-26235][Core] Change log level for ClassNotFoundEx...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23189 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99514/ Test PASSed. ---

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-11-30 Thread rezasafi
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/22612 The test is flaky and not related to my change. It passed when we re run it last time. Jenkins retest this please. --- - To uns

[GitHub] spark pull request #23186: [SPARK-26230][SQL]FileIndex: if case sensitive, v...

2018-11-30 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/23186#discussion_r237963856 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -345,15 +346,18 @@ object PartitioningUti

[GitHub] spark issue #23189: [SPARK-26235][Core] Change log level for ClassNotFoundEx...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23189 **[Test build #99514 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99514/testReport)** for PR 23189 at commit [`158d421`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #23055: [SPARK-26080][PYTHON] Skips Python resource limit on Win...

2018-11-30 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/23055 +1 once the docs are updated to note that resource requests still include python memory, even in Windows. --- - To unsubscribe, e

[GitHub] spark issue #23166: [SPARK-26201] Fix python broadcast with encryption

2018-11-30 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/23166 thanks @redsanket @squito, committed master, 2.4, and 2.3.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #23055: [SPARK-26080][PYTHON] Skips Python resource limit...

2018-11-30 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/23055#discussion_r237963488 --- Diff: docs/configuration.md --- @@ -190,6 +190,8 @@ of the most common options to set are: and it is up to the application to avoid exceeding the

[GitHub] spark issue #23174: [SPARK-26194][k8s] Auto generate auth secret for k8s app...

2018-11-30 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23174 > while leaving it an exercise for the reader to understand how to properly run spark such that the secrets are actually secured. I don't think that's an exercise for the user, but for the ad

[GitHub] spark issue #23189: [SPARK-26235][Core] Change log level for ClassNotFoundEx...

2018-11-30 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/23189 @vanzin The `logWarning` call as the other handler below is also not overridden: ``` case e: NoClassDefFoundError => logWarning(s"Failed to load $childMainClass: ${e.g

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22612 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99515/ Test FAILed. ---

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22612 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22612 **[Test build #99515 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99515/testReport)** for PR 22612 at commit [`ef4be38`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23181 **[Test build #99524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99524/testReport)** for PR 23181 at commit [`63ea05a`](https://github.com/apache/spark/commit/63

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5588/

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23181 **[Test build #99523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99523/testReport)** for PR 23181 at commit [`0406b92`](https://github.com/apache/spark/commit/04

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5587/

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23181 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23181#discussion_r237959492 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends Spark

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23181#discussion_r237959457 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends Spark

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r237956662 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/FPGrowthExample.scala --- @@ -64,4 +64,3 @@ object FPGrowthExample { spark.sto

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22957 **[Test build #99522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99522/testReport)** for PR 22957 at commit [`5bca5e3`](https://github.com/apache/spark/commit/5b

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5586/

[GitHub] spark issue #23174: [SPARK-26194][k8s] Auto generate auth secret for k8s app...

2018-11-30 Thread gdearment
Github user gdearment commented on the issue: https://github.com/apache/spark/pull/23174 The issue with requiring the use of secrets is quite a bit of work must be done in order to secure a cluster to ensure that the secrets are themselves secured. Most of the high level concerns are

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

2018-11-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23181#discussion_r237952522 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends SparkFunS

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

2018-11-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23181#discussion_r237952479 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends SparkFunS

[GitHub] spark issue #23189: [SPARK-26235][Core] Change log level for ClassNotFoundEx...

2018-11-30 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23189 Actually the problem is here: ``` case e: ClassNotFoundException => logWarning(s"Failed to load $childMainClass.", e) ``` That particular `logWarning` is not

[GitHub] spark issue #23177: [SPARK-26212][Build][test-maven] Upgrade maven version t...

2018-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23177 Oh, @kiszk . Could you update AppVoyer script together? AppVoyer still uses a static version number unlike with `build/mvn`. - https://github.com/apache/spark/blob/master/dev/appveyor-inst

[GitHub] spark pull request #23108: [Spark-25993][SQL][TEST]Add test cases for resolu...

2018-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23108#discussion_r237948799 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -186,6 +186,54 @@ abstract class OrcSuit

[GitHub] spark pull request #23177: [SPARK-26212][Build][test-maven] Upgrade maven ve...

2018-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23177#discussion_r237945252 --- Diff: pom.xml --- @@ -114,7 +114,7 @@ 1.8 ${java.version} ${java.version} -3.5.4 +3.6.0 --- End diff

[GitHub] spark issue #23177: [SPARK-26212][Build][test-maven] Upgrade maven version t...

2018-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23177 Thank you for pinging me, @HyukjinKwon . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20788: [SPARK-23647][PYTHON][SQL] Adds more types for hint in p...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20788 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20788: [SPARK-23647][PYTHON][SQL] Adds more types for hint in p...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20788 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99521/ Test PASSed. ---

[GitHub] spark issue #20788: [SPARK-23647][PYTHON][SQL] Adds more types for hint in p...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20788 **[Test build #99521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99521/testReport)** for PR 20788 at commit [`00a4888`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #23153: [SPARK-26147][SQL] only pull out unevaluable pyth...

2018-11-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/23153#discussion_r237944306 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -155,19 +155,20 @@ object EliminateOuterJoin extends Ru

[GitHub] spark pull request #23055: [SPARK-26080][PYTHON] Skips Python resource limit...

2018-11-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23055#discussion_r237941472 --- Diff: docs/configuration.md --- @@ -190,6 +190,8 @@ of the most common options to set are: and it is up to the application to avoid exceeding the

[GitHub] spark issue #20788: [SPARK-23647][PYTHON][SQL] Adds more types for hint in p...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20788 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20788: [SPARK-23647][PYTHON][SQL] Adds more types for hint in p...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20788 **[Test build #99521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99521/testReport)** for PR 20788 at commit [`00a4888`](https://github.com/apache/spark/commit/00

[GitHub] spark issue #20788: [SPARK-23647][PYTHON][SQL] Adds more types for hint in p...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20788 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5585/

[GitHub] spark issue #20788: [SPARK-23647][PYTHON][SQL] Adds more types for hint in p...

2018-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20788 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99518/ Test FAILed. ---

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22957 **[Test build #99518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99518/testReport)** for PR 22957 at commit [`6eee1e4`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23159: [SPARK-26191][SQL] Control truncation of Spark plans via...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23159 **[Test build #99520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99520/testReport)** for PR 23159 at commit [`a5e82f7`](https://github.com/apache/spark/commit/a5

[GitHub] spark issue #23159: [SPARK-26191][SQL] Control truncation of Spark plans via...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5584/

[GitHub] spark issue #23159: [SPARK-26191][SQL] Control truncation of Spark plans via...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23159 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23173: [SPARK-26208][SQL] add headers to empty csv files when h...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23173 **[Test build #99519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99519/testReport)** for PR 23173 at commit [`6165e1a`](https://github.com/apache/spark/commit/61

[GitHub] spark issue #23173: [SPARK-26208][SQL] add headers to empty csv files when h...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23173 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23173: [SPARK-26208][SQL] add headers to empty csv files when h...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23173 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5583/

[GitHub] spark issue #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of `Colum...

2018-11-30 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/23152 This reproes it: ``` sql("create table all_null (attr1 int, attr2 int)") sql("insert into all_null values (null, null)") sql("analyze table all_null co

[GitHub] spark issue #23152: [SPARK-26181][SQL] the `hasMinMaxStats` method of `Colum...

2018-11-30 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/23152 While at it, could we kill one more potential for a bug? In `FilterEstimation.evaluateBinaryForTwoColumns` there is a ``` attrLeft.dataType match { case StringType |

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5582/

[GitHub] spark pull request #23160: [SPARK-26196][WebUI] Total tasks title in the sta...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23160#discussion_r237907702 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -610,7 +610,8 @@ $(document).ready(function () { $(

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5581/

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #23183: [SPARK-26226][SQL] Update query tracker to report timeli...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23183 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23183: [SPARK-26226][SQL] Update query tracker to report timeli...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23183 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5580/

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22957 **[Test build #99518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99518/testReport)** for PR 22957 at commit [`6eee1e4`](https://github.com/apache/spark/commit/6e

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark pull request #22957: [SPARK-25951][SQL] Ignore aliases for distributio...

2018-11-30 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22957#discussion_r237905754 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala --- @@ -780,6 +780,23 @@ class PlannerSuite extends SharedSQLContext {

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99517/ Test FAILed. ---

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22957 **[Test build #99517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99517/testReport)** for PR 22957 at commit [`5c6b9fc`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22957 **[Test build #99517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99517/testReport)** for PR 22957 at commit [`5c6b9fc`](https://github.com/apache/spark/commit/5c

[GitHub] spark issue #23183: [SPARK-26226][SQL] Update query tracker to report timeli...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23183 **[Test build #99516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99516/testReport)** for PR 23183 at commit [`eb7d194`](https://github.com/apache/spark/commit/eb

[GitHub] spark pull request #23184: [SPARK-26227][R] from_[csv|json] should accept sc...

2018-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23184#discussion_r237896514 --- Diff: R/pkg/R/functions.R --- @@ -202,8 +202,9 @@ NULL #' \itemize{ #' \item \code{from_json}: a structType object to use as t

[GitHub] spark pull request #23184: [SPARK-26227][R] from_[csv|json] should accept sc...

2018-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23184#discussion_r237898787 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -225,4 +225,10 @@ private[sql] object SQLUtils extends Logging {

[GitHub] spark pull request #23184: [SPARK-26227][R] from_[csv|json] should accept sc...

2018-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23184#discussion_r237899057 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -225,4 +225,10 @@ private[sql] object SQLUtils extends Logging {

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22612 **[Test build #99515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99515/testReport)** for PR 22612 at commit [`ef4be38`](https://github.com/apache/spark/commit/ef

[GitHub] spark issue #23185: [MINOR][Docs] Fix typos

2018-11-30 Thread kjmrknsn
Github user kjmrknsn commented on the issue: https://github.com/apache/spark/pull/23185 Thanks for checking and merging. I used a spell checker software and checked its result. --- - To unsubscribe, e-mail: reviews-

[GitHub] spark issue #18406: [SPARK-21195] Automatically register new metrics from so...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18406 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23150: [SPARK-26178][SQL] Use java.time API for parsing timesta...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23150 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99510/ Test PASSed. ---

[GitHub] spark issue #23150: [SPARK-26178][SQL] Use java.time API for parsing timesta...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23150 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23150: [SPARK-26178][SQL] Use java.time API for parsing timesta...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23150 **[Test build #99510 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99510/testReport)** for PR 23150 at commit [`f3f46c7`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #23185: [MINOR][Docs] Fix typos

2018-11-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23185 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23185: [MINOR][Docs] Fix typos

2018-11-30 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/23185 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@

[GitHub] spark pull request #23186: [SPARK-26230][SQL]FileIndex: if case sensitive, v...

2018-11-30 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/23186#discussion_r237889346 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -345,15 +346,18 @@ object PartitioningUtils {

[GitHub] spark pull request #23186: [SPARK-26230][SQL]FileIndex: if case sensitive, v...

2018-11-30 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/23186#discussion_r237889521 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala --- @@ -65,6 +65,34 @@ class FileIndexSuite extends Shar

[GitHub] spark pull request #23186: [SPARK-26230][SQL]FileIndex: if case sensitive, v...

2018-11-30 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/23186#discussion_r237888926 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -345,15 +346,18 @@ object PartitioningUtils {

[GitHub] spark pull request #23146: [SPARK-26173] [MLlib] Prior regularization for Lo...

2018-11-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23146#discussion_r237888294 --- Diff: mllib/src/test/scala/org/apache/spark/ml/optim/loss/DifferentiableRegularizationSuite.scala --- @@ -16,9 +16,13 @@ */ package org.apach

[GitHub] spark pull request #23146: [SPARK-26173] [MLlib] Prior regularization for Lo...

2018-11-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23146#discussion_r237888645 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -250,6 +250,66 @@ private[classification] trait LogisticRe

[GitHub] spark pull request #23160: [SPARK-26196][WebUI] Total tasks title in the sta...

2018-11-30 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/23160#discussion_r237843584 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -610,7 +610,8 @@ $(document).ready(function () { $(

[GitHub] spark pull request #23146: [SPARK-26173] [MLlib] Prior regularization for Lo...

2018-11-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23146#discussion_r237888349 --- Diff: mllib/src/test/scala/org/apache/spark/ml/optim/loss/DifferentiableRegularizationSuite.scala --- @@ -16,9 +16,13 @@ */ package org.apach

[GitHub] spark pull request #23146: [SPARK-26173] [MLlib] Prior regularization for Lo...

2018-11-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23146#discussion_r237888585 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -250,6 +250,66 @@ private[classification] trait LogisticRe

[GitHub] spark pull request #22957: [SPARK-25951][SQL] Ignore aliases for distributio...

2018-11-30 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22957#discussion_r237887275 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala --- @@ -780,6 +780,23 @@ class PlannerSuite extends SharedSQLContext {

[GitHub] spark pull request #23151: [SPARK-26180][CORE][TEST] Reuse withTempDir funct...

2018-11-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23151#discussion_r237886340 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -105,5 +105,16 @@ abstract class SparkFunSuite logInfo(s"\n\n= FINISH

<    1   2   3   4   5   >