[GitHub] spark issue #22363: [SPARK-25375][SQL][TEST] Reenable qualified perm. functi...

2018-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22363 Thank you for review, @HyukjinKwon . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comm

[GitHub] spark pull request #21618: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-09-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21618#discussion_r216132926 --- Diff: core/src/main/java/org/apache/hadoop/fs/SparkGlobber.java --- @@ -0,0 +1,293 @@ +/** + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request #19773: [SPARK-22546][SQL] Supporting for changing column...

2018-09-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19773#discussion_r216132779 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -350,16 +366,11 @@ case class AlterTableChangeColumnCommand(

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22337 **[Test build #95833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95833/testReport)** for PR 22337 at commit [`309e265`](https://github.com/apache/spark/commit/30

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2948/

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attributes f...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22326 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attributes f...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22326 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95830/ Test PASSed. ---

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attributes f...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22326 **[Test build #95830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95830/testReport)** for PR 22326 at commit [`6875719`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2947/

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22337 **[Test build #95832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95832/testReport)** for PR 22337 at commit [`a314776`](https://github.com/apache/spark/commit/a3

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216131276 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -927,7 +927,7 @@ class SparkSqlAstBuilder(conf: SQLConf) extends

[GitHub] spark issue #22316: [SPARK-25048][SQL] Pivoting by multiple columns in Scala...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22316 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95829/ Test PASSed. ---

[GitHub] spark issue #22316: [SPARK-25048][SQL] Pivoting by multiple columns in Scala...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22316 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22316: [SPARK-25048][SQL] Pivoting by multiple columns in Scala...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22316 **[Test build #95829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95829/testReport)** for PR 22316 at commit [`8ccf845`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #22337: [SPARK-25338][Test] Ensure to call super.beforeAl...

2018-09-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22337#discussion_r216130925 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/WideSchemaBenchmark.scala --- @@ -54,8 +54,11 @@ class WideSchemaBenchmark extend

[GitHub] spark issue #22360: [MINOR][ML] Remove `BisectingKMeansModel.setDistanceMeas...

2018-09-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22360 OK so the wrapper model's constructor needs to set this value once from the wrapped model? It does seem like it should be immutable and seems to not be exposed to set in k means for example. ---

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22364 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95828/ Test PASSed. ---

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22364 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22364 **[Test build #95828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95828/testReport)** for PR 22364 at commit [`14edbe6`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #22349: [SPARK-25345][ML] Deprecate public APIs from ImageSchema

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95831/ Test PASSed. ---

[GitHub] spark issue #22349: [SPARK-25345][ML] Deprecate public APIs from ImageSchema

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22349 **[Test build #95831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95831/testReport)** for PR 22349 at commit [`576b7d8`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #22349: [SPARK-25345][ML] Deprecate public APIs from ImageSchema

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22213 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95826/ Test PASSed. ---

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22213 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22213 **[Test build #95826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95826/testReport)** for PR 22213 at commit [`603fdc9`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216128234 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) extends

[GitHub] spark pull request #19773: [SPARK-22546][SQL] Supporting for changing column...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19773#discussion_r216128008 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -350,16 +366,11 @@ case class AlterTableChangeColumnCommand(

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216127950 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -261,6 +261,14 @@ partitionVal : identifier (EQ consta

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216127931 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -29,10 +29,10 @@ import org.apache.hadoop.mapred.{FileInputFormat,

[GitHub] spark pull request #22326: [SPARK-25314][SQL] Fix Python UDF accessing attri...

2018-09-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22326#discussion_r216127932 --- Diff: python/pyspark/sql/tests.py --- @@ -547,6 +547,74 @@ def test_udf_in_filter_on_top_of_join(self): df = left.crossJoin(right).filt

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22262 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22262 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95827/ Test PASSed. ---

[GitHub] spark pull request #19773: [SPARK-22546][SQL] Supporting for changing column...

2018-09-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19773#discussion_r216127904 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -350,16 +366,11 @@ case class AlterTableChangeColumnCommand(

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22262 **[Test build #95827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95827/testReport)** for PR 22262 at commit [`26b4710`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #19773: [SPARK-22546][SQL] Supporting for changing column...

2018-09-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19773#discussion_r216127880 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -350,16 +366,11 @@ case class AlterTableChangeColumnCommand(

[GitHub] spark issue #22349: [SPARK-25345][ML] Deprecate public APIs from ImageSchema

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22349 **[Test build #95831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95831/testReport)** for PR 22349 at commit [`576b7d8`](https://github.com/apache/spark/commit/57

[GitHub] spark issue #22349: [SPARK-25345][ML] Deprecate public APIs from ImageSchema

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2946/

[GitHub] spark issue #22349: [SPARK-25345][ML] Deprecate public APIs from ImageSchema

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attibutes fr...

2018-09-08 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22326 @holdenk Thanks, sorry for the typo. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216127734 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) extends

[GitHub] spark pull request #22326: [SPARK-25314][SQL] Fix Python UDF accessing attib...

2018-09-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22326#discussion_r216127710 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1202,15 +1243,38 @@ object PushPredicateThroughJ

[GitHub] spark pull request #22326: [SPARK-25314][SQL] Fix Python UDF accessing attib...

2018-09-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22326#discussion_r216127673 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1202,15 +1243,38 @@ object PushPredicateThroughJ

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attibutes fr...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22326 **[Test build #95830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95830/testReport)** for PR 22326 at commit [`6875719`](https://github.com/apache/spark/commit/68

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attibutes fr...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22326 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attibutes fr...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22326 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2945/

[GitHub] spark pull request #22326: [SPARK-25314][SQL] Fix Python UDF accessing attib...

2018-09-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22326#discussion_r216127605 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1149,6 +1149,47 @@ object PushPredicateThroughJo

[GitHub] spark pull request #22335: [SPARK-25091][Core] reduce the storage memory in ...

2018-09-08 Thread cfangplus
Github user cfangplus closed the pull request at: https://github.com/apache/spark/pull/22335 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22335: [SPARK-25091][Core] reduce the storage memory in Executo...

2018-09-08 Thread cfangplus
Github user cfangplus commented on the issue: https://github.com/apache/spark/pull/22335 viirya did a great job, seems this pr should be closed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #22341: [SPARK-24889][Core] Update block info when unpers...

2018-09-08 Thread cfangplus
Github user cfangplus commented on a diff in the pull request: https://github.com/apache/spark/pull/22341#discussion_r216127013 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -646,7 +647,47 @@ private[spark] class AppStatusListener( }

[GitHub] spark issue #22360: [MINOR][ML] Remove `BisectingKMeansModel.setDistanceMeas...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22360 Thanks for pinging me @srowen. > But I don't see that its getDistanceMeasure would actually return the value inside that wrapped model? I am missing where the wrapper gets updated to retur

[GitHub] spark issue #22316: [SPARK-25048][SQL] Pivoting by multiple columns in Scala...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22316 **[Test build #95829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95829/testReport)** for PR 22316 at commit [`8ccf845`](https://github.com/apache/spark/commit/8c

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216126194 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -261,6 +261,14 @@ partitionVal : identifier (EQ cons

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216126249 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) exten

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216126480 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala --- @@ -861,7 +861,8 @@ class DDLParserSuite extends PlanTes

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216126332 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) exten

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216126453 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1015,6 +1037,23 @@ class AstBuilder(conf: SQLConf) ext

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216126471 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -29,10 +29,10 @@ import org.apache.hadoop.mapred.{FileInputFormat

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216126217 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) exten

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216126491 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -577,6 +578,143 @@ class HiveDDLSuite } }

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216126328 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) exten

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22364 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2944/

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22364 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21791: [SPARK-24925][SQL] input bytesRead metrics fluctuate fro...

2018-09-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21791 I see. thank you for sharing the dependency and status. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addi

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22364 **[Test build #95828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95828/testReport)** for PR 22364 at commit [`14edbe6`](https://github.com/apache/spark/commit/14

[GitHub] spark pull request #22364: [SPARK-25379][SQL] Improve AttributeSet and Colum...

2018-09-08 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/22364 [SPARK-25379][SQL] Improve AttributeSet and ColumnPruning performance ## What changes were proposed in this pull request? This PR contains 3 optimizations: 1) it improves significant

[GitHub] spark issue #22335: [SPARK-25091][Core] reduce the storage memory in Executo...

2018-09-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22335 Yea, looks like this is duplicate to #22341. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional com

[GitHub] spark issue #21816: [SPARK-24794][CORE] Driver launched through rest should ...

2018-09-08 Thread bsikander
Github user bsikander commented on the issue: https://github.com/apache/spark/pull/21816 could someone please have a look at this change? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additio

[GitHub] spark issue #22335: [SPARK-25091][Core] reduce the storage memory in Executo...

2018-09-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22335 @cfangplus just for your information, #22341 is modifying the similar parts in `AppStatusListener.scala`. --- - To unsubscribe, e-

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216124388 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1015,6 +1037,23 @@ class AstBuilder(conf: SQLConf) exten

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216124365 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) extends

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216124252 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) extends

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216124163 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) extends

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216124059 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf) extends

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216123948 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -29,10 +29,10 @@ import org.apache.hadoop.mapred.{FileInputFormat,

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-08 Thread cfangplus
Github user cfangplus commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r216123919 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -33,7 +29,9 @@ import org.apache.spark.storage.RDDInfo import org.apache

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-08 Thread cfangplus
Github user cfangplus commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r216123864 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -646,8 +646,17 @@ private[spark] class AppStatusListener( }

[GitHub] spark issue #22311: [WIP][SPARK-25305][SQL] Respect attribute name in Collap...

2018-09-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22311 ok, thanks for the check. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark pull request #19773: [SPARK-22546][SQL] Supporting for changing column...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19773#discussion_r216123860 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -350,16 +366,11 @@ case class AlterTableChangeColumnCommand(

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-08 Thread cfangplus
Github user cfangplus commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r216123775 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -710,9 +719,21 @@ private[spark] class AppStatusListener( va

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216123701 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala --- @@ -861,7 +861,8 @@ class DDLParserSuite extends PlanTest

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216122595 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -577,6 +578,143 @@ class HiveDDLSuite } }

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-09-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r216122503 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -261,6 +261,14 @@ partitionVal : identifier (EQ consta

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22262 **[Test build #95827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95827/testReport)** for PR 22262 at commit [`26b4710`](https://github.com/apache/spark/commit/26

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22262 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22213 **[Test build #95826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95826/testReport)** for PR 22213 at commit [`603fdc9`](https://github.com/apache/spark/commit/60

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22213 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22213 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95819/ Test FAILed. ---

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22213 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22262 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95824/ Test FAILed. ---

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22262 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22262 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22262 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95825/ Test FAILed. ---

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22262 **[Test build #95825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95825/testReport)** for PR 22262 at commit [`26b4710`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #21654: [SPARK-24671][PySpark] DataFrame length using a dunder/m...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21654 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95823/ Test FAILed. ---

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22262 **[Test build #95824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95824/testReport)** for PR 22262 at commit [`366bb35`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95822/ Test FAILed. ---

<    1   2   3   >