[GitHub] spark issue #19148: [SPARK-21936][SQL] backward compatibility test framework...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19148 LGTM except two minor comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark pull request #19148: [SPARK-21936][SQL] backward compatibility test fr...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137683665 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -0,0 +1,193 @@ +/* + * Licensed to the

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 Thank you for review, @gatorsmile , @HyukjinKwon , @maropu . In this issue, I've learned how to track the unused stuff correctly. Thank you again. --- ---

[GitHub] spark pull request #19148: [SPARK-21936][SQL] backward compatibility test fr...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137682740 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -0,0 +1,193 @@ +/* + * Licensed to the

[GitHub] spark pull request #19148: [SPARK-21936][SQL] backward compatibility test fr...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137682115 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -0,0 +1,193 @@ +/* + * Licensed to the

[GitHub] spark issue #19148: [SPARK-21936][SQL] backward compatibility test framework...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19148 Less than 2 mins to finish the suite. It looks pretty good! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18956 LGTM except two minor comments --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-m

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137681045 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -64,6 +64,14 @@ abstract class RuleExecutor[TreeType

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137680999 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -64,6 +64,14 @@ abstract class RuleExecutor[TreeType

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137680545 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,176 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137680153 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -360,6 +360,31 @@ case class Insert

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19129 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19129 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137678987 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizerSICheckerSuite.scala --- @@ -0,0 +1,60 @@ +/* + * Licensed to

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137679007 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -64,6 +64,14 @@ abstract class RuleExecutor[TreeType <:

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #81529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81529/testReport)** for PR 18956 at commit [`d1db7cf`](https://github.com/apache/spark/commit/d1

[GitHub] spark pull request #19046: [SPARK-18769][yarn] Limit resource requests based...

2017-09-07 Thread vanzin
Github user vanzin closed the pull request at: https://github.com/apache/spark/pull/19046 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19046: [SPARK-18769][yarn] Limit resource requests based on RM'...

2017-09-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19046 I'm going to close this; when I find some free time I might take a closer at the issue described in Wilfred's message. --- - To u

[GitHub] spark pull request #19148: [SPARK-21936][SQL] backward compatibility test fr...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137663570 --- Diff: sql/hive/pom.xml --- @@ -177,6 +177,10 @@ libfb303 + org.apache.derby + derby --- End di

[GitHub] spark issue #19148: [SPARK-21936][SQL] backward compatibility test framework...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19148 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81524/ Test PASSed. ---

[GitHub] spark issue #19148: [SPARK-21936][SQL] backward compatibility test framework...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19148 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19148: [SPARK-21936][SQL] backward compatibility test framework...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19148 **[Test build #81524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81524/testReport)** for PR 19148 at commit [`08dcf22`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #17589: [SPARK-16544][SQL] Support for conversion from numeric c...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17589 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19129 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19129 **[Test build #81525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81525/testReport)** for PR 19129 at commit [`8e3d8fe`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #17589: [SPARK-16544][SQL] Support for conversion from numeric c...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17589 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81527/ Test PASSed. ---

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81525/ Test PASSed. ---

[GitHub] spark issue #17589: [SPARK-16544][SQL] Support for conversion from numeric c...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17589 **[Test build #81527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81527/testReport)** for PR 17589 at commit [`cbf8a22`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #19103: [SPARK-21890] Credentials not being passed to add...

2017-09-07 Thread redsanket
Github user redsanket closed the pull request at: https://github.com/apache/spark/pull/19103 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81528/ Test FAILed. ---

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81528/testReport)** for PR 18975 at commit [`4a5ff29`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #19157: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19157 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19157: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19157 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81526/ Test FAILed. ---

[GitHub] spark issue #19157: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19157 **[Test build #81526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81526/testReport)** for PR 19157 at commit [`8b38300`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137650844 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -659,62 +659,67 @@ class HiveDDLSuite }

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19129 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81523/ Test PASSed. ---

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19129 **[Test build #81523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81523/testReport)** for PR 19129 at commit [`40ed9ff`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #19130: [SPARK-21917][CORE][YARN] Supporting adding http(s) reso...

2017-09-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19130 sorry probably wont' get to this today, will look tomorrow. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19155: [MINOR][TEST] Tables created in unit tests should be dro...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19155 Since it's a big patch, `+408 −359`, we had better have a JIRA issue with 'TRIVIAL` priority. --- - To unsubscribe, e-ma

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137638893 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -1603,7 +1617,7 @@ class SQLQuerySuite extends Query

[GitHub] spark pull request #19153: [SPARK-21941] Stop storing unused attemptId in SQ...

2017-09-07 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/19153#discussion_r137637831 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala --- @@ -478,10 +476,11 @@ private[ui] class SQLStageMetrics(

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137637931 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -370,21 +370,24 @@ class HiveQuerySuite extends Hiv

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137637228 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -349,21 +351,23 @@ class SQLQuerySuite extends Query

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137637074 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -116,69 +116,71 @@ class SQLQuerySuite extends Query

[GitHub] spark pull request #19155: [MINOR][TEST] Tables created in unit tests should...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19155#discussion_r137636117 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -461,19 +471,23 @@ class InsertIntoHiveTableSuite e

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-07 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 @snir It's hard to pin down exactly where the problem is and how to fix it. We removed the `NullIntolerant` trait from the implementations of the `ExtractValue` trait in `complexTypeExtractors.scala

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-07 Thread snir
Github user snir commented on the issue: https://github.com/apache/spark/pull/16578 @mallman Yes, the issue is only reproducible with codegen on. However the performance improvement by codegen is too significant to disable. I managed to fix it with some changes to the codegen (Added

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81528/testReport)** for PR 18975 at commit [`4a5ff29`](https://github.com/apache/spark/commit/4a

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137629284 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,150 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18941: [SPARK-21715][WebUI] History Server should not re...

2017-09-07 Thread zhouyejoe
Github user zhouyejoe closed the pull request at: https://github.com/apache/spark/pull/18941 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #17589: [SPARK-16544][SQL] Support for conversion from numeric c...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17589 **[Test build #81527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81527/testReport)** for PR 17589 at commit [`cbf8a22`](https://github.com/apache/spark/commit/cb

[GitHub] spark issue #19157: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19157 **[Test build #81526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81526/testReport)** for PR 19157 at commit [`8b38300`](https://github.com/apache/spark/commit/8b

[GitHub] spark issue #19157: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/19157 Reopened this because CI was having issues with the previous PR. [18950](https://github.com/apache/spark/pull/18950) --- - To uns

[GitHub] spark issue #19157: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/19157 @squito @markhamstra @tgravescs Can you review this PR. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #19157: [SPARK-20589][Core][Scheduler] Allow limiting tas...

2017-09-07 Thread dhruve
GitHub user dhruve opened a pull request: https://github.com/apache/spark/pull/19157 [SPARK-20589][Core][Scheduler] Allow limiting task concurrency per job group ## What changes were proposed in this pull request? This change allows the user to specify the maximum no. of tasks ru

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19129 **[Test build #81525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81525/testReport)** for PR 19129 at commit [`8e3d8fe`](https://github.com/apache/spark/commit/8e

[GitHub] spark issue #18950: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-07 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/18950 CI is having issues downloading my repo. Closing this PR and opening a new one. --- - To unsubscribe, e-mail: reviews-unsubscr...

[GitHub] spark pull request #18950: [SPARK-20589][Core][Scheduler] Allow limiting tas...

2017-09-07 Thread dhruve
Github user dhruve closed the pull request at: https://github.com/apache/spark/pull/18950 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137622298 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - A

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137621847 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - A

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137621549 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -0,0 +1,193 @@ +/* + * Licensed to the

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137621065 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogBackwardCompatibilitySuite.scala --- @@ -1,260 +0,0 @@ -/* - * Lic

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137621130 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala --- @@ -1354,31 +1354,4 @@ class MetastoreDataSourcesSuite ex

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137621009 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - A

[GitHub] spark pull request #19148: [SPARK-21936][SQL][WIP] backward compatibility te...

2017-09-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19148#discussion_r137620814 --- Diff: sql/hive/pom.xml --- @@ -177,6 +177,10 @@ libfb303 + org.apache.derby + derby --- End diff -

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137620763 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - A

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137620489 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - A

[GitHub] spark issue #19148: [SPARK-21936][SQL][WIP] backward compatibility test fram...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19148 **[Test build #81524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81524/testReport)** for PR 19148 at commit [`08dcf22`](https://github.com/apache/spark/commit/08

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137619510 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - As a

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137619452 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - A

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137619209 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - As a

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137617487 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - A

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-09-07 Thread erikerlandson
Github user erikerlandson commented on the issue: https://github.com/apache/spark/pull/19041 I have been thinking about a different but related [use case](https://github.com/apache-spark-on-k8s/spark/issues/261); supporting the ability to operate in dynamic allocation mode without req

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r137616632 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - As a

[GitHub] spark pull request #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_...

2017-09-07 Thread parente
Github user parente commented on a diff in the pull request: https://github.com/apache/spark/pull/18339#discussion_r137616450 --- Diff: python/pyspark/java_gateway.py --- @@ -33,14 +33,18 @@ from pyspark.serializers import read_int -def launch_gateway(conf=None)

[GitHub] spark pull request #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_...

2017-09-07 Thread parente
Github user parente commented on a diff in the pull request: https://github.com/apache/spark/pull/18339#discussion_r137616353 --- Diff: python/pyspark/java_gateway.py --- @@ -33,14 +33,18 @@ from pyspark.serializers import read_int -def launch_gateway(conf=None)

[GitHub] spark issue #19148: [SPARK-21936][SQL][WIP] backward compatibility test fram...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19148 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19148: [SPARK-21936][SQL][WIP] backward compatibility test fram...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19148 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81520/ Test FAILed. ---

[GitHub] spark issue #19148: [SPARK-21936][SQL][WIP] backward compatibility test fram...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19148 **[Test build #81520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81520/testReport)** for PR 19148 at commit [`3d827f9`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19129 **[Test build #81523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81523/testReport)** for PR 19129 at commit [`40ed9ff`](https://github.com/apache/spark/commit/40

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137613115 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -64,6 +64,14 @@ abstract class RuleExecutor[TreeType

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137611075 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -93,6 +101,13 @@ abstract class RuleExecutor[TreeTyp

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137610235 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizerSICheckerSuite.scala --- @@ -0,0 +1,60 @@ +/* + * License

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-09-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r137609405 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala --- @@ -93,6 +101,13 @@ abstract class RuleExecutor[TreeTyp

[GitHub] spark issue #19147: [WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs in Pytho...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19147 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81519/ Test FAILed. ---

[GitHub] spark issue #19147: [WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs in Pytho...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19147 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137607929 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,81 @@ +/* + * Licen

[GitHub] spark issue #19147: [WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs in Pytho...

2017-09-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19147 **[Test build #81519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81519/testReport)** for PR 19147 at commit [`2f929d8`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137607749 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,81 @@ +/* + * Licen

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137607545 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to t

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137607475 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to t

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 The PR title resolved two issues under title `[SPARK-15639][SPARK-16321][SQL] Push down filter at RowGroups level for parquet reader` I'll add like the following. Is it enough? ``` -

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137606401 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to t

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137605940 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTmpPath.scala --- @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137605602 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala --- @@ -366,4 +367,43 @@ class InsertSuite extends DataSourceTest with

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137605509 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -140,6 +143,14 @@ case class DataSourceAnal

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137605311 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala --- @@ -524,6 +525,50 @@ class DDLParserSuite extends Plan

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-09-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18956 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

<    1   2   3   4   >