[GitHub] spark pull request #19140: [SPARK-21890] Credentials not being passed to add...

2017-09-05 Thread redsanket
GitHub user redsanket opened a pull request: https://github.com/apache/spark/pull/19140 [SPARK-21890] Credentials not being passed to add the tokens I observed this while running a oozie job trying to connect to hbase via spark. It look like the creds are not being passed in

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81425/testReport)** for PR 19140 at commit

[GitHub] spark pull request #19099: [SPARK-21652][SQL] Fix rule confliction between I...

2017-09-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19099 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19139 BTW, please update the PR title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #18692: [SPARK-21417][SQL] Infer join conditions using pr...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18692#discussion_r137111457 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -152,3 +152,71 @@ object EliminateOuterJoin extends

[GitHub] spark pull request #19139: [MINOR][DOC] Update `Partition Discovery` section...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19139#discussion_r137115627 --- Diff: docs/sql-programming-guide.md --- @@ -733,8 +733,9 @@ SELECT * FROM parquetTable Table partitioning is a common optimization

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81431/testReport)** for PR 19140 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137120663 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -145,15 +146,27 @@ class DetermineTableStats(session:

[GitHub] spark issue #18950: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18950 **[Test build #81427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81427/testReport)** for PR 18950 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137102745 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -85,6 +88,14 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137108746 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137108805 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137108881 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18704 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18966: [SPARK-21751][SQL] CodeGeneraor.splitExpressions ...

2017-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18966#discussion_r137164501 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -769,16 +769,27 @@ class CodegenContext {

[GitHub] spark issue #19131: [MINOR][SQL]remove unuse import class

2017-09-05 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19131 Jenkins, test this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137167216 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -875,4 +876,70 @@ class SubquerySuite extends QueryTest with

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137167260 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -875,4 +876,70 @@ class SubquerySuite extends QueryTest with

[GitHub] spark issue #19124: [SPARK-21912][SQL] ORC/Parquet table should not create i...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81435/testReport)** for PR 19124 at commit

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19050 LGTM pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18935 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18935: [SPARK-9104][CORE] Expose Netty memory metrics in...

2017-09-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18935 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137169152 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137171798 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -130,10 +130,12 @@ case class

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137173190 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -130,10 +130,12 @@ case class

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137174463 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137174337 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -206,6 +206,9 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-05 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r137175343 --- Diff: python/pyspark/ml/tuning.py --- @@ -255,18 +257,24 @@ def _fit(self, dataset): randCol = self.uid + "_rand" df =

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r137162378 --- Diff: python/pyspark/ml/tuning.py --- @@ -255,18 +257,24 @@ def _fit(self, dataset): randCol = self.uid + "_rand" df =

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19140 @redsanket can you please test this with a secure Hadoop environment using spark-submit (not Oozie), I don't want to bring in any regression here. ---

[GitHub] spark issue #17254: [SPARK-19917][SQL]qualified partition path stored in cat...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17254 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #17254: [SPARK-19917][SQL]qualified partition path stored in cat...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17254 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81437/ Test FAILed. ---

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137171079 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -206,6 +206,9 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19050 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81439/ Test PASSed. ---

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19050 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19050 **[Test build #81439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81439/testReport)** for PR 19050 at commit

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r137162089 --- Diff: python/pyspark/ml/tuning.py --- @@ -255,18 +257,23 @@ def _fit(self, dataset): randCol = self.uid + "_rand" df =

[GitHub] spark issue #17254: [SPARK-19917][SQL]qualified partition path stored in cat...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17254 **[Test build #81437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81437/testReport)** for PR 17254 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137170172 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,19 @@ object DDLUtils { } }

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137170969 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,19 @@ object DDLUtils { } }

[GitHub] spark issue #19124: [SPARK-21912][SQL] ORC/Parquet table should not create i...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19124 Oh, thank you for review, @viirya, @HyukjinKwon and @gatorsmile ! I'll follow up your comments! --- - To unsubscribe,

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-05 Thread jmchung
Github user jmchung commented on the issue: https://github.com/apache/spark/pull/18865 Could @gatorsmile and @HyukjinKwon please share some instructions for revised details on exception message? The current message indicates the reason of disallowance when users just select the

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19050 **[Test build #81439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81439/testReport)** for PR 19050 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137169608 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137169805 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137174215 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -206,6 +206,9 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @ajbozarth I have updated the implementation which only access FS in FSHistoryServerProvider --- - To unsubscribe, e-mail:

[GitHub] spark issue #19124: [SPARK-21912][SQL] ORC/Parquet table should not create i...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19124: [SPARK-21912][SQL] ORC/Parquet table should not create i...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81435/ Test PASSed. ---

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19050 Thanks @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-09-05 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/18692 @cloud-fan : In event when the (set of join keys) is a superset of (child node's partitioning keys), its possible to avoid shuffle : https://github.com/apache/spark/pull/19054 ... this can help

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19050 **[Test build #81440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81440/testReport)** for PR 19050 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137170635 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,19 @@ object DDLUtils { } }

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137171539 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -206,6 +206,9 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137174591 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,19 @@ object DDLUtils { } }

[GitHub] spark issue #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19136 **[Test build #81441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81441/testReport)** for PR 19136 at commit

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2017-09-05 Thread KevinZwx
Github user KevinZwx commented on the issue: https://github.com/apache/spark/pull/18903 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #81416 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81416/testReport)** for PR 18875 at commit

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19056 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81416/ Test PASSed. ---

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81415/ Test PASSed. ---

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19056 **[Test build #81415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81415/testReport)** for PR 19056 at commit

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19139 I added `(including TEXT/CSV/JSON/ORC/Parquet)` and updated the PR title, too. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137108608 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137111624 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-09-05 Thread brad-kaiser
Github user brad-kaiser commented on the issue: https://github.com/apache/spark/pull/19041 Thanks vanzin. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137096383 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -145,15 +146,27 @@ class DetermineTableStats(session:

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19140 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81424/testReport)** for PR 19124 at commit

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19099 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81418/ Test PASSed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19132 Though this may fix the problem it is 100% the wrong way to do it for multiple reasons: 1. The UI should not touch the FS, FS access it abstracted out so the UI is can work with multiple FS

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19099 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19139: [MINOR][DOC] Add ORC in `Partition Discovery` sec...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19139#discussion_r137101003 --- Diff: docs/sql-programming-guide.md --- @@ -733,7 +733,7 @@ SELECT * FROM parquetTable Table partitioning is a common optimization

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137103061 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -83,6 +83,8 @@ class OrcFileFormat extends FileFormat with

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137107658 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -83,6 +83,8 @@ class OrcFileFormat extends FileFormat with

[GitHub] spark pull request #19139: [MINOR][DOC] Update `Partition Discovery` section...

2017-09-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19139 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137122638 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -145,15 +146,27 @@ class DetermineTableStats(session:

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81426/testReport)** for PR 19140 at commit

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81426/ Test FAILed. ---

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19099 **[Test build #81418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81418/testReport)** for PR 19099 at commit

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19124 That is normal. When we find a bug, it normally means we ignore it in more than one place. Thus, we need to check all the other code paths that could trigger it. ---

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137106989 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -241,11 +241,21 @@ query : ctes? queryNoWith

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137107152 --- Diff: sql/core/src/test/resources/sql-tests/results/show_columns.sql.out --- @@ -19,23 +19,23 @@ struct<> -- !query 2

[GitHub] spark pull request #19139: [MINOR][DOC] Update `Partition Discovery` section...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19139#discussion_r137112679 --- Diff: docs/sql-programming-guide.md --- @@ -733,8 +733,9 @@ SELECT * FROM parquetTable Table partitioning is a common optimization

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19099 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18935 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18935 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81420/ Test PASSed. ---

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81430/testReport)** for PR 19124 at commit

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81429/testReport)** for PR 19139 at commit

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19139 Thanks! Merged to master/2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-09-05 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/16774 This should be ready to merge @jkbradley @MLnick --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19139 @gatorsmile . For this PR, I just update the doc only. Thank you so much! --- - To unsubscribe, e-mail:

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81423/testReport)** for PR 19139 at commit

[GitHub] spark pull request #19140: [SPARK-21890] Credentials not being passed to add...

2017-09-05 Thread redsanket
Github user redsanket commented on a diff in the pull request: https://github.com/apache/spark/pull/19140#discussion_r137096611 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopFSDelegationTokenProvider.scala --- @@ -103,15 +103,17 @@ private[deploy] class

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread redsanket
Github user redsanket commented on the issue: https://github.com/apache/spark/pull/19140 Previous discussion on this PR is here https://github.com/apache/spark/pull/19103 --- - To unsubscribe, e-mail:

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19124 Thank you, @gatorsmile . The PR becomes much general. When I started this PR, I didn't notice that the current Spark had so many missing cases like this. ---

[GitHub] spark issue #19138: [SPARK-21925] Update trigger interval documentation in d...

2017-09-05 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/19138 LGTM. Thanks @brkyvz. Merging to master and 2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   3   4   5   6   >