[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2017-09-05 Thread KevinZwx
Github user KevinZwx commented on the issue: https://github.com/apache/spark/pull/18903 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19136 **[Test build #81441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81441/testReport)** for PR 19136 at commit

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-05 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r137175343 --- Diff: python/pyspark/ml/tuning.py --- @@ -255,18 +257,24 @@ def _fit(self, dataset): randCol = self.uid + "_rand" df =

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-05 Thread jmchung
Github user jmchung commented on the issue: https://github.com/apache/spark/pull/18865 Could @gatorsmile and @HyukjinKwon please share some instructions for revised details on exception message? The current message indicates the reason of disallowance when users just select the

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19050 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19050 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81439/ Test PASSed. ---

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19050 **[Test build #81439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81439/testReport)** for PR 19050 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137174591 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,19 @@ object DDLUtils { } }

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137174463 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137174337 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -206,6 +206,9 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137174215 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -206,6 +206,9 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137173190 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -130,10 +130,12 @@ case class

[GitHub] spark issue #19124: [SPARK-21912][SQL] ORC/Parquet table should not create i...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19124 Oh, thank you for review, @viirya, @HyukjinKwon and @gatorsmile ! I'll follow up your comments! --- - To unsubscribe,

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137171798 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -130,10 +130,12 @@ case class

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137171539 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -206,6 +206,9 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137171079 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -206,6 +206,9 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137170969 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,19 @@ object DDLUtils { } }

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137170635 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,19 @@ object DDLUtils { } }

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137170172 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,19 @@ object DDLUtils { } }

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137169805 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137169608 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137169152 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the

[GitHub] spark pull request #18935: [SPARK-9104][CORE] Expose Netty memory metrics in...

2017-09-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18935 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18935 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17254: [SPARK-19917][SQL]qualified partition path stored in cat...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17254 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81437/ Test FAILed. ---

[GitHub] spark issue #17254: [SPARK-19917][SQL]qualified partition path stored in cat...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17254 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #17254: [SPARK-19917][SQL]qualified partition path stored in cat...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17254 **[Test build #81437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81437/testReport)** for PR 17254 at commit

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-09-05 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/18692 @cloud-fan : In event when the (set of join keys) is a superset of (child node's partitioning keys), its possible to avoid shuffle : https://github.com/apache/spark/pull/19054 ... this can help

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19050 Thanks @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19050 **[Test build #81440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81440/testReport)** for PR 19050 at commit

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19050 LGTM pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19124: [SPARK-21912][SQL] ORC/Parquet table should not create i...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81435/ Test PASSed. ---

[GitHub] spark issue #19124: [SPARK-21912][SQL] ORC/Parquet table should not create i...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19124: [SPARK-21912][SQL] ORC/Parquet table should not create i...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81435/testReport)** for PR 19124 at commit

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137167260 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -875,4 +876,70 @@ class SubquerySuite extends QueryTest with

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137167216 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -875,4 +876,70 @@ class SubquerySuite extends QueryTest with

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @ajbozarth I have updated the implementation which only access FS in FSHistoryServerProvider --- - To unsubscribe, e-mail:

[GitHub] spark issue #19131: [MINOR][SQL]remove unuse import class

2017-09-05 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19131 Jenkins, test this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18966: [SPARK-21751][SQL] CodeGeneraor.splitExpressions ...

2017-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18966#discussion_r137164501 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -769,16 +769,27 @@ class CodegenContext {

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18704 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19140 @redsanket can you please test this with a secure Hadoop environment using spark-submit (not Oozie), I don't want to bring in any regression here. ---

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r137162378 --- Diff: python/pyspark/ml/tuning.py --- @@ -255,18 +257,24 @@ def _fit(self, dataset): randCol = self.uid + "_rand" df =

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r137162089 --- Diff: python/pyspark/ml/tuning.py --- @@ -255,18 +257,23 @@ def _fit(self, dataset): randCol = self.uid + "_rand" df =

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19050 **[Test build #81439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81439/testReport)** for PR 19050 at commit

[GitHub] spark issue #19110: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19110 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19110: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19110 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81438/ Test PASSed. ---

[GitHub] spark issue #19110: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19110 **[Test build #81438 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81438/testReport)** for PR 19110 at commit

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137159908 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -49,6 +49,30 @@ object RewritePredicateSubquery extends

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137159871 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -875,4 +876,71 @@ class SubquerySuite extends QueryTest with

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137159884 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -98,6 +122,7 @@ object RewritePredicateSubquery extends

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137159896 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -49,6 +49,30 @@ object RewritePredicateSubquery extends

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81434/ Test PASSed. ---

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19056 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19056 **[Test build #81434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81434/testReport)** for PR 19056 at commit

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-09-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18692 > After adding the inferred join conditions, it might lead to the child node's partitioning NOT satisfying the JOIN node's requirements which otherwise could have. Isn't it an existing

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 So, it's removed before 2.0.0. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19110: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19110 **[Test build #81438 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81438/testReport)** for PR 19110 at commit

[GitHub] spark pull request #18628: [SPARK-18061][ThriftServer] Add spnego auth suppo...

2017-09-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18628 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 Thanks for your recommendation @ajbozarth .Could you put a link for your pr?For the problem you mentioned,i have thought about them. 1、FsHistoryServer will always use FS to get event log

[GitHub] spark issue #19110: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19110 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81436/ Test FAILed. ---

[GitHub] spark issue #19110: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19110 **[Test build #81436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81436/testReport)** for PR 19110 at commit

[GitHub] spark issue #18628: [SPARK-18061][ThriftServer] Add spnego auth support for ...

2017-09-05 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18628 Thanks @jiangxb1987 , let me merge it to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19110: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19110 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17254: [SPARK-19917][SQL]qualified partition path stored in cat...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17254 **[Test build #81437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81437/testReport)** for PR 17254 at commit

[GitHub] spark issue #19110: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19110 **[Test build #81436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81436/testReport)** for PR 19110 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137153437 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2000,4 +2000,38 @@ class SQLQuerySuite extends

[GitHub] spark pull request #19124: [SPARK-21912][SQL] ORC/Parquet table should not c...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137153372 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -206,6 +206,9 @@ case class AlterTableAddColumnsCommand(

[GitHub] spark issue #19124: [SPARK-21912][SQL] ORC/Parquet table should not create i...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81435 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81435/testReport)** for PR 19124 at commit

[GitHub] spark pull request #19102: [SPARK-21859][CORE] Fix SparkFiles.get failed on ...

2017-09-05 Thread lgrcyanny
Github user lgrcyanny closed the pull request at: https://github.com/apache/spark/pull/19102 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19117 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81433/ Test FAILed. ---

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-05 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/19086 @gatorsmile More comments on this ? Regarding the behavior change, should we follow Spark previous behavior or follow Hive? I'm ok with both. ---

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19117 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19140: [SPARK-21890] Credentials not being passed to add...

2017-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19140#discussion_r137152903 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopFSDelegationTokenProvider.scala --- @@ -103,15 +103,17 @@ private[deploy] class

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19117 **[Test build #81433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81433/testReport)** for PR 19117 at commit

[GitHub] spark issue #19142: When the number of attempting to restart receiver greate...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19142 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19142: When the number of attempting to restart receiver...

2017-09-05 Thread liuxianjiao
GitHub user liuxianjiao opened a pull request: https://github.com/apache/spark/pull/19142 When the number of attempting to restart receiver greater than 0,spark do nothing in 'else' When the number of attempting to restart receiver greater than 0,spark do nothing in 'else'.So I

[GitHub] spark issue #19135: [SPARK-21923][CORE]Avoid call reserveUnrollMemoryForThis...

2017-09-05 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/19135 hi @cloud-fan, The previous writing is the same as `putIteratorAsValues`. Now I have modified the code, each application for an additional `chunkSize` bytes of memory, because the size of

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18865 @jmchung, just to be clear, sure, let's go in this way and I guess we have the only comment left to address now: > Please update the error message and also add it to the migration

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19124 I created SPARK-21929 for **"Support `ALTER TABLE table_name ADD COLUMNS(..)` for ORC data source"**. For Parquet ALTER TABLE, yes. I think I can include that here. But, for the

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81432/ Test PASSed. ---

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81432/testReport)** for PR 19124 at commit

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/19056 LGTM. Will merge after tests pass. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19124 Could this PR cover this scenario? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81431/ Test PASSed. ---

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18865 @gatorsmile, Thanks for elaborating this. Looks a fair point. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81431/testReport)** for PR 19140 at commit

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19124 For that, no. It's not considered yet like the other code path. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19124 Altering table add column with illegal column names will issue an error message? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19124 Parquet works. I tested. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19124 How about Parquet? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137148036 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -98,6 +122,7 @@ object RewritePredicateSubquery

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19124 Ah, there are too many missing things in ORC code path. `AlterTableAddColumnsCommand` seems not to allow ORC in

[GitHub] spark issue #19020: [SPARK-3181] [ML] Implement huber loss for LinearRegress...

2017-09-05 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19020 Looks good. cc @jkbradley Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19141: [SPARK-21384] [YARN] Spark 2.2 + YARN without spark.yarn...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19141 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19126: [SPARK-21915][ML][PySpark]Model 1 and Model 2 ParamMaps ...

2017-09-05 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19126 Yeah, I checked and this is not a problem in master since #17849 --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19141: [SPARK-21384] [YARN] Spark 2.2 + YARN without spa...

2017-09-05 Thread devaraj-kavali
GitHub user devaraj-kavali opened a pull request: https://github.com/apache/spark/pull/19141 [SPARK-21384] [YARN] Spark 2.2 + YARN without spark.yarn.jars / spark.yarn.archive fails ## What changes were proposed in this pull request? When the libraries temp directory(i.e.

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137146867 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -130,10 +130,12 @@ case class

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137146817 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -130,10 +130,12 @@ case class

  1   2   3   4   5   6   >