[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81426/testReport)** for PR 19140 at commit

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137100218 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19139 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81423/ Test PASSed. ---

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19139 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81423 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81423/testReport)** for PR 19139 at commit

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19138: [SPARK-21925] Update trigger interval documentation in d...

2017-09-05 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/19138 LGTM. Thanks @brkyvz. Merging to master and 2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81425/ Test FAILed. ---

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81425 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81425/testReport)** for PR 19140 at commit

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81425/testReport)** for PR 19140 at commit

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread redsanket
Github user redsanket commented on the issue: https://github.com/apache/spark/pull/19140 Previous discussion on this PR is here https://github.com/apache/spark/pull/19103 --- - To unsubscribe, e-mail:

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19124 Thank you, @gatorsmile . The PR becomes much general. When I started this PR, I didn't notice that the current Spark had so many missing cases like this. ---

[GitHub] spark pull request #19140: [SPARK-21890] Credentials not being passed to add...

2017-09-05 Thread redsanket
Github user redsanket commented on a diff in the pull request: https://github.com/apache/spark/pull/19140#discussion_r137096611 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopFSDelegationTokenProvider.scala --- @@ -103,15 +103,17 @@ private[deploy] class

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19140 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81424/testReport)** for PR 19124 at commit

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137096383 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -145,15 +146,27 @@ class DetermineTableStats(session:

[GitHub] spark issue #19103: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread redsanket
Github user redsanket commented on the issue: https://github.com/apache/spark/pull/19103 @vanzin @tgravescs sorry for the delay, will put up a PR against master, we can move further discussion there, about the suggested improvements, I put up a PR against master just for workaround.

[GitHub] spark pull request #19140: [SPARK-21890] Credentials not being passed to add...

2017-09-05 Thread redsanket
GitHub user redsanket opened a pull request: https://github.com/apache/spark/pull/19140 [SPARK-21890] Credentials not being passed to add the tokens I observed this while running a oozie job trying to connect to hbase via spark. It look like the creds are not being passed in

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19139 @gatorsmile . For this PR, I just update the doc only. Thank you so much! --- - To unsubscribe, e-mail:

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81423/testReport)** for PR 19139 at commit

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-09-05 Thread brad-kaiser
Github user brad-kaiser commented on the issue: https://github.com/apache/spark/pull/19041 Thanks vanzin. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-09-05 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/16774 This should be ready to merge @jkbradley @MLnick --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81416/ Test PASSed. ---

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #81416 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81416/testReport)** for PR 18875 at commit

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19056 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81415/ Test PASSed. ---

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19056 **[Test build #81415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81415/testReport)** for PR 19056 at commit

[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-09-05 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18029#discussion_r137086237 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to

[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-09-05 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18029#discussion_r137086738 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to

[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-09-05 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18029#discussion_r136725034 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to

[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-09-05 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18029#discussion_r137083713 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to

[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-09-05 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18029#discussion_r137085859 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/InitialPosition.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137085144 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137085104 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137084856 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19139 `abstract class HadoopFsRelationTest` is extended by four data sources. You can move some test cases into it. However, I am not sure whether you do it for all of them. ---

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137082903 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19041 It might take some time for me to find time to review this, but in the meantime others who might be interested: @cloud-fan @JoshRosen ---

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137082186 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137081917 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19139 As you pointed out, I checked it again. Right, `HadoopFsRelationTest` have it. Should we merge `ParquetPartitionDiscoverySuite` and `OrcPartitionDiscoverySuite` into

[GitHub] spark issue #19138: [SPARK-21925] Update trigger interval documentation in d...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19138 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19139 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81421/ Test PASSed. ---

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19139 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19138: [SPARK-21925] Update trigger interval documentation in d...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19138 **[Test build #81422 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81422/testReport)** for PR 19138 at commit

[GitHub] spark issue #19138: [SPARK-21925] Update trigger interval documentation in d...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19138 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81422/ Test PASSed. ---

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81421 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81421/testReport)** for PR 19139 at commit

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19139 Are the partition discovery applicable to all the built-in data sources? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137078673 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -85,6 +87,13 @@ case class

[GitHub] spark issue #19138: [SPARK-21925] Update trigger interval documentation in d...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19138 **[Test build #81422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81422/testReport)** for PR 19138 at commit

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81421 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81421/testReport)** for PR 19139 at commit

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 Sure, I'll. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19118: [SPARK-21882][CORE] OutputMetrics doesn't count written ...

2017-09-05 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19118 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19139: [MINOR][DOC] Add ORC in `Partition Discovery` sec...

2017-09-05 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/19139 [MINOR][DOC] Add ORC in `Partition Discovery` section. ## What changes were proposed in this pull request? Like Parquet, ORC data source supports `Partition Discovery`. We had better

[GitHub] spark pull request #19138: [SPARK-21925] Update trigger interval documentati...

2017-09-05 Thread brkyvz
GitHub user brkyvz opened a pull request: https://github.com/apache/spark/pull/19138 [SPARK-21925] Update trigger interval documentation in docs with behavior change in Spark 2.2 Forgot to update docs with behavior change. You can merge this pull request into a Git repository by

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137075768 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -85,6 +87,13 @@ case class

[GitHub] spark issue #19103: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19103 yes user stated he will be opening one for master, but that is quite a bit different due to the credentials stuff moving around so I think this one will have to stay open anyway. But I

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19129 Could you check the change history when we remove the usage of this SQLConf? It sounds like we do not have a test case coverage for this in the past. We did not realize it when removing the

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16774 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16774 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81417/ Test PASSed. ---

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16774 **[Test build #81417 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81417/testReport)** for PR 16774 at commit

[GitHub] spark pull request #19060: [WIP][SQL] Add DataSourceSuite validating data so...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19060#discussion_r137071948 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/DataSourceSuite.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18628: [SPARK-18061][ThriftServer] Add spnego auth support for ...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18628 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81419/ Test PASSed. ---

[GitHub] spark issue #18628: [SPARK-18061][ThriftServer] Add spnego auth support for ...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18628 **[Test build #81419 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81419/testReport)** for PR 18628 at commit

[GitHub] spark issue #18628: [SPARK-18061][ThriftServer] Add spnego auth support for ...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18628 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18991: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18991 Of course, I want to proceed in any part of ORC! As you know, I tried many trials to get a chance to be reviewed. Some PR gets it, but the other ORC PR like #18953 didn't get a feedback

[GitHub] spark issue #19103: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19103 BTW Oozie can also disable the HDFS provider (`spark.yarn.security.credentials.hadoopfs.enabled=false`, I think). But it would be nice if Spark was able to do that by itself is the current UGI does

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18935 **[Test build #81420 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81420/testReport)** for PR 18935 at commit

[GitHub] spark pull request #19110: [SPARK-21027][ML][PYTHON] Added tunable paralleli...

2017-09-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19110#discussion_r137068863 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala --- @@ -297,6 +298,15 @@ final class OneVsRest @Since("1.4.0") (

[GitHub] spark pull request #19110: [SPARK-21027][ML][PYTHON] Added tunable paralleli...

2017-09-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19110#discussion_r137068269 --- Diff: python/pyspark/ml/param/_shared_params_code_gen.py --- @@ -152,6 +152,8 @@ def get$Name(self): ("varianceCol", "column name for

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18935 Just realized the last build is 18 days ago. Triggered a new now. Will merge after passing tests. --- - To unsubscribe, e-mail:

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18935 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18935 LGTM since enableVerboseMetrics is off by default. Merging to master. Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-05 Thread Gauravshah
Github user Gauravshah commented on the issue: https://github.com/apache/spark/pull/16578 just in case someone wants to try: ``` resolvers += "jitpack" at "https://jitpack.io; libraryDependencies += "com.github.VideoAmp" % "spark-public" %

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18935 The change itself looks good, but I'm afraid it's a bit too verbose to add all of those metrics. cc @zsxwing . --- - To

[GitHub] spark issue #18991: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18991 If you want to hold, we can wait for the completion of data source API v2. Otherwise, we can start it now and change it if needed. Conceptually, the test coverage improvement should be

[GitHub] spark pull request #19060: [WIP][SQL] Add DataSourceSuite validating data so...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19060#discussion_r137066894 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/DataSourceSuite.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19134: [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0.8 behin...

2017-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19134 For python I think you just do what kinesis does: ``` # Must be same as the variable and condition defined in KinesisTestUtils.scala kinesis_test_environ_var = "ENABLE_KINESIS_TESTS"

[GitHub] spark issue #18991: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18991 Is the plan aligned with the ongoing [Data Source V2](#19136) ? --- - To unsubscribe, e-mail:

[GitHub] spark issue #18628: [SPARK-18061][ThriftServer] Add spnego auth support for ...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18628 **[Test build #81419 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81419/testReport)** for PR 18628 at commit

[GitHub] spark issue #18991: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18991 To avoid duplicating the efforts, we should have a unified testing framework for covering the PPD of all the sources. Parquet and ORC should be part of it. In the future, when we add the other

[GitHub] spark pull request #19060: [WIP][SQL] Add DataSourceSuite validating data so...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19060#discussion_r137064595 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/DataSourceSuite.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18628: [SPARK-18061][ThriftServer] Add spnego auth support for ...

2017-09-05 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18628 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18628: [SPARK-18061][ThriftServer] Add spnego auth support for ...

2017-09-05 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18628 LGTM pending jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r13706 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,132 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137063137 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,132 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137063179 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,132 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137063033 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,132 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark issue #18991: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18991 To be more clear, I mean the existing Parquet coverage in Apache Spark code base. --- - To unsubscribe, e-mail:

[GitHub] spark issue #18991: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18991 The test case coverage parity between Parquet and ORC should be the criteria for this, right? --- - To unsubscribe,

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137062047 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,145 @@ +/* + * Licensed to

[GitHub] spark issue #18991: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18991 Left a few comments in the another PR: https://github.com/apache/spark/pull/19060#discussion_r137060959. I think it is a right time to improve the test case coverage before turning

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137061444 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,145 @@ +/* + * Licensed to

[GitHub] spark pull request #19060: [WIP][SQL] Add DataSourceSuite validating data so...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19060#discussion_r137060959 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/DataSourceSuite.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137060998 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,81 @@ +/* + *

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137060557 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1509,4 +1509,86 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19099 **[Test build #81418 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81418/testReport)** for PR 19099 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137058514 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1509,4 +1509,86 @@ class SparkSqlAstBuilder(conf: SQLConf)

<    1   2   3   4   5   6   >