[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20484 @dongjoon-hyun What we need to add in the migration guide is to help the existing ORC users migrate their existing queries that are built on ORC tables. The Hive ORC confs can be set in

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87128 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87128/testReport)** for PR 20474 at commit

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87126/ Test FAILed. ---

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20490 **[Test build #87126 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87126/testReport)** for PR 20490 at commit

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20513 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20513 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87118/ Test PASSed. ---

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87127 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87127/testReport)** for PR 20474 at commit

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20513 **[Test build #87118 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87118/testReport)** for PR 20513 at commit

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20490 **[Test build #87126 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87126/testReport)** for PR 20490 at commit

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/640/

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20226: [SPARK-23034][SQL] Override `nodeName` for all *S...

2018-02-06 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/20226#discussion_r166426610 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -86,6 +86,9 @@ case class RowDataSourceScanExec(

[GitHub] spark pull request #20226: [SPARK-23034][SQL] Override `nodeName` for all *S...

2018-02-06 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/20226#discussion_r166426184 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala --- @@ -169,10 +171,12 @@ case class LogicalRDD( case class

[GitHub] spark pull request #20226: [SPARK-23034][SQL] Override `nodeName` for all *S...

2018-02-06 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/20226#discussion_r166425954 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala --- @@ -103,6 +103,8 @@ case class ExternalRDDScanExec[T](

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166425787 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark pull request #20519: [Spark-23240][python] Don't let python site custo...

2018-02-06 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/20519#discussion_r166422848 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -181,17 +181,33 @@ private[spark] class

[GitHub] spark pull request #20519: [Spark-23240][python] Don't let python site custo...

2018-02-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20519#discussion_r166422566 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -181,17 +181,33 @@ private[spark] class

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87125/ Test FAILed. ---

[GitHub] spark pull request #20519: [Spark-23240][python] Don't let python site custo...

2018-02-06 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/20519#discussion_r166422170 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -181,17 +181,33 @@ private[spark] class

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20490 **[Test build #87125 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87125/testReport)** for PR 20490 at commit

[GitHub] spark issue #20517: [SPARK-23342][SQL][TEST] Add ORC configuration tests for...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20517 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87117/ Test PASSed. ---

[GitHub] spark issue #20517: [SPARK-23342][SQL][TEST] Add ORC configuration tests for...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20517 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-06 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20507 I made https://issues.apache.org/jira/browse/ARROW-2101 to track the issue in Arrow --- - To unsubscribe, e-mail:

[GitHub] spark issue #20517: [SPARK-23342][SQL][TEST] Add ORC configuration tests for...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20517 **[Test build #87117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87117/testReport)** for PR 20517 at commit

[GitHub] spark pull request #20519: [Spark-23240][python] Don't let python site custo...

2018-02-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20519#discussion_r166421329 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -181,17 +181,33 @@ private[spark] class

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166421002 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark pull request #20519: [Spark-23240][python] Don't let python site custo...

2018-02-06 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/20519#discussion_r166420442 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -181,17 +181,33 @@ private[spark] class

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/639/

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20490: [SPARK-23323][SQL]: Support commit coordinator fo...

2018-02-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20490#discussion_r166418424 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala --- @@ -117,20 +118,43 @@ object

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20495 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/638/

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20490 **[Test build #87125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87125/testReport)** for PR 20490 at commit

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20521 @cloud-fan, the tests failed but I believe it was unrelated to this change. As you noticed, the `table` parameter that was removed wasn't actually used anywhere. ---

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20495 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87119/ Test FAILed. ---

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20521 **[Test build #87119 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87119/testReport)** for PR 20521 at commit

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20495 **[Test build #87124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87124/testReport)** for PR 20495 at commit

[GitHub] spark pull request #20519: [Spark-23240][python] Don't let python site custo...

2018-02-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20519#discussion_r166415978 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -181,17 +181,33 @@ private[spark] class

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/637/

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87115/ Test PASSed. ---

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166414290 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87115/testReport)** for PR 20474 at commit

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20387 **[Test build #87123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87123/testReport)** for PR 20387 at commit

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166413015 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20387 Will do. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20387 I think there will be a lot of discussion about data source v2 table support. For now how about we remove the table/path stuff and get this PR in ASAP? ---

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166410690 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20387 > Can you give a use case about this? We've added a resolution rule from `UnresolvedRelation` to `DataSourceV2Relation` that uses our implementation. `UnresolvedRelation` needs to pass its

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r166409901 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -17,7 +17,7 @@ package org.apache.spark.ui -import

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r166409000 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +51,31 @@ private[v1] class

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r166409259 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +51,31 @@ private[v1] class

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r166409473 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +51,31 @@ private[v1] class

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166409729 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20387 > The reader is to be created and configured by the relation, then the relation needs to be able to set the table, path, and other properties. This adds necessary data to the relation that is no

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #87122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87122/testReport)** for PR 20445 at commit

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/636/

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166407366 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark issue #20485: [SPARK-23315][SQL] failed to get output from canonicaliz...

2018-02-06 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20485 @gatorsmile @rdblue please review and LGTM this. This will unblock my PR - https://github.com/apache/spark/pull/20445 --- - To

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-06 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20445 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20387 The reader is to be created and configured by the relation, then the relation needs to be able to set the table, path, and other properties. This adds necessary data to the relation that is no

[GitHub] spark pull request #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style chec...

2018-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20514 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166403941 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87121 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87121/testReport)** for PR 20474 at commit

[GitHub] spark issue #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issue...

2018-02-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20514 Merging to master / 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87120/testReport)** for PR 20474 at commit

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20474 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87114/ Test FAILed. ---

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20387 This PR does 3 things: 1. make `DataSourceV2Relation` immutable. This extends the constructor of `DataSourceV2Relation` to include pushed filters and pruned columns. 2. carry some standard

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87114/testReport)** for PR 20474 at commit

[GitHub] spark issue #20244: [SPARK-23053][CORE] taskBinarySerialization and task par...

2018-02-06 Thread ivoson
Github user ivoson commented on the issue: https://github.com/apache/spark/pull/20244 @squito Hi Rashid, thanks for your review and advise. The PR description and JIRA have been updated , and also put the stack trace on the JIRA. The last commit addresses the comments you left.

[GitHub] spark pull request #20490: [SPARK-23323][SQL]: Support commit coordinator fo...

2018-02-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20490#discussion_r166398432 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala --- @@ -117,20 +118,43 @@ object

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20521 Overall this looks fine to me as a fix. Thanks for noting that we want to move away from commands-within-commands over the next few releases. ---

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166396780 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark pull request #20490: [SPARK-23323][SQL]: Support commit coordinator fo...

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20490#discussion_r166395788 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala --- @@ -117,20 +118,43 @@ object

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r166394747 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,151 @@ package

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r166392277 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,151 @@ package

[GitHub] spark issue #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issue...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20514 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87110/ Test PASSed. ---

[GitHub] spark issue #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issue...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20514 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issue...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20514 **[Test build #87110 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87110/testReport)** for PR 20514 at commit

[GitHub] spark pull request #20244: [SPARK-23053][CORE] taskBinarySerialization and t...

2018-02-06 Thread ivoson
Github user ivoson commented on a diff in the pull request: https://github.com/apache/spark/pull/20244#discussion_r166387840 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1016,15 +1016,23 @@ class DAGScheduler( // might modify state

[GitHub] spark pull request #20244: [SPARK-23053][CORE] taskBinarySerialization and t...

2018-02-06 Thread ivoson
Github user ivoson commented on a diff in the pull request: https://github.com/apache/spark/pull/20244#discussion_r166387716 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #20244: [SPARK-23053][CORE] taskBinarySerialization and t...

2018-02-06 Thread ivoson
Github user ivoson commented on a diff in the pull request: https://github.com/apache/spark/pull/20244#discussion_r166387660 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166386827 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -482,9 +484,24 @@ case class DataSource( /**

[GitHub] spark pull request #20244: [SPARK-23053][CORE] taskBinarySerialization and t...

2018-02-06 Thread ivoson
Github user ivoson commented on a diff in the pull request: https://github.com/apache/spark/pull/20244#discussion_r166386639 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r166386661 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,151 @@ package

[GitHub] spark pull request #20244: [SPARK-23053][CORE] taskBinarySerialization and t...

2018-02-06 Thread ivoson
Github user ivoson commented on a diff in the pull request: https://github.com/apache/spark/pull/20244#discussion_r166386624 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #20244: [SPARK-23053][CORE] taskBinarySerialization and t...

2018-02-06 Thread ivoson
Github user ivoson commented on a diff in the pull request: https://github.com/apache/spark/pull/20244#discussion_r166386259 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/635/

[GitHub] spark pull request #20244: [SPARK-23053][CORE] taskBinarySerialization and t...

2018-02-06 Thread ivoson
Github user ivoson commented on a diff in the pull request: https://github.com/apache/spark/pull/20244#discussion_r166385409 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #20244: [SPARK-23053][CORE] taskBinarySerialization and t...

2018-02-06 Thread ivoson
Github user ivoson commented on a diff in the pull request: https://github.com/apache/spark/pull/20244#discussion_r166385020 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2424,115 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20521 **[Test build #87119 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87119/testReport)** for PR 20521 at commit

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20521 cc @sameeragarwal @rdblue @gatorsmile @wangyum --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

<    1   2   3   4   5   6   >