[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20226 sure, will do, too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20226 @maropu Could you take this over? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94786/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #94786 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94786/testReport)** for PR 20226 at commit [`1facc05`](https://github.com/apache/spark/commit/1facc0554aae0829a19bbb7607b25ff7eda4ef8d). * This patch **fails due to an unknown error code, -9**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #94786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94786/testReport)** for PR 20226 at commit [`1facc05`](https://github.com/apache/spark/commit/1facc0554aae0829a19bbb7607b25ff7eda4ef8d). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20226 After went through the changes here, I think we only need to update 2 nodes to include table name in `nodeName`: hive table scan and in-memory table scan. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20226 After more thoughts, I feel it's reasonable to include table information in the node name. The UI displays `nodeName` in the plan graph, and displays `simpleString` in a pop-up window when users hover over the plan graph. Since table information is pretty important, it makes sense to display it in the plan graph instead of the pop-up window. Data Source table scan does follow this rule   +1 on this PR to fix the hive table scan, or any other scan nodes that don't follow this rule. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20226 By default `simpleString` is defined as `s"$nodeName $argString".trim`, if we overwrite `nodeName` in some node, we should also overwrite `argString`, otherwise we may have duplicated information in `simpleString`, which is used with `explain`. Can we just change the UI code to put `plan.simpleString` in the plan graph? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20226 It sounds like we still need to fix a test in PySpark. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86850/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86850/testReport)** for PR 20226 at commit [`1facc05`](https://github.com/apache/spark/commit/1facc0554aae0829a19bbb7607b25ff7eda4ef8d). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/406/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86850/testReport)** for PR 20226 at commit [`1facc05`](https://github.com/apache/spark/commit/1facc0554aae0829a19bbb7607b25ff7eda4ef8d). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/220/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86407/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86407 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86407/testReport)** for PR 20226 at commit [`bf90ac7`](https://github.com/apache/spark/commit/bf90ac713f1ea909572486b136c44f9e4badc50c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86407/testReport)** for PR 20226 at commit [`bf90ac7`](https://github.com/apache/spark/commit/bf90ac713f1ea909572486b136c44f9e4badc50c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/54/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86353/testReport)** for PR 20226 at commit [`0c0aa94`](https://github.com/apache/spark/commit/0c0aa94fe3cedc7ab63552a506b96549e75f2882). * This patch **fails from timeout after a configured wait of \`300m\`**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86353/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86353/testReport)** for PR 20226 at commit [`0c0aa94`](https://github.com/apache/spark/commit/0c0aa94fe3cedc7ab63552a506b96549e75f2882). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/11/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86299/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86299/testReport)** for PR 20226 at commit [`65ec7a2`](https://github.com/apache/spark/commit/65ec7a2ab28b1277bf1e53b889ae8f215d42aacf). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86300/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86300/testReport)** for PR 20226 at commit [`65ec7a2`](https://github.com/apache/spark/commit/65ec7a2ab28b1277bf1e53b889ae8f215d42aacf). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86300/testReport)** for PR 20226 at commit [`65ec7a2`](https://github.com/apache/spark/commit/65ec7a2ab28b1277bf1e53b889ae8f215d42aacf). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86299/testReport)** for PR 20226 at commit [`65ec7a2`](https://github.com/apache/spark/commit/65ec7a2ab28b1277bf1e53b889ae8f215d42aacf). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/20226 Jenkins retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20226 > build/sbt -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver assembly "hive-thriftserver/test-only org.apache.spark.sql.hive.thriftserver.HiveThriftBinaryServerSuite" Try this? It worked for me before. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/20226 The test failure does look legit to me. I have been not able to repro it on my laptop. Intellij doesn't treat it as a test case. Command-line does recognize it as test case but hits runtime failure with jar mismatch. I am using this to run the test: ``` build/mvn -Phive-thriftserver -Dtest=none -DwildcardSuites=org.apache.spark.sql.hive.thriftserver.HiveThriftBinaryServerSuite test ``` Is there special setup needed to run these tests ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20226 Overall, the fixes looks good to me. We just need to resolve the test cases. Thanks for improving it! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86044/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86044/testReport)** for PR 20226 at commit [`f16b73b`](https://github.com/apache/spark/commit/f16b73b9c9100599942c92c2f10768ce921ffb8a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86044/testReport)** for PR 20226 at commit [`f16b73b`](https://github.com/apache/spark/commit/f16b73b9c9100599942c92c2f10768ce921ffb8a). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
