[GitHub] [spark] beliefer commented on a diff in pull request #40467: [SPARK-42584][CONNECT] Improve output of `Column.explain`
beliefer commented on code in PR #40467:
URL: https://github.com/apache/spark/pull/40467#discussion_r1144208941
##
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala:
##
@@ -1211,13 +1211,11 @@ class Column private[sql] (private[sql] val expr:
proto.Expression) extends Logg
* @group df_ops
* @since 3.4.0
*/
- def explain(extended: Boolean): Unit = {
+ def explain(extended: Boolean)(implicit spark: SparkSession): Unit = {
Review Comment:
I understand what you mean roughly. We don't need follow SQL, but the output
reflects the hierarchy of the tree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #40467: [SPARK-42584][CONNECT] Improve output of `Column.explain`
beliefer commented on code in PR #40467:
URL: https://github.com/apache/spark/pull/40467#discussion_r1144182236
##
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala:
##
@@ -1211,13 +1211,11 @@ class Column private[sql] (private[sql] val expr:
proto.Expression) extends Logg
* @group df_ops
* @since 3.4.0
*/
- def explain(extended: Boolean): Unit = {
+ def explain(extended: Boolean)(implicit spark: SparkSession): Unit = {
Review Comment:
@hvanhovell Do you mean to construct local catalyst expression tree at
connect client side ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #40467: [SPARK-42584][CONNECT] Improve output of `Column.explain`
beliefer commented on code in PR #40467:
URL: https://github.com/apache/spark/pull/40467#discussion_r1144175741
##
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala:
##
@@ -1211,13 +1211,11 @@ class Column private[sql] (private[sql] val expr:
proto.Expression) extends Logg
* @group df_ops
* @since 3.4.0
*/
- def explain(extended: Boolean): Unit = {
+ def explain(extended: Boolean)(implicit spark: SparkSession): Unit = {
Review Comment:
@hvanhovell If so, the output of `Column.explain` can't keep consistent with
Spark SQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #40467: [SPARK-42584][CONNECT] Improve output of `Column.explain`
beliefer commented on code in PR #40467:
URL: https://github.com/apache/spark/pull/40467#discussion_r1142827066
##
connector/connect/common/src/main/protobuf/spark/connect/base.proto:
##
@@ -163,6 +164,14 @@ message AnalyzePlanRequest {
// (Required) The logical plan to get a hashCode.
Plan plan = 1;
}
+
+ // Explains the expression based on extended is true or not.
+ message ExplainExpression {
+// (Required) The expression to be analyzed.
+Expression expr = 1;
+
+bool extended = 2;
Review Comment:
Thank you! I forgot it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #40467: [SPARK-42584][CONNECT] Improve output of `Column.explain`
beliefer commented on code in PR #40467:
URL: https://github.com/apache/spark/pull/40467#discussion_r1142826734
##
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala:
##
@@ -1211,13 +1211,11 @@ class Column private[sql] (private[sql] val expr:
proto.Expression) extends Logg
* @group df_ops
* @since 3.4.0
*/
- def explain(extended: Boolean): Unit = {
+ def explain(extended: Boolean)(implicit spark: SparkSession): Unit = {
Review Comment:
Good question. It seems we must send the msg to server side. So we require
`SparkSession`.
But it's a question how to get `SparkSession` in connect's `Column` ? or we
define the SparkSession in `Column` directly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
