[GitHub] [spark] maropu commented on a change in pull request #28420: [SPARK-31615][SQL] Pretty string output for sql method of RuntimeReplaceable expressions

2020-05-01 Thread GitBox 


maropu commented on a change in pull request #28420:
URL: https://github.com/apache/spark/pull/28420#discussion_r418420802



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
##
@@ -323,6 +323,20 @@ trait RuntimeReplaceable extends UnaryExpression with 
Unevaluable {
   // two `RuntimeReplaceable` are considered to be semantically equal if their 
"child" expressions
   // are semantically equal.
   override lazy val canonicalized: Expression = child.canonicalized
+
+  override def innerChildren: Seq[Expression] = sys.error("RuntimeReplaceable 
must implement" +
+" innerChildren with the original parameters")
+
+  protected val sqlStrSeparator: String = ", "
+
+  override def sql: String = RuntimeReplaceable.this.prettyName +
+prettyChildren.map(_.sql).mkString("(", sqlStrSeparator, ")")
+
+  protected var prettyChildren: Seq[Expression] = innerChildren

Review comment:
   I feel having the 'pretty' functionality in `RuntimeReplaceable` is a 
bit intrusive. Dropping the idea above is okay, but I think we need a more 
general solution for getting pretty strings outside `RuntimeReplaceable`. WDYT? 
@cloud-fan 

##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
##
@@ -323,6 +323,20 @@ trait RuntimeReplaceable extends UnaryExpression with 
Unevaluable {
   // two `RuntimeReplaceable` are considered to be semantically equal if their 
"child" expressions
   // are semantically equal.
   override lazy val canonicalized: Expression = child.canonicalized
+
+  override def innerChildren: Seq[Expression] = sys.error("RuntimeReplaceable 
must implement" +
+" innerChildren with the original parameters")
+
+  protected val sqlStrSeparator: String = ", "
+
+  override def sql: String = RuntimeReplaceable.this.prettyName +
+prettyChildren.map(_.sql).mkString("(", sqlStrSeparator, ")")
+
+  protected var prettyChildren: Seq[Expression] = innerChildren

Review comment:
   I feel having the 'pretty' functionality in `RuntimeReplaceable` is a 
bit intrusive. Dropping my idea above is okay, but I think we need a more 
general solution for getting pretty strings outside `RuntimeReplaceable`. WDYT? 
@cloud-fan 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #28420: [SPARK-31615][SQL] Pretty string output for sql method of RuntimeReplaceable expressions

2020-04-30 Thread GitBox 


maropu commented on a change in pull request #28420:
URL: https://github.com/apache/spark/pull/28420#discussion_r418348605



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
##
@@ -323,6 +323,20 @@ trait RuntimeReplaceable extends UnaryExpression with 
Unevaluable {
   // two `RuntimeReplaceable` are considered to be semantically equal if their 
"child" expressions
   // are semantically equal.
   override lazy val canonicalized: Expression = child.canonicalized
+
+  override def innerChildren: Seq[Expression] = sys.error("RuntimeReplaceable 
must implement" +
+" innerChildren with the original parameters")
+
+  protected val sqlStrSeparator: String = ", "
+
+  override def sql: String = RuntimeReplaceable.this.prettyName +
+prettyChildren.map(_.sql).mkString("(", sqlStrSeparator, ")")
+
+  protected var prettyChildren: Seq[Expression] = innerChildren

Review comment:
   I think its better not to use `var` where possible, so how about it like 
this?
   ```
 // RuntimeReplaceable
 def newInstance(innerChildren: Seq[Expression]): RuntimeReplaceable
   
 // e.g., ParseToTimestamp
 override def newInstance(innerChildren: Seq[Expression]): ParseToTimestamp 
= {
   innerChildren match {
 case Seq(left) => ParseToTimestamp(left, None, child)
 case Seq(left, format) => ParseToTimestamp(left, Some(format), child)
   }
 }
   ```
   Then, update the `toPrettySQL` in `utils.package`;
   ```
 def toPrettySQL(e: Expression): String = e match {
   case r: RuntimeReplaceable =>
 r.newInstance(r.innerChildren.map(usePrettyExpression)).sql
   case _ => usePrettyExpression(e).sql
 }
   ```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #28420: [SPARK-31615][SQL] Pretty string output for sql method of RuntimeReplaceable expressions

2020-04-30 Thread GitBox 


maropu commented on a change in pull request #28420:
URL: https://github.com/apache/spark/pull/28420#discussion_r418420802



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
##
@@ -323,6 +323,20 @@ trait RuntimeReplaceable extends UnaryExpression with 
Unevaluable {
   // two `RuntimeReplaceable` are considered to be semantically equal if their 
"child" expressions
   // are semantically equal.
   override lazy val canonicalized: Expression = child.canonicalized
+
+  override def innerChildren: Seq[Expression] = sys.error("RuntimeReplaceable 
must implement" +
+" innerChildren with the original parameters")
+
+  protected val sqlStrSeparator: String = ", "
+
+  override def sql: String = RuntimeReplaceable.this.prettyName +
+prettyChildren.map(_.sql).mkString("(", sqlStrSeparator, ")")
+
+  protected var prettyChildren: Seq[Expression] = innerChildren

Review comment:
   I feel having the pretty functionality in `RuntimeReplaceable` is a bit 
intrusive. Dropping the idea above is okay, but I think we need a more general 
solution for getting pretty strings outside `RuntimeReplaceable`. WDYT? 
@cloud-fan 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #28420: [SPARK-31615][SQL] Pretty string output for sql method of RuntimeReplaceable expressions

2020-04-30 Thread GitBox 


maropu commented on a change in pull request #28420:
URL: https://github.com/apache/spark/pull/28420#discussion_r418346518



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
##
@@ -323,6 +323,20 @@ trait RuntimeReplaceable extends UnaryExpression with 
Unevaluable {
   // two `RuntimeReplaceable` are considered to be semantically equal if their 
"child" expressions
   // are semantically equal.
   override lazy val canonicalized: Expression = child.canonicalized
+
+  override def innerChildren: Seq[Expression] = sys.error("RuntimeReplaceable 
must implement" +
+" innerChildren with the original parameters")
+
+  protected val sqlStrSeparator: String = ", "

Review comment:
   We need this variable? It seems only `Extract` uses this.

##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
##
@@ -323,6 +323,20 @@ trait RuntimeReplaceable extends UnaryExpression with 
Unevaluable {
   // two `RuntimeReplaceable` are considered to be semantically equal if their 
"child" expressions
   // are semantically equal.
   override lazy val canonicalized: Expression = child.canonicalized
+
+  override def innerChildren: Seq[Expression] = sys.error("RuntimeReplaceable 
must implement" +

Review comment:
   Since compilers cannot catch the case where one forget to override this, 
how about it like this?
   ```
 // RuntimeReplaceable
 def exprsReplaced: Seq[Expression]
 override def innerChildren: Seq[Expression] = exprsReplaced
   
 // e.g., ParseToTimestamp
 override def exprsReplaced: Seq[Expression] = Seq(Some(left), 
format).flatten
   
   ```

##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
##
@@ -323,6 +323,20 @@ trait RuntimeReplaceable extends UnaryExpression with 
Unevaluable {
   // two `RuntimeReplaceable` are considered to be semantically equal if their 
"child" expressions
   // are semantically equal.
   override lazy val canonicalized: Expression = child.canonicalized
+
+  override def innerChildren: Seq[Expression] = sys.error("RuntimeReplaceable 
must implement" +
+" innerChildren with the original parameters")
+
+  protected val sqlStrSeparator: String = ", "
+
+  override def sql: String = RuntimeReplaceable.this.prettyName +
+prettyChildren.map(_.sql).mkString("(", sqlStrSeparator, ")")
+
+  protected var prettyChildren: Seq[Expression] = innerChildren

Review comment:
   I think its better to user `var` where possible, so how about it like 
this?
   ```
 // RuntimeReplaceable
 def newInstance(innerChildren: Seq[Expression]): RuntimeReplaceable
   
 // e.g., ParseToTimestamp
 override def newInstance(innerChildren: Seq[Expression]): ParseToTimestamp 
= {
   innerChildren match {
 case Seq(left) => ParseToTimestamp(left, None, child)
 case Seq(left, format) => ParseToTimestamp(left, Some(format), child)
   }
 }
   ```
   Then, update the `toPrettySQL` in `utils.package`;
   ```
 def toPrettySQL(e: Expression): String = e match {
   case r: RuntimeReplaceable =>
 r.newInstance(r.innerChildren.map(usePrettyExpression)).sql
   case _ => usePrettyExpression(e).sql
 }
   ```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org