andygrove commented on code in PR #255:
URL:
https://github.com/apache/arrow-datafusion-comet/pull/255#discussion_r1570671008
##########
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##########
@@ -833,4 +949,36 @@ object CometSparkSessionExtensions extends Logging {
ByteUnit.MiB.toBytes(shuffleMemorySize)
}
}
+
+ /**
+ * Attaches explain information to a TreeNode, rolling up the corresponding
information tags
+ * from any child nodes
+ *
+ * @param node
+ * The node to attach the explain information to. Typically a SparkPlan
+ * @param info
+ * Information text. Optional, may be null or empty. If not provided, then
only information
+ * from child nodes will be included.
+ * @param exprs
+ * Child nodes. Information attached in these nodes will be be included in
the information
+ * attached to @node
+ * @tparam T
+ * The type of the TreeNode. Typically SparkPlan, AggregateExpression, or
Expression
+ * @return
+ * The node with information (if any) attached
+ */
+ def withInfo[T <: TreeNode[_]](node: T, info: String, exprs: T*): T = {
Review Comment:
I love the idea of storing the metadata on the plan using Spark's tagging
mechanism (we did the same thing in Spark RAPIDS).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
