HyukjinKwon commented on a change in pull request #24414: [SPARK-22044][SQL]
Add `cost` and `codegen` arguments to `explain`
URL: https://github.com/apache/spark/pull/24414#discussion_r316953726
##########
File path: python/pyspark/sql/dataframe.py
##########
@@ -253,29 +253,55 @@ def printSchema(self):
print(self._jdf.schema().treeString())
@since(1.3)
- def explain(self, extended=False):
- """Prints the (logical and physical) plans to the console for
debugging purpose.
+ def explain(self, extended=False, codegen=False, cost=False):
+ """Prints the plans to the console for debugging purpose.
- :param extended: boolean, default ``False``. If ``False``, prints only
the physical plan.
+ With no (or, all ``False``) options, this prints the physical plan.
The options are mutually
+ exclusive: at most one can be ``True``.
+
+ :param extended: boolean, default ``False``. If ``True``, prints the
logical plans as well
+ as the physical plans.
+ :param codegen: boolean, default ``False``. If ``True``, prints the
generated code for
+ whole-stage codegen.
+ :param cost: boolean, default ``False``. If ``True``, prints the
optimized logical plan
+ with operator costs.
>>> df.explain()
== Physical Plan ==
*(1) Scan ExistingRDD[age#0,name#1]
- >>> df.explain(True)
+ >>> df.explain(extended = True) # doctest: +SKIP
== Parsed Logical Plan ==
- ...
+ LogicalRDD [age#0, name#1], false
+ <BLANKLINE>
== Analyzed Logical Plan ==
- ...
+ age: int, name: string
+ LogicalRDD [age#0, name#1], false
+ <BLANKLINE>
== Optimized Logical Plan ==
- ...
+ LogicalRDD [age#0, name#1], false
+ <BLANKLINE>
== Physical Plan ==
+ *(1) Scan ExistingRDD[age#0,name#1]
+
+ >>> df.explain(codegen = True)
+ Found 1 WholeStageCodegen subtrees.
+ == Subtree 1 / 1 ==
+ *(1) Scan ExistingRDD[age#0,name#1]
+ <BLANKLINE>
Review comment:
I don't think this annotation is needed since we enabled
`NORMALIZE_WHITESPACE` for doctests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
