Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/22674#discussion_r224000145
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -3356,21 +3356,11 @@ class Dataset[T] private[sql](
* user-registered callback functions.
*/
private def withAction[U](name: String, qe: QueryExecution)(action:
SparkPlan => U) = {
- try {
- qe.executedPlan.foreach { plan =>
- plan.resetMetrics()
- }
- val start = System.nanoTime()
- val result = SQLExecution.withNewExecutionId(sparkSession, qe) {
- action(qe.executedPlan)
- }
- val end = System.nanoTime()
- sparkSession.listenerManager.onSuccess(name, qe, end - start)
- result
- } catch {
- case e: Exception =>
- sparkSession.listenerManager.onFailure(name, qe, e)
- throw e
+ qe.executedPlan.foreach { plan =>
--- End diff --
can this throw an exception? Imagine if `df.count()` threw an exception,
and then you run it again.
Won't this be a behavior change in that case?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
