Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/22674#discussion_r224755348
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala ---
@@ -39,7 +39,22 @@ case class SparkListenerSQLExecutionStart(
@DeveloperApi
case class SparkListenerSQLExecutionEnd(executionId: Long, time: Long)
- extends SparkListenerEvent
+ extends SparkListenerEvent {
+
+ // The name of the execution, e.g. `df.collect` will trigger a SQL
execution with name "collect".
+ @JsonIgnore private[sql] var executionName: Option[String] = None
+
+ // The following 3 fields are only accessed when `executionName` is
defined.
+
+ // The duration of the SQL execution, in nanoseconds.
+ @JsonIgnore private[sql] var duration: Long = 0L
--- End diff --
did you verify that the JsonIgnore annotation actually works? For some
reason, I actually needed to annotate the class as
```scala
@JsonIgnoreProperties(Array("a", b", "c"))
class SomeClass {
@JsonProperty("a") val a: ...
@JsonProperty("b") val a: ...
}
```
the reason being Json4s understands that API better. I believe we use
Json4s for all of these events
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
