Hi, I'm curious why EventTimeStatsAccum is not a named accumulator (not to mention SQLMetric) so the event-time watermark could be monitored in web UI?
I've changed the code for EventTimeWatermarkExec physical operator to register EventTimeStatsAccum as a named accumulator and the values are properly propagated back to the driver and the web UI. It seems to be working fine (and it's just a one-day coding). It went fairly easy to have a very initial prototype so I'm wondering why it's not been included? Has this been considered? Should I file a JIRA ticket and send a pull request for review? Please guide as I found it very helpful (and surprisingly easy to implement so I'm worried I'm missing something important). Thanks. Pozdrawiam, Jacek Laskowski ---- https://about.me/JacekLaskowski The Internals of Spark SQL https://bit.ly/spark-sql-internals The Internals of Spark Structured Streaming https://bit.ly/spark-structured-streaming The Internals of Apache Kafka https://bit.ly/apache-kafka-internals Follow me at https://twitter.com/jaceklaskowski
