Hi, I'm running Spark 2.4.4. When I execute a simple query "select * from table group by col", I found the SparkListenerTaskEnd event in event log reports all negative time duration for aggregate time total:
{"ID":6,"Name":"aggregate time total (min, med, max)","Update":"2","Value":"-46","Internal":true,"Count Failed Values":true,"Metadata":"sql"} The same thing happens in SparkListenerStageCompleted event: {"ID":6,"Name":"aggregate time total (min, med, max)","Value":"-133","Internal":true,"Count Failed Values":true,"Metadata":"sql"} Then I checked the history server web UI, but the SQL tab displays positive numbers for the HashAggregate operator: aggregate time total (min, med, max): 35 ms (0 ms, 2 ms, 6 ms) I'm wondering is this a bug in Spark 2.4? If not, how does Spark compute the "aggregate time total" from those negative numbers? Thanks