DjvuLee created SPARK-17500: ------------------------------- Summary: The DiskBytesSpilled metric in ExternalMerger && ExternalGroupBy is wrong Key: SPARK-17500 URL: https://issues.apache.org/jira/browse/SPARK-17500 Project: Spark Issue Type: Bug Components: PySpark Affects Versions: 2.0.0, 1.6.2, 1.6.1, 1.6.0 Reporter: DjvuLee
The DiskBytesSpilled metric in ExternalMerger && ExternalGroupBy is increment by file size in each spill, but we only need the file size at the last time. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org