Matt Cheah created SPARK-31801: ---------------------------------- Summary: Register shuffle map output metadata with a shuffle output tracker Key: SPARK-31801 URL: https://issues.apache.org/jira/browse/SPARK-31801 Project: Spark Issue Type: Sub-task Components: Shuffle Affects Versions: 3.1.0 Reporter: Matt Cheah
Part of the design as discussed inĀ [this document|https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit#]. Establish a {{ShuffleOutputTracker}} API that resides on the driver, and handle accepting map output metadata returned by the map output writers and send them to the output tracker module accordingly. Requires https://issues.apache.org/jira/browse/SPARK-31798. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org