Matt Cheah created SPARK-31798:
----------------------------------
Summary: Return map output metadata from shuffle writers
Key: SPARK-31798
URL: https://issues.apache.org/jira/browse/SPARK-31798
Project: Spark
Issue Type: Sub-task
Components: Shuffle
Affects Versions: 3.0.0
Reporter: Matt Cheah
Part of the overall sub-push for shuffle metadata management as proposed in[
this
document|[https://docs.google.com/document/d/1Aj6IyMsbS2sdIfHxLvIbHUNjHIWHTabfknIPoxOrTjk/edit]].
The shuffle plugin system needs a means for storing custom metadata for shuffle
readers to eventually use. This task tracks the API changes required for the
map output writer to return some such metadata back to the driver.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
