mccheah opened a new pull request #28618:

   ### What changes were proposed in this pull request?
   Adds a `ShuffleOutputTracker` API that can be used for managing shuffle 
metadata on the driver. Accepts map output metadata returned by the map output 
   ### Why are the changes needed?
   Part of the design as discussed in [this 
 and part of the wider effort of SPARK-25299.
   ### Does this PR introduce _any_ user-facing change?
   Enables additional APIs for the shuffle storage plugin tree. Usage will 
become more apparent when the read side of the shuffle plugin tree is 
   ### How was this patch tested?
   We've added a mock implementation of the shuffle plugin tree here, to prove 
that a Spark job using a different implementation of the plugin can use all of 
the plugin points for an alternative shuffle data storage solution.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to