attilapiros commented on pull request #34672: URL: https://github.com/apache/spark/pull/34672#issuecomment-1012301827
> I definitely think if we have basically 4 implementations of the shuffle we should make an api that fits and get something done rather then getting caught up in trying to cover all future cases. In this case the most important part is `MapStatus` where filling out the `blockmanger ID` has some consequences and there is no place for arbitrary metadata. > so looking at other things the ExternalBlockStoreClient is not part of the shuffle manager, the intention there would be to keep like the api introduced in this PR? I would keep it as it is. As we already have same implementations of ShuffleManager I would keep the final opened API as close as possible. This way we can easily use those project to test our solutions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
