Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19788
@yucai I'm thinking of the necessity to add this new configuration
`spark.shuffle.continuousFetch` like you mentioned above. This PR you proposed
is actually a superset of previous way, it is compatible with original shuffle
way if length = 1. The configuration here is only used to keep compatible for
external shuffle service, I think it is not so intuitive and user may confused
whether this should be enabled or not (since since conf is
functionality-orientiated). Besides do we need to guarantee forward compatible,
also is there a transparent way to automatically switch between two shuffles
without configuration?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
