Hi devs, the docs of org.apache.spark.shuffle.api.ShuffleDataIO read:
An interface for plugging in modules for storing and reading temporary shuffle data. but the API does only provide interface for writing shuffle data: - ShuffleExecutorComponents.createMapOutputWriter - ShuffleExecutorComponents.createSingleFileMapOutputWriter Where is the reading part of ShuffleDataIO? How can I plug in implementations for reading shuffle data via ShuffleDataIO? I am aware of org.apache.spark.shuffle.ShuffleManager. Cheers, Enrico --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
