Hi, (mostly @fabian) why is the re-partition operator special-cased like it is now? You can only do map/filter on partitioned data. Wouldn't it be nice if we had a general re-partition operator. Operators that normally do their own repartitioning would notice that the data is already partitioned and use that. The way it is now, the implementation relies heavily on special-case handling.
In the long run we could even introduce combine and sort as special operators that users could insert themselves. The optimiser would then also insert these before operations when required. This would simplify/generalise things a bit. What do you think? Cheers, Aljoscha
