I have committed, on my branch https://github.com/julianhyde/incubator-calcite/tree/calcite-594, a fix for https://issues.apache.org/jira/browse/CALCITE-594 that I think meets Drill's and Ashutosh's requirements. Please review it before I commit to master.
I have 3 questions: 1. Can we please shorten the distribution type name HASH_DISTRIBUTED to HASH (similarly RANDOM_DISTRIBUTED etc.)? 2. Drill has DrillDistributionTrait.DEFAULT, but I'd rather not create RelDistributions.DEFAULT. (The default will be different for different systems using Calcite.) 3. RelDistributionTraitDef.convert does not create specific physical operators as it does in Drill, because there are no physical implementations of Exchange, just LogicalExchange. Drill will need to create rules to convert LogicalExchange to HashToRandomExchangePrel, etc. Hope that is OK. Julian On Feb 12, 2015, at 1:57 PM, Jacques Nadeau <[email protected]> wrote: Got it. On Thu, Feb 12, 2015 at 11:44 AM, Julian Hyde <[email protected]> wrote: RelMdDistribution (the "Md" that distinguishes it from "RelDistribution" stands for "metadata") is just theory, but it will look a lot like RelMdCollation. https://github.com/apache/incubator-calcite/blob/master/core/src/main/java/org/apache/calcite/rel/metadata/RelMdCollation.java On Wed, Feb 11, 2015 at 6:47 PM, Ashutosh Chauhan <[email protected]> wrote: I think RelDistribution as a member of Exchange (as in Julian proposal) makes more sense as compared to mine at field level. On Wed, Feb 11, 2015 at 6:42 PM, Jacques Nadeau <[email protected]> wrote: Does RelMdDistribution exist or as theory? Definitely inclined to "Exchange" over "Shuffle" as more generic/consistent with Volcano. Note, looking at Ashutosh I wonder a little about the need to have range versus hash at the field information level. I'm sure we can come up with a use case but it seems to overcomplicate. We already have space exploration problems when trying to do distribution subsumption for joins with trait propagation so I would be cautious about adding more dimensions to that attribute.
