Stefan De Smit created CRUNCH-598:
-------------------------------------
Summary: scaleFactor for JoinStrategy
Key: CRUNCH-598
URL: https://issues.apache.org/jira/browse/CRUNCH-598
Project: Crunch
Issue Type: Improvement
Reporter: Stefan De Smit
Priority: Minor
the scaleFactor method has a big influence on planner.
For joins, there currently isn't a clean way to set this, while it often is
required, as a join can have a big multiply factor.
for the DefaultJoinStrategy, it's possible to add a custom JoinFn with proper
scaleFactor, or just extend the default InnerJoinFn with a scaleFactor.
For the ShardedJoinStrategy, this isn't possible, while it often is needed more
(as ShardedJoin is especially handy for 1 to really many).
For the default ConstantShardingStrategy, it might make sense to use the
numShards also as scalingFactor for left side. as that's kind of what happens:
emit every left entry numShards times.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
