Thanks for reviewing!
Sure thing to explain a bit more:
Firstly, in order to support custom operator, we need to pass in the operator
object as well as specifying the grouping strategy.
Currently, the implementation of Streamlet classes are like:
bldr.setBolt(getName(), new FlatMapOperator<R, T>(flatMapFn),
getNumPartitions()).shuffleGrouping(parent.getName());
You can say the grouping is a function call with the parent name. It is ok for
predefined operators because the grouping is likely to be decided, but this is
tricky for custom operators (and there might be new operators that allow users
to specify grouping strategy in future).
One way is to pass an enum, but this is not flexible and there will be a
switch(grouping) {....} to maintain. So I check the code and it seems fairly
stranghtforward to refactor the grouping strategies into their own classes and
here it is~ After this change, user can create any the grouping object and use
it in streamlet.
[ Full content available at:
https://github.com/apache/incubator-heron/pull/3040 ]
This message was relayed via gitbox.apache.org for [email protected]
