Github user StephanEwen commented on the pull request:
https://github.com/apache/incubator-flink/pull/128#issuecomment-56957293
I think this variant is a lot simpler and keeps the API clean. Calling
partitioning is something that a user should use, when he wants explicit
control, and I think it is okay if that leads to situations where the
partitioning turns out redundant (we could also fix this later with optimizer
rules, but we cannot fix the API once public).
When calling explicit partitioning before a join, the optimizer will reuse
the partitioning. When calling it before reduce, only the combiner will be
dropped.
Regarding parallelism: Calling rebalancing when the DOP changes is
redundant, because the DOP change will cause a rebalance by itself, but it does
not hurt. Calling partition() and then change the DOP voids the partitioning,
that is true. I would accept that for now and rather throw an error now (e.g.
in the optimizer "partition operator followed by DOP change", and later fix it
with an optimizer rule or so)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
