Github user vasia commented on the pull request:
https://github.com/apache/flink/pull/1124#issuecomment-139959564
Hi @andralungu,
I was under the impression that we never really reached consensus in the
mailing list thread regarding this addition.
I definitely think we need to handle skewed graphs and you've done great
work, but I wouldn't add this to Gelly at its current state.
- This is a very recent method that has not been tested thoroughly (apart
from the experiments in your thesis work). Its benefits and overheads are not
yet well understood.
- The API certainly needs rethinking. This should be a transparent method
that would be easy to activate with a flag/option. Right now, it seems to me
that it's too complicated to use and can easily allow erroneous implementations.
For now I would suggest that we keep this in your personal repository and
we link to it from the Gelly documentation as additional/experimental feature.
After we have better understanding of the technique and we have thought of a
nicer API, we can reconsider adding it. What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
