Hi Joshua, It is possible, but fragile, as it depends on the internal code that Flink uses to calculate a hash for a key, and the max operator parallelism, etc.
See makeKeyForOperatorIndex <https://github.com/ScaleUnlimited/flink-crawler/blob/master/src/main/java/com/scaleunlimited/flinkcrawler/utils/FlinkUtils.java#L185>, which will generate a String that can be used for partitioning to a specific subtask (operator index) — Ken > On Jun 21, 2019, at 10:15 AM, Joshua Griffith <josh.griff...@gmail.com> wrote: > > Is it possible to use a custom Partitioner to forward messages to a > downstream substask by the subtask's index? I realize that it would not be > safe to rely upon this across job restarts but this does not effect my > particular application. I attempted to return a partition number identical to > the subtask number but this did not work. For example, assigning a message to > partition 1 would send it to subtask 0. > > Thanks, > Josh -------------------------- Ken Krugler +1 530-210-6378 http://www.scaleunlimited.com Custom big data solutions & training Flink, Solr, Hadoop, Cascading & Cassandra
