Lu created FLINK-30193:
--------------------------

             Summary: could flink has dynamic rebalance  according to the load 
of downstream operators 
                 Key: FLINK-30193
                 URL: https://issues.apache.org/jira/browse/FLINK-30193
             Project: Flink
          Issue Type: New Feature
          Components: API / DataStream
    Affects Versions: 1.15.2
            Reporter: Lu


Different records cause different loads to the downstream and different task 
slot has different compute power. So rebalancing according to the load of 
downstream operators will greatly reduce data skew and improve resource 
utilization.

I thaught the point is how to gain the load of downstream.  I have seen the 

StreamPartitioner , and it dosen't has any context or method to get the load 
information of downstream operators. So ordinary developers cannot easily 
implement such functions.

finally,Could flink provide such a function?

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to