Re: [DISCUSS] Detection Flink Backpressure

2019-01-08 Thread Piotr Nowojski
t;, and reflecting the ratio as > backpressure if not affect theperformance. > > Best, > Zhijiang > > > ---------- > From:Piotr Nowojski > Send Time:2019年1月4日(星期五) 17:01 > To:dev > Subject:Re: [DISCUSS] Detection

Re: [DISCUSS] Detection Flink Backpressure

2019-01-07 Thread zhijiang
reflecting the ratio as backpressure if not affect theperformance. Best, Zhijiang -- From:Piotr Nowojski Send Time:2019年1月4日(星期五) 17:01 To:dev Subject:Re: [DISCUSS] Detection Flink Backpressure Hi, In that case I think instead o

Re: [DISCUSS] Detection Flink Backpressure

2019-01-04 Thread Piotr Nowojski
at causes the backpressure. >>>>> >>>>> However, currently we think that the InputBufferPoolUsage and >>> OutputBufferPoolUsage requires some modification to be more accurate: >>>>> 1. When there are multiple inputs or outputs, the &g

Re: [DISCUSS] Detection Flink Backpressure

2019-01-03 Thread Ken Krugler
e pre-allocate logic in the >> receiver side, the InputBufferPoolUsage may be 100% even if the data have >> not been received yet [2]. >>>> >>>> We may need to address these problems before adopting the >> InputBufferPoolUsage and OutputBufferPool

Re: [DISCUSS] Detection Flink Backpressure

2019-01-03 Thread Jamie Grier
logic in the >> receiver side, the InputBufferPoolUsage may be 100% even if the data have >> not been received yet [2]. >> >> >> >> We may need to address these problems before adopting the >> InputBufferPoolUsage and OutputBufferPoolUsage as the backpr

Re: [DISCUSS] Detection Flink Backpressure

2019-01-03 Thread Jamie Grier
gt; >> Besides, another similar thought is that we may also add new > InputBufferUsage and OutputBufferUsage metrics to show (number of queued > buffers / number of all buffers) instead. > >> > >> > >> Best, > >>Yun Gao > >> >

Re: [DISCUSS] Detection Flink Backpressure

2019-01-03 Thread Piotr Nowojski
eued >> buffers / number of all buffers) instead. >> >> >> Best, >>Yun Gao >> >> >> [1] https://issues.apache.org/jira/browse/FLINK-10981 >> [2] https://issues.apache.org/jira/browse/FLINK-11082 >> >> >>

Re: [DISCUSS] Detection Flink Backpressure

2019-01-03 Thread peiliping
owse/FLINK-10981 [2] https://issues.apache.org/jira/browse/FLINK-11082 -- From:裴立平 Send Time:2019 Jan. 3 (Thu.) 13:39 To:dev Subject:[DISCUSS] Detection Flink Backpressure Recently I want to optimize the way to

Re: [DISCUSS] Detection Flink Backpressure

2019-01-02 Thread Yun Gao
he.org/jira/browse/FLINK-11082 -- From:裴立平 Send Time:2019 Jan. 3 (Thu.) 13:39 To:dev Subject:[DISCUSS] Detection Flink Backpressure Recently I want to optimize the way to find the positions where the backpressures occured . I rea

[DISCUSS] Detection Flink Backpressure

2019-01-02 Thread 裴立平
Recently I want to optimize the way to find the positions where the backpressures occured . I read some blogs about flink-backpressure and have a rough idea of it . The method which Flink adopted is thread-stack-sample , it's heavy and no-lasting . The positions where backpressures occured are