Today, Ning, Huijun, Sanjeev and Maosong discussed about the bp(back-pressure) today, and here are the notes:
*Two major issues of current bp algorithm*: 1. one single point of bp will stop the whole topology 2. when a topology in bp state, the overall throughput for the whole topology can reduce *Some proposals*: For 1. - load shedding -- for example, drop tuples in stmgr. - better bp algo. -- no need to stop the whole topology; can just handle the slowness to particular instances, for example, spill the buffer to the disk For 2. - Rate control on the source(spout) side. So even in bp state, the overall throughput can be higher than normal - Run-time scale-up Let me know if I missing anything. -- With my best Regards ------------------ Fu Maosong Twitter Inc. Mobile: +001-415-244-7520
