Hi Fu, Thanks for the proposals, is it possible to outline the pros and cons that you guys discussed? That would help the community in weighing in on this. Thanks in advance
Sent from my iPhone On Nov 17, 2017, at 4:15 PM, Fu Maosong <[email protected]<mailto:[email protected]>> wrote: Today, Ning, Huijun, Sanjeev and Maosong discussed about the bp(back-pressure) today, and here are the notes: *Two major issues of current bp algorithm*: 1. one single point of bp will stop the whole topology 2. when a topology in bp state, the overall throughput for the whole topology can reduce *Some proposals*: For 1. - load shedding -- for example, drop tuples in stmgr. - better bp algo. -- no need to stop the whole topology; can just handle the slowness to particular instances, for example, spill the buffer to the disk For 2. - Rate control on the source(spout) side. So even in bp state, the overall throughput can be higher than normal - Run-time scale-up Let me know if I missing anything. -- With my best Regards ------------------ Fu Maosong Twitter Inc. Mobile: +001-415-244-7520
