Re: [DISCUSS] Altering storage write rate limiting, and adding read rate limiting

2023-05-14 Thread Chen Luo
Hi Ian, I remembered I introduced this rate limiter before :) For 1 and 2, the original thought was that log writes will be much less compared with LSM writes (consider every record is written to log once but will be flushed once and merged multiple times). By bounding flush/merge rate, it will

Re: [DISCUSS] Altering storage write rate limiting, and adding read rate limiting

2023-05-13 Thread Mike Carey
I've been meaning to reply to this - it's time!  I suspect that #1 is just historical, and happened because the limiter's initial focus was on keeping LSM activity (in particular big merges) from overrunning the other activities in the system.  It seems like this is the right move, IMO.  I

[DISCUSS] Altering storage write rate limiting, and adding read rate limiting

2023-05-03 Thread Ian Maxon
Hi everyone, I've been working on a patch that adds read rate limiting to AsterixDB so that multiple NCs can share the same disk in a more cooperative fashion. Clearly since we already have write rate limiting, I looked at how that is done as a first step. It seemed easy enough to perform the read