Agree with Xiaoyu, +1 for the merge. Thanks, Li Cheng for working on this feature and taking it to completion.
Best, Sid On Fri, Feb 7, 2020 at 1:38 PM Xiaoyu Yao <[email protected]> wrote: > Thanks for sharing the data. Given the issues raised earlier have been > addressed in the follow up JIRA. I'm +1 for merge. > > Xiaoyu > > On Fri, Feb 7, 2020 at 8:34 AM timmycheng(程力) <[email protected]> > wrote: > > > Hey all, > > > > Just wanna follow up on multi-raft feature progress. I’ve collect some > > feedbacks from Xiaoyu, Anu and Sid ( > > > https://docs.google.com/document/d/1NxCiHhn0u9BqgjuUXB8zxGtny69Qek4yTFe1QqUHiqM/edit > ) > > and address them all in HDDS-2913. Shout out to Xiaoyu, Anu and Sid for > the > > feedbacks and help on resolving them as well. Also would like to know if > > there are other comments and reviews. > > > > We at Tencent has already deployed the multi-raft version to our internal > > production cluster and it’s serving reasonable amount of traffic now. So > > far there are over 16K times of write into our Ozone cluster and I > compare > > with the single-raft version’s performance. Both are measured in similar > > pattern of traffic on daily basis. > > > > Write finishes in: > > > > Single raft > > > > Multi raft > > > > > 3s > > > > 0.009% > > > > 0.006% > > > > 2s ~ 3s > > > > 27.4% > > > > 1.46% > > > > 1s ~ 2s > > > > 1.64% > > > > 0.07% > > > > 0.2s ~ 1s > > > > 2.7% > > > > 0.53% > > > > < 0.2s > > > > 68.2% > > > > 97.9% > > > > > > Our internal customer writes to ozone every day and there are schedules > > jobs as well as on-demand jobs. Size could be from KB to GB every write, > > but every daes y’s traffic share the same pattern. Therefore, we see that > > multi-raft version makes ~98% of write finish within 0.2s, which is 20% > > more than what single-raft version can do. At the same time, those who > > finishes from 2s to 3s reduces from 27.4% to 1.46%. Multi-raft has made > our > > internal cluster more stable and the latency fluctuates way less, which > is > > pretty helpful. > > > > Cheers, > > Li > > > > 发件人: "timmycheng(程力)" <[email protected]> > > 日期: 2020年1月13日 星期一 下午4:24 > > 收件人: "[email protected]" <[email protected]> > > 主题: [DISCUSS] - Merge Multi-Raft Support - HDDS-1564 > > > > Hey all, > > > > Happy to present the multi-raft feature to ozone community ( > > https://issues.apache.org/jira/browse/HDDS-1564). This feature is to > > allow every datanode to host more than 1 pipeline based on user config to > > better utilize every datanode’s disks IO. > > > > All dev work have been done and I’ve conducted performance tests in > > different scenarios. Based on my testing, multi-raft ozone cluster can > help > > to make writing latency as low as 1/3 of single-raft’s one. Please check > > the attachment in the above JIRA for test brief and more details as well > as > > the code patch. > > > > I would like to use this thread to discuss about this feature and it’s > > merge back to master. > > > > -Li > > >
