Hi Rui, I did not post the comparison result of async api and streaming, because FileStore's asyncApi's implementation is different from Ozone. FileStore's asyncApi does not have any optimization, such as Ozone implement dataStateMachineCache to improve readStateMachineData, but FileStore's asyncApi need to read disk to read StateMachineData, so FileStore's asyncApi is slower than Ozone, and it's meaningless to compare FileStore's asyncApi and FileStore's streaming. Even though FileStore's streaming is faster than FileStore's asyncApi, we can not say FileStore's streaming is faster than Ozone. So I compare Ozone and FileStore's streaming directly, because the improvement is great, We can make sure FileStore's streaming is going to be a greate improvement on Ozone.
Thanks, runzhiwang Rui Wang <amaliu...@apache.org> 于2020年12月15日周二 上午4:58写道： > The improvements of Ratis streaming is solid based on the doc! Thanks > runzhiwang for your work! > > After walking through the performance doc, I think the comparison seemed to > happen between an Ozone cluster and a filestore? > > If so, my question is, will the Ozone cluster be too heavy to compare with > filestore, which is a simple implementation to write files? Do we need a > comparison for filesstore only between async api and streaming? > > > Hi Liyun, > > Currently the Ratis streaming write feature should be ready to use (though > improvements could keep happening). The read feature is under development. > > -Rui > > On Mon, Dec 14, 2020 at 8:02 AM Lin, Yiqun <yiq...@ebay.com.invalid> > wrote: > > > Hi runzhiwang/ Tsz-Wo, > > > > Seems a very good improvement, I am interested for this. > > As I see the Ratis Streaming feature (RATIS-979) is still in progress > > state. Do you know current status of this feature and if it's ready and > > stable for Ozone to use now? > > > > Thanks, > > Yiqun > > > > On 2020/12/14, 5:40 PM, "Tsz Wo Sze" <szets...@gmail.com> wrote: > > > > External Email > > > > runzhiwang, > > > > Thanks a lot for working hard on Ratis Streaming and running the > > benchmarks! Ratis Streaming is going to be a big improvement in > Ozone. > > > > It is anticipated that Ratis Streaming can > > - improve the performance as shown in the benchmark results; > > - improve the scalability of Ozone since Streaming can reduce the > > number of > > pipelines in Ozone; and > > - reduce the memory requirement in datanodes. > > > > Regards, > > Tsz-Wo > > > > On Mon, Dec 14, 2020 at 4:35 PM runzhiwang <runzhiwan...@gmail.com> > > wrote: > > > > > Hi all, > > > I test the performance of ratis streaming. As the image shows, > Raits > > > streaming can improve Ozone's throughput, memory usage, and > stability > > > greatly. > > > > > > [image: screenshot-1.png] > > > > > > If you are interested with the test and want to reproduce it, > please > > find > > > the details here: > > > > > > > > > https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1TkR4qN1ymm9UY2nRLGlvL_TjI_gTabByDN5uCUEpJiM%2Fedit&data=04%7C01%7Cyiqlin%40ebay.com%7C4a8c74e9a369402a3c8f08d8a014502e%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637435356398333595%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=moz51OUIBm98n%2B%2FYdUtLT4Az0CE%2BgQ%2FnhpU47G5HWOY%3D&reserved=0 > > > > > > > > > Thanks, > > > runzhiwang > > > > > > > >
