zhangyue19921010 commented on PR #6843: URL: https://github.com/apache/hudi/pull/6843#issuecomment-1312423485
Hey @alexeykudinkin Thanks a lot for this PR! Maybe we could have more discuss here if you'd like. Based on our experience, the performance is different between disruptor, BIMQ, and no queue at all in different scenarios. For example, schema is pretty simple, and the bottleneck is lock so disruptor can be helpful. For example in this PR https://github.com/apache/hudi/pull/7174 test(Although it is only a local benchmark, it can also reflect certain problems) different queue do has different advantages. So would u think we can just remove the design of Executor **OR** we can implement different executor including this `no inner message queue`. One more reason I think we can keep this `executor design` is that we can both support multi-writers ex `RealtimeUnmergedRecordReader` and single-writer/single-reader model. Also this no inner queue is my favor although can not support multi-writers, maybe we can use it as default one instead of `BoundedInMemoryQueue ` as next step. Looking forward your replay. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
