I will appreciate the community feedback/review on a design proposal I submitted: https://github.com/apache/druid/issues/10321.
This design proposal introduces an API that allows users to choose which in-memory indexing approach to use during ingestion (e.g., OnheapIncrementalIndex). As suggested by Jonathan Wei during one discussion ( https://github.com/apache/druid/issues/5698#issuecomment-506881097), this API will allow extensions to offer their own in-memory index implementations. It is motivated by the understanding that different indexing approaches might have different tradeoffs. Thus, it can be beneficial to allow users to select the best approach for their workload. A notable example is #10001 <https://github.com/apache/druid/pull/10001> which shows significant improvements in throughput and resource utilization for batch ingestion. This proposal is accompanied by a work-in-progress PR: https://github.com/apache/druid/pull/10335 Thanks, Liran
