Hi Liran, thanks for an interesting proposal! I believe this proposal will help to compare different implementations. I haven't looked through the proposal closely yet, but will try soon.
On Mon, Aug 31, 2020 at 5:22 AM Liran Funaro <liran.fun...@verizonmedia.com.invalid> wrote: > I will appreciate the community feedback/review on a design proposal I > submitted: https://github.com/apache/druid/issues/10321. > > This design proposal introduces an API that allows users to choose which > in-memory indexing approach to use during ingestion (e.g., > OnheapIncrementalIndex). > > As suggested by Jonathan Wei during one discussion ( > https://github.com/apache/druid/issues/5698#issuecomment-506881097), this > API will allow extensions to offer their own in-memory index > implementations. > > It is motivated by the understanding that different indexing approaches > might have different tradeoffs. Thus, it can be beneficial to allow users > to select the best approach for their workload. A notable example is #10001 > <https://github.com/apache/druid/pull/10001> which shows significant > improvements in throughput and resource utilization for batch ingestion. > > This proposal is accompanied by a work-in-progress PR: > https://github.com/apache/druid/pull/10335 > > Thanks, > > Liran >
