Hi Devs, I'm noticing a behavior during the ingestion is that it's getting slower by time. I know that is an expected behavior in LSM-indexes. But what I'm seeing is that I can notice the drop in ingestion rate roughly after having 10 components (around ~13 GB). That's what I'm not sure if it's expected?
I tried multiple setups (increasing Memory component size + max-mergable-component-size). All of which delayed the problem but not solved it. The only part I've never changed is the bloom-filter false-positive rate (1%). Which I want to investigate next. So.. What I want to suggest is that when the primary key is auto-generated, why AsterixDB looks for duplicates? it seems a wasteful operation to me. Also, can we give the user the ability to tell the index that all keys are unique ? I know I should not trust the user .. but in certain cases, probably the user is certain that the key is unique. Or a more elegant solution can shine in the end :-) -- *Regards,* Wail Alkowaileet
