There is a fundamental reason behind that (unless that has changed). All partitions of a dataset (primary and secondary indexes) share the memory budget for their in-memory components. So they use the same optracker to synchronize their operations.
Sattam On Oct 24, 2017 5:59 AM, "Wail Alkowaileet" <[email protected]> wrote: > Thanks Abdullah! > > On Mon, Oct 23, 2017 at 7:15 PM, abdullah alamoudi <[email protected]> > wrote: > > > Hi Wail, > > There is no fundamental reason why it is one. In fact, it has been on our > > todo for a long time to make it one per partition. > > > > Cheers, > > Abdullah. > > > > > On Oct 23, 2017, at 7:14 PM, Wail Alkowaileet <[email protected]> > > wrote: > > > > > > Dear devs, > > > > > > I have a question regarding the opTracker. Currently, we initialize one > > > opTracker per dataset in every NC. > > > > > > My question is why it's per dataset not per partition ? Is there a > > > transactional constraints for that ? > > > > > > From what I can see that the opTracker can create a lot of contention > in > > > case there're many IO devices. For instance, each insert will call > > > *LSMHarness.getAndEnterComponents()* [1], which > > *synchronize(opTracker). *That > > > means (correct me if I'm wrong), insert is going to serialize the > > > *enterComponent()* part among partitions. > > > > > > [1] > > > https://github.com/apache/asterixdb/blob/master/hyracks- > > fullstack/hyracks/hyracks-storage-am-lsm-common/src/ > > main/java/org/apache/hyracks/storage/am/lsm/common/impls/ > > LSMHarness.java#L86 > > > > > > -- > > > > > > *Regards,* > > > Wail Alkowaileet > > > > > > > -- > > *Regards,* > Wail Alkowaileet >
