Hi All, Last week, we ran some performance tests in which we were running count(*) queries while doing high modification load. We noticed that whenever we run the scan query, the modification rate drops to 0 op/s.
We did some investigation, found two issues causing this and thought that this was worth sharing: 1) Memory component gets full, is flushed but can't be recycled because of the search operation is still inside the component and it only exits at the end of the operation. 2) Read latches on the memory component not being released and the memory component search cursor is not advancing. This happens if the key where the cursor is at in the memory component is larger than other cursors in other immutable components. We addressed these issues for the LSMBTree in a proposed change https://asterix-gerrit.ics.uci.edu/#/c/2166/ <https://asterix-gerrit.ics.uci.edu/#/c/2166/> but not yet for other indexes. - The proposed solution for case (1) is to poll memory components states every n records during the search operation. If a memory component was found to have been flushed, its cursor is moved to the corresponding disk component allowing the memory component to be recycled. - The proposed solution for case (2) is to check memory component cursor every n records. If the cursor has not advanced and the component has writers, then the latches over the leaf page are released, and the cursor re-do the operation entering from the component tree root. Cheers, Abdullah.
