[
https://issues.apache.org/jira/browse/ASTERIXDB-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17787508#comment-17787508
]
ASF subversion and git services commented on ASTERIXDB-3314:
------------------------------------------------------------
Commit ea4ae3a1a8df45d6f575400ce7da2d2099eaa416 in asterixdb's branch
refs/heads/master from Wail Alkowaileet
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=ea4ae3a1a8 ]
[ASTERIXDB-3314][STO] Reduce buffer cache pressure on columnar
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Make max merging components count to 4 in columnar datasets
- Fix not unpinning columnar filter pages
- Allocate initial 32KB buffers for columnar writers
Change-Id: I809109b232bc5a5db0c47a52cb98c838ff55e27f
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17965
Integration-Tests: Jenkins <[email protected]>
Tested-by: Jenkins <[email protected]>
Reviewed-by: Murtadha Hubail <[email protected]>
> Reduce buffer cache pressure when operating against columnar datasets
> ---------------------------------------------------------------------
>
> Key: ASTERIXDB-3314
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-3314
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: STO - Storage
> Affects Versions: 0.9.9
> Reporter: Wail Y. Alkowaileet
> Assignee: Wail Y. Alkowaileet
> Priority: Major
> Labels: triaged
> Fix For: 0.9.9
>
>
> When ingesting columnar datasets, the bulkloader and its columnar writers
> rely on the buffer cache to provide the necessary buffers for writing.
> Specifically, the buffer cache provides temporary buffers to the columnar
> writers. However, not all columns require a full 128KB buffers (e.g., sparse
> columns). Instead of using precious buffer cache pages for such columns, we
> should allow column writers to allocate smaller buffers to be used initially.
> In case they needed more space, they can ask the buffer cache for more. This
> approach relieve the pressure against the buffer cache.
>
> Another issue/bug (related to the buffer cache), columnar filters are not
> unpinning their pages when they're done – holding those pages indefinitely
> until the next restart.
>
> A third issue is the merge policy. Currently, we allow merging upto 10
> components in one shot. Reducing this number could also alleviate the
> pressure on the buffer cache.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
