[jira] [Updated] (KUDU-2356) Idle WALs can consume significant memory
[ https://issues.apache.org/jira/browse/KUDU-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KUDU-2356: -- Fix Version/s: (was: 1.8.0) > Idle WALs can consume significant memory > > > Key: KUDU-2356 > URL: https://issues.apache.org/jira/browse/KUDU-2356 > Project: Kudu > Issue Type: Improvement > Components: log, tserver >Affects Versions: 1.7.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > Attachments: heap.svg > > > I grabbed a heap sample of a tserver which has been running a write workload > for a little while and found that 750MB of memory is used by faststring > allocations inside WritableLogSegment::WriteEntryBatch. It seems like this is > the 'compress_buf_' member. This buffer always resizes up during a log write > but never shrinks back down, even when the WAL is idle. We should consider > clearing the buffer after each append, or perhaps after a short timeout like > 100ms after a WAL becomes idle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KUDU-2356) Idle WALs can consume significant memory
[ https://issues.apache.org/jira/browse/KUDU-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KUDU-2356: -- Target Version/s: (was: 1.8.0) > Idle WALs can consume significant memory > > > Key: KUDU-2356 > URL: https://issues.apache.org/jira/browse/KUDU-2356 > Project: Kudu > Issue Type: Improvement > Components: log, tserver >Affects Versions: 1.7.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > Attachments: heap.svg > > > I grabbed a heap sample of a tserver which has been running a write workload > for a little while and found that 750MB of memory is used by faststring > allocations inside WritableLogSegment::WriteEntryBatch. It seems like this is > the 'compress_buf_' member. This buffer always resizes up during a log write > but never shrinks back down, even when the WAL is idle. We should consider > clearing the buffer after each append, or perhaps after a short timeout like > 100ms after a WAL becomes idle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
