[
https://issues.apache.org/jira/browse/IGNITE-13831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254588#comment-17254588
]
shivakumar edited comment on IGNITE-13831 at 12/24/20, 6:10 PM:
----------------------------------------------------------------
Hi [~ktkale...@gridgain.com] [~ilyak] [~akalashnikov]
With latest master branch code I'm facing WAL issue, when there is a continuous
stream of data (sql inserts) to sql table then WAL is growing without any limit.
This happens when there is a stream of data and at that time if I connect to
visor then WAL usage will continuously grow (I have seen WAL growing more than
300GB and disk runs out of space )
The same test was working fine without any issue with December 13th master
branch code, from that time this is the major change(also IGNITE13847) related
to WAL segment removal, could you please check if there is any issue.
These are the WAL related parameters set as part of ignite configurations;
<property name="writeThrottlingEnabled" value="true"/>
<property name="checkpointFrequency" value="#\{1 * 60 * 1000}"/>
<property name="walSegmentSize" value="#\{1L * 1024 * 1024 * 1024}"/>
<property name="maxWalArchiveSize" value="#\{10L * 1024 * 1024 * 1024}"/>
<property name="storagePath" value="/opt/ignite/persistence/"/>
<property name="walPath" value="/opt/ignite/wal/"/>
<property name="walArchivePath" value="/opt/ignite/wal/"/>
<property name="walMode" value="FSYNC"/>
Regards,
Shiva
was (Author: shm):
Hi
With latest master branch code I'm facing WAL issue, when there is a continuous
stream of data (sql inserts) to sql table then WAL is growing without any limit.
This happens when there is a stream of data and at that time if I connect to
visor then WAL usage will continuously grow (I have seen WAL growing more than
300GB and disk runs out of space )
The same test was working fine without any issue with December 13th master
branch code, from that time this is the major change(also
[IGNITE13847|https://issues.apache.org/jira/browse/IGNITE-13847]) related to
WAL segment removal, could you please check if there is any issue.
These are the WAL related parameters set as part of ignite configurations;
<property name="writeThrottlingEnabled" value="true"/>
<property name="checkpointFrequency" value="#\{1 * 60 * 1000}"/>
<property name="walSegmentSize" value="#\{1L * 1024 * 1024 * 1024}"/>
<property name="maxWalArchiveSize" value="#\{10L * 1024 * 1024 * 1024}"/>
<property name="storagePath" value="/opt/ignite/persistence/"/>
<property name="walPath" value="/opt/ignite/wal/"/>
<property name="walArchivePath" value="/opt/ignite/wal/"/>
<property name="walMode" value="FSYNC"/>
Regards,
Shiva
> Move WAL archive cleanup from checkpoint to rollover
> ----------------------------------------------------
>
> Key: IGNITE-13831
> URL: https://issues.apache.org/jira/browse/IGNITE-13831
> Project: Ignite
> Issue Type: Improvement
> Components: persistence
> Reporter: Kirill Tkalenko
> Assignee: Kirill Tkalenko
> Priority: Major
> Fix For: 2.10
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Users expect *DataStorageConfiguration#maxWalArchiveSize* to mean that WAL
> archive will not exceed this value, but it is not.
> It seems that to reduce the chance of getting into a situation when we exceed
> WAL archive, it will be lowed when we clean it when switching to a new
> segment than at the end of the checkpoint. It is proposed to move the archive
> cleanup to *FileWriteAheadLogManager#rollOver* when the
> *DataStorageConfiguration#maxWalArchiveSize* is reached.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
