[
https://issues.apache.org/jira/browse/HBASE-25065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211965#comment-17211965
]
Hudson commented on HBASE-25065:
--------------------------------
Results for branch master
[build #92 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/92/]:
(/) *{color:green}+1 overall{color}*
----
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/92/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/92/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/92/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> WAL archival to be done by a separate thread
> --------------------------------------------
>
> Key: HBASE-25065
> URL: https://issues.apache.org/jira/browse/HBASE-25065
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Affects Versions: 3.0.0-alpha-1, 2.4.0
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Priority: Major
>
> Currently we do clean up of logs once we ensure that the region data has been
> flushed. We track the sequence number and if we ensure that the seq number
> has been flushed for any given region and the WAL that was rolled has that
> seq number then those WAL can be archived.
> When we have around ~50 files to archive (per RS) - we do the archiving one
> after the other. Since archiving is nothing but a rename operation it adds to
> the meta operation load of Cloud based FS.
> Not only that - the entire archival is done inside the rollWriterLock. Though
> we have closed the writer and created a new writer and the writes are ongoing
> - we never release the lock until we are done with the archiving.
> What happens is that during that period our logs grow in size compared to the
> default size configured (when we have consistent writes happening).
> So the proposal is to move the log archival to a seperate thread and ensure
> we can do some kind of throttling or batching so that we don't do archival at
> one shot.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
