[ 
https://issues.apache.org/jira/browse/HBASE-25065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214584#comment-17214584
 ] 

Hudson commented on HBASE-25065:
--------------------------------

Results for branch master
        [build #95 on 
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/95/]:
 (x) *{color:red}-1 overall{color}*
----
details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/95/General_20Nightly_20Build_20Report/]






(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/95/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/95/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> WAL archival to be done by a separate thread
> --------------------------------------------
>
>                 Key: HBASE-25065
>                 URL: https://issues.apache.org/jira/browse/HBASE-25065
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>    Affects Versions: 3.0.0-alpha-1, 2.4.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>            Priority: Major
>             Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Currently we do clean up of logs once we ensure that the region data has been 
> flushed. We track the sequence number and if we ensure that the seq number 
> has been flushed for any given region and the WAL that was rolled has that 
> seq number then those WAL can be archived.
> When we have around ~50 files to archive (per RS) - we do the archiving one 
> after the other. Since archiving is nothing but a rename operation it adds to 
> the meta operation load of Cloud based FS. 
> Not only that - the entire archival is done inside the rollWriterLock. Though 
> we have closed the writer and created a new writer and the writes are ongoing 
> - we never release the lock until we are done with the archiving. 
> What happens is that during that period our logs grow in size compared to the 
> default size configured (when we have consistent writes happening). 
> So the proposal is to move the log archival to a seperate thread and ensure 
> we can do some kind of throttling or batching so that we don't do archival at 
> one shot. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to