[
https://issues.apache.org/jira/browse/HBASE-26658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17473262#comment-17473262
]
chenglei edited comment on HBASE-26658 at 1/12/22, 3:35 AM:
------------------------------------------------------------
[~zhangduo]，thank you very much for reply.
??The toWriteAppends will be cleared in appendAndAsync, so if you clear
unackedAppends after transferring to toWriteAppends, if we face a HDFS failure
again, there will be data loss...??
Why there is data loss ? In appendAndAsync, we indeed clear toWriteAppends
after we successfully send them, but we also put them in unackedAppends at the
same time, so if we face a HDFS again, unackedAppends would be transferred to
the toWriteAppends to send them again, it seems no data loss.
was (Author: comnetwork):
[~zhangduo]，thank you very much for reply.
??The toWriteAppends will be cleared in appendAndAsync, so if you clear
unackedAppends after transferring to toWriteAppends, if we face a HDFS failure
again, there will be data loss...??
Why there is data loss ? In appendAndAsync, we indeed clear toWriteAppends
after we successfully send them, but we also put them in unackedAppends at the
same time, so if we face a HDFS again, unackedAppends would be transferred to
the toWriteAppends to send them again, it seems no data loss.
> AsyncFSWAL.unackedAppends should clear after transfered to
> AsyncFSWAL.toWriteAppends
> --------------------------------------------------------------------------------------
>
> Key: HBASE-26658
> URL: https://issues.apache.org/jira/browse/HBASE-26658
> Project: HBase
> Issue Type: Bug
> Components: wal
> Affects Versions: 3.0.0-alpha-2, 2.4.9
> Reporter: chenglei
> Assignee: chenglei
> Priority: Major
>
> When {{ASyncFSWAL}} syncing to HDFS failed, {{AsyncFSWAL.unackedAppends}}
> are transfered to {{AsyncFSWAL.toWriteAppends}} to avoid data loss, but
> {{AsyncFSWAL.unackedAppends}} itself is not cleared. I think there is no need
> to continue retain them in {{AsyncFSWAL.unackedAppends}} because we would
> open a new HDFS pipeline to resend the {{AsyncFSWAL.unackedAppends}}. To make
> the things worse, if syncing to HDFS failed all the time,
> {{AsyncFSWAL.unackedAppends}} would be overwhelmed with repetitive datas.
> BTW : It would also simplify the logic for fixing HBASE-25905, current fix
> for HBASE-25905 is somewhat hard to understand. I think the problem to cause
> HBASE-25905 is that {{AsyncFSWAL.unackedAppends}} could not exactly reflect
> the *unacked* for current HDFS pipeline. If we clear
> {{AsyncFSWAL.unackedAppends}} after transferring them to
> {{AsyncFSWAL.toWriteAppends}}, HBASE-25905 could also avoid.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
