[
https://issues.apache.org/jira/browse/HBASE-26658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472708#comment-17472708
]
Duo Zhang commented on HBASE-26658:
-----------------------------------
No, you can not do this. The toWriteAppends will be cleared in appendAndAsync,
so if you clear unackedAppends after transferring to toWriteAppends, if we face
a HDFS failure again, there will be data loss...
And I do not understand this
{quote}
To make the things worse, if syncing to HDFS failed all the time,
AsyncFSWAL.unackedAppends would be overwhelmed with repetitive datas.
{quote}
If we fail all the time, we just keep transferring the data from unackedAppends
to toWriteAppends, how could this operation introduce repetitive data in
unackedAppends?
Thanks.
> AsyncFSWAL.unackedAppends should clear after transfered to
> AsyncFSWAL.toWriteAppends
> --------------------------------------------------------------------------------------
>
> Key: HBASE-26658
> URL: https://issues.apache.org/jira/browse/HBASE-26658
> Project: HBase
> Issue Type: Bug
> Components: wal
> Affects Versions: 3.0.0-alpha-2, 2.4.9
> Reporter: chenglei
> Assignee: chenglei
> Priority: Major
>
> When {{ASyncFSWAL}} syncing to HDFS failed, {{AsyncFSWAL.unackedAppends}}
> are transfered to {{AsyncFSWAL.toWriteAppends}} to avoid data loss, but
> {{AsyncFSWAL.unackedAppends}} itself is not cleared. I think there is no need
> to continue retain them in {{AsyncFSWAL.unackedAppends}} because we would
> open a new HDFS pipeline to resend the {{AsyncFSWAL.unackedAppends}}. To make
> the things worse, if syncing to HDFS failed all the time,
> {{AsyncFSWAL.unackedAppends}} would be overwhelmed with repetitive datas.
> BTW : It would also simplify the logic for fixing HBASE-25905, current fix
> for HBASE-25905 is somewhat hard to understand. I think the problem to cause
> HBASE-25905 is that {{AsyncFSWAL.unackedAppends}} could not exactly reflect
> the *unacked* for current HDFS pipeline. If we clear
> {{AsyncFSWAL.unackedAppends}} after transferring them to
> {{AsyncFSWAL.toWriteAppends}}, HBASE-25905 could also avoid.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
