[
https://issues.apache.org/jira/browse/HBASE-26658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26658:
-----------------------------
Description:
When {{ASyncFSWAL}} syncing to HDFS failed, {{AsyncFSWAL.unackedAppends}} are
transfered to {{AsyncFSWAL.toWriteAppends}} to avoid data loss, but
{{AsyncFSWAL.unackedAppends}} itself is not cleared. I think there is no need
to continue retain them in {{AsyncFSWAL.unackedAppends}} because we would open
a new HDFS pipeline to resend the {{AsyncFSWAL.unackedAppends}}.To make the
things worse, if syncing to HDFS failed all the
time,{{AsyncFSWAL.unackedAppends}} would be overwhelmed with repetitive data.
BTW : It would also simplify the logic for fixing HBASE-25905, current fix for
HBASE-25905 is somewhat hard to understand. I think the problem to cause
HBASE-25905 is that {{AsyncFSWAL.unackedAppends}} could not exactly reflect
the *unacked* for current HDFS pipeline. If we clear
{{AsyncFSWAL.unackedAppends}} after transferring them to
{{AsyncFSWAL.toWriteAppends}}, HBASE-25905 could also avoid.
was:
When {{ASyncFSWAL}} syncing to HDFS failed, in {{AsyncFSWAL.syncFailed}} ,
{{AsyncFSWAL.unackedAppends}} are transfered to {{AsyncFSWAL.toWriteAppends}}
to avoid data loss, but {{AsyncFSWAL.unackedAppends}} itself is not cleared. I
think there is no need to continue retain them in {{AsyncFSWAL.unackedAppends}}
because we would open a new HDFS pipeline to resend the
{{AsyncFSWAL.unackedAppends}}.To make the things worse, if syncing to HDFS
failed all the time,{{AsyncFSWAL.unackedAppends}} would be overwhelmed with
repetitive data.
BTW : It would also simplify the logic for fixing HBASE-25905, current fix for
HBASE-25905 is somewhat hard to understand. I think the problem to cause
HBASE-25905 is that {{AsyncFSWAL.unackedAppends}} could not exactly reflect
the *unacked* for current HDFS pipeline. If we clear
{{AsyncFSWAL.unackedAppends}} after transferring them to
{{AsyncFSWAL.toWriteAppends}}, HBASE-25905 could also avoid.
> AsyncFSWAL.unackedAppends should clear after transfered to
> AsyncFSWAL.toWriteAppends
> --------------------------------------------------------------------------------------
>
> Key: HBASE-26658
> URL: https://issues.apache.org/jira/browse/HBASE-26658
> Project: HBase
> Issue Type: Bug
> Components: wal
> Affects Versions: 3.0.0-alpha-2, 2.4.9
> Reporter: chenglei
> Priority: Major
>
> When {{ASyncFSWAL}} syncing to HDFS failed, {{AsyncFSWAL.unackedAppends}}
> are transfered to {{AsyncFSWAL.toWriteAppends}} to avoid data loss, but
> {{AsyncFSWAL.unackedAppends}} itself is not cleared. I think there is no need
> to continue retain them in {{AsyncFSWAL.unackedAppends}} because we would
> open a new HDFS pipeline to resend the {{AsyncFSWAL.unackedAppends}}.To make
> the things worse, if syncing to HDFS failed all the
> time,{{AsyncFSWAL.unackedAppends}} would be overwhelmed with repetitive data.
> BTW : It would also simplify the logic for fixing HBASE-25905, current fix
> for HBASE-25905 is somewhat hard to understand. I think the problem to cause
> HBASE-25905 is that {{AsyncFSWAL.unackedAppends}} could not exactly reflect
> the *unacked* for current HDFS pipeline. If we clear
> {{AsyncFSWAL.unackedAppends}} after transferring them to
> {{AsyncFSWAL.toWriteAppends}}, HBASE-25905 could also avoid.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
