[
https://issues.apache.org/jira/browse/HBASE-26658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477228#comment-17477228
]
chenglei edited comment on HBASE-26658 at 1/17/22, 2:06 PM:
------------------------------------------------------------
[~zhangduo], thank you very much for reply. Yes , from you point, I think you
are reasonable. but if we look at unackedAppends as a mechanism to resend the
data when there is any exception, since the data is already transferred to
toWriteAppends, there is no need to still in unackedAppends , may be I was
influenced by {{DFSOutputStream.ackQueue}} in HDFS client, which although the
name {{ackQueue}} is opposite, but the function is similar, it also represent
packets which are sent out but haven't been acked yet:
{code:java}
private boolean processDatanodeError() throws IOException {
......
// move packets from ack queue to front of the data queue
synchronized (dataQueue) {
dataQueue.addAll(0, ackQueue);
ackQueue.clear();
packetSendTime.clear();
}
}
{code}
I would close this JIRA because this is not a bug now, and I think you are also
reasonable from your point.It's a question of how we use unackedAppends and it
doesn't affect the AsyncFSWAL sync result.
was (Author: comnetwork):
[~zhangduo], thank you very much for reply. Yes , from you point, I think you
are reasonable. but if we look at unackedAppends as a mechanism to resend the
data when there is any exception, since the data is already transferred to
toWriteAppends, there is no need to still in unackedAppends , may be I was
influenced by {{DFSOutputStream.ackQueue}} in HDFS client, which although the
name {{ackQueue}} is opposite, but the function is similar, it also represent
packets which are sent out but haven't been acked yet:
{code:java}
private boolean processDatanodeError() throws IOException {
......
// move packets from ack queue to front of the data queue
synchronized (dataQueue) {
dataQueue.addAll(0, ackQueue);
ackQueue.clear();
packetSendTime.clear();
}
}
{code}
I would close this JIRA because this is not a bug now, and I think you are also
reasonable from your point.It's a question of how we use unackedAppends and it
doesn't affect the final result.
> AsyncFSWAL.unackedAppends should clear after transfered to
> AsyncFSWAL.toWriteAppends
> --------------------------------------------------------------------------------------
>
> Key: HBASE-26658
> URL: https://issues.apache.org/jira/browse/HBASE-26658
> Project: HBase
> Issue Type: Bug
> Components: wal
> Affects Versions: 3.0.0-alpha-2, 2.4.9
> Reporter: chenglei
> Assignee: chenglei
> Priority: Major
>
> When {{ASyncFSWAL}} syncing to HDFS failed, {{AsyncFSWAL.unackedAppends}}
> are transfered to {{AsyncFSWAL.toWriteAppends}} to avoid data loss, but
> {{AsyncFSWAL.unackedAppends}} itself is not cleared. I think there is no need
> to continue retain them in {{AsyncFSWAL.unackedAppends}} because we would
> open a new HDFS pipeline to resend the {{AsyncFSWAL.unackedAppends}}.
> BTW : It would also simplify the logic for fixing HBASE-25905, current fix
> for HBASE-25905 is somewhat hard to understand. I think the problem to cause
> HBASE-25905 is that {{AsyncFSWAL.unackedAppends}} could not exactly reflect
> the *unacked* for current HDFS pipeline. If we clear
> {{AsyncFSWAL.unackedAppends}} after transferring them to
> {{AsyncFSWAL.toWriteAppends}}, HBASE-25905 could also avoid.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
