[
https://issues.apache.org/jira/browse/HBASE-17085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663855#comment-15663855
]
Duo Zhang commented on HBASE-17085:
-----------------------------------
[~ramkrishna] Could you please confirm your code again? Do you use the latest
master and then apply the v1 patch? I've added a log in sync method and tried a
WALPE run. The txid of the last SyncFuture in syncFutures maybe the same
between calls, but the currentHighestProcessedAppendTxid always increases.
Thanks.
{code}
LOG.info("=================" + fileLengthAtLastSync + ":" +
currentHighestProcessedAppendTxid + ":" + (syncFutures.isEmpty() ? -1 :
syncFutures.last().getTxid()));
{code}
fileLength:currentHighestProcessedAppendTxid:syncFutures.last().getTxid()
{noformat}
2016-11-14 20:57:38,644 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================18954972:16712758:16712902
2016-11-14 20:57:38,644 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19025752:16712778:16712902
2016-11-14 20:57:38,645 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19096532:16712798:16712902
2016-11-14 20:57:38,645 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19167312:16712819:16712902
2016-11-14 20:57:38,645 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19238092:16712839:16712902
2016-11-14 20:57:38,645 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19308872:16712857:16712902
2016-11-14 20:57:38,646 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19379652:16712879:16712902
2016-11-14 20:57:38,646 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19450432:16712899:16712902
2016-11-14 20:57:38,646 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19457510:16712900:16712902
2016-11-14 20:57:38,646 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19528290:16712921:16713043
2016-11-14 20:57:38,647 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19599070:16712941:16713078
2016-11-14 20:57:38,647 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19669850:16712960:16713078
2016-11-14 20:57:38,647 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19740630:16712980:16713078
2016-11-14 20:57:38,647 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19811410:16713000:16713078
2016-11-14 20:57:38,648 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19882190:16713020:16713078
2016-11-14 20:57:38,648 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================19952970:16713040:16713078
2016-11-14 20:57:38,648 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================20023750:16713060:16713078
2016-11-14 20:57:38,648 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================20080374:16713077:16713078
2016-11-14 20:57:38,649 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================20151154:16713097:16713248
2016-11-14 20:57:38,649 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================20221934:16713117:16713258
2016-11-14 20:57:38,650 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================20292714:16713137:16713258
2016-11-14 20:57:38,650 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================20363494:16713157:16713258
2016-11-14 20:57:38,650 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================20434274:16713177:16713258
2016-11-14 20:57:38,650 INFO [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL:
=================20505054:16713196:16713258
{noformat}
> AsyncFSWAL may issue unnecessary AsyncDFSOutput.sync
> ----------------------------------------------------
>
> Key: HBASE-17085
> URL: https://issues.apache.org/jira/browse/HBASE-17085
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Affects Versions: 2.0.0
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17085-v1.patch, HBASE-17085.patch
>
>
> The problem is in appendAndSync method, we will issue an AsyncDFSOutput.sync
> if syncFutures is not empty. The SyncFutures in syncFutures can only be
> removed after an AsyncDFSOutput.sync comes back, so before the
> AsyncDFSOutput.sync actually returns, we will always issue an
> AsyncDFSOutput.sync after an append even if there is no new sync request.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
