[
https://issues.apache.org/jira/browse/HBASE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473580#comment-13473580
]
Jean-Daniel Cryans commented on HBASE-6758:
-------------------------------------------
bq. please let me know if I missed something or misunderstood your concern
Consider this scenario. First this runs:
bq. Path newPath = computeFilename();
Then with your patch we add this file in ZK during:
bq. i.preLogRoll(oldPath, newPath);
Now let's say HDFS becomes unavailable or the RS fails and never gets to this
line:
bq. HLog.Writer nextWriter = this.createWriterInstance(fs, newPath, conf);
You end up with a log tracked in ZK that doesn't exist. This RS's queue will be
recovered by another RS that will eventually try to read from that non-existing
file. My concern is how we're going to treat that file.
> [replication] The replication-executor should make sure the file that it is
> replicating is closed before declaring success on that file
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6758
> URL: https://issues.apache.org/jira/browse/HBASE-6758
> Project: HBase
> Issue Type: Bug
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: 6758-1-0.92.patch, 6758-2-0.92.patch,
> 6758-trunk-1.patch, 6758-trunk-2.patch, 6758-trunk-3.patch,
> 6758-trunk-4.patch,
> TEST-org.apache.hadoop.hbase.replication.TestReplication.xml
>
>
> I have seen cases where the replication-executor would lose data to replicate
> since the file hasn't been closed yet. Upon closing, the new data becomes
> visible. Before that happens the ZK node shouldn't be deleted in
> ReplicationSourceManager.logPositionAndCleanOldLogs. Changes need to be made
> in ReplicationSource.processEndOfFile as well (currentPath related).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
