[
https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17346606#comment-17346606
]
Virajith Jalaparti commented on HDFS-15915:
-------------------------------------------
Looking at this again and answering my question (3) - when
{{FSEditLog#getLastWrittenTxIdWithoutLock()}} is called with the current
implementation (without the patch here), it may or may not return the txid of
the latest operation that was actually applied as the txid may not be assigned
to the operation at all. It is assigned only when {{beginTransaction}} is
called and {{txid}} is incremented in {{FSEditLog}}. Hence, even though the
operations are assigned the correct txids in the current implementation, they
might be assigned at a later time than when it's applied to the namesystem.
This change forces the txid to be assigned when the operation takes place under
the FSN lock.
> Race condition with async edits logging due to updating txId outside of the
> namesystem log
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-15915
> URL: https://issues.apache.org/jira/browse/HDFS-15915
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Major
> Attachments: HDFS-15915-01.patch, HDFS-15915-02.patch,
> HDFS-15915-03.patch, testMkdirsRace.patch
>
>
> {{FSEditLogAsync}} creates an {{FSEditLogOp}} and populates its fields inside
> {{FSNamesystem.writeLock}}. But one essential field the transaction id of the
> edits op remains unset until the time when the operation is scheduled for
> synching. At that time {{beginTransaction()}} will set the the
> {{FSEditLogOp.txid}} and increment the global transaction count. On busy
> NameNode this event can fall outside the write lock.
> This causes problems for Observer reads. It also can potentially reshuffle
> transactions and Standby will apply them in a wrong order.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
