Erik Krogen created HDFS-13689: ---------------------------------- Summary: NameNodeRpcServer getEditsFromTxid assumes it is run on active NameNode Key: HDFS-13689 URL: https://issues.apache.org/jira/browse/HDFS-13689 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs, namenode Reporter: Erik Krogen Assignee: Erik Krogen
{{NameNodeRpcServer#getEditsFromTxid}} currently decides which transactions are able to be served, i.e. which transactions are durable, using the following logic: {code} long syncTxid = log.getSyncTxId(); // If we haven't synced anything yet, we can only read finalized // segments since we can't reliably determine which txns in in-progress // segments have actually been committed (e.g. written to a quorum of JNs). // If we have synced txns, we can definitely read up to syncTxid since // syncTxid is only updated after a transaction is committed to all // journals. (In-progress segments written by old writers are already // discarded for us, so if we read any in-progress segments they are // guaranteed to have been written by this NameNode.) boolean readInProgress = syncTxid > 0; {code} This assumes that the NameNode serving this request is the current writer/active NameNode, which may not be true in the ObserverNode situation. Since {{selectInputStreams}} now has a {{onlyDurableTxns}} flag, which, if enabled, will only return durable/committed transactions, we can instead leverage this to provide the same functionality. We should utilize this to avoid consistency issues when serving this request from the ObserverNode. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org