[jira] [Updated] (HDFS-13236) Standby NN down with error encountered while tailing edits
[ https://issues.apache.org/jira/browse/HDFS-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy Malygin updated HDFS-13236: - Description: After update Hadoop from 2.7.3 to 3.0.0 standby NN down with error encountered while tailing edits from JN: {code:java} Feb 28 01:58:31 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:31,594 INFO [FSImageSaver for /one/hadoop-data/dfs of type IMAGE_AND_EDITS] FSImageFormatProtobuf - Image file /one/hadoop-data/dfs/current/fsimage.ckpt_012748979 98 of size 4595971949 bytes saved in 93 seconds. Feb 28 01:58:33 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:33,445 INFO [Standby State Checkpointer] NNStorageRetentionManager - Going to retain 2 images with txid >= 1274897935 Feb 28 01:58:33 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:33,445 INFO [Standby State Checkpointer] NNStorageRetentionManager - Purging old image FSImageFile(file=/one/hadoop-data/dfs/current/fsimage_01274897875, cpktTxId =01274897875) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,660 INFO [Edit log tailer] FSImage - Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@6a168e6f expecting start txid #1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,660 INFO [Edit log tailer] FSImage - Start loading edits file http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A10 56233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true, http://srve2916.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848; inProgressOk=true Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,661 INFO [Edit log tailer] RedundantEditLogInputStream - Fast-forwarding stream 'http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999 torageInfo=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true, http://srve2916.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217 -aef5-6ed206893848=true' to transaction ID 1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,661 INFO [Edit log tailer] RedundantEditLogInputStream - Fast-forwarding stream 'http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true' to transaction ID 1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,680 ERROR [Edit log tailer] FSEditLogLoader - Encountered exception on operation AddOp [length=0, inodeId=145550319, path=/kafka/parquet/infrastructureGrace/date=2018-02-28/_temporary/1/_temporary/attempt_1516181147167_20856_r_98_0/part-r-00098.gz.parquet, replication=3, mtime=1519772206615, atime=1519772206615, blockSize=134217728, blocks=[], permissions=root:supergroup:rw-r--r--, aclEntries=null, clientName=DFSClient_attempt_1516181147167_20856_r_98_0_1523538799_1, clientMachine=10.137.2.142, overwrite=false, RpcClientId=, RpcCallId=271996603, storagePolicyId=0, erasureCodingPolicyId=0, opCode=OP_ADD, txid=1274898002] Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: java.lang.IllegalArgumentException: Invalid clientId - length is 0 expected length 16 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntry.(RetryCache.java:74) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntry.(RetryCache.java:86) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntryWithPayload.(RetryCache.java:163) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache.addCacheEntryWithPayload(RetryCache.java:322) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.addCacheEntryWithPayload(FSNamesystem.java:946) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:397) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:249) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:158) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:882) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:863) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at
[jira] [Updated] (HDFS-13236) Standby NN down with error encountered while tailing edits
[ https://issues.apache.org/jira/browse/HDFS-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy Malygin updated HDFS-13236: - Description: After update Hadoop from 2.7.3 to 3.0.0 standby NN down with error encountered while tailing edits from JN: {code:java} Feb 28 01:58:31 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:31,594 INFO [FSImageSaver for /one/hadoop-data/dfs of type IMAGE_AND_EDITS] FSImageFormatProtobuf - Image file /one/hadoop-data/dfs/current/fsimage.ckpt_012748979 98 of size 4595971949 bytes saved in 93 seconds. Feb 28 01:58:33 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:33,445 INFO [Standby State Checkpointer] NNStorageRetentionManager - Going to retain 2 images with txid >= 1274897935 Feb 28 01:58:33 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:33,445 INFO [Standby State Checkpointer] NNStorageRetentionManager - Purging old image FSImageFile(file=/one/hadoop-data/dfs/current/fsimage_01274897875, cpktTxId =01274897875) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,660 INFO [Edit log tailer] FSImage - Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@6a168e6f expecting start txid #1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,660 INFO [Edit log tailer] FSImage - Start loading edits file http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A10 56233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true, http://srve2916.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848; inProgressOk=true Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,661 INFO [Edit log tailer] RedundantEditLogInputStream - Fast-forwarding stream 'http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999 torageInfo=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true, http://srve2916.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217 -aef5-6ed206893848=true' to transaction ID 1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,661 INFO [Edit log tailer] RedundantEditLogInputStream - Fast-forwarding stream 'http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true' to transaction ID 1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,680 ERROR [Edit log tailer] FSEditLogLoader - Encountered exception on operation AddOp [length=0, inodeId=145550319, path=/kafka/parquet/infrastructureGrace/date=2018-02-28/_temporary/1/_temporary/attempt_1516181147167_20856_r_98_0/part-r-00098.gz.parquet, replication=3, mtime=1519772206615, atime=1519772206615, blockSize=134217728, blocks=[], permissions=root:supergroup:rw-r--r--, aclEntries=null, clientName=DFSClient_attempt_1516181147167_20856_r_98_0_1523538799_1, clientMachine=10.137.2.142, overwrite=false, RpcClientId=, RpcCallId=271996603, storagePolicyId=0, erasureCodingPolicyId=0, opCode=OP_ADD, txid=1274898002] Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: java.lang.IllegalArgumentException: Invalid clientId - length is 0 expected length 16 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntry.(RetryCache.java:74) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntry.(RetryCache.java:86) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntryWithPayload.(RetryCache.java:163) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache.addCacheEntryWithPayload(RetryCache.java:322) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.addCacheEntryWithPayload(FSNamesystem.java:946) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:397) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:249) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:158) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:882) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:863) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at
[jira] [Updated] (HDFS-13236) Standby NN down with error encountered while tailing edits
[ https://issues.apache.org/jira/browse/HDFS-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy Malygin updated HDFS-13236: - Description: After update Hadoop from 2.7.3 to 3.0.0 standby NN down with error encountered while tailing edits from JN: {code:java} Feb 28 01:58:31 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:31,594 INFO [FSImageSaver for /one/hadoop-data/dfs of type IMAGE_AND_EDITS] FSImageFormatProtobuf - Image file /one/hadoop-data/dfs/current/fsimage.ckpt_012748979 98 of size 4595971949 bytes saved in 93 seconds. Feb 28 01:58:33 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:33,445 INFO [Standby State Checkpointer] NNStorageRetentionManager - Going to retain 2 images with txid >= 1274897935 Feb 28 01:58:33 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:33,445 INFO [Standby State Checkpointer] NNStorageRetentionManager - Purging old image FSImageFile(file=/one/hadoop-data/dfs/current/fsimage_01274897875, cpktTxId =01274897875) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,660 INFO [Edit log tailer] FSImage - Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@6a168e6f expecting start txid #1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,660 INFO [Edit log tailer] FSImage - Start loading edits file http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A10 56233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true, http://srve2916.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848; inProgressOk=true Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,661 INFO [Edit log tailer] RedundantEditLogInputStream - Fast-forwarding stream 'http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999 torageInfo=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true, http://srve2916.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217 -aef5-6ed206893848=true' to transaction ID 1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,661 INFO [Edit log tailer] RedundantEditLogInputStream - Fast-forwarding stream 'http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true' to transaction ID 1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,680 ERROR [Edit log tailer] FSEditLogLoader - Encountered exception on operation AddOp [length=0, inodeId=145550319, path=/kafka/parquet/infrastructureGrace/date=2018-02-28/_temporary/1/_temporary/attempt_1516181147167_20856_r_98_0/part-r-00098.gz.parquet, replication=3, mtime=1519772206615, atime=1519772206615, blockSize=134217728, blocks=[], permissions=root:supergroup:rw-r--r--, aclEntries=null, clientName=DFSClient_attempt_1516181147167_20856_r_98_0_1523538799_1, clientMachine=10.137.2.142, overwrite=false, RpcClientId=, RpcCallId=271996603, storagePolicyId=0, erasureCodingPolicyId=0, opCode=OP_ADD, txid=1274898002] Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: java.lang.IllegalArgumentException: Invalid clientId - length is 0 expected length 16 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntry.(RetryCache.java:74) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntry.(RetryCache.java:86) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntryWithPayload.(RetryCache.java:163) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache.addCacheEntryWithPayload(RetryCache.java:322) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.addCacheEntryWithPayload(FSNamesystem.java:946) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:397) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:249) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:158) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:882) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:863) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at
[jira] [Updated] (HDFS-13236) Standby NN down with error encountered while tailing edits
[ https://issues.apache.org/jira/browse/HDFS-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy Malygin updated HDFS-13236: - Description: After update Hadoop from 2.7.3 to 3.0.0 standby NN down with error encountered while tailing edits from JN: {code:java} Feb 28 01:58:31 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:31,594 INFO [FSImageSaver for /one/hadoop-data/dfs of type IMAGE_AND_EDITS] FSImageFormatProtobuf - Image file /one/hadoop-data/dfs/current/fsimage.ckpt_012748979 98 of size 4595971949 bytes saved in 93 seconds. Feb 28 01:58:33 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:33,445 INFO [Standby State Checkpointer] NNStorageRetentionManager - Going to retain 2 images with txid >= 1274897935 Feb 28 01:58:33 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:33,445 INFO [Standby State Checkpointer] NNStorageRetentionManager - Purging old image FSImageFile(file=/one/hadoop-data/dfs/current/fsimage_01274897875, cpktTxId =01274897875) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,660 INFO [Edit log tailer] FSImage - Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@6a168e6f expecting start txid #1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,660 INFO [Edit log tailer] FSImage - Start loading edits file http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A10 56233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true, http://srve2916.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848; inProgressOk=true Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,661 INFO [Edit log tailer] RedundantEditLogInputStream - Fast-forwarding stream 'http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999 torageInfo=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true, http://srve2916.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217 -aef5-6ed206893848=true' to transaction ID 1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,661 INFO [Edit log tailer] RedundantEditLogInputStream - Fast-forwarding stream 'http://srvd87.local:8480/getJournal?jid=datalab-hadoop-backup=1274897999=-64%3A1056233980%3A0%3ACID-1fba08aa-c8bd-4217-aef5-6ed206893848=true' to transaction ID 1274897999 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: 2018-02-28 01:58:34,680 ERROR [Edit log tailer] FSEditLogLoader - Encountered exception on operation AddOp [length=0, inodeId=145550319, path=/kafka/parquet/infrastructureGrace/date=2018-02-28/_temporary/1/_temporary/attempt_1516181147167_20856_r_98_0/part-r-00098.gz.parquet, replication=3, mtime=1519772206615, atime=1519772206615, blockSize=134217728, blocks=[], permissions=root:supergroup:rw-r--r--, aclEntries=null, clientName=DFSClient_attempt_1516181147167_20856_r_98_0_1523538799_1, clientMachine=10.137.2.142, overwrite=false, RpcClientId=, RpcCallId=271996603, storagePolicyId=0, erasureCodingPolicyId=0, opCode=OP_ADD, txid=1274898002] Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: java.lang.IllegalArgumentException: Invalid clientId - length is 0 expected length 16 Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntry.(RetryCache.java:74) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntry.(RetryCache.java:86) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache$CacheEntryWithPayload.(RetryCache.java:163) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.ipc.RetryCache.addCacheEntryWithPayload(RetryCache.java:322) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.addCacheEntryWithPayload(FSNamesystem.java:946) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:397) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:249) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:158) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:882) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:863) Feb 28 01:58:34 srvd2135 datalab-namenode[15566]: at