[ 
https://issues.apache.org/jira/browse/HDFS-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475894#comment-15475894
 ] 

Dinesh edited comment on HDFS-9096 at 10/18/16 10:39 AM:
---------------------------------------------------------

Facing same issue when I Rollback (after rolling upgrade) from Hadoop 2.7.2 to 
2.5.2; Could any one please tell, is this a known bug?
My Name node log details:

|C:\Hadoop\bin>hdfs namenode -rollingUpgrade rollback
16/10/18 15:54:47 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = myservernode/192.168.1.21
STARTUP_MSG:   args = [-rollingUpgrade, rollback]
STARTUP_MSG:   version = 2.7.2
STARTUP_MSG:   classpath = ....
16/10/18 15:54:47 INFO namenode.NameNode: createNameNode [-rollingUpgrade, 
rollback]
16/10/18 15:54:48 INFO impl.MetricsConfig: loaded properties from 
hadoop-metrics2.properties
16/10/18 15:54:48 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 
second(s).
16/10/18 15:54:48 INFO impl.MetricsSystemImpl: NameNode metrics system started
16/10/18 15:54:48 INFO namenode.NameNode: fs.defaultFS is hdfs://hacluster
16/10/18 15:54:48 INFO namenode.NameNode: Clients are to use hacluster to 
access this namenode/service.
16/10/18 15:54:48 INFO hdfs.DFSUtil: Starting Web-server for hdfs at: 
http://myservernode.root.server.lan:50070
16/10/18 15:54:48 INFO mortbay.log: Logging to 
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
16/10/18 15:54:48 INFO server.AuthenticationFilter: Unable to initialize 
FileSignerSecretProvider, falling back to use random secrets.
16/10/18 15:54:48 INFO http.HttpRequestLog: Http request log for 
http.requests.namenode is not defined
16/10/18 15:54:48 INFO http.HttpServer2: Added global filter 'safety' 
(class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
16/10/18 15:54:48 INFO http.HttpServer2: Added filter static_user_filter 
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to 
context hdfs
16/10/18 15:54:48 INFO http.HttpServer2: Added filter static_user_filter 
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to 
context logs
16/10/18 15:54:48 INFO http.HttpServer2: Added filter static_user_filter 
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to 
context static
16/10/18 15:54:48 INFO http.HttpServer2: Added filter 
'org.apache.hadoop.hdfs.web.AuthFilter' 
(class=org.apache.hadoop.hdfs.web.AuthFilter)
16/10/18 15:54:48 INFO http.HttpServer2: addJerseyResourcePackage: 
packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources,
 pathSpec=/webhdfs/v1/*
16/10/18 15:54:48 INFO http.HttpServer2: Jetty bound to port 50070
16/10/18 15:54:48 INFO mortbay.log: jetty-6.1.26
16/10/18 15:54:48 INFO mortbay.log: Started 
HttpServer2$selectchannelconnectorwithsafestar...@myservernode.root.server.lan:50070
16/10/18 15:54:48 WARN namenode.FSNamesystem: Only one image storage directory 
(dfs.namenode.name.dir) configured. Beware of data loss due to lack of 
redundant storage directories!
16/10/18 15:54:48 INFO namenode.FSNamesystem: No KeyProvider found.
16/10/18 15:54:48 INFO namenode.FSNamesystem: fsLock is fair:true
16/10/18 15:54:48 INFO blockmanagement.DatanodeManager: 
dfs.block.invalidate.limit=1000
16/10/18 15:54:48 INFO blockmanagement.DatanodeManager: 
dfs.namenode.datanode.registration.ip-hostname-check=true
16/10/18 15:54:48 INFO blockmanagement.BlockManager: 
dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
16/10/18 15:54:48 INFO blockmanagement.BlockManager: The block deletion will 
start around 2016 Oct 18 15:54:48
16/10/18 15:54:48 INFO util.GSet: Computing capacity for map BlocksMap
16/10/18 15:54:48 INFO util.GSet: VM type       = 64-bit
16/10/18 15:54:48 INFO util.GSet: 2.0% max memory 455 MB = 9.1 MB
16/10/18 15:54:48 INFO util.GSet: capacity      = 2^20 = 1048576 entries
16/10/18 15:54:48 INFO blockmanagement.BlockManager: 
dfs.block.access.token.enable=false
16/10/18 15:54:48 INFO blockmanagement.BlockManager: defaultReplication         
= 3
16/10/18 15:54:48 INFO blockmanagement.BlockManager: maxReplication             
= 512
16/10/18 15:54:48 INFO blockmanagement.BlockManager: minReplication             
= 1
16/10/18 15:54:48 INFO blockmanagement.BlockManager: maxReplicationStreams      
= 2
16/10/18 15:54:48 INFO blockmanagement.BlockManager: replicationRecheckInterval 
= 3000
16/10/18 15:54:48 INFO blockmanagement.BlockManager: encryptDataTransfer        
= false
16/10/18 15:54:48 INFO blockmanagement.BlockManager: maxNumBlocksToLog          
= 1000
16/10/18 15:54:48 INFO namenode.FSNamesystem: fsOwner             = SYSTEM 
(auth:SIMPLE)
16/10/18 15:54:48 INFO namenode.FSNamesystem: supergroup          = supergroup
16/10/18 15:54:48 INFO namenode.FSNamesystem: isPermissionEnabled = false
16/10/18 15:54:48 INFO namenode.FSNamesystem: Determined nameservice ID: 
hacluster
16/10/18 15:54:48 INFO namenode.FSNamesystem: HA Enabled: true
16/10/18 15:54:48 INFO namenode.FSNamesystem: Append Enabled: true
16/10/18 15:54:48 INFO util.GSet: Computing capacity for map INodeMap
16/10/18 15:54:48 INFO util.GSet: VM type       = 64-bit
16/10/18 15:54:48 INFO util.GSet: 1.0% max memory 455 MB = 4.5 MB
16/10/18 15:54:48 INFO util.GSet: capacity      = 2^19 = 524288 entries
16/10/18 15:54:48 INFO namenode.FSDirectory: ACLs enabled? false
16/10/18 15:54:48 INFO namenode.FSDirectory: XAttrs enabled? true
16/10/18 15:54:48 INFO namenode.FSDirectory: Maximum size of an xattr: 16384
16/10/18 15:54:48 INFO namenode.NameNode: Caching file names occuring more than 
10 times
16/10/18 15:54:48 INFO util.GSet: Computing capacity for map cachedBlocks
16/10/18 15:54:48 INFO util.GSet: VM type       = 64-bit
16/10/18 15:54:48 INFO util.GSet: 0.25% max memory 455 MB = 1.1 MB
16/10/18 15:54:48 INFO util.GSet: capacity      = 2^17 = 131072 entries
16/10/18 15:54:48 INFO namenode.FSNamesystem: 
dfs.namenode.safemode.threshold-pct = 0.9990000128746033
16/10/18 15:54:48 INFO namenode.FSNamesystem: 
dfs.namenode.safemode.min.datanodes = 0
16/10/18 15:54:48 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension   
  = 30000
16/10/18 15:54:48 INFO metrics.TopMetrics: NNTop conf: 
dfs.namenode.top.window.num.buckets = 10
16/10/18 15:54:48 INFO metrics.TopMetrics: NNTop conf: 
dfs.namenode.top.num.users = 10
16/10/18 15:54:48 INFO metrics.TopMetrics: NNTop conf: 
dfs.namenode.top.windows.minutes = 1,5,25
16/10/18 15:54:48 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
16/10/18 15:54:48 INFO namenode.FSNamesystem: Retry cache will use 0.03 of 
total heap and retry cache entry expiry time is 600000 millis
16/10/18 15:54:48 INFO util.GSet: Computing capacity for map NameNodeRetryCache
16/10/18 15:54:48 INFO util.GSet: VM type       = 64-bit
16/10/18 15:54:48 INFO util.GSet: 0.029999999329447746% max memory 455 MB = 
139.8 KB
16/10/18 15:54:48 INFO util.GSet: capacity      = 2^14 = 16384 entries
16/10/18 15:54:48 INFO common.Storage: Lock on 
\Metadata\data\dfs\namenode\in_use.lock acquired by nodename 13228@myservernode
16/10/18 15:54:49 INFO client.QuorumJournalManager: Starting recovery process 
for unclosed journal segments...
16/10/18 15:54:49 INFO client.QuorumJournalManager: Successfully started new 
epoch 6
16/10/18 15:54:49 INFO client.QuorumJournalManager: Beginning recovery of 
unclosed segment starting at txid 5879
16/10/18 15:54:49 INFO client.QuorumJournalManager: Recovery prepare phase 
complete. Responses:
192.168.1.22:8485: segmentState { startTxId: 5879 endTxId: 5942 isInProgress: 
true } lastWriterEpoch: 5 lastCommittedTxId: 5941
192.168.1.23:8485: segmentState { startTxId: 5879 endTxId: 5942 isInProgress: 
true } lastWriterEpoch: 5 lastCommittedTxId: 5941
16/10/18 15:54:49 INFO client.QuorumJournalManager: Using longest log: 
172.16.100.93:8485=segmentState {
  startTxId: 5879
  endTxId: 5942
  isInProgress: true
}
lastWriterEpoch: 5
lastCommittedTxId: 5941
16/10/18 15:54:49 INFO namenode.FileJournalManager: Recovering unfinalized 
segments in \Metadata\data\dfs\namenode\current
16/10/18 15:54:49 INFO namenode.FileJournalManager: Finalizing edits file 
\Metadata\data\dfs\namenode\current\edits_inprogress_0000000000000005718 -> 
\Metadata\data\dfs\namenode\current\edits_0000000000000005718-0000000000000005718
16/10/18 15:54:50 ERROR namenode.FSImage: Failed to load image from 
FSImageFile(file=\Metadata\data\dfs\namenode\current\fsimage_rollback_0000000000000005713,
 cpktTxId=0000000000000005713)
java.io.IOException: Image version -57 is not equal to the software version -63
        at 
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:196)
        at 
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:179)
        at 
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:226)
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:957)
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:941)
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:740)
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:676)
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:584)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:644)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/10/18 15:54:50 WARN namenode.FSNamesystem: Encountered exception loading 
fsimage
java.io.IOException: Failed to load an FSImage file!
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:687)
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:584)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:644)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/10/18 15:54:50 INFO mortbay.log: Stopped 
HttpServer2$selectchannelconnectorwithsafestar...@myservernode.root.server.lan:50070
16/10/18 15:54:50 INFO impl.MetricsSystemImpl: Stopping NameNode metrics 
system...
16/10/18 15:54:50 INFO impl.MetricsSystemImpl: NameNode metrics system stopped.
16/10/18 15:54:50 INFO impl.MetricsSystemImpl: NameNode metrics system shutdown 
complete.
16/10/18 15:54:50 ERROR namenode.NameNode: Failed to start namenode.
java.io.IOException: Failed to load an FSImage file!
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:687)
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:584)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:644)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/10/18 15:54:50 INFO util.ExitUtil: Exiting with status 1
16/10/18 15:54:50 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at myservernode/192.168.1.21
************************************************************/|


was (Author: dineshkumarp):
Facing same issue when I Rollback (after rolling upgrade) from Hadoop 2.7.2 to 
2.5.2; Could any one please tell, is this a known bug?

> Issue in Rollback (after rolling upgrade) from hadoop 2.7.1 to 2.4.0
> --------------------------------------------------------------------
>
>                 Key: HDFS-9096
>                 URL: https://issues.apache.org/jira/browse/HDFS-9096
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: rolling upgrades
>    Affects Versions: 2.4.0
>            Reporter: Harpreet Kaur
>
> I tried to do rolling upgrade from hadoop 2.4.0 to hadoop 2.7.1. As per 
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html#dfsadmin_-rollingUpgrade
>  one can rollback to previous release provided the finalise step is not done. 
> I upgraded the setup but didnot finalise the upgrade and tried to rollback 
> HDFS to 2.4.0
> I tried the following steps
>   1.  Shutdown all NNs and DNs.
>   2.  Restore the pre-upgrade release in all machines.
>   3.  Start NN1 as Active with the "-rollingUpgrade 
> rollback<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html#namenode_-rollingUpgrade>"
>  option.
> I am getting the following error after 3rd step
> 15/09/01 17:53:35 INFO namenode.AclConfigFlag: ACLs enabled? false
> 15/09/01 17:53:35 INFO common.Storage: Lock on <<NameNode dir>>/in_use.lock 
> acquired by nodename 12152@VM-2
> 15/09/01 17:53:35 WARN namenode.FSNamesystem: Encountered exception loading 
> fsimage
> org.apache.hadoop.hdfs.server.common.IncorrectVersionException: Unexpected 
> version of storage directory /data/yarn/namenode. Reported: -63. Expecting = 
> -56.
> at 
> org.apache.hadoop.hdfs.server.common.StorageInfo.setLayoutVersion(StorageInfo.java:178)
> at 
> org.apache.hadoop.hdfs.server.common.StorageInfo.setFieldsFromProperties(StorageInfo.java:131)
> at 
> org.apache.hadoop.hdfs.server.namenode.NNStorage.setFieldsFromProperties(NNStorage.java:608)
> at 
> org.apache.hadoop.hdfs.server.common.StorageInfo.readProperties(StorageInfo.java:228)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:309)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:202)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:882)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:639)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:455)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:511)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:670)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:655)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1304)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1370)
> 15/09/01 17:53:35 INFO mortbay.log: Stopped 
> SelectChannelConnector@0.0.0.0:50070
> 15/09/01 17:53:35 INFO impl.MetricsSystemImpl: Stopping NameNode metrics 
> system...
> 15/09/01 17:53:35 INFO impl.MetricsSystemImpl: NameNode metrics system 
> stopped.
> 15/09/01 17:53:35 INFO impl.MetricsSystemImpl: NameNode metrics system 
> shutdown complete.
> 15/09/01 17:53:35 FATAL namenode.NameNode: Exception in namenode join
> From rolling upgrade documentation it can be inferred that rolling upgrade is 
> supported Hadoop 2.4.0 onwards but rollingUpgrade rollback to Hadoop 2.4.0 
> seems to be broken in Hadoop 2.4.0. It throws above mentioned error.
> Are there any other steps to perform rollback (from rolling upgrade) or is it 
> not supported to rollback to Hadoop 2.4.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to