[
https://issues.apache.org/jira/browse/HDFS-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068330#comment-13068330
]
Ravi Prakash commented on HDFS-2177:
------------------------------------
Here's a relevant SNN log
{noformat}
2011-07-19 23:07:30,466 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Web server init done
2011-07-19 23:07:30,467 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Secondary Web-server
up at: virtual.somehost.com:50090
2011-07-19 23:07:30,467 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Secondary image
servlet up at: virtual.somehost.com:50490
2011-07-19 23:07:30,467 WARN
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Checkpoint Period
:60 secs (1 min)
2011-07-19 23:07:30,467 WARN
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Log Size Trigger
:2048000000 bytes (2000000 KB)
2011-07-19 23:08:30,982 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Downloaded file
fsimage size 7576 bytes.
2011-07-19 23:08:30,993 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Downloaded file edits
size 30 bytes.
2011-07-19 23:08:31,093 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: defaultReplication = 3
2011-07-19 23:08:31,093 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: maxReplication = 50
2011-07-19 23:08:31,093 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: minReplication = 1
2011-07-19 23:08:31,093 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: maxReplicationStreams = 2
2011-07-19 23:08:31,093 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: shouldCheckForEnoughRacks
= true
2011-07-19 23:08:31,099 INFO org.apache.hadoop.hdfs.util.GSet: VM type =
64-bit
2011-07-19 23:08:31,099 INFO org.apache.hadoop.hdfs.util.GSet: 2% max memory =
273.85625 MB
2011-07-19 23:08:31,099 INFO org.apache.hadoop.hdfs.util.GSet: capacity =
2^25 = 33554432 entries
2011-07-19 23:08:31,099 INFO org.apache.hadoop.hdfs.util.GSet:
recommended=33554432, actual=33554432
2011-07-19 23:08:31,291 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
fsOwner=hdfs/[email protected]
2011-07-19 23:08:31,291 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=hdfs
2011-07-19 23:08:31,291 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled=true
2011-07-19 23:08:31,305 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
dfs.block.invalidate.limit=1000
2011-07-19 23:08:31,305 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isBlockTokenEnabled=true
blockKeyUpdateInterval=600 min(s), blockTokenLifetime=600 min(s)
2011-07-19 23:08:31,333 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
Caching file names occuring more than 10 times
2011-07-19 23:08:31,349 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Loading image file /tmp/hadoop-someuser/dfs/namesecondary/current/fsimage using
no compression
2011-07-19 23:08:31,349 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Number of files = 71
2011-07-19 23:08:31,370 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Number of files under construction = 0
2011-07-19 23:08:31,375 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Image file of size 7576 loaded in 0 seconds.
2011-07-19 23:08:31,386 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Edits file /tmp/hadoop-someuser/dfs/namesecondary/current/edits of size 30
edits # 2 loaded in 0 seconds.
2011-07-19 23:08:31,388 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog:
Number of transactions: 0 Total time for transactions(ms): 0Number of
transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
2011-07-19 23:08:31,398 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Saving image file /tmp/hadoop-someuser/dfs/namesecondary/current/fsimage using
no compression
2011-07-19 23:08:31,408 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Image file of size 7600 saved in 0 seconds.
2011-07-19 23:08:31,417 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Posted URL
somehost.com:50470putimage=1&port=50490&machine=virtual.somehost.com&token=-35:1272447189:0:13111169
10000:1311116479833:4396bbd3f5b9947166068f39b1307441:CID-1c3abcd4-60e7-4b3c-be9a-420e9df14b74:BP-1333203313-98.137.97.97-1311113548658&newChecksum=5650481e7cb891340dbcc1b22db43eb5
2011-07-19 23:08:31,628 WARN
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Checkpoint done. New
Image Size: 7600
2011-07-19 23:09:31,687 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog:
Number of transactions: 0 Total time for transactions(ms): 0Number of
transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
2011-07-19 23:09:31,703 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Image has not
changed. Will not download image.
2011-07-19 23:09:31,720 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Downloaded file edits
size 4 bytes.
2011-07-19 23:09:31,721 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: defaultReplication = 3
2011-07-19 23:09:31,721 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: maxReplication = 50
2011-07-19 23:09:31,721 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: minReplication = 1
2011-07-19 23:09:31,721 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: maxReplicationStreams = 2
2011-07-19 23:09:31,721 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: shouldCheckForEnoughRacks
= true
2011-07-19 23:09:31,721 INFO org.apache.hadoop.hdfs.util.GSet: VM type =
64-bit
2011-07-19 23:09:31,721 INFO org.apache.hadoop.hdfs.util.GSet: 2% max memory =
273.85625 MB
2011-07-19 23:09:31,721 INFO org.apache.hadoop.hdfs.util.GSet: capacity =
2^25 = 33554432 entries
2011-07-19 23:09:31,721 INFO org.apache.hadoop.hdfs.util.GSet:
recommended=33554432, actual=33554432
2011-07-19 23:09:31,909 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
fsOwner=hdfs/[email protected]
2011-07-19 23:09:31,909 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=hdfs
2011-07-19 23:09:31,909 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled=true
2011-07-19 23:09:31,909 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
dfs.block.invalidate.limit=1000
2011-07-19 23:09:31,909 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isBlockTokenEnabled=true
blockKeyUpdateInterval=600 min(s), blockTokenLifetime=600 min(s)
2011-07-19 23:09:31,909 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
Caching file names occuring more than 10 times
2011-07-19 23:09:31,911 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Edits file /tmp/hadoop-someuser/dfs/namesecondary/current/edits of size 4 edits
# 0 loaded in 0 seconds.
2011-07-19 23:09:31,911 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog:
Number of transactions: 0 Total time for transactions(ms): 0Number of
transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
2011-07-19 23:09:31,914 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Saving image file /tmp/hadoop-someuser/dfs/namesecondary/current/fsimage using
no compression
2011-07-19 23:09:31,918 INFO org.apache.hadoop.hdfs.server.namenode.FSImage:
Image file of size 107 saved in 0 seconds.
2011-07-19 23:09:31,927 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Posted URL
somehost.com:50470putimage=1&port=50490&machine=virtual.somehost.com&token=-35:1272447189:0:13111169
71000:1311116911612:5650481e7cb891340dbcc1b22db43eb5:CID-1c3abcd4-60e7-4b3c-be9a-420e9df14b74:BP-1333203313-98.137.97.97-1311113548658&newChecksum=fab43b7e4a607aa4611d16d20e1b261c
2011-07-19 23:09:31,971 WARN
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Checkpoint done. New
Image Size: 107
{noformat}
As you can see, the image size which was 7600 shrunk to 107 in the very next
checkpoint. (Checkpointing was configured to be every 60 seconds). This was
exactly the same time as the namenode was shutdown and restarted.
> Restarting the namenode when the secondary namenode is checkpointing seems to
> remove everything from /
> ------------------------------------------------------------------------------------------------------
>
> Key: HDFS-2177
> URL: https://issues.apache.org/jira/browse/HDFS-2177
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.23.0
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
> Priority: Blocker
> Fix For: 0.23.0
>
>
> This was again discovered by Arpit Gupta! Restarting the namenode when the
> secondary namenode is checkpointing seems to remove everything from /
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
