[
https://issues.apache.org/jira/browse/HDFS-15308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216885#comment-17216885
]
Ahmed Hussein commented on HDFS-15308:
--------------------------------------
[~touchida], Thanks for taking a look at that failure.
I still see failures from this test. Can you please check it again to see if
your patch still applies?
I also suggest to change the while loop into {{GenericTestUtils.waitFor()}}.
{code:bash}
Error Message
Found 4 timeout pending reconstruction tasks
Stacktrace
java.lang.AssertionError: Found 4 timeout pending reconstruction tasks
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hdfs.TestReconstructStripedFile.testNNSendsErasureCodingTasks(TestReconstructStripedFile.java:502)
at
org.apache.hadoop.hdfs.TestReconstructStripedFile.testNNSendsErasureCodingTasks(TestReconstructStripedFile.java:458)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Standard Output
2020-10-16 23:05:16,252 [Listener at localhost/35179] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:<init>(529)) - starting cluster: numNameNodes=1,
numDataNodes=12
2020-10-16 23:05:16,254 [Listener at localhost/35179] INFO namenode.NameNode
(NameNode.java:format(1249)) - Formatting using clusterid: testClusterID
2020-10-16 23:05:16,254 [Listener at localhost/35179] INFO namenode.FSEditLog
(FSEditLog.java:newInstance(229)) - Edit logging is async:true
2020-10-16 23:05:16,262 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(828)) - KeyProvider: null
2020-10-16 23:05:16,263 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystemLock.java:<init>(131)) - fsLock is fair: true
2020-10-16 23:05:16,263 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystemLock.java:<init>(149)) - Detailed lock hold
time metrics enabled: false
2020-10-16 23:05:16,263 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(861)) - fsOwner
= jenkins (auth:SIMPLE)
2020-10-16 23:05:16,263 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(862)) - supergroup
= supergroup
2020-10-16 23:05:16,263 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(863)) - isPermissionEnabled
= true
2020-10-16 23:05:16,263 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(864)) - isStoragePolicyEnabled
= true
2020-10-16 23:05:16,263 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(875)) - HA Enabled: false
2020-10-16 23:05:16,264 [Listener at localhost/35179] INFO common.Util
(Util.java:isDiskStatsEnabled(395)) -
dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO
profiling
2020-10-16 23:05:16,264 [Listener at localhost/35179] INFO
blockmanagement.DatanodeManager (DatanodeManager.java:<init>(309)) -
dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2020-10-16 23:05:16,264 [Listener at localhost/35179] INFO
blockmanagement.DatanodeManager (DatanodeManager.java:<init>(317)) -
dfs.namenode.datanode.registration.ip-hostname-check=true
2020-10-16 23:05:16,264 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (InvalidateBlocks.java:printBlockDeletionTime(77))
- dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2020-10-16 23:05:16,265 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (InvalidateBlocks.java:printBlockDeletionTime(83))
- The block deletion will start around 2020 Oct 16 23:05:16
2020-10-16 23:05:16,265 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(395)) - Computing capacity for map
BlocksMap
2020-10-16 23:05:16,265 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(396)) - VM type = 64-bit
2020-10-16 23:05:16,265 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(397)) - 2.0% max memory 1.8 GB = 36.4 MB
2020-10-16 23:05:16,265 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(402)) - capacity = 2^22 = 4194304
entries
2020-10-16 23:05:16,268 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:createSPSManager(5439)) -
Storage policy satisfier is disabled
2020-10-16 23:05:16,268 [Listener at localhost/35179] INFO
blockmanagement.BlockManager
(BlockManager.java:createBlockTokenSecretManager(641)) -
dfs.block.access.token.enable = false
2020-10-16 23:05:16,268 [Listener at localhost/35179] INFO
blockmanagement.BlockManagerSafeMode (BlockManagerSafeMode.java:<init>(160)) -
dfs.namenode.safemode.threshold-pct = 0.999
2020-10-16 23:05:16,268 [Listener at localhost/35179] INFO
blockmanagement.BlockManagerSafeMode (BlockManagerSafeMode.java:<init>(161)) -
dfs.namenode.safemode.min.datanodes = 0
2020-10-16 23:05:16,269 [Listener at localhost/35179] INFO
blockmanagement.BlockManagerSafeMode (BlockManagerSafeMode.java:<init>(163)) -
dfs.namenode.safemode.extension = 0
2020-10-16 23:05:16,269 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(627)) -
defaultReplication = 3
2020-10-16 23:05:16,269 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(628)) - maxReplication
= 512
2020-10-16 23:05:16,269 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(629)) - minReplication
= 1
2020-10-16 23:05:16,269 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(630)) -
maxReplicationStreams = 2
2020-10-16 23:05:16,269 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(631)) -
redundancyRecheckInterval = 1000ms
2020-10-16 23:05:16,269 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(632)) -
encryptDataTransfer = false
2020-10-16 23:05:16,269 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(633)) -
maxNumBlocksToLog = 1000
2020-10-16 23:05:16,270 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(395)) - Computing capacity for map
INodeMap
2020-10-16 23:05:16,270 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(396)) - VM type = 64-bit
2020-10-16 23:05:16,270 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(397)) - 1.0% max memory 1.8 GB = 18.2 MB
2020-10-16 23:05:16,271 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(402)) - capacity = 2^21 = 2097152
entries
2020-10-16 23:05:16,272 [Listener at localhost/35179] INFO
namenode.FSDirectory (FSDirectory.java:<init>(335)) - ACLs enabled? true
2020-10-16 23:05:16,272 [Listener at localhost/35179] INFO
namenode.FSDirectory (FSDirectory.java:<init>(339)) - POSIX ACL inheritance
enabled? true
2020-10-16 23:05:16,272 [Listener at localhost/35179] INFO
namenode.FSDirectory (FSDirectory.java:<init>(343)) - XAttrs enabled? true
2020-10-16 23:05:16,272 [Listener at localhost/35179] INFO namenode.NameNode
(FSDirectory.java:<init>(406)) - Caching file names occurring more than 10 times
2020-10-16 23:05:16,272 [Listener at localhost/35179] INFO
snapshot.SnapshotManager (SnapshotManager.java:<init>(162)) - Loaded config
captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false,
snapshotDiffAllowSnapRootDescendant: true, maxSnapshotFSLimit: 65536,
maxSnapshotLimit: 65536
2020-10-16 23:05:16,272 [Listener at localhost/35179] INFO
snapshot.SnapshotManager (SnapshotManager.java:<init>(175)) -
dfs.namenode.snapshot.deletion.ordered = false
2020-10-16 23:05:16,273 [Listener at localhost/35179] INFO
snapshot.SnapshotManager (DirectoryDiffListFactory.java:init(43)) - SkipList is
disabled
2020-10-16 23:05:16,273 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(395)) - Computing capacity for map
cachedBlocks
2020-10-16 23:05:16,273 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(396)) - VM type = 64-bit
2020-10-16 23:05:16,273 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(397)) - 0.25% max memory 1.8 GB = 4.6 MB
2020-10-16 23:05:16,273 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(402)) - capacity = 2^19 = 524288
entries
2020-10-16 23:05:16,274 [Listener at localhost/35179] INFO metrics.TopMetrics
(TopMetrics.java:logConf(76)) - NNTop conf: dfs.namenode.top.window.num.buckets
= 10
2020-10-16 23:05:16,274 [Listener at localhost/35179] INFO metrics.TopMetrics
(TopMetrics.java:logConf(78)) - NNTop conf: dfs.namenode.top.num.users = 10
2020-10-16 23:05:16,274 [Listener at localhost/35179] INFO metrics.TopMetrics
(TopMetrics.java:logConf(80)) - NNTop conf: dfs.namenode.top.windows.minutes =
1,5,25
2020-10-16 23:05:16,274 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:initRetryCache(1105)) - Retry cache on
namenode is enabled
2020-10-16 23:05:16,274 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:initRetryCache(1113)) - Retry cache
will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2020-10-16 23:05:16,274 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(395)) - Computing capacity for map
NameNodeRetryCache
2020-10-16 23:05:16,274 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(396)) - VM type = 64-bit
2020-10-16 23:05:16,275 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(397)) - 0.029999999329447746% max memory
1.8 GB = 559.3 KB
2020-10-16 23:05:16,275 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(402)) - capacity = 2^16 = 65536
entries
2020-10-16 23:05:16,275 [Listener at localhost/35179] INFO namenode.FSImage
(FSImage.java:format(185)) - Allocated new BlockPoolId:
BP-523381985-172.17.0.2-1602889516275
2020-10-16 23:05:16,282 [Listener at localhost/35179] INFO common.Storage
(NNStorage.java:format(595)) - Storage directory
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1
has been successfully formatted.
2020-10-16 23:05:16,300 [Listener at localhost/35179] INFO common.Storage
(NNStorage.java:format(595)) - Storage directory
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-2
has been successfully formatted.
2020-10-16 23:05:16,311 [FSImageSaver for
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1
of type IMAGE_AND_EDITS] INFO namenode.FSImageFormatProtobuf
(FSImageFormatProtobuf.java:save(717)) - Saving image file
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1/current/fsimage.ckpt_0000000000000000000
using no compression
2020-10-16 23:05:16,311 [FSImageSaver for
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-2
of type IMAGE_AND_EDITS] INFO namenode.FSImageFormatProtobuf
(FSImageFormatProtobuf.java:save(717)) - Saving image file
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-2/current/fsimage.ckpt_0000000000000000000
using no compression
2020-10-16 23:05:16,320 [FSImageSaver for
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-2
of type IMAGE_AND_EDITS] INFO namenode.FSImageFormatProtobuf
(FSImageFormatProtobuf.java:save(721)) - Image file
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-2/current/fsimage.ckpt_0000000000000000000
of size 454 bytes saved in 0 seconds .
2020-10-16 23:05:16,320 [FSImageSaver for
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1
of type IMAGE_AND_EDITS] INFO namenode.FSImageFormatProtobuf
(FSImageFormatProtobuf.java:save(721)) - Image file
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1/current/fsimage.ckpt_0000000000000000000
of size 454 bytes saved in 0 seconds .
2020-10-16 23:05:16,324 [Listener at localhost/35179] INFO
namenode.NNStorageRetentionManager
(NNStorageRetentionManager.java:getImageTxIdToRetain(203)) - Going to retain 1
images with txid >= 0
2020-10-16 23:05:16,326 [Listener at localhost/35179] INFO namenode.NameNode
(NameNode.java:createNameNode(1700)) - createNameNode []
2020-10-16 23:05:16,327 [Listener at localhost/35179] INFO impl.MetricsConfig
(MetricsConfig.java:loadFirst(120)) - Loaded properties from
hadoop-metrics2-namenode.properties
2020-10-16 23:05:16,329 [Listener at localhost/35179] INFO
impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(378)) - Scheduled
Metric snapshot period at 1 second(s).
2020-10-16 23:05:16,329 [Listener at localhost/35179] INFO
impl.MetricsSystemImpl (MetricsSystemImpl.java:start(191)) - NameNode metrics
system started
2020-10-16 23:05:16,330 [Listener at localhost/35179] INFO
namenode.NameNodeUtils (NameNodeUtils.java:getClientNamenodeAddress(79)) -
fs.defaultFS is hdfs://127.0.0.1:0
2020-10-16 23:05:16,344
[org.apache.hadoop.util.JvmPauseMonitor$Monitor@ab24484] INFO
util.JvmPauseMonitor (JvmPauseMonitor.java:run(188)) - Starting JVM pause
monitor
2020-10-16 23:05:16,345 [Listener at localhost/35179] INFO hdfs.DFSUtil
(DFSUtil.java:httpServerTemplateForNNAndJN(1668)) - Filter initializers set :
org.apache.hadoop.http.lib.StaticUserWebFilter,org.apache.hadoop.hdfs.web.AuthFilterInitializer
2020-10-16 23:05:16,345 [Listener at localhost/35179] INFO hdfs.DFSUtil
(DFSUtil.java:httpServerTemplateForNNAndJN(1690)) - Starting Web-server for
hdfs at: http://localhost:0
2020-10-16 23:05:16,345 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:getWebAppsPath(1141)) - Web server is in development mode.
Resources will be read from the source tree.
2020-10-16 23:05:16,346 [Listener at localhost/35179] INFO
server.AuthenticationFilter
(AuthenticationFilter.java:constructSecretProvider(240)) - Unable to initialize
FileSignerSecretProvider, falling back to use random secrets.
2020-10-16 23:05:16,347 [Listener at localhost/35179] INFO http.HttpRequestLog
(HttpRequestLog.java:getRequestLog(82)) - Http request log for
http.requests.namenode is not defined
2020-10-16 23:05:16,347 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:getWebAppsPath(1141)) - Web server is in development mode.
Resources will be read from the source tree.
2020-10-16 23:05:16,349 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:addGlobalFilter(1059)) - Added global filter 'safety'
(class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2020-10-16 23:05:16,349 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:addFilter(1032)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to
context hdfs
2020-10-16 23:05:16,349 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:addFilter(1042)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to
context static
2020-10-16 23:05:16,349 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:addFilter(1042)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to
context logs
2020-10-16 23:05:16,351 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:addFilter(1032)) - Added filter AuthFilter
(class=org.apache.hadoop.hdfs.web.AuthFilter) to context hdfs
2020-10-16 23:05:16,351 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:addFilter(1042)) - Added filter AuthFilter
(class=org.apache.hadoop.hdfs.web.AuthFilter) to context static
2020-10-16 23:05:16,351 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:addFilter(1042)) - Added filter AuthFilter
(class=org.apache.hadoop.hdfs.web.AuthFilter) to context logs
2020-10-16 23:05:16,351 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:addJerseyResourcePackage(886)) - addJerseyResourcePackage:
packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources,
pathSpec=/webhdfs/v1/*
2020-10-16 23:05:16,352 [Listener at localhost/35179] INFO http.HttpServer2
(HttpServer2.java:bindListener(1280)) - Jetty bound to port 38731
2020-10-16 23:05:16,352 [Listener at localhost/35179] INFO server.Server
(Server.java:doStart(359)) - jetty-9.4.20.v20190813; built:
2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm
1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
2020-10-16 23:05:16,354 [Listener at localhost/35179] INFO server.session
(DefaultSessionIdManager.java:doStart(333)) - DefaultSessionIdManager
workerName=node0
2020-10-16 23:05:16,354 [Listener at localhost/35179] INFO server.session
(DefaultSessionIdManager.java:doStart(338)) - No SessionScavenger set, using
defaults
2020-10-16 23:05:16,354 [Listener at localhost/35179] INFO server.session
(HouseKeeper.java:startScavenging(140)) - node0 Scavenging every 660000ms
2020-10-16 23:05:16,355 [Listener at localhost/35179] INFO
server.AuthenticationFilter
(AuthenticationFilter.java:constructSecretProvider(240)) - Unable to initialize
FileSignerSecretProvider, falling back to use random secrets.
2020-10-16 23:05:16,356 [Listener at localhost/35179] INFO
handler.ContextHandler (ContextHandler.java:doStart(824)) - Started
o.e.j.s.ServletContextHandler@9f86dc3{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,AVAILABLE}
2020-10-16 23:05:16,356 [Listener at localhost/35179] INFO
handler.ContextHandler (ContextHandler.java:doStart(824)) - Started
o.e.j.s.ServletContextHandler@662e682a{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,AVAILABLE}
2020-10-16 23:05:16,360 [Listener at localhost/35179] INFO
handler.ContextHandler (ContextHandler.java:doStart(824)) - Started
o.e.j.w.WebAppContext@5820070c{hdfs,/,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/,AVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs}
2020-10-16 23:05:16,364 [Listener at localhost/35179] INFO
server.AbstractConnector (AbstractConnector.java:doStart(293)) - Started
ServerConnector@2fd72332{HTTP/1.1,[http/1.1]}{localhost:38731}
2020-10-16 23:05:16,364 [Listener at localhost/35179] INFO server.Server
(Server.java:doStart(399)) - Started @22037ms
2020-10-16 23:05:16,366 [Listener at localhost/35179] INFO namenode.FSEditLog
(FSEditLog.java:newInstance(229)) - Edit logging is async:true
2020-10-16 23:05:16,374 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(828)) - KeyProvider: null
2020-10-16 23:05:16,374 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystemLock.java:<init>(131)) - fsLock is fair: true
2020-10-16 23:05:16,374 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystemLock.java:<init>(149)) - Detailed lock hold
time metrics enabled: false
2020-10-16 23:05:16,374 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(861)) - fsOwner
= jenkins (auth:SIMPLE)
2020-10-16 23:05:16,375 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(862)) - supergroup
= supergroup
2020-10-16 23:05:16,375 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(863)) - isPermissionEnabled
= true
2020-10-16 23:05:16,375 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(864)) - isStoragePolicyEnabled
= true
2020-10-16 23:05:16,375 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:<init>(875)) - HA Enabled: false
2020-10-16 23:05:16,375 [Listener at localhost/35179] INFO common.Util
(Util.java:isDiskStatsEnabled(395)) -
dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO
profiling
2020-10-16 23:05:16,375 [Listener at localhost/35179] INFO
blockmanagement.DatanodeManager (DatanodeManager.java:<init>(309)) -
dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2020-10-16 23:05:16,376 [Listener at localhost/35179] INFO
blockmanagement.DatanodeManager (DatanodeManager.java:<init>(317)) -
dfs.namenode.datanode.registration.ip-hostname-check=true
2020-10-16 23:05:16,376 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (InvalidateBlocks.java:printBlockDeletionTime(77))
- dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2020-10-16 23:05:16,376 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (InvalidateBlocks.java:printBlockDeletionTime(83))
- The block deletion will start around 2020 Oct 16 23:05:16
2020-10-16 23:05:16,376 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(395)) - Computing capacity for map
BlocksMap
2020-10-16 23:05:16,376 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(396)) - VM type = 64-bit
2020-10-16 23:05:16,377 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(397)) - 2.0% max memory 1.8 GB = 36.4 MB
2020-10-16 23:05:16,377 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(402)) - capacity = 2^22 = 4194304
entries
2020-10-16 23:05:16,379 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:createSPSManager(5439)) -
Storage policy satisfier is disabled
2020-10-16 23:05:16,379 [Listener at localhost/35179] INFO
blockmanagement.BlockManager
(BlockManager.java:createBlockTokenSecretManager(641)) -
dfs.block.access.token.enable = false
2020-10-16 23:05:16,380 [Listener at localhost/35179] INFO
blockmanagement.BlockManagerSafeMode (BlockManagerSafeMode.java:<init>(160)) -
dfs.namenode.safemode.threshold-pct = 0.999
2020-10-16 23:05:16,380 [Listener at localhost/35179] INFO
blockmanagement.BlockManagerSafeMode (BlockManagerSafeMode.java:<init>(161)) -
dfs.namenode.safemode.min.datanodes = 0
2020-10-16 23:05:16,380 [Listener at localhost/35179] INFO
blockmanagement.BlockManagerSafeMode (BlockManagerSafeMode.java:<init>(163)) -
dfs.namenode.safemode.extension = 0
2020-10-16 23:05:16,380 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(627)) -
defaultReplication = 3
2020-10-16 23:05:16,380 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(628)) - maxReplication
= 512
2020-10-16 23:05:16,380 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(629)) - minReplication
= 1
2020-10-16 23:05:16,380 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(630)) -
maxReplicationStreams = 2
2020-10-16 23:05:16,381 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(631)) -
redundancyRecheckInterval = 1000ms
2020-10-16 23:05:16,381 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(632)) -
encryptDataTransfer = false
2020-10-16 23:05:16,381 [Listener at localhost/35179] INFO
blockmanagement.BlockManager (BlockManager.java:<init>(633)) -
maxNumBlocksToLog = 1000
2020-10-16 23:05:16,381 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(395)) - Computing capacity for map
INodeMap
2020-10-16 23:05:16,381 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(396)) - VM type = 64-bit
2020-10-16 23:05:16,382 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(397)) - 1.0% max memory 1.8 GB = 18.2 MB
2020-10-16 23:05:16,382 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(402)) - capacity = 2^21 = 2097152
entries
2020-10-16 23:05:16,383 [Listener at localhost/35179] INFO
namenode.FSDirectory (FSDirectory.java:<init>(335)) - ACLs enabled? true
2020-10-16 23:05:16,383 [Listener at localhost/35179] INFO
namenode.FSDirectory (FSDirectory.java:<init>(339)) - POSIX ACL inheritance
enabled? true
2020-10-16 23:05:16,383 [Listener at localhost/35179] INFO
namenode.FSDirectory (FSDirectory.java:<init>(343)) - XAttrs enabled? true
2020-10-16 23:05:16,384 [Listener at localhost/35179] INFO namenode.NameNode
(FSDirectory.java:<init>(406)) - Caching file names occurring more than 10 times
2020-10-16 23:05:16,384 [Listener at localhost/35179] INFO
snapshot.SnapshotManager (SnapshotManager.java:<init>(162)) - Loaded config
captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false,
snapshotDiffAllowSnapRootDescendant: true, maxSnapshotFSLimit: 65536,
maxSnapshotLimit: 65536
2020-10-16 23:05:16,384 [Listener at localhost/35179] INFO
snapshot.SnapshotManager (SnapshotManager.java:<init>(175)) -
dfs.namenode.snapshot.deletion.ordered = false
2020-10-16 23:05:16,384 [Listener at localhost/35179] INFO
snapshot.SnapshotManager (DirectoryDiffListFactory.java:init(43)) - SkipList is
disabled
2020-10-16 23:05:16,384 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(395)) - Computing capacity for map
cachedBlocks
2020-10-16 23:05:16,384 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(396)) - VM type = 64-bit
2020-10-16 23:05:16,385 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(397)) - 0.25% max memory 1.8 GB = 4.6 MB
2020-10-16 23:05:16,385 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(402)) - capacity = 2^19 = 524288
entries
2020-10-16 23:05:16,385 [Listener at localhost/35179] INFO metrics.TopMetrics
(TopMetrics.java:logConf(76)) - NNTop conf: dfs.namenode.top.window.num.buckets
= 10
2020-10-16 23:05:16,385 [Listener at localhost/35179] INFO metrics.TopMetrics
(TopMetrics.java:logConf(78)) - NNTop conf: dfs.namenode.top.num.users = 10
2020-10-16 23:05:16,385 [Listener at localhost/35179] INFO metrics.TopMetrics
(TopMetrics.java:logConf(80)) - NNTop conf: dfs.namenode.top.windows.minutes =
1,5,25
2020-10-16 23:05:16,386 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:initRetryCache(1105)) - Retry cache on
namenode is enabled
2020-10-16 23:05:16,386 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:initRetryCache(1113)) - Retry cache
will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2020-10-16 23:05:16,386 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(395)) - Computing capacity for map
NameNodeRetryCache
2020-10-16 23:05:16,386 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(396)) - VM type = 64-bit
2020-10-16 23:05:16,386 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(397)) - 0.029999999329447746% max memory
1.8 GB = 559.3 KB
2020-10-16 23:05:16,386 [Listener at localhost/35179] INFO util.GSet
(LightWeightGSet.java:computeCapacity(402)) - capacity = 2^16 = 65536
entries
2020-10-16 23:05:16,390 [Listener at localhost/35179] INFO common.Storage
(Storage.java:tryLock(928)) - Lock on
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1/in_use.lock
acquired by nodename 7533@87df745ebcb4
2020-10-16 23:05:16,391 [Listener at localhost/35179] INFO common.Storage
(Storage.java:tryLock(928)) - Lock on
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-2/in_use.lock
acquired by nodename 7533@87df745ebcb4
2020-10-16 23:05:16,393 [Listener at localhost/35179] INFO
namenode.FileJournalManager
(FileJournalManager.java:recoverUnfinalizedSegments(428)) - Recovering
unfinalized segments in
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1/current
2020-10-16 23:05:16,393 [Listener at localhost/35179] INFO
namenode.FileJournalManager
(FileJournalManager.java:recoverUnfinalizedSegments(428)) - Recovering
unfinalized segments in
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-2/current
2020-10-16 23:05:16,393 [Listener at localhost/35179] INFO namenode.FSImage
(FSImage.java:loadFSImage(733)) - No edit log streams selected.
2020-10-16 23:05:16,393 [Listener at localhost/35179] INFO namenode.FSImage
(FSImage.java:loadFSImageFile(799)) - Planning to load image:
FSImageFile(file=/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1/current/fsimage_0000000000000000000,
cpktTxId=0000000000000000000)
2020-10-16 23:05:16,395 [Listener at localhost/35179] INFO
namenode.FSImageFormatPBINode
(FSImageFormatPBINode.java:loadINodeSectionHeader(413)) - Loading 1 INodes.
2020-10-16 23:05:16,401 [Listener at localhost/35179] INFO
namenode.FSImageFormatPBINode (FSImageFormatPBINode.java:loadINodeSection(371))
- Successfully loaded 1 inodes
2020-10-16 23:05:16,401 [Listener at localhost/35179] INFO
namenode.FSImageFormatPBINode
(FSImageFormatPBINode.java:waitBlocksMapAndNameCacheUpdateFinished(344)) -
Completed update blocks map and name cache, total waiting duration 0ms.
2020-10-16 23:05:16,402 [Listener at localhost/35179] INFO
namenode.FSImageFormatProtobuf (FSImageFormatProtobuf.java:load(253)) - Loaded
FSImage in 0 seconds.
2020-10-16 23:05:16,402 [Listener at localhost/35179] INFO namenode.FSImage
(FSImage.java:loadFSImage(977)) - Loaded image for txid 0 from
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1/current/fsimage_0000000000000000000
2020-10-16 23:05:16,402 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:loadFSImage(1227)) - Need to save fs
image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
2020-10-16 23:05:16,403 [Listener at localhost/35179] INFO namenode.FSEditLog
(FSEditLog.java:startLogSegment(1391)) - Starting log segment at 1
2020-10-16 23:05:16,422 [Listener at localhost/35179] INFO namenode.NameCache
(NameCache.java:initialized(143)) - initialized with 0 entries 0 lookups
2020-10-16 23:05:16,422 [Listener at localhost/35179] INFO
namenode.FSNamesystem (FSNamesystem.java:loadFromDisk(800)) - Finished loading
FSImage in 35 msecs
2020-10-16 23:05:16,423 [Listener at localhost/35179] INFO namenode.NameNode
(NameNodeRpcServer.java:<init>(454)) - RPC server is binding to localhost:0
2020-10-16 23:05:16,423 [Listener at localhost/35179] INFO namenode.NameNode
(NameNodeRpcServer.java:<init>(459)) - Enable NameNode state context:false
2020-10-16 23:05:16,423 [Listener at localhost/35179] INFO
ipc.CallQueueManager (CallQueueManager.java:<init>(92)) - Using callQueue:
class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 1000, scheduler:
class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2020-10-16 23:05:16,424 [Socket Reader #1 for port 0] INFO ipc.Server
(Server.java:run(1250)) - Starting Socket Reader #1 for port 0
2020-10-16 23:05:16,438 [Listener at localhost/39159] INFO namenode.NameNode
(NameNode.java:initialize(774)) - Clients are to use localhost:39159 to access
this namenode/service.
2020-10-16 23:05:16,439 [Listener at localhost/39159] INFO
namenode.FSNamesystem (FSNamesystem.java:registerMBean(5508)) - Registered
FSNamesystemState, ReplicatedBlocksState and ECBlockGroupsState MBeans.
2020-10-16 23:05:16,488 [Listener at localhost/39159] INFO
namenode.LeaseManager (LeaseManager.java:getNumUnderConstructionBlocks(178)) -
Number of blocks under construction: 0
2020-10-16 23:05:16,489
[org.apache.hadoop.hdfs.server.blockmanagement.PendingReconstructionBlocks$PendingReconstructionMonitor@4735d6e5]
DEBUG blockmanagement.BlockManager
(PendingReconstructionBlocks.java:pendingReconstructionCheck(274)) -
PendingReconstructionMonitor checking Q
2020-10-16 23:05:16,489 [Listener at localhost/39159] INFO
blockmanagement.DatanodeAdminDefaultMonitor
(DatanodeAdminDefaultMonitor.java:processConf(116)) - Initialized the Default
Decommission and Maintenance monitor
2020-10-16 23:05:16,491 [Listener at localhost/39159] INFO
blockmanagement.BlockManager (BlockManager.java:initializeReplQueues(5179)) -
initializing replication queues
2020-10-16 23:05:16,491 [Listener at localhost/39159] INFO hdfs.StateChange
(BlockManagerSafeMode.java:leaveSafeMode(408)) - STATE* Leaving safe mode after
0 secs
2020-10-16 23:05:16,491 [Listener at localhost/39159] INFO hdfs.StateChange
(BlockManagerSafeMode.java:leaveSafeMode(414)) - STATE* Network topology has 0
racks and 0 datanodes
2020-10-16 23:05:16,491 [Listener at localhost/39159] INFO hdfs.StateChange
(BlockManagerSafeMode.java:leaveSafeMode(416)) - STATE* UnderReplicatedBlocks
has 0 blocks
2020-10-16 23:05:16,501 [Reconstruction Queue Initializer] INFO
blockmanagement.BlockManager
(BlockManager.java:processMisReplicatesAsync(3721)) - Total number of blocks
= 0
2020-10-16 23:05:16,502 [Reconstruction Queue Initializer] INFO
blockmanagement.BlockManager
(BlockManager.java:processMisReplicatesAsync(3722)) - Number of invalid blocks
= 0
2020-10-16 23:05:16,502 [Reconstruction Queue Initializer] INFO
blockmanagement.BlockManager
(BlockManager.java:processMisReplicatesAsync(3723)) - Number of
under-replicated blocks = 0
2020-10-16 23:05:16,502 [Reconstruction Queue Initializer] INFO
blockmanagement.BlockManager
(BlockManager.java:processMisReplicatesAsync(3724)) - Number of
over-replicated blocks = 0
2020-10-16 23:05:16,502 [Reconstruction Queue Initializer] INFO
blockmanagement.BlockManager
(BlockManager.java:processMisReplicatesAsync(3726)) - Number of blocks being
written = 0
2020-10-16 23:05:16,502 [Reconstruction Queue Initializer] INFO
hdfs.StateChange (BlockManager.java:processMisReplicatesAsync(3729)) - STATE*
Replication Queue initialization scan for invalid, over- and under-replicated
blocks completed in 11 msec
2020-10-16 23:05:16,503 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1492)) - IPC Server Responder: starting
2020-10-16 23:05:16,503 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1329)) - IPC Server listener on 0: starting
2020-10-16 23:05:16,528 [Listener at localhost/39159] INFO namenode.NameNode
(NameNode.java:startCommonServices(891)) - NameNode RPC up at:
localhost/127.0.0.1:39159
2020-10-16 23:05:16,528 [Listener at localhost/39159] INFO
namenode.FSNamesystem (FSNamesystem.java:startActiveServices(1339)) - Starting
services required for active state
2020-10-16 23:05:16,529 [Listener at localhost/39159] INFO
namenode.FSDirectory (FSDirectory.java:updateCountForQuota(849)) - Initializing
quota with 4 thread(s)
2020-10-16 23:05:16,530 [Listener at localhost/39159] INFO
namenode.FSDirectory (FSDirectory.java:updateCountForQuota(858)) - Quota
initialization completed in 1 milliseconds
name space=1
storage space=0
storage types=RAM_DISK=0, NVDIMM=0, SSD=0, DISK=0, ARCHIVE=0, PROVIDED=0
2020-10-16 23:05:16,536 [CacheReplicationMonitor(1697427407)] INFO
blockmanagement.CacheReplicationMonitor (CacheReplicationMonitor.java:run(160))
- Starting CacheReplicationMonitor with interval 30000 milliseconds
2020-10-16 23:05:16,543 [Listener at localhost/39159] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:startDataNodes(1701)) - Starting DataNode 0 with
dfs.datanode.data.dir:
[DISK]file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data1,[DISK]file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data2
2020-10-16 23:05:16,544 [Listener at localhost/39159] INFO
checker.ThrottledAsyncChecker (ThrottledAsyncChecker.java:schedule(137)) -
Scheduling a check for
[DISK]file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data1
2020-10-16 23:05:16,544 [Listener at localhost/39159] INFO
checker.ThrottledAsyncChecker (ThrottledAsyncChecker.java:schedule(137)) -
Scheduling a check for
[DISK]file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data2
2020-10-16 23:05:16,546 [Listener at localhost/39159] INFO
impl.MetricsSystemImpl (MetricsSystemImpl.java:init(158)) - DataNode metrics
system started (again)
2020-10-16 23:05:16,546 [Listener at localhost/39159] INFO common.Util
(Util.java:isDiskStatsEnabled(395)) -
dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO
profiling
2020-10-16 23:05:16,547 [Listener at localhost/39159] INFO
datanode.BlockScanner (BlockScanner.java:<init>(201)) - Initialized block
scanner with targetBytesPerSec 1048576
2020-10-16 23:05:16,547 [Listener at localhost/39159] INFO datanode.DataNode
(DataNode.java:<init>(512)) - Configured hostname is 127.0.0.1
2020-10-16 23:05:16,547 [Listener at localhost/39159] INFO common.Util
(Util.java:isDiskStatsEnabled(395)) -
dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO
profiling
2020-10-16 23:05:16,548 [Listener at localhost/39159] INFO datanode.DataNode
(DataNode.java:startDataNode(1422)) - Starting DataNode with maxLockedMemory = 0
2020-10-16 23:05:16,548 [Listener at localhost/39159] INFO datanode.DataNode
(DataNode.java:initDataXceiver(1170)) - Opened streaming server at
/127.0.0.1:36247
2020-10-16 23:05:16,549 [Listener at localhost/39159] INFO datanode.DataNode
(DataXceiverServer.java:<init>(94)) - Balancing bandwidth is 104857600 bytes/s
2020-10-16 23:05:16,549 [Listener at localhost/39159] INFO datanode.DataNode
(DataXceiverServer.java:<init>(95)) - Number threads for balancing is 100
2020-10-16 23:05:16,550 [Listener at localhost/39159] INFO http.HttpServer2
(HttpServer2.java:getWebAppsPath(1141)) - Web server is in development mode.
Resources will be read from the source tree.
2020-10-16 23:05:16,563 [Listener at localhost/39159] INFO
server.AuthenticationFilter
(AuthenticationFilter.java:constructSecretProvider(240)) - Unable to initialize
FileSignerSecretProvider, falling back to use random secrets.
2020-10-16 23:05:16,564 [Listener at localhost/39159] INFO http.HttpRequestLog
(HttpRequestLog.java:getRequestLog(82)) - Http request log for
http.requests.datanode is not defined
2020-10-16 23:05:16,564 [Listener at localhost/39159] INFO http.HttpServer2
(HttpServer2.java:getWebAppsPath(1141)) - Web server is in development mode.
Resources will be read from the source tree.
2020-10-16 23:05:16,567 [Listener at localhost/39159] INFO http.HttpServer2
(HttpServer2.java:addGlobalFilter(1059)) - Added global filter 'safety'
(class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2020-10-16 23:05:16,568 [Listener at localhost/39159] INFO http.HttpServer2
(HttpServer2.java:addFilter(1032)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to
context datanode
2020-10-16 23:05:16,568 [Listener at localhost/39159] INFO http.HttpServer2
(HttpServer2.java:addFilter(1042)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to
context static
2020-10-16 23:05:16,568 [Listener at localhost/39159] INFO http.HttpServer2
(HttpServer2.java:addFilter(1042)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to
context logs
2020-10-16 23:05:16,569 [Listener at localhost/39159] INFO http.HttpServer2
(HttpServer2.java:bindListener(1280)) - Jetty bound to port 39633
2020-10-16 23:05:16,569 [Listener at localhost/39159] INFO server.Server
(Server.java:doStart(359)) - jetty-9.4.20.v20190813; built:
2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm
1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
2020-10-16 23:05:16,571 [Listener at localhost/39159] INFO server.session
(DefaultSessionIdManager.java:doStart(333)) - DefaultSessionIdManager
workerName=node0
2020-10-16 23:05:16,571 [Listener at localhost/39159] INFO server.session
(DefaultSessionIdManager.java:doStart(338)) - No SessionScavenger set, using
defaults
2020-10-16 23:05:16,572 [Listener at localhost/39159] INFO server.session
(HouseKeeper.java:startScavenging(140)) - node0 Scavenging every 660000ms
2020-10-16 23:05:16,573 [Listener at localhost/39159] INFO
handler.ContextHandler (ContextHandler.java:doStart(824)) - Started
o.e.j.s.ServletContextHandler@4e3e11b9{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,AVAILABLE}
2020-10-16 23:05:16,573 [Listener at localhost/39159] INFO
handler.ContextHandler (ContextHandler.java:doStart(824)) - Started
o.e.j.s.ServletContextHandler@214beff9{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,AVAILABLE}
2020-10-16 23:05:16,580 [Listener at localhost/39159] INFO
handler.ContextHandler (ContextHandler.java:doStart(824)) - Started
o.e.j.w.WebAppContext@45b32dfe{datanode,/,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/,AVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode}
2020-10-16 23:05:16,581 [Listener at localhost/39159] INFO
server.AbstractConnector (AbstractConnector.java:doStart(293)) - Started
ServerConnector@5fcd1c5f{HTTP/1.1,[http/1.1]}{localhost:39633}
2020-10-16 23:05:16,582 [Listener at localhost/39159] INFO server.Server
(Server.java:doStart(399)) - Started @22254ms
2020-10-16 23:05:16,590 [Listener at localhost/39159] WARN
web.DatanodeHttpServer (RestCsrfPreventionFilterHandler.java:<init>(75)) - Got
null for restCsrfPreventionFilter - will not do any filtering.
2020-10-16 23:05:16,597 [Listener at localhost/39159] INFO
web.DatanodeHttpServer (DatanodeHttpServer.java:start(322)) - Listening HTTP
traffic on /127.0.0.1:38979
2020-10-16 23:05:16,598 [Listener at localhost/39159] INFO datanode.DataNode
(DataNode.java:startDataNode(1450)) - dnUserName = jenkins
2020-10-16 23:05:16,598 [Listener at localhost/39159] INFO datanode.DataNode
(DataNode.java:startDataNode(1451)) - supergroup = supergroup
2020-10-16 23:05:16,598
[org.apache.hadoop.util.JvmPauseMonitor$Monitor@566f1852] INFO
util.JvmPauseMonitor (JvmPauseMonitor.java:run(188)) - Starting JVM pause
monitor
2020-10-16 23:05:16,598 [Listener at localhost/39159] INFO
ipc.CallQueueManager (CallQueueManager.java:<init>(92)) - Using callQueue:
class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 1000, scheduler:
class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2020-10-16 23:05:16,599 [Socket Reader #1 for port 0] INFO ipc.Server
(Server.java:run(1250)) - Starting Socket Reader #1 for port 0
2020-10-16 23:05:16,603 [Listener at localhost/42577] INFO datanode.DataNode
(DataNode.java:initIpcServer(1057)) - Opened IPC server at /127.0.0.1:42577
2020-10-16 23:05:16,620 [Listener at localhost/42577] INFO datanode.DataNode
(BlockPoolManager.java:refreshNamenodes(149)) - Refresh request received for
nameservices: null
2020-10-16 23:05:16,620 [Listener at localhost/42577] INFO datanode.DataNode
(BlockPoolManager.java:doRefreshNamenodes(210)) - Starting BPOfferServices for
nameservices: <default>
2020-10-16 23:05:16,621 [Thread-1724] INFO datanode.DataNode
(BPServiceActor.java:run(846)) - Block pool <registering> (Datanode Uuid
unassigned) service to localhost/127.0.0.1:39159 starting to offer service
2020-10-16 23:05:16,625 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1492)) - IPC Server Responder: starting
2020-10-16 23:05:16,625 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1329)) - IPC Server listener on 0: starting
2020-10-16 23:05:16,630 [Listener at localhost/42577] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:startDataNodes(1701)) - Starting DataNode 1 with
dfs.datanode.data.dir:
[DISK]file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data3,[DISK]file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data4
2020-10-16 23:05:16,630 [Thread-1724] INFO datanode.DataNode
(BPOfferService.java:verifyAndSetNamespaceInfo(380)) - Acknowledging ACTIVE
Namenode during handshakeBlock pool <registering> (Datanode Uuid unassigned)
service to localhost/127.0.0.1:39159
2020-10-16 23:05:16,632 [Thread-1724] INFO common.Storage
(DataStorage.java:getParallelVolumeLoadThreadsNum(356)) - Using 2 threads to
upgrade data directories (dfs.datanode.parallel.volumes.load.threads.num=2,
dataDirs=2)
2020-10-16 23:05:16,633 [Listener at localhost/42577] INFO
checker.ThrottledAsyncChecker (ThrottledAsyncChecker.java:schedule(137)) -
Scheduling a check for
[DISK]file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data3
2020-10-16 23:05:16,633 [Listener at localhost/42577] INFO
checker.ThrottledAsyncChecker (ThrottledAsyncChecker.java:schedule(137)) -
Scheduling a check for
[DISK]file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data4
2020-10-16 23:05:16,635 [Listener at localhost/42577] INFO
impl.MetricsSystemImpl (MetricsSystemImpl.java:init(158)) - DataNode metrics
system started (again)
2020-10-16 23:05:16,636 [Listener at localhost/42577] INFO common.Util
(Util.java:isDiskStatsEnabled(395)) -
dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO
profiling
2020-10-16 23:05:16,636 [Listener at localhost/42577] INFO
datanode.BlockScanner (BlockScanner.java:<init>(201)) - Initialized block
scanner with targetBytesPerSec 1048576
2020-10-16 23:05:16,636 [Listener at localhost/42577] INFO datanode.DataNode
(DataNode.java:<init>(512)) - Configured hostname is 127.0.0.1
2020-10-16 23:05:16,636 [Listener at localhost/42577] INFO common.Util
(Util.java:isDiskStatsEnabled(395)) -
dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO
profiling
2020-10-16 23:05:16,637 [Listener at localhost/42577] INFO datanode.DataNode
(DataNode.java:startDataNode(1422)) - Starting DataNode with maxLockedMemory = 0
2020-10-16 23:05:16,637 [Thread-1724] INFO common.Storage
(Storage.java:tryLock(928)) - Lock on
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data1/in_use.lock
acquired by nodename 7533@87df745ebcb4
2020-10-16 23:05:16,637 [Listener at localhost/42577] INFO datanode.DataNode
(DataNode.java:initDataXceiver(1170)) - Opened streaming server at
/127.0.0.1:46669
2020-10-16 23:05:16,637 [Thread-1724] INFO common.Storage
(DataStorage.java:loadStorageDirectory(284)) - Storage directory with location
[DISK]file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data1
is not formatted for namespace 1205853636. Formatting...
2020-10-16 23:05:16,637 [Listener at localhost/42577] INFO datanode.DataNode
(DataXceiverServer.java:<init>(94)) - Balancing bandwidth is 104857600 bytes/s
2020-10-16 23:05:16,638 [Thread-1724] INFO common.Storage
(DataStorage.java:createStorageID(160)) - Generated new storageID
DS-851e14b2-1e07-4c18-9d3a-9f9336d0c3e2 for directory
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data1
2020-10-16 23:05:16,638 [Listener at localhost/42577] INFO datanode.DataNode
(DataXceiverServer.java:<init>(95)) - Number threads for balancing is 100
2020-10-16 23:05:16,639 [Listener at localhost/42577] INFO http.HttpServer2
(HttpServer2.java:getWebAppsPath(1141)) - Web server is in development mode.
Resources will be read from the source tree.
2020-10-16 23:05:16,640 [Listener at localhost/42577] INFO
server.AuthenticationFilter
(AuthenticationFilter.java:constructSecretProvider(240)) - Unable to initialize
FileSignerSecretProvider, falling back to use random secrets.
2020-10-16 23:05:16,640 [Listener at localhost/42577] INFO http.HttpRequestLog
(HttpRequestLog.java:getRequestLog(82)) - Http request log for
http.requests.datanode is not defined
2020-10-16 23:05:16,641 [Listener at localhost/42577] INFO http.HttpServer2
(HttpServer2.java:getWebAppsPath(1141)) - Web server is in development mode.
Resources will be read from the source tree.
2020-10-16 23:05:16,642 [Listener at localhost/42577] INFO http.HttpServer2
(HttpServer2.java:addGlobalFilter(1059)) - Added global filter 'safety'
(class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2020-10-16 23:05:16,643 [Listener at localhost/42577] INFO http.HttpServer2
(HttpServer2.java:addFilter(1032)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to
context datanode
2020-10-16 23:05:16,643 [Listener at localhost/42577] INFO http.HttpServer2
(HttpServer2.java:addFilter(1042)) - Added filter static_user_filter
(class=org.apache.hadoop.ht
...[truncated 13066090 chars]...
{datanode,/,null,UNAVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode}
2020-10-16 23:05:47,202 [Listener at localhost/39255] INFO
server.AbstractConnector (AbstractConnector.java:doStop(343)) - Stopped
ServerConnector@1c5f94a1{HTTP/1.1,[http/1.1]}{localhost:0}
2020-10-16 23:05:47,203 [Listener at localhost/39255] INFO server.session
(HouseKeeper.java:stopScavenging(158)) - node0 Stopped scavenging
2020-10-16 23:05:47,205 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@1ab30db7{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,UNAVAILABLE}
2020-10-16 23:05:47,210 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@77926c4a{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,UNAVAILABLE}
2020-10-16 23:05:47,214 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2104)) - Waiting up to 30 seconds for transfer threads
to complete
2020-10-16 23:05:47,214 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 37479
2020-10-16 23:05:47,214 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1361)) - Stopping IPC Server listener on 0
2020-10-16 23:05:47,215 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1497)) - Stopping IPC Server Responder
2020-10-16 23:05:47,215 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.IncrementalBlockReportManager
(IncrementalBlockReportManager.java:waitTillNextIBR(160)) -
IncrementalBlockReportManager interrupted
2020-10-16 23:05:47,215 [Command processor] ERROR datanode.DataNode
(BPServiceActor.java:processQueue(1329)) - Command processor encountered
interrupt and exit.
2020-10-16 23:05:47,215 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.DataNode
(BPServiceActor.java:run(889)) - Ending block pool service for: Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
e4172dab-86c8-4fa5-b0a4-deaacb38089d) service to localhost/127.0.0.1:46689
2020-10-16 23:05:47,216 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO datanode.DataNode
(BlockPoolManager.java:remove(102)) - Removed Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
e4172dab-86c8-4fa5-b0a4-deaacb38089d)
2020-10-16 23:05:47,216 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO impl.FsDatasetImpl
(FsDatasetImpl.java:shutdownBlockPool(2946)) - Removing block pool
BP-1639228104-172.17.0.2-1602889526611
2020-10-16 23:05:47,217
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data17/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,217
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data18/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,217 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(193)) -
Shutting down all async disk service threads
2020-10-16 23:05:47,217 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(201)) -
All async disk service threads have been shut down
2020-10-16 23:05:47,218 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(177)) - Shutting down all async
lazy persist service threads
2020-10-16 23:05:47,218 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(184)) - All async lazy persist
service threads have been shut down
2020-10-16 23:05:47,218 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2193)) - Shutdown complete.
2020-10-16 23:05:47,218 [Listener at localhost/39255] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2139)) - Shutting down DataNode 7
2020-10-16 23:05:47,218 [Listener at localhost/39255] INFO
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(392)) - Shutdown has
been called
2020-10-16 23:05:47,219
[org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@4f7c40bb] INFO
datanode.DataNode (DataXceiverServer.java:closeAllPeers(396)) - Closing all
peers.
2020-10-16 23:05:47,219
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data16)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data16,
DS-d4fd7f71-0a95-4b89-8995-ee4edbbfc7c8) exiting.
2020-10-16 23:05:47,219
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data15)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data15,
DS-6a420396-23a3-450a-84b7-02d8cecf2dfa) exiting.
2020-10-16 23:05:47,261 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.w.WebAppContext@45f2b938{datanode,/,null,UNAVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode}
2020-10-16 23:05:47,262 [Listener at localhost/39255] INFO
server.AbstractConnector (AbstractConnector.java:doStop(343)) - Stopped
ServerConnector@eb4a643{HTTP/1.1,[http/1.1]}{localhost:0}
2020-10-16 23:05:47,262 [Listener at localhost/39255] INFO server.session
(HouseKeeper.java:stopScavenging(158)) - node0 Stopped scavenging
2020-10-16 23:05:47,264 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@7017db7d{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,UNAVAILABLE}
2020-10-16 23:05:47,267 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@7aff65ef{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,UNAVAILABLE}
2020-10-16 23:05:47,269 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2104)) - Waiting up to 30 seconds for transfer threads
to complete
2020-10-16 23:05:47,269 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 37823
2020-10-16 23:05:47,270 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1361)) - Stopping IPC Server listener on 0
2020-10-16 23:05:47,270 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1497)) - Stopping IPC Server Responder
2020-10-16 23:05:47,271 [Command processor] ERROR datanode.DataNode
(BPServiceActor.java:processQueue(1329)) - Command processor encountered
interrupt and exit.
2020-10-16 23:05:47,271 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.IncrementalBlockReportManager
(IncrementalBlockReportManager.java:waitTillNextIBR(160)) -
IncrementalBlockReportManager interrupted
2020-10-16 23:05:47,271 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.DataNode
(BPServiceActor.java:run(889)) - Ending block pool service for: Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
9b7d0727-3eb2-4bc5-81a2-de6293d4aa25) service to localhost/127.0.0.1:46689
2020-10-16 23:05:47,271 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO datanode.DataNode
(BlockPoolManager.java:remove(102)) - Removed Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
9b7d0727-3eb2-4bc5-81a2-de6293d4aa25)
2020-10-16 23:05:47,271 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO impl.FsDatasetImpl
(FsDatasetImpl.java:shutdownBlockPool(2946)) - Removing block pool
BP-1639228104-172.17.0.2-1602889526611
2020-10-16 23:05:47,272
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data15/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,272
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data16/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,272 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(193)) -
Shutting down all async disk service threads
2020-10-16 23:05:47,273 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(201)) -
All async disk service threads have been shut down
2020-10-16 23:05:47,273 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(177)) - Shutting down all async
lazy persist service threads
2020-10-16 23:05:47,273 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(184)) - All async lazy persist
service threads have been shut down
2020-10-16 23:05:47,273 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2193)) - Shutdown complete.
2020-10-16 23:05:47,274 [Listener at localhost/39255] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2139)) - Shutting down DataNode 6
2020-10-16 23:05:47,274 [Listener at localhost/39255] INFO
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(392)) - Shutdown has
been called
2020-10-16 23:05:47,274
[org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@178bf2a0] INFO
datanode.DataNode (DataXceiverServer.java:closeAllPeers(396)) - Closing all
peers.
2020-10-16 23:05:47,274
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data14)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data14,
DS-dc2b9672-23a1-41d8-9efb-bd98697e526a) exiting.
2020-10-16 23:05:47,274
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data13)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data13,
DS-4dd40ca3-4c03-459c-ac9c-05a6a0990496) exiting.
2020-10-16 23:05:47,304 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.w.WebAppContext@700eb422{datanode,/,null,UNAVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode}
2020-10-16 23:05:47,305 [Listener at localhost/39255] INFO
server.AbstractConnector (AbstractConnector.java:doStop(343)) - Stopped
ServerConnector@34d8430c{HTTP/1.1,[http/1.1]}{localhost:0}
2020-10-16 23:05:47,305 [Listener at localhost/39255] INFO server.session
(HouseKeeper.java:stopScavenging(158)) - node0 Stopped scavenging
2020-10-16 23:05:47,314 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@270733ea{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,UNAVAILABLE}
2020-10-16 23:05:47,317 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@581520c2{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,UNAVAILABLE}
2020-10-16 23:05:47,319 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2104)) - Waiting up to 30 seconds for transfer threads
to complete
2020-10-16 23:05:47,320 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 33553
2020-10-16 23:05:47,320 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1361)) - Stopping IPC Server listener on 0
2020-10-16 23:05:47,321 [Command processor] ERROR datanode.DataNode
(BPServiceActor.java:processQueue(1329)) - Command processor encountered
interrupt and exit.
2020-10-16 23:05:47,321 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.IncrementalBlockReportManager
(IncrementalBlockReportManager.java:waitTillNextIBR(160)) -
IncrementalBlockReportManager interrupted
2020-10-16 23:05:47,321 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1497)) - Stopping IPC Server Responder
2020-10-16 23:05:47,321 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.DataNode
(BPServiceActor.java:run(889)) - Ending block pool service for: Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
a6728116-dd42-40a2-a9d5-fbd54c591e61) service to localhost/127.0.0.1:46689
2020-10-16 23:05:47,322 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO datanode.DataNode
(BlockPoolManager.java:remove(102)) - Removed Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
a6728116-dd42-40a2-a9d5-fbd54c591e61)
2020-10-16 23:05:47,322 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO impl.FsDatasetImpl
(FsDatasetImpl.java:shutdownBlockPool(2946)) - Removing block pool
BP-1639228104-172.17.0.2-1602889526611
2020-10-16 23:05:47,323
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data13/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,323
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data14/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,323 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(193)) -
Shutting down all async disk service threads
2020-10-16 23:05:47,324 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(201)) -
All async disk service threads have been shut down
2020-10-16 23:05:47,324 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(177)) - Shutting down all async
lazy persist service threads
2020-10-16 23:05:47,324 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(184)) - All async lazy persist
service threads have been shut down
2020-10-16 23:05:47,324 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2193)) - Shutdown complete.
2020-10-16 23:05:47,324 [Listener at localhost/39255] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2139)) - Shutting down DataNode 5
2020-10-16 23:05:47,325 [Listener at localhost/39255] INFO
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(392)) - Shutdown has
been called
2020-10-16 23:05:47,325
[org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@38ab69e8] INFO
datanode.DataNode (DataXceiverServer.java:closeAllPeers(396)) - Closing all
peers.
2020-10-16 23:05:47,325
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data12)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data12,
DS-481e47d0-637c-4f5a-b894-624be31bca11) exiting.
2020-10-16 23:05:47,325
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data11)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data11,
DS-6c8fc150-0520-4045-a085-c997609ea62a) exiting.
2020-10-16 23:05:47,352 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.w.WebAppContext@4fda3606{datanode,/,null,UNAVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode}
2020-10-16 23:05:47,353 [Listener at localhost/39255] INFO
server.AbstractConnector (AbstractConnector.java:doStop(343)) - Stopped
ServerConnector@799bf89d{HTTP/1.1,[http/1.1]}{localhost:0}
2020-10-16 23:05:47,353 [Listener at localhost/39255] INFO server.session
(HouseKeeper.java:stopScavenging(158)) - node0 Stopped scavenging
2020-10-16 23:05:47,353 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@7c7935b6{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,UNAVAILABLE}
2020-10-16 23:05:47,356 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@21e91a2d{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,UNAVAILABLE}
2020-10-16 23:05:47,359 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2104)) - Waiting up to 30 seconds for transfer threads
to complete
2020-10-16 23:05:47,359 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 33289
2020-10-16 23:05:47,360 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1361)) - Stopping IPC Server listener on 0
2020-10-16 23:05:47,360 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1497)) - Stopping IPC Server Responder
2020-10-16 23:05:47,361 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.IncrementalBlockReportManager
(IncrementalBlockReportManager.java:waitTillNextIBR(160)) -
IncrementalBlockReportManager interrupted
2020-10-16 23:05:47,361 [Command processor] ERROR datanode.DataNode
(BPServiceActor.java:processQueue(1329)) - Command processor encountered
interrupt and exit.
2020-10-16 23:05:47,361 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.DataNode
(BPServiceActor.java:run(889)) - Ending block pool service for: Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
722dc1f2-6d6c-479a-82d3-443580e0f6ea) service to localhost/127.0.0.1:46689
2020-10-16 23:05:47,361 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO datanode.DataNode
(BlockPoolManager.java:remove(102)) - Removed Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
722dc1f2-6d6c-479a-82d3-443580e0f6ea)
2020-10-16 23:05:47,361 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO impl.FsDatasetImpl
(FsDatasetImpl.java:shutdownBlockPool(2946)) - Removing block pool
BP-1639228104-172.17.0.2-1602889526611
2020-10-16 23:05:47,362
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data11/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,362
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data12/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,362 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(193)) -
Shutting down all async disk service threads
2020-10-16 23:05:47,363 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(201)) -
All async disk service threads have been shut down
2020-10-16 23:05:47,363 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(177)) - Shutting down all async
lazy persist service threads
2020-10-16 23:05:47,363 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(184)) - All async lazy persist
service threads have been shut down
2020-10-16 23:05:47,363 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2193)) - Shutdown complete.
2020-10-16 23:05:47,363 [Listener at localhost/39255] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2139)) - Shutting down DataNode 4
2020-10-16 23:05:47,364 [Listener at localhost/39255] INFO
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(392)) - Shutdown has
been called
2020-10-16 23:05:47,364
[org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@58619deb] INFO
datanode.DataNode (DataXceiverServer.java:closeAllPeers(396)) - Closing all
peers.
2020-10-16 23:05:47,364
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data10)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data10,
DS-a3e3bb6f-fa26-4a72-9b2a-b635ee73564e) exiting.
2020-10-16 23:05:47,364
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data9)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data9,
DS-fbe2e8e9-4e61-4e00-aa56-699774dd3028) exiting.
2020-10-16 23:05:47,396 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.w.WebAppContext@9bd9c40{datanode,/,null,UNAVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode}
2020-10-16 23:05:47,397 [Listener at localhost/39255] INFO
server.AbstractConnector (AbstractConnector.java:doStop(343)) - Stopped
ServerConnector@1e0a25c1{HTTP/1.1,[http/1.1]}{localhost:0}
2020-10-16 23:05:47,398 [Listener at localhost/39255] INFO server.session
(HouseKeeper.java:stopScavenging(158)) - node0 Stopped scavenging
2020-10-16 23:05:47,398 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@16658969{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,UNAVAILABLE}
2020-10-16 23:05:47,401 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@108a3c49{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,UNAVAILABLE}
2020-10-16 23:05:47,404 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2104)) - Waiting up to 30 seconds for transfer threads
to complete
2020-10-16 23:05:47,404 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 35559
2020-10-16 23:05:47,405 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1361)) - Stopping IPC Server listener on 0
2020-10-16 23:05:47,405 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1497)) - Stopping IPC Server Responder
2020-10-16 23:05:47,405 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.IncrementalBlockReportManager
(IncrementalBlockReportManager.java:waitTillNextIBR(160)) -
IncrementalBlockReportManager interrupted
2020-10-16 23:05:47,405 [Command processor] ERROR datanode.DataNode
(BPServiceActor.java:processQueue(1329)) - Command processor encountered
interrupt and exit.
2020-10-16 23:05:47,405 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.DataNode
(BPServiceActor.java:run(889)) - Ending block pool service for: Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
690746a8-27bf-430c-a71a-5d58643bc56c) service to localhost/127.0.0.1:46689
2020-10-16 23:05:47,406 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO datanode.DataNode
(BlockPoolManager.java:remove(102)) - Removed Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
690746a8-27bf-430c-a71a-5d58643bc56c)
2020-10-16 23:05:47,406 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO impl.FsDatasetImpl
(FsDatasetImpl.java:shutdownBlockPool(2946)) - Removing block pool
BP-1639228104-172.17.0.2-1602889526611
2020-10-16 23:05:47,407
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data9/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,407
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data10/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,407 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(193)) -
Shutting down all async disk service threads
2020-10-16 23:05:47,407 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(201)) -
All async disk service threads have been shut down
2020-10-16 23:05:47,407 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(177)) - Shutting down all async
lazy persist service threads
2020-10-16 23:05:47,408 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(184)) - All async lazy persist
service threads have been shut down
2020-10-16 23:05:47,408 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2193)) - Shutdown complete.
2020-10-16 23:05:47,408 [Listener at localhost/39255] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2139)) - Shutting down DataNode 3
2020-10-16 23:05:47,409 [Listener at localhost/39255] INFO
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(392)) - Shutdown has
been called
2020-10-16 23:05:47,409
[org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@290dbf5f] INFO
datanode.DataNode (DataXceiverServer.java:closeAllPeers(396)) - Closing all
peers.
2020-10-16 23:05:47,409
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data8)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data8,
DS-957f82cd-d955-415b-a34f-d881b6f999a4) exiting.
2020-10-16 23:05:47,409
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data7)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data7,
DS-9d5c0a4a-27b5-48fd-a1cd-ddb5e693cd2f) exiting.
2020-10-16 23:05:47,433 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.w.WebAppContext@228a7865{datanode,/,null,UNAVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode}
2020-10-16 23:05:47,434 [Listener at localhost/39255] INFO
server.AbstractConnector (AbstractConnector.java:doStop(343)) - Stopped
ServerConnector@71cd2b0f{HTTP/1.1,[http/1.1]}{localhost:0}
2020-10-16 23:05:47,434 [Listener at localhost/39255] INFO server.session
(HouseKeeper.java:stopScavenging(158)) - node0 Stopped scavenging
2020-10-16 23:05:47,434 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@485255f0{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,UNAVAILABLE}
2020-10-16 23:05:47,437 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@11be91d9{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,UNAVAILABLE}
2020-10-16 23:05:47,448 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2104)) - Waiting up to 30 seconds for transfer threads
to complete
2020-10-16 23:05:47,449 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 40611
2020-10-16 23:05:47,449 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1361)) - Stopping IPC Server listener on 0
2020-10-16 23:05:47,450 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1497)) - Stopping IPC Server Responder
2020-10-16 23:05:47,451 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.IncrementalBlockReportManager
(IncrementalBlockReportManager.java:waitTillNextIBR(160)) -
IncrementalBlockReportManager interrupted
2020-10-16 23:05:47,451 [Command processor] ERROR datanode.DataNode
(BPServiceActor.java:processQueue(1329)) - Command processor encountered
interrupt and exit.
2020-10-16 23:05:47,451 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.DataNode
(BPServiceActor.java:run(889)) - Ending block pool service for: Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
bfd05b11-9912-4b37-a45e-6d3be7cdf49c) service to localhost/127.0.0.1:46689
2020-10-16 23:05:47,451 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO datanode.DataNode
(BlockPoolManager.java:remove(102)) - Removed Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
bfd05b11-9912-4b37-a45e-6d3be7cdf49c)
2020-10-16 23:05:47,451 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO impl.FsDatasetImpl
(FsDatasetImpl.java:shutdownBlockPool(2946)) - Removing block pool
BP-1639228104-172.17.0.2-1602889526611
2020-10-16 23:05:47,452
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data7/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,452
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data8/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,452 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(193)) -
Shutting down all async disk service threads
2020-10-16 23:05:47,452 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(201)) -
All async disk service threads have been shut down
2020-10-16 23:05:47,453 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(177)) - Shutting down all async
lazy persist service threads
2020-10-16 23:05:47,453 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(184)) - All async lazy persist
service threads have been shut down
2020-10-16 23:05:47,453 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2193)) - Shutdown complete.
2020-10-16 23:05:47,453 [Listener at localhost/39255] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2139)) - Shutting down DataNode 2
2020-10-16 23:05:47,453 [Listener at localhost/39255] INFO
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(392)) - Shutdown has
been called
2020-10-16 23:05:47,453 [Listener at localhost/39255] WARN
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(394)) - Shutdown has
been called, but periodic scanner not started
2020-10-16 23:05:47,456 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2104)) - Waiting up to 30 seconds for transfer threads
to complete
2020-10-16 23:05:47,456 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 37423
2020-10-16 23:05:47,456 [Listener at localhost/39255] WARN
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(191)) -
AsyncDiskService has already shut down.
2020-10-16 23:05:47,456 [Listener at localhost/39255] WARN
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(175)) - AsyncLazyPersistService
has already shut down.
2020-10-16 23:05:47,458 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2193)) - Shutdown complete.
2020-10-16 23:05:47,458 [Listener at localhost/39255] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2139)) - Shutting down DataNode 1
2020-10-16 23:05:47,458 [Listener at localhost/39255] INFO
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(392)) - Shutdown has
been called
2020-10-16 23:05:47,459 [Listener at localhost/39255] WARN
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(394)) - Shutdown has
been called, but periodic scanner not started
2020-10-16 23:05:47,460 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2104)) - Waiting up to 30 seconds for transfer threads
to complete
2020-10-16 23:05:47,460 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 43821
2020-10-16 23:05:47,461 [Listener at localhost/39255] WARN
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(191)) -
AsyncDiskService has already shut down.
2020-10-16 23:05:47,461 [Listener at localhost/39255] WARN
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(175)) - AsyncLazyPersistService
has already shut down.
2020-10-16 23:05:47,461 [Listener at localhost/39255] INFO
impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(210)) - Stopping DataNode
metrics system...
2020-10-16 23:05:47,462 [Listener at localhost/39255] INFO
impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(216)) - DataNode metrics
system stopped.
2020-10-16 23:05:47,462 [Listener at localhost/39255] INFO
impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(611)) - DataNode
metrics system shutdown complete.
2020-10-16 23:05:47,464 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2193)) - Shutdown complete.
2020-10-16 23:05:47,464 [Listener at localhost/39255] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2139)) - Shutting down DataNode 0
2020-10-16 23:05:47,465 [Listener at localhost/39255] INFO
datanode.DirectoryScanner (DirectoryScanner.java:shutdown(392)) - Shutdown has
been called
2020-10-16 23:05:47,465
[org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@65d8b33] INFO
datanode.DataNode (DataXceiverServer.java:closeAllPeers(396)) - Closing all
peers.
2020-10-16 23:05:47,465
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data1)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data1,
DS-006b1cc2-728b-411d-ae7d-bba76ab8a56d) exiting.
2020-10-16 23:05:47,465
[VolumeScannerThread(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data2)]
INFO datanode.VolumeScanner (VolumeScanner.java:run(672)) -
VolumeScanner(/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data2,
DS-5aea8683-6571-40ee-a5db-f8c850e23623) exiting.
2020-10-16 23:05:47,490 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.w.WebAppContext@4fd6abb8{datanode,/,null,UNAVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode}
2020-10-16 23:05:47,491 [Listener at localhost/39255] INFO
server.AbstractConnector (AbstractConnector.java:doStop(343)) - Stopped
ServerConnector@30bdc2af{HTTP/1.1,[http/1.1]}{localhost:0}
2020-10-16 23:05:47,491 [Listener at localhost/39255] INFO server.session
(HouseKeeper.java:stopScavenging(158)) - node0 Stopped scavenging
2020-10-16 23:05:47,492 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@26635145{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,UNAVAILABLE}
2020-10-16 23:05:47,492 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@3c9c631f{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,UNAVAILABLE}
2020-10-16 23:05:47,493 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2104)) - Waiting up to 30 seconds for transfer threads
to complete
2020-10-16 23:05:47,493 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 37701
2020-10-16 23:05:47,494 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1361)) - Stopping IPC Server listener on 0
2020-10-16 23:05:47,494 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1497)) - Stopping IPC Server Responder
2020-10-16 23:05:47,495 [Command processor] ERROR datanode.DataNode
(BPServiceActor.java:processQueue(1329)) - Command processor encountered
interrupt and exit.
2020-10-16 23:05:47,495 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.IncrementalBlockReportManager
(IncrementalBlockReportManager.java:waitTillNextIBR(160)) -
IncrementalBlockReportManager interrupted
2020-10-16 23:05:47,495 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] WARN datanode.DataNode
(BPServiceActor.java:run(889)) - Ending block pool service for: Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
65dd21f3-1cc0-4642-aedb-ca6f7b78f559) service to localhost/127.0.0.1:46689
2020-10-16 23:05:47,496 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO datanode.DataNode
(BlockPoolManager.java:remove(102)) - Removed Block pool
BP-1639228104-172.17.0.2-1602889526611 (Datanode Uuid
65dd21f3-1cc0-4642-aedb-ca6f7b78f559)
2020-10-16 23:05:47,496 [BP-1639228104-172.17.0.2-1602889526611 heartbeating to
localhost/127.0.0.1:46689] INFO impl.FsDatasetImpl
(FsDatasetImpl.java:shutdownBlockPool(2946)) - Removing block pool
BP-1639228104-172.17.0.2-1602889526611
2020-10-16 23:05:47,497
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data1/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,497
[refreshUsed-/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/data/data2/current/BP-1639228104-172.17.0.2-1602889526611]
WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(211)) - Thread
Interrupted waiting to refresh disk information: sleep interrupted
2020-10-16 23:05:47,497 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(193)) -
Shutting down all async disk service threads
2020-10-16 23:05:47,498 [Listener at localhost/39255] INFO
impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:shutdown(201)) -
All async disk service threads have been shut down
2020-10-16 23:05:47,498 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(177)) - Shutting down all async
lazy persist service threads
2020-10-16 23:05:47,498 [Listener at localhost/39255] INFO
impl.RamDiskAsyncLazyPersistService
(RamDiskAsyncLazyPersistService.java:shutdown(184)) - All async lazy persist
service threads have been shut down
2020-10-16 23:05:47,498 [Listener at localhost/39255] INFO datanode.DataNode
(DataNode.java:shutdown(2193)) - Shutdown complete.
2020-10-16 23:05:47,498 [Listener at localhost/39255] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:stopAndJoinNameNode(2172)) - Shutting down the namenode
2020-10-16 23:05:47,498 [Listener at localhost/39255] INFO
namenode.FSNamesystem (FSNamesystem.java:stopActiveServices(1456)) - Stopping
services started for active state
2020-10-16 23:05:47,499 [Listener at localhost/39255] INFO namenode.FSEditLog
(FSEditLog.java:endCurrentLogSegment(1441)) - Ending log segment 1, 253
2020-10-16 23:05:47,499
[org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeEditLogRoller@1bcebcc5]
INFO namenode.FSNamesystem (FSNamesystem.java:run(4513)) -
NameNodeEditLogRoller was interrupted, exiting
2020-10-16 23:05:47,499
[org.apache.hadoop.hdfs.server.namenode.FSNamesystem$LazyPersistFileScrubber@3255b2bc]
INFO namenode.FSNamesystem (FSNamesystem.java:run(4609)) -
LazyPersistFileScrubber was interrupted, exiting
2020-10-16 23:05:47,500 [Listener at localhost/39255] INFO namenode.FSEditLog
(FSEditLog.java:printStatistics(778)) - Number of transactions: 254 Total time
for transactions(ms): 34 Number of transactions batched in Syncs: 50 Number of
syncs: 205 SyncTimes(ms): 13 3
2020-10-16 23:05:47,503 [Listener at localhost/39255] INFO
namenode.FileJournalManager (FileJournalManager.java:finalizeLogSegment(145)) -
Finalizing edits file
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1/current/edits_inprogress_0000000000000000001
->
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-1/current/edits_0000000000000000001-0000000000000000254
2020-10-16 23:05:47,506 [Listener at localhost/39255] INFO
namenode.FileJournalManager (FileJournalManager.java:finalizeLogSegment(145)) -
Finalizing edits file
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-2/current/edits_inprogress_0000000000000000001
->
/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/test/data/4/dwiOtdiBM1/name-0-2/current/edits_0000000000000000001-0000000000000000254
2020-10-16 23:05:47,506 [FSEditLogAsync] INFO namenode.FSEditLog
(FSEditLogAsync.java:run(253)) - FSEditLogAsync was interrupted, exiting
2020-10-16 23:05:47,507 [CacheReplicationMonitor(593159301)] INFO
blockmanagement.CacheReplicationMonitor (CacheReplicationMonitor.java:run(169))
- Shutting down CacheReplicationMonitor
2020-10-16 23:05:47,507 [Listener at localhost/39255] INFO ipc.Server
(Server.java:stop(3400)) - Stopping server on 46689
2020-10-16 23:05:47,508 [IPC Server listener on 0] INFO ipc.Server
(Server.java:run(1361)) - Stopping IPC Server listener on 0
2020-10-16 23:05:47,508 [RedundancyMonitor] INFO blockmanagement.BlockManager
(BlockManager.java:run(4944)) - Stopping RedundancyMonitor.
2020-10-16 23:05:47,511 [StorageInfoMonitor] INFO blockmanagement.BlockManager
(BlockManager.java:run(4979)) - Stopping thread.
2020-10-16 23:05:47,511 [IPC Server Responder] INFO ipc.Server
(Server.java:run(1497)) - Stopping IPC Server Responder
2020-10-16 23:05:47,511
[org.apache.hadoop.hdfs.server.blockmanagement.PendingReconstructionBlocks$PendingReconstructionMonitor@10da9fc3]
DEBUG blockmanagement.BlockManager (PendingReconstructionBlocks.java:run(261))
- PendingReconstructionMonitor thread is interrupted.
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hdfs.server.blockmanagement.PendingReconstructionBlocks$PendingReconstructionMonitor.run(PendingReconstructionBlocks.java:259)
at java.lang.Thread.run(Thread.java:748)
2020-10-16 23:05:47,522 [Listener at localhost/39255] INFO
namenode.FSNamesystem (FSNamesystem.java:stopActiveServices(1456)) - Stopping
services started for active state
2020-10-16 23:05:47,523 [Listener at localhost/39255] INFO
namenode.FSNamesystem (FSNamesystem.java:stopStandbyServices(1557)) - Stopping
services started for standby state
2020-10-16 23:05:47,525 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.w.WebAppContext@534a1627{hdfs,/,null,UNAVAILABLE}{file:/home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs}
2020-10-16 23:05:47,528 [Listener at localhost/39255] INFO
server.AbstractConnector (AbstractConnector.java:doStop(343)) - Stopped
ServerConnector@208b6f4d{HTTP/1.1,[http/1.1]}{localhost:0}
2020-10-16 23:05:47,528 [Listener at localhost/39255] INFO server.session
(HouseKeeper.java:stopScavenging(158)) - node0 Stopped scavenging
2020-10-16 23:05:47,528 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@25928105{static,/static,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/,UNAVAILABLE}
2020-10-16 23:05:47,529 [Listener at localhost/39255] INFO
handler.ContextHandler (ContextHandler.java:doStop(1007)) - Stopped
o.e.j.s.ServletContextHandler@2e577a5b{logs,/logs,file:///home/jenkins/jenkins-agent/workspace/PreCommit-HDFS-Build/sourcedir/hadoop-hdfs-project/hadoop-hdfs/target/log/,UNAVAILABLE}
{code}
> TestReconstructStripedFile.testNNSendsErasureCodingTasks is flaky
> -----------------------------------------------------------------
>
> Key: HDFS-15308
> URL: https://issues.apache.org/jira/browse/HDFS-15308
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
> Affects Versions: 3.3.0
> Reporter: Toshihiko Uchida
> Priority: Minor
> Labels: flaky-test
> Attachments: HDFS-15308.001.patch
>
>
> In HDFS-14353, TestReconstructStripedFile.testNNSendsErasureCodingTasks
> failed once due to pending reconstruction timeout as follows.
> {code}
> java.lang.AssertionError: Found 4 timeout pending reconstruction tasks
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hdfs.TestReconstructStripedFile.testNNSendsErasureCodingTasks(TestReconstructStripedFile.java:502)
> at
> org.apache.hadoop.hdfs.TestReconstructStripedFile.testNNSendsErasureCodingTasks(TestReconstructStripedFile.java:458)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The error occurred on the following assertion.
> {code}
> // Make sure that all pending reconstruction tasks can be processed.
> while (ns.getPendingReconstructionBlocks() > 0) {
> long timeoutPending = ns.getNumTimedOutPendingReconstructions();
> assertTrue(String.format("Found %d timeout pending reconstruction tasks",
> timeoutPending), timeoutPending == 0);
> Thread.sleep(1000);
> }
> {code}
> The failure could not be reproduced in the reporter's docker environment
> (start-build-environment.sh).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
