See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/27/changes
Changes:
[cdouglas] MAPREDUCE-245. Change Job and jobcontrol classes to use the List
interface
rather than ArrayList in APIs. Contributed by Tom White
[acmurthy] MAPREDUCE-740. Log a job-summary at the end of a job, while allowing
it to be configured to use a custom appender if desired.
[cdouglas] MAPREDUCE-739. Remove errant import.
[matei] MAPREDUCE-546. Provide sample fair scheduler config file in conf/ and
use it by default if no other config file is specified.
------------------------------------------
[...truncated 132451 lines...]
[junit] 2009-07-19 16:09:45,031 INFO mapred.JobClient
(JobClient.java:monitorAndPrintJob(1349)) - Task Id :
attempt_200907191608_0002_r_000000_0, Status : SUCCEEDED
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:33,730 INFO
jvm.JvmMetrics (JvmMetrics.java:init(71)) - Initializing JVM Metrics with
processName=SHUFFLE, sessionId=
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:33,856 INFO
mapred.ReduceTask (ReduceTask.java:<init>(1012)) - ShuffleRamManager:
MemoryLimit=141387360, MaxSingleShuffleLimit=35346840
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:33,862 INFO
mapred.ReduceTask (ReduceTask.java:run(2487)) -
attempt_200907191608_0002_r_000000_0 Thread started: Thread for merging on-disk
files
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:33,862 INFO
mapred.ReduceTask (ReduceTask.java:run(2586)) -
attempt_200907191608_0002_r_000000_0 Thread started: Thread for merging in
memory files
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:33,863 INFO
mapred.ReduceTask (ReduceTask.java:run(2492)) -
attempt_200907191608_0002_r_000000_0 Thread waiting: Thread for merging on-disk
files
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:33,864 INFO
mapred.ReduceTask (ReduceTask.java:fetchOutputs(1894)) -
attempt_200907191608_0002_r_000000_0 Need another 3 map output(s) where 0 is
already in progress
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:33,864 INFO
mapred.ReduceTask (ReduceTask.java:run(2688)) -
attempt_200907191608_0002_r_000000_0 Thread started: Thread for polling Map
Completion Events
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:33,865 INFO
mapred.ReduceTask (ReduceTask.java:fetchOutputs(2007)) -
attempt_200907191608_0002_r_000000_0 Scheduled 0 outputs (0 slow hosts and0 dup
hosts)
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:33,871 INFO
mapred.ReduceTask (ReduceTask.java:run(2694)) -
attempt_200907191608_0002_r_000000_0: Got 2 new map-outputs
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:37,875 INFO
mapred.ReduceTask (ReduceTask.java:run(2694)) -
attempt_200907191608_0002_r_000000_0: Got 1 new map-outputs
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,865 INFO
mapred.ReduceTask (ReduceTask.java:fetchOutputs(2007)) -
attempt_200907191608_0002_r_000000_0 Scheduled 1 outputs (0 slow hosts and0 dup
hosts)
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,917 INFO
mapred.ReduceTask (ReduceTask.java:getMapOutput(1425)) - header:
attempt_200907191608_0002_m_000000_0, compressed len: 11, decompressed len: 7
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,918 INFO
mapred.ReduceTask (ReduceTask.java:getMapOutput(1439)) - Shuffling 7 bytes (11
raw bytes) into RAM from attempt_200907191608_0002_m_000000_0
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,921 INFO
mapred.ReduceTask (ReduceTask.java:shuffleInMemory(1568)) - Read 7 bytes from
map-output for attempt_200907191608_0002_m_000000_0
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,921 INFO
mapred.ReduceTask (ReduceTask.java:shuffleInMemory(1627)) - Rec #1 from
attempt_200907191608_0002_m_000000_0 -> (2, 1) from localhost
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,923 INFO
mapred.ReduceTask (ReduceTask.java:fetchOutputs(2007)) -
attempt_200907191608_0002_r_000000_0 Scheduled 1 outputs (0 slow hosts and0 dup
hosts)
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,928 INFO
mapred.ReduceTask (ReduceTask.java:getMapOutput(1425)) - header:
attempt_200907191608_0002_m_000001_0, compressed len: 11, decompressed len: 7
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,929 INFO
mapred.ReduceTask (ReduceTask.java:getMapOutput(1439)) - Shuffling 7 bytes (11
raw bytes) into RAM from attempt_200907191608_0002_m_000001_0
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,929 INFO
mapred.ReduceTask (ReduceTask.java:shuffleInMemory(1568)) - Read 7 bytes from
map-output for attempt_200907191608_0002_m_000001_0
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,929 INFO
mapred.ReduceTask (ReduceTask.java:shuffleInMemory(1627)) - Rec #1 from
attempt_200907191608_0002_m_000001_0 -> (2, 1) from localhost
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,930 INFO
mapred.ReduceTask (ReduceTask.java:fetchOutputs(2007)) -
attempt_200907191608_0002_r_000000_0 Scheduled 1 outputs (0 slow hosts and0 dup
hosts)
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,933 INFO
mapred.ReduceTask (ReduceTask.java:getMapOutput(1425)) - header:
attempt_200907191608_0002_m_000002_0, compressed len: 11, decompressed len: 7
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,933 INFO
mapred.ReduceTask (ReduceTask.java:getMapOutput(1439)) - Shuffling 7 bytes (11
raw bytes) into RAM from attempt_200907191608_0002_m_000002_0
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,933 INFO
mapred.ReduceTask (ReduceTask.java:shuffleInMemory(1568)) - Read 7 bytes from
map-output for attempt_200907191608_0002_m_000002_0
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:38,933 INFO
mapred.ReduceTask (ReduceTask.java:shuffleInMemory(1627)) - Rec #1 from
attempt_200907191608_0002_m_000002_0 -> (2, 1) from localhost
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,876 INFO
mapred.ReduceTask (ReduceTask.java:run(2712)) - GetMapEventsThread exiting
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,877 INFO
mapred.ReduceTask (ReduceTask.java:fetchOutputs(2205)) - getMapsEventsThread
joined.
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,877 INFO
mapred.ReduceTask (ReduceTask.java:close(1118)) - Closed ram manager
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,877 INFO
mapred.ReduceTask (ReduceTask.java:fetchOutputs(2233)) - Interleaved on-disk
merge complete: 0 files left.
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,877 INFO
mapred.ReduceTask (ReduceTask.java:fetchOutputs(2238)) - In-memory merge
complete: 3 files left.
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,942 INFO
mapred.Merger (Merger.java:merge(469)) - Merging 3 sorted segments
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,943 INFO
mapred.Merger (Merger.java:merge(568)) - Down to the last merge-pass, with 3
segments left of total size: 9 bytes
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,951 INFO
mapred.ReduceTask (ReduceTask.java:createKVIterator(2351)) - Merged 3 segments,
21 bytes to disk to satisfy reduce memory limit
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,952 INFO
mapred.ReduceTask (ReduceTask.java:createKVIterator(2371)) - Merging 1 files,
21 bytes from disk
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,953 INFO
mapred.ReduceTask (ReduceTask.java:createKVIterator(2385)) - Merging 0
segments, 0 bytes from memory into reduce
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,953 INFO
mapred.Merger (Merger.java:merge(469)) - Merging 1 sorted segments
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:39,957 INFO
mapred.Merger (Merger.java:merge(568)) - Down to the last merge-pass, with 1
segments left of total size: 13 bytes
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:40,074 INFO
mapred.Task (Task.java:done(633)) - Task:attempt_200907191608_0002_r_000000_0
is done. And is in the process of commiting
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:42,082 INFO
mapred.Task (Task.java:commit(733)) - Task attempt_200907191608_0002_r_000000_0
is allowed to commit now
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:42,211 INFO
mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(92)) - Saved
output of task 'attempt_200907191608_0002_r_000000_0' to
hdfs://localhost:58036/user/hudson/mapout
[junit] attempt_200907191608_0002_r_000000_0: 2009-07-19 16:09:42,214 INFO
mapred.Task (Task.java:sendDone(704)) - Task
'attempt_200907191608_0002_r_000000_0' done.
[junit] 2009-07-19 16:09:45,044 INFO mapred.JvmManager
(JvmManager.java:<init>(383)) - In JvmRunner constructed JVM ID:
jvm_200907191608_0002_m_2136229430
[junit] 2009-07-19 16:09:45,044 INFO mapred.JvmManager
(JvmManager.java:spawnNewJvm(351)) - JVM Runner
jvm_200907191608_0002_m_2136229430 spawned.
[junit] 2009-07-19 16:09:45,492 INFO mapred.TaskTracker
(TaskTracker.java:getTask(2534)) - JVM with ID:
jvm_200907191608_0002_m_2136229430 given task:
attempt_200907191608_0002_m_000003_0
[junit] 2009-07-19 16:09:45,855 INFO mapred.TaskTracker
(TaskTracker.java:reportProgress(1986)) - attempt_200907191608_0002_m_000003_0
0.0%
[junit] 2009-07-19 16:09:45,898 INFO FSNamesystem.audit
(FSNamesystem.java:logAuditEvent(114)) - ugi=hudson,hudson ip=/127.0.0.1
cmd=delete src=/user/hudson/mapout/_temporary dst=null
perm=null
[junit] 2009-07-19 16:09:45,902 INFO mapred.TaskTracker
(TaskTracker.java:reportProgress(1986)) - attempt_200907191608_0002_m_000003_0
0.0% cleanup > map
[junit] 2009-07-19 16:09:45,903 INFO mapred.TaskTracker
(TaskTracker.java:reportDone(2067)) - Task attempt_200907191608_0002_m_000003_0
is done.
[junit] 2009-07-19 16:09:45,903 INFO mapred.TaskTracker
(TaskTracker.java:reportDone(2068)) - reported output size for
attempt_200907191608_0002_m_000003_0 was 0
[junit] 2009-07-19 16:09:45,904 INFO mapred.TaskTracker
(TaskTracker.java:addFreeSlots(1618)) - addFreeSlot : current free slots : 2
[junit] 2009-07-19 16:09:46,037 INFO mapred.JobClient
(JobClient.java:monitorAndPrintJob(1300)) - map 100% reduce 100%
[junit] 2009-07-19 16:09:46,124 WARN util.ProcessTree
(ProcessTree.java:terminateProcessGroup(136)) - Error executing shell command
org.apache.hadoop.util.Shell$ExitCodeException: kill: No such process
[junit]
[junit] 2009-07-19 16:09:46,124 INFO util.ProcessTree
(ProcessTree.java:terminateProcessGroup(138)) - Killing all processes in the
process group 18785 with SIGTERM. Exit code 1
[junit] 2009-07-19 16:09:47,361 INFO mapred.JvmManager
(JvmManager.java:runChild(411)) - JVM : jvm_200907191608_0002_r_-280748229
exited. Number of tasks it ran: 1
[junit] 2009-07-19 16:09:47,997 INFO mapred.TaskTracker
(TaskTracker.java:tryToGetOutputSize(1570)) -
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_200907191608_0002/attempt_200907191608_0002_m_000003_0/output/file.out
in any of the configured local directories
[junit] 2009-07-19 16:09:47,999 INFO mapred.JobInProgress
(JobInProgress.java:completedTask(2443)) - Task
'attempt_200907191608_0002_m_000003_0' has completed
task_200907191608_0002_m_000003 successfully.
[junit] 2009-07-19 16:09:48,000 INFO mapred.JobInProgress
(JobInProgress.java:jobComplete(2608)) - Job job_200907191608_0002 has
completed successfully.
[junit] 2009-07-19 16:09:48,000 INFO mapred.JobInProgress$JobSummary
(JobInProgress.java:logJobSummary(3267)) -
jobId=job_200907191608_0002,submitTime=1248019765924,launchTime=1248019766116,finishTime=1248019788000,numMaps=3,numSlotsPerMap=1,numReduces=1,numSlotsPerReduce=1,user=hudson,queue=default,status=SUCCEEDED,mapSlotSeconds=5,reduceSlotsSeconds=9,clusterMapCapacity=4,clusterReduceCapacity=4
[junit] 2009-07-19 16:09:48,003 INFO hdfs.StateChange
(FSNamesystem.java:allocateBlock(1286)) - BLOCK* NameSystem.allocateBlock:
/user/hudson/mapout/_logs/history/localhost_1248019728755_job_200907191608_0002_hudson_NA.
blk_-1743757223564922330_1015
[junit] 2009-07-19 16:09:48,005 INFO datanode.DataNode
(DataXceiver.java:writeBlock(240)) - Receiving block
blk_-1743757223564922330_1015 src: /127.0.0.1:51035 dest: /127.0.0.1:59012
[junit] 2009-07-19 16:09:48,006 INFO datanode.DataNode
(DataXceiver.java:writeBlock(240)) - Receiving block
blk_-1743757223564922330_1015 src: /127.0.0.1:39403 dest: /127.0.0.1:35072
[junit] 2009-07-19 16:09:48,009 INFO DataNode.clienttrace
(BlockReceiver.java:lastDataNodeRun(807)) - src: /127.0.0.1:39403, dest:
/127.0.0.1:35072, bytes: 11831, op: HDFS_WRITE, cliID: DFSClient_-2011435373,
offset: 0, srvID: DS-544367877-67.195.138.9-35072-1248019728628, blockid:
blk_-1743757223564922330_1015, duration: 1975476
[junit] 2009-07-19 16:09:48,010 INFO datanode.DataNode
(BlockReceiver.java:lastDataNodeRun(831)) - PacketResponder 0 for block
blk_-1743757223564922330_1015 terminating
[junit] 2009-07-19 16:09:48,010 INFO hdfs.StateChange
(BlockManager.java:addStoredBlock(955)) - BLOCK* NameSystem.addStoredBlock:
blockMap updated: 127.0.0.1:35072 is added to blk_-1743757223564922330_1015
size 11831
[junit] 2009-07-19 16:09:48,010 INFO DataNode.clienttrace
(BlockReceiver.java:run(933)) - src: /127.0.0.1:51035, dest: /127.0.0.1:59012,
bytes: 11831, op: HDFS_WRITE, cliID: DFSClient_-2011435373, offset: 0, srvID:
DS-1812677761-67.195.138.9-59012-1248019727997, blockid:
blk_-1743757223564922330_1015, duration: 2951350
[junit] 2009-07-19 16:09:48,011 INFO hdfs.StateChange
(BlockManager.java:addStoredBlock(955)) - BLOCK* NameSystem.addStoredBlock:
blockMap updated: 127.0.0.1:59012 is added to blk_-1743757223564922330_1015
size 11831
[junit] 2009-07-19 16:09:48,011 INFO datanode.DataNode
(BlockReceiver.java:run(997)) - PacketResponder 1 for block
blk_-1743757223564922330_1015 terminating
[junit] 2009-07-19 16:09:48,013 INFO hdfs.StateChange
(FSNamesystem.java:completeFileInternal(1252)) - DIR* NameSystem.completeFile:
file
/user/hudson/mapout/_logs/history/localhost_1248019728755_job_200907191608_0002_hudson_NA
is closed by DFSClient_-2011435373
[junit] 2009-07-19 16:09:48,013 INFO namenode.FSNamesystem
(FSEditLog.java:printStatistics(876)) - Number of transactions: 87 Total time
for transactions(ms): 6Number of transactions batched in Syncs: 6 Number of
syncs: 60 SyncTimes(ms): 827 589
[junit] 2009-07-19 16:09:48,024 INFO mapred.JobHistory
(JobHistory.java:markCompleted(967)) - Moving completed job from
http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/ws/trunk/build/test/logs/history/localhost_1248019728755_job_200907191608_0002_hudson_NA
to
http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/ws/trunk/build/test/logs/history/done/localhost_1248019728755_job_200907191608_0002_hudson_NA
[junit] 2009-07-19 16:09:48,025 INFO mapred.JobHistory
(JobHistory.java:markCompleted(977)) - Moving configuration of completed job
from
http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/ws/trunk/build/test/logs/history/localhost_1248019728755_job_200907191608_0002_conf.xml
to
http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/ws/trunk/build/test/logs/history/done/localhost_1248019728755_job_200907191608_0002_conf.xml
[junit] 2009-07-19 16:09:48,027 INFO hdfs.StateChange
(BlockManager.java:addToInvalidates(365)) - BLOCK* NameSystem.addToInvalidates:
blk_6722452888608395195 is added to invalidSet of 127.0.0.1:59012
[junit] 2009-07-19 16:09:48,027 INFO hdfs.StateChange
(BlockManager.java:addToInvalidates(365)) - BLOCK* NameSystem.addToInvalidates:
blk_6722452888608395195 is added to invalidSet of 127.0.0.1:35072
[junit] 2009-07-19 16:09:48,027 INFO hdfs.StateChange
(BlockManager.java:addToInvalidates(365)) - BLOCK* NameSystem.addToInvalidates:
blk_3832403828249354844 is added to invalidSet of 127.0.0.1:35072
[junit] 2009-07-19 16:09:48,027 INFO hdfs.StateChange
(BlockManager.java:addToInvalidates(365)) - BLOCK* NameSystem.addToInvalidates:
blk_3832403828249354844 is added to invalidSet of 127.0.0.1:59012
[junit] 2009-07-19 16:09:48,028 INFO mapred.JobTracker
(JobTracker.java:removeMarkedTasks(2286)) - Removed completed task
'attempt_200907191608_0002_m_000000_0' from
'tracker_host1.foo.com:localhost/127.0.0.1:50481'
[junit] 2009-07-19 16:09:48,028 INFO mapred.JobTracker
(JobTracker.java:removeMarkedTasks(2286)) - Removed completed task
'attempt_200907191608_0002_m_000001_0' from
'tracker_host1.foo.com:localhost/127.0.0.1:50481'
[junit] 2009-07-19 16:09:48,028 INFO mapred.JobTracker
(JobTracker.java:removeMarkedTasks(2286)) - Removed completed task
'attempt_200907191608_0002_m_000003_0' from
'tracker_host1.foo.com:localhost/127.0.0.1:50481'
[junit] 2009-07-19 16:09:48,028 INFO mapred.JobTracker
(JobTracker.java:removeMarkedTasks(2286)) - Removed completed task
'attempt_200907191608_0002_m_000004_0' from
'tracker_host1.foo.com:localhost/127.0.0.1:50481'
[junit] 2009-07-19 16:09:48,028 INFO mapred.JobTracker
(JobTracker.java:removeMarkedTasks(2286)) - Removed completed task
'attempt_200907191608_0002_r_000000_0' from
'tracker_host1.foo.com:localhost/127.0.0.1:50481'
[junit] 2009-07-19 16:09:48,029 INFO mapred.TaskTracker
(TaskTracker.java:purgeJob(1366)) - Received 'KillJobAction' for job:
job_200907191608_0002
[junit] 2009-07-19 16:09:48,029 INFO mapred.TaskRunner
(MapTaskRunner.java:close(43)) - attempt_200907191608_0002_m_000001_0 done;
removing files.
[junit] 2009-07-19 16:09:48,030 INFO mapred.TaskRunner
(MapTaskRunner.java:close(43)) - attempt_200907191608_0002_m_000000_0 done;
removing files.
[junit] 2009-07-19 16:09:48,031 INFO mapred.TaskRunner
(MapTaskRunner.java:close(43)) - attempt_200907191608_0002_m_000003_0 done;
removing files.
[junit] 2009-07-19 16:09:48,031 INFO mapred.IndexCache
(IndexCache.java:removeMap(140)) - Map ID attempt_200907191608_0002_m_000003_0
not found in cache
[junit] 2009-07-19 16:09:48,041 INFO mapred.JobClient
(JobClient.java:monitorAndPrintJob(1349)) - Task Id :
attempt_200907191608_0002_m_000003_0, Status : SUCCEEDED
[junit] attempt_200907191608_0002_m_000003_0: 2009-07-19 16:09:45,737 INFO
jvm.JvmMetrics (JvmMetrics.java:init(71)) - Initializing JVM Metrics with
processName=MAP, sessionId=
[junit] attempt_200907191608_0002_m_000003_0: 2009-07-19 16:09:45,899 INFO
mapred.Task (Task.java:done(633)) - Task:attempt_200907191608_0002_m_000003_0
is done. And is in the process of commiting
[junit] attempt_200907191608_0002_m_000003_0: 2009-07-19 16:09:45,904 INFO
mapred.Task (Task.java:sendDone(704)) - Task
'attempt_200907191608_0002_m_000003_0' done.
[junit] 2009-07-19 16:09:48,043 INFO FSNamesystem.audit
(FSNamesystem.java:logAuditEvent(114)) - ugi=hudson,hudson ip=/127.0.0.1
cmd=delete src=/tmp/hadoop-hudson/mapred/system/job_200907191608_0002
dst=null perm=null
[junit] 2009-07-19 16:09:48,044 INFO mapred.JobClient
(JobClient.java:monitorAndPrintJob(1355)) - Job complete: job_200907191608_0002
[junit] 2009-07-19 16:09:48,047 INFO mapred.JobClient
(Counters.java:log(534)) - Counters: 22
[junit] 2009-07-19 16:09:48,047 INFO mapred.JobClient
(Counters.java:log(536)) - Job Counters
[junit] 2009-07-19 16:09:48,047 INFO mapred.JobClient
(Counters.java:log(538)) - Launched reduce tasks=1
[junit] 2009-07-19 16:09:48,047 INFO mapred.JobClient
(Counters.java:log(538)) - SLOTS_MILLIS_MAPS=5094
[junit] 2009-07-19 16:09:48,047 INFO mapred.JobClient
(Counters.java:log(538)) - Total time spent by all reduces waiting after
reserving slots (ms)=0
[junit] 2009-07-19 16:09:48,047 INFO mapred.JobClient
(Counters.java:log(538)) - Total time spent by all maps waiting after
reserving slots (ms)=0
[junit] 2009-07-19 16:09:48,047 INFO mapred.JobClient
(Counters.java:log(538)) - Rack-local map tasks=3
[junit] 2009-07-19 16:09:48,048 INFO mapred.JobClient
(Counters.java:log(538)) - Launched map tasks=3
[junit] 2009-07-19 16:09:48,048 INFO mapred.JobClient
(Counters.java:log(538)) - SLOTS_MILLIS_REDUCES=9149
[junit] 2009-07-19 16:09:48,048 INFO mapred.JobClient
(Counters.java:log(536)) - FileInputFormatCounters
[junit] 2009-07-19 16:09:48,048 INFO mapred.JobClient
(Counters.java:log(538)) - BYTES_READ=3
[junit] 2009-07-19 16:09:48,048 INFO mapred.JobClient
(Counters.java:log(536)) - FileSystemCounters
[junit] 2009-07-19 16:09:48,048 INFO mapred.JobClient
(Counters.java:log(538)) - FILE_BYTES_READ=21
[junit] 2009-07-19 16:09:48,048 INFO mapred.JobClient
(Counters.java:log(538)) - HDFS_BYTES_READ=828
[junit] 2009-07-19 16:09:48,049 INFO mapred.JobClient
(Counters.java:log(538)) - FILE_BYTES_WRITTEN=150
[junit] 2009-07-19 16:09:48,049 INFO mapred.JobClient
(Counters.java:log(538)) - HDFS_BYTES_WRITTEN=6
[junit] 2009-07-19 16:09:48,049 INFO mapred.JobClient
(Counters.java:log(536)) - Map-Reduce Framework
[junit] 2009-07-19 16:09:48,049 INFO mapred.JobClient
(Counters.java:log(538)) - Reduce input groups=3
[junit] 2009-07-19 16:09:48,049 INFO mapred.JobClient
(Counters.java:log(538)) - Combine output records=0
[junit] 2009-07-19 16:09:48,049 INFO mapred.JobClient
(Counters.java:log(538)) - Map input records=3
[junit] 2009-07-19 16:09:48,049 INFO mapred.JobClient
(Counters.java:log(538)) - Reduce shuffle bytes=22
[junit] 2009-07-19 16:09:48,050 INFO mapred.JobClient
(Counters.java:log(538)) - Reduce output records=3
[junit] 2009-07-19 16:09:48,050 INFO mapred.JobClient
(Counters.java:log(538)) - Spilled Records=6
[junit] 2009-07-19 16:09:48,050 INFO mapred.JobClient
(Counters.java:log(538)) - Map output bytes=9
[junit] 2009-07-19 16:09:48,050 INFO mapred.JobClient
(Counters.java:log(538)) - Combine input records=0
[junit] 2009-07-19 16:09:48,050 INFO mapred.JobClient
(Counters.java:log(538)) - Map output records=3
[junit] 2009-07-19 16:09:48,050 INFO mapred.JobClient
(Counters.java:log(538)) - Reduce input records=3
[junit] 2009-07-19 16:09:48,053 INFO FSNamesystem.audit
(FSNamesystem.java:logAuditEvent(114)) - ugi=hudson,hudson ip=/127.0.0.1
cmd=listStatus src=/user/hudson/mapout dst=null perm=null
[junit] Found 2 items
[junit] drwxr-xr-x - hudson supergroup 0 2009-07-19 16:09
/user/hudson/mapout/_logs
[junit] -rw-r--r-- 3 hudson supergroup 6 2009-07-19 16:09
/user/hudson/mapout/part-00000
[junit] 2009-07-19 16:09:48,055 INFO FSNamesystem.audit
(FSNamesystem.java:logAuditEvent(114)) - ugi=hudson,hudson ip=/127.0.0.1
cmd=listStatus src=/user/hudson/mapout dst=null perm=null
[junit] 2009-07-19 16:09:48,057 INFO FSNamesystem.audit
(FSNamesystem.java:logAuditEvent(114)) - ugi=hudson,hudson ip=/127.0.0.1
cmd=open src=/user/hudson/mapout/part-00000 dst=null
perm=null
[junit] 2009-07-19 16:09:48,058 INFO DataNode.clienttrace
(BlockSender.java:sendBlock(417)) - src: /127.0.0.1:59012, dest:
/127.0.0.1:51037, bytes: 10, op: HDFS_READ, cliID: DFSClient_-2011435373,
offset: 0, srvID: DS-1812677761-67.195.138.9-59012-1248019727997, blockid:
blk_-2784186997596440368_1015, duration: 251425
[junit] 2009-07-19 16:09:48,058 INFO mapred.MiniMRCluster
(MiniMRCluster.java:waitTaskTrackers(306)) - Waiting for task tracker
tracker_host0.foo.com:localhost/127.0.0.1:45026 to be idle.
[junit] 2009-07-19 16:09:48,886 INFO mapred.JobTracker
(JobTracker.java:removeMarkedTasks(2286)) - Removed completed task
'attempt_200907191608_0002_m_000002_0' from
'tracker_host0.foo.com:localhost/127.0.0.1:45026'
[junit] 2009-07-19 16:09:48,887 INFO mapred.TaskTracker
(TaskTracker.java:purgeJob(1366)) - Received 'KillJobAction' for job:
job_200907191608_0002
[junit] 2009-07-19 16:09:48,887 INFO mapred.TaskRunner
(MapTaskRunner.java:close(43)) - attempt_200907191608_0002_m_000002_0 done;
removing files.
[junit] 2009-07-19 16:09:49,059 INFO mapred.TaskTracker
(TaskTracker.java:run(623)) - Shutting down: Map-events fetcher for all reduce
tasks on tracker_host0.foo.com:localhost/127.0.0.1:45026
[junit] 2009-07-19 16:09:49,060 INFO ipc.Server (Server.java:stop(1103)) -
Stopping server on 45026
[junit] 2009-07-19 16:09:49,061 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 0 on 45026: exiting
[junit] 2009-07-19 16:09:49,061 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 1 on 45026: exiting
[junit] 2009-07-19 16:09:49,061 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 3 on 45026: exiting
[junit] 2009-07-19 16:09:49,061 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 2 on 45026: exiting
[junit] 2009-07-19 16:09:49,062 INFO ipc.Server (Server.java:run(352)) -
Stopping IPC Server listener on 45026
[junit] 2009-07-19 16:09:49,062 INFO ipc.Server (Server.java:run(539)) -
Stopping IPC Server Responder
[junit] 2009-07-19 16:09:49,062 INFO mapred.TaskTracker
(TaskTracker.java:shutdown(855)) - Shutting down StatusHttpServer
[junit] 2009-07-19 16:09:49,164 INFO mapred.TaskTracker
(TaskTracker.java:offerService(1134)) - Interrupted. Closing down.
[junit] 2009-07-19 16:09:49,165 INFO mapred.TaskTracker
(TaskTracker.java:run(623)) - Shutting down: Map-events fetcher for all reduce
tasks on tracker_host1.foo.com:localhost/127.0.0.1:50481
[junit] 2009-07-19 16:09:49,166 INFO ipc.Server (Server.java:stop(1103)) -
Stopping server on 50481
[junit] 2009-07-19 16:09:49,166 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 2 on 50481: exiting
[junit] 2009-07-19 16:09:49,166 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 3 on 50481: exiting
[junit] 2009-07-19 16:09:49,166 INFO mapred.TaskTracker
(TaskTracker.java:shutdown(855)) - Shutting down StatusHttpServer
[junit] 2009-07-19 16:09:49,166 INFO ipc.Server (Server.java:run(539)) -
Stopping IPC Server Responder
[junit] 2009-07-19 16:09:49,166 INFO ipc.Server (Server.java:run(352)) -
Stopping IPC Server listener on 50481
[junit] 2009-07-19 16:09:49,167 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 1 on 50481: exiting
[junit] 2009-07-19 16:09:49,167 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 0 on 50481: exiting
[junit] 2009-07-19 16:09:49,269 INFO mapred.TaskTracker
(TaskTracker.java:offerService(1134)) - Interrupted. Closing down.
[junit] 2009-07-19 16:09:49,270 INFO mapred.JobTracker
(JobTracker.java:close(2104)) - Stopping infoServer
[junit] 2009-07-19 16:09:49,397 INFO mapred.JobTracker
(JobTracker.java:close(2112)) - Stopping interTrackerServer
[junit] 2009-07-19 16:09:49,397 INFO ipc.Server (Server.java:stop(1103)) -
Stopping server on 59662
[junit] 2009-07-19 16:09:49,397 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 1 on 59662: exiting
[junit] 2009-07-19 16:09:49,397 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 7 on 59662: exiting
[junit] 2009-07-19 16:09:49,398 INFO ipc.Server (Server.java:run(352)) -
Stopping IPC Server listener on 59662
[junit] 2009-07-19 16:09:49,398 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 2 on 59662: exiting
[junit] 2009-07-19 16:09:49,397 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 6 on 59662: exiting
[junit] 2009-07-19 16:09:49,397 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 8 on 59662: exiting
[junit] 2009-07-19 16:09:49,398 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 0 on 59662: exiting
[junit] 2009-07-19 16:09:49,398 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 5 on 59662: exiting
[junit] 2009-07-19 16:09:49,398 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 3 on 59662: exiting
[junit] 2009-07-19 16:09:49,398 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 9 on 59662: exiting
[junit] 2009-07-19 16:09:49,398 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 4 on 59662: exiting
[junit] 2009-07-19 16:09:49,398 INFO mapred.JobTracker
(JobTracker.java:offerService(2099)) - Stopped interTrackerServer
[junit] 2009-07-19 16:09:49,398 INFO ipc.Server (Server.java:run(539)) -
Stopping IPC Server Responder
[junit] 2009-07-19 16:09:49,398 INFO mapred.JobTracker
(JobTracker.java:stopExpireTrackersThread(2155)) - Stopping expireTrackers
[junit] 2009-07-19 16:09:49,401 INFO mapred.JobTracker
(JobTracker.java:close(2119)) - Stopping retirer
[junit] 2009-07-19 16:09:49,402 INFO
mapred.EagerTaskInitializationListener
(EagerTaskInitializationListener.java:terminate(106)) - Stopping Job Init
Manager thread
[junit] 2009-07-19 16:09:49,402 INFO
mapred.EagerTaskInitializationListener
(EagerTaskInitializationListener.java:run(60)) - JobInitManagerThread
interrupted.
[junit] 2009-07-19 16:09:49,402 INFO
mapred.EagerTaskInitializationListener
(EagerTaskInitializationListener.java:run(64)) - Shutting down thread pool
[junit] 2009-07-19 16:09:49,402 INFO mapred.JobTracker
(JobTracker.java:close(2131)) - Stopping expireLaunchingTasks
[junit] 2009-07-19 16:09:49,403 INFO mapred.JobTracker
(JobTracker.java:close(2149)) - stopped all jobtracker services
[junit] Shutting down the Mini HDFS Cluster
[junit] Shutting down DataNode 1
[junit] 2009-07-19 16:09:49,505 INFO ipc.Server (Server.java:stop(1103)) -
Stopping server on 35635
[junit] 2009-07-19 16:09:49,506 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 0 on 35635: exiting
[junit] 2009-07-19 16:09:49,506 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 1 on 35635: exiting
[junit] 2009-07-19 16:09:49,506 INFO ipc.Server (Server.java:run(352)) -
Stopping IPC Server listener on 35635
[junit] 2009-07-19 16:09:49,506 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 2 on 35635: exiting
[junit] 2009-07-19 16:09:49,506 INFO datanode.DataNode
(DataNode.java:shutdown(639)) - Waiting for threadgroup to exit, active threads
is 1
[junit] 2009-07-19 16:09:49,507 WARN datanode.DataNode
(DataXceiverServer.java:run(137)) - DatanodeRegistration(127.0.0.1:35072,
storageID=DS-544367877-67.195.138.9-35072-1248019728628, infoPort=49256,
ipcPort=35635):DataXceiveServer: java.nio.channels.AsynchronousCloseException
[junit] at
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:185)
[junit] at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:152)
[junit] at
sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:84)
[junit] at
org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:130)
[junit] at java.lang.Thread.run(Thread.java:619)
[junit]
[junit] 2009-07-19 16:09:49,506 INFO ipc.Server (Server.java:run(539)) -
Stopping IPC Server Responder
[junit] 2009-07-19 16:09:49,508 INFO datanode.DataNode
(DataNode.java:shutdown(639)) - Waiting for threadgroup to exit, active threads
is 0
[junit] 2009-07-19 16:09:49,509 INFO datanode.DataBlockScanner
(DataBlockScanner.java:run(616)) - Exiting DataBlockScanner thread.
[junit] 2009-07-19 16:09:49,509 INFO datanode.DataNode
(DataNode.java:run(1285)) - DatanodeRegistration(127.0.0.1:35072,
storageID=DS-544367877-67.195.138.9-35072-1248019728628, infoPort=49256,
ipcPort=35635):Finishing DataNode in:
FSDataset{dirpath='http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/ws/trunk/build/test/data/dfs/data/data3/current,/home/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk/trunk/build/test/data/dfs/data/data4/current'}
[junit] 2009-07-19 16:09:49,509 INFO ipc.Server (Server.java:stop(1103)) -
Stopping server on 35635
[junit] 2009-07-19 16:09:49,510 INFO datanode.DataNode
(DataNode.java:shutdown(639)) - Waiting for threadgroup to exit, active threads
is 0
[junit] Shutting down DataNode 0
[junit] 2009-07-19 16:09:49,511 INFO ipc.Server (Server.java:stop(1103)) -
Stopping server on 60587
[junit] 2009-07-19 16:09:49,511 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 0 on 60587: exiting
[junit] 2009-07-19 16:09:49,512 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 2 on 60587: exiting
[junit] 2009-07-19 16:09:49,512 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 1 on 60587: exiting
[junit] 2009-07-19 16:09:49,512 INFO ipc.Server (Server.java:run(539)) -
Stopping IPC Server Responder
[junit] 2009-07-19 16:09:49,512 INFO ipc.Server (Server.java:run(352)) -
Stopping IPC Server listener on 60587
[junit] 2009-07-19 16:09:49,512 INFO datanode.DataNode
(DataNode.java:shutdown(639)) - Waiting for threadgroup to exit, active threads
is 1
[junit] 2009-07-19 16:09:49,513 WARN datanode.DataNode
(DataXceiverServer.java:run(137)) - DatanodeRegistration(127.0.0.1:59012,
storageID=DS-1812677761-67.195.138.9-59012-1248019727997, infoPort=36088,
ipcPort=60587):DataXceiveServer: java.nio.channels.AsynchronousCloseException
[junit] at
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:185)
[junit] at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:152)
[junit] at
sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:84)
[junit] at
org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:130)
[junit] at java.lang.Thread.run(Thread.java:619)
[junit]
[junit] 2009-07-19 16:09:49,515 INFO datanode.DataNode
(DataNode.java:shutdown(639)) - Waiting for threadgroup to exit, active threads
is 0
[junit] 2009-07-19 16:09:49,515 INFO datanode.DataBlockScanner
(DataBlockScanner.java:run(616)) - Exiting DataBlockScanner thread.
[junit] 2009-07-19 16:09:49,516 INFO datanode.DataNode
(DataNode.java:run(1285)) - DatanodeRegistration(127.0.0.1:59012,
storageID=DS-1812677761-67.195.138.9-59012-1248019727997, infoPort=36088,
ipcPort=60587):Finishing DataNode in:
FSDataset{dirpath='http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/ws/trunk/build/test/data/dfs/data/data1/current,/home/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk/trunk/build/test/data/dfs/data/data2/current'}
[junit] 2009-07-19 16:09:49,516 INFO ipc.Server (Server.java:stop(1103)) -
Stopping server on 60587
[junit] 2009-07-19 16:09:49,516 INFO datanode.DataNode
(DataNode.java:shutdown(639)) - Waiting for threadgroup to exit, active threads
is 0
[junit] 2009-07-19 16:09:49,621 WARN namenode.FSNamesystem
(FSNamesystem.java:run(2057)) - ReplicationMonitor thread received
InterruptedException.java.lang.InterruptedException: sleep interrupted
[junit] 2009-07-19 16:09:49,621 WARN namenode.DecommissionManager
(DecommissionManager.java:run(67)) - Monitor interrupted:
java.lang.InterruptedException: sleep interrupted
[junit] 2009-07-19 16:09:49,621 INFO namenode.FSNamesystem
(FSEditLog.java:printStatistics(876)) - Number of transactions: 88 Total time
for transactions(ms): 6Number of transactions batched in Syncs: 6 Number of
syncs: 62 SyncTimes(ms): 841 601
[junit] 2009-07-19 16:09:49,634 INFO ipc.Server (Server.java:stop(1103)) -
Stopping server on 58036
[junit] 2009-07-19 16:09:49,634 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 1 on 58036: exiting
[junit] 2009-07-19 16:09:49,634 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 3 on 58036: exiting
[junit] 2009-07-19 16:09:49,634 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 9 on 58036: exiting
[junit] 2009-07-19 16:09:49,634 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 2 on 58036: exiting
[junit] 2009-07-19 16:09:49,634 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 5 on 58036: exiting
[junit] 2009-07-19 16:09:49,634 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 7 on 58036: exiting
[junit] 2009-07-19 16:09:49,635 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 0 on 58036: exiting
[junit] 2009-07-19 16:09:49,635 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 8 on 58036: exiting
[junit] 2009-07-19 16:09:49,635 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 4 on 58036: exiting
[junit] 2009-07-19 16:09:49,635 INFO ipc.Server (Server.java:run(997)) -
IPC Server handler 6 on 58036: exiting
[junit] 2009-07-19 16:09:49,635 INFO ipc.Server (Server.java:run(352)) -
Stopping IPC Server listener on 58036
[junit] 2009-07-19 16:09:49,635 INFO ipc.Server (Server.java:run(539)) -
Stopping IPC Server Responder
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 103.369 sec
[junit] Running org.apache.hadoop.util.TestReflectionUtils
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.579 sec
[junit] Running org.apache.hadoop.util.TestRunJar
[junit] Creating
filehttp://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/ws/trunk/build/test/data/out
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.164 sec
checkfailure:
[touch] Creating
http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/ws/trunk/build/test/testsfailed
BUILD FAILED
http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/ws/trunk/build.xml
:614: Tests failed!
Total time: 142 minutes 13 seconds
Publishing Javadoc
Recording test results
Recording fingerprints
Publishing Clover coverage report...
