See <https://builds.apache.org/job/Blur-master-jdk7/org.apache.blur$blur-mapred/10/>
------------------------------------------ [...truncated 1430 lines...] 13/09/22 03:02:09 INFO mapred.JobInProgress: Job job_20130922025930838_0004 has completed successfully. 13/09/22 03:02:09 INFO mapred.JobInProgress$JobSummary: jobId=job_20130922025930838_0004,submitTime=1379818846750,launchTime=1379818846779,firstMapTaskLaunchTime=1379818848435,firstReduceTaskLaunchTime=1379818850258,firstJobSetupTaskLaunchTime=1379818846921,firstJobCleanupTaskLaunchTime=1379818927792,finishTime=1379818929001,numMaps=2,numSlotsPerMap=1,numReduces=14,numSlotsPerReduce=1,user=jenkins,queue=default,status=SUCCEEDED,mapSlotSeconds=6,reduceSlotsSeconds=153,clusterMapCapacity=2,clusterReduceCapacity=2,jobName=blur index 13/09/22 03:02:09 INFO mapred.JobHistory: Moving <https://builds.apache.org/job/Blur-master-jdk7/org.apache.blur$blur-mapred/ws/target/BlurOutputFormatTest/hadoop_log/history/job_20130922025930838_0004_1379818846750_jenkins_blur+index> to <https://builds.apache.org/job/Blur-master-jdk7/org.apache.blur$blur-mapred/ws/target/BlurOutputFormatTest/hadoop_log/history/done/version-1/localhost_1379818771196_/2013/09/22/000000> 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_m_000000_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_m_000001_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_m_000002_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_m_000003_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000000_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000001_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000002_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000003_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000004_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000005_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000006_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000007_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000008_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000009_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000010_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000011_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000012_0' 13/09/22 03:02:09 INFO mapred.JobTracker: Removing task 'attempt_20130922025930838_0004_r_000013_0' 13/09/22 03:02:09 INFO mapred.JobHistory: Moving <https://builds.apache.org/job/Blur-master-jdk7/org.apache.blur$blur-mapred/ws/target/BlurOutputFormatTest/hadoop_log/history/job_20130922025930838_0004_conf.xml> to <https://builds.apache.org/job/Blur-master-jdk7/org.apache.blur$blur-mapred/ws/target/BlurOutputFormatTest/hadoop_log/history/done/version-1/localhost_1379818771196_/2013/09/22/000000> 13/09/22 03:02:09 INFO mapred.JvmManager: JVM : jvm_20130922025930838_0004_m_-301704684 exited with exit code 0. Number of tasks it ran: 1 13/09/22 03:02:09 INFO mapred.TaskTracker: Received KillTaskAction for task: attempt_20130922025930838_0004_r_000013_0 13/09/22 03:02:09 INFO mapred.TaskTracker: About to purge task: attempt_20130922025930838_0004_r_000013_0 13/09/22 03:02:09 INFO mapred.TaskTracker: Received 'KillJobAction' for job: job_20130922025930838_0004 13/09/22 03:02:09 INFO mapred.IndexCache: Map ID attempt_20130922025930838_0004_m_000002_0 not found in cache 13/09/22 03:02:09 INFO mapred.UserLogCleaner: Adding job_20130922025930838_0004 for user-log deletion with retainTimeStamp:1379905329284 13/09/22 03:02:09 INFO mapred.JobClient: map 100% reduce 100% 13/09/22 03:02:09 INFO mapred.JobClient: Task Id : attempt_20130922025930838_0004_m_000002_0, Status : SUCCEEDED 13/09/22 03:02:09 INFO mapred.JobClient: Job complete: job_20130922025930838_0004 13/09/22 03:02:09 INFO mapred.JobClient: Counters: 37 13/09/22 03:02:09 INFO mapred.JobClient: org.apache.blur.mapreduce.lib.BlurCounters 13/09/22 03:02:09 INFO mapred.JobClient: RECORD_COUNT=160000 13/09/22 03:02:09 INFO mapred.JobClient: RECORD_RATE=0 13/09/22 03:02:09 INFO mapred.JobClient: COLUMN_COUNT=160000 13/09/22 03:02:09 INFO mapred.JobClient: LUCENE_FIELD_COUNT=400000 13/09/22 03:02:09 INFO mapred.JobClient: ROW_DELETE_COUNT=0 13/09/22 03:02:09 INFO mapred.JobClient: RECORD_DUPLICATE_COUNT=0 13/09/22 03:02:09 INFO mapred.JobClient: ROW_OVERFLOW_COUNT=50 13/09/22 03:02:09 INFO mapred.JobClient: COPY_RATE=0 13/09/22 03:02:09 INFO mapred.JobClient: ROW_RATE=0 13/09/22 03:02:09 INFO mapred.JobClient: ROW_COUNT=64 13/09/22 03:02:09 INFO mapred.JobClient: Job Counters 13/09/22 03:02:09 INFO mapred.JobClient: Launched reduce tasks=14 13/09/22 03:02:09 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=6571 13/09/22 03:02:09 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/09/22 03:02:09 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/09/22 03:02:09 INFO mapred.JobClient: Rack-local map tasks=2 13/09/22 03:02:09 INFO mapred.JobClient: Launched map tasks=2 13/09/22 03:02:09 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=153260 13/09/22 03:02:09 INFO mapred.JobClient: File Output Format Counters 13/09/22 03:02:09 INFO mapred.JobClient: Bytes Written=0 13/09/22 03:02:09 INFO mapred.JobClient: FileSystemCounters 13/09/22 03:02:09 INFO mapred.JobClient: FILE_BYTES_READ=4744764 13/09/22 03:02:09 INFO mapred.JobClient: FILE_BYTES_WRITTEN=7328653 13/09/22 03:02:09 INFO mapred.JobClient: File Input Format Counters 13/09/22 03:02:09 INFO mapred.JobClient: Bytes Read=1945350 13/09/22 03:02:09 INFO mapred.JobClient: Map-Reduce Framework 13/09/22 03:02:09 INFO mapred.JobClient: Map output materialized bytes=2796018 13/09/22 03:02:09 INFO mapred.JobClient: Map input records=80000 13/09/22 03:02:09 INFO mapred.JobClient: Reduce shuffle bytes=2796018 13/09/22 03:02:09 INFO mapred.JobClient: Spilled Records=160000 13/09/22 03:02:09 INFO mapred.JobClient: Map output bytes=2635850 13/09/22 03:02:09 INFO mapred.JobClient: Total committed heap usage (bytes)=1949433856 13/09/22 03:02:09 INFO mapred.JobClient: CPU time spent (ms)=85090 13/09/22 03:02:09 INFO mapred.JobClient: Combine input records=0 13/09/22 03:02:09 INFO mapred.JobClient: SPLIT_RAW_BYTES=356 13/09/22 03:02:09 INFO mapred.JobClient: Reduce input records=80000 13/09/22 03:02:09 INFO mapred.JobClient: Reduce input groups=50 13/09/22 03:02:09 INFO mapred.JobClient: Combine output records=0 13/09/22 03:02:09 INFO mapred.JobClient: Physical memory (bytes) snapshot=2034704384 13/09/22 03:02:09 INFO mapred.JobClient: Reduce output records=80000 13/09/22 03:02:09 INFO mapred.JobClient: Virtual memory (bytes) snapshot=6717722624 13/09/22 03:02:09 INFO mapred.JobClient: Map output records=80000 Counters: Counters: 37 FileSystemCounters FILE_BYTES_READ=4744764 FILE_BYTES_WRITTEN=7328653 org.apache.blur.mapreduce.lib.BlurCounters COLUMN_COUNT=160000 COPY_RATE=0 LUCENE_FIELD_COUNT=400000 RECORD_COUNT=160000 RECORD_DUPLICATE_COUNT=0 RECORD_RATE=0 ROW_COUNT=64 ROW_DELETE_COUNT=0 ROW_OVERFLOW_COUNT=50 ROW_RATE=0 Job Counters Total time spent by all maps waiting after reserving slots (ms)=0 Total time spent by all reduces waiting after reserving slots (ms)=0 Rack-local map tasks=2 SLOTS_MILLIS_MAPS=6571 SLOTS_MILLIS_REDUCES=153260 Launched map tasks=2 Launched reduce tasks=14 Map-Reduce Framework Combine input records=0 Combine output records=0 Total committed heap usage (bytes)=1949433856 CPU time spent (ms)=85090 Map input records=80000 Map output bytes=2635850 Map output materialized bytes=2796018 Map output records=80000 Physical memory (bytes) snapshot=2034704384 Reduce input groups=50 Reduce input records=80000 Reduce output records=80000 Reduce shuffle bytes=2796018 Spilled Records=160000 SPLIT_RAW_BYTES=356 Virtual memory (bytes) snapshot=6717722624 File Input Format Counters Bytes Read=1945350 File Output Format Counters Bytes Written=0 13/09/22 03:02:09 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurMutate in order to ship it to the cluster. 13/09/22 03:02:09 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurMutate in order to ship it to the cluster. 13/09/22 03:02:09 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurOutputFormat in order to ship it to the cluster. 13/09/22 03:02:09 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/09/22 03:02:09 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String). 13/09/22 03:02:09 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-jenkins/mapred/staging/jenkins/.staging/job_20130922025930838_0005 13/09/22 03:02:09 INFO mapred.TaskTracker: Shutting down: Map-events fetcher for all reduce tasks on tracker_host0.foo.com:localhost/127.0.0.1:50578 13/09/22 03:02:09 INFO ipc.Server: Stopping server on 50578 13/09/22 03:02:09 INFO ipc.Server: IPC Server handler 1 on 50578: exiting 13/09/22 03:02:09 INFO ipc.Server: IPC Server handler 2 on 50578: exiting 13/09/22 03:02:09 INFO ipc.Server: IPC Server handler 3 on 50578: exiting 13/09/22 03:02:09 INFO ipc.Server: Stopping IPC Server listener on 50578 13/09/22 03:02:09 INFO filecache.TrackerDistributedCacheManager: Cleanup... java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.filecache.TrackerDistributedCacheManager$CleanupThread.run(TrackerDistributedCacheManager.java:1020) 13/09/22 03:02:09 INFO ipc.Server: IPC Server handler 0 on 50578: exiting 13/09/22 03:02:09 INFO metrics.RpcInstrumentation: shut down 13/09/22 03:02:09 INFO ipc.Server: Stopping IPC Server Responder 13/09/22 03:02:09 INFO mapred.TaskTracker: Shutting down StatusHttpServer 13/09/22 03:02:09 INFO mortbay.log: Stopped [email protected]:0 13/09/22 03:03:10 ERROR mapred.TaskTracker: Caught exception: java.io.IOException: Call to localhost/127.0.0.1:51672 failed on local exception: java.nio.channels.ClosedByInterruptException at org.apache.hadoop.ipc.Client.wrapException(Client.java:1150) at org.apache.hadoop.ipc.Client.call(Client.java:1118) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229) at org.apache.hadoop.mapred.$Proxy10.heartbeat(Unknown Source) at org.apache.hadoop.mapred.TaskTracker.transmitHeartBeat(TaskTracker.java:2030) at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:1824) at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:2676) at org.apache.hadoop.mapred.MiniMRCluster$TaskTrackerRunner.run(MiniMRCluster.java:217) at java.lang.Thread.run(Thread.java:724) Caused by: java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:473) at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.DataOutputStream.flush(DataOutputStream.java:123) at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:824) at org.apache.hadoop.ipc.Client.call(Client.java:1094) ... 7 more 13/09/22 03:03:10 INFO mapred.JobTracker: Stopping infoServer 13/09/22 03:03:10 INFO mortbay.log: Stopped SelectChannelConnector@localhost:0 13/09/22 03:03:10 INFO mapred.JobTracker: Stopping interTrackerServer 13/09/22 03:03:10 INFO ipc.Server: Stopping server on 51672 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 0 on 51672: exiting 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 4 on 51672: exiting 13/09/22 03:03:10 INFO metrics.RpcInstrumentation: shut down 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 2 on 51672: exiting 13/09/22 03:03:10 INFO mapred.JobTracker: Stopped interTrackerServer 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 8 on 51672: exiting 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 3 on 51672: exiting 13/09/22 03:03:10 INFO ipc.Server: Stopping IPC Server Responder 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 6 on 51672: exiting 13/09/22 03:03:10 INFO ipc.Server: Stopping IPC Server listener on 51672 13/09/22 03:03:10 INFO mapred.JobTracker: Stopping expireTrackers 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 1 on 51672: exiting 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 7 on 51672: exiting 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 9 on 51672: exiting 13/09/22 03:03:10 INFO ipc.Server: IPC Server handler 5 on 51672: exiting 13/09/22 03:03:10 INFO mapred.JobTracker: Stopping retirer 13/09/22 03:03:10 INFO mapred.EagerTaskInitializationListener: Stopping Job Init Manager thread 13/09/22 03:03:10 INFO mapred.EagerTaskInitializationListener: JobInitManagerThread interrupted. 13/09/22 03:03:10 INFO mapred.EagerTaskInitializationListener: Shutting down thread pool 13/09/22 03:03:10 INFO mapred.JobTracker: Stopping expireLaunchingTasks 13/09/22 03:03:10 INFO mapred.JobTracker: Stopping job history server 13/09/22 03:03:10 INFO mapred.JobTracker: stopped all jobtracker services Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 219.443 sec Running org.apache.blur.mapreduce.lib.CsvBlurDriverTest 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurMutate in order to ship it to the cluster. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurMutate in order to ship it to the cluster. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurOutputFormat in order to ship it to the cluster. Missing required options: c, t, d usage: csvloader The "csvloader" command is used to load delimited into a Blur table. The required options are "-c", "-t", "-d". The standard format for the contents of a file is:"rowid,recordid,family,col1,col2,...". However there are several options, such as the rowid and recordid can be generated based on the data in the record via the "-A" and "-a" options. The family can assigned based on the path via the "-I" option. The column name order can be mapped via the "-d" option. Also you can set the input format to either sequence files vie the "-S" option or leave the default text files. -A No Row Ids - Automatically generate row ids for each record based on a MD5 has of the data within the record. -a No Record Ids - Automatically generate record ids for each record based on a MD5 has of the data within the record. -b <size> The maximum number of Lucene documents to buffer in the reducer for a single row before spilling over to disk. (default 1000) -c <controller*> * Thrift controller connection string. (host1:40010 host2:40010 ...) -C <minimum maximum> Enables a combine file input to help deal with many small files as the input. Provide the minimum and maximum size per mapper. For a minimum of 1GB and a maximum of 2.5GB: (1000000000 2500000000) -d <family column*> * Define the mapping of fields in the CSV file to column names. (family col1 col2 col3 ...) -I <family path*> The directory to index with a family name, the family name is assumed to NOT be present in the file contents. (family hdfs://namenode/input/in1) -i <path*> The directory to index, the family name is assumed to BE present in the file contents. (hdfs://namenode/input/in1) -l Disable the use storage local on the server that is running the reducing task and copy to Blur table once complete. (enabled by default) -o Disable optimize indexes during copy, this has very little overhead. (enabled by default) -p <codec> Sets the compression codec for the map compress output setting. (SNAPPY,GZIP,BZIP,DEFAULT, or classname) -r <multiplier> The reducer multipler allows for an increase in the number of reducers per shard in the given table. For example if the table has 128 shards and the reducer multiplier is 4 the total number of reducers will be 512, 4 reducers per shard. (default 1) -s <delimiter> The file delimiter to be used. (default value ',') NOTE: For special charactors like the default hadoop separator of ASCII value 1, you can use standard java escaping (\u0001) -S The input files are sequence files. -t <tablename> * Blur table name. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurMutate in order to ship it to the cluster. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.CsvBlurDriver$CsvBlurCombineSequenceFileInputFormat in order to ship it to the cluster. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurMutate in order to ship it to the cluster. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurOutputFormat in order to ship it to the cluster. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurMutate in order to ship it to the cluster. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.CsvBlurDriver$CsvBlurCombineSequenceFileInputFormat in order to ship it to the cluster. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurMutate in order to ship it to the cluster. 13/09/22 03:03:10 WARN lib.BlurMapReduceUtil: Could not find jar for class class org.apache.blur.mapreduce.lib.BlurOutputFormat in order to ship it to the cluster. Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.08 sec Results : Tests run: 16, Failures: 0, Errors: 0, Skipped: 0
