Hi,
I want to collect and analyse user logs every 5 minutes. Now we
have origin log file which generated by nginx and divided by hour, about
30,000,000 logs per hour. The log format is like this:
60.222.199.118 - - [11/Mar/2013:16:00:00 +0800] "GET ....
Because I want to firstly collect the logs into file. so I wrote a
FileEventSink, just did some modification based on
org.apache.flume.sink.hdfs.BucketWriter.java and
org.apache.flume.sink.hdfs.HDFSEventSink.java. Following is my flume
config file:
==================
a1.sources = r1
a1.channels = c1
a1.sinks = k1
a1.sources.r1.type = cn.larry.flume.source.MyExecSource //I need to
fetch time and other info into headers, so I add these logics based on
ExecSource
a1.sources.r1.command = tail -n +0 -F /data2/log/log_2013031117.log
a1.sources.r1.channels = c1
a1.sources.r1.batchSize = 1 //I set this to 1 because otherwise
it will lost data at the end of the log file, I apply this patch
https://issues.apache.org/jira/browse/FLUME-1819 but it seems no help...
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000000
a1.channels.c1.transactionCapacity = 10000
a1.sinks.k1.type = cn.larry.flume.sink.FileEventSink
a1.sinks.k1.channel = c1
a1.sinks.k1.file.path = /opt/livedata/%Y%m%d/%H
a1.sinks.k1.file.filePrefix = log-%Y%m%d%H%M
a1.sinks.k1.file.round = true
a1.sinks.k1.file.roundValue = 5
a1.sinks.k1.file.roundUnit = minute
a1.sinks.k1.file.rollInterval=300
a1.sinks.k1.file.rollSize=0
a1.sinks.k1.file.rollCount=0
a1.sinks.k1.file.batchSize=100
And because I need to change the source log file name each hour, so
I wrote a script, which does 3 things:
1. At the 1st minute per hour:
->copy a new config file, which just change the source log
file name(a1.sources.r1.command = tail -n +0 -F /data2/log/log_<new
time>.log)
->start new flume process which use the new conifg. (I did
this because if flume process die, it won't affect next hour)
2. At the 30th minute per hour:
-> kill the flume process of last hour.
This project has been run more than 10 days, most of time it works
well, but sometimes flume process crashed due to JVM error, about once
every 2 days! I used jvm version 1.6.0_27-ea. Here's the log of error
which happened on 2013-03-11:
2013-03-11 05:45:11,302 (file-k1-roll-timer-0) [INFO -
cn.larry.flume.sink.FileBucketWriter.renameBucket(FileBucketWriter.java:408)]
Renaming /opt/livedata/20130311/05/log-201303110540.1362951611255.tmp to
/opt/livedata/20130311/05/log-201303110540.1362951611255
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002b4247be034e, pid=1463, tid=1098979648
#
# JRE version: 6.0_18-b07
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.0-b13 mixed mode
linux-amd64 )
# Problematic frame:
# V [libjvm.so+0x2de34e]
#
# An error report file with more information is saved as:
# /opt/scripts/tvhadoop/flume/flume-1.3.0/bin/hs_err_pid1463.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
+ exec /usr/local/jdk/bin/java -Xmx2048m
-Dflume.root.logger=INFO,console -cp
'/opt/tvhadoop/apache-flume-1.3.1-bin/conf:/opt/tvhadoop/apache-flume-1.3.1-bin/lib/*'
-Djava.library.path= org.apache.flume.node.Application -f
/opt/tvhadoop/apache-flume-1.3.1-bin/conf/flume_2013031106.conf -n a1
and the jvm dump file is in the attachments.
I wonder how to handle this problem.
And another question is about the execSource, I don't know why it
loss data if batchsize > 1. If I use file channel, I must make a large
batchsize to fulfill the throughputs...
Thanks & Best regards,
larry
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002b4247be034e, pid=1463, tid=1098979648
#
# JRE version: 6.0_18-b07
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.0-b13 mixed mode linux-amd64 )
# Problematic frame:
# V [libjvm.so+0x2de34e]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x0000000051074800): GCTaskThread [stack: 0x0000000041711000,0x0000000041812000] [id=1490]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0xfffffffffffffffc
Registers:
RAX=0x0000000000000001, RBX=0x00002aaab6adab78, RCX=0x00002aaaae62a878, RDX=0xfffffffffffffffc
RSP=0x0000000041810bd0, RBP=0x0000000041810bf0, RSI=0x00002aaab6adab78, RDI=0x00000000510ca840
R8 =0x00002aaaae62a7e8, R9 =0x0000000000000002, R10=0x00002b424821c201, R11=0x00002b42482296e0
R12=0x00000000510ca840, R13=0x00002aaaae62a878, R14=0x00002aaaae62a878, R15=0x00000000510ca840
RIP=0x00002b4247be034e, EFL=0x0000000000010206, CSGSFS=0x0000000000000033, ERR=0x0000000000000005
TRAPNO=0x000000000000000e
Top of Stack: (sp=0x0000000041810bd0)
0x0000000041810bd0: 00000000510ca840 00002aaab58f0530
0x0000000041810be0: 00002aaab6adab78 00002aaab6adab70
0x0000000041810bf0: 0000000041810c30 00002b4247cb0e32
0x0000000041810c00: 00002aaab6adab50 00000000ffffc018
0x0000000041810c10: 00002aaab6adab50 00002aaab6adac00
0x0000000041810c20: 00002aab336446d6 0000000000000240
0x0000000041810c30: 0000000041810cc0 00002b4247b3f8d8
0x0000000041810c40: 00002aaab6ae0080 00002aab33644701
0x0000000041810c50: 00002aab33644685 00002aab33644680
0x0000000041810c60: 00002aaab6ad0c18 00002aab336448d6
0x0000000041810c70: 000000000108cf10 00000000510ca840
0x0000000041810c80: 00002aaab6b1ab98 000000005106e1b0
0x0000000041810c90: 000000005106def0 000000005108cf10
0x0000000041810ca0: 00000000510ca840 0000000051074800
0x0000000041810cb0: 00002b4248071e51 0000000000000000
0x0000000041810cc0: 0000000041810ce0 00002b4247f2783b
0x0000000041810cd0: 00002b4248213b4c 000000005108cf10
0x0000000041810ce0: 0000000041810de0 00002b4247c67dda
0x0000000041810cf0: 0000000041810d30 0000000041810d08
0x0000000041810d00: 00002b424822d160 0000000000000000
0x0000000041810d10: 0000000051075090 00000000510750c0
0x0000000041810d20: 00000000510750d0 00000000510754a8
0x0000000041810d30: 0000000051074800 00000000510754b0
0x0000000041810d40: 00000000510754e0 00000000510754f0
0x0000000041810d50: 00000000510758c8 0000000041810d80
0x0000000041810d60: 0000000051075090 00000000510750c0
0x0000000041810d70: 00000000510750d0 00000000510754a8
0x0000000041810d80: 0000000051074800 00000000510754b0
0x0000000041810d90: 00000000510754e0 00000000510754f0
0x0000000041810da0: 00000000510758c8 00000000510758d0
0x0000000041810db0: 00000000510761f0 00002b4248227b30
0x0000000041810dc0: 00000000510761f0 0000000051076110
Instructions: (pc=0x00002b4247be034e)
0x00002b4247be033e: f0 4c 89 65 f8 48 89 f3 49 89 fc 74 5d 48 8b 16
0x00002b4247be034e: 48 8b 0a 83 e1 03 48 83 f9 03 75 58 48 8b 0d 17
Stack: [0x0000000041711000,0x0000000041812000], sp=0x0000000041810bd0, free space=3fe0000000000000018k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x2de34e]
V [libjvm.so+0x3aee32]
V [libjvm.so+0x23d8d8]
V [libjvm.so+0x62583b]
V [libjvm.so+0x365dda]
V [libjvm.so+0x5da2af]
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x00002aab4004b000 JavaThread "file-k1-call-runner-9" [_thread_blocked, id=1587, stack(0x0000000043d37000,0x0000000043e38000)]
0x00002aab40048800 JavaThread "file-k1-call-runner-8" [_thread_blocked, id=1586, stack(0x0000000043c36000,0x0000000043d37000)]
0x00002aab40055800 JavaThread "file-k1-call-runner-7" [_thread_blocked, id=1585, stack(0x0000000043b35000,0x0000000043c36000)]
0x00002aab40052800 JavaThread "file-k1-call-runner-6" [_thread_blocked, id=1584, stack(0x0000000043a34000,0x0000000043b35000)]
0x00002aab40051000 JavaThread "file-k1-call-runner-5" [_thread_blocked, id=1583, stack(0x0000000043933000,0x0000000043a34000)]
0x00002aab40050000 JavaThread "file-k1-call-runner-4" [_thread_blocked, id=1582, stack(0x0000000043832000,0x0000000043933000)]
0x00002aab4005b800 JavaThread "file-k1-call-runner-3" [_thread_blocked, id=1581, stack(0x0000000043731000,0x0000000043832000)]
0x0000000051ae5800 JavaThread "Flume Avro RPC Client Call Invoker 1" daemon [_thread_blocked, id=1578, stack(0x0000000043630000,0x0000000043731000)]
0x00002aab344d6000 JavaThread "SinkRunner-PollingRunner-DefaultSinkProcessor" [_thread_blocked, id=1576, stack(0x000000004352f000,0x0000000043630000)]
0x00002aab4003f000 JavaThread "file-k1-call-runner-2" [_thread_blocked, id=1575, stack(0x0000000041160000,0x0000000041261000)]
0x00002aab4003c800 JavaThread "file-k1-call-runner-1" [_thread_blocked, id=1571, stack(0x0000000040f5d000,0x000000004105e000)]
0x000000005179a000 JavaThread "file-k1-roll-timer-0" [_thread_blocked, id=1569, stack(0x0000000040e5c000,0x0000000040f5d000)]
0x00002aab4003b800 JavaThread "file-k1-call-runner-0" [_thread_blocked, id=1568, stack(0x00000000409bd000,0x0000000040abe000)]
0x00002aab342b4000 JavaThread "Avro NettyTransceiver I/O Worker 1" daemon [_thread_in_native, id=1563, stack(0x00000000408bc000,0x00000000409bd000)]
0x00002aab34353800 JavaThread "StderrReader-[tail -n +0 -F /data2/log/vvaccess_2013031105.log]" daemon [_thread_in_native, id=1557, stack(0x000000004342e000,0x000000004352f000)]
0x000000005140d800 JavaThread "process reaper" daemon [_thread_in_native, id=1555, stack(0x000000004332d000,0x000000004342e000)]
0x00000000514e9800 JavaThread "pool-6-thread-1" [_thread_blocked, id=1554, stack(0x000000004322c000,0x000000004332d000)]
0x00002aab34189800 JavaThread "SinkRunner-PollingRunner-DefaultSinkProcessor" [_thread_blocked, id=1553, stack(0x000000004312b000,0x000000004322c000)]
0x00002aab34187800 JavaThread "lifecycleSupervisor-1-5" [_thread_blocked, id=1552, stack(0x000000004302a000,0x000000004312b000)]
0x00002aab34186000 JavaThread "lifecycleSupervisor-1-4" [_thread_blocked, id=1551, stack(0x0000000042f29000,0x000000004302a000)]
0x00002aab341b4000 JavaThread "lifecycleSupervisor-1-3" [_thread_blocked, id=1550, stack(0x0000000042e28000,0x0000000042f29000)]
0x00002aab341b3800 JavaThread "lifecycleSupervisor-1-2" [_thread_blocked, id=1541, stack(0x0000000042d27000,0x0000000042e28000)]
0x0000000051469000 JavaThread "lifecycleSupervisor-1-1" [_thread_blocked, id=1540, stack(0x0000000040d5b000,0x0000000040e5c000)]
0x00000000513d9000 JavaThread "conf-file-poller-0" [_thread_blocked, id=1537, stack(0x0000000042c26000,0x0000000042d27000)]
0x00000000513e1000 JavaThread "lifecycleSupervisor-1-0" [_thread_blocked, id=1536, stack(0x0000000042b25000,0x0000000042c26000)]
0x00000000513d5800 JavaThread "lifecycleSupervisor-1-2" [_thread_blocked, id=1535, stack(0x0000000042a24000,0x0000000042b25000)]
0x00000000510e6800 JavaThread "lifecycleSupervisor-1-1" [_thread_blocked, id=1534, stack(0x0000000042923000,0x0000000042a24000)]
0x00000000513ca000 JavaThread "lifecycleSupervisor-1-0" [_thread_blocked, id=1533, stack(0x00000000414a2000,0x00000000415a3000)]
0x00000000510ff800 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=1526, stack(0x0000000042721000,0x0000000042822000)]
0x00000000510fd800 JavaThread "CompilerThread1" daemon [_thread_blocked, id=1525, stack(0x0000000042620000,0x0000000042721000)]
0x00000000510fa800 JavaThread "CompilerThread0" daemon [_thread_blocked, id=1524, stack(0x00000000407bb000,0x00000000408bc000)]
0x00000000510f8800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=1523, stack(0x000000004010a000,0x000000004020b000)]
0x00000000510d9000 JavaThread "Finalizer" daemon [_thread_blocked, id=1517, stack(0x000000004251f000,0x0000000042620000)]
0x00000000510d1800 JavaThread "Reference Handler" daemon [_thread_blocked, id=1516, stack(0x000000004241e000,0x000000004251f000)]
0x0000000051060000 JavaThread "main" [_thread_blocked, id=1485, stack(0x000000004105f000,0x0000000041160000)]
Other Threads:
0x00000000510cd000 VMThread [stack: 0x000000004231d000,0x000000004241e000] [id=1513]
0x0000000051102800 WatcherThread [stack: 0x0000000042822000,0x0000000042923000] [id=1527]
=>0x0000000051074800 (exited) GCTaskThread [stack: 0x0000000041711000,0x0000000041812000] [id=1490]
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x000000005105da30] Threads_lock - owner thread: 0x00000000510cd000
[0x000000005105df30] Heap_lock - owner thread: 0x00000000514e9800
Heap
PSYoungGen total 12096K, used 12096K [0x00002aab08b60000, 0x00002aab18380000, 0x00002aab33600000)
eden space 11712K, 100% used [0x00002aab08b60000,0x00002aab096d0000,0x00002aab096d0000)
from space 384K, 100% used [0x00002aab18320000,0x00002aab18380000,0x00002aab18380000)
to space 576K, 61% used [0x00002aab18260000,0x00002aab182b8000,0x00002aab182f0000)
PSOldGen total 256896K, used 54490K [0x00002aaab3600000, 0x00002aaac30e0000, 0x00002aab08b60000)
object space 256896K, 21% used [0x00002aaab3600000,0x00002aaab6b36b98,0x00002aaac30e0000)
PSPermGen total 21248K, used 14588K [0x00002aaaae200000, 0x00002aaaaf6c0000, 0x00002aaab3600000)
object space 21248K, 68% used [0x00002aaaae200000,0x00002aaaaf03f3b0,0x00002aaaaf6c0000)
Dynamic libraries:
40000000-40009000 r-xp 00000000 08:11 27510390 /data1/app/jdk1.6.0_18/bin/java
40108000-4010a000 rwxp 00008000 08:11 27510390 /data1/app/jdk1.6.0_18/bin/java
4010a000-4010d000 ---p 4010a000 00:00 0
4010d000-4020b000 rwxp 4010d000 00:00 0
4020b000-4020e000 ---p 4020b000 00:00 0
4020e000-4030c000 rwxp 4020e000 00:00 0
407bb000-407be000 ---p 407bb000 00:00 0
407be000-408bc000 rwxp 407be000 00:00 0
408bc000-408bf000 ---p 408bc000 00:00 0
408bf000-409bd000 rwxp 408bf000 00:00 0
409bd000-409c0000 ---p 409bd000 00:00 0
409c0000-40abe000 rwxp 409c0000 00:00 0
40d5b000-40d5e000 ---p 40d5b000 00:00 0
40d5e000-40e5c000 rwxp 40d5e000 00:00 0
40e5c000-40e5f000 ---p 40e5c000 00:00 0
40e5f000-40f5d000 rwxp 40e5f000 00:00 0
40f5d000-40f60000 ---p 40f5d000 00:00 0
40f60000-4105e000 rwxp 40f60000 00:00 0
4105f000-41062000 ---p 4105f000 00:00 0
41062000-41160000 rwxp 41062000 00:00 0
41160000-41163000 ---p 41160000 00:00 0
41163000-41261000 rwxp 41163000 00:00 0
414a2000-414a5000 ---p 414a2000 00:00 0
414a5000-415a3000 rwxp 414a5000 00:00 0
41610000-41611000 ---p 41610000 00:00 0
41611000-41711000 rwxp 41611000 00:00 0
41711000-41712000 ---p 41711000 00:00 0
41712000-41812000 rwxp 41712000 00:00 0
41812000-41813000 ---p 41812000 00:00 0
41813000-41913000 rwxp 41813000 00:00 0
41913000-41914000 ---p 41913000 00:00 0
41914000-41a14000 rwxp 41914000 00:00 0
41a14000-41a15000 ---p 41a14000 00:00 0
41a15000-41b15000 rwxp 41a15000 00:00 0
41b15000-41b16000 ---p 41b15000 00:00 0
41b16000-41c16000 rwxp 41b16000 00:00 0
41c16000-41c17000 ---p 41c16000 00:00 0
41c17000-41d17000 rwxp 41c17000 00:00 0
41d17000-41d18000 ---p 41d17000 00:00 0
41d18000-41e18000 rwxp 41d18000 00:00 0
41e18000-41e19000 ---p 41e18000 00:00 0
41e19000-41f19000 rwxp 41e19000 00:00 0
41f19000-41f1a000 ---p 41f19000 00:00 0
41f1a000-4201a000 rwxp 41f1a000 00:00 0
4201a000-4201b000 ---p 4201a000 00:00 0
4201b000-4211b000 rwxp 4201b000 00:00 0
4211b000-4211c000 ---p 4211b000 00:00 0
4211c000-4221c000 rwxp 4211c000 00:00 0
4221c000-4221d000 ---p 4221c000 00:00 0
4221d000-4231d000 rwxp 4221d000 00:00 0
4231d000-4231e000 ---p 4231d000 00:00 0
4231e000-4241e000 rwxp 4231e000 00:00 0
4241e000-42421000 ---p 4241e000 00:00 0
42421000-4251f000 rwxp 42421000 00:00 0
4251f000-42522000 ---p 4251f000 00:00 0
42522000-42620000 rwxp 42522000 00:00 0
42620000-42623000 ---p 42620000 00:00 0
42623000-42721000 rwxp 42623000 00:00 0
42721000-42724000 ---p 42721000 00:00 0
42724000-42822000 rwxp 42724000 00:00 0
42822000-42823000 ---p 42822000 00:00 0
42823000-42923000 rwxp 42823000 00:00 0
42923000-42926000 ---p 42923000 00:00 0
42926000-42a24000 rwxp 42926000 00:00 0
42a24000-42a27000 ---p 42a24000 00:00 0
42a27000-42b25000 rwxp 42a27000 00:00 0
42b25000-42b28000 ---p 42b25000 00:00 0
42b28000-42c26000 rwxp 42b28000 00:00 0
42c26000-42c29000 ---p 42c26000 00:00 0
42c29000-42d27000 rwxp 42c29000 00:00 0
42d27000-42d2a000 ---p 42d27000 00:00 0
42d2a000-42e28000 rwxp 42d2a000 00:00 0
42e28000-42e2b000 ---p 42e28000 00:00 0
42e2b000-42f29000 rwxp 42e2b000 00:00 0
42f29000-42f2c000 ---p 42f29000 00:00 0
42f2c000-4302a000 rwxp 42f2c000 00:00 0
4302a000-4302d000 ---p 4302a000 00:00 0
4302d000-4312b000 rwxp 4302d000 00:00 0
4312b000-4312e000 ---p 4312b000 00:00 0
4312e000-4322c000 rwxp 4312e000 00:00 0
4322c000-4322f000 ---p 4322c000 00:00 0
4322f000-4332d000 rwxp 4322f000 00:00 0
4332d000-43330000 ---p 4332d000 00:00 0
43330000-4342e000 rwxp 43330000 00:00 0
4342e000-43431000 ---p 4342e000 00:00 0
43431000-4352f000 rwxp 43431000 00:00 0
4352f000-43532000 ---p 4352f000 00:00 0
43532000-43630000 rwxp 43532000 00:00 0
43630000-43633000 ---p 43630000 00:00 0
43633000-43731000 rwxp 43633000 00:00 0
43731000-43734000 ---p 43731000 00:00 0
43734000-43832000 rwxp 43734000 00:00 0
43832000-43835000 ---p 43832000 00:00 0
43835000-43933000 rwxp 43835000 00:00 0
43933000-43936000 ---p 43933000 00:00 0
43936000-43a34000 rwxp 43936000 00:00 0
43a34000-43a37000 ---p 43a34000 00:00 0
43a37000-43b35000 rwxp 43a37000 00:00 0
43b35000-43b38000 ---p 43b35000 00:00 0
43b38000-43c36000 rwxp 43b38000 00:00 0
43c36000-43c39000 ---p 43c36000 00:00 0
43c39000-43d37000 rwxp 43c39000 00:00 0
43d37000-43d3a000 ---p 43d37000 00:00 0
43d3a000-43e38000 rwxp 43d3a000 00:00 0
51055000-52bc4000 rwxp 51055000 00:00 0 [heap]
330e400000-330e41c000 r-xp 00000000 08:01 393218 /lib64/ld-2.5.so
330e61b000-330e61c000 r-xp 0001b000 08:01 393218 /lib64/ld-2.5.so
330e61c000-330e61d000 rwxp 0001c000 08:01 393218 /lib64/ld-2.5.so
330e800000-330e94d000 r-xp 00000000 08:01 393225 /lib64/libc-2.5.so
330e94d000-330eb4d000 ---p 0014d000 08:01 393225 /lib64/libc-2.5.so
330eb4d000-330eb51000 r-xp 0014d000 08:01 393225 /lib64/libc-2.5.so
330eb51000-330eb52000 rwxp 00151000 08:01 393225 /lib64/libc-2.5.so
330eb52000-330eb57000 rwxp 330eb52000 00:00 0
330ec00000-330ec02000 r-xp 00000000 08:01 393262 /lib64/libdl-2.5.so
330ec02000-330ee02000 ---p 00002000 08:01 393262 /lib64/libdl-2.5.so
330ee02000-330ee03000 r-xp 00002000 08:01 393262 /lib64/libdl-2.5.so
330ee03000-330ee04000 rwxp 00003000 08:01 393262 /lib64/libdl-2.5.so
330f000000-330f082000 r-xp 00000000 08:01 393267 /lib64/libm-2.5.so
330f082000-330f281000 ---p 00082000 08:01 393267 /lib64/libm-2.5.so
330f281000-330f282000 r-xp 00081000 08:01 393267 /lib64/libm-2.5.so
330f282000-330f283000 rwxp 00082000 08:01 393267 /lib64/libm-2.5.so
330f400000-330f416000 r-xp 00000000 08:01 393231 /lib64/libpthread-2.5.so
330f416000-330f615000 ---p 00016000 08:01 393231 /lib64/libpthread-2.5.so
330f615000-330f616000 r-xp 00015000 08:01 393231 /lib64/libpthread-2.5.so
330f616000-330f617000 rwxp 00016000 08:01 393231 /lib64/libpthread-2.5.so
330f617000-330f61b000 rwxp 330f617000 00:00 0
330fc00000-330fc07000 r-xp 00000000 08:01 393233 /lib64/librt-2.5.so
330fc07000-330fe07000 ---p 00007000 08:01 393233 /lib64/librt-2.5.so
330fe07000-330fe08000 r-xp 00007000 08:01 393233 /lib64/librt-2.5.so
330fe08000-330fe09000 rwxp 00008000 08:01 393233 /lib64/librt-2.5.so
3314000000-3314015000 r-xp 00000000 08:01 393410 /lib64/libnsl-2.5.so
3314015000-3314214000 ---p 00015000 08:01 393410 /lib64/libnsl-2.5.so
3314214000-3314215000 r-xp 00014000 08:01 393410 /lib64/libnsl-2.5.so
3314215000-3314216000 rwxp 00015000 08:01 393410 /lib64/libnsl-2.5.so
3314216000-3314218000 rwxp 3314216000 00:00 0
2aaaaaaab000-2aaaaaaaf000 r-xs 0002a000 08:06 63832844 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/avro-ipc-1.7.2.jar
2aaaaaaaf000-2aaaaaab1000 r-xs 00003000 08:06 63832873 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/jcl-over-slf4j-1.7.2.jar
2aaaaaab1000-2aaaaaaba000 r-xs 0006d000 08:06 63832838 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/log4j-1.2.16.jar
2aaaaaaba000-2aaaaaabc000 r-xs 0000f000 08:06 63834166 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/sohu-flume-1.0.jar
2aaaaaac0000-2aaaaaacd000 r-xp 00000000 08:11 27511026 /data1/app/jdk1.6.0_18/jre/lib/amd64/libverify.so
2aaaaaacd000-2aaaaabcc000 ---p 0000d000 08:11 27511026 /data1/app/jdk1.6.0_18/jre/lib/amd64/libverify.so
2aaaaabcc000-2aaaaabcf000 rwxp 0000c000 08:11 27511026 /data1/app/jdk1.6.0_18/jre/lib/amd64/libverify.so
2aaaaabcf000-2aaaaabf8000 r-xp 00000000 08:11 27511044 /data1/app/jdk1.6.0_18/jre/lib/amd64/libjava.so
2aaaaabf8000-2aaaaacf7000 ---p 00029000 08:11 27511044 /data1/app/jdk1.6.0_18/jre/lib/amd64/libjava.so
2aaaaacf7000-2aaaaacfe000 rwxp 00028000 08:11 27511044 /data1/app/jdk1.6.0_18/jre/lib/amd64/libjava.so
2aaaaacfe000-2aaaaacff000 ---p 2aaaaacfe000 00:00 0
2aaaaacff000-2aaaaad00000 rwxp 2aaaaacff000 00:00 0
2aaaaad00000-2aaaaad07000 r-xp 00000000 08:11 27511033 /data1/app/jdk1.6.0_18/jre/lib/amd64/native_threads/libhpi.so
2aaaaad07000-2aaaaae08000 ---p 00007000 08:11 27511033 /data1/app/jdk1.6.0_18/jre/lib/amd64/native_threads/libhpi.so
2aaaaae08000-2aaaaae0a000 rwxp 00008000 08:11 27511033 /data1/app/jdk1.6.0_18/jre/lib/amd64/native_threads/libhpi.so
2aaaaae0a000-2aaaaae0b000 rwxp 2aaaaae0a000 00:00 0
2aaaaae0b000-2aaaaae13000 rwxs 00000000 00:13 3645065185 /tmp/hsperfdata_tvhadoop/1463
2aaaaae13000-2aaaaae18000 r-xs 0002a000 08:06 63832857 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/gson-2.2.2.jar
2aaaaae1f000-2aaaaae29000 r-xp 00000000 08:01 393240 /lib64/libnss_files-2.5.so
2aaaaae29000-2aaaab028000 ---p 0000a000 08:01 393240 /lib64/libnss_files-2.5.so
2aaaab028000-2aaaab029000 r-xp 00009000 08:01 393240 /lib64/libnss_files-2.5.so
2aaaab029000-2aaaab02a000 rwxp 0000a000 08:01 393240 /lib64/libnss_files-2.5.so
2aaaab02a000-2aaaab038000 r-xp 00000000 08:11 27511031 /data1/app/jdk1.6.0_18/jre/lib/amd64/libzip.so
2aaaab038000-2aaaab13a000 ---p 0000e000 08:11 27511031 /data1/app/jdk1.6.0_18/jre/lib/amd64/libzip.so
2aaaab13a000-2aaaab13d000 rwxp 00010000 08:11 27511031 /data1/app/jdk1.6.0_18/jre/lib/amd64/libzip.so
2aaaab13d000-2aaaab3ae000 rwxp 2aaaab13d000 00:00 0
2aaaab3ae000-2aaaae13e000 rwxp 2aaaab3ae000 00:00 0
2aaaae13e000-2aaaae148000 rwxp 2aaaae13e000 00:00 0
2aaaae148000-2aaaae1fe000 rwxp 2aaaae148000 00:00 0
2aaaae200000-2aaaaf6c0000 rwxp 2aaaae200000 00:00 0
2aaaaf6c0000-2aaab3600000 rwxp 2aaaaf6c0000 00:00 0
2aaab3600000-2aaac30e0000 rwxp 2aaab3600000 00:00 0
2aaac30e0000-2aab08b60000 rwxp 2aaac30e0000 00:00 0
2aab08b60000-2aab18380000 rwxp 2aab08b60000 00:00 0
2aab18380000-2aab20800000 ---p 2aab18380000 00:00 0
2aab20800000-2aab33600000 rwxp 2aab20800000 00:00 0
2aab33600000-2aab3360b000 rwxp 2aab33600000 00:00 0
2aab3360b000-2aab3362a000 rwxp 2aab3360b000 00:00 0
2aab3362a000-2aab336a8000 rwxp 2aab3362a000 00:00 0
2aab336a8000-2aab338d4000 rwxp 2aab336a8000 00:00 0
2aab338d4000-2aab33951000 rwxp 2aab338d4000 00:00 0
2aab33951000-2aab33993000 ---p 2aab33951000 00:00 0
2aab33993000-2aab33a2a000 rwxp 2aab33993000 00:00 0
2aab33a2a000-2aab33aa9000 rwxp 2aab33a2a000 00:00 0
2aab33aa9000-2aab33cd6000 rwxp 2aab33aa9000 00:00 0
2aab33cd6000-2aab33ce1000 rwxp 2aab33cd6000 00:00 0
2aab33ce1000-2aab33d00000 rwxp 2aab33ce1000 00:00 0
2aab33d00000-2aab33d28000 rwxp 2aab33d00000 00:00 0
2aab33d28000-2aab33ebe000 r-xs 02fc6000 08:11 27511110 /data1/app/jdk1.6.0_18/jre/lib/rt.jar
2aab33ebe000-2aab33ff2000 rwxp 2aab33ebe000 00:00 0
2aab33ff2000-2aab33ffa000 r-xs 00053000 08:06 63832834 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-ng-core-1.3.1.jar
2aab33ffa000-2aab33ffc000 r-xs 0001f000 08:06 63832861 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/irclib-1.10.jar
2aab33ffc000-2aab33ffe000 r-xs 00004000 08:06 63832874 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-ng-elasticsearch-sink-1.3.1.jar
2aab34000000-2aab34ae1000 rwxp 2aab34000000 00:00 0
2aab34ae1000-2aab38000000 ---p 2aab34ae1000 00:00 0
2aab38000000-2aab38d0d000 rwxp 2aab38000000 00:00 0
2aab38d0d000-2aab3c2de000 r-xp 00000000 08:05 483601 /usr/lib/locale/locale-archive
2aab3c2de000-2aab3c2e2000 r-xs 00028000 08:06 63832846 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/jetty-util-6.1.26.jar
2aab3c2e2000-2aab3c2f0000 r-xs 00090000 08:06 63832858 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/mina-core-2.0.4.jar
2aab3c2f0000-2aab3c2f5000 r-xs 00040000 08:06 63832849 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/commons-lang-2.5.jar
2aab3c2f5000-2aab3c2f7000 r-xs 0000b000 08:06 63832862 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-jdbc-channel-1.3.1.jar
2aab3c2f7000-2aab3c2fa000 r-xs 0001e000 08:06 63832856 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/servlet-api-2.5-20110124.jar
2aab3c2fa000-2aab3c306000 r-xs 000b3000 08:06 63832872 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/zookeeper-3.4.5.jar
2aab3c306000-2aab3c308000 r-xs 000f2000 08:06 63832843 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/snappy-java-1.0.4.1.jar
2aab3c308000-2aab3c30a000 r-xs 00002000 08:06 63832860 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-irc-sink-1.3.1.jar
2aab3c30a000-2aab3c30d000 r-xs 00015000 08:06 63832864 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/commons-pool-1.5.4.jar
2aab3c30d000-2aab3c30e000 r-xs 00004000 08:06 63832877 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-avro-source-1.3.1.jar
2aab3c30e000-2aab3c310000 r-xs 0000c000 08:06 63832878 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-thrift-source-1.3.1.jar
2aab3c310000-2aab3c313000 r-xs 0000b000 08:06 63832832 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-ng-sdk-1.3.1.jar
2aab3c313000-2aab3c316000 r-xs 00020000 08:06 63832870 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/asynchbase-1.2.0.jar
2aab3c316000-2aab3c329000 r-xs 000d7000 08:06 63832850 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/netty-3.4.0.Final.jar
2aab3c329000-2aab3c32f000 r-xs 00068000 08:06 63832867 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/protobuf-java-2.4.1.jar
2aab3c32f000-2aab3c331000 r-xs 00007000 08:06 63832869 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-ng-hbase-sink-1.3.1.jar
2aab3c331000-2aab3c333000 r-xs 00006000 08:06 63832842 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/paranamer-2.3.jar
2aab3c333000-2aab3c335000 r-xs 0000c000 08:06 63832833 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-ng-configuration-1.3.1.jar
2aab3c335000-2aab3c33d000 r-xs 00066000 08:06 63832847 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/velocity-1.7.jar
2aab3c33d000-2aab3c34b000 r-xs 0007f000 08:06 63832848 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/commons-collections-3.2.1.jar
2aab3c34b000-2aab3c34d000 r-xs 0000a000 08:06 63832859 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-hdfs-sink-1.3.1.jar
2aab3c34d000-2aab3c34e000 r-xs 0000a000 08:06 63832875 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-scribe-source-1.3.1.jar
2aab3c34e000-2aab3c354000 r-xs 00043000 08:06 63832839 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/avro-1.7.2.jar
2aab3c354000-2aab3c372000 r-xs 00151000 08:06 63832851 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/guava-10.0.1.jar
2aab3c372000-2aab3c382000 r-xs 0007c000 08:06 63832855 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/joda-time-2.1.jar
2aab3c382000-2aab3c385000 r-xs 00025000 08:06 63832863 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/commons-dbcp-1.4.jar
2aab3c385000-2aab3c389000 r-xs 0003a000 08:06 63832866 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-file-channel-1.3.1.jar
2aab3c389000-2aab3c38a000 r-xs 00002000 08:06 63832879 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-ng-log4jappender-1.3.1.jar
2aab3c38a000-2aab3c38c000 r-xs 00009000 08:06 63832854 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/commons-cli-1.2.jar
2aab3c38c000-2aab3c39a000 r-xs 000af000 08:06 63832841 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/jackson-mapper-asl-1.9.3.jar
2aab3c39a000-2aab3c39b000 r-xs 00002000 08:06 63832837 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/slf4j-log4j12-1.6.1.jar
2aab3c3a7000-2aab3c3ad000 r-xs 00043000 08:06 63832876 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/libthrift-0.6.1.jar
2aab3c3ad000-2aab3c3b0000 r-xs 00025000 08:06 63832853 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/commons-io-2.1.jar
2aab3c3b0000-2aab3c3b7000 r-xs 0007d000 08:06 63832845 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/jetty-6.1.26.jar
2aab3c3b7000-2aab3c3ba000 r-xs 00006000 08:06 63832852 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/jsr305-1.3.9.jar
2aab3c3ba000-2aab3c3be000 r-xs 00034000 08:06 63832840 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/jackson-core-asl-1.9.3.jar
2aab3c3be000-2aab3c3c0000 r-xs 00005000 08:06 63832836 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/slf4j-api-1.6.1.jar
2aab3c3c0000-2aab3c3c1000 r-xs 00007000 08:06 63832835 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-ng-node-1.3.1.jar
2aab3c3c1000-2aab3c3c3000 r-xs 00003000 08:06 63832871 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/async-1.3.1.jar
2aab3c3c3000-2aab3c3c5000 r-xs 00009000 08:06 63832868 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/flume-recoverable-memory-channel-1.3.1.jar
2aab3c3c5000-2aab3c3ed000 r-xs 00265000 08:06 63832865 /opt/tvhadoop/apache-flume-1.3.1-bin/lib/derby-10.8.2.2.jar
2aab3c3ed000-2aab3c400000 r-xp 00000000 08:11 27511020 /data1/app/jdk1.6.0_18/jre/lib/amd64/libnet.so
2aab3c400000-2aab3c501000 ---p 00013000 08:11 27511020 /data1/app/jdk1.6.0_18/jre/lib/amd64/libnet.so
2aab3c501000-2aab3c504000 rwxp 00014000 08:11 27511020 /data1/app/jdk1.6.0_18/jre/lib/amd64/libnet.so
2aab3c504000-2aab3c804000 rwxp 2aab3c504000 00:00 0
2aab3c804000-2aab3c80a000 r-xp 00000000 08:11 27511061 /data1/app/jdk1.6.0_18/jre/lib/amd64/libmanagement.so
2aab3c80a000-2aab3c909000 ---p 00006000 08:11 27511061 /data1/app/jdk1.6.0_18/jre/lib/amd64/libmanagement.so
2aab3c909000-2aab3c90b000 rwxp 00005000 08:11 27511061 /data1/app/jdk1.6.0_18/jre/lib/amd64/libmanagement.so
2aab3c90b000-2aab3c912000 r-xp 00000000 08:11 27511021 /data1/app/jdk1.6.0_18/jre/lib/amd64/libnio.so
2aab3c912000-2aab3ca11000 ---p 00007000 08:11 27511021 /data1/app/jdk1.6.0_18/jre/lib/amd64/libnio.so
2aab3ca11000-2aab3ca13000 rwxp 00006000 08:11 27511021 /data1/app/jdk1.6.0_18/jre/lib/amd64/libnio.so
2aab40000000-2aab4105d000 rwxp 2aab40000000 00:00 0
2aab4105d000-2aab44000000 ---p 2aab4105d000 00:00 0
2b42477e0000-2b42477e1000 rwxp 2b42477e0000 00:00 0
2b42477f5000-2b42477f6000 rwxp 2b42477f5000 00:00 0
2b42477f6000-2b42477fd000 r-xp 00000000 08:11 27511016 /data1/app/jdk1.6.0_18/jre/lib/amd64/jli/libjli.so
2b42477fd000-2b42478fe000 ---p 00007000 08:11 27511016 /data1/app/jdk1.6.0_18/jre/lib/amd64/jli/libjli.so
2b42478fe000-2b4247900000 rwxp 00008000 08:11 27511016 /data1/app/jdk1.6.0_18/jre/lib/amd64/jli/libjli.so
2b4247900000-2b4247902000 rwxp 2b4247900000 00:00 0
2b4247902000-2b42480b6000 r-xp 00000000 08:11 27525122 /data1/app/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so
2b42480b6000-2b42481b6000 ---p 007b4000 08:11 27525122 /data1/app/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so
2b42481b6000-2b4248340000 rwxp 007b4000 08:11 27525122 /data1/app/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so
2b4248340000-2b4248379000 rwxp 2b4248340000 00:00 0
7fff9c8e9000-7fff9c8fe000 rwxp 7ffffffea000 00:00 0 [stack]
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
VM Arguments:
jvm_args: -Xmx2048m -Dflume.root.logger=INFO,console -Djava.library.path=
java_command: org.apache.flume.node.Application -f /opt/tvhadoop/apache-flume-1.3.1-bin/conf/flume_vv_2013031105.conf -n a1
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=/usr/local/jdk
PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/jdk/bin:/usr/local/ant/bin:/usr/local/mysql/bin:/home/tvhadoop/bin:/usr/local/jdk/bin:/usr/local/ant/bin:/usr/local/mysql/bin:/home/tvhadoop/bin:/usr/local/jdk/bin:/usr/local/ant/bin:/usr/local/mysql/bin:/home/tvhadoop/bin
LD_LIBRARY_PATH=/data1/app/jdk1.6.0_18/jre/lib/amd64/server:/data1/app/jdk1.6.0_18/jre/lib/amd64:/data1/app/jdk1.6.0_18/jre/../lib/amd64
SHELL=/bin/bash
Signal Handlers:
SIGSEGV: [libjvm.so+0x70f1a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGBUS: [libjvm.so+0x70f1a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGFPE: [libjvm.so+0x5d7f70], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGPIPE: [libjvm.so+0x5d7f70], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGXFSZ: [libjvm.so+0x5d7f70], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGILL: [libjvm.so+0x5d7f70], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: [libjvm.so+0x5da790], sa_mask[0]=0x00000000, sa_flags=0x10000004
SIGHUP: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGINT: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGTERM: [libjvm.so+0x5da4e0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGQUIT: [libjvm.so+0x5da4e0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
--------------- S Y S T E M ---------------
OS:Red Hat Enterprise Linux Server release 5.4 (Tikanga)
uname:Linux 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64
libc:glibc 2.5 NPTL 2.5
rlimit: STACK 10240k, CORE 0k, NPROC 16384, NOFILE 65535, AS infinity
load average:0.07 0.20 0.17
CPU:total 16 (8 cores per cpu, 2 threads per core) family 6 model 26 stepping 5, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, ht
Memory: 4k page, physical 24661048k(5775876k free), swap 4192956k(4192860k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (16.0-b13) for linux-amd64 JRE (1.6.0_18-b07), built on Dec 17 2009 13:42:22 by "java_re" with gcc 3.2.2 (SuSE Linux)
time: Mon Mar 11 05:46:19 2013
elapsed time: 2667 seconds
