I change phoenix lib from 4.6.0 to 4.5.1, logs come back.......
2016-02-14 15:27 GMT+08:00 Heng Chen <heng.chen.1...@gmail.com>: > I find some hints, the log seems to be disappear after i install > phoenix, some suspicious logs below i found > > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/home/maintain/hadoop/hbase/hbase-1.1.1/lib/phoenix-4.6.0-HBase-1.1-client.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/home/maintain/hadoop/hbase/hbase-1.1.1/lib/phoenix-server-4.6.0-HBase-1.1-runnable.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/home/maintain/hadoop/hbase/hbase-1.1.1/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > > 2016-02-14 15:17 GMT+08:00 Heng Chen <heng.chen.1...@gmail.com>: > >> This happens after i upgrade my cluster from 0.98 to 1.1 >> >> >> >> 2016-02-14 12:47 GMT+08:00 Heng Chen <heng.chen.1...@gmail.com>: >> >>> I am not sure why this happens, this is my command >>> >>> maintain 11444 66.9 1.1 10386988 1485888 pts/0 Sl 12:33 6:30 >>> /usr/java/jdk/bin/java -Dproc_regionserver -XX:OnOutOfMemoryError=kill -9 >>> %p -Xmx8000m -XX:+UseConcMarkSweepGC -verbose:gc -XX:+PrintGCDetails >>> -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 >>> -XX:GCLogFileSize=512M >>> -Dhbase.log.dir=/home/maintain/hadoop/hbase/hbase-1.1.1/logs >>> -Dhbase.log.file=hbase-maintain-regionserver-dx-pipe-regionserver7-online.log >>> -Dhbase.home.dir=/home/maintain/hadoop/hbase/hbase-1.1.1 >>> -Dhbase.id.str=maintain -Dhbase.root.logger=INFO,RFA >>> -Dhbase.security.logger=INFO,RFAS >>> org.apache.hadoop.hbase.regionserver.HRegionServer start >>> >>> >>> in hbase-maintain-regionserver-dx-pipe-regionserver7-online.log there >>> is only information below: >>> >>> Sun Feb 14 12:33:19 CST 2016 Starting regionserver on >>> dx-pipe-regionserver7-online >>> core file size (blocks, -c) 1024 >>> data seg size (kbytes, -d) unlimited >>> scheduling priority (-e) 0 >>> file size (blocks, -f) unlimited >>> pending signals (-i) 514904 >>> max locked memory (kbytes, -l) 64 >>> max memory size (kbytes, -m) unlimited >>> open files (-n) 65536 >>> pipe size (512 bytes, -p) 8 >>> POSIX message queues (bytes, -q) 819200 >>> real-time priority (-r) 0 >>> stack size (kbytes, -s) 8192 >>> cpu time (seconds, -t) unlimited >>> max user processes (-u) 32764 >>> virtual memory (kbytes, -v) unlimited >>> file locks (-x) unlimited >>> >>> >>> >>> It seems there are some logs in >>> hbase-maintain-regionserver-dx-pipe-regionserver7-online.out, but not >>> complete. >>> >>> 713517 >>> [RpcServer.reader=5,bindAddress=dx-pipe-regionserver7-online,port=16020] >>> INFO SecurityLogger.org.apache.hadoop.hbase.Server - Connection from >>> 10.11.51.75 port: 51239 with version info: version: "1.1.1" url: >>> "git://hw11397.local/Volumes/hbase-1.1.1RC0/hbase" revision: >>> "d0a115a7267f54e01c72c603ec53e91ec418292f" user: "ndimiduk" date: "Tue Jun >>> 23 14:56:34 PDT 2015" src_checksum: "6e2d8cecbd28738ad86daacb25dc467e" >>> 2016-02-14T12:46:20.880+0800: [GC (Allocation Failure) [ParNew: >>> 599568K->41393K(618048K), 0.0334895 secs] 1230469K->686558K(1991616K), >>> 0.0336985 secs] [Times: user=0.36 sys=0.06, real=0.04 secs] >>> 723538 >>> [RpcServer.reader=6,bindAddress=dx-pipe-regionserver7-online,port=16020] >>> INFO SecurityLogger.org.apache.hadoop.hbase.Server - Connection from >>> 10.11.53.52 port: 18965 with version info: version: "1.1.1" url: >>> "git://hw11397.local/Volumes/hbase-1.1.1RC0/hbase" revision: >>> "d0a115a7267f54e01c72c603ec53e91ec418292f" user: "ndimiduk" date: "Tue Jun >>> 23 14:56:34 PDT 2015" src_checksum: "6e2d8cecbd28738ad86daacb25dc467e" >>> 733529 >>> [RpcServer.reader=7,bindAddress=dx-pipe-regionserver7-online,port=16020] >>> INFO SecurityLogger.org.apache.hadoop.hbase.Server - Connection from >>> 10.11.51.75 port: 51325 with version info: version: "1.1.1" url: >>> "git://hw11397.local/Volumes/hbase-1.1.1RC0/hbase" revision: >>> "d0a115a7267f54e01c72c603ec53e91ec418292f" user: "ndimiduk" date: "Tue Jun >>> 23 14:56:34 PDT 2015" src_checksum: "6e2d8cecbd28738ad86daacb25dc467e" >>> 733529 >>> [RpcServer.reader=8,bindAddress=dx-pipe-regionserver7-online,port=16020] >>> INFO SecurityLogger.org.apache.hadoop.hbase.Server - Connection from >>> 10.11.51.62 port: 32504 with version info: version: "1.1.1" url: >>> "git://hw11397.local/Volumes/hbase-1.1.1RC0/hbase" revision: >>> "d0a115a7267f54e01c72c603ec53e91ec418292f" user: "ndimiduk" date: "Tue Jun >>> 23 14:56:34 PDT 2015" src_checksum: "6e2d8cecbd28738ad86daacb25dc467e" >>> 733552 >>> [RpcServer.reader=9,bindAddress=dx-pipe-regionserver7-online,port=16020] >>> INFO SecurityLogger.org.apache.hadoop.hbase.Server - Connection from >>> 10.11.53.52 port: 19005 with version info: version: "1.1.1" url: >>> "git://hw11397.local/Volumes/hbase-1.1.1RC0/hbase" revision: >>> "d0a115a7267f54e01c72c603ec53e91ec418292f" user: "ndimiduk" date: "Tue Jun >>> 23 14:56:34 PDT 2015" src_checksum: "6e2d8cecbd28738ad86daacb25dc467e" >>> 2016-02-14T12:46:40.371+0800: [GC (Allocation Failure) [ParNew: >>> 590769K->67969K(618048K), 0.0282183 secs] 1235934K->713134K(1991616K), >>> 0.0284249 secs] [Times: user=0.27 sys=0.06, real=0.03 secs] >>> >>> >>> >>> >>> >>> Any one meet this problem? >>> >>> >> >