Finally I copied the hadoop 2.2.0 log4j.properties in place of the HBase one and got what I was looking for ;)
Thanks for your recommendations. 2013/12/14 Ted Yu <[email protected]> > TRACE,RFA was used in the command you posted - it overrode DEBUG,console in > your properties file. > > I should have mentioned that I got the snippet from > /grid/0/var/log/hbase/hbase.log > where /grid/0/var/log/hbase is the log dir. > > You can try specifying TRACE,console > > Cheers > > > On Sat, Dec 14, 2013 at 7:47 AM, Jean-Marc Spaggiari < > [email protected]> wrote: > > > I have updated the log4.properties. > > > > Here is the output with log4j debug enabled: > > hbase@hbasetest1:~$ bin/hbase -Dlog4j.debug > > org.apache.hadoop.hbase.util.CompressionTest file:///tmp/test.txt snappy > > log4j: Trying to find [log4j.xml] using context classloader > > sun.misc.Launcher$AppClassLoader@713c817. > > log4j: Trying to find [log4j.xml] using > > sun.misc.Launcher$AppClassLoader@713c817 class loader. > > log4j: Trying to find [log4j.xml] using ClassLoader.getSystemResource(). > > log4j: Trying to find [log4j.properties] using context classloader > > sun.misc.Launcher$AppClassLoader@713c817. > > log4j: Using URL [file:/home/hbase/conf/log4j.properties] for automatic > > log4j configuration. > > log4j: Reading configuration from URL > > file:/home/hbase/conf/log4j.properties > > log4j: Hierarchy threshold set to [ALL]. > > log4j: Parsing for [root] with value=[INFO,console]. > > log4j: Level token is [INFO]. > > log4j: Category root set to INFO > > log4j: Parsing appender named "console". > > log4j: Parsing layout options for "console". > > log4j: Setting property [conversionPattern] to [%d{ISO8601} %-5p [%t] > > %c{2}: %m%n]. > > log4j: End of parsing for "console". > > log4j: Setting property [target] to [System.err]. > > log4j: Parsed "console" options. > > log4j: Parsing for [SecurityLogger] with value=[INFO,NullAppender]. > > log4j: Level token is [INFO]. > > log4j: Category SecurityLogger set to INFO > > log4j: Parsing appender named "NullAppender". > > log4j: Parsed "NullAppender" options. > > log4j: Handling log4j.additivity.SecurityLogger=[false] > > log4j: Setting additivity for "SecurityLogger" to false > > log4j: Finished configuring. > > 2013-12-14 10:36:45,985 INFO [main] Configuration.deprecation: > > hadoop.native.lib is deprecated. Instead, use io.native.lib.available > > Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library > > /home/hbase/lib/native/Linux-amd64-64/libhadoop.so which might have > > disabled stack guard. The VM will try to fix the stack guard now. > > It's highly recommended that you fix the library with 'execstack -c > > <libfile>', or link it with '-z noexecstack'. > > 2013-12-14 10:36:46,577 WARN [main] util.NativeCodeLoader: Unable to > load > > native-hadoop library for your platform... using builtin-java classes > where > > applicable > > 2013-12-14 10:36:46,881 INFO [main] util.ChecksumType: Checksum using > > org.apache.hadoop.util.PureJavaCrc32 > > 2013-12-14 10:36:46,883 INFO [main] util.ChecksumType: Checksum can use > > org.apache.hadoop.util.PureJavaCrc32C > > Exception in thread "main" java.lang.UnsatisfiedLinkError: > > org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z > > at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native > > Method) > > at > > > > > org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:62) > > at > > > > > org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:131) > > at > > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:147) > > at > > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:162) > > at > > > > > org.apache.hadoop.hbase.io.compress.Compression$Algorithm.getCompressor(Compression.java:312) > > at > > > > > org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultEncodingContext.<init>(HFileBlockDefaultEncodingContext.java:79) > > at > > > > > org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:719) > > at > > > > > org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:131) > > at > > > > > org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:122) > > at > > > > > org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:105) > > at > > > org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:426) > > at > > > > > org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(CompressionTest.java:115) > > at > > > org.apache.hadoop.hbase.util.CompressionTest.main(CompressionTest.java:145) > > > > > > As you can see, it's reading from the default log4j.property file. > > > > Inside of which I have this: > > # Define some default values that can be overridden by system properties > > hbase.root.logger=DEBUG,console > > hbase.security.logger=DEBUG,console > > hbase.log.dir=. > > hbase.log.file=hbase.log > > root.logger=DEBUG.console > > log4j.rootLogger=DEBUG,console > > > > > > However as you can see, no debug info displayed. > > > > Using your parameter seems to be working: > > hbase@hbasetest1:~$ bin/hbase -Dlog4j.debug > -Dhbase.root.logger=TRACE,RFA > > org.apache.hadoop.hbase.util.CompressionTest file:///tmp/test.txt snappy > > /home/hbase/bin/../lib/native/Linux-amd64-64 > > log4j: Trying to find [log4j.xml] using context classloader > > sun.misc.Launcher$AppClassLoader@713c817. > > log4j: Trying to find [log4j.xml] using > > sun.misc.Launcher$AppClassLoader@713c817 class loader. > > log4j: Trying to find [log4j.xml] using ClassLoader.getSystemResource(). > > log4j: Trying to find [log4j.properties] using context classloader > > sun.misc.Launcher$AppClassLoader@713c817. > > log4j: Using URL [file:/home/hbase/conf/log4j.properties] for automatic > > log4j configuration. > > log4j: Reading configuration from URL > > file:/home/hbase/conf/log4j.properties > > log4j: Hierarchy threshold set to [ALL]. > > log4j: Parsing for [root] with value=[TRACE,RFA]. > > log4j: Level token is [TRACE]. > > log4j: Category root set to TRACE > > log4j: Parsing appender named "RFA". > > log4j: Parsing layout options for "RFA". > > log4j: Setting property [conversionPattern] to [%d{ISO8601} %-5p [%t] > > %c{2}: %m%n]. > > log4j: End of parsing for "RFA". > > log4j: Setting property [maxBackupIndex] to [20]. > > log4j: Setting property [file] to [/home/hbase/bin/../logs/hbase.log]. > > log4j: Setting property [maxFileSize] to [256MB]. > > log4j: setFile called: /home/hbase/bin/../logs/hbase.log, true > > log4j: setFile ended > > log4j: Parsed "RFA" options. > > log4j: Parsing for [SecurityLogger] with value=[INFO,NullAppender]. > > log4j: Level token is [INFO]. > > log4j: Category SecurityLogger set to INFO > > log4j: Parsing appender named "NullAppender". > > log4j: Parsed "NullAppender" options. > > log4j: Handling log4j.additivity.SecurityLogger=[false] > > log4j: Setting additivity for "SecurityLogger" to false > > log4j: Finished configuring. > > Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library > > /home/hbase/lib/native/Linux-amd64-64/libhadoop.so which might have > > disabled stack guard. The VM will try to fix the stack guard now. > > It's highly recommended that you fix the library with 'execstack -c > > <libfile>', or link it with '-z noexecstack'. > > Exception in thread "main" java.lang.UnsatisfiedLinkError: > > org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z > > at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native > > Method) > > at > > > > > org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:62) > > at > > > > > org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:131) > > at > > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:147) > > at > > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:162) > > at > > > > > org.apache.hadoop.hbase.io.compress.Compression$Algorithm.getCompressor(Compression.java:312) > > at > > > > > org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultEncodingContext.<init>(HFileBlockDefaultEncodingContext.java:79) > > at > > > > > org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:719) > > at > > > > > org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:131) > > at > > > > > org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:122) > > at > > > > > org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:105) > > at > > > org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:426) > > at > > > > > org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(CompressionTest.java:115) > > at > > > org.apache.hadoop.hbase.util.CompressionTest.main(CompressionTest.java:145) > > > > > > But still no debug info displayed. My goal is to display the debug logs > of > > NativeCodeLoader for its static piece, but so far, no luck... > > > > > > 2013/12/14 Ted Yu <[email protected]> > > > > > How did you set log level to DEBUG ? > > > > > > I tried the following command and it worked: > > > > > > hbase -Dhbase.root.logger=DEBUG,RFA > > > org.apache.hadoop.hbase.util.CompressionTest file:///tmp/test.txt > snappy > > > > > > Snippet of log file: > > > > > > 2013-12-14 15:23:37,853 DEBUG [main] hdfs.BlockReaderLocal: The > > > short-circuit local reads feature is enabled. > > > 2013-12-14 15:23:37,900 INFO [main] compress.CodecPool: Got brand-new > > > decompressor [.snappy] > > > 2013-12-14 15:23:37,901 DEBUG [main] compress.CodecPool: Got recycled > > > decompressor > > > > > > Cheers > > > > > > > > > On Sat, Dec 14, 2013 at 5:42 AM, Jean-Marc Spaggiari < > > > [email protected]> wrote: > > > > > > > Hi there, > > > > > > > > I'm trying this tool: > > > > bin/hbase org.apache.hadoop.hbase.util.CompressionTest > > > file:///tmp/test.txt > > > > snappy > > > > > > > > And I want to set the log level to debug to see why it fails. But it > > > seems > > > > that it's not taking the log4j.conf into consideraion. I tried to > > remove > > > > it, same result. I tried to set to debug, same result. > > > > > > > > Any idea how to change the loglevel and why it's not taking our > default > > > > config file into consideration? > > > > > > > > JM > > > > > > > > > >
