[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13570938#comment-13570938
]
Hudson commented on HBASE-5458:
---
Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See
[https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/])
HBASE-5458 Thread safety issues with Compression.Algorithm.GZ and
CompressionTest (Revision 1435317)
Result = FAILURE
eclark :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Fix For: 0.90.7, 0.92.3, 0.96.0, 0.94.5
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571099#comment-13571099
]
Hudson commented on HBASE-5458:
---
Integrated in HBase-0.92-security #148 (See
[https://builds.apache.org/job/HBase-0.92-security/148/])
HBASE-5458 Thread safety issues with Compression.Algorithm.GZ and
CompressionTest (Revision 1435318)
Result = ABORTED
eclark :
Files :
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Fix For: 0.90.7, 0.92.3, 0.96.0, 0.94.5
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561705#comment-13561705
]
Hudson commented on HBASE-5458:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-5458 Thread safety issues with Compression.Algorithm.GZ and
CompressionTest (Revision 1435317)
Result = FAILURE
eclark :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Fix For: 0.90.7, 0.92.3, 0.96.0, 0.94.5
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13557451#comment-13557451
]
Elliott Clark commented on HBASE-5458:
--
This is called on every hfile reader open, so pretty low volume, but not never.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13557458#comment-13557458
]
Lars Hofhansl commented on HBASE-5458:
--
AbstractFSReader.decompress is called for each block read, no? Anyway, still
not super high volume.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13557478#comment-13557478
]
Elliott Clark commented on HBASE-5458:
--
Yeah you're correct. every block not every file.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13557485#comment-13557485
]
Elliott Clark commented on HBASE-5458:
--
going ahead to commit then.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13557488#comment-13557488
]
Lars Hofhansl commented on HBASE-5458:
--
+1
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13557559#comment-13557559
]
Hudson commented on HBASE-5458:
---
Integrated in HBase-TRUNK #3766 (See
[https://builds.apache.org/job/HBase-TRUNK/3766/])
HBASE-5458 Thread safety issues with Compression.Algorithm.GZ and
CompressionTest (Revision 1435316)
Result = FAILURE
eclark :
Files :
*
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/io/compress/Compression.java
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Fix For: 0.90.7, 0.92.3, 0.96.0, 0.94.5
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13557599#comment-13557599
]
Hudson commented on HBASE-5458:
---
Integrated in HBase-0.94 #746 (See
[https://builds.apache.org/job/HBase-0.94/746/])
HBASE-5458 Thread safety issues with Compression.Algorithm.GZ and
CompressionTest (Revision 1435317)
Result = SUCCESS
eclark :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Fix For: 0.90.7, 0.92.3, 0.96.0, 0.94.5
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13557741#comment-13557741
]
Hudson commented on HBASE-5458:
---
Integrated in HBase-0.92 #611 (See
[https://builds.apache.org/job/HBase-0.92/611/])
HBASE-5458 Thread safety issues with Compression.Algorithm.GZ and
CompressionTest (Revision 1435318)
Result = FAILURE
eclark :
Files :
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Fix For: 0.90.7, 0.92.3, 0.96.0, 0.94.5
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13557799#comment-13557799
]
Hudson commented on HBASE-5458:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #360 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/360/])
HBASE-5458 Thread safety issues with Compression.Algorithm.GZ and
CompressionTest (Revision 1435316)
Result = FAILURE
eclark :
Files :
*
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/io/compress/Compression.java
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Fix For: 0.90.7, 0.92.3, 0.96.0, 0.94.5
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556068#comment-13556068
]
Jonathan Hsieh commented on HBASE-5458:
---
I didn't look at all the context, but why is lock transient?
Do we really need double-checked locking (how often is this called and is it in
the critical path)? simpler to just do the normal locking pattern (I'd prefer
this)?
If we do double-checked locking, let's do it correctly -- I believe the
lzoCodec and snappyCodec need to be volatile instead of transient. See
http://en.wikipedia.org/wiki/Double-checked_locking
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556312#comment-13556312
]
Elliott Clark commented on HBASE-5458:
--
The lock's transient so that if someone tries to serialize an HFile that's been
compressed there's no lock object that's serialized. It's only there as a
helper, and has no real state to serialize.
This method is called on every open of HFiles. So it's in a pretty critical
path; which is why I went with double checked locking. Though there aren't a
lot of these calls made so it could be ok.
I'll make the codecs volatile as well as transient.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556335#comment-13556335
]
Jonathan Hsieh commented on HBASE-5458:
---
I'm fine with the double checked locking.
Hm.. I thought transient was used only for java serialization -- we're actually
using that for the Compression class? That seems funny since I thought we
always did writables/pbufs. (if we aren't using java serialization file a
follow on to remove the transients?)
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556342#comment-13556342
]
David McIntosh commented on HBASE-5458:
---
This patch prevents instantiating the same codec multiple times but it doesn't
look like it addresses the race condition I originally ran into. The
problematic sequence is like this:
Thread 1 calls GZ.getCodec for the first time. It sees that codec is null and
makes it into the buildCodec method. It executes codec = new GzipCodec() but
before it can set the configuration thread 2 also calls GZ.getCodec. Thread 2
sees that codec is not null and returns it resulting in an exception if it is
used before Thread 1 has set the configuration on it.
The GzipCodec is the only one affected since the other codecs are able to set
the configuration in the constructor.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556399#comment-13556399
]
Ted Yu commented on HBASE-5458:
---
{code}
+
ClassLoader.getSystemClassLoader().loadClass(org.apache.hadoop.io.compress.SnappyCodec);
{code}
Mind wrapping long line ?
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556506#comment-13556506
]
Ted Yu commented on HBASE-5458:
---
Latest patch looks good.
Can you prepare patch for trunk ?
Thanks
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556537#comment-13556537
]
Jonathan Hsieh commented on HBASE-5458:
---
[~eclark] lgtm +1.
[~dmcintosh] it looks good to me -- this idiom is called double-checked locking
and is an optimization for lazy initializers. In your example let's say thread
1 is already in buildCodec. This means it has the lock. thread 2 could see
null, and would get blocked by the lock. thread 1 would finish initialization,
set the variable, and then release the lock. thread 2 enters, sees that the
variable is not null and falls through. I don't see where the conf object gets
passed elsewhere. Am I missing something?
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556545#comment-13556545
]
Elliott Clark commented on HBASE-5458:
--
[~yuzhih...@gmail.com] Sure I'll post a version for all the versions.
[~jmhsieh] The order was wrong on version 0, and it would be possible (not easy
but still possible) for a thread to get a partially initialized Gzip algorithm.
Fixed since then.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556610#comment-13556610
]
Jonathan Hsieh commented on HBASE-5458:
---
Got it, thanks for the clarification (I was looking at the later versions).
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556739#comment-13556739
]
Hadoop QA commented on HBASE-5458:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12565392/HBASE-5458-trunk-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestScanWithBloomError
org.apache.hadoop.hbase.io.hfile.TestHFile
org.apache.hadoop.hbase.io.hfile.TestHFileBlockCompatibility
org.apache.hadoop.hbase.io.hfile.TestChecksum
org.apache.hadoop.hbase.io.hfile.TestHFileWriterV2
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4077//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4077//console
This message is automatically generated.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556758#comment-13556758
]
Hadoop QA commented on HBASE-5458:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12565390/HBASE-5458-092-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4078//console
This message is automatically generated.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556824#comment-13556824
]
Hadoop QA commented on HBASE-5458:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12565398/HBASE-5458-trunk-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestLocalHBaseCluster
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4079//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4079//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4079//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4079//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4079//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4079//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4079//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4079//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4079//console
This message is automatically generated.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556828#comment-13556828
]
Ted Yu commented on HBASE-5458:
---
+1 on latest patch.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556858#comment-13556858
]
Elliott Clark commented on HBASE-5458:
--
[~lhofhansl] Thoughts on this in 0.94 ?
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13556872#comment-13556872
]
Lars Hofhansl commented on HBASE-5458:
--
Looks good to me.
This is not on a hot code path, right? The hottest I found was
AbstractFSReader.decompress.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
Reporter: David McIntosh
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
HBASE-5458-trunk-2.patch
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214068#comment-13214068
]
Zhihong Yu commented on HBASE-5458:
---
{code}
try {
Compressor c = algo.getCompressor();
algo.returnCompressor(c);
compressionTestResults[algo.ordinal()] = true; // passes
} catch (Throwable t) {
compressionTestResults[algo.ordinal()] = false; // failure
throw new IOException(t);
}
{code}
How about catching NPE in the above code and call algo.getCompressor() again ?
We set compressionTestResults to false after the retry fails.
Thread safety issues with Compression.Algorithm.GZ and CompressionTest
--
Key: HBASE-5458
URL: https://issues.apache.org/jira/browse/HBASE-5458
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.5
Reporter: David McIntosh
Priority: Minor
I've seen some occasional NullPointerExceptions in
ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
completebulkload process. This is being caused by a null configuration
getting passed to the isNativeZlibLoaded method. I think this happens when 2
or more threads call the CompressionTest.testCompression method at once. If
the GZ algorithm has not been tested yet both threads could continue on and
attempt to load the compressor. For GZ the getCodec method is not thread
safe which could lead to one thread getting a reference to a GzipCodec that
has a null configuration.
{code}
current:
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
codec = new GzipCodec();
codec.setConf(new Configuration(conf));
}
return codec;
}
{code}
one possible fix would be something like this:
{code}
DefaultCodec getCodec(Configuration conf) {
if (codec == null) {
GzipCodec gzip = new GzipCodec();
gzip.setConf(new Configuration(conf));
codec = gzip;
}
return codec;
}
{code}
But that may not be totally safe without some synchronization. An upstream
fix in CompressionTest could also prevent multi thread access to
GZ.getCodec(conf)
exceptions:
12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at
org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at
org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
... 9 more
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at
