[jira] [Commented] (HBASE-5881) BuiltIn Gzip compressor decompressor not getting pooled, leading to native memory leak
[ https://issues.apache.org/jira/browse/HBASE-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262790#comment-13262790 ] stack commented on HBASE-5881: -- Is the existance of this JIRA sufficient documentation until above is fixed? BuiltIn Gzip compressor decompressor not getting pooled, leading to native memory leak Key: HBASE-5881 URL: https://issues.apache.org/jira/browse/HBASE-5881 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.92.1 Reporter: Gopinathan A Assignee: ramkrishna.s.vasudevan Priority: Critical Fix For: 0.92.2, 0.96.0, 0.94.1 This issue will occur only in hadoop 23.x above/ In hadoop 0.20.x {code} public static void returnDecompressor(Decompressor decompressor) { if (decompressor == null) { return; } decompressor.reset(); payback(decompressorPool, decompressor); } {code} In hadoop 0.23.x {code} public static void returnDecompressor(Decompressor decompressor) { if (decompressor == null) { return; } // if the decompressor can't be reused, don't pool it. if (decompressor.getClass().isAnnotationPresent(DoNotPool.class)) { return; } decompressor.reset(); payback(decompressorPool, decompressor); } {code} Here annotation has been added. By default this library will be loaded if there are no native library. {code} @DoNotPool public class BuiltInGzipDecompressor {code} Due to this each time new compressor/decompressor will be loaded, this leads to native memory leak. {noformat} 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz] 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz] 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz] {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5881) BuiltIn Gzip compressor decompressor not getting pooled, leading to native memory leak
[ https://issues.apache.org/jira/browse/HBASE-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263332#comment-13263332 ] Gopinathan A commented on HBASE-5881: - OK, I feel. BuiltIn Gzip compressor decompressor not getting pooled, leading to native memory leak Key: HBASE-5881 URL: https://issues.apache.org/jira/browse/HBASE-5881 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.92.1 Reporter: Gopinathan A Assignee: ramkrishna.s.vasudevan Priority: Critical Fix For: 0.92.2, 0.96.0, 0.94.1 This issue will occur only in hadoop 23.x above/ In hadoop 0.20.x {code} public static void returnDecompressor(Decompressor decompressor) { if (decompressor == null) { return; } decompressor.reset(); payback(decompressorPool, decompressor); } {code} In hadoop 0.23.x {code} public static void returnDecompressor(Decompressor decompressor) { if (decompressor == null) { return; } // if the decompressor can't be reused, don't pool it. if (decompressor.getClass().isAnnotationPresent(DoNotPool.class)) { return; } decompressor.reset(); payback(decompressorPool, decompressor); } {code} Here annotation has been added. By default this library will be loaded if there are no native library. {code} @DoNotPool public class BuiltInGzipDecompressor {code} Due to this each time new compressor/decompressor will be loaded, this leads to native memory leak. {noformat} 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz] 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz] 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz] {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5881) BuiltIn Gzip compressor decompressor not getting pooled, leading to native memory leak
[ https://issues.apache.org/jira/browse/HBASE-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262388#comment-13262388 ] ramkrishna.s.vasudevan commented on HBASE-5881: --- We need to document this? BuiltIn Gzip compressor decompressor not getting pooled, leading to native memory leak Key: HBASE-5881 URL: https://issues.apache.org/jira/browse/HBASE-5881 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.92.1 Reporter: Gopinathan A Priority: Critical Fix For: 0.92.2, 0.94.0, 0.96.0 This issue will occur only in hadoop 23.x above/ In hadoop 0.20.x {code} public static void returnDecompressor(Decompressor decompressor) { if (decompressor == null) { return; } decompressor.reset(); payback(decompressorPool, decompressor); } {code} In hadoop 0.23.x {code} public static void returnDecompressor(Decompressor decompressor) { if (decompressor == null) { return; } // if the decompressor can't be reused, don't pool it. if (decompressor.getClass().isAnnotationPresent(DoNotPool.class)) { return; } decompressor.reset(); payback(decompressorPool, decompressor); } {code} Here annotation has been added. By default this library will be loaded if there are no native library. {code} @DoNotPool public class BuiltInGzipDecompressor {code} Due to this each time new compressor/decompressor will be loaded, this leads to native memory leak. {noformat} 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz] 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz] 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz] {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira