[
https://issues.apache.org/jira/browse/HBASE-14598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ian Friedman updated HBASE-14598:
---------------------------------
Description:
We noticed that in returning a Scan against a region containing particularly
large (wide) rows that it is possible during ByteBufferOutputStream.
checkSizeAndGrow() to attempt to create a new ByteBuffer larger than the JVM
allows (which I believe is Integer.MAX_VALUE), which then throws a
OutOfMemoryError. This lead to us dealing with cascading region server death as
the RegionServer hosting the region died, opened on a new server, the client
retried the scan, and the new RS died as well.
I believe ByteBufferOutputStream should not try to create ByteBuffers that
large and instead throw an exception back up. The limit should probably be
something like Integer.MAX_VALUE-8, as that is what ArrayList uses. ref:
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/ArrayList.java#221
was:
We noticed that in returning a Scan against a region containing particularly
large (wide) rows that it is possible during ByteBufferOutputStream.
checkSizeAndGrow() to attempt to create a new ByteBuffer larger than the JVM
allows (which I believe is Integer.MAX_VALUE), which then throws a
OutOfMemoryError. This lead to us dealing with cascading region server death as
the RegionServer hosting the region died, opened on a new server, the client
retried the scan, and the new RS died as well.
I believe ByteBufferOutputStream should not try to create ByteBuffers that
large and instead throw an exception back up.
> ByteBufferOutputStream grows its HeapByteBuffer beyond JVM limitations
> ----------------------------------------------------------------------
>
> Key: HBASE-14598
> URL: https://issues.apache.org/jira/browse/HBASE-14598
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.98.12
> Reporter: Ian Friedman
> Assignee: Ian Friedman
>
> We noticed that in returning a Scan against a region containing particularly
> large (wide) rows that it is possible during ByteBufferOutputStream.
> checkSizeAndGrow() to attempt to create a new ByteBuffer larger than the JVM
> allows (which I believe is Integer.MAX_VALUE), which then throws a
> OutOfMemoryError. This lead to us dealing with cascading region server death
> as the RegionServer hosting the region died, opened on a new server, the
> client retried the scan, and the new RS died as well.
> I believe ByteBufferOutputStream should not try to create ByteBuffers that
> large and instead throw an exception back up. The limit should probably be
> something like Integer.MAX_VALUE-8, as that is what ArrayList uses. ref:
> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/ArrayList.java#221
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
