[
https://issues.apache.org/jira/browse/PHOENIX-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037666#comment-14037666
]
Praveen Murugesan commented on PHOENIX-990:
-------------------------------------------
[~prkommireddi] I think yes and no, it will be good to have finite control on
settings, but this can soon become over-whelming (It makes me yelp everytime i
look at the list of settings hadoop offers :p). This is something very internal
to how phoenix allocates buffers, so I'm not sure if this is a good candidate
for exposure, because if exposed you expect the user to have finite details
about how phoenix does things which increases the barrier to entry to use
phoenix
That said, one thing i forgot to add to this thread is, I really am not sure of
the need of mappedbytebuffers in this case, from the code i looked at, it
looked like only one thread is operating on the file at a particular time (I
might be wrong) [~giacomotaylor] Can you elaborate the choice of using a mapped
byte buffer. I'm just curious and want to learn.
> OOM caused by order by query returning all rows
> -----------------------------------------------
>
> Key: PHOENIX-990
> URL: https://issues.apache.org/jira/browse/PHOENIX-990
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 3.0.0, 4.0.0
> Reporter: Mujtaba Chohan
> Assignee: Praveen Murugesan
> Attachments: PHOENIX-990-TEST.patch, PHOENIX-990.patch
>
>
> OOM error with the following stack trace with large number of rows for query
> with order by returning all rows without limit or aggregation (ex. select *
> from table order by col1). Created a local perf. test to verify it when this
> gets fixed. Also performance.py script can be used to generate a few mil.
> rows.
> Originally reported by @zenmehra.
> Stack:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> org.apache.hadoop.hbase.DoNotRetryIOException:
> PERFORMANCE_5000000,,1400524730456.c62cccdac8cffd098d236f5e282564bb.:
> java.io.IOException: Map failed
> at
> org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:73)
> at
> org.apache.phoenix.coprocessor.ScanRegionObserver.getTopNScanner(ScanRegionObserver.java:221)
> at
> org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:197)
> at
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2573)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2541)
> at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> Caused by: java.lang.RuntimeException: java.io.IOException: Map failed
> at
> org.apache.phoenix.iterate.MappedByteBufferSortedQueue.offer(MappedByteBufferSortedQueue.java:73)
> at
> org.apache.phoenix.iterate.MappedByteBufferSortedQueue.offer(MappedByteBufferSortedQueue.java:43)
> at java.util.AbstractQueue.add(AbstractQueue.java:68)
> at
> org.apache.phoenix.iterate.OrderedResultIterator.getResultIterator(OrderedResultIterator.java:224)
> at
> org.apache.phoenix.iterate.OrderedResultIterator.next(OrderedResultIterator.java:169)
> at
> org.apache.phoenix.coprocessor.ScanRegionObserver.getTopNScanner(ScanRegionObserver.java:216)
> ... 10 more
> Caused by: java.io.IOException: Map failed
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
> at
> org.apache.phoenix.iterate.MappedByteBufferSortedQueue$MappedByteBufferPriorityQueue.writeResult(MappedByteBufferSortedQueue.java:299)
> at
> org.apache.phoenix.iterate.MappedByteBufferSortedQueue.offer(MappedByteBufferSortedQueue.java:65)
> ... 15 more
> Caused by: java.lang.OutOfMemoryError: Map failed
> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
