[
https://issues.apache.org/jira/browse/PHOENIX-34?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897473#comment-13897473
]
Lars Hofhansl commented on PHOENIX-34:
--------------------------------------
We should not give Phoenix 50% of the heap on a RegionServer.
By default a RegionServer is configured with 40% of the heap to the memstore
and another 40% to the blockcache, and 20% for "everything else".
5-10% to Phoenix seems like a good default. When more is needed other parts of
the RegionServer need to reconfigured.
Also with the hash-join types Phoenix supports I had assumed we'd limit the
smaller hashed to 10k rows or so.
It's nice to support a wider range of scenarios, but we need to accept the
realities of a RegionServer, where heap is already at a premium.
(I think we need to generally think through Phoenix's resource consumption
anyway and if/how they scale with the number of available resources. For
example the number of a threads to use for a query should not be fixed, etc.)
> Insufficient memory exception on join when RHS rows count > 250K
> -----------------------------------------------------------------
>
> Key: PHOENIX-34
> URL: https://issues.apache.org/jira/browse/PHOENIX-34
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 3.0.0
> Environment: HBase 0.94.14, r1543222, Hadoop 1.0.4, r1393290, 2 RS +
> 1 Master, Heap 4GB per RS
> Reporter: Mujtaba Chohan
> Fix For: 3.0.0
>
>
> Join fails when rows count of RHS table is >250K. Detail on table schema is
> and performance numbers with different LHS/RHS row count is on
> http://phoenix-bin.github.io/client/performance/phoenix-20140210023154.htm.
> James comment:
> So that's with a 4GB heap allowing Phoenix to use 50% of it. With a pretty
> narrow table: 3 KV columns of 30bytes. Topping out at 250K is a bit low. I
> wonder if our memory estimation matches reality.
> What do you think Maryann?
> How about filing a JIRA, Mujtaba. This is a good conversation to have on the
> dev list. Can we move it there, please?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
