Here it is,
java.sql.SQLException: Encountered exception in hash plan [1] execution.
at
org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:146)
at
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:211)
at
org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:204)
at
org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:54)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:204)
at
org.apache.phoenix.jdbc.PhoenixPreparedStatement.executeQuery(PhoenixPreparedStatement.java:158)
at Query.sel_Cust_Order_OrderLine_Tables(Query.java:135)
at Query.main(Query.java:25)
Caused by: org.apache.phoenix.join.MaxServerCacheSizeExceededException:
Size of hash cache (104857684 bytes) exceeds the maximum allowed size
(104857600 bytes)
at
org.apache.phoenix.join.HashCacheClient.serialize(HashCacheClient.java:106)
at
org.apache.phoenix.join.HashCacheClient.addHashCache(HashCacheClient.java:77)
at
org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:119)
at
org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:114)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
I am setting hbase heap to 4 GB and phoenix properties are set as below
<property>
<name>phoenix.query.maxServerCacheBytes</name>
<value>2004857600</value>
</property>
<property>
<name>phoenix.query.maxGlobalMemoryPercentage</name>
<value>40</value>
</property>
<property>
<name>phoenix.query.maxGlobalMemorySize</name>
<value>1504857600</value>
</property>
Thanks,
~Ashish
On Tue, Sep 30, 2014 at 12:13 PM, Maryann Xue <maryann....@gmail.com> wrote:
> Hi Ashish,
>
> Could you please let us see your error message?
>
>
> Thanks,
> Maryann
>
> On Tue, Sep 30, 2014 at 12:58 PM, ashish tapdiya <ashishtapd...@gmail.com>
> wrote:
>
>> Hey Maryann,
>>
>> Thanks for your input. I tried both the properties but no luck.
>>
>> ~Ashish
>>
>> On Sun, Sep 28, 2014 at 8:31 PM, Maryann Xue <maryann....@gmail.com>
>> wrote:
>>
>>> Hi Ashish,
>>>
>>> The global cache size is set to either "
>>> *phoenix.query.maxGlobalMemorySize*" or
>>> "phoenix.query.maxGlobalMemoryPercentage
>>> * heapSize" (Sorry about the mistake I made earlier). The ""
>>> phoenix.query.maxServerCacheBytes" is a client parameter and is most
>>> likely NOT the thing you should worry about. So you can try adjusting "
>>> phoenix.query.maxGlobalMemoryPercentage" and the heap size in region
>>> server configurations and see how it works.
>>>
>>>
>>> Thanks,
>>> Maryann
>>>
>>> On Fri, Sep 26, 2014 at 10:48 PM, ashish tapdiya <
>>> ashishtapd...@gmail.com> wrote:
>>>
>>>> I have tried that as well...but "phoenix.query.maxServerCacheBytes"
>>>> remains the default value of 100 MB. I get to see it when join fails.
>>>>
>>>> Thanks,
>>>> ~Ashish
>>>>
>>>> On Fri, Sep 26, 2014 at 8:02 PM, Maryann Xue <maryann....@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Ashish,
>>>>>
>>>>> The global cache size is set to either "phoenix.query.maxServerCacheBytes"
>>>>> or "phoenix.query.maxGlobalMemoryPercentage * heapSize", whichever is
>>>>> *smaller*. You can try setting "phoenix.query.
>>>>> maxGlobalMemoryPercentage" instead, which is recommended, and see how
>>>>> it goes.
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Maryann
>>>>>
>>>>> On Fri, Sep 26, 2014 at 5:37 PM, ashish tapdiya <
>>>>> ashishtapd...@gmail.com> wrote:
>>>>>
>>>>>> Hi Maryann,
>>>>>>
>>>>>> I am having the same issue where star join is failing with
>>>>>> MaxServerCacheSizeExceededException.
>>>>>> I set phoenix.query.maxServerCacheBytes to 1 GB both in client and
>>>>>> server hbase-site.xml's. However, it does not take effect.
>>>>>>
>>>>>> Phoenix 3.1
>>>>>> HBase .94
>>>>>>
>>>>>> Thanks,
>>>>>> ~Ashish
>>>>>>
>>>>>> On Fri, Sep 26, 2014 at 2:56 PM, Maryann Xue <maryann....@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yes, you should make your modification on each region server, since
>>>>>>> this is a server-side configuration.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Sep 25, 2014 at 4:15 AM, G.S.Vijay Raajaa <
>>>>>>> gsvijayraa...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Xue,
>>>>>>>>
>>>>>>>> Thanks for replying. I did modify the hbase-site.xml by
>>>>>>>> increasing the default value of phoenix.query.maxGlobalMemoryPercentage
>>>>>>>> . Also increased the Region server heap space memory . The
>>>>>>>> change didn't get reflected and I still get the error with an
>>>>>>>> indication
>>>>>>>> that "global pool of 319507660 bytes" is present. Should I modify
>>>>>>>> the hbase-site.xml in every region server or just the file present in
>>>>>>>> the class path of Phoenix client?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Vijay Raajaa G S
>>>>>>>>
>>>>>>>> On Thu, Sep 25, 2014 at 1:47 AM, Maryann Xue <maryann....@gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Hi Vijay,
>>>>>>>>>
>>>>>>>>> I think here the query plan is scanning table *CUSTOMER_30000 *while
>>>>>>>>> joining the other two tables at the same time, which means the region
>>>>>>>>> server memory for Phoenix should be large enough to hold 2 tables
>>>>>>>>> together
>>>>>>>>> and you also need to expect some memory expansion for java objects.
>>>>>>>>>
>>>>>>>>> Do you mean that after you had modified the parameters you
>>>>>>>>> mentioned, you were still getting the same error message with exactly
>>>>>>>>> the
>>>>>>>>> same numbers as "global pool of 319507660 bytes"? Did you make
>>>>>>>>> sure that the parameters actually took effect after modification?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Maryann
>>>>>>>>>
>>>>>>>>> On Tue, Sep 23, 2014 at 1:43 AM, G.S.Vijay Raajaa <
>>>>>>>>> gsvijayraa...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I am trying to do a join of three tables usng the following
>>>>>>>>>> query:
>>>>>>>>>>
>>>>>>>>>> *select c.c_first_name, ca.ca_city, cd.cd_education_status from
>>>>>>>>>> CUSTOMER_30000 c join CUSTOMER_DEMOGRAPHICS_1 cd on
>>>>>>>>>> c.c_current_cdemo_sk =
>>>>>>>>>> cd.cd_demo_sk join CUSTOMER_ADDRESS_1 ca on c.c_current_addr_sk =
>>>>>>>>>> ca.ca_address_sk group by ca.ca_city, cd.cd_education_status,
>>>>>>>>>> c.c_first_name;*
>>>>>>>>>>
>>>>>>>>>> *The size of CUSTOMER_30000 is 4.1 GB with 30million records.*
>>>>>>>>>>
>>>>>>>>>> *I get the following error:*
>>>>>>>>>>
>>>>>>>>>> ./psql.py 10.10.5.55 test.sql
>>>>>>>>>> java.sql.SQLException: Encountered exception in hash plan [0]
>>>>>>>>>> execution.
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:146)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:211)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:204)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:54)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:204)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:193)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:147)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:152)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixConnection.executeStatements(PhoenixConnection.java:220)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.util.PhoenixRuntime.executeStatements(PhoenixRuntime.java:193)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.util.PhoenixRuntime.main(PhoenixRuntime.java:140)
>>>>>>>>>> Caused by: java.sql.SQLException:
>>>>>>>>>> java.util.concurrent.ExecutionException:
>>>>>>>>>> java.lang.reflect.UndeclaredThrowableException
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:199)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.join.HashCacheClient.addHashCache(HashCacheClient.java:78)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:119)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:114)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>> at java.lang.Thread.run(Thread.java:662)
>>>>>>>>>> Caused by: java.util.concurrent.ExecutionException:
>>>>>>>>>> java.lang.reflect.UndeclaredThrowableException
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
>>>>>>>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:91)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:191)
>>>>>>>>>> ... 8 more
>>>>>>>>>> Caused by: java.lang.reflect.UndeclaredThrowableException
>>>>>>>>>> at $Proxy10.addServerCache(Unknown Source)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:169)
>>>>>>>>>> at
>>>>>>>>>> org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:164)
>>>>>>>>>> ... 5 more
>>>>>>>>>> Caused by:
>>>>>>>>>> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed
>>>>>>>>>> after
>>>>>>>>>> attempts=14, exceptions:
>>>>>>>>>> Tue Sep 23 00:25:53 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:26:02 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:26:18 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:26:43 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:27:01 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:27:10 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:27:24 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:28:16 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:28:35 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:29:09 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:30:16 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:31:22 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:32:29 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>> Tue Sep 23 00:33:35 CDT 2014,
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>> memory of
>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:187)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPCInvoker.java:79)
>>>>>>>>>> ... 8 more
>>>>>>>>>>
>>>>>>>>>> Trials:
>>>>>>>>>>
>>>>>>>>>> I tried to increase the Region Server Heap space ,
>>>>>>>>>> modified phoenix.query.maxGlobalMemoryPercentage as well.
>>>>>>>>>>
>>>>>>>>>> I am not able to increase the global memory .
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Vijay Raajaa
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks,
>>>>>>>>> Maryann
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> Maryann
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Maryann
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Maryann
>>>
>>
>>
>
>
> --
> Thanks,
> Maryann
>
