Hi Maryann,
After executing the same query after increasing the heap
space in region server, I get a strange error:
./psql.py 10.10.5.55 test.sql
14/10/07 02:59:52 WARN execute.HashJoinPlan: Hash plan [0] execution seems
too slow. Earlier hash cache(s) might have expired on servers.
org.apache.phoenix.exception.PhoenixIOException:
org.apache.phoenix.exception.PhoenixIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: Could not find hash cache
for joinId: {/�PY�L�. The cache might have expired and have been removed.
at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:97)
at
org.apache.phoenix.iterate.ParallelIterators.getIterators(ParallelIterators.java:279)
at
org.apache.phoenix.iterate.MergeSortResultIterator.getIterators(MergeSortResultIterator.java:48)
at
org.apache.phoenix.iterate.MergeSortResultIterator.minIterator(MergeSortResultIterator.java:63)
at
org.apache.phoenix.iterate.MergeSortResultIterator.next(MergeSortResultIterator.java:90)
at
org.apache.phoenix.iterate.GroupedAggregatingResultIterator.next(GroupedAggregatingResultIterator.java:68)
at
org.apache.phoenix.iterate.DelegateResultIterator.next(DelegateResultIterator.java:40)
at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:732)
at
org.apache.phoenix.jdbc.PhoenixConnection.executeStatements(PhoenixConnection.java:223)
at
org.apache.phoenix.util.PhoenixRuntime.executeStatements(PhoenixRuntime.java:193)
at org.apache.phoenix.util.PhoenixRuntime.main(PhoenixRuntime.java:140)
Caused by: java.util.concurrent.ExecutionException:
org.apache.phoenix.exception.PhoenixIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: Could not find hash cache
for joinId: {/�PY�L�. The cache might have expired and have been removed.
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
at java.util.concurrent.FutureTask.get(FutureTask.java:91)
at
org.apache.phoenix.iterate.ParallelIterators.getIterators(ParallelIterators.java:275)
... 9 more
Caused by: org.apache.phoenix.exception.PhoenixIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: Could not find hash cache
for joinId: {/�PY�L�. The cache might have expired and have been removed.
at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:97)
at
org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:57)
at
org.apache.phoenix.iterate.ParallelIterators$3.call(ParallelIterators.java:351)
at
org.apache.phoenix.iterate.ParallelIterators$3.call(ParallelIterators.java:346)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.hadoop.hbase.DoNotRetryIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: Could not find hash cache
for joinId: {/�PY�L�. The cache might have expired and have been removed.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.client.ServerCallable.translateException(ServerCallable.java:256)
at
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:166)
at
org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:211)
at
org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:126)
at
org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:121)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:702)
at
org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:54)
... 7 more
Caused by: org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hbase.DoNotRetryIOException: Could not find hash cache
for joinId: {/�PY�L�. The cache might have expired and have been removed.
at
org.apache.phoenix.coprocessor.HashJoinRegionScanner.<init>(HashJoinRegionScanner.java:90)
at
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.doPostScannerOpen(GroupedAggregateRegionObserver.java:121)
at
org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:89)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1313)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2429)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1428)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1012)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87)
at $Proxy6.openScanner(Unknown Source)
at
org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:224)
at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:126)
at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:42)
at
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:164)
Kindly direct me.
Regards,
Vijay Raajaa G S
On Sun, Oct 5, 2014 at 2:10 AM, Maryann Xue <maryann....@gmail.com> wrote:
> Hi Ashish,
>
> The "phoenix.query.maxServerCacheBytes" is a client parameter while the
> other two are server parameters. But it looks like the configuration change
> did not take effect at your client side. Could you please make sure that
> this is the only configuration that goes to the CLASSPATH of your phoenix
> client execution environment?
>
> Another thing is the exception you got was a different problem from
> Vijay's. It happened in an even earlier stage. Could you please also share
> you query? We could probably re-write it so that it can better fit the
> hash-join scheme. (Since table stats are not used in joins yet, we
> currently have to do it manually.)
>
>
> Thanks,
> Maryann
>
> On Tue, Sep 30, 2014 at 1:22 PM, ashish tapdiya <ashishtapd...@gmail.com>
> wrote:
>
>> Here it is,
>>
>> java.sql.SQLException: Encountered exception in hash plan [1] execution.
>> at
>> org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:146)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:211)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:204)
>> at
>> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:54)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:204)
>> at
>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.executeQuery(PhoenixPreparedStatement.java:158)
>> at Query.sel_Cust_Order_OrderLine_Tables(Query.java:135)
>> at Query.main(Query.java:25)
>> Caused by: org.apache.phoenix.join.MaxServerCacheSizeExceededException:
>> Size of hash cache (104857684 bytes) exceeds the maximum allowed size
>> (104857600 bytes)
>> at
>> org.apache.phoenix.join.HashCacheClient.serialize(HashCacheClient.java:106)
>> at
>> org.apache.phoenix.join.HashCacheClient.addHashCache(HashCacheClient.java:77)
>> at
>> org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:119)
>> at
>> org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:114)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:744)
>>
>> I am setting hbase heap to 4 GB and phoenix properties are set as below
>>
>> <property>
>> <name>phoenix.query.maxServerCacheBytes</name>
>> <value>2004857600</value>
>> </property>
>> <property>
>> <name>phoenix.query.maxGlobalMemoryPercentage</name>
>> <value>40</value>
>> </property>
>> <property>
>> <name>phoenix.query.maxGlobalMemorySize</name>
>> <value>1504857600</value>
>> </property>
>>
>> Thanks,
>> ~Ashish
>>
>> On Tue, Sep 30, 2014 at 12:13 PM, Maryann Xue <maryann....@gmail.com>
>> wrote:
>>
>>> Hi Ashish,
>>>
>>> Could you please let us see your error message?
>>>
>>>
>>> Thanks,
>>> Maryann
>>>
>>> On Tue, Sep 30, 2014 at 12:58 PM, ashish tapdiya <
>>> ashishtapd...@gmail.com> wrote:
>>>
>>>> Hey Maryann,
>>>>
>>>> Thanks for your input. I tried both the properties but no luck.
>>>>
>>>> ~Ashish
>>>>
>>>> On Sun, Sep 28, 2014 at 8:31 PM, Maryann Xue <maryann....@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Ashish,
>>>>>
>>>>> The global cache size is set to either "
>>>>> *phoenix.query.maxGlobalMemorySize*" or
>>>>> "phoenix.query.maxGlobalMemoryPercentage
>>>>> * heapSize" (Sorry about the mistake I made earlier). The ""
>>>>> phoenix.query.maxServerCacheBytes" is a client parameter and is most
>>>>> likely NOT the thing you should worry about. So you can try adjusting "
>>>>> phoenix.query.maxGlobalMemoryPercentage" and the heap size in region
>>>>> server configurations and see how it works.
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Maryann
>>>>>
>>>>> On Fri, Sep 26, 2014 at 10:48 PM, ashish tapdiya <
>>>>> ashishtapd...@gmail.com> wrote:
>>>>>
>>>>>> I have tried that as well...but "phoenix.query.maxServerCacheBytes"
>>>>>> remains the default value of 100 MB. I get to see it when join fails.
>>>>>>
>>>>>> Thanks,
>>>>>> ~Ashish
>>>>>>
>>>>>> On Fri, Sep 26, 2014 at 8:02 PM, Maryann Xue <maryann....@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ashish,
>>>>>>>
>>>>>>> The global cache size is set to either
>>>>>>> "phoenix.query.maxServerCacheBytes"
>>>>>>> or "phoenix.query.maxGlobalMemoryPercentage * heapSize", whichever
>>>>>>> is *smaller*. You can try setting "phoenix.query.
>>>>>>> maxGlobalMemoryPercentage" instead, which is recommended, and see
>>>>>>> how it goes.
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Maryann
>>>>>>>
>>>>>>> On Fri, Sep 26, 2014 at 5:37 PM, ashish tapdiya <
>>>>>>> ashishtapd...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Maryann,
>>>>>>>>
>>>>>>>> I am having the same issue where star join is failing with
>>>>>>>> MaxServerCacheSizeExceededException.
>>>>>>>> I set phoenix.query.maxServerCacheBytes to 1 GB both in client and
>>>>>>>> server hbase-site.xml's. However, it does not take effect.
>>>>>>>>
>>>>>>>> Phoenix 3.1
>>>>>>>> HBase .94
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> ~Ashish
>>>>>>>>
>>>>>>>> On Fri, Sep 26, 2014 at 2:56 PM, Maryann Xue <maryann....@gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Yes, you should make your modification on each region server,
>>>>>>>>> since this is a server-side configuration.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Sep 25, 2014 at 4:15 AM, G.S.Vijay Raajaa <
>>>>>>>>> gsvijayraa...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Xue,
>>>>>>>>>>
>>>>>>>>>> Thanks for replying. I did modify the hbase-site.xml by
>>>>>>>>>> increasing the default value of
>>>>>>>>>> phoenix.query.maxGlobalMemoryPercentage
>>>>>>>>>> . Also increased the Region server heap space memory . The
>>>>>>>>>> change didn't get reflected and I still get the error with an
>>>>>>>>>> indication
>>>>>>>>>> that "global pool of 319507660 bytes" is present. Should I
>>>>>>>>>> modify the hbase-site.xml in every region server or just the file
>>>>>>>>>> present
>>>>>>>>>> in the class path of Phoenix client?
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Vijay Raajaa G S
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 25, 2014 at 1:47 AM, Maryann Xue <
>>>>>>>>>> maryann....@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Vijay,
>>>>>>>>>>>
>>>>>>>>>>> I think here the query plan is scanning table *CUSTOMER_30000 *while
>>>>>>>>>>> joining the other two tables at the same time, which means the
>>>>>>>>>>> region
>>>>>>>>>>> server memory for Phoenix should be large enough to hold 2 tables
>>>>>>>>>>> together
>>>>>>>>>>> and you also need to expect some memory expansion for java objects.
>>>>>>>>>>>
>>>>>>>>>>> Do you mean that after you had modified the parameters you
>>>>>>>>>>> mentioned, you were still getting the same error message with
>>>>>>>>>>> exactly the
>>>>>>>>>>> same numbers as "global pool of 319507660 bytes"? Did you make
>>>>>>>>>>> sure that the parameters actually took effect after modification?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Maryann
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Sep 23, 2014 at 1:43 AM, G.S.Vijay Raajaa <
>>>>>>>>>>> gsvijayraa...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I am trying to do a join of three tables usng the following
>>>>>>>>>>>> query:
>>>>>>>>>>>>
>>>>>>>>>>>> *select c.c_first_name, ca.ca_city, cd.cd_education_status from
>>>>>>>>>>>> CUSTOMER_30000 c join CUSTOMER_DEMOGRAPHICS_1 cd on
>>>>>>>>>>>> c.c_current_cdemo_sk =
>>>>>>>>>>>> cd.cd_demo_sk join CUSTOMER_ADDRESS_1 ca on c.c_current_addr_sk =
>>>>>>>>>>>> ca.ca_address_sk group by ca.ca_city, cd.cd_education_status,
>>>>>>>>>>>> c.c_first_name;*
>>>>>>>>>>>>
>>>>>>>>>>>> *The size of CUSTOMER_30000 is 4.1 GB with 30million records.*
>>>>>>>>>>>>
>>>>>>>>>>>> *I get the following error:*
>>>>>>>>>>>>
>>>>>>>>>>>> ./psql.py 10.10.5.55 test.sql
>>>>>>>>>>>> java.sql.SQLException: Encountered exception in hash plan [0]
>>>>>>>>>>>> execution.
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:146)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:211)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:204)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:54)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:204)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:193)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:147)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:152)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.jdbc.PhoenixConnection.executeStatements(PhoenixConnection.java:220)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.util.PhoenixRuntime.executeStatements(PhoenixRuntime.java:193)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.util.PhoenixRuntime.main(PhoenixRuntime.java:140)
>>>>>>>>>>>> Caused by: java.sql.SQLException:
>>>>>>>>>>>> java.util.concurrent.ExecutionException:
>>>>>>>>>>>> java.lang.reflect.UndeclaredThrowableException
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:199)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.join.HashCacheClient.addHashCache(HashCacheClient.java:78)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:119)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:114)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:662)
>>>>>>>>>>>> Caused by: java.util.concurrent.ExecutionException:
>>>>>>>>>>>> java.lang.reflect.UndeclaredThrowableException
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
>>>>>>>>>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:91)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:191)
>>>>>>>>>>>> ... 8 more
>>>>>>>>>>>> Caused by: java.lang.reflect.UndeclaredThrowableException
>>>>>>>>>>>> at $Proxy10.addServerCache(Unknown Source)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:169)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:164)
>>>>>>>>>>>> ... 5 more
>>>>>>>>>>>> Caused by:
>>>>>>>>>>>> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed
>>>>>>>>>>>> after
>>>>>>>>>>>> attempts=14, exceptions:
>>>>>>>>>>>> Tue Sep 23 00:25:53 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:26:02 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:26:18 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:26:43 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:27:01 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:27:10 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:27:24 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:28:16 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:28:35 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:29:09 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:30:16 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:31:22 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:32:29 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>> Tue Sep 23 00:33:35 CDT 2014,
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@100e398,
>>>>>>>>>>>> java.io.IOException: java.io.IOException:
>>>>>>>>>>>> org.apache.phoenix.memory.InsufficientMemoryException: Requested
>>>>>>>>>>>> memory of
>>>>>>>>>>>> 446623727 bytes is larger than global pool of 319507660 bytes.
>>>>>>>>>>>>
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:187)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPCInvoker.java:79)
>>>>>>>>>>>> ... 8 more
>>>>>>>>>>>>
>>>>>>>>>>>> Trials:
>>>>>>>>>>>>
>>>>>>>>>>>> I tried to increase the Region Server Heap space ,
>>>>>>>>>>>> modified phoenix.query.maxGlobalMemoryPercentage as well.
>>>>>>>>>>>>
>>>>>>>>>>>> I am not able to increase the global memory .
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Vijay Raajaa
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Maryann
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks,
>>>>>>>>> Maryann
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> Maryann
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Maryann
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Maryann
>>>
>>
>>
>
>
> --
> Thanks,
> Maryann
>
