[
https://issues.apache.org/jira/browse/TRAFODION-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511467#comment-16511467
]
Sandhya Sundaresan commented on TRAFODION-3107:
-----------------------------------------------
|Looking at the .hprof it shows that due to scanner caching , we are occupying
too much space in the client side and the JVM runs out of memory. The explain
plan shows that the entire 5 MB row is being selected sisince it's n aligned
row table.THis is how select (count(0) and select count(*) work when they are
not pushed down. The same issue would occur for a select * for both sligned and
non aligned tables.
With the current calculations of how many to store in the scanner cache, we
calculate 100. So 100x5 MB is 500MB which is too large.
Limiting hte scanner cache, ensures that this does not run into a memory error.
So we can make this change in the code to limit the scanner cache size when the
row size is larger than a set size .|
> selct count(0) with long varchar hitting jvm OOM error
> --------------------------------------------------------
>
> Key: TRAFODION-3107
> URL: https://issues.apache.org/jira/browse/TRAFODION-3107
> Project: Apache Trafodion
> Issue Type: Bug
> Reporter: Sandhya Sundaresan
> Assignee: Sandhya Sundaresan
> Priority: Major
> Fix For: 2.3
>
>
> selct count(0) hitting jvm OOM error when clob as varchar and insert some 5MB
> images.
> org.trafodion.jdbc.t4.TrafT4Exception: *** ERROR[8448] Unable to access Hbase
> interface. Call to ExpHbaseInterface::nextRow returned error
> HBASE_ACCESS_ERROR(-706). Cause: java.util.concurrent.ExecutionException:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=3, exceptions:
> Tue Jun 05 10:53:33 CST 2018,
> RpcRetryingCaller\{globalStartTime=1528167213257, pause=100, retries=3},
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> Tue Jun 05 10:53:34 CST 2018,
> RpcRetryingCaller\{globalStartTime=1528167213257, pause=100, retries=3},
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> Tue Jun 05 10:53:34 CST 2018,
> RpcRetryingCaller\{globalStartTime=1528167213257, pause=100, retries=3},
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
>
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> org.trafodion.sql.HTableClient.fetchRows(HTableClient.java:1287) Caused by
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=3, exceptions:
> Tue Jun 05 10:53:33 CST 2018,
> RpcRetryingCaller\{globalStartTime=1528167213257, pause=100, retries=3},
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> Tue Jun 05 10:53:34 CST 2018,
> RpcRetryingCaller\{globalStartTime=1528167213257, pause=100, retries=3},
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> Tue Jun 05 10:53:34 CST 2018,
> RpcRetryingCaller\{globalStartTime=1528167213257, pause=100, retries=3},
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
>
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:157)
> org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745) Caused by
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:218)
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:397)
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:371)
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136)
> org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745) Caused by
> java.lang.OutOfMemoryError: Java heap space. [2018-06-05 10:53:34]
> at
> org.trafodion.jdbc.t4.TrafT4Messages.throwSQLException(TrafT4Messages.java:211)
> at
> org.trafodion.jdbc.t4.InterfaceResultSet.fetch(InterfaceResultSet.java:857)
> at org.trafodion.jdbc.t4.TrafT4ResultSet.next(TrafT4ResultSet.java:2786)
> at
> org.jdbctest.performance.lob.LobPerformance.test_clob_photo_normal(LobPerformance.java:1469)
> at
> org.jdbctest.performance.lob.LobPerformance.test_jdbc_api(LobPerformance.java:374)
> at
> org.jdbctest.performance.lob.LobPerformance.main(LobPerformance.java:319)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
