[
https://issues.apache.org/jira/browse/TAJO-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101360#comment-15101360
]
ASF GitHub Bot commented on TAJO-2036:
--------------------------------------
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/928#issuecomment-171892339
@combineads, sorry for late response.
Actually, my intention was to keep both factors, but clean some unnecessary
codes.
I think we need to keep only the max number of rows rather than max byte
size in the future. However, for now, it would be ok if we support both
factors. How about adding the max number of rows again?
> Prevent out of memory in the master server, if the query result is large
> ------------------------------------------------------------------------
>
> Key: TAJO-2036
> URL: https://issues.apache.org/jira/browse/TAJO-2036
> Project: Tajo
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 0.11.1
> Reporter: Byunghwa Yun
> Assignee: Byunghwa Yun
> Priority: Minor
> Fix For: 0.12.0, 0.11.1
>
>
> Tajo Master server is a no response below situation.
> Test SQL:
> select col1, col2, .... col100 from big_table
> (big_table has rows: 1,000,000,000)
> Web UI parameters:
> Limit : 10MB
> Rows/Page : 25
> Master Server Log:
> 2016-01-06 10:35:59,973 INFO
> org.apache.tajo.tuple.memory.ResizableMemoryBlock: Increase DirectRowBlock to
> 64.0 KiB
> 2016-01-06 10:36:00,739 INFO
> org.apache.tajo.tuple.memory.ResizableMemoryBlock: Increase DirectRowBlock to
> 64.0 KiB
> 2016-01-06 10:36:01,316 INFO
> org.apache.tajo.master.exec.NonForwardQueryResultFileScanner: "Sent result to
> client for 7cba1175-e77b-4876-957d-2ff8722e940b, queryId:
> q_1452043069985_0001 rows: 3986313
> 2016-01-06 10:36:49,561 INFO org.apache.tajo.util.JvmPauseMonitor: Detected
> pause in JVM or host machine (eg GC): pause of approximately 6385ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=5460ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=993ms
> 2016-01-06 10:37:03,102 WARN org.apache.tajo.util.JvmPauseMonitor: Detected
> pause in JVM or host machine (eg GC): pause of approximately 10587ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=10669ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=401ms
> Master Server Stacktrace:
> "pool-11-thread-1" #379 prio=5 os_prio=0 tid=0x00007f47d424d000 nid=0xe26c
> waiting on condition [0x00007f4834cfa000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000000076e9c7220> (a
> com.google.common.util.concurrent.AbstractFuture$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:275)
> at
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)
> at
> org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:77)
> at
> org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:711)
> at
> org.apache.tajo.webapp.QueryExecutorServlet$QueryRunner.MakeResultText(QueryExecutorServlet.java:511)
> at
> org.apache.tajo.webapp.QueryExecutorServlet$QueryRunner.getQueryResult(QueryExecutorServlet.java:466)
> at
> org.apache.tajo.webapp.QueryExecutorServlet$QueryRunner.run(QueryExecutorServlet.java:347)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
