[
https://issues.apache.org/jira/browse/TRAFODION-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596668#comment-16596668
]
Sandhya Sundaresan commented on TRAFODION-3192:
-----------------------------------------------
Here are my findings.
Each TDB has 4 values for queue sizes :
* Initial Size up (ISU)
* Initial Size Down (ISD)
* Max size up (MSU)
* Max size down (MSD)
With certain operators like Union, Sequence and Transport, we currently
statically allocate queues based whatever the CQD values for the max queue
sizes for these TDBs specify. Some operators like Sort, we do dynamic queue
resizing.
Eg For Union
GEN_UN_SIZE_DOWN =2
GEN_UN_SIZE_UP =16
This means Union TCB gets allocated with sizes 2 and 16.
With dynamic queue resizing, all queues start off with a default initial size
ISD,ISU of 4 and 4. (CQDs DYN_QUEUE_RESIZE_INIT_DOWN, DYN_QUEUE_RESIZE_INIT_UP)
The way queue resizing works is a ranking system. It looks for a pattern in how
often the queue changes from
Empty->Full (Increment resizePoints by 3)
Full->Full (Decrement resizePoints by 1)
Full->Empty (increment resizePoints by 3)
Empty->Empty (decrement resizePoints by 2)
Each time the transition happens we increment or decrement the resizePoints
with the "rank" for that case.
By default only if the resize points is 9 (controlled by CQD
DYN_QUEUE_RESIZE_LIMIT) , we allow resizing the queue.
We do have a special case at codegen time for Nested Join where we resize the
right side of Nested Join to always have larger queues.
But the fix I have added allows Sequence, Union and Transport to hvae dynamic
queue resizing and also increased the max that those queues can be resized to.
Tests show a good improvement in performance.
> UNION ALL up and down queue size default value too small, or not dynamically
> adjusted
> -------------------------------------------------------------------------------------
>
> Key: TRAFODION-3192
> URL: https://issues.apache.org/jira/browse/TRAFODION-3192
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-exe
> Affects Versions: any
> Reporter: Sandhya Sundaresan
> Assignee: Sandhya Sundaresan
> Priority: Major
> Fix For: 2.4
>
>
> The union up queue size is hardcoded in nadefaults.cpp as a start value as 16
> which
> is very low if large number of rows are being returned. Scan operator has it
> set to 2048.
> Some operators further adjust this up size from the original value based on
> estimate of
> the number of rows being returned. We can do that for union codegen method and
> also change the original up queue size to 2048.
>
> Noticed that Transpose, Sequence also have the same issue and dynamic queue
> resizing is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
