[
https://issues.apache.org/jira/browse/IMPALA-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-12192 started by Riza Suminto.
---------------------------------------------
> Scan fragment parallelism is limited by its ScanNode cost.
> ----------------------------------------------------------
>
> Key: IMPALA-12192
> URL: https://issues.apache.org/jira/browse/IMPALA-12192
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
>
> IMPALA-12091 implement processing cost for ScanNode. Scan fragment
> parallelism is capped by this ScanNode cost, originally intended to prevent
> scanner over-parallelization (small files problem).
> However, there is a bug in the implementation where scan fragment parallelism
> will always be limited solely by the ScanNode cost. If ScanNode is colocated
> with other query node operator that have higher processing cost, Planner will
> not scale it up beyond what is allowed by the ScanNode cost. Scan fragment
> should be allowed to scale up, as long as it is within total processing cost
> of the fragment and number of effective scan ranges.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
