[
https://issues.apache.org/jira/browse/IMPALA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17892267#comment-17892267
]
Michael Smith commented on IMPALA-12909:
----------------------------------------
If I set up a cluster with a dedicated coordinator
{code}
start-impala-cluster.py --use_exclusive_coordinators -c 1 -s 3
{code}
the SCAN DATA SOURCE fragment is scheduled to one of the executors.
So I suspect this may specifically be an executor scheduling issue, where we
want to more evenly distribute DataSourceScanNodes across executors. I would
start by experimenting with queries using multiple external data sources and a
cluster with a dedicated coordinator as above, and get familiar with how
scheduler.cc schedules the fragments.
> Generate distributed plan for query accessing multiple JDBC tables
> ------------------------------------------------------------------
>
> Key: IMPALA-12909
> URL: https://issues.apache.org/jira/browse/IMPALA-12909
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Wenzhe Zhou
> Assignee: Pranav Yogi Lodha
> Priority: Major
>
> For a query which access multiple JDBC tables, Planner generate single node
> plan. It's better to generate distributed plan so that JDBC read could be
> scheduled on executors. This restriction is due to current design of External
> data source framework because scan is single threaded. DataSourceScanNode
> cannot run in node other than coordinator.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
