[
https://issues.apache.org/jira/browse/DRILL-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rahul Challapalli updated DRILL-5104:
-------------------------------------
Attachment: physical.json
2711af55-e1f7-bf4e-3dcf-b9f66f69fc33.sys.drill
Looks like I am still hitting this problem. Below is the query I used to
generate the attached physical plan (physical.json)
{code}
ALTER SESSION SET `exec.sort.disable_managed` = false;
alter session set `planner.width.max_per_node` = 1;
alter session set `planner.width.max_per_node` = 1;
alter session set `planner.disable_exchanges` = true;
alter session set `planner.memory.max_query_memory_per_node` = 62914560;
select count(*) from (select * from
dfs.`/drill/testdata/resource-manager/250wide-small.tbl` order by columns[0])d
where d.columns[0] = 'ljdfhwuehnoiueyf';
{code}
Now I submitted the physical plan using the below command
{code}
/opt/drill/bin/submit_plan -f /home/framework/physical.json -t physical -z
10.10.100.183:5181
{code}
>From the attached query profile, external sort consumed 104 MB before
>spilling. Can you take a look [~Paul.Rogers] ?
> Foreman sets external sort memory allocation even for a physical plan
> ---------------------------------------------------------------------
>
> Key: DRILL-5104
> URL: https://issues.apache.org/jira/browse/DRILL-5104
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.8.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Labels: ready-to-commit
> Fix For: 1.10.0
>
> Attachments: 2711af55-e1f7-bf4e-3dcf-b9f66f69fc33.sys.drill,
> physical.json
>
>
> Consider the (disabled) unit test
> {{TestSimpleExternalSort.outOfMemoryExternalSort}} which uses the physical
> plan {{xsort/oom_sort_test.json}} that contains a setting for the amount of
> memory to allocate:
> {code}
> {
> ...
> pop:"external-sort",
> ...
> initialAllocation: 1000000,
> maxAllocation: 30000000
> },
> {code}
> When run, the amount of memory is set to 715827882. The reason is that code
> was added to {{Foreman}} to compute the memory to allocate to the external
> sort:
> {code}
> private void runPhysicalPlan(final PhysicalPlan plan) throws
> ExecutionSetupException {
> validatePlan(plan);
> MemoryAllocationUtilities.setupSortMemoryAllocations(plan, queryContext);
> {code}
> The problem is that a physical plan should execute as provided to enable
> detailed testing.
> To solve this problem, move the sort memory setup to the path taken by SQL
> queries, but not via physical plans.
> This change is necessary to re-enable the previously-disabled external sort
> tests.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
