[
https://issues.apache.org/jira/browse/IMPALA-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088265#comment-18088265
]
Arnab Karmakar commented on IMPALA-15076:
-----------------------------------------
Assigning this to myself on [~kdeschle]'s instruction.
I think the root cause is that the scheduler pins each JDBC scan node to a
different executor by hashing node_id
([scheduler.cc).|https://github.com/apache/impala/blob/670872bc8b868ef6a6f08d524603f99c16cfb061/be/src/scheduling/scheduler.cc#L1186]
{code:java}
size_t idx = std::hash<int>{}(node_id) % all_execs.size(); {code}
JDBC scan nodes in the same union fragment are hashed to different executors,
so each executor runs every scan and sums triple.
We should try pinning by fragment_idx instead of node_id.
> TestTpcdsQueryForJdbcTables tests fail
> --------------------------------------
>
> Key: IMPALA-15076
> URL: https://issues.apache.org/jira/browse/IMPALA-15076
> Project: IMPALA
> Issue Type: Bug
> Reporter: Balazs Hevele
> Assignee: Arnab Karmakar
> Priority: Major
>
> Several test cases fail in exhaustive builds.
> E.g. test_tpcds_q5:
> {code}
> AssertionError: assert Comparing QueryTestResults (expected vs actual):
> 'catalog channel','catalog_pageAAAAAAAAAABAAAAA',73724.99,0.00,-34506.26 !=
> 'catalog channel','catalog_pageAAAAAAAAAABAAAAA',221174.97,0.00,-103518.78
> 'catalog channel','catalog_pageAAAAAAAAAADAAAAA',73182.60,0.00,-17828.06 !=
> 'catalog channel','catalog_pageAAAAAAAAAADAAAAA',219547.80,0.00,-53484.18
> 'catalog channel','catalog_pageAAAAAAAAABAAAAAA',0.00,4543.46,-2785.79 !=
> 'catalog channel','catalog_pageAAAAAAAAABAAAAAA',0.00,13630.38,-8357.37
> 'catalog channel','catalog_pageAAAAAAAAABBAAAAA',72671.60,0.00,6590.95 !=
> 'catalog channel','catalog_pageAAAAAAAAABBAAAAA',218014.80,0.00,19772.85
> 'catalog channel','catalog_pageAAAAAAAAABCAAAAA',0.00,3646.32,-514.18 !=
> 'catalog channel','catalog_pageAAAAAAAAABCAAAAA',0.00,10938.96,-1542.54
> 'catalog channel','catalog_pageAAAAAAAAABDAAAAA',69548.37,0.00,-6749.40 !=
> 'catalog channel','catalog_pageAAAAAAAAABDAAAAA',208645.11,0.00,-20248.20
> 'catalog channel','catalog_pageAAAAAAAAACAAAAAA',0.00,10499.54,-5521.37 !=
> 'catalog channel','catalog_pageAAAAAAAAACAAAAAA',0.00,31498.62,-16564.11
> 'catalog channel','catalog_pageAAAAAAAAACBAAAAA',153295.93,5351.63,4632.81
> != 'catalog
> channel','catalog_pageAAAAAAAAACBAAAAA',459887.79,16054.89,13898.43
> 'catalog channel','catalog_pageAAAAAAAAACDAAAAA',54336.42,0.00,-18212.06 !=
> 'catalog channel','catalog_pageAAAAAAAAACDAAAAA',163009.26,0.00,-54636.18
> 'catalog channel','catalog_pageAAAAAAAAADAAAAAA',0.00,9992.13,-6117.85 !=
> 'catalog channel','catalog_pageAAAAAAAAADAAAAAA',0.00,29976.39,-18353.55
> 'catalog channel','catalog_pageAAAAAAAAADBAAAAA',131954.65,50.05,5527.80 !=
> 'catalog channel','catalog_pageAAAAAAAAADBAAAAA',395863.95,150.15,16583.40
> 'catalog channel','catalog_pageAAAAAAAAADDAAAAA',76814.28,0.00,-3457.30 !=
> 'catalog channel','catalog_pageAAAAAAAAADDAAAAA',230442.84,0.00,-10371.90
> 'catalog channel','catalog_pageAAAAAAAAAEAAAAAA',0.00,3197.25,-478.68 !=
> 'catalog channel','catalog_pageAAAAAAAAAEAAAAAA',0.00,9591.75,-1436.04
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
