[
https://issues.apache.org/jira/browse/IMPALA-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18089838#comment-18089838
]
ASF subversion and git services commented on IMPALA-15076:
----------------------------------------------------------
Commit cffe56518feb0a899c9cee9dd7bdb10f96106abc in impala's branch
refs/heads/master from Arnab Karmakar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=cffe56518 ]
IMPALA-15076: Fix JDBC scan duplication in UNION ALL and ResultSet race
After IMPALA-14523 patch (shared JDBC cursor fetching), UNION ALL
queries with JDBC tables produced incorrect results due to
duplicate table scans. The scheduler assigned each JDBC scan node to a
different executor based on node_id, causing UNPARTITIONED union
fragments to spawn instances on multiple executors. Each instance would
execute all scan nodes in the union, but only one had assigned ranges,
leading to 3x result duplication in queries.
The fix changes JDBC executor selection to hash fragment_idx instead of
node_id, ensuring all JDBC scans within a union fragment are co-located
on the same executor. This preserves the shared connection optimization
while preventing duplicate reads.
Additionally, fixed a race condition in JdbcRecordIterator where multiple
threads could call ResultSet.next() after end-of-stream. Per JDBC spec,
this behavior is vendor-specific for TYPE_FORWARD_ONLY result sets and
may throw SQLException. Added an endOfStream flag guarded by fetchLock
to prevent post-EOS next() calls.
Testing:
- Core TPC-DS JDBC tests now pass
- All tests passed with exhaustive exploration strategy using release build
- Added JDBC UNION ALL planner tests
Change-Id: I60ed011faa2177af67ea681c2cd2967648e4a963
Reviewed-on: http://gerrit.cloudera.org:8080/24464
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> TestTpcdsQueryForJdbcTables tests fail
> --------------------------------------
>
> Key: IMPALA-15076
> URL: https://issues.apache.org/jira/browse/IMPALA-15076
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 5.0.0
> Reporter: Balazs Hevele
> Assignee: Arnab Karmakar
> Priority: Major
>
> Several test cases fail in exhaustive builds.
> E.g. test_tpcds_q5:
> {code}
> AssertionError: assert Comparing QueryTestResults (expected vs actual):
> 'catalog channel','catalog_pageAAAAAAAAAABAAAAA',73724.99,0.00,-34506.26 !=
> 'catalog channel','catalog_pageAAAAAAAAAABAAAAA',221174.97,0.00,-103518.78
> 'catalog channel','catalog_pageAAAAAAAAAADAAAAA',73182.60,0.00,-17828.06 !=
> 'catalog channel','catalog_pageAAAAAAAAAADAAAAA',219547.80,0.00,-53484.18
> 'catalog channel','catalog_pageAAAAAAAAABAAAAAA',0.00,4543.46,-2785.79 !=
> 'catalog channel','catalog_pageAAAAAAAAABAAAAAA',0.00,13630.38,-8357.37
> 'catalog channel','catalog_pageAAAAAAAAABBAAAAA',72671.60,0.00,6590.95 !=
> 'catalog channel','catalog_pageAAAAAAAAABBAAAAA',218014.80,0.00,19772.85
> 'catalog channel','catalog_pageAAAAAAAAABCAAAAA',0.00,3646.32,-514.18 !=
> 'catalog channel','catalog_pageAAAAAAAAABCAAAAA',0.00,10938.96,-1542.54
> 'catalog channel','catalog_pageAAAAAAAAABDAAAAA',69548.37,0.00,-6749.40 !=
> 'catalog channel','catalog_pageAAAAAAAAABDAAAAA',208645.11,0.00,-20248.20
> 'catalog channel','catalog_pageAAAAAAAAACAAAAAA',0.00,10499.54,-5521.37 !=
> 'catalog channel','catalog_pageAAAAAAAAACAAAAAA',0.00,31498.62,-16564.11
> 'catalog channel','catalog_pageAAAAAAAAACBAAAAA',153295.93,5351.63,4632.81
> != 'catalog
> channel','catalog_pageAAAAAAAAACBAAAAA',459887.79,16054.89,13898.43
> 'catalog channel','catalog_pageAAAAAAAAACDAAAAA',54336.42,0.00,-18212.06 !=
> 'catalog channel','catalog_pageAAAAAAAAACDAAAAA',163009.26,0.00,-54636.18
> 'catalog channel','catalog_pageAAAAAAAAADAAAAAA',0.00,9992.13,-6117.85 !=
> 'catalog channel','catalog_pageAAAAAAAAADAAAAAA',0.00,29976.39,-18353.55
> 'catalog channel','catalog_pageAAAAAAAAADBAAAAA',131954.65,50.05,5527.80 !=
> 'catalog channel','catalog_pageAAAAAAAAADBAAAAA',395863.95,150.15,16583.40
> 'catalog channel','catalog_pageAAAAAAAAADDAAAAA',76814.28,0.00,-3457.30 !=
> 'catalog channel','catalog_pageAAAAAAAAADDAAAAA',230442.84,0.00,-10371.90
> 'catalog channel','catalog_pageAAAAAAAAAEAAAAAA',0.00,3197.25,-478.68 !=
> 'catalog channel','catalog_pageAAAAAAAAAEAAAAAA',0.00,9591.75,-1436.04
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
