[
https://issues.apache.org/jira/browse/ARROW-17115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weston Pace resolved ARROW-17115.
---------------------------------
Resolution: Fixed
Issue resolved by pull request 13679
[https://github.com/apache/arrow/pull/13679]
> [C++] HashJoin fails if it encounters a batch with more than 32Ki rows
> ----------------------------------------------------------------------
>
> Key: ARROW-17115
> URL: https://issues.apache.org/jira/browse/ARROW-17115
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Reporter: Weston Pace
> Assignee: Weston Pace
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 9.0.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> The new swiss join assumes that batches are being broken according to the
> morsel/batch model and it assumes those batches have, at most, 32Ki rows
> (signed 16-bit indices are used in various places).
> However, we are not currently slicing all of our inputs to batches this
> small. This is causing conbench to fail and would likely be a problem with
> any large inputs.
> We should fix this by slicing batches in the engine to the appropriate
> maximum size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
