Weston Pace created ARROW-17115:
-----------------------------------
Summary: [C++] HashJoin fails if it encounters a batch with more
than 32Ki rows
Key: ARROW-17115
URL: https://issues.apache.org/jira/browse/ARROW-17115
Project: Apache Arrow
Issue Type: Bug
Components: C++
Reporter: Weston Pace
Assignee: Weston Pace
The new swiss join assumes that batches are being broken according to the
morsel/batch model and it assumes those batches have, at most, 32Ki rows
(signed 16-bit indices are used in various places).
However, we are not currently slicing all of our inputs to batches this small.
This is causing conbench to fail and would likely be a problem with any large
inputs.
We should fix this by slicing batches in the engine to the appropriate maximum
size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
