Shiva Jahangiri created ASTERIXDB-2656:
------------------------------------------
Summary: No need for frame constraint in probe phase of hybrid
hash join
Key: ASTERIXDB-2656
URL: https://issues.apache.org/jira/browse/ASTERIXDB-2656
Project: Apache AsterixDB
Issue Type: Bug
Components: *DB - AsterixDB
Reporter: Shiva Jahangiri
In an optimized hybrid hash join, there is a constraint that a spilled
partition cannot have more than 1 frame. While it seems to be necessary during
the build phase to help with having as much data for in-memory partitions, it
seems an unnecessary constraint during the probe phase.
During the probe, we should let the spilled partitions to grow and use the
leftover frames not used by in-memory partitions. This also solves a bug that
hybrid hash join currently has.
During the probe phase, in case there is not enough memory for inserting a
large record, asterixdb may either flush the record directly to the disk or
spill the largest spilled partition to the disk, depending on which one is
larger. In the case that the biggest spilled partition is the vicitim to flush
and release the frames, the large record cannot use the freed frames as it
belongs to a spilled partition(otherwise it would have been probed) and as such
1-frame limit is applied on it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
