[
https://issues.apache.org/jira/browse/ASTERIXDB-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962597#comment-16962597
]
ASF subversion and git services commented on ASTERIXDB-2656:
------------------------------------------------------------
Commit 95bf1b695829ee790374e6d1002bbe0019bf33f5 in asterixdb's branch
refs/heads/master from Shiva
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=95bf1b6 ]
[ASTERIXDB-2656] Remove frame constraint in HHJ probe
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
During the probe phase, we should not have the 1-frame
limit for spilled partitions. It wastes the memory and
leads to flushing small objects directly to the disk as
a large object.
Change-Id: I7580ecbe7db36d018348cd82d3864ca6c2ff439d
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3703
Contrib: Jenkins <[email protected]>
Integration-Tests: Jenkins <[email protected]>
Tested-by: Jenkins <[email protected]>
Reviewed-by: Ali Alsuliman <[email protected]>
> No need for frame constraint in probe phase of hybrid hash join
> ---------------------------------------------------------------
>
> Key: ASTERIXDB-2656
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2656
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: *DB - AsterixDB
> Reporter: Shiva Jahangiri
> Priority: Major
>
> In an optimized hybrid hash join, there is a constraint that a spilled
> partition cannot have more than 1 frame. While it seems to be necessary
> during the build phase to help with having as much data for in-memory
> partitions, it seems an unnecessary constraint during the probe phase.
> During the probe, we should let the spilled partitions to grow and use the
> leftover frames not used by in-memory partitions. This also solves a bug that
> hybrid hash join currently has.
> During the probe phase, in case there is not enough memory for inserting a
> large record, asterixdb may either flush the record directly to the disk or
> spill the largest spilled partition to the disk, depending on which one is
> larger. In the case that the biggest spilled partition is the vicitim to
> flush and release the frames, the large record cannot use the freed frames as
> it belongs to a spilled partition(otherwise it would have been probed) and as
> such 1-frame limit is applied on it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
