[
https://issues.apache.org/jira/browse/ASTERIXDB-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876333#comment-17876333
]
ASF subversion and git services commented on ASTERIXDB-3410:
------------------------------------------------------------
Commit 2c0f5d83aa445569c84370c7e7f6d55cf0806000 in asterixdb's branch
refs/heads/master from Peeyush Gupta
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=2c0f5d83aa ]
[ASTERIXDB-3410][COMP] Stack overflow with heavily nested queries
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
With this change we avoid inlining an expression if it is referencing the
same variable more than a fixed number of times (128 by default). We add a
compiler property compiler.max.variable.occurrences.inlining to update this
value.
Ext-ref: MB-63183
Change-Id: Ie3fed69bf915535302664beb335d32ccc4988afe
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18685
Reviewed-by: Ali Alsuliman <[email protected]>
Reviewed-by: Peeyush Gupta <[email protected]>
Integration-Tests: Jenkins <[email protected]>
Tested-by: Peeyush Gupta <[email protected]>
> OOM Error with Heavily Nested Queries
> -------------------------------------
>
> Key: ASTERIXDB-3410
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-3410
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: COMP - Compiler
> Reporter: Glenn Justo Galvizo
> Priority: Major
> Labels: triaged
>
> Something I came across while trying to use AFrame with AsterixDB
> ([https://dzone.com/articles/dataframes-with-big-data-in-couchbase]). A
> pattern used by AFrame and early versions of Graphix involved repeatedly
> nesting queries:
> SELECT VALUE OBJECT_REMOVE(t, 'categories')
> FROM (
> SELECT t.*, to_bigint(contains(categories, "Home Services")) AS `Home
> Services`
> FROM (
> SELECT t.*, to_bigint(contains(categories, "Bars")) AS `Bars`
> FROM ...
>
> .
> .
> .
>
> ) t
> ) t
>
> Unfortunately, for heavily nested queries, AsterixDB throws an out of memory
> error. This was an issue I faced with the compiler, not the runtime (I ran
> this with EXPLAIN, and I still got the OOM error).
>
> Potential remedy: we do have a decorrelation Algebricks rule, but it may
> (potentially) be less memory intensive to have an AST-level rule that tries
> to do the same.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
