[
https://issues.apache.org/jira/browse/HIVE-27858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17820717#comment-17820717
]
okumin commented on HIVE-27858:
-------------------------------
[~dkuzmenko]
I would like to simply VOID this ticket as we have nothing to do with this
ticket. No versions can successfully finish the query without CTE
materialization. So, this is not degradation. Also, I think we are not
obligated to process a query with 10,000 vertices without any tuning.
* Hive 2 with hive.optimize.cte.materialize.threshold=-1 => OOM
* Hive 3 with hive.optimize.cte.materialize.threshold=-1 => OOM
* Hive 4 or the master with hive.optimize.cte.materialize.threshold=-1 => OOM
Though Hive 3 fails even with CTE materialization, it's been fixed on the
master branch. We have nothing to do.
* Hive 2 with hive.optimize.cte.materialize.threshold=2 => OK
* Hive 3 with hive.optimize.cte.materialize.threshold=2 => Failed with
"Grouping sets size cannot be greater than 64"
* Hive 4 or the master branch with hive.optimize.cte.materialize.threshold=2
and hive.optimize.cte.materialize.full.aggregate.only=false => OK
Does that make sense? We can keep this ticket if we would like to support such
queries without CTE materialization. It sounds like a new capability rather
than a bug fix.
> OOM happens when selecting many columns and JOIN.
> --------------------------------------------------
>
> Key: HIVE-27858
> URL: https://issues.apache.org/jira/browse/HIVE-27858
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
> Affects Versions: 4.0.0-beta-1
> Reporter: Ryu Kobayashi
> Assignee: okumin
> Priority: Major
> Fix For: Not Applicable
>
> Attachments: HIVE-27858.full.q, HIVE-27858.less.columns.q, ddl.sql,
> query.sql
>
>
> OOM happens when executing [^query.sql] using a table in [^ddl.sql]. These
> did not happen in Hive 2 previously.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
