[
https://issues.apache.org/jira/browse/FLINK-31059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17692559#comment-17692559
]
dalongliu commented on FLINK-31059:
-----------------------------------
[~miamiaoxyz] For point a, I've tested it on my machine, yes, it doesn't work
in SqlClient if we load the HiveModule first and then enable the option. But If
we enable the option first and load the HiveModule, this option works well.
After deep dive into the related code, I think there exists some problem in the
code implementation on SqlClient side. If we want this option can work at the
job level, we need to modify the public API `Module`. Therefore, I have created
an [issue|https://issues.apache.org/jira/browse/FLINK-31193]to track it, we
will fix it in Flink 1.18. Currently, if we want to use this option in
SqlClient, here's what we should:
{code:java}
// enable option first
SET 'table.exec.hive.native-agg-function.enabled' = 'true';
// then laod hive module
LOAD MODULE hive WITH ('hive-version' = '...') {code}
In addition, I will add some notes to the document to remind users.
> Release Testing: Verify FLINK-29717 Supports hive udaf such as sum/count by
> native implementation
> -------------------------------------------------------------------------------------------------
>
> Key: FLINK-31059
> URL: https://issues.apache.org/jira/browse/FLINK-31059
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Hive
> Affects Versions: 1.17.0
> Reporter: dalongliu
> Assignee: miamiaoxyz
> Priority: Blocker
> Fix For: 1.17.0
>
> Attachments: image-2023-02-21-15-45-48-226.png,
> image-2023-02-21-15-46-13-966.png, image-2023-02-21-15-47-54-043.png,
> image-2023-02-21-15-49-58-854.png, image-2023-02-21-15-59-44-470.png,
> image-2023-02-21-16-28-22-038.png, image-2023-02-21-16-29-42-983.png,
> image-2023-02-21-16-31-58-361.png, image-2023-02-21-16-35-46-294.png
>
>
> This task aims to verify
> [FLINK-29717|https://issues.apache.org/jira/browse/FLINK-29717] which
> improves the hive udaf performance.
> As the document [PR|https://github.com/apache/flink/pull/21789] description,
> please veriy:
> 1. Enabling the option `table.exec.hive.native-agg-function.enabled`, use the
> sum/count/avg/min/max functions separately in the query to verify if the
> hash-agg strategy is chosen via plan, and verify if the data results are the
> same as when the option `table.exec.hive.native-agg-function.enabled` is
> disabled.
> 2. Enabling the option `table.exec.hive.native-agg-function.enabled`, combine
> sum/count/avg/min/max functions in query, verify if the hash-agg strategy is
> chosen via plan, and verify if the data results are the same as when option
> `table.exec.hive.native-agg-function.enabled` is disabled.
> 3. Enabling the option `table.exec.hive.native-agg-function.enabled`, count
> or max array&struct and other complex types in query, verify whether the
> sort-agg strategy is chosen via plan, verify whether the data result is the
> same as when option `table.exec.hive.native-agg-function.enabled` is disabled.
> 4. Enabling the option `table.exec.hive.native-agg-function.enabled`, use the
> sum/count and first_value/last_value functions in the query simultaneously,
> verify that the sort-agg strategy is chosen via plan, verify that the data is
> the same as when option `table.exec.hive.native-agg-function.enabled` is
> disabled.
> 5. Enabling the option `table.exec.hive.native-agg-function.enabled`, use the
> sum/count/avg/min/max functions in the query and open sort-agg strategy
> forcibly, verify that the data results are the same as when option
> `table.exec.hive.native-agg-function.enabled` is disabled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
