[
https://issues.apache.org/jira/browse/FLINK-29425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-29425:
-----------------------------------
Labels: pull-request-available (was: )
> Hybrid full spilling strategy triggering spilling frequently
> ------------------------------------------------------------
>
> Key: FLINK-29425
> URL: https://issues.apache.org/jira/browse/FLINK-29425
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Network
> Affects Versions: 1.16.0
> Reporter: Weijie Guo
> Priority: Critical
> Labels: pull-request-available
>
> In hybrid shuffle mode, we have an internal config option
> 'DEFAULT_FULL_STRATEGY_NUM_BUFFERS_TRIGGER_SPILLED' to control spilling
> frequency in the full spilling strategy. Unfortunately, the default value(10)
> is too small. As a result, frequent disk spilling calls are observed in the
> TPC-DS test, which seriously affects performance. When we increase the value,
> the query performance is improved significantly. We should set a more
> reasonable default value, or adopt an adaptive strategy to determine the
> spilling frequency.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
