[
https://issues.apache.org/jira/browse/FLINK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuxin Tan reassigned FLINK-32779:
---------------------------------
Assignee: Weijie Guo (was: Weijie Tong)
> Release Testing: Verify FLIP-301: Hybrid Shuffle supports Remote Storage
> ------------------------------------------------------------------------
>
> Key: FLINK-32779
> URL: https://issues.apache.org/jira/browse/FLINK-32779
> Project: Flink
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 1.18.0
> Reporter: Qingsheng Ren
> Assignee: Weijie Guo
> Priority: Major
> Fix For: 1.18.0
>
>
> This ticket aims to verify https://issues.apache.org/jira/browse/FLINK-31634.
> This verification mainly contains two parts.
> Part 1. Run without remote storage.
> This part mainly is to verify the new mode can use the Memory tier and Disk
> tier dynamically when shuffling.
> Set the mode to new hybrid shuffle mode(execution.batch-shuffle-mode:
> ALL_EXCHANGES_HYBRID_SELECTIVE), and run a simple job. For example(tpcds
> q1.sql). When the resource is enough, then the upstream and the downstream
> can run at the same time.
> Part2. Run with remote storage.
> This part mainly is to verify the new mode can use the Memory tier, Disk
> tier, Remote tier dynamically when shuffling.
> 2.1 Set the mode to new hybrid shuffle mode(execution.batch-shuffle-mode:
> ALL_EXCHANGES_HYBRID_SELECTIVE)
> 2.2 set the remote storage path with the
> option(taskmanager.network.hybrid-shuffle.remote.path:
> oss://flink-runtime/runtime/shuffle, note that the path
> oss://flink-runtime/runtime/shuffle in oss should be exist).
> 2.3 Modify the
> option TieredStorageConfiguration#DEFAULT_MIN_RESERVE_DISK_SPACE_FRACTION to
> 1, compile the package, then run a simple job. For example(tpcds q1.sql).
> Check the shuffle data is written to the remote storage in the path
> oss://flink-runtime/runtime/shuffle.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
