[jira] [Commented] (FLINK-32783) Release Testing: Improve parallel download of RocksDB incremental state
[ https://issues.apache.org/jira/browse/FLINK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762308#comment-17762308 ] Sergey Nuyanzin commented on FLINK-32783: - Thanks for the advice [~knaufk] I'm going to close it > Release Testing: Improve parallel download of RocksDB incremental state > --- > > Key: FLINK-32783 > URL: https://issues.apache.org/jira/browse/FLINK-32783 > Project: Flink > Issue Type: Sub-task > Components: Tests >Affects Versions: 1.18.0 >Reporter: Qingsheng Ren >Priority: Major > Fix For: 1.18.0 > > > This feature is automatically used whenever we download state during a > restart from a RocksDB incremental checkpoint. This should be tested with and > without task-local recovery. > Will be covered by the nightly tests: > * run_test "Resuming Externalized Checkpoint (rocks, incremental, no > parallelism change) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 2 > rocks true true" "skip_check_exceptions" > * run_test "Resuming Externalized Checkpoint (rocks, incremental, scale > up) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 4 > rocks true true" "skip_check_exceptions" > * run_test "Resuming Externalized Checkpoint (rocks, incremental, scale > down) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 4 2 > rocks true true" "skip_check_exceptions" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32783) Release Testing: Improve parallel download of RocksDB incremental state
[ https://issues.apache.org/jira/browse/FLINK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762002#comment-17762002 ] Yun Tang commented on FLINK-32783: -- +1 for [~knaufk]'s advice, let's close this ticket [~renqs]. > Release Testing: Improve parallel download of RocksDB incremental state > --- > > Key: FLINK-32783 > URL: https://issues.apache.org/jira/browse/FLINK-32783 > Project: Flink > Issue Type: Sub-task > Components: Tests >Affects Versions: 1.18.0 >Reporter: Qingsheng Ren >Priority: Major > Fix For: 1.18.0 > > > This feature is automatically used whenever we download state during a > restart from a RocksDB incremental checkpoint. This should be tested with and > without task-local recovery. > Will be covered by the nightly tests: > * run_test "Resuming Externalized Checkpoint (rocks, incremental, no > parallelism change) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 2 > rocks true true" "skip_check_exceptions" > * run_test "Resuming Externalized Checkpoint (rocks, incremental, scale > up) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 4 > rocks true true" "skip_check_exceptions" > * run_test "Resuming Externalized Checkpoint (rocks, incremental, scale > down) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 4 2 > rocks true true" "skip_check_exceptions" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32783) Release Testing: Improve parallel download of RocksDB incremental state
[ https://issues.apache.org/jira/browse/FLINK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761982#comment-17761982 ] Konstantin Knauf commented on FLINK-32783: -- I don't think we need to test it after talking to [~srichter]. The correctness is verified by our nightly tests. If someone wants to verify the performance improvements, we could leave it open, but I don't think this is strictly necessary. > Release Testing: Improve parallel download of RocksDB incremental state > --- > > Key: FLINK-32783 > URL: https://issues.apache.org/jira/browse/FLINK-32783 > Project: Flink > Issue Type: Sub-task > Components: Tests >Affects Versions: 1.18.0 >Reporter: Qingsheng Ren >Priority: Major > Fix For: 1.18.0 > > > This feature is automatically used whenever we download state during a > restart from a RocksDB incremental checkpoint. This should be tested with and > without task-local recovery. > Will be covered by the nightly tests: > * run_test "Resuming Externalized Checkpoint (rocks, incremental, no > parallelism change) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 2 > rocks true true" "skip_check_exceptions" > * run_test "Resuming Externalized Checkpoint (rocks, incremental, scale > up) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 4 > rocks true true" "skip_check_exceptions" > * run_test "Resuming Externalized Checkpoint (rocks, incremental, scale > down) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 4 2 > rocks true true" "skip_check_exceptions" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32783) Release Testing: Improve parallel download of RocksDB incremental state
[ https://issues.apache.org/jira/browse/FLINK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761682#comment-17761682 ] Yun Tang commented on FLINK-32783: -- Why do we need to test this internal improvement manually? From my understanding, this is not a FLIP and not so visible to users. If we stick to adding such a release testing, I could help to verify this. > Release Testing: Improve parallel download of RocksDB incremental state > --- > > Key: FLINK-32783 > URL: https://issues.apache.org/jira/browse/FLINK-32783 > Project: Flink > Issue Type: Sub-task > Components: Tests >Affects Versions: 1.18.0 >Reporter: Qingsheng Ren >Priority: Major > Fix For: 1.18.0 > > > This feature is automatically used whenever we download state during a > restart from a RocksDB incremental checkpoint. This should be tested with and > without task-local recovery. > Will be covered by the nightly tests: > * run_test "Resuming Externalized Checkpoint (rocks, incremental, no > parallelism change) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 2 > rocks true true" "skip_check_exceptions" > * run_test "Resuming Externalized Checkpoint (rocks, incremental, scale > up) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 4 > rocks true true" "skip_check_exceptions" > * run_test "Resuming Externalized Checkpoint (rocks, incremental, scale > down) end-to-end test" > "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 4 2 > rocks true true" "skip_check_exceptions" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32783) Release Testing: Improve parallel download of RocksDB incremental state
[ https://issues.apache.org/jira/browse/FLINK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757266#comment-17757266 ] Qingsheng Ren commented on FLINK-32783: --- [~srichter] Could you provide some instructions or link to the documentation in the "Description" field about how to verify the feature? Thanks! > Release Testing: Improve parallel download of RocksDB incremental state > --- > > Key: FLINK-32783 > URL: https://issues.apache.org/jira/browse/FLINK-32783 > Project: Flink > Issue Type: Sub-task > Components: Tests >Affects Versions: 1.18.0 >Reporter: Qingsheng Ren >Priority: Major > Fix For: 1.18.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32783) Release Testing: Improve parallel download of RocksDB incremental state
[ https://issues.apache.org/jira/browse/FLINK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757264#comment-17757264 ] Qingsheng Ren commented on FLINK-32783: --- Unassigning the issue from the original author of the new feature. Please re-assign the ticket to the actual person doing the test. > Release Testing: Improve parallel download of RocksDB incremental state > --- > > Key: FLINK-32783 > URL: https://issues.apache.org/jira/browse/FLINK-32783 > Project: Flink > Issue Type: Sub-task > Components: Tests >Affects Versions: 1.18.0 >Reporter: Qingsheng Ren >Priority: Major > Fix For: 1.18.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
