[jira] [Commented] (FLINK-25835) The task initialization duration is recorded in logs

2022-01-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484354#comment-17484354 ] Yun Tang commented on FLINK-25835: -- Current state backend would print the time to restore the state

[jira] [Updated] (FLINK-25862) Refactor SharedStateRegistry to not limit StreamStateHandle to register/unregister

2022-01-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25862: - Summary: Refactor SharedStateRegistry to not limit StreamStateHandle to register/unregister (was:

[jira] [Resolved] (FLINK-25767) Translation of page 'Working with State' is incomplete

2022-01-28 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-25767. -- Fix Version/s: 1.14.4 Resolution: Fixed Merged master:

[jira] [Created] (FLINK-25872) Restoring from incremental checkpoint with changelog state-backend cannot work well with snapshot CLAIM mode

2022-01-28 Thread Yun Tang (Jira)
Yun Tang created FLINK-25872: Summary: Restoring from incremental checkpoint with changelog state-backend cannot work well with snapshot CLAIM mode Key: FLINK-25872 URL:

[jira] [Created] (FLINK-25862) Refactor SharedStateRegistry to limit StreamStateHandle to register/unregister

2022-01-28 Thread Yun Tang (Jira)
Yun Tang created FLINK-25862: Summary: Refactor SharedStateRegistry to limit StreamStateHandle to register/unregister Key: FLINK-25862 URL: https://issues.apache.org/jira/browse/FLINK-25862 Project:

[jira] [Reopened] (FLINK-25343) HBaseConnectorITCase.testTableSourceSinkWithDDL fail on azure

2022-01-26 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reopened FLINK-25343: -- Reopen the issue as another instance happened:

[jira] [Resolved] (FLINK-25557) Introduce incremental/full checkpoint size stats

2022-01-26 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-25557. -- Fix Version/s: 1.15.0 Resolution: Fixed merged in master:

[jira] [Created] (FLINK-25816) Changelog keyed state backend would come across NPE during notification

2022-01-25 Thread Yun Tang (Jira)
Yun Tang created FLINK-25816: Summary: Changelog keyed state backend would come across NPE during notification Key: FLINK-25816 URL: https://issues.apache.org/jira/browse/FLINK-25816 Project: Flink

[jira] [Commented] (FLINK-15507) Activate local recovery for RocksDB backends by default

2022-01-24 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481203#comment-17481203 ] Yun Tang commented on FLINK-15507: -- [~pnowojski] TaskStateManagerImpl would clean up local checkpoint

[jira] [Commented] (FLINK-25200) Implement duplicating for s3 filesystem

2022-01-24 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481048#comment-17481048 ] Yun Tang commented on FLINK-25200: -- [~pnowojski] Actually, I mean we should implement Aliyun OSS

[jira] [Commented] (FLINK-25200) Implement duplicating for s3 filesystem

2022-01-24 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480904#comment-17480904 ] Yun Tang commented on FLINK-25200: -- [~pnowojski] From my knowledge, S3 will involve copy on server side

[jira] [Closed] (FLINK-22578) deployment configuration page miss content navigate menu

2022-01-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-22578. Fix Version/s: (was: 1.15.0) (was: 1.13.6) Resolution: Implemented >

[jira] [Reopened] (FLINK-22578) deployment configuration page miss content navigate menu

2022-01-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reopened FLINK-22578: -- > deployment configuration page miss content navigate menu >

[jira] [Commented] (FLINK-25767) Translation of page 'Working with State' is incomplete

2022-01-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480830#comment-17480830 ] Yun Tang commented on FLINK-25767: -- [~paul8263] already assigned to you, please go ahead. >

[jira] [Assigned] (FLINK-25767) Translation of page 'Working with State' is incomplete

2022-01-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-25767: Assignee: Yao Zhang > Translation of page 'Working with State' is incomplete >

[jira] [Assigned] (FLINK-15507) Activate local recovery for RocksDB backends by default

2022-01-20 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-15507: Assignee: Yuan Mei (was: Yuan Mei) > Activate local recovery for RocksDB backends by default >

[jira] [Assigned] (FLINK-15507) Activate local recovery for RocksDB backends by default

2022-01-20 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-15507: Assignee: Yuan Mei > Activate local recovery for RocksDB backends by default >

[jira] [Updated] (FLINK-15507) Activate local recovery for RocksDB backends by default

2022-01-20 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-15507: - Fix Version/s: (was: 1.15.0) > Activate local recovery for RocksDB backends by default >

[jira] [Commented] (FLINK-25200) Implement duplicating for s3 filesystem

2022-01-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479139#comment-17479139 ] Yun Tang commented on FLINK-25200: -- [~akalashnikov] I think we should run tests to mock the condition

[jira] [Updated] (FLINK-24210) Window related serailizer should not return 0 as its serialized length

2022-01-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-24210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-24210: - Fix Version/s: 1.15.0 > Window related serailizer should not return 0 as its serialized length >

[jira] [Resolved] (FLINK-24210) Window related serailizer should not return 0 as its serialized length

2022-01-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-24210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-24210. -- Resolution: Fixed Merged in master: 573d2b038f8fd5dc1177733ee3087c5b6c847fa4 > Window related

[jira] [Commented] (FLINK-25200) Implement duplicating for s3 filesystem

2022-01-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478674#comment-17478674 ] Yun Tang commented on FLINK-25200: -- For general RocksDB incremental checkpoint, we should also test

[jira] [Closed] (FLINK-25494) Duplicate element serializer during DefaultOperatorStateBackendSnapshotStrategy#syncPrepareResources

2022-01-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-25494. Resolution: Invalid > Duplicate element serializer during >

[jira] [Assigned] (FLINK-25580) Minor improvements for RocksDBIncrementalCheckpointUtils

2022-01-16 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-25580: Assignee: Aitozi > Minor improvements for RocksDBIncrementalCheckpointUtils >

[jira] [Updated] (FLINK-25580) Minor improvements for RocksDBIncrementalCheckpointUtils

2022-01-16 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25580: - Fix Version/s: 1.15.0 > Minor improvements for RocksDBIncrementalCheckpointUtils >

[jira] [Resolved] (FLINK-25580) Minor improvements for RocksDBIncrementalCheckpointUtils

2022-01-16 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-25580. -- Resolution: Fixed Megred in master: b7fd63b41500dbfeadbf2ee48e1a83d77cd1259b > Minor improvements

[jira] [Closed] (FLINK-24797) GlobalWindow.Serializer#getLength() provides a wrong length.

2022-01-14 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-24797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-24797. Resolution: Information Provided [~bx123] Thanks for your effect to finding this problem. However, this

[jira] [Updated] (FLINK-23378) ContinuousProcessingTimeTrigger最后一个定时器无法触发

2022-01-12 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-23378: - Fix Version/s: (was: 1.14.3) > ContinuousProcessingTimeTrigger最后一个定时器无法触发 >

[jira] [Commented] (FLINK-23378) ContinuousProcessingTimeTrigger最后一个定时器无法触发

2022-01-12 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17474365#comment-17474365 ] Yun Tang commented on FLINK-23378: -- [~frey] Please make the title and description of this ticket is in

[jira] [Reopened] (FLINK-23378) ContinuousProcessingTimeTrigger最后一个定时器无法触发

2022-01-12 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reopened FLINK-23378: -- > ContinuousProcessingTimeTrigger最后一个定时器无法触发 > -- > >

[jira] [Resolved] (FLINK-25601) Update 'state.backend' in flink-conf.yaml

2022-01-11 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-25601. -- Resolution: Fixed merged in master: 41e96d2488e4caba17eef5172bb5ae493eb9c742 > Update

[jira] [Assigned] (FLINK-25478) Same materialized state handle should not register multi times

2022-01-11 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-25478: Assignee: Yun Tang > Same materialized state handle should not register multi times >

[jira] [Commented] (FLINK-25601) Update 'state.backend' in flink-conf.yaml

2022-01-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17472503#comment-17472503 ] Yun Tang commented on FLINK-25601: -- [~ana4] Already assigned to you, please go ahead. > Update

[jira] [Assigned] (FLINK-25601) Update 'state.backend' in flink-conf.yaml

2022-01-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-25601: Assignee: Ada Wong > Update 'state.backend' in flink-conf.yaml >

[jira] [Commented] (FLINK-25557) Introduce incremental/full checkpoint size stats

2022-01-09 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17471713#comment-17471713 ] Yun Tang commented on FLINK-25557: -- [~liufangqi] Thanks for your attention. I will introduce the new

[jira] [Resolved] (FLINK-25536) Minor Fix: Adjust the order of variable declaration and comment in StateAssignmentOperation

2022-01-07 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-25536. -- Fix Version/s: 1.15.0 Resolution: Fixed merged in master:

[jira] [Commented] (FLINK-25360) Add State Desc to CheckpointMetadata

2022-01-06 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17470355#comment-17470355 ] Yun Tang commented on FLINK-25360: -- [~liufangqi] I suggest to create a design doc via google doc first.

[jira] [Created] (FLINK-25557) Introduce incremental/full checkpoint size stats

2022-01-06 Thread Yun Tang (Jira)
Yun Tang created FLINK-25557: Summary: Introduce incremental/full checkpoint size stats Key: FLINK-25557 URL: https://issues.apache.org/jira/browse/FLINK-25557 Project: Flink Issue Type:

[jira] [Assigned] (FLINK-25536) Minor Fix: Adjust the order of variable declaration and comment in StateAssignmentOperation

2022-01-05 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-25536: Assignee: Junfan Zhang > Minor Fix: Adjust the order of variable declaration and comment in >

[jira] [Commented] (FLINK-25536) Minor Fix: Adjust the order of variable declaration and comment in StateAssignmentOperation

2022-01-05 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469694#comment-17469694 ] Yun Tang commented on FLINK-25536: -- [~zuston] I think the PR is easy for review. Please remember to

[jira] [Commented] (FLINK-25360) Add State Desc to CheckpointMetadata

2022-01-05 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469693#comment-17469693 ] Yun Tang commented on FLINK-25360: -- [~liufangqi] I think we could at least to write a discuss email for

[jira] [Updated] (FLINK-25458) Support local recovery

2022-01-05 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25458: - Description: Current changelog state-backend implementation cannot work well with local recovery

[jira] [Created] (FLINK-25524) If enabled changelog, RocksDB incremental checkpoint would always be full

2022-01-05 Thread Yun Tang (Jira)
Yun Tang created FLINK-25524: Summary: If enabled changelog, RocksDB incremental checkpoint would always be full Key: FLINK-25524 URL: https://issues.apache.org/jira/browse/FLINK-25524 Project: Flink

[jira] [Commented] (FLINK-25478) Same materialized state handle should not register multi times

2022-01-04 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468426#comment-17468426 ] Yun Tang commented on FLINK-25478: -- I created a branch to reproce this problem

[jira] [Updated] (FLINK-25477) The directory structure of the State Backends document is not standardized

2022-01-03 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25477: - Fix Version/s: 1.14.3 > The directory structure of the State Backends document is not standardized >

[jira] [Closed] (FLINK-25477) The directory structure of the State Backends document is not standardized

2022-01-03 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-25477. Merged in master: 332eee0a0d6edb9c9aa455d182f79977496af756   release-1.14:

[jira] [Updated] (FLINK-25429) Avoid to close output streams twice during uploading changelogs

2021-12-31 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25429: - Issue Type: Bug (was: Improvement) > Avoid to close output streams twice during uploading changelogs >

[jira] [Updated] (FLINK-25494) Duplicate element serializer during DefaultOperatorStateBackendSnapshotStrategy#syncPrepareResources

2021-12-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25494: - Description: Currently, during DefaultOperatorStateBackendSnapshotStrategy#syncPrepareResources, it

[jira] [Updated] (FLINK-25494) Duplicate element serializer during DefaultOperatorStateBackendSnapshotStrategy#syncPrepareResources

2021-12-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25494: - Description: Currently, during DefaultOperatorStateBackendSnapshotStrategy#syncPrepareResources, it

[jira] [Created] (FLINK-25494) Duplicate element serializer during DefaultOperatorStateBackendSnapshotStrategy#syncPrepareResources

2021-12-30 Thread Yun Tang (Jira)
Yun Tang created FLINK-25494: Summary: Duplicate element serializer during DefaultOperatorStateBackendSnapshotStrategy#syncPrepareResources Key: FLINK-25494 URL: https://issues.apache.org/jira/browse/FLINK-25494

[jira] [Closed] (FLINK-10954) Hardlink from files of previous local stored state might cross devices

2021-12-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-10954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-10954. Resolution: Information Provided After FLINK-25468, we can at least not fail the job. > Hardlink from

[jira] [Updated] (FLINK-25478) Same materialized state handle should not register multi times

2021-12-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25478: - Summary: Same materialized state handle should not register multi times (was: Same materialized state

[jira] [Updated] (FLINK-25478) Same materialized state handle should register multi times

2021-12-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25478: - Parent: FLINK-21352 Issue Type: Sub-task (was: Bug) > Same materialized state handle should

[jira] [Updated] (FLINK-25478) Same materialized state handle should register multi times

2021-12-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25478: - Description: Currently, changelog materialization would call RocksDB state backend's snapshot method

[jira] [Updated] (FLINK-25478) Same materialized state handle should register multi times

2021-12-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25478: - Summary: Same materialized state handle should register multi times (was: Changelog materialization

[jira] [Updated] (FLINK-25478) Changelog materialization with incremental checkpoint could cause checkpointed data lost

2021-12-30 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25478: - Description: Currently, changelog materialization would call RocksDB state backend's snapshot method

[jira] [Assigned] (FLINK-25477) The directory structure of the State Backends document is not standardized

2021-12-29 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-25477: Assignee: Hangxiang Yu > The directory structure of the State Backends document is not

[jira] [Created] (FLINK-25479) Changlog materialization with incremental checkpoint cannot work well in local tests

2021-12-29 Thread Yun Tang (Jira)
Yun Tang created FLINK-25479: Summary: Changlog materialization with incremental checkpoint cannot work well in local tests Key: FLINK-25479 URL: https://issues.apache.org/jira/browse/FLINK-25479

[jira] [Created] (FLINK-25478) Changelog materialization with incremental checkpoint could cause checkpointed data lost

2021-12-29 Thread Yun Tang (Jira)
Yun Tang created FLINK-25478: Summary: Changelog materialization with incremental checkpoint could cause checkpointed data lost Key: FLINK-25478 URL: https://issues.apache.org/jira/browse/FLINK-25478

[jira] [Resolved] (FLINK-25465) Correct the logic of materizating wrapped changelog state

2021-12-28 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-25465. -- Resolution: Fixed merged in master: 57391962d5119816beebec56e78a034130e0bfdd > Correct the logic of

[jira] [Resolved] (FLINK-25446) Avoid sanity check on read bytes on DataInputStream#read(byte[])

2021-12-28 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-25446. -- Resolution: Fixed Merged master: c0f46ef324c35b3ed7813c74931ab9cb589896f7 release-1.14:

[jira] [Commented] (FLINK-5279) Improve error message when trying to access keyed state in non-keyed operator

2021-12-28 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466060#comment-17466060 ] Yun Tang commented on FLINK-5279: - Current error message has been "{{{}Keyed state can only be used on a

[jira] [Closed] (FLINK-21218) "state.checkpoints.dir" should be required only when "execution.checkpointing.interval" is specified

2021-12-28 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-21218. Resolution: Information Provided Since Flink already use new API EmbeddedRocksDBStateBackend to replace

[jira] [Commented] (FLINK-25426) UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on AZP because it cannot allocate enough network buffers

2021-12-28 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466044#comment-17466044 ] Yun Tang commented on FLINK-25426: -- Another instance:

[jira] [Created] (FLINK-25466) TTL configuration could parse in StateTtlConfig#DISABLED

2021-12-27 Thread Yun Tang (Jira)
Yun Tang created FLINK-25466: Summary: TTL configuration could parse in StateTtlConfig#DISABLED Key: FLINK-25466 URL: https://issues.apache.org/jira/browse/FLINK-25466 Project: Flink Issue Type:

[jira] [Updated] (FLINK-25465) Correct the logic of materizating wrapped changelog state

2021-12-27 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25465: - Summary: Correct the logic of materizating wrapped changelog state (was: Correct the logic to

[jira] [Created] (FLINK-25465) Correct the logic to materizate on wrapped changelog state

2021-12-27 Thread Yun Tang (Jira)
Yun Tang created FLINK-25465: Summary: Correct the logic to materizate on wrapped changelog state Key: FLINK-25465 URL: https://issues.apache.org/jira/browse/FLINK-25465 Project: Flink Issue

[jira] [Commented] (FLINK-25426) UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on AZP because it cannot allocate enough network buffers

2021-12-27 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465955#comment-17465955 ] Yun Tang commented on FLINK-25426: -- Another instance:

[jira] [Resolved] (FLINK-24785) Relocate RocksDB's log under flink log directory by default

2021-12-27 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-24785. -- Resolution: Fixed merged in master: cdef0973b21196ca23a31486f5a996f0b397136d > Relocate RocksDB's

[jira] [Commented] (FLINK-25260) Recovery fails when using changelog+s3+presto

2021-12-27 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465646#comment-17465646 ] Yun Tang commented on FLINK-25260: -- I think this issue would be resolved by

[jira] [Updated] (FLINK-25458) Supports local recovery

2021-12-27 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25458: - Component/s: Runtime / Checkpointing Runtime / State Backends > Supports local

[jira] [Created] (FLINK-25458) Supports local recovery

2021-12-27 Thread Yun Tang (Jira)
Yun Tang created FLINK-25458: Summary: Supports local recovery Key: FLINK-25458 URL: https://issues.apache.org/jira/browse/FLINK-25458 Project: Flink Issue Type: Sub-task Reporter:

[jira] [Created] (FLINK-25446) Avoid sanity check on read bytes on DataInputStream#read(byte[])

2021-12-26 Thread Yun Tang (Jira)
Yun Tang created FLINK-25446: Summary: Avoid sanity check on read bytes on DataInputStream#read(byte[]) Key: FLINK-25446 URL: https://issues.apache.org/jira/browse/FLINK-25446 Project: Flink

[jira] [Resolved] (FLINK-25429) Avoid to close output streams twice during uploading changelogs

2021-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-25429. -- Resolution: Fixed > Avoid to close output streams twice during uploading changelogs >

[jira] [Resolved] (FLINK-25094) The verify code in LatencyTrackingMapStateTest#verifyIterator is not actually executed

2021-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-25094. -- Resolution: Fixed > The verify code in LatencyTrackingMapStateTest#verifyIterator is not actually >

[jira] [Commented] (FLINK-25094) The verify code in LatencyTrackingMapStateTest#verifyIterator is not actually executed

2021-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17464914#comment-17464914 ] Yun Tang commented on FLINK-25094: -- Merged in master: d08a1d0f4035485283f1063d4d0fc0a8aaca > The

[jira] [Commented] (FLINK-25429) Avoid to close output streams twice during uploading changelogs

2021-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17464911#comment-17464911 ] Yun Tang commented on FLINK-25429: -- merged in master: 804eb8dda556a2bea35c69a2662f13d1dafb9255 > Avoid

[jira] [Created] (FLINK-25438) KafkaProducerExactlyOnceITCase.testMultipleSinkOperators failed due to topic 'exactlyTopicCustomOperator20' already exists

2021-12-23 Thread Yun Tang (Jira)
Yun Tang created FLINK-25438: Summary: KafkaProducerExactlyOnceITCase.testMultipleSinkOperators failed due to topic 'exactlyTopicCustomOperator20' already exists Key: FLINK-25438 URL:

[jira] [Updated] (FLINK-25429) Avoid to close output streams twice during uploading changelogs

2021-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25429: - Fix Version/s: 1.15.0 > Avoid to close output streams twice during uploading changelogs >

[jira] [Created] (FLINK-25429) Avoid to close output streams twice during uploading changelogs

2021-12-23 Thread Yun Tang (Jira)
Yun Tang created FLINK-25429: Summary: Avoid to close output streams twice during uploading changelogs Key: FLINK-25429 URL: https://issues.apache.org/jira/browse/FLINK-25429 Project: Flink

[jira] [Assigned] (FLINK-17808) Rename checkpoint meta file to "_metadata" until it has completed writing

2021-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-17808: Assignee: Junfan Zhang > Rename checkpoint meta file to "_metadata" until it has completed

[jira] [Commented] (FLINK-17808) Rename checkpoint meta file to "_metadata" until it has completed writing

2021-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17464459#comment-17464459 ] Yun Tang commented on FLINK-17808: -- Since [~zuston] had already provided a PR to review, already

[jira] [Created] (FLINK-25423) Enable loading state backend via configuration in state processor api

2021-12-22 Thread Yun Tang (Jira)
Yun Tang created FLINK-25423: Summary: Enable loading state backend via configuration in state processor api Key: FLINK-25423 URL: https://issues.apache.org/jira/browse/FLINK-25423 Project: Flink

[jira] [Commented] (FLINK-25360) Add State Desc to CheckpointMetadata

2021-12-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17464242#comment-17464242 ] Yun Tang commented on FLINK-25360: -- [~liufangqi] Since RocksDB incremenatl checkpoints already have

[jira] [Commented] (FLINK-25360) Add State Desc to CheckpointMetadata

2021-12-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17463703#comment-17463703 ] Yun Tang commented on FLINK-25360: -- Raw state is not recomended to use anymore, and flink would only

[jira] [Commented] (FLINK-25360) Add State Desc to CheckpointMetadata

2021-12-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17463660#comment-17463660 ] Yun Tang commented on FLINK-25360: -- It seems you're trying to read checkpoint to modify previous

[jira] [Commented] (FLINK-25360) Add State Desc to CheckpointMetadata

2021-12-21 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17463622#comment-17463622 ] Yun Tang commented on FLINK-25360: -- [~liufangqi], I think your concerns have two different problems: #

[jira] [Commented] (FLINK-17808) Rename checkpoint meta file to "_metadata" until it has completed writing

2021-12-21 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17463604#comment-17463604 ] Yun Tang commented on FLINK-17808: -- [~zuston] It seems FileSystem#createRecoverableWriter is only

[jira] [Commented] (FLINK-25360) Add State Desc to CheckpointMetadata

2021-12-21 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17463557#comment-17463557 ] Yun Tang commented on FLINK-25360: -- [~liufangqi] Any changes to checkpoint persistent content requires

[jira] [Commented] (FLINK-17808) Rename checkpoint meta file to "_metadata" until it has completed writing

2021-12-21 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17463546#comment-17463546 ] Yun Tang commented on FLINK-17808: -- Before moving to review PR, [~zuston] could you share your ideas

[jira] [Commented] (FLINK-25360) Add State Desc to CheckpointMetadata

2021-12-17 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461276#comment-17461276 ] Yun Tang commented on FLINK-25360: -- The state name and state serializer have been persisted, what else

[jira] [Commented] (FLINK-9114) Enable user-provided, custom CheckpointRecoveryFactory for non-HA deployments

2021-12-11 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-9114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457630#comment-17457630 ] Yun Tang commented on FLINK-9114: - [~jparkie] What's the status of your idea to support user provided

[jira] [Updated] (FLINK-25099) flink on yarn Accessing two HDFS Clusters

2021-12-11 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-25099: - Component/s: Runtime / Checkpointing (was: Runtime / State Backends) > flink on

[jira] [Created] (FLINK-25237) ElasticsearchSinkITCase fails due to NoNodeAvailableException

2021-12-09 Thread Yun Tang (Jira)
Yun Tang created FLINK-25237: Summary: ElasticsearchSinkITCase fails due to NoNodeAvailableException Key: FLINK-25237 URL: https://issues.apache.org/jira/browse/FLINK-25237 Project: Flink Issue

[jira] [Resolved] (FLINK-21504) State ownership: notify TMs about checkpoint subsumption

2021-12-09 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-21504. -- Resolution: Fixed merged in master: def3855690f8a5ac24a889dc4ed5909d4f9b31aa > State ownership:

[jira] [Commented] (FLINK-4266) Cassandra SplitOver Statebackend

2021-12-08 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456215#comment-17456215 ] Yun Tang commented on FLINK-4266: - [~foxss] What's the progress of thie ticket, can we close it now? >

[jira] [Closed] (FLINK-3947) Provide low level access to RocksDB state backend

2021-12-08 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-3947. --- Resolution: Information Provided MapState and keyedStatebackend#getKeys have been implemented. Closing this

[jira] [Closed] (FLINK-4413) Improve savepoint restore error messages

2021-12-08 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-4413. --- Resolution: Information Provided Closing this ticket due to lack of activity. > Improve savepoint restore

[jira] [Closed] (FLINK-4493) Unify the snapshot output format for keyed-state backends

2021-12-08 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-4493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-4493. --- Resolution: Information Provided Already implemented in FLINK-20976. > Unify the snapshot output format for

[jira] [Closed] (FLINK-4916) BufferSpiller should distribute files across temp directories

2021-12-08 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-4916. --- Resolution: Information Provided {{BufferSpiller}} had been dropped. > BufferSpiller should distribute

<    2   3   4   5   6   7   8   9   10   11   >