[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33934: Description: In our product we encounter a case that lead to data lost, the job info:     1.

[jira] [Comment Edited] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17800273#comment-17800273 ] Cai Liuyang edited comment on FLINK-33934 at 12/25/23 11:41 AM:

[jira] [Commented] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17800273#comment-17800273 ] Cai Liuyang commented on FLINK-33934: - [~hackergin] Yeah,  turn off Object Reuse for all

[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33934: Description: In our product we encounter a case that lead to data lost, the job info:     1.

[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33934: Description: In our product we encounter a case that lead to data lost, the job info:     1.

[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33934: Description: In our product we encounter a case that lead to data lost, the job info:     1.

[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33934: Description: In our product we encounter a case that lead to data lost, the job info:     1.

[jira] [Created] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

Cai Liuyang created FLINK-33934: --- Summary: Flink SQL Source use raw format maybe lead to data lost Key: FLINK-33934 URL: https://issues.apache.org/jira/browse/FLINK-33934 Project: Flink Issue

[jira] [Updated] (FLINK-33759) flink parquet writer support write nested array or map type

[ https://issues.apache.org/jira/browse/FLINK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33759: Description: When we use flink-parquet format wirte Map[] type (which will be read by spark

[jira] [Updated] (FLINK-33759) flink parquet writer support write nested array or map type

[ https://issues.apache.org/jira/browse/FLINK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33759: Description: When we use flink-parquet format wirte Map[] type (which will be read by spark

[jira] [Updated] (FLINK-33759) flink parquet writer support write nested array or map type

[ https://issues.apache.org/jira/browse/FLINK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33759: Description: When we use flink-parquet format wirte Map[] type (which will be read by spark

[jira] [Updated] (FLINK-33759) flink parquet writer support write nested array or map type

[ https://issues.apache.org/jira/browse/FLINK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33759: Description: When we use flink-parquet format wirte Map[] type (which will be read by spark job),

[jira] [Updated] (FLINK-33759) flink parquet writer support write nested array or map type

[ https://issues.apache.org/jira/browse/FLINK-33759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33759: Description: When we use flink-parquet format wirte Map[] type (which will be read by spark

[jira] [Created] (FLINK-33759) flink parquet writer support write nested array or map type

Cai Liuyang created FLINK-33759: --- Summary: flink parquet writer support write nested array or map type Key: FLINK-33759 URL: https://issues.apache.org/jira/browse/FLINK-33759 Project: Flink

[jira] [Commented] (FLINK-32362) SourceAlignment announceCombinedWatermark period task maybe lost

[ https://issues.apache.org/jira/browse/FLINK-32362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733451#comment-17733451 ] Cai Liuyang commented on FLINK-32362: - [~fanrui] I compare the two ways, in common case if one task

[jira] [Commented] (FLINK-32362) SourceAlignment announceCombinedWatermark period task maybe lost

[ https://issues.apache.org/jira/browse/FLINK-32362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733349#comment-17733349 ] Cai Liuyang commented on FLINK-32362: - Two ways: # the simple way code like: {code:java} // code

[jira] [Comment Edited] (FLINK-32362) SourceAlignment announceCombinedWatermark period task maybe lost

[ https://issues.apache.org/jira/browse/FLINK-32362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733349#comment-17733349 ] Cai Liuyang edited comment on FLINK-32362 at 6/16/23 6:38 AM: -- [~fanrui]

[jira] (FLINK-32362) SourceAlignment announceCombinedWatermark period task maybe lost

[ https://issues.apache.org/jira/browse/FLINK-32362 ] Cai Liuyang deleted comment on FLINK-32362: - was (Author: cailiuyang): [~fanrui] Two ways: > SourceAlignment announceCombinedWatermark period task maybe lost >

[jira] [Commented] (FLINK-32362) SourceAlignment announceCombinedWatermark period task maybe lost

[ https://issues.apache.org/jira/browse/FLINK-32362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733345#comment-17733345 ] Cai Liuyang commented on FLINK-32362: - [~fanrui] Two ways: > SourceAlignment

[jira] [Updated] (FLINK-32316) Duplicated announceCombinedWatermark task maybe scheduled if jobmanager failover

[ https://issues.apache.org/jira/browse/FLINK-32316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-32316: Description: When we try SourceAlignment feature, we found there will be a duplicated

[jira] [Updated] (FLINK-32316) Duplicated announceCombinedWatermark task maybe scheduled if jobmanager failover

[ https://issues.apache.org/jira/browse/FLINK-32316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-32316: Description: When we try SourceAlignment feature, we found there will be a duplicated

[jira] [Created] (FLINK-32362) SourceAlignment announceCombinedWatermark period task maybe lost

Cai Liuyang created FLINK-32362: --- Summary: SourceAlignment announceCombinedWatermark period task maybe lost Key: FLINK-32362 URL: https://issues.apache.org/jira/browse/FLINK-32362 Project: Flink

[jira] [Commented] (FLINK-32316) Duplicated announceCombinedWatermark task maybe scheduled if jobmanager failover

[ https://issues.apache.org/jira/browse/FLINK-32316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733278#comment-17733278 ] Cai Liuyang commented on FLINK-32316: - [~fanrui] OK，please assign to me，thks～ > Duplicated

[jira] [Updated] (FLINK-32316) Duplicated announceCombinedWatermark task maybe scheduled if jobmanager failover

[ https://issues.apache.org/jira/browse/FLINK-32316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-32316: Description: When we try SourceAlignment feature, we found there will be a duplicated

[jira] [Updated] (FLINK-32316) Duplicated announceCombinedWatermark task maybe scheduled if jobmanager failover

[ https://issues.apache.org/jira/browse/FLINK-32316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-32316: Description: When we try SourceAlignment feature, we found there will be a duplicated

[jira] [Updated] (FLINK-32316) Duplicated announceCombinedWatermark task maybe scheduled if jobmanager failover

[ https://issues.apache.org/jira/browse/FLINK-32316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-32316: Description: When we try SourceAlignment feature, we found there will be a duplicated

[jira] [Commented] (FLINK-32316) Duplicated announceCombinedWatermark task maybe scheduled if jobmanager failover

[ https://issues.apache.org/jira/browse/FLINK-32316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17731608#comment-17731608 ] Cai Liuyang commented on FLINK-32316: - [~pnowojski]  please take a look? thks~ > Duplicated

[jira] [Created] (FLINK-32316) Duplicated announceCombinedWatermark task maybe scheduled if jobmanager failover

Cai Liuyang created FLINK-32316: --- Summary: Duplicated announceCombinedWatermark task maybe scheduled if jobmanager failover Key: FLINK-32316 URL: https://issues.apache.org/jira/browse/FLINK-32316

[jira] [Closed] (FLINK-29577) Disable rocksdb wal when restore from full snapshot

[ https://issues.apache.org/jira/browse/FLINK-29577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang closed FLINK-29577. --- Resolution: Fixed > Disable rocksdb wal when restore from full snapshot >

[jira] [Comment Edited] (FLINK-29577) Disable rocksdb wal when restore from full snapshot

[ https://issues.apache.org/jira/browse/FLINK-29577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620889#comment-17620889 ] Cai Liuyang edited comment on FLINK-29577 at 10/20/22 8:11 AM: --- [~ym] 

[jira] [Commented] (FLINK-29577) Disable rocksdb wal when restore from full snapshot

[ https://issues.apache.org/jira/browse/FLINK-29577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620889#comment-17620889 ] Cai Liuyang commented on FLINK-29577: - [~ym]  This is my test case: 1. disable rocksdb managed

[jira] [Updated] (FLINK-29577) Disable rocksdb wal when restore from full snapshot

[ https://issues.apache.org/jira/browse/FLINK-29577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-29577: Attachment: image-2022-10-20-16-08-15-746.png > Disable rocksdb wal when restore from full

[jira] [Updated] (FLINK-29577) Disable rocksdb wal when restore from full snapshot

[ https://issues.apache.org/jira/browse/FLINK-29577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-29577: Attachment: (was: image-2022-10-12-20-15-11-219.png) > Disable rocksdb wal when restore from

[jira] [Updated] (FLINK-29577) Disable rocksdb wal when restore from full snapshot

[ https://issues.apache.org/jira/browse/FLINK-29577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-29577: Attachment: image-2022-10-12-20-15-11-219.png > Disable rocksdb wal when restore from full

[jira] [Updated] (FLINK-29577) Disable rocksdb wal when restore from full snapshot

[ https://issues.apache.org/jira/browse/FLINK-29577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-29577: Description: For now, RocksDBFullRestoreOperation and RocksDBHeapTimersFullRestoreOperation

[jira] [Commented] (FLINK-29577) Disable rocksdb wal when restore from full snapshot

[ https://issues.apache.org/jira/browse/FLINK-29577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615862#comment-17615862 ] Cai Liuyang commented on FLINK-29577: - [~yunta] Sorry, i only observe disk usage doubled during

[jira] [Created] (FLINK-29577) Disable rocksdb wal when restore from full snapshot

Cai Liuyang created FLINK-29577: --- Summary: Disable rocksdb wal when restore from full snapshot Key: FLINK-29577 URL: https://issues.apache.org/jira/browse/FLINK-29577 Project: Flink Issue

[jira] [Commented] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17494359#comment-17494359 ] Cai Liuyang commented on FLINK-26080: - Yeah, looks like the same problem, thks [~pnowojski] , i'll

[jira] [Closed] (FLINK-25664) Notify will be not triggered for PipelinedSubpartition if more than one buffer is added during isBlocked == true

[ https://issues.apache.org/jira/browse/FLINK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang closed FLINK-25664. --- Resolution: Abandoned > Notify will be not triggered for PipelinedSubpartition if more than one >

[jira] [Commented] (FLINK-25664) Notify will be not triggered for PipelinedSubpartition if more than one buffer is added during isBlocked == true

[ https://issues.apache.org/jira/browse/FLINK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490882#comment-17490882 ] Cai Liuyang commented on FLINK-25664: - yeah, got it. I forgot this logic before, thanks ~ > Notify

[jira] [Comment Edited] (FLINK-25664) Notify will be not triggered for PipelinedSubpartition if more than one buffer is added during isBlocked == true

[ https://issues.apache.org/jira/browse/FLINK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490882#comment-17490882 ] Cai Liuyang edited comment on FLINK-25664 at 2/11/22, 12:20 PM: yeah,

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:     upstreamTask is

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:     upstreamTask is

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:     upstreamTask is

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:     upstreamTask is

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:     upstreamTask is

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:     upstreamTask is

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:     upstreamTask is

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:     upstreamTask is

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:     upstreamTask is

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Description: In out production environment, we encounter one abnormal case:   upstreamTask is

[jira] [Created] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

Cai Liuyang created FLINK-26080: --- Summary: PartitionRequest client use Netty's IdleStateHandler to monitor channel's status Key: FLINK-26080 URL: https://issues.apache.org/jira/browse/FLINK-26080

[jira] [Updated] (FLINK-26080) PartitionRequest client use Netty's IdleStateHandler to monitor channel's status

[ https://issues.apache.org/jira/browse/FLINK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-26080: Affects Version/s: 1.14.3 > PartitionRequest client use Netty's IdleStateHandler to monitor

[jira] [Updated] (FLINK-25664) Notify will be not triggered for PipelinedSubpartition if more than one buffer is added during isBlocked == true

[ https://issues.apache.org/jira/browse/FLINK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-25664: Component/s: Runtime / Network > Notify will be not triggered for PipelinedSubpartition if more

[jira] [Updated] (FLINK-25664) Notify will be not triggered for PipelinedSubpartition if more than one buffer is added during isBlocked == true

[ https://issues.apache.org/jira/browse/FLINK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-25664: Affects Version/s: 1.14.3 > Notify will be not triggered for PipelinedSubpartition if more than

[jira] [Updated] (FLINK-25664) Notify will be not triggered for PipelinedSubpartition if more than one buffer is added during isBlocked == true

[ https://issues.apache.org/jira/browse/FLINK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-25664: Description: For now, there might be case like: # PipelinedSubPartition only have one

[jira] [Created] (FLINK-25664) Notify will be not triggered for PipelinedSubpartition if more than one buffer is added during isBlocked == true

Cai Liuyang created FLINK-25664: --- Summary: Notify will be not triggered for PipelinedSubpartition if more than one buffer is added during isBlocked == true Key: FLINK-25664 URL: