[jira] [Commented] (FLINK-31655) Adaptive Channel selection for partitioner

2023-04-13 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711755#comment-17711755 ] Gen Luo commented on FLINK-31655: - Hi [~akalash], thanks for the remind. I agree that the impaction to

[jira] [Commented] (FLINK-30387) support job singleton within TM

2023-02-09 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17686335#comment-17686335 ] Gen Luo commented on FLINK-30387: - As you said we can use the flink user classloader, there can be a

[jira] [Commented] (FLINK-30943) Hide sensitive command-line configurations

2023-02-09 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17686320#comment-17686320 ] Gen Luo commented on FLINK-30943: - Hiding sensitive configurations is already supported if the key and

[jira] [Commented] (FLINK-30719) flink-runtime-web failed due to a corrupted

2023-02-02 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683265#comment-17683265 ] Gen Luo commented on FLINK-30719: - Seems that the corrupting file has been deleted and the issue just

[jira] [Commented] (FLINK-30448) "filter(Objects::nonNull)" will bring down task with failure cause: ClassCastException

2023-01-05 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654893#comment-17654893 ] Gen Luo commented on FLINK-30448: - Hi Yong, I looked into the issue and found that this may be a

[jira] [Commented] (FLINK-30244) When task using udf/udtf with jni, on k8s session the old TM will shut down and create new TM or the task will fail

2022-12-15 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17647953#comment-17647953 ] Gen Luo commented on FLINK-30244: - [~AlexXXX] Hi, do you have any question about this? If so you may

[jira] [Commented] (FLINK-30193) could flink has dynamic rebalance according to the load of downstream operators

2022-11-24 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17638336#comment-17638336 ] Gen Luo commented on FLINK-30193: - HI [~landlord], We did some work like this, which makes rebalance

[jira] [Updated] (FLINK-29927) AkkaUtils#getAddress may cause memory leak

2022-11-08 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-29927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gen Luo updated FLINK-29927: Attachment: RemoteAddressExtensionLeaking.png > AkkaUtils#getAddress may cause memory leak >

[jira] [Updated] (FLINK-29927) AkkaUtils#getAddress may cause memory leak

2022-11-08 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-29927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gen Luo updated FLINK-29927: Attachment: (was: 截屏2022-11-08 下午5.19.38.png) > AkkaUtils#getAddress may cause memory leak >

[jira] [Created] (FLINK-29927) AkkaUtils#getAddress may cause memory leak

2022-11-08 Thread Gen Luo (Jira)
Gen Luo created FLINK-29927: --- Summary: AkkaUtils#getAddress may cause memory leak Key: FLINK-29927 URL: https://issues.apache.org/jira/browse/FLINK-29927 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-29132) SubtaskMetricStore causes memory leak.

2022-08-31 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-29132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17598314#comment-17598314 ] Gen Luo commented on FLINK-29132: - [~nagist] Thanks a lot for reporting the issue! And thanks

[jira] [Commented] (FLINK-26394) CheckpointCoordinator.isTriggering can not be reset if a checkpoint expires while the checkpointCoordinator task is queuing in the SourceCoordinator executor.

2022-08-26 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585199#comment-17585199 ] Gen Luo commented on FLINK-26394: - [~gezhiwei8899] , It's expected because, though the fix is merged to

[jira] [Commented] (FLINK-26394) CheckpointCoordinator.isTriggering can not be reset if a checkpoint expires while the checkpointCoordinator task is queuing in the SourceCoordinator executor.

2022-08-26 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585177#comment-17585177 ] Gen Luo commented on FLINK-26394: - Hi [~gezhiwei8899], The issue should be existing in any version that

[jira] [Created] (FLINK-28864) DynamicPartitionPruningRule#isNewSource should check if the source used by the DataStreamScanProvider is actually a new sourc

2022-08-08 Thread Gen Luo (Jira)
Gen Luo created FLINK-28864: --- Summary: DynamicPartitionPruningRule#isNewSource should check if the source used by the DataStreamScanProvider is actually a new sourc Key: FLINK-28864 URL:

[jira] [Created] (FLINK-28589) Enhance Web UI for Speculative Execution

2022-07-18 Thread Gen Luo (Jira)
Gen Luo created FLINK-28589: --- Summary: Enhance Web UI for Speculative Execution Key: FLINK-28589 URL: https://issues.apache.org/jira/browse/FLINK-28589 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-28588) Enhance REST API for Speculative Execution

2022-07-18 Thread Gen Luo (Jira)
Gen Luo created FLINK-28588: --- Summary: Enhance REST API for Speculative Execution Key: FLINK-28588 URL: https://issues.apache.org/jira/browse/FLINK-28588 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-28587) FLIP-249: Flink Web UI Enhancement for Speculative Execution

2022-07-18 Thread Gen Luo (Jira)
Gen Luo created FLINK-28587: --- Summary: FLIP-249: Flink Web UI Enhancement for Speculative Execution Key: FLINK-28587 URL: https://issues.apache.org/jira/browse/FLINK-28587 Project: Flink Issue

[jira] [Created] (FLINK-28240) NettyShuffleMetricFactory#RequestedMemoryUsageMetric#getValue may throw ArithmeticException when the total segments of NetworkBufferPool is 0

2022-06-24 Thread Gen Luo (Jira)
Gen Luo created FLINK-28240: --- Summary: NettyShuffleMetricFactory#RequestedMemoryUsageMetric#getValue may throw ArithmeticException when the total segments of NetworkBufferPool is 0 Key: FLINK-28240 URL:

[jira] [Commented] (FLINK-27910) FileSink not registered the timer to enforce rolling policy if started from scratch

2022-06-07 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-27910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550823#comment-17550823 ] Gen Luo commented on FLINK-27910: - This is a mistake while migrating the FileSink to the new sink API.

[jira] [Commented] (FLINK-27148) UnalignedCheckpointITCase fails on AZP

2022-04-13 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521505#comment-17521505 ] Gen Luo commented on FLINK-27148: - I'm working on the issue now. The solution seems to be simple, but

[jira] [Commented] (FLINK-26394) CheckpointCoordinator.isTriggering can not be reset if a checkpoint expires while the checkpointCoordinator task is queuing in the SourceCoordinator executor.

2022-03-14 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506047#comment-17506047 ] Gen Luo commented on FLINK-26394: - I agree with you, [~becket_qin]. IMO CheckpointCoordinator should

[jira] [Commented] (FLINK-26322) Test FileSink compaction manually

2022-03-14 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506013#comment-17506013 ] Gen Luo commented on FLINK-26322: - Now all issues known are resolved, and the patches are merged. I

[jira] [Created] (FLINK-26610) FileSink can not upgrade from 1.13 if the uid of the origin sink is not set.

2022-03-11 Thread Gen Luo (Jira)
Gen Luo created FLINK-26610: --- Summary: FileSink can not upgrade from 1.13 if the uid of the origin sink is not set. Key: FLINK-26610 URL: https://issues.apache.org/jira/browse/FLINK-26610 Project: Flink

[jira] [Commented] (FLINK-26322) Test FileSink compaction manually

2022-03-10 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17504708#comment-17504708 ] Gen Luo commented on FLINK-26322: - I found issues that may relate to 4. The jira is FLINK-26580 and a PR

[jira] [Updated] (FLINK-26580) FileSink Compactor is not properly processing in-progress files.

2022-03-10 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gen Luo updated FLINK-26580: Summary: FileSink Compactor is not properly processing in-progress files. (was: FileSink Compactor state

[jira] [Updated] (FLINK-26580) FileSink Compactor state handlers are not properly processing remaining in-progress files.

2022-03-10 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gen Luo updated FLINK-26580: Summary: FileSink Compactor state handlers are not properly processing remaining in-progress files.

[jira] [Updated] (FLINK-26580) FileSink Compactor is not properly processing in-progress files.

2022-03-10 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gen Luo updated FLINK-26580: Summary: FileSink Compactor is not properly processing in-progress files. (was: FileSink

[jira] [Created] (FLINK-26580) FileSink CompactCoordinator add illegal committable as toCompacted.

2022-03-10 Thread Gen Luo (Jira)
Gen Luo created FLINK-26580: --- Summary: FileSink CompactCoordinator add illegal committable as toCompacted. Key: FLINK-26580 URL: https://issues.apache.org/jira/browse/FLINK-26580 Project: Flink

[jira] [Created] (FLINK-26564) CompactCoordinatorStateHandler doesn't properly handle the cleanup-in-progress requests.

2022-03-09 Thread Gen Luo (Jira)
Gen Luo created FLINK-26564: --- Summary: CompactCoordinatorStateHandler doesn't properly handle the cleanup-in-progress requests. Key: FLINK-26564 URL: https://issues.apache.org/jira/browse/FLINK-26564

[jira] [Commented] (FLINK-26322) Test FileSink compaction manually

2022-03-09 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503975#comment-17503975 ] Gen Luo commented on FLINK-26322: - Thanks for the testing, [~alexanderpreuss]! 3 has been addressed.

[jira] [Created] (FLINK-26440) CompactorOperatorStateHandler can not work with unaligned checkpoint

2022-03-01 Thread Gen Luo (Jira)
Gen Luo created FLINK-26440: --- Summary: CompactorOperatorStateHandler can not work with unaligned checkpoint Key: FLINK-26440 URL: https://issues.apache.org/jira/browse/FLINK-26440 Project: Flink

[jira] [Commented] (FLINK-26394) CheckpointCoordinator.isTriggering can not be reset if a checkpoint expires while the checkpointCoordinator task is queuing in the SourceCoordinator executor.

2022-03-01 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17499474#comment-17499474 ] Gen Luo commented on FLINK-26394: - [~yunta] The problem can be reproduced by: # add a 10s sleeping (or

[jira] [Comment Edited] (FLINK-26394) CheckpointCoordinator.isTriggering can not be reset if a checkpoint expires while the checkpointCoordinator task is queuing in the SourceCoordinator executor.

2022-03-01 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17499474#comment-17499474 ] Gen Luo edited comment on FLINK-26394 at 3/1/22, 11:29 AM: --- [~yunta] The

[jira] [Created] (FLINK-26394) CheckpointCoordinator.isTriggering can not be reset if a checkpoint expires while the checkpointCoordinator task is queuing in the SourceCoordinator executor.

2022-02-28 Thread Gen Luo (Jira)
Gen Luo created FLINK-26394: --- Summary: CheckpointCoordinator.isTriggering can not be reset if a checkpoint expires while the checkpointCoordinator task is queuing in the SourceCoordinator executor. Key: FLINK-26394

[jira] [Commented] (FLINK-26314) StreamingCompactingFileSinkITCase.testFileSink failed on azure

2022-02-24 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497887#comment-17497887 ] Gen Luo commented on FLINK-26314: - I found that the PseudoRandomValueSelector is selecting PT0S for

[jira] [Commented] (FLINK-26314) StreamingCompactingFileSinkITCase.testFileSink failed on azure

2022-02-23 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497225#comment-17497225 ] Gen Luo commented on FLINK-26314: - I'll take a look soon. >

[jira] [Commented] (FLINK-26233) FileSinkCompactionSwitchITCase.testSwitchingCompaction() fails in CI

2022-02-18 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17494456#comment-17494456 ] Gen Luo commented on FLINK-26233: - This is because when the compaction is switch from on to off, the

[jira] [Updated] (FLINK-26235) CompactingFileWriter and PendingFileRecoverable should not be exposed to users.

2022-02-17 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gen Luo updated FLINK-26235: Description: In FLINK-25583, we added a FileCompactingWriter to write the compacting file. The writer is

[jira] [Created] (FLINK-26235) CompactingFileWriter and PendingFileRecoverable should not be exposed to users.

2022-02-17 Thread Gen Luo (Jira)
Gen Luo created FLINK-26235: --- Summary: CompactingFileWriter and PendingFileRecoverable should not be exposed to users. Key: FLINK-26235 URL: https://issues.apache.org/jira/browse/FLINK-26235 Project: Flink

[jira] [Created] (FLINK-26180) Update docs to introduce the compaction for FileSink

2022-02-16 Thread Gen Luo (Jira)
Gen Luo created FLINK-26180: --- Summary: Update docs to introduce the compaction for FileSink Key: FLINK-26180 URL: https://issues.apache.org/jira/browse/FLINK-26180 Project: Flink Issue Type:

[jira] [Commented] (FLINK-25937) SQL Client end-to-end test e2e fails on AZP

2022-02-09 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489997#comment-17489997 ] Gen Luo commented on FLINK-25937: - The reason has been identified.  Parallelism of a transformation

[jira] [Created] (FLINK-24965) Improper usage of Map.Entry after Entry Iterator.remove in TaskLocaStateStoreImpl#pruneCheckpoints

2021-11-19 Thread Gen Luo (Jira)
Gen Luo created FLINK-24965: --- Summary: Improper usage of Map.Entry after Entry Iterator.remove in TaskLocaStateStoreImpl#pruneCheckpoints Key: FLINK-24965 URL: https://issues.apache.org/jira/browse/FLINK-24965

[jira] [Updated] (FLINK-23216) RM keeps allocating and freeing slots after a TM lost until its heartbeat timeout

2021-07-05 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gen Luo updated FLINK-23216: Description: In Flink 1.13, it's observed that the ResourceManager keeps allocating and freeing slots

[jira] [Commented] (FLINK-23216) RM keeps allocating and freeing slots after a TM lost until its heartbeat timeout

2021-07-05 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17374650#comment-17374650 ] Gen Luo commented on FLINK-23216: - I agree the improvement should be done regardless of FLINK-23209.

[jira] [Created] (FLINK-23216) RM keeps allocating and freeing slots after a TM lost until its heartbeat timeout

2021-07-02 Thread Gen Luo (Jira)
Gen Luo created FLINK-23216: --- Summary: RM keeps allocating and freeing slots after a TM lost until its heartbeat timeout Key: FLINK-23216 URL: https://issues.apache.org/jira/browse/FLINK-23216 Project:

[jira] [Commented] (FLINK-22945) StackOverflowException can happen when a large scale job is CANCELING/FAILING

2021-06-28 Thread Gen Luo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370454#comment-17370454 ] Gen Luo commented on FLINK-22945: - Hi [~zhuzh], I'd like to work on this, would you please assign the