[jira] [Created] (FLINK-33804) Add Option to disable showing metrics in JobMananger UI

2023-12-12 Thread Lu Niu (Jira)
Lu Niu created FLINK-33804: -- Summary: Add Option to disable showing metrics in JobMananger UI Key: FLINK-33804 URL: https://issues.apache.org/jira/browse/FLINK-33804 Project: Flink Issue Type: Impro

[jira] [Commented] (FLINK-33804) Add Option to disable showing metrics in JobMananger UI

2023-12-12 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17795945#comment-17795945 ] Lu Niu commented on FLINK-33804: [~martijnvisser] disabling the metric aggregation to Jo

[jira] [Created] (FLINK-33806) Async IO Allows Custom Action after Final Retry Failure

2023-12-12 Thread Lu Niu (Jira)
Lu Niu created FLINK-33806: -- Summary: Async IO Allows Custom Action after Final Retry Failure Key: FLINK-33806 URL: https://issues.apache.org/jira/browse/FLINK-33806 Project: Flink Issue Type: Impr

[jira] [Comment Edited] (FLINK-33804) Add Option to disable showing metrics in JobMananger UI

2023-12-13 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17796455#comment-17796455 ] Lu Niu edited comment on FLINK-33804 at 12/13/23 8:36 PM: -- [~ma

[jira] [Commented] (FLINK-33804) Add Option to disable showing metrics in JobMananger UI

2023-12-13 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17796455#comment-17796455 ] Lu Niu commented on FLINK-33804: The problem we want to address: In default setting, th

[jira] [Commented] (FLINK-33804) Add Option to disable showing metrics in JobMananger UI

2023-12-13 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17796474#comment-17796474 ] Lu Niu commented on FLINK-33804: the whitelist will be a subset of metrics listed here

[jira] [Created] (FLINK-19985) job went into zombie state after ZK session timeout

2020-11-04 Thread Lu Niu (Jira)
Lu Niu created FLINK-19985: -- Summary: job went into zombie state after ZK session timeout Key: FLINK-19985 URL: https://issues.apache.org/jira/browse/FLINK-19985 Project: Flink Issue Type: Bug A

[jira] [Updated] (FLINK-19985) job went into zombie state after ZK session timeout

2020-11-04 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Niu updated FLINK-19985: --- Description: Recently we had an issue that one flink job went into zombie state after ZK session timeout. D

[jira] [Created] (FLINK-16931) Large _metadata file lead to JobManager not responding when restart

2020-04-01 Thread Lu Niu (Jira)
Lu Niu created FLINK-16931: -- Summary: Large _metadata file lead to JobManager not responding when restart Key: FLINK-16931 URL: https://issues.apache.org/jira/browse/FLINK-16931 Project: Flink Issu

[jira] [Updated] (FLINK-16931) Large _metadata file lead to JobManager not responding when restart

2020-04-01 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Niu updated FLINK-16931: --- Description: When _metadata file is big, JobManager could never recover from checkpoint. It fall into a loo

[jira] [Updated] (FLINK-16931) Large _metadata file lead to JobManager not responding when restart

2020-04-01 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Niu updated FLINK-16931: --- Description: When _metadata file is big, JobManager could never recover from checkpoint. It fall into a loo

[jira] [Commented] (FLINK-16931) Large _metadata file lead to JobManager not responding when restart

2020-04-02 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073879#comment-17073879 ] Lu Niu commented on FLINK-16931: Hi, [~trohrmann] and [~pnowojski] Could you assign thi

[jira] [Commented] (FLINK-16931) Large _metadata file lead to JobManager not responding when restart

2020-04-03 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075023#comment-17075023 ] Lu Niu commented on FLINK-16931: Thanks in advance! Will share with the plan later.  >

[jira] [Created] (FLINK-21263) Job hangs under backpressure

2021-02-03 Thread Lu Niu (Jira)
Lu Niu created FLINK-21263: -- Summary: Job hangs under backpressure Key: FLINK-21263 URL: https://issues.apache.org/jira/browse/FLINK-21263 Project: Flink Issue Type: Bug Affects Versions: 1.11.0

[jira] [Commented] (FLINK-17089) Checkpoint fail because RocksDBException: Error While opening a file for sequentially reading

2020-05-21 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113355#comment-17113355 ] Lu Niu commented on FLINK-17089: [~sewen] thanks for replying.  # The root cause is sth

[jira] [Created] (FLINK-17364) Support StreamingFileSink in PrestoS3FileSystem

2020-04-23 Thread Lu Niu (Jira)
Lu Niu created FLINK-17364: -- Summary: Support StreamingFileSink in PrestoS3FileSystem Key: FLINK-17364 URL: https://issues.apache.org/jira/browse/FLINK-17364 Project: Flink Issue Type: Improvement

[jira] [Commented] (FLINK-17364) Support StreamingFileSink in PrestoS3FileSystem

2020-04-24 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091726#comment-17091726 ] Lu Niu commented on FLINK-17364: Hi, [~kkl0u] . Thanks for replying!  PrestoS3FileSystem

[jira] [Commented] (FLINK-17364) Support StreamingFileSink in PrestoS3FileSystem

2020-04-28 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094714#comment-17094714 ] Lu Niu commented on FLINK-17364: [~kkl0u] Are you OK with the approach? thanks > Suppor

[jira] [Commented] (FLINK-17364) Support StreamingFileSink in PrestoS3FileSystem

2020-04-29 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17095752#comment-17095752 ] Lu Niu commented on FLINK-17364: Sure. will create a PR for this. thanks! > Support Str

[jira] [Commented] (FLINK-17468) Provide more detailed metrics why asynchronous part of checkpoint is taking long time

2020-04-29 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17095773#comment-17095773 ] Lu Niu commented on FLINK-17468: +1. I think it will be very helpful to debug checkpoint

[jira] [Commented] (FLINK-17468) Provide more detailed metrics why asynchronous part of checkpoint is taking long time

2020-05-08 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102742#comment-17102742 ] Lu Niu commented on FLINK-17468: [~klion26] [~yunta] Thanks for reviewing! I think addi

[jira] [Created] (FLINK-17089) Checkpoint fail because RocksDBException: Error While opening a file for sequentially reading

2020-04-10 Thread Lu Niu (Jira)
Lu Niu created FLINK-17089: -- Summary: Checkpoint fail because RocksDBException: Error While opening a file for sequentially reading Key: FLINK-17089 URL: https://issues.apache.org/jira/browse/FLINK-17089 Pro

[jira] [Commented] (FLINK-17089) Checkpoint fail because RocksDBException: Error While opening a file for sequentially reading

2020-04-10 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081039#comment-17081039 ] Lu Niu commented on FLINK-17089: The root cause is actually user brings in rocksdb 6.1.0

[jira] [Commented] (FLINK-16931) Large _metadata file lead to JobManager not responding when restart

2020-04-15 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084463#comment-17084463 ] Lu Niu commented on FLINK-16931: [~SleePy] [~pnowojski] Could you help review the design

[jira] [Commented] (FLINK-16931) Large _metadata file lead to JobManager not responding when restart

2020-04-19 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087284#comment-17087284 ] Lu Niu commented on FLINK-16931: Thanks! No hurries > Large _metadata file lead to J

[jira] [Commented] (FLINK-16931) Large _metadata file lead to JobManager not responding when restart

2020-04-20 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088282#comment-17088282 ] Lu Niu commented on FLINK-16931: [~trohrmann] Thanks for the advice. Certainly there are

[jira] [Commented] (FLINK-17627) Add support for writing _SUCCESS file with StreamingFileSink

2020-07-15 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17158704#comment-17158704 ] Lu Niu commented on FLINK-17627: [~lzljs3620320] We also want StreamingFileSink write _S

[jira] [Commented] (FLINK-17627) Add support for writing _SUCCESS file with StreamingFileSink

2020-07-16 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159544#comment-17159544 ] Lu Niu commented on FLINK-17627: Hi, [~lzljs3620320]. thanks for replying. I took a loo

[jira] [Commented] (FLINK-17627) Add support for writing _SUCCESS file with StreamingFileSink

2020-07-16 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159628#comment-17159628 ] Lu Niu commented on FLINK-17627: hmm... It seems not because I don't have DataStream to

[jira] [Commented] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-06-10 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361080#comment-17361080 ] Lu Niu commented on FLINK-22326: [~dwysakowicz] thanks for reply.  I understand that cur

[jira] [Created] (FLINK-22162) Make Max Operator name Length Configurable

2021-04-08 Thread Lu Niu (Jira)
Lu Niu created FLINK-22162: -- Summary: Make Max Operator name Length Configurable Key: FLINK-22162 URL: https://issues.apache.org/jira/browse/FLINK-22162 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-22164) Add jobId and JobName variable to JobManager metrics in per-job deployment mode

2021-04-08 Thread Lu Niu (Jira)
Lu Niu created FLINK-22164: -- Summary: Add jobId and JobName variable to JobManager metrics in per-job deployment mode Key: FLINK-22164 URL: https://issues.apache.org/jira/browse/FLINK-22164 Project: Flink

[jira] [Commented] (FLINK-22164) Add jobId and JobName variable to JobManager metrics in per-job deployment mode

2021-05-13 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344078#comment-17344078 ] Lu Niu commented on FLINK-22164: [~trohrmann] This is the POC code for adding this featu

[jira] [Commented] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-05-19 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17347886#comment-17347886 ] Lu Niu commented on FLINK-22326: [~trohrmann] I have a fix for this issue But need some

[jira] [Commented] (FLINK-13483) PrestoS3FileSystemITCase.testDirectoryListing fails on Travis

2020-03-17 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-13483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17061334#comment-17061334 ] Lu Niu commented on FLINK-13483: Could you assign it to me if no one's working on it? I

[jira] [Commented] (FLINK-16555) Preflight check for known unstable hashCodes.

2020-03-17 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17061342#comment-17061342 ] Lu Niu commented on FLINK-16555: Is someone working on this? If not, could you assign it

[jira] [Created] (FLINK-16640) Expose listStatus latency in flink filesystem

2020-03-17 Thread Lu Niu (Jira)
Lu Niu created FLINK-16640: -- Summary: Expose listStatus latency in flink filesystem Key: FLINK-16640 URL: https://issues.apache.org/jira/browse/FLINK-16640 Project: Flink Issue Type: Improvement

[jira] [Commented] (FLINK-16555) Preflight check for known unstable hashCodes.

2020-03-19 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062853#comment-17062853 ] Lu Niu commented on FLINK-16555: [~sewen] would add more checking in  [https://github.co

[jira] [Commented] (FLINK-13483) PrestoS3FileSystemITCase.testDirectoryListing fails on Travis

2020-03-19 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-13483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062855#comment-17062855 ] Lu Niu commented on FLINK-13483: [~liyu] thanks! Unfortunately I cannot reproduce the fa

[jira] [Commented] (FLINK-13483) PrestoS3FileSystemITCase.testDirectoryListing fails on Travis

2020-03-25 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-13483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067259#comment-17067259 ] Lu Niu commented on FLINK-13483: [~liyu] thanks for the reply. I find the problem might

[jira] [Created] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-16 Thread Lu Niu (Jira)
Lu Niu created FLINK-22326: -- Summary: Job contains Iterate Operator always fails on Checkpoint Key: FLINK-22326 URL: https://issues.apache.org/jira/browse/FLINK-22326 Project: Flink Issue Type: Imp

[jira] [Updated] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-16 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Niu updated FLINK-22326: --- Attachment: Screen Shot 2021-04-16 at 12.43.38 PM.png > Job contains Iterate Operator always fails on Checkp

[jira] [Updated] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-16 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Niu updated FLINK-22326: --- Attachment: Screen Shot 2021-04-16 at 12.40.34 PM.png > Job contains Iterate Operator always fails on Checkp

[jira] [Updated] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-16 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Niu updated FLINK-22326: --- Description: Job contains Iterate Operator will always fail on checkpoint. How to reproduce:  [https://gis

[jira] [Updated] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-16 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Niu updated FLINK-22326: --- Description: Job contains Iterate Operator will always fail on checkpoint. How to reproduce:  [https://gis

[jira] [Updated] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-16 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Niu updated FLINK-22326: --- Description: Job contains Iterate Operator will always fail on checkpoint. How to reproduce:  [https://gis

[jira] [Commented] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-16 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17324045#comment-17324045 ] Lu Niu commented on FLINK-22326: This is a bit urgent on our side. Please let me know ho

[jira] [Commented] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-20 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17326168#comment-17326168 ] Lu Niu commented on FLINK-22326: Hi, [~Thesharing] we confirmed this is a bug in current

[jira] [Comment Edited] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-20 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17326168#comment-17326168 ] Lu Niu edited comment on FLINK-22326 at 4/21/21, 12:13 AM: --- Hi

[jira] [Updated] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-20 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Niu updated FLINK-22326: --- Issue Type: Bug (was: Improvement) > Job contains Iterate Operator always fails on Checkpoint > --

[jira] [Comment Edited] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-20 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17326168#comment-17326168 ] Lu Niu edited comment on FLINK-22326 at 4/21/21, 12:14 AM: --- Hi

[jira] [Commented] (FLINK-22164) Add jobId and JobName variable to JobManager metrics in per-job deployment mode

2021-04-20 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17326171#comment-17326171 ] Lu Niu commented on FLINK-22164: Thanks for replying. We are actually more cirous about

[jira] [Commented] (FLINK-22164) Add jobId and JobName variable to JobManager metrics in per-job deployment mode

2021-04-21 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17327026#comment-17327026 ] Lu Niu commented on FLINK-22164: [~trohrmann] yes. that's why I suggest these optional t

[jira] [Commented] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-04-21 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17327031#comment-17327031 ] Lu Niu commented on FLINK-22326: [~trohrmann] Here is a short summary: before 1.9: job c

[jira] [Commented] (FLINK-22326) Job contains Iterate Operator always fails on Checkpoint

2021-06-03 Thread Lu Niu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17356983#comment-17356983 ] Lu Niu commented on FLINK-22326: trying to figure out unit test by myself.  > Job conta