[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-02-12 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035182#comment-17035182 ] Yuan Mei edited comment on FLINK-15670 at 2/12/20 9:17 AM: --- I have spent a

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-02-12 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035182#comment-17035182 ] Yuan Mei commented on FLINK-15670: -- I have spent a couple of days hacking a prototype using existing

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-02-12 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035182#comment-17035182 ] Yuan Mei edited comment on FLINK-15670 at 2/12/20 9:24 AM: ---   I have spent a

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-02-12 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035182#comment-17035182 ] Yuan Mei edited comment on FLINK-15670 at 2/12/20 9:22 AM: --- {code:java} //

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-03 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050031#comment-17050031 ] Yuan Mei edited comment on FLINK-15670 at 3/4/20, 6:31 AM: --- [~sewen] Need to

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-03 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050031#comment-17050031 ] Yuan Mei edited comment on FLINK-15670 at 3/4/20, 6:37 AM: --- [~sewen] Need to

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-03 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050031#comment-17050031 ] Yuan Mei edited comment on FLINK-15670 at 3/4/20, 6:41 AM: --- [~sewen] Need to

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-04 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050031#comment-17050031 ] Yuan Mei edited comment on FLINK-15670 at 3/4/20, 9:10 AM: --- [~sewen] Need to

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-04 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050031#comment-17050031 ] Yuan Mei edited comment on FLINK-15670 at 3/4/20, 9:11 AM: --- [~sewen] Need to

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-02-28 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047415#comment-17047415 ] Yuan Mei commented on FLINK-15670: -- Thanks Stephan! This is definitely a cleaner interface. The reason

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-06 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053232#comment-17053232 ] Yuan Mei edited comment on FLINK-15670 at 3/6/20, 10:19 AM: *Meeting

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-06 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053232#comment-17053232 ] Yuan Mei commented on FLINK-15670: -- *Meeting Outline* Sketch the meeting agenda here to make the

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-06 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053232#comment-17053232 ] Yuan Mei edited comment on FLINK-15670 at 3/6/20, 10:50 AM: *Meeting

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-06 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053232#comment-17053232 ] Yuan Mei edited comment on FLINK-15670 at 3/6/20, 10:50 AM: *Meeting

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-03 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050031#comment-17050031 ] Yuan Mei commented on FLINK-15670: -- [~sewen] Need to chat a bit for two things: # Redefine the scope

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-02-20 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040874#comment-17040874 ] Yuan Mei commented on FLINK-15670: -- Some updates: I wrapped the shuffle producer and consumer into

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-02-20 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040874#comment-17040874 ] Yuan Mei edited comment on FLINK-15670 at 2/20/20 11:56 AM: Some updates: I

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011657#comment-17011657 ] Yuan Mei commented on FLINK-14163: -- [~zhuzh] Thank you sooo much. I prefer option 3 better as well,

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011657#comment-17011657 ] Yuan Mei edited comment on FLINK-14163 at 1/9/20 10:16 AM: --- [~zhuzh] Thank you

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011568#comment-17011568 ] Yuan Mei commented on FLINK-14163: -- Thanks for assigning the task to me!   I have written a first

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011568#comment-17011568 ] Yuan Mei edited comment on FLINK-14163 at 1/9/20 9:34 AM: -- Thanks for assigning

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011568#comment-17011568 ] Yuan Mei edited comment on FLINK-14163 at 1/9/20 9:06 AM: -- Thanks for assigning

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011568#comment-17011568 ] Yuan Mei edited comment on FLINK-14163 at 1/9/20 9:06 AM: -- Thanks for assigning

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011568#comment-17011568 ] Yuan Mei edited comment on FLINK-14163 at 1/9/20 9:46 AM: -- Thanks for assigning

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-10 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013353#comment-17013353 ] Yuan Mei commented on FLINK-14163: -- PR: pull request #10832 > Execution#producedPartitions is possibly

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-13 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014147#comment-17014147 ] Yuan Mei edited comment on FLINK-14163 at 1/13/20 11:04 AM:   Synced up

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-13 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014147#comment-17014147 ] Yuan Mei edited comment on FLINK-14163 at 1/13/20 9:59 AM: ---   Synced up with

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-13 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014147#comment-17014147 ] Yuan Mei edited comment on FLINK-14163 at 1/13/20 9:20 AM: ---   Synced up with

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-13 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014147#comment-17014147 ] Yuan Mei commented on FLINK-14163: --   Synced up with [~zjwang] offline.    In general, I would

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-13 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014147#comment-17014147 ] Yuan Mei edited comment on FLINK-14163 at 1/13/20 9:58 AM: ---   Synced up with

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-01-19 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019222#comment-17019222 ] Yuan Mei commented on FLINK-15670: -- I like this idea. If understanding correctly, we provide

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-01-19 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019222#comment-17019222 ] Yuan Mei edited comment on FLINK-15670 at 1/20/20 6:15 AM: --- I like this idea.

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-13 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014886#comment-17014886 ] Yuan Mei commented on FLINK-14163: --   Thanks [~azagrebin] and [~zhuzh]! I do not have a strong opinion

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-10 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012507#comment-17012507 ] Yuan Mei edited comment on FLINK-14163 at 1/10/20 8:10 AM: --- Hey, [~chesnay], 

[jira] [Comment Edited] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-10 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012507#comment-17012507 ] Yuan Mei edited comment on FLINK-14163 at 1/10/20 8:16 AM: --- Hey, [~chesnay], 

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012507#comment-17012507 ] Yuan Mei commented on FLINK-14163: -- Hey, [~chesnay], [~zhuzh], [~azagrebin] and [~gjy], thank you so

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012426#comment-17012426 ] Yuan Mei commented on FLINK-14163: -- Hey, [~chesnay], [~zhuzh] and [~azagrebin], thank you so much for

[jira] [Issue Comment Deleted] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-14163: - Comment: was deleted (was: Hey, [~chesnay], [~zhuzh] and [~azagrebin], thank you so much for the

[jira] [Issue Comment Deleted] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-14163: - Comment: was deleted (was: Hey, [~chesnay], [~zhuzh] and [~azagrebin], thank you so much for the

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-09 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012427#comment-17012427 ] Yuan Mei commented on FLINK-14163: -- Hey, [~chesnay], [~zhuzh] and [~azagrebin], thank you so much for

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-07 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009452#comment-17009452 ] Yuan Mei commented on FLINK-14163: -- Accidentally navigating here when digging into the ShuffleMaster

[jira] [Commented] (FLINK-14163) Execution#producedPartitions is possibly not assigned when used

2020-01-07 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010280#comment-17010280 ] Yuan Mei commented on FLINK-14163: -- That's a good point: fail loudly if acting in an unexpected way for

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-11 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17056757#comment-17056757 ] Yuan Mei commented on FLINK-15670: -- Some updates since last sync up: *Watermark* For watermark,

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-04-14 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082909#comment-17082909 ] Yuan Mei commented on FLINK-15670: -- Pull Request is filed: 

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-04-14 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082909#comment-17082909 ] Yuan Mei edited comment on FLINK-15670 at 4/14/20, 6:40 AM: Pull Request is

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-04-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093323#comment-17093323 ] Yuan Mei commented on FLINK-15670: -- Thanks [~AHeise] for reviewing the code! Appreciate :) We've

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-04-28 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093323#comment-17093323 ] Yuan Mei edited comment on FLINK-15670 at 4/28/20, 9:10 AM: Thanks [~AHeise] 

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-18 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110072#comment-17110072 ] Yuan Mei commented on FLINK-15670: -- The code has conflicts with  [FLINK-17659] Rework

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei commented on FLINK-15670: -- Things to follow up and discuss (listed here in case forgotten):  

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei edited comment on FLINK-15670 at 5/16/20, 2:42 AM: Things to follow

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei edited comment on FLINK-15670 at 5/16/20, 2:39 AM: Things to follow

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei edited comment on FLINK-15670 at 5/16/20, 12:34 AM: - Things to

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei edited comment on FLINK-15670 at 5/16/20, 12:33 AM: - Things to

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei edited comment on FLINK-15670 at 5/16/20, 12:30 AM: - Things to

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei edited comment on FLINK-15670 at 5/16/20, 12:31 AM: - Things to

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei edited comment on FLINK-15670 at 5/16/20, 4:26 AM: Things to follow

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei edited comment on FLINK-15670 at 5/16/20, 2:38 AM: Things to follow

[jira] [Comment Edited] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-05-15 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108783#comment-17108783 ] Yuan Mei edited comment on FLINK-15670 at 5/16/20, 3:44 AM: Things to follow

[jira] [Commented] (FLINK-15670) Provide a Kafka Source/Sink pair that aligns Kafka's Partitions and Flink's KeyGroups

2020-03-18 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17061669#comment-17061669 ] Yuan Mei commented on FLINK-15670: --   Some updates: I have finished a new version of POC as promised

[jira] [Commented] (FLINK-17351) CheckpointCoordinator and CheckpointFailureManager ignores checkpoint timeouts

2020-05-19 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17111716#comment-17111716 ] Yuan Mei commented on FLINK-17351: --   Thanks for the pointers [~roman_khachatryan]. I have quite a

[jira] [Comment Edited] (FLINK-17873) Add check for max concurrent checkpoints under UC mode

2020-05-22 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113842#comment-17113842 ] Yuan Mei edited comment on FLINK-17873 at 5/22/20, 8:34 AM: Looks like the

[jira] [Commented] (FLINK-17873) Add check for max concurrent checkpoints under UC mode

2020-05-22 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113842#comment-17113842 ] Yuan Mei commented on FLINK-17873: -- Looks like the issue has already been resolved by accident in

[jira] [Comment Edited] (FLINK-17873) Add check for max concurrent checkpoints under UC mode

2020-05-22 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113842#comment-17113842 ] Yuan Mei edited comment on FLINK-17873 at 5/22/20, 8:36 AM: Looks like the

[jira] [Comment Edited] (FLINK-17873) Add check for max concurrent checkpoints under UC mode

2020-05-22 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113842#comment-17113842 ] Yuan Mei edited comment on FLINK-17873 at 5/22/20, 8:36 AM: Looks like the

[jira] [Created] (FLINK-17873) Add check for max concurrent checkpoints under UC mode

2020-05-21 Thread Yuan Mei (Jira)
Yuan Mei created FLINK-17873: Summary: Add check for max concurrent checkpoints under UC mode Key: FLINK-17873 URL: https://issues.apache.org/jira/browse/FLINK-17873 Project: Flink Issue Type:

[jira] [Closed] (FLINK-17873) Add check for max concurrent checkpoints under UC mode

2020-05-22 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei closed FLINK-17873. Resolution: Fixed > Add check for max concurrent checkpoints under UC mode >

[jira] [Updated] (FLINK-19547) Partial record clean up after receiver task fails and restart

2020-10-08 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-19547: - Description: Partial records happen if a record can not fit into one buffer, then the remaining part

[jira] [Updated] (FLINK-19547) Partial record clean up after receiver task fails and restart

2020-10-08 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-19547: - Summary: Partial record clean up after receiver task fails and restart (was: Partial record clean Up

[jira] [Created] (FLINK-19547) Partial record clean Up after receiver task fails and restart

2020-10-08 Thread Yuan Mei (Jira)
Yuan Mei created FLINK-19547: Summary: Partial record clean Up after receiver task fails and restart Key: FLINK-19547 URL: https://issues.apache.org/jira/browse/FLINK-19547 Project: Flink Issue

[jira] [Comment Edited] (FLINK-19547) Partial record clean up after receiver task fails and restart

2020-10-14 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213701#comment-17213701 ] Yuan Mei edited comment on FLINK-19547 at 10/14/20, 7:40 AM: - *Some other

[jira] [Comment Edited] (FLINK-19547) Partial record clean up after receiver task fails and restart

2020-10-14 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213701#comment-17213701 ] Yuan Mei edited comment on FLINK-19547 at 10/14/20, 9:09 AM: - *Some other

[jira] [Commented] (FLINK-19547) Partial record clean up after receiver task fails and restart

2020-10-14 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213701#comment-17213701 ] Yuan Mei commented on FLINK-19547: -- Some other prototypes for reference: # encode partial length at

[jira] [Updated] (FLINK-19632) Introduce a new ResultPartitionType for Approximate Local Recovery

2020-10-14 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-19632: - Description: # On downstream node failure, the upstream node needs to release sub-partition view while

[jira] [Comment Edited] (FLINK-19441) Performance regression on 24.09.2020

2020-10-13 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213559#comment-17213559 ] Yuan Mei edited comment on FLINK-19441 at 10/14/20, 3:21 AM: - I have

[jira] [Commented] (FLINK-19441) Performance regression on 24.09.2020

2020-10-13 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213559#comment-17213559 ] Yuan Mei commented on FLINK-19441: -- forward this to [~kevin.cyj] and pinged him in person. >

[jira] [Created] (FLINK-19632) Introduce a new ResultPartitionType for Approximate Local Recovery

2020-10-13 Thread Yuan Mei (Jira)
Yuan Mei created FLINK-19632: Summary: Introduce a new ResultPartitionType for Approximate Local Recovery Key: FLINK-19632 URL: https://issues.apache.org/jira/browse/FLINK-19632 Project: Flink

[jira] [Created] (FLINK-19693) Scheduler Change for Approximate Local Recovery

2020-10-18 Thread Yuan Mei (Jira)
Yuan Mei created FLINK-19693: Summary: Scheduler Change for Approximate Local Recovery Key: FLINK-19693 URL: https://issues.apache.org/jira/browse/FLINK-19693 Project: Flink Issue Type: Sub-task

[jira] [Commented] (FLINK-18955) Add snapshot path to job startup message

2020-08-17 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178920#comment-17178920 ] Yuan Mei commented on FLINK-18955: -- Hey [~NicoK], just to confirm, do you mean something similar to the

[jira] [Updated] (FLINK-18112) Single Task Approximate Failure Recovery

2020-09-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-18112: - Summary: Single Task Approximate Failure Recovery (was: Single Task Failure Recovery Prototype) >

[jira] [Updated] (FLINK-18112) Approximate Task-Local Recovery

2020-09-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-18112: - Summary: Approximate Task-Local Recovery (was: Single Task Approximate Failure Recovery) >

[jira] [Updated] (FLINK-18112) Approximate Task-Local Recovery -- Milestone One

2020-09-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-18112: - Summary: Approximate Task-Local Recovery -- Milestone One (was: Approximate Task-Local Recovery) >

[jira] [Updated] (FLINK-18112) Approximate Task-Local Recovery -- Milestone One

2020-09-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-18112: - Description: This is the Jira ticket for Milestone One of [FLIP-135 Approximate Task-Local

[jira] [Updated] (FLINK-18112) Approximate Task-Local Recovery -- Milestone One

2020-09-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-18112: - Description: This is the Jira ticket for Milestone One of [FLIP-135 Approximate Task-Local

[jira] [Commented] (FLINK-17949) KafkaShuffleITCase.testSerDeIngestionTime:156->testRecordSerDe:388 expected:<310> but was:<0>

2020-05-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17117799#comment-17117799 ] Yuan Mei commented on FLINK-17949: -- Wondering usually how can I reproduce this? Or where can I find

[jira] [Updated] (FLINK-17916) Provide API to separate KafkaShuffle's Producer and Consumer to different jobs

2020-05-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17916: - Affects Version/s: (was: 1.12.0) 1.11.0 > Provide API to separate

[jira] [Commented] (FLINK-17916) Provide API to separate KafkaShuffle's Producer and Consumer to different jobs

2020-05-28 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118473#comment-17118473 ] Yuan Mei commented on FLINK-17916: -- This one is moved to 1.12.0 PR is ready, the review is postponed

[jira] [Commented] (FLINK-17949) KafkaShuffleITCase.testSerDeIngestionTime:156->testRecordSerDe:388 expected:<310> but was:<0>

2020-05-28 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118525#comment-17118525 ] Yuan Mei commented on FLINK-17949: -- Thank [~rmetzger] for the log reference!  In both cases, SerDe

[jira] [Updated] (FLINK-17949) KafkaShuffleITCase.testSerDeIngestionTime:156->testRecordSerDe:388 expected:<310> but was:<0>

2020-05-28 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17949: - Attachment: logs-ci-kafkagelly-1590500380.zip >

[jira] [Updated] (FLINK-17949) KafkaShuffleITCase.testSerDeIngestionTime:156->testRecordSerDe:388 expected:<310> but was:<0>

2020-05-28 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17949: - Attachment: logs-ci-kafkagelly-1590524911.zip >

[jira] [Commented] (FLINK-17949) KafkaShuffleITCase.testSerDeIngestionTime:156->testRecordSerDe:388 expected:<310> but was:<0>

2020-05-28 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118527#comment-17118527 ] Yuan Mei commented on FLINK-17949: -- uploaded two logs  >

[jira] [Updated] (FLINK-17916) Provide API to separate KafkaShuffle's Producer and Consumer to different jobs

2020-05-25 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17916: - Summary: Provide API to separate KafkaShuffle's Producer and Consumer to different jobs (was: Separate

[jira] [Updated] (FLINK-17916) Separate KafkaShuffle's Producer and Consumer to different jobs

2020-05-25 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17916: - Description: Follow up of FLINK-15670 *Separate sink (producer) and source (consumer) to different

[jira] [Updated] (FLINK-17916) Separate KafkaShuffle read/write to different environments

2020-05-25 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17916: - Description: Follow up of FLINK-15670 Separate sink (producer) and source (consumer) to different

[jira] [Updated] (FLINK-17916) Separate KafkaShuffle's Producer and Consumer to different jobs

2020-05-25 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17916: - Summary: Separate KafkaShuffle's Producer and Consumer to different jobs (was: Separate KafkaShuffle

[jira] [Created] (FLINK-17916) Separate KafkaShuffle read/write to different environments

2020-05-25 Thread Yuan Mei (Jira)
Yuan Mei created FLINK-17916: Summary: Separate KafkaShuffle read/write to different environments Key: FLINK-17916 URL: https://issues.apache.org/jira/browse/FLINK-17916 Project: Flink Issue

[jira] [Updated] (FLINK-17916) Separate KafkaShuffle read/write to different environments

2020-05-25 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17916: - Fix Version/s: (was: 1.12.0) 1.11.0 > Separate KafkaShuffle read/write to

[jira] [Updated] (FLINK-17916) Provide API to separate KafkaShuffle's Producer and Consumer to different jobs

2020-05-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17916: - Fix Version/s: (was: 1.11.0) 1.12.0 > Provide API to separate KafkaShuffle's

[jira] [Updated] (FLINK-17916) Provide API to separate KafkaShuffle's Producer and Consumer to different jobs

2020-05-27 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-17916: - Affects Version/s: (was: 1.11.0) 1.12.0 > Provide API to separate

[jira] [Updated] (FLINK-19774) Introduce sub partition view version for approximate Failover

2020-10-23 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Mei updated FLINK-19774: - Description:   This ticket is to solve a corner case where a downstream task continuously fails

[jira] [Commented] (FLINK-19774) Introduce sub partition view version for approximate Failover

2020-10-23 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219524#comment-17219524 ] Yuan Mei commented on FLINK-19774: -- Places need to be changed: 1. set the parent of view -> invalid

  1   2   3   4   5   6   7   8   >