[jira] [Updated] (FLINK-31173) Fix several bugs in flink-ml-iteration module
[ https://issues.apache.org/jira/browse/FLINK-31173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhipeng Zhang updated FLINK-31173: -- Description: In flink-ml-iteration, there are several bugs as follows: # TailOperator should have one input operator. We have added a Tail operator to increment the epoch watermark at each iteration. We have made an assumption that each Tail operator have only one input and did not align the epoch watermarks from different inputs. This assumption might not be true if the input is an `union`. # ProxyOperatorStateBackend does not correctly initialize the state descriptor. was: In flink-ml-iteration, there are several bugs as follows: # TailOperator should have one input operator. We have added a Tail operator to increment the epoch watermark at each iteration. We have made an assumption that each Tail operator have only one input and did not align the epoch watermarks from different inputs. This assumption might not be true if the input is an `union`. # ReplayOperator should replay the records when it receives max epochwatermark. Currently ReplayOperator does not replay the records when it receives the max epochwatermark. However, it is inconsistent with the HeadOperator. As HeadOperator will always forward the record to downstreams. # ProxyOperatorStateBackend does not correctly initialize the state descriptor. > Fix several bugs in flink-ml-iteration module > - > > Key: FLINK-31173 > URL: https://issues.apache.org/jira/browse/FLINK-31173 > Project: Flink > Issue Type: Bug > Components: Library / Machine Learning >Affects Versions: ml-2.0.0, ml-2.1.0, ml-2.2.0 >Reporter: Zhipeng Zhang >Assignee: Zhipeng Zhang >Priority: Major > Labels: pull-request-available > Fix For: ml-2.3.0 > > > In flink-ml-iteration, there are several bugs as follows: > # TailOperator should have one input operator. We have added a Tail operator > to increment the epoch watermark at each iteration. We have made an > assumption that each Tail operator have only one input and did not align the > epoch watermarks from different inputs. This assumption might not be true if > the input is an `union`. > # ProxyOperatorStateBackend does not correctly initialize the state > descriptor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-31173) Fix several bugs in flink-ml-iteration module
[ https://issues.apache.org/jira/browse/FLINK-31173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated FLINK-31173: - Fix Version/s: ml-2.3.0 > Fix several bugs in flink-ml-iteration module > - > > Key: FLINK-31173 > URL: https://issues.apache.org/jira/browse/FLINK-31173 > Project: Flink > Issue Type: Bug > Components: Library / Machine Learning >Affects Versions: ml-2.0.0, ml-2.1.0, ml-2.2.0 >Reporter: Zhipeng Zhang >Assignee: Zhipeng Zhang >Priority: Major > Labels: pull-request-available > Fix For: ml-2.3.0 > > > In flink-ml-iteration, there are several bugs as follows: > # TailOperator should have one input operator. We have added a Tail operator > to increment the epoch watermark at each iteration. We have made an > assumption that each Tail operator have only one input and did not align the > epoch watermarks from different inputs. This assumption might not be true if > the input is an `union`. > # ReplayOperator should replay the records when it receives max > epochwatermark. Currently ReplayOperator does not replay the records when it > receives the max epochwatermark. However, it is inconsistent with the > HeadOperator. As HeadOperator will always forward the record to downstreams. > # ProxyOperatorStateBackend does not correctly initialize the state > descriptor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-31173) Fix several bugs in flink-ml-iteration module
[ https://issues.apache.org/jira/browse/FLINK-31173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-31173: --- Labels: pull-request-available (was: ) > Fix several bugs in flink-ml-iteration module > - > > Key: FLINK-31173 > URL: https://issues.apache.org/jira/browse/FLINK-31173 > Project: Flink > Issue Type: Bug > Components: Library / Machine Learning >Affects Versions: ml-2.0.0, ml-2.1.0, ml-2.2.0 >Reporter: Zhipeng Zhang >Assignee: Zhipeng Zhang >Priority: Major > Labels: pull-request-available > > In flink-ml-iteration, there are several bugs as follows: > # TailOperator should have one input operator. We have added a Tail operator > to increment the epoch watermark at each iteration. We have made an > assumption that each Tail operator have only one input and did not align the > epoch watermarks from different inputs. This assumption might not be true if > the input is an `union`. > # ReplayOperator should replay the records when it receives max > epochwatermark. Currently ReplayOperator does not replay the records when it > receives the max epochwatermark. However, it is inconsistent with the > HeadOperator. As HeadOperator will always forward the record to downstreams. > # ProxyOperatorStateBackend does not correctly initialize the state > descriptor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-31173) Fix several bugs in flink-ml-iteration module
[ https://issues.apache.org/jira/browse/FLINK-31173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhipeng Zhang updated FLINK-31173: -- Description: In flink-ml-iteration, there are several bugs as follows: # TailOperator should have one input operator. We have added a Tail operator to increment the epoch watermark at each iteration. We have made an assumption that each Tail operator have only one input and did not align the epoch watermarks from different inputs. This assumption might not be true if the input is an `union`. # ReplayOperator should replay the records when it receives max epochwatermark. Currently ReplayOperator does not replay the records when it receives the max epochwatermark. However, it is inconsistent with the HeadOperator. As HeadOperator will always forward the record to downstreams. # ProxyOperatorStateBackend does not correctly initialize the state descriptor. was: In flink-ml-iteration, there are several bugs as follows: # TailOperator should have one input operator. We have added a Tail operator to increment the epoch watermark at each iteration. We have made an assumption that each Tail operator have only one input and did not align the epoch watermarks from different inputs. This assumption might not be true if the input is an `union`. # ReplayOperator should replay the records when it receives max epochwatermark. Currently ReplayOperator does not replay the records when it receives the max epochwatermark. However, it is inconsistent with the HeadOperator. As HeadOperator will always forward the record to downstreams. # ProxyOperatorStateBackend does not correctly initialize the state descriptor. > Fix several bugs in flink-ml-iteration module > - > > Key: FLINK-31173 > URL: https://issues.apache.org/jira/browse/FLINK-31173 > Project: Flink > Issue Type: Bug > Components: Library / Machine Learning >Affects Versions: ml-2.0.0, ml-2.1.0, ml-2.2.0 >Reporter: Zhipeng Zhang >Priority: Major > > In flink-ml-iteration, there are several bugs as follows: > # TailOperator should have one input operator. We have added a Tail operator > to increment the epoch watermark at each iteration. We have made an > assumption that each Tail operator have only one input and did not align the > epoch watermarks from different inputs. This assumption might not be true if > the input is an `union`. > # ReplayOperator should replay the records when it receives max > epochwatermark. Currently ReplayOperator does not replay the records when it > receives the max epochwatermark. However, it is inconsistent with the > HeadOperator. As HeadOperator will always forward the record to downstreams. > # ProxyOperatorStateBackend does not correctly initialize the state > descriptor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-31173) Fix several bugs in flink-ml-iteration module
[ https://issues.apache.org/jira/browse/FLINK-31173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhipeng Zhang updated FLINK-31173: -- Description: In flink-ml-iteration, there are several bugs as follows: # TailOperator should have one input operator. We have added a Tail operator to increment the epoch watermark at each iteration. We have made an assumption that each Tail operator have only one input and did not align the epoch watermarks from different inputs. This assumption might not be true if the input is an `union`. # ReplayOperator should replay the records when it receives max epochwatermark. Currently ReplayOperator does not replay the records when it receives the max epochwatermark. However, it is inconsistent with the HeadOperator. As HeadOperator will always forward the record to downstreams. # ProxyOperatorStateBackend does not correctly initialize the state descriptor. was: In flink-ml-iteration, we have added a Tail operator to increment the epoch watermark at each iteration. We have made an assumption that each Tail operator have only one input and did not align the epoch watermarks from different inputs. This assumption might not be true if the input is an `union`. I propose to add an explicit check to prevent from having multiple input for the TailOperator. If a the input does contain multiple inputs, then users can add a map operator after union. Summary: Fix several bugs in flink-ml-iteration module (was: TailOperator should only have one input) > Fix several bugs in flink-ml-iteration module > - > > Key: FLINK-31173 > URL: https://issues.apache.org/jira/browse/FLINK-31173 > Project: Flink > Issue Type: Bug > Components: Library / Machine Learning >Affects Versions: ml-2.0.0, ml-2.1.0, ml-2.2.0 >Reporter: Zhipeng Zhang >Priority: Major > > In flink-ml-iteration, there are several bugs as follows: > > # TailOperator should have one input operator. We have added a Tail operator > to increment the epoch watermark at each iteration. We have made an > assumption that each Tail operator have only one input and did not align the > epoch watermarks from different inputs. This assumption might not be true if > the input is an `union`. > # ReplayOperator should replay the records when it receives max > epochwatermark. Currently ReplayOperator does not replay the records when it > receives the max epochwatermark. However, it is inconsistent with the > HeadOperator. As HeadOperator will always forward the record to downstreams. > # ProxyOperatorStateBackend does not correctly initialize the state > descriptor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
