[
https://issues.apache.org/jira/browse/FLINK-23959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dong Lin updated FLINK-23959:
-----------------------------
Description:
The FLIP design doc can be found at
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181311363.
The existing Flink ML library allows users to compose an Estimator/Transformer
from a pipeline (i.e. linear sequence) of Estimator/Transformer. Users only
need to construct this Pipeline once and generate the corresponding
PipelineModel, without having to explicitly construct the fitted PipelineModel
as a linear sequence of stages. However, in order to train a DAG of
Estimator/Transformer and uses the trained model for inference, users currently
need to construct the DAG twice, once for the training logic and once for the
inference logic. This experience is inferior to the experience of training and
using a chain of Estimator/Transformer. In addition to requiring more work from
users, this approach is more error prone because the DAG for the training logic
may be inconsistent from the DAG for the inference logic.
In order to address the issues described above, we propose to add several
helper classes that allow users to compose Estimator/Transformer/AlgoOperator
from a DAG of Estimator/Transformer/AlgoOperator.
was:
The existing Flink ML library allows users to compose an Estimator/Transformer
from a pipeline (i.e. linear sequence) of Estimator/Transformer. Users only
need to construct this Pipeline once and generate the corresponding
PipelineModel, without having to explicitly construct the fitted PipelineModel
as a linear sequence of stages.
However, in the use-case that needs a DAG of Estimator/Transformer, users
currently needs to separately build the DAG separately, once for the training
logic and once for the inference logic. This experience is inferior to the
cases supported by the Pipeline.
To improve the user experience, we propose to add several helper classes that
allow users to compose Estimator/Transformer/AlgoOperator from a DAG of
Estimator/Transformer/AlgoOperator.
> FLIP-175: Add GraphBuilder to compose Estimator/Transformer/AlgoOperator from
> a DAG of stages
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-23959
> URL: https://issues.apache.org/jira/browse/FLINK-23959
> Project: Flink
> Issue Type: Improvement
> Reporter: Dong Lin
> Priority: Major
>
> The FLIP design doc can be found at
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181311363.
> The existing Flink ML library allows users to compose an
> Estimator/Transformer from a pipeline (i.e. linear sequence) of
> Estimator/Transformer. Users only need to construct this Pipeline once and
> generate the corresponding PipelineModel, without having to explicitly
> construct the fitted PipelineModel as a linear sequence of stages. However,
> in order to train a DAG of Estimator/Transformer and uses the trained model
> for inference, users currently need to construct the DAG twice, once for the
> training logic and once for the inference logic. This experience is inferior
> to the experience of training and using a chain of Estimator/Transformer. In
> addition to requiring more work from users, this approach is more error prone
> because the DAG for the training logic may be inconsistent from the DAG for
> the inference logic.
> In order to address the issues described above, we propose to add several
> helper classes that allow users to compose Estimator/Transformer/AlgoOperator
> from a DAG of Estimator/Transformer/AlgoOperator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
