[
https://issues.apache.org/jira/browse/FLINK-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luo Gen updated FLINK-13167:
----------------------------
Description:
Import/export a trained model is an important part in machine learning
lifecycle, especially for serving purpose, hence it triggered a heatedly
discussion in the previous pull request for FlinkML basic interfaces. Some
thoughts about this are summarized in the following doc.
[https://docs.google.com/document/d/1B84b-1CvOXtwWQ6_tQyiaHwnSeiRqh-V96Or8uHqCp8/edit?usp=sharing]
The jira will introduce the import/export framework based on these discussion.
Import and export framework have almost the same structure so we take export as
example. Export framework has 3 basic components, including 2 interfaces for
exporter authors to implement, and one util class for end-users to acquire an
Exporter.
Interfaces:
- Exporter: Interface of model exporters. Implementations can export supported
models to a specific target model type, such as PMML.
- ExporterFactory: Interface of factories to create Exporters. Each
implementation supports only one model type. Typically an ExporterFactory is
given a model instance with a property map if needed, and returns an Exporter
that supports the model to export to target model type. ExporterFactories are
loaded with ServiceLoader framework of Java, and should be registered in the
file
"META-INF/services/org.apache.flink.ml.api.misc.exportation.ExporterFactory".
Utility:
- ExporterLoader: Loader to get an Exporter for a specific model format and a
model instance. Users would get Exporters via this class rather than directly
creating with constructors. This is a concrete util class and no one needs to
override it.
Besides, a new enumeration named ModelType is also introduced, listing all
types FlinkML may support. This would be extended when a new exporter support a
new type.
was:
Import/export a trained model is an important part in machine learning
lifecycle, especially for serving purpose, hence it triggered a heatedly
discussion in the previous pull request for FlinkML basic interfaces. Some
thoughts about this are summarized in the following doc.
https://docs.google.com/document/d/1B84b-1CvOXtwWQ6_tQyiaHwnSeiRqh-V96Or8uHqCp8/edit?usp=sharing
The jira will introduce the import/export framework based on these discussion.
Import and export framework have almost the same structure so we take export
for example. Export framework has 3 basic components, including 2 interfaces
for exporter authors to implement, and one util class for end-users to acquire
an Exporter.
Interfaces:
- Exporter: Interface of model exporters. Implementations can export supported
models to a specific target model type, such as PMML.
- ExporterFactory: Interface of factories to create Exporters. Each
implementation supports only one model type. Typically an ExporterFactory is
given a model instance with a property map if needed, and returns an Exporter
that supports the model to export to target model type. ExporterFactories are
loaded with ServiceLoader framework of Java, and should be registered in the
file
"META-INF/services/org.apache.flink.ml.api.misc.exportation.ExporterFactory".
Utility:
- ExporterLoader: Loader to get an Exporter for a specific model format and a
model instance. Users would get Exporters via this class rather than directly
creating with constructors. This is a concrete util class and no one needs to
override it.
Besides, a new enumeration named ModelType is also introduced, listing all
types FlinkML may support. This would be extended when a new exporter support a
new type.
> Introduce FlinkML import/export framework
> -----------------------------------------
>
> Key: FLINK-13167
> URL: https://issues.apache.org/jira/browse/FLINK-13167
> Project: Flink
> Issue Type: Sub-task
> Components: Library / Machine Learning
> Reporter: Luo Gen
> Assignee: Luo Gen
> Priority: Major
>
> Import/export a trained model is an important part in machine learning
> lifecycle, especially for serving purpose, hence it triggered a heatedly
> discussion in the previous pull request for FlinkML basic interfaces. Some
> thoughts about this are summarized in the following doc.
> [https://docs.google.com/document/d/1B84b-1CvOXtwWQ6_tQyiaHwnSeiRqh-V96Or8uHqCp8/edit?usp=sharing]
> The jira will introduce the import/export framework based on these
> discussion. Import and export framework have almost the same structure so we
> take export as example. Export framework has 3 basic components, including 2
> interfaces for exporter authors to implement, and one util class for
> end-users to acquire an Exporter.
> Interfaces:
> - Exporter: Interface of model exporters. Implementations can export
> supported models to a specific target model type, such as PMML.
> - ExporterFactory: Interface of factories to create Exporters. Each
> implementation supports only one model type. Typically an ExporterFactory is
> given a model instance with a property map if needed, and returns an Exporter
> that supports the model to export to target model type. ExporterFactories are
> loaded with ServiceLoader framework of Java, and should be registered in the
> file
> "META-INF/services/org.apache.flink.ml.api.misc.exportation.ExporterFactory".
> Utility:
> - ExporterLoader: Loader to get an Exporter for a specific model format and
> a model instance. Users would get Exporters via this class rather than
> directly creating with constructors. This is a concrete util class and no one
> needs to override it.
> Besides, a new enumeration named ModelType is also introduced, listing all
> types FlinkML may support. This would be extended when a new exporter support
> a new type.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
