[
https://issues.apache.org/jira/browse/HUDI-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yuemeng updated HUDI-3953:
--------------------------
Description:
Currently. Flink Hudi Module only supports SQL APIs. People who want to use
low-level APIs such used for operating Flink state or another purpose don't
have a friendly way.
It can be provided a low-level APIs for users to write/read hoodie data
The API design and main change will be:
# add sink and source API in Pipelines
# getSinkRuntimeProvider in HoodieTableSink call Pipelines.sink(...) to return
DataStreamSink
# getScanRuntimeProvider in HoodieTableSource call Pipelines.source() to
return DataStream
# move some common methods such as getInputFormat in util class
# low-level API such as read and write just call Pipelines.sink(...) and
Pipelines.source()
was:
Currently. Flink Hudi Module only supports SQL APIs. People who want to use
low-level APIs such used for operating Flink state or another purpose don't
have a friendly way.
It can be provided a low-level APIs for users to write/read hoodie data
the API design and main change will be:
# add sink and source API in Pipelines
# getSinkRuntimeProvider in HoodieTableSink call Pipelines.sink(...) to return
DataStreamSink
# getScanRuntimeProvider in HoodieTableSource call Pipelines.source() to
return DataStream
# move some common methods such as getInputFormat in util class
# low-level API such as read and write just call Pipelines.sink(...) and
Pipelines.source()
> Flink Hudi module should support low-level read and write APIs
> ---------------------------------------------------------------
>
> Key: HUDI-3953
> URL: https://issues.apache.org/jira/browse/HUDI-3953
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: yuemeng
> Priority: Major
> Labels: pull-request-available
>
> Currently. Flink Hudi Module only supports SQL APIs. People who want to use
> low-level APIs such used for operating Flink state or another purpose don't
> have a friendly way.
> It can be provided a low-level APIs for users to write/read hoodie data
> The API design and main change will be:
> # add sink and source API in Pipelines
> # getSinkRuntimeProvider in HoodieTableSink call Pipelines.sink(...) to
> return DataStreamSink
> # getScanRuntimeProvider in HoodieTableSource call Pipelines.source() to
> return DataStream
> # move some common methods such as getInputFormat in util class
> # low-level API such as read and write just call Pipelines.sink(...) and
> Pipelines.source()
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
