[
https://issues.apache.org/jira/browse/FLINK-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ShenDa updated FLINK-15182:
---------------------------
Description:
Flink already has FileSystem connector that can sink data into specified file
system, but it just writes data to a target path without bukcet and it can't
configure file rolling policy neither. We can use StreamingFileSink to avoid
those shortcomings instead, however, the StreamingFileSink is a SinkFunction
that users have to implement their processing logic with DataStreamAPI.
Therefore, we want to add a new FileTableSink based on StreamingFileSink. This
will help us connect file system by using TableAPI&SQL. But according to the
two ways of StreamingFileSink to sinking data, bulk format and row format, it
comes a question to us. Does the FileTableSink have to support bulk format and
row format simultaneously? Or separating this sink into two table sinks,
BulkFileTableSink and RowFileTableSink, which writes data by using one format
respectively.
was:
Flink already has FileSystem connector that can sink data into specified file
system, but it just writes data to a target path without bukcet and it can't
configure file rolling policy neither. We can use StreamingFileSink to avoid
those shortcomings instead, however, the StreamingFileSink is a SinkFunction
that users have to implement their processing logic with DataStreamAPI.
Therefore, we want to add a new FileTableSink based on StreamingFileSink. This
will help us connect file system by using TableAPI&SQL. But according to the
two ways of StreamingFileSink to sinking data, bulk format and row format, it
comes a question to us. Does the FileTableSink have to support bulk format and
row format simultaneously? Or separating this sink into two sinks,
BulkFileTableSink and RowFileTableSink, which writes data by using one format
respectively.
> Add FileTableSink based on StreamingFileSink
> --------------------------------------------
>
> Key: FLINK-15182
> URL: https://issues.apache.org/jira/browse/FLINK-15182
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.9.0
> Reporter: ShenDa
> Priority: Major
>
> Flink already has FileSystem connector that can sink data into specified file
> system, but it just writes data to a target path without bukcet and it can't
> configure file rolling policy neither. We can use StreamingFileSink to avoid
> those shortcomings instead, however, the StreamingFileSink is a SinkFunction
> that users have to implement their processing logic with DataStreamAPI.
> Therefore, we want to add a new FileTableSink based on StreamingFileSink.
> This will help us connect file system by using TableAPI&SQL. But according to
> the two ways of StreamingFileSink to sinking data, bulk format and row
> format, it comes a question to us. Does the FileTableSink have to support
> bulk format and row format simultaneously? Or separating this sink into two
> table sinks, BulkFileTableSink and RowFileTableSink, which writes data by
> using one format respectively.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
