[
https://issues.apache.org/jira/browse/FLINK-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ShenDa updated FLINK-15182:
---------------------------
Description:
Flink already has FileSystem connector that can sink data into specified file
system, but it just writes data to a target path without bukcet and it can't
configure file rolling policy neither. We can use StreamingFileSink to avoid
those shortcomings instead, however, the StreamingFileSink is a SinkFunction
that users have to implement their processing logic with DataStreamAPI.
Therefore, we want to add a new FileTableSink based on StreamingFileSink. This
will help us connect file system by using TableAPI&SQL. But according to the
two ways of StreamingFileSink to sinking data, bulk format and row format, it
comes a question to us. Does the FileTableSink support bulk format and row
format simultaneously? Or separating this sink into two sinks,
BulkFileTableSink and RowFileTableSink, which writes data by using one format
respectively.
was:
Flink already has FileSystem connector that can sink data into specified file
system, but it just writes data to a target path without bukcet and it can't
configure file rolling policy neither. We can use StreamingFileSink to avoid
those shortcomings instead, however, the StreamingFileSink is a SinkFunction
that users have to implement their processing logic with DataStreamAPI.
Therefore, we want to add a new FileTableSink based on StreamingFileSink. This
will help us connect file system by using TableAPI&SQL. But according the two
ways of StreamingFileSink to sinking data, bulk format and row format, it comes
a question to us. Does the FileTableSink support bulk format and row format
simultaneously? Or separating this sink into two sinks, BulkFileTableSink and
RowFileTableSink, which writes data by using one format respectively.
> Add FileTableSink based on StreamingFileSink
> --------------------------------------------
>
> Key: FLINK-15182
> URL: https://issues.apache.org/jira/browse/FLINK-15182
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.9.0
> Reporter: ShenDa
> Priority: Major
>
> Flink already has FileSystem connector that can sink data into specified file
> system, but it just writes data to a target path without bukcet and it can't
> configure file rolling policy neither. We can use StreamingFileSink to avoid
> those shortcomings instead, however, the StreamingFileSink is a SinkFunction
> that users have to implement their processing logic with DataStreamAPI.
> Therefore, we want to add a new FileTableSink based on StreamingFileSink.
> This will help us connect file system by using TableAPI&SQL. But according to
> the two ways of StreamingFileSink to sinking data, bulk format and row
> format, it comes a question to us. Does the FileTableSink support bulk format
> and row format simultaneously? Or separating this sink into two sinks,
> BulkFileTableSink and RowFileTableSink, which writes data by using one format
> respectively.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
