[
https://issues.apache.org/jira/browse/FLINK-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-2672:
----------------------------------
Labels: features stale-minor (was: features)
> Add partitioned output format to HDFS RollingSink
> -------------------------------------------------
>
> Key: FLINK-2672
> URL: https://issues.apache.org/jira/browse/FLINK-2672
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Common, Connectors / FileSystem
> Affects Versions: 0.10.0
> Reporter: Mohamed Amine ABDESSEMED
> Priority: Minor
> Labels: features, stale-minor
>
> An interesting use case of the HDFS Sink is to dispatch data into multiple
> directories depending of attributes present in the source data.
> For example, for some data with a timestamp and a status fields, we want to
> write it into different directories using a pattern like :
> /somepath/%{timestamp}/%{status}
> The expected results are somethings like:
> /somepath/some_timestamp/wellformed
> /somepath/some_timestamp/malformed
> /somepath/some_timestamp/incomplete
> ...
> etc
> To support this functionality the bucketing and checkpointing logics need to
> be changed.
> Note: For now, this can be done using the current version of the Rolling HDFS
> Sink (https://github.com/apache/flink/pull/1084) with the help of splitting
> data streams and having multiple HDFS sinks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
