[
https://issues.apache.org/jira/browse/FLINK-13609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916537#comment-16916537
]
Joao Boto commented on FLINK-13609:
-----------------------------------
And we could not use the max counter for that bucket? or we could have problems
with a already closed bucket no?
Maybe for this situation could be more intuitive the counter be the timestamp
of the initialization of file?
> StreamingFileSink - reset part counter on bucket change
> -------------------------------------------------------
>
> Key: FLINK-13609
> URL: https://issues.apache.org/jira/browse/FLINK-13609
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Reporter: Joao Boto
> Priority: Major
>
> When writing to files using StreamingFileSink on bucket change we expect that
> partcounter will reset its counter to 0
> as a example
> * using DateTimeBucketAssigner using ({color:#6a8759}yyyy/MM/dd/HH{color})
> * and ten files hour (for simplicity)
> this will create the:
> * bucket 2019/08/07/00 with files partfile-0-0 to partfile-0-9
> * bucket 2019/08/07/01 with files partfile-0-10 to partfile-0-19
> * bucket 2019/08/07/02 with files partfile-0-20 to partfile-0-29
> and we expect this:
> * bucket 2019/08/07/00 with files partfile-0-0 to partfile-0-9
> * bucket 2019/08/07/01 with files partfile-0-0 to partfile-0-9
> * bucket 2019/08/07/02 with files partfile-0-0 to partfile-0-9
>
> [~kkl0u] i don't know if it's the expected behavior (or this can be
> configured)
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
