[
https://issues.apache.org/jira/browse/FLINK-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114074#comment-17114074
]
Jark Wu commented on FLINK-17878:
---------------------------------
I don't think removing {{transient}} can help here. The serialized watermark
doesn't help when job is restored, the value is still the initial value. If you
want to restore the watermark to the previous one, you have to use {{State}} in
streaming. However, this is another big topic and make the logic complex and
will involve some performance concern. Besides, we didn't find any problem with
current design so far. The watermark will be adjust to a correct value once
there is new records coming in.
> Transient watermark attribute should be initial at runtime in streaming file
> operators
> --------------------------------------------------------------------------------------
>
> Key: FLINK-17878
> URL: https://issues.apache.org/jira/browse/FLINK-17878
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.11.0
> Reporter: xiaogang zhou
> Assignee: xiaogang zhou
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> StreamingFileWriter has a
> private transient long currentWatermark = Long.MIN_VALUE;
>
> in case developer wants to create a custom bucket assigner, it will receive a
> currentWatermark as 0, this might be conflict with the original flink
> approach to handle a min_long.
>
> should we remove the transient key word?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
