[
https://issues.apache.org/jira/browse/FLINK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aljoscha Krettek updated FLINK-4796:
------------------------------------
Description:
The current {{SinkFunction}} cannot access the timestamps of elements which
resulted in the (somewhat hacky) {{FlinkKafkaProducer010}}. Due to other
limitations {{GenericWriteAheadSink}} is currently also a {{StreamOperator}}
and not a {{SinkFunction}}.
We should add a new interface for sinks that takes a context parameter, similar
to {{ProcessFunction}}. This will allow sinks to query additional meta data
about the element that they're receiving.
This is one ML thread where a user ran into a problem caused by this:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Why-I-am-getting-Null-pointer-exception-while-accessing-RuntimeContext-in-FlinkKafkaProducer010-td12633.html#a12635
h3. Original Text (that is still valid but not general)
The Kafka 0.10 connector supports writing event timestamps to Kafka.
Currently, the regular DataStream APIs don't allow user code to access the
event timestamp easily. That's why the Kafka connector is using a custom
operator ({{transform()}}) to access the event time.
With this JIRA, I would like to provide the event timestamp in the regular
DataStream APIs.
Once I'll look into the issue, I'll post some proposals how to add the
timestamp.
was:
The Kafka 0.10 connector supports writing event timestamps to Kafka.
Currently, the regular DataStream APIs don't allow user code to access the
event timestamp easily. That's why the Kafka connector is using a custom
operator ({{transform()}}) to access the event time.
With this JIRA, I would like to provide the event timestamp in the regular
DataStream APIs.
Once I'll look into the issue, I'll post some proposals how to add the
timestamp.
> Extend SinkFunction to include also the event timestamp
> -------------------------------------------------------
>
> Key: FLINK-4796
> URL: https://issues.apache.org/jira/browse/FLINK-4796
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
> Affects Versions: 1.2.0
> Reporter: Robert Metzger
>
> The current {{SinkFunction}} cannot access the timestamps of elements which
> resulted in the (somewhat hacky) {{FlinkKafkaProducer010}}. Due to other
> limitations {{GenericWriteAheadSink}} is currently also a {{StreamOperator}}
> and not a {{SinkFunction}}.
> We should add a new interface for sinks that takes a context parameter,
> similar to {{ProcessFunction}}. This will allow sinks to query additional
> meta data about the element that they're receiving.
> This is one ML thread where a user ran into a problem caused by this:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Why-I-am-getting-Null-pointer-exception-while-accessing-RuntimeContext-in-FlinkKafkaProducer010-td12633.html#a12635
> h3. Original Text (that is still valid but not general)
> The Kafka 0.10 connector supports writing event timestamps to Kafka.
> Currently, the regular DataStream APIs don't allow user code to access the
> event timestamp easily. That's why the Kafka connector is using a custom
> operator ({{transform()}}) to access the event time.
> With this JIRA, I would like to provide the event timestamp in the regular
> DataStream APIs.
> Once I'll look into the issue, I'll post some proposals how to add the
> timestamp.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
