[
https://issues.apache.org/jira/browse/FLINK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17436651#comment-17436651
]
Arvid Heise commented on FLINK-24705:
-------------------------------------
I'm assuming you want to have data lineage information added to the records.
DataStream is a low-level API where you have to do add it to your record
yourself.
For example, you can use {{ConsumerRecord#topic}} in a
{{KafkaDeserializationSchema}} to retrieve the value and add it into a field of
your record. Similarly, if you create a {{IndexRequest}} for the ES sink, you
add the respective information to the document.
There are no plans to add some default way for that metadata as the solution
space is so big that Flink would barely satisfy the needs of a specific user.
It might be something for SQL but I think it's also fair to assume that we
still expect the user to manually select metadata columns that carry the
information.
> Flink blood relationship
> ------------------------
>
> Key: FLINK-24705
> URL: https://issues.apache.org/jira/browse/FLINK-24705
> Project: Flink
> Issue Type: New Feature
> Components: API / DataStream
> Reporter: HideOnBush
> Priority: Critical
>
> Please tell me how to do blood relationship in Flink DataStream. For example,
> source is topic, sink is ES, I want to capture this specific information. For
> example, the connection information of topic, the SINK information of ES, and
> then the blood relationship, ask for ideas
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
