[
https://issues.apache.org/jira/browse/FLINK-25483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472124#comment-17472124
]
陈磊 commented on FLINK-25483:
----------------------------
[~MartijnVisser] oh,no.I think you misunderstood me. Field a,b and c is an
example.In actual development, the user does not know when a field is a null
value and when is a non-null value. At this time, the a field is a null value,
but the next moment it is a non-null value, and the same is true for the b
field. If you filter in advance as you said, how to do this? In addition, I
believe you should also know that in the design of the Elasticsearch connector,
the fields defined in the DDL are fixed as serialized fields, and the RowData
data is serialized into the corresponding byte array by the serializer, and
then written to the Elasticsearch. During this process, null values are also
written or overwritten into Elasticsearch. Users do not want our null values to
affect the original records in Elasticsearch. For this requirement, I have
encountered many times in our company, and have provided a corresponding
solution, that is, encapsulating non-null fields and field types as Json Node,
then encapsulating them into UpdateRequest, and writing them to Elasticsearch.
Of course, enabling this function requires parameter configuration. I don't
think the current design affects the SQL standard. What do you think?
> When FlinkSQL writes ES, it will not write and update the null value field
> --------------------------------------------------------------------------
>
> Key: FLINK-25483
> URL: https://issues.apache.org/jira/browse/FLINK-25483
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Ecosystem
> Reporter: 陈磊
> Priority: Minor
>
> Using Flink SQL to consume Kafka to write ES, sometimes some fields do not
> exist, and those that do not exist do not want to write ES, how to deal with
> this situation?
> For example: the source data has 3 fields, a, b, c
> insert into table2
> select
> a,b,c
> from table1
> When b=null, only hope to write a and c
> When c=null, only hope to write a and b
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
