[
https://issues.apache.org/jira/browse/FLINK-32712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yuan updated FLINK-32712:
-------------------------
Description:
The BEFORE field in ogg-json can be configured to contain no table fields, as
in the ogg-json example below.
For this case, the current version of ogg-json format, sends downstream a
Record of type UPDATE_BEFORE with null fields, eg. `Record(-U, null, null,
null,...)`, which obviously makes no sense.
{code:java}
{
"table": "ZBZZZ",
"op_type": "U",
"op_ts": "2023-07-20 21:45:34.860817",
"current_ts": "2023-07-21T05:45:36.615000",
"pos": "00002564940142073691",
"before": {},
"after": {
"ID": 1461242,
"PROPERTY_01": "tc",
"PROPERTY_02": null,
"PROPERTY_03": null,
"PROPERTY_04": "K",
"PROPERTY_05": "5",
"PROPERTY_06": null,
"PROPERTY_07": null,
"PROPERTY_08": null,
"PROPERTY_09": null,
"PROPERTY_10": null
}
}{code}
For this case, ogg-json format should not send a Record of type UPDATE_BEFORE
downstream。 Worse, it can sometimes cause the flink job to crash out.
was:
The BEFORE field in ogg-json can be configured to contain no table fields, as
in the ogg-json example below.
For this case, the current version of ogg-json format, sends downstream a
Record of type UPDATE_BEFORE with null fields, eg. Record(-U, null, null,
null,...), which obviously makes no sense.
{code:java}
{
"table": "ZBZZZ",
"op_type": "U",
"op_ts": "2023-07-20 21:45:34.860817",
"current_ts": "2023-07-21T05:45:36.615000",
"pos": "00002564940142073691",
"before": {},
"after": {
"ID": 1461242,
"PROPERTY_01": "tc",
"PROPERTY_02": null,
"PROPERTY_03": null,
"PROPERTY_04": "K",
"PROPERTY_05": "5",
"PROPERTY_06": null,
"PROPERTY_07": null,
"PROPERTY_08": null,
"PROPERTY_09": null,
"PROPERTY_10": null
}
}{code}
For this case, ogg-json format should not send a Record of type UPDATE_BEFORE
downstream。 Worse, it can sometimes cause the flink job to crash out.
> Record with RowKind of type -U should not be sent downstream when the before
> field is empty in ogg-json which op_type is U .
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-32712
> URL: https://issues.apache.org/jira/browse/FLINK-32712
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.17.1
> Reporter: yuan
> Priority: Major
> Labels: pull-request-available
>
> The BEFORE field in ogg-json can be configured to contain no table fields, as
> in the ogg-json example below.
> For this case, the current version of ogg-json format, sends downstream a
> Record of type UPDATE_BEFORE with null fields, eg. `Record(-U, null, null,
> null,...)`, which obviously makes no sense.
> {code:java}
> {
> "table": "ZBZZZ",
> "op_type": "U",
> "op_ts": "2023-07-20 21:45:34.860817",
> "current_ts": "2023-07-21T05:45:36.615000",
> "pos": "00002564940142073691",
> "before": {},
> "after": {
> "ID": 1461242,
> "PROPERTY_01": "tc",
> "PROPERTY_02": null,
> "PROPERTY_03": null,
> "PROPERTY_04": "K",
> "PROPERTY_05": "5",
> "PROPERTY_06": null,
> "PROPERTY_07": null,
> "PROPERTY_08": null,
> "PROPERTY_09": null,
> "PROPERTY_10": null
> }
> }{code}
> For this case, ogg-json format should not send a Record of type UPDATE_BEFORE
> downstream。 Worse, it can sometimes cause the flink job to crash out.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
