[
https://issues.apache.org/jira/browse/FLINK-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonard Xu updated FLINK-18758:
-------------------------------
Description:
Currently debezium json format wrapper the json format to
serialize/deserialize the debezuim json format data.
But debezium json format has its own data type which consists of Literal Type
and Semantic Type[1], i.g. Date type in debezium is an integer which represents
the number of days since epoch rather than a string with 'yyyy-MM-dd' pattern.
{code:java}
{ "schema":{
"fields":[
{
"fields":[
{
"type":"int32", //Literal Type
"optional":false,
"name":"io.debezium.time.Date", //semantic Type
"version":1,
"field":"order_date"
},
{ "type":"int32",
"optional":false,
"field":"quantity"
}
]
}
]
},
"payload":{
"before":null,
"after":{
"order_date":16852, // Literal Value, the number of days since epoch
"quantity":1
},
"op":"c",
"ts_ms":1596081813474
}
} {code}
I think we need obtain the debezuim data type from schema information and then
serialize/deserialize the data in payload.
[1][https://debezium.io/documentation/reference/1.2/connectors/mysql.html]
was:
Currently debezium json format wrapper the json format to
serialize/deserialize the debezuim json format data.
But debezium json format has its own data type which consists of Literal Type
and Semantic Type[1], i.g. Date type in debezium is the an integer which
represents the number of days since epoch rather than a string with
'yyyy-MM-dd' pattern.
{code:java}
{ "schema":{ "fields":[ { "fields":[ {
"type":"int32",
"optional":false,
"name":"io.debezium.time.Date",
"version":1,
"field":"order_date"
},
{ "type":"int32",
"optional":false,
"field":"quantity"
}
]
}
]
},
"payload":{
"before":null,
"after":{
"order_date":16852,
"quantity":1
},
"op":"c",
"ts_ms":1596081813474
}
} {code}
I think we need obtain the debezuim data type from schema information and then
serialize/deserialize the data in payload.
[1][https://debezium.io/documentation/reference/1.2/connectors/mysql.html]
> Support debezium data type in debezium format
> ----------------------------------------------
>
> Key: FLINK-18758
> URL: https://issues.apache.org/jira/browse/FLINK-18758
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Reporter: Leonard Xu
> Priority: Major
>
> Currently debezium json format wrapper the json format to
> serialize/deserialize the debezuim json format data.
> But debezium json format has its own data type which consists of Literal Type
> and Semantic Type[1], i.g. Date type in debezium is an integer which
> represents the number of days since epoch rather than a string with
> 'yyyy-MM-dd' pattern.
> {code:java}
> { "schema":{
> "fields":[
> {
> "fields":[
> {
> "type":"int32", //Literal Type
> "optional":false,
> "name":"io.debezium.time.Date", //semantic Type
> "version":1,
> "field":"order_date"
> },
> { "type":"int32",
> "optional":false,
> "field":"quantity"
> }
> ]
> }
> ]
> },
> "payload":{
> "before":null,
> "after":{
> "order_date":16852, // Literal Value, the number of days since epoch
> "quantity":1
> },
> "op":"c",
> "ts_ms":1596081813474
> }
> } {code}
>
> I think we need obtain the debezuim data type from schema information and
> then serialize/deserialize the data in payload.
> [1][https://debezium.io/documentation/reference/1.2/connectors/mysql.html]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
