[GitHub] [inlong] yunqingmoswu opened a new issue, #7477: [Bug][Sort] Fix the metadata of table write error for canal-json

[via GitHub]

yunqingmoswu opened a new issue, #7477:
URL: https://github.com/apache/inlong/issues/7477

   ### What happened
   
   The metadata of table write error for canal-json.
   <img width="516" alt="image" 
src="https://user-images.githubusercontent.com/44659300/222085818-3b637989-4619-4352-96f1-0b50da04210e.png";>
   
   
   ### What you expected to happen
   
   The metadata of table write right for canal-json.
   
   ### How to reproduce
   
   It can reproduce using the follow flink sql 
   ```
   CREATE TABLE `input`(
       PRIMARY KEY (`id`) NOT ENFORCED,
       `database_wedata_di` STRING METADATA FROM 'meta.database_name' VIRTUAL,
       `table_wedata_di` STRING METADATA FROM 'meta.table_name' VIRTUAL,
       `type_wedata_di` STRING METADATA FROM 'meta.op_type' VIRTUAL,
       `ts_wedata_di` TIMESTAMP_LTZ(3) METADATA FROM 'meta.op_ts' VIRTUAL,
       `processing_time_wedata_di` AS PROCTIME(),
       `id` INT,
       `name` STRING,
       `PROCESS_TIME` AS PROCTIME())
       WITH (
       'debezium.snapshot.locking.mode' = 'none',
       'scan.startup.mode' = 'initial',
       'debezium.binary.handling.mode' = 'base64',
       'inlong.metric.labels' = 'groupId=1&streamId=1&nodeId=1',
       'connector' = 'mysql-cdc-inlong',
       'hostname' = 'localhost',
       'database-name' = 'test',
       'port' = '3306',
       'scan.incremental.snapshot.enabled' = 'true',
       'server-time-zone' = 'Asia/Shanghai',
       'username' = 'root',
       'password' = 'password',
       'table-name' = 'test_db.test_table'
   );
   CREATE TABLE `output`(
       `id` INT,
       `name` STRING,
       `TABLE_NAME_wedata_di` STRING METADATA FROM 'value.table',
       `OP_TS_wedata_di` TIMESTAMP_LTZ(3) METADATA FROM 'value.event-timestamp',
       `DATABASE_NAME_wedata_di` STRING METADATA FROM 'value.database',
       `OP_TYPE_wedata_di` STRING METADATA FROM 'value.type')
       WITH (
       'dirty.side-output.field-delimiter' = ',',
       'dirty.side-output.format' = 'csv',
       'dirty.side-output.log-tag' = '脏数据][test_table',
       'dirty.identifier' = 'test_table',
       'dirty.side-output.enable' = 'true',
       'dirty.side-output.labels' = 'SYSTEM_TIME=${SYSTEM_TIME},test_table',
       'dirty.side-output.connector' = 'log',
       'dirty.ignore' = 'true',
       'inlong.metric.labels' = 'groupId=1&streamId=1&nodeId=2',
       'topic' = 'test_table',
       'properties.bootstrap.servers' = 'localhost:9092',
       'connector' = 'kafka-inlong',
       'canal-json-inlong.ignore-parse-errors' = 'true',
       'canal-json-inlong.map-null-key.mode' = 'DROP',
       'format' = 'canal-json-inlong',
       'canal-json-inlong.encode.decimal-as-plain-number' = 'true',
       'canal-json-inlong.timestamp-format.standard' = 'SQL',
       'canal-json-inlong.map-null-key.literal' = 'null'
   );
   INSERT INTO `output`
       SELECT 
       `id` AS `id`,
       `name` AS `name`,
       `table_wedata_di` AS `TABLE_NAME_wedata_di`,
       `ts_wedata_di` AS `OP_TS_wedata_di`,
       `database_wedata_di` AS `DATABASE_NAME_wedata_di`,
       `type_wedata_di` AS `OP_TYPE_wedata_di`
       FROM `input` ;
   ```
   
   ### Environment
   
   _No response_
   
   ### InLong version
   
   master
   
   ### InLong Component
   
   InLong Sort
   
   ### Are you willing to submit PR?
   
   - [X] Yes, I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@inlong.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org