zjureel opened a new pull request, #423: URL: https://github.com/apache/flink-table-store/pull/423
The column type may be updated when table store read data from file meta, it should convert the data from old column type to new type. For example, there're four fields with types as follows 1->a int, 2->b int, 3->c int, 4->d int After column type evolution, the new types are as follows 1->c bigint, 2->a float, 3->d double, 4->d bigint When table store reads data from file meta according to the old column type, it should convert the data to the new column type. The main changes in this PR are as followed 1. Added `DataValueConverter` to convert value from old `LogicalType` to new type. 2. Added `SchemaEvolutionUtil.createConvertMapping` to create converter mapping between table and data fields 3. Updated `FieldStatsArraySerializer` to get new type value from old type value 4. Updated `AppendOnlyFileStoreScan` and `KeyValueFileStoreScan` to create converter mapping The main tests are as followed 1. Added `AppendOnlyTableColumnTypeFileMetaTest` to test column type evolution for file meta in append-only table 2. Added `ChangelogValueCountColumnTypeFileMetaTest` to test column type evolution for file meta in changelog value count table 3. Added `ChangelogWithKeyTableColumnTypeFileMetaTest` to test column type evolution for file meta in changelog with key table -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
