PDGGK opened a new pull request, #11022: URL: https://github.com/apache/paimon/pull/11022
### Purpose Fixes #5092. With Maxwell's `output_ddl` enabled, DDL events such as `table-alter` and `table-create` go to the same Kafka topic as the data changes, unless `ddl_kafka_topic` is set. These events carry `def`, `old` and `sql`, and have no `data` field. `MaxwellRecordParser.extractRecords` therefore throws `Invalid maxwell-json format: missing 'data' field.`. The Kafka sync job fails, and after a restart it fails again on the same offset. Schema discovery at job start (`buildSchema`) fails the same way when it reaches such an event. The canal-json, aliyun-json and aws-dms-json parsers already skip their DDL records through `isDDL()`. This PR does the same for Maxwell, using the six DDL event types Maxwell emits: `database-create`, `database-alter`, `database-drop`, `table-create`, `table-alter` and `table-drop`. `buildSchema` returns null for them, so schema discovery moves on to the next record, and `extractRecords` returns no records. The reporter's earlier PR #5099 caught every `RuntimeException` around the `data` lookup. It was closed by the stale bot without review. Checking the event type instead keeps malformed data events failing as before. ### Tests New `MaxwellRecordParserTest#testSkipDdlEvents`, parameterized over the six DDL types. For each one, `buildSchema` returns null and `flatMap` emits nothing. An insert event parsed afterwards is still emitted. - `mvn -pl paimon-flink/paimon-flink-cdc -am -Pflink1,fast-build -Dtest=MaxwellRecordParserTest test` on JDK 8: 6 tests pass. - With only the change to `MaxwellRecordParser` reverted, all 6 fail with `Invalid maxwell-json format: missing 'data' field.`, the error from the issue. - `spotless:check` and `checkstyle:check` pass for `paimon-flink-cdc`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
