hutiefang76 opened a new issue, #12864:
URL: https://github.com/apache/seatunnel/issues/12864
### Search before asking
- [x] Checked existing issues and open/closed PRs, including the separate
default-value work in #11633 and custom CDC format in #12728.
### What happened
CDC update before-images can lose unchanged column values because membership
in `old` is checked recursively instead of within the current row's top-level
old image.
For Canal, use a four-column schema (`id INT`, `name STRING`, `description
STRING`, `weight FLOAT`) and deserialize:
```json
{"type":"UPDATE","database":"d","table":"t","ts":1,"data":[{"id":1,"name":"alpha","description":"d1","weight":5.0},{"id":2,"name":"b0","description":"d2","weight":9.0}],"old":[{"weight":4.0},{"name":"a0"}]}
```
Expected before rows: `[1, alpha, d1, 4.0]` and `[2, a0, d2, 9.0]`. Actual:
`[1, null, d1, 4.0]` and `[2, a0, d2, null]`. A changed field in another row is
incorrectly treated as changed in this row too.
The same lookup problem occurs in Maxwell when `old` contains a nested
object with the name of an unchanged top-level column:
```json
{"database":"d","table":"t","type":"update","ts":1,"data":{"id":1,"name":"alpha","description":"d1","weight":5.0},"old":{"meta":{"name":"nested"},"weight":4.0}}
```
Expected before row: `[1, alpha, d1, 4.0]`; actual: `[1, null, d1, 4.0]`.
Both cases fail in local format-level regressions. Explicit nulls in the
current old image already pass and must remain null. I will limit lookup to
each matching old row and its direct fields, with regression coverage in the
existing Canal and Maxwell suites.
### SeaTunnel Version
Current dev, e2957d27afa29d4de4324cf642450757d865b902.
### SeaTunnel Config
Direct Canal/Maxwell format API, using the four-column schema above. No
engine or external connector is required for the reproduction.
### Running Command
JUnit format-level deserialization tests on JDK 17; no engine job was run.
### Error Exception
No exception: the UPDATE_BEFORE rows silently contain incorrect null fields,
as shown above.
### Zeta or Flink or Spark Version
Not involved.
### Java or Scala Version
OpenJDK 17.0.18.
### Are you willing to submit PR?
- [x] Yes, I am willing to submit a PR.
### Code of Conduct
- [x] I agree to follow the project's Code of Conduct.
<sub>Reproduction and report prepared by an AI assistant for
@hutiefang76.</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
