waterWang opened a new pull request, #11634:
URL: https://github.com/apache/seatunnel/pull/11634
### What problem does this PR solve?
Issue Number: close #11632
### Problem Summary
When a source connector using the JSON format (Kafka, Pulsar, HTTP, etc.)
configures `schema { columns = [ { name = "GROUPID", type = "int", defaultValue
= 0 } ] }`, the `defaultValue` is parsed and stored on the `Column` but never
applied during deserialization: if the JSON message is missing the field or
contains an explicit `null`, the row field stays `null` instead of the
configured default value.
### Root Cause
The `defaultValue` is lost on the way from `CatalogTable` to the row
converter:
1. `CatalogTable.getSeaTunnelRowType()` calls `toPhysicalRowDataType()`,
which creates a `SeaTunnelRowType` with only `fieldNames[]` and `fieldTypes[]`
— the `defaultValue` is dropped.
2. `JsonToRowConverters` builds converters from `SeaTunnelRowType` and has
no access to `Column` metadata.
3. When `convertField()` encounters a `null` or missing field, it returns
`null` without consulting the default value.
### What Changed
1. **`JsonToRowConverters`**: Added an optional `defaultValues` array
parameter. When a field is `null`/missing and a non-null default value exists
for that column, the default value is used instead of `null`.
2. **`JsonDeserializationSchema`**: Extracts default values from
`CatalogTable.getTableSchema().getColumns()` (filtering to physical columns to
match `toPhysicalRowDataType()` behavior) and passes them to the converters.
### How was this tested?
- Existing tests pass without modification (the `defaultValues` array is
`null` by default, preserving backward compatibility).
- Manually verified that the scenario from the issue (Kafka JSON source with
`defaultValue = 0` / `defaultValue = "unknown"`) now correctly applies defaults
when fields are missing or `null`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
