goutamadwant opened a new issue, #12363:
URL: https://github.com/apache/seatunnel/issues/12363
### Search before asking
- [x] Searched existing issues and open/closed PRs for Replace schema,
ReplaceTransform and related schema-restoration changes. No matching
field-rebinding fix was found.
### What happened
`ReplaceTransform` initializes configured field indexes in its constructor,
but does not rebuild them when the input schema changes. The shared transform
lifecycle refreshes the schema after restoration, supported column events and
input-schema handoff; Replace continues using positions from the old layout.
This can leave the configured field unchanged and apply replacement to an
unrelated field. The problem concerns pipelines whose supported schema
lifecycle changes physical column positions, including MySQL CDC with a
leading-column addition followed by savepoint/restore.
### SeaTunnel Version
Reproduced on `dev`, commit `b37af3a9bf634735cabe4908014c2d390b288558`
(3.0.0-SNAPSHOT). Released versions were not separately tested.
### SeaTunnel Config
The relevant transform configuration is:
```hocon
Replace {
replace_fields = ["name", "title"]
pattern = "before"
replacement = "after"
}
```
### Reproduction
1. Construct a Replace transform with the original schema `id INT, name
STRING, title STRING` and the configuration above.
2. Obtain its produced catalog table, as in normal initialization.
3. Deliver `RestoreTableSchemaEvent` with the same table identity and
reordered columns `title STRING, id INT, name STRING`.
4. Immediately map the row `["before title", 1, "before name"]`.
5. Assert output `["after title", 1, "after name"]`.
On the unchanged implementation, `title` remains `"before title"` because
the old indexes still point at positions 1 and 2. The same defect is reproduced
by calling `setInputCatalogTable` with a reordered schema before mapping the
next row.
The implementation adds these regressions to the existing
`ReplaceTransformTest` as `testRestoreSchemaRebindsFieldsBeforeNextRow` and
`testInputSchemaHandoffRebindsFieldsBeforeNextRow`. To reproduce against the
original behavior, apply those tests without the production hook change.
### Running Command
With the regression tests present and the original production implementation:
```sh
./mvnw -B -pl seatunnel-transforms-v2 -am package \
'-Dtest=ReplaceTransformTest#testRestoreSchemaRebindsFieldsBeforeNextRow+testInputSchemaHandoffRebindsFieldsBeforeNextRow'
\
-Dsurefire.failIfNoSpecifiedTests=false -Dskip.ui=true
```
### Error Exception / before and after
Both baseline regressions fail with `before title` instead of `after title`.
An exception is not required in normal operation: the transform can produce
incorrect field values using stale indexes.
- Before: field bindings belong to the constructor-time schema, even after
the lifecycle supplies a different layout.
- After: configured names are resolved against the refreshed schema before
the next row is processed.
- After: a configured target absent from the refreshed schema fails with the
existing field-validation error instead of processing rows through stale
bindings.
### Proposed fix and advantages
Call the existing `initializeFieldIndexes()` from `transformTableSchema()`.
The shared base already invokes this hook for schema refresh, restore and input
handoff, so no new event-dispatch mechanism is needed.
This keeps replacement attached to the intended fields through schema
evolution and recovery. Rebinding occurs at schema refresh, not per row. The
patch adds no dependencies or configuration options and preserves existing
input-row isolation, row metadata, literal/regex behavior and null handling.
### Breaking changes / compatibility
No public API, checkpoint-format, option-name or default-value changes.
Previously incorrect output after column-position changes is corrected. Missing
configured fields now fail during schema refresh; this is not suppressed by a
row-level skip policy. Users should retain or update configured target names
when changing schemas. EN/ZH documentation and compatibility notes are included.
The fix does not add general Flink/Spark schema-evolution support, change
non-string replacement conversion, or repair unrelated chained-rename behavior.
### Validation and Java or Scala Version
- Java 8 baseline: fresh package reactor, two selected regressions fail.
- Java 11 baseline: the same two regressions fail when executing Java
8-compiled baseline classes under Java 11; prerequisite artifacts came from
installed Maven snapshots, not a fresh Java 11 baseline reactor.
- Fixed full transform suites: 1,165 tests on each of Java 8u172 and Java
11.0.19, zero failures/errors/skips.
- Actual MySQL 8.0.43 E2E: snapshot, live `ADD prefix FIRST`, updated source
data, savepoint, stopped-source DML and restore all pass. Two replacement
fields and untouched guard fields are asserted.
- The E2E used SeaTunnel engine Java 8u342 with a Java 11 host driver.
Runtime JARs were freshly built and hash-checked; host test-harness
dependencies included installed snapshots. It was not an engine-Java-11
recovery test or a live baseline-before/fixed-after pair.
- Combined validation with the separate array-extrema fix passed the
310-module Java 11 build with tests skipped; full repository test execution is
not claimed.
### Are you willing to submit PR?
Yes, the implementation and regression coverage are ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
