goutamadwant opened a new issue, #12077:
URL: https://github.com/apache/seatunnel/issues/12077
### Search before asking
- [x] I searched existing issues and pull requests for transform dependency
cycles, unresolved `plugin_input` values, partial multi-input resolution, and
parser retry loops. I found no issue or pull request implementing this fix.
### What happened
The Zeta transform resolver has two related correctness failures on the
current `dev` branch:
1. A cycle between transform outputs never makes progress. The runtime
parser and `--dry-run connect` repeatedly move the same transform
configurations to the end of a queue, so submission or validation does not
return an error.
2. A valid out-of-order multi-input transform is considered ready when only
one declared input exists. Missing inputs are filtered out, and the transform
is created from the remaining inputs. This silently removes an edge from the
configured DAG.
For example, when `t2` declares `plugin_input = [src, t1]` before `t1` is
declared, the current parser creates `t2` from `src` alone instead of waiting
for `t1`. This can change job results without reporting an invalid
configuration.
Expected behavior:
- Cyclic or otherwise unresolvable transform dependencies should fail
promptly with a contextual configuration error.
- A transform should be created only after every distinct declared input ID
is available.
- Repeated input IDs should remain compatible with the existing action and
catalog-table deduplication behavior.
- Existing simple-graph and explicit empty-input fallback behavior should
remain compatible.
- Reverse-declared valid DAGs should not require quadratic retry scans or
repeated plugin discovery.
### SeaTunnel Version
`dev` at `9ebb27e8f92059dee76d71029adbfbaa1c099285`.
### SeaTunnel Config
Cycle reproduction:
```conf
env {
execution.parallelism = 1
job.mode = BATCH
}
source {
FakeSource {
plugin_output = src
schema = { fields { name = string } }
}
}
transform {
sql {
plugin_input = [t2]
plugin_output = t1
query = "select 1 from dual"
}
sql {
plugin_input = [t1]
plugin_output = t2
query = "select 1 from dual"
}
}
sink {
console { plugin_input = [src] }
}
```
Partial-input reproduction:
```conf
env {
execution.parallelism = 1
job.mode = BATCH
}
source {
FakeSource {
plugin_output = src
schema = { fields { name = string } }
}
}
transform {
sql {
plugin_input = [src, t1]
plugin_output = t2
query = "select * from dual"
}
sql {
plugin_input = [src]
plugin_output = t1
query = "select * from dual"
}
}
sink {
console { plugin_input = [t2] }
}
```
### Running Command
```shell
./bin/seatunnel.sh --config /path/to/reproduction.conf -e local
```
The same resolver is exercised by `--dry-run connect`.
### Error Exception
The cycle case produces no exception. The process remains in transform
parsing until it is externally terminated.
The partial-input case also produces no exception. Inspection of the
generated action graph shows that `t2` has only the source action as its
upstream; the declared `t1` upstream is absent.
In deterministic runs on the current `dev` code, the cycle remained active
until terminated after 50 seconds on Java 8 and 35 seconds on Java 11. The
partial-input action-graph reproduction also produced the same missing edge on
both JDKs.
### Zeta or Flink or Spark Version
Zeta engine from the `dev` revision above.
### Java or Scala Version
- Java 8: `1.8.0_172`
- Java 11: `11.0.19`
### Proposed fix and compatibility contract
- Validate transform-to-transform dependencies with a deterministic graph
traversal and report a concrete cycle path.
- Resolve transforms through an indexed ready queue so each dependency edge
is processed once and plugin discovery begins only after all inputs are ready.
- Preserve the legacy queue's evaluation order and generated action names
for existing valid configurations.
- Retain the intentional single-transform and terminal empty-input
compatibility fallbacks, while failing ambiguous unresolved graphs.
- Add runtime parser, graph validation, and connect dry-run coverage for
cycles, self-cycles, partial inputs, duplicate inputs, reverse-declared chains,
fallback compatibility, and action-index boundaries.
### Are you willing to submit a PR?
- [x] Yes, I am willing to submit a PR.
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
