nzw921rx opened a new issue, #12781: URL: https://github.com/apache/seatunnel/issues/12781
### Search before asking - [X] I searched in the [issues](https://github.com/apache/seatunnel/issues) and found nothing similar. ### SeaTunnel Version dev (any version that contains #11095) ### Minimal Reproduction Transforms that support `table_transform` (multi-table rules) still require their transform-specific options **at the transform level**, so a configuration that only uses `table_transform` entries is rejected before the transform is created: ```hocon transform { FieldMapper { plugin_input = "fake" plugin_output = "fake1" table_transform = [{ table_path = "test.xyz" field_mapper = { name = name1 } }] } } ``` ### Expected Behavior The job starts; `test.xyz` uses the `field_mapper` from its `table_transform` entry, other tables are unchanged (the docs state that a `table_transform` entry takes precedence and the outer rules do not apply to that table). ### Actual Behavior The job fails deterministically before any data is processed: ``` option: 'field_mapper' type: required constraint: required option is not configured ``` `FactoryUtil#createAndPrepareMultiTableTransform` validates only the transform-level options with `ConfigValidator.of(options).validate(factory.optionRule())`, so any top-level `required(...)` option that the user prefers to set per table (inside `table_transform`) fails the whole job. ### Affected transforms Verified by running `ConfigValidator` against every transform factory that declares `TransformCommonOptions.MULTI_TABLES`, with a `table_transform`-only configuration. 15 of 24 factories fail this way: | Transform | Required transform-level option(s) with no default | | --- | --- | | `FieldMapper` | `field_mapper` | | `Replace` | `replace_fields`, `pattern`, `replacement` | | `Split` | `separator`, `split_field`, `output_fields` | | `Sql` | `query` | | `Calcite` | `sql` | | `Metadata` | `metadata_fields` | | `JsonPath` | `columns` | | `DataValidator` | `field_rules` | | `DefineSinkType` | `columns` | | `Python` | `columns` | | `FieldEncrypt` | `fields`, `key` | | `TableMerge` | `table` | | `RegexExtract` | `source_field`, `regex_pattern`, `output_fields` | | `TextChunk` | `text_field` | | `DynamicCompile` | `compile_language` | Factories that only declare `optional`/`conditional` options are unaffected: `FieldRename`, `TableRename`, `RowKindExtractor`, `TableFilter`, `Embedding`, `LLM`. ### Additional context - Regression from #11095 ("Migrate transforms validation to declarative OptionRule"), which turned previously runtime-checked options into top-level `required` rules. - The same rule migration also produced a stricter variant for three transforms whose option groups are `exclusive`: `Filter` (already reported), `Copy` and `FilterRowKind` (separate report to follow). - `TextChunkTransformFactory` already validates each `table_transform` entry (`TableTransformRulesValidator`), but its top-level `required(text_field)` still blocks entries-only configurations. ### Suggested fix - Validate the options that can be overridden per table against the effective configuration: either move the requirement into the transform (where the per-table config is known, like `TextChunkTransformFactory` does for its table entries), or keep the top-level rule but make it conditional on `table_transform` being absent. - Add regression tests that validate the rule and build the transform with only `table_transform` configured, for each affected transform. ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
