nzw921rx opened a new issue, #12809:
URL: https://github.com/apache/seatunnel/issues/12809
We need community help to bring the transform plugin documents up to the
connector documentation
standard: a complete options table, one section per option, the multi-table
semantics, runnable
examples and English/Chinese kept in sync.
Please comment on this issue to claim one or more documents (one plugin per
pull request is
preferred).
## Background
Connector documents already follow a documented standard
(`docs/en/connectors/source/Kafka.md`,
`docs/en/connectors/sink/Jdbc.md`). Most **transform** documents are still
the early placeholder
version — a four column options table and a single example — which causes
concrete user-facing
problems:
* the multi-table options `table_transform`, `table_match_regex` and
`rule_match_mode` are missing from
almost every transform document, although every transform supports them
(see
`docs/en/transforms/transform-multi-table.md`);
* deprecated aliases are not mentioned, for example `fields` as the fallback
of `include_fields` in
`Filter`;
* option types, required flags and default values are missing or outdated
(compare
`seatunnel-transforms-v2/src/main/java/.../<Plugin>TransformConfig.java`
and
`<Plugin>TransformFactory#optionRule()`);
* options that no longer exist are still documented, and newly added options
are not documented at all
(`separators`, `skip_empty_text`, `column_rule.sink_type`, ...).
## Documentation Template
Every rewritten document has to follow this structure, in English **and**
Chinese:
1. `## Support Those Engines` — the engines that really run the plugin
(SeaTunnel Zeta / Flink / Spark).
2. `## Description` — what the plugin does and when to use it (3-5
sentences).
3. `## Key Features` — optional, short bullet list.
4. `## Options` — five columns `Name | Type | Required | Default |
Description`, including
`table_transform`, `table_match_regex` and `rule_match_mode` when the
plugin supports multi-table
configuration, plus `common options`.
5. One `### <option> [type]` section per option: meaning, accepted values,
default value, constraints
and how it interacts with the other options.
6. `## Multi-Table Configuration` — entries of `table_transform` take
precedence over the transform
level rule, every entry has to be a complete single table rule, and when
only `table_transform` is
configured the tables without an entry stay unchanged. Add a runnable
example.
7. `## Example` — input table, output table and the complete HOCON, ideally
covering the three usages
(transform level rule only / transform level rule plus per-table rules /
per-table rules only).
8. Option names, types and defaults are copied from the code, and `docs/en`
/ `docs/zh` stay in sync.
The options table and a per-option section look like this:
````markdown
## Options
| Name | Type | Required | Default |
Description
|
|-------------------|--------------------|----------|---------------|---------------------------------------------------------------------------------|
| include_fields | Array<String> | no | - |
Fields to keep. All other fields are removed.
|
| exclude_fields | Array<String> | no | - |
Fields to remove. All other fields are kept.
|
| table_match_regex | String | no | `.*` | Regex
of the table paths that the transform level rule applies to. |
| table_transform | Array<Map> | no | - |
Per-table rules, see [Multi-Table Transform](transform-multi-table.md).
|
| rule_match_mode | String | no | `FIRST_MATCH` | How
multiple `table_transform` entries of the same table are evaluated. |
| common options | - | no | - |
Transform plugin common parameters, see [Transform
Plugin](common-options/common-options.md). |
### include_fields [Array<String>]
The fields to keep, for example `include_fields = [name, card]` keeps only
those two fields.
````
## Transforms Open for Claim
| Type | Transform | Document
| Contributor | Status | PR |
| --------- | ---------------- |
---------------------------------------------- | ----------- | ------ | --- |
| Transform | Filter | `docs/{en,zh}/transforms/filter.md`
| | Todo | |
| Transform | Copy | `docs/{en,zh}/transforms/copy.md`
| | Todo | |
| Transform | FilterRowKind | `docs/{en,zh}/transforms/filter-rowkind.md`
| | Todo | |
| Transform | FieldMapper | `docs/{en,zh}/transforms/field-mapper.md`
| | Todo | |
| Transform | Replace | `docs/{en,zh}/transforms/replace.md`
| | Todo | |
| Transform | Split | `docs/{en,zh}/transforms/split.md`
| | Todo | |
| Transform | Metadata | `docs/{en,zh}/transforms/metadata.md`
| | Todo | |
| Transform | JsonPath | `docs/{en,zh}/transforms/jsonpath.md`
| | Todo | |
| Transform | DataValidator | `docs/{en,zh}/transforms/data-validator.md`
| | Todo | |
| Transform | DefineSinkType |
`docs/{en,zh}/transforms/define-sink-type.md` | | Todo | |
| Transform | TextChunk | `docs/{en,zh}/transforms/text-chunk.md`
| | Todo | |
| Transform | RegexExtract | `docs/{en,zh}/transforms/regexextract.md`
| | Todo | |
| Transform | TableMerge | `docs/{en,zh}/transforms/table-merge.md`
| | Todo | |
| Transform | FieldEncrypt | `docs/{en,zh}/transforms/encrypt.md`
| | Todo | |
| Transform | DynamicCompile |
`docs/{en,zh}/transforms/dynamic-compile.md` | | Todo | |
| Transform | Python | `docs/{en,zh}/transforms/python.md`
| | Todo | |
| Transform | FieldRename | `docs/{en,zh}/transforms/field-rename.md`
| | Todo | |
| Transform | TableRename | `docs/{en,zh}/transforms/table-rename.md`
| | Todo | |
| Transform | RowKindExtractor |
`docs/{en,zh}/transforms/rowkind-extractor.md` | | Todo | |
| Transform | TableFilter | `docs/{en,zh}/transforms/table-filter.md`
| | Todo | |
| Transform | Embedding | `docs/{en,zh}/transforms/embedding.md`
| | Todo | |
| Transform | LLM | `docs/{en,zh}/transforms/llm.md`
| | Todo | |
## Note
This issue covers the plugin documents only. The cross-plugin documents
`overview.md`, `recipes.md`,
`transform-faq.md`, `transform-multi-table.md` and
`multi-table-transform-and-join-boundary.md` do not
follow the plugin template above and are out of scope here; maintainers will
handle them separately.
`docs/{en,zh}/transforms/sql.md` and `docs/{en,zh}/transforms/calcite.md`
are the reference quality
level and are excluded from this issue. `sql-functions.md`, `sql-udf.md` and
`calcite-udf.md` are
already detailed; please open a separate issue if something is missing there.
## How to Contribute
1. **Pick a document**: choose one row from the table above (one plugin per
pull request is preferred).
2. **Claim the task**: comment on this issue, for example `I would like to
work on Filter`, and update
the `Contributor`/`Status` columns if you have permission to edit the
issue.
3. **Rewrite both languages**: `docs/en/...` and `docs/zh/...` in the same
pull request, following
[the documentation format
specification](docs/en/developer/docs-format-specification.md) for the
admonitions (`:::tip`, `:::info`, `:::caution`).
4. **Reference the code**: mention the `*TransformConfig` /
`*TransformFactory#optionRule()` you
compared against in the pull request description, so reviewers can verify
the option names, types
and defaults.
5. **Submit the pull request**: open it against `dev`, use the title
`[Docs][Transform-V2] <description>` and link it back to this issue.
Please keep code changes out of
the documentation pull request.
Thank you for your contribution.
## Code of Conduct
I agree to follow this project's Code of Conduct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
