chovy-3012 commented on PR #12817:
URL: https://github.com/apache/seatunnel/pull/12817#issuecomment-6231551314
Thanks for the review! I've added `compress_codec` and
`archive_compress_codec` as optional options to the 4 file source factories
that were missing them:
- **Obs** (`ObsFileSourceFactory`)
- **Sftp** (`SftpFileSourceFactory`)
- **Smb** (`SmbFileSourceFactory`)
- **Jindo-Oss** (`OssFileSourceFactory` — added `archive_compress_codec`;
`compress_codec` was already present)
Both options are consumed by shared read strategies in `connector-file-base`
(`AbstractReadStrategy`, `CsvReadStrategy`, `JsonReadStrategy`,
`TextReadStrategy`) but were not declared in these factories' `optionRule()`,
causing `--check` / `--dry-run` to reject valid configs as unknown keys.
**Reproduced stack trace** (running `--check` with an `ObsFile` config):
```
Exception in thread "main"
org.apache.seatunnel.core.starter.exception.ConfigCheckException:
Static analysis failed: ErrorCode:[API-02], ErrorDescription:[Option item
validate failed]
- Connector 'ObsFile' has unknown option keys: [archive_compress_codec,
compress_codec].
Please check for typos.
at
org.apache.seatunnel.core.starter.seatunnel.command.SeaTunnelConfValidateCommand.execute(SeaTunnelConfValidateCommand.java:196)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at
org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:41)
Caused by:
org.apache.seatunnel.api.configuration.util.OptionValidationException:
ErrorCode:[API-02], ErrorDescription:[Option item validate failed]
- Connector 'ObsFile' has unknown option keys: [archive_compress_codec,
compress_codec].
Please check for typos.
at
org.apache.seatunnel.api.configuration.util.ConfigValidator.validateUnknownKeys(ConfigValidator.java:102)
at
org.apache.seatunnel.core.starter.seatunnel.command.SeaTunnelConfValidateCommand.validatePluginConfigs(SeaTunnelConfValidateCommand.java:325)
at
org.apache.seatunnel.core.starter.seatunnel.command.SeaTunnelConfValidateCommand.execute(SeaTunnelConfValidateCommand.java:162)
```
Changes:
- Added `.optional()` declarations to 4 factories
- Added regression assertions to 4 factory test classes
- Added `archive_compress_codec` docs to `SmbFile` (en+zh) and `OssFile`
(en+zh)
All 12 file source factories now consistently declare both options. Unit
tests pass on all 4 affected modules.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
