karthik-0306 opened a new pull request, #12858:
URL: https://github.com/apache/seatunnel/pull/12858
### Purpose of this pull request
Related to #12707 (pip-installable SeaTunnel launcher).
This is the `install-plugin` piece of the launcher core that @SEZ9 outlined
on #12707. It adds `tools/seatunnel-python-sdk/seatunnel/plugin_installer.py`
with two functions:
- `select_plugins(plugin_config_path, connectors)` rewrites
`config/plugin_config` so that only the requested connectors are enabled. It
accepts short names (`jdbc`) or artifact IDs (`connector-jdbc`).
- `install_plugins(seatunnel_home, connectors, version)` checks the
distribution layout and that `bash` is available, then calls `select_plugins`
and runs `bin/install-plugin.sh <version>`.
Behaviour:
1. Section headers (`--connectors-v2--`), the license header, comments,
blank lines and line endings (LF or CRLF) are preserved.
2. Idempotent: a second run with the same connectors leaves the file
byte-identical, and the file is not rewritten.
3. If a requested connector is not listed in `plugin_config`, the request is
empty, or a name contains characters `install-plugin.sh` rejects, it raises
`PluginInstallError` and does not modify the file.
4. Everything that can fail before the script starts (missing
`bin/install-plugin.sh`, missing `config/plugin_config`, no `bash`, empty
version) is checked before `plugin_config` is touched.
5. The file is written through a temporary file plus `os.replace`, so a
failure never leaves it half written.
6. The script's output is not captured, so download progress is shown.
`SEATUNNEL_PLUGIN_DOWNLOAD_METHOD` and `SEATUNNEL_MAVEN_REPOSITORY` are passed
through unchanged.
7. `version` is required and has no default, matching the issue's "wheel
version equals engine version" rule.
**Design note:** connectors that were not requested are **disabled with a
leading `#`** (`#connector-kafka`) instead of being deleted.
`install-plugin.sh` already skips `#` lines, so the result is the same, but the
full connector list stays in the file and a later call can enable a different
connector. If the full list were deleted, a second call asking for another
connector would fail with "not found". Happy to switch to deleting the lines if
reviewers prefer.
Scope: two new files under `tools/seatunnel-python-sdk`. The helper is not
exported from `seatunnel/__init__.py`, and `install-plugin.sh`, `plugin_config`
and the engine are untouched.
### Does this PR introduce _any_ user-facing change?
No. It is an internal helper that nothing calls yet, and it is not part of
the SDK's public API.
### How was this patch tested?
Added 22 unit tests in
`tools/seatunnel-python-sdk/tests/test_plugin_installer.py`. They use a
temporary directory and mock `subprocess.run` / `shutil.which`, so no network
or real distribution is needed. They cover:
- Only the requested connectors stay enabled, and the others are disabled
with `#`
- License header, comments and section header preserved
- Idempotent second run (byte-identical, no rewrite)
- Re-enabling a connector disabled by an earlier call
- Short name vs artifact ID, duplicate names, a single string argument
- Unknown connector, empty request and invalid names raise, and the file is
left unchanged
- CRLF line endings preserved, and a last line without a trailing newline
- `install_plugins`: the config is rewritten before the script runs with
`[bash, script, version]`; non-zero exit, `OSError`, missing `bash`, missing
script, missing `plugin_config`, unknown connector and empty version all raise,
and none of these runs the script or rewrites the file
Results: all 50 SDK tests (22 new + 28 existing) pass on Python 3.9, 3.11
and 3.13 on Windows.
I also ran `install_plugins` against copies of the real
`config/plugin_config` and `bin/install-plugin.sh`. Only `#` toggles changed
compared with the original file, the section header was kept, a second run was
byte-identical, and `connector-assert-3.0.0.jar` was downloaded and passed the
script's checksum check.
Side note, not changed here: the default version in `install-plugin.sh` is
`3.0.1`, which is not on Maven Central yet (404). Only `3.0.0` is published.
The CI workflow for these tests is in #12857. This branch is based on `dev`,
so that workflow does not run here until #12857 is merged.
To run the tests (from `tools/seatunnel-python-sdk/`):
python -m pytest tests/ -v
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
