junjunclub opened a new pull request, #12431:
URL: https://github.com/apache/seatunnel/pull/12431
### Purpose of this pull request
Adds `--dry-run connect` support to the Redis **source and sink** in one
slice.
Both factories now implement the existing opt-in SPI only:
- `RedisSourceFactory implements SupportSourceDryRunValidation`
- `RedisSinkFactory implements SupportSinkDryRunValidation`
`DryRunConnectValidator` and the shared SPI are **not** touched.
**What the validation does** (`RedisDryRunValidator`, shared by source and
sink):
- Single-node: opens one `Jedis` with a bounded connect/socket timeout
(10s), sends
`AUTH` (when `auth` is set), `SELECT db_num`, `PING`, and closes the
client.
- Cluster: builds a `JedisCluster` with the same bounded timeouts and a
single
attempt — the constructor initializes the slot cache, which already proves
connectivity and authentication — then reads node-level `INFO` through the
existing `JedisWrapper` and closes it.
- Every client is closed on success **and** on failure.
- Only `AUTH`, `SELECT`, `PING`, `CLUSTER SLOTS` and `INFO` are issued: no
key is
read, scanned, written or expired, and no key space is created.
- Source schemas come from the configured `schema` / `tables_configs`
through the
same path the runtime uses, so downstream plugins see the real output
tables.
No Redis value is inspected and no reader is created.
- Failures are reported as `RedisConnectorException(REDIS_CONNECTION_ERROR)`
naming
the validated target (host:port or node list) without echoing credentials.
**What a successful validation does _not_ prove** (documented in the
connector docs):
- Key existence, key-pattern matches, or that stored values match
`data_type` / `format`.
- Read or write permission on the configured keys.
- Sink field compatibility. The Redis sink resolves `key`, `value_field`,
`hash_key_field` and `hash_value_field` against the upstream fields and
falls back
to the configured name as a literal value, so a name missing from the
schema is not
a runtime failure and must not be rejected here.
- Cluster deployments with partially unreachable nodes: `JedisWrapper#info`
succeeds
as long as one node answers.
**One deliberate divergence from the runtime client, for reviewers:**
`RedisParameters#buildJedis()` is not reused. On the single-node path it
calls
`jedis.aclSetUser(user)`, which mutates server-side ACLs — not acceptable in
a dry
run. The validation therefore mirrors the runtime `AUTH` only and never
applies the
`user` option, in either mode. A deployment that relies on `user` is not
fully proven
by `--dry-run connect`; this is stated in the docs. The runtime path is
unchanged.
### Does this PR introduce _any_ user-facing change?
Yes, for `--dry-run connect` only:
- Redis source and sink are now reported as `VALIDATED` instead of
`SKIPPED`, and a
configuration pointing at an unreachable or misauthenticated Redis now
fails the
dry run instead of passing silently.
- `--dry-run connect` may now open a short-lived connection to Redis. No job
submission, reader, writer or save-mode logic is involved, and normal job
execution
is unchanged.
- Docs: EN/ZH Redis source and sink pages get a "Connectivity dry-run"
section, and
the Layer 1 support matrix in `docs/{en,zh}/engines/zeta/user-command.md`
gains a
Redis row. _(pending in this draft)_
### How was this patch tested?
`RedisDryRunValidatorTest` (new, Mockito construction mocking — no server
required):
- single-node success: `AUTH` → `SELECT` → `PING` → `close` in order, plus
`verifyNoMoreInteractions`, which proves no key command is issued;
- single-node without `auth`: no authentication is attempted;
- authentication failure (`WRONGPASS`): reported with the target in the
message, and
the client is still closed;
- connection failure: same, with the `JedisConnectionException` preserved as
cause;
- cluster: asserts the bounded constructor arguments (10s/10s, 1 attempt,
password)
and that the client is closed;
- cluster with no reachable node: reported, client closed.
`RedisFactoryTest` (extended): `inferSchemaForDryRun` returns the configured
columns
for a single table and one table per `tables_configs` entry — passing
without any
Redis instance, which is what proves no value is read.
```shell
./mvnw -pl seatunnel-connectors-v2/connector-redis -am test
TODO before this leaves draft:
- [ ] EN/ZH docs for the Redis source and sink connectors
- [ ] Redis row in the EN/ZH Layer 1 support matrix
- [ ] RedisConnectDryRunIT against a password-protected container (success,
wrong password, unreachable port, and DBSIZE == 0 after validation)
Check list
- [x] If any new Jar binary package adding in your PR, please add License
Notice according
New License Guide
(https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
— no new dependency
- [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
- [x] If necessary, please update incompatible-changes.md to describe the
incompatibility caused by this PR — not applicable, runtime behavior is
unchanged
- [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update plugin-mapping.properties
(https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and
add new connector information in it — not applicable, existing connector
2. Update the pom file of seatunnel-dist
(https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml) — not
applicable
3. Add ci label in label-scope-conf
(https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
— not applicable
4. Add e2e testcase in seatunnel-e2e
(https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
— pending
5. Update connector plugin_config
(https://github.com/apache/seatunnel/blob/dev/config/plugin_config) — not
applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
