goutamadwant opened a new pull request, #13005:
URL: https://github.com/apache/seatunnel/pull/13005
### Purpose of this pull request
Closes #13004.
The REST v2 submit endpoints can now fill `${...}` placeholders the same way
`-i` does on the command line:
- `/submit-job` (json, hocon, sql) and `/submit-job/upload` take a
`variables` request parameter in the `-i` syntax.
- A JSON body can carry a top-level `variables` object instead, and
`/submit-jobs` reads `params.variables` or a `variables` object per job.
Substitution goes through `ConfigBuilder` (new `substituteVariables`), so
the result matches the CLI. Because it runs inside the engine, a few things
differ on purpose:
- it never reads the engine's system properties or environment variables;
- a `[...]` / `{...}` value cannot `include` a file or URL;
- sink table placeholders such as `${table_name}` are left for the sink.
Bad input returns 400, and the message never contains a value. Requests
without `variables` behave as before.
### Does this PR introduce _any_ user-facing change?
Yes. There is a new optional `variables` parameter / field on the three
submit endpoints, documented in the en and zh REST v2 docs.
On `dev` today, a HOCON body with `${rows}` fails with a 500, and a JSON
body passes `"${rows}"` through as a literal, so the source fails to start.
With this change:
```bash
curl -X POST
'http://127.0.0.1:8080/submit-job?format=hocon&variables=region%3Deu%2Crows%3D100'
\
-H 'Content-Type: text/plain' --data-binary @job.conf
```
submits the same job as `./bin/seatunnel.sh -c job.conf -i region=eu -i
rows=100`.
`/submit-jobs` also accepts an empty or missing `params` now; before, that
request failed.
### How was this patch tested?
- `RestSubmitVariablesTest`:
- HOCON, SQL and JSON bodies produce the same config as
`ConfigBuilder.of(file, variables)`. The fixture covers quoted commas, `=` and
Unicode in values, list values, defaults and missing variables.
- Engine system properties and environment variables are ignored.
- An `include` in a value is rejected, and a local HTTP server receives no
request.
- Bad input is rejected without echoing values.
- Sink table placeholders and `/encrypt-config` are unchanged.
- `RestApiSubmitJobVariablesTest`: every submit endpoint against a running
engine, including the 400 cases.
- `RestSubmitJobVariablesIT`:
- one config submitted twice with different variables finishes both times
with the expected row counts;
- `/submit-jobs` with per-job variables;
- the CLI variables fixture submitted over REST matches the same job
submitted through the client.
- The existing REST and `ConfigBuilder` tests pass on JDK 11 and 17.
### Check list
* [x] No new Jar binary package is added.
* [x] Documentation updated (`docs/en` and `docs/zh` `rest-api-v2.md`).
* [x] No incompatible change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
