goutamadwant opened a new issue, #13004: URL: https://github.com/apache/seatunnel/issues/13004
### Search before asking - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. ### Description On the command line a job config can keep `${...}` placeholders and fill them at submit time with `-i key=value`. The REST v2 submit endpoints on `dev` have nothing like that, so anything that submits over REST has to build the full config text itself, including quoting. Before (current `dev`), with this config: ```hocon env { job.mode = "BATCH" job.name = "sync_${region}" } source { FakeSource { row.num = ${rows} schema = { fields { name = "string" } } } } sink { Console {} } ``` - `POST /submit-job?format=hocon` returns 500 with `String json deserialization exception ...`, because nothing can fill the placeholders. - The same job as a JSON body (`"row.num": "${rows}"`) is accepted as a literal string and fails with `Unable to create a source for identifier 'FakeSource'`. After: ```bash # same result as: ./bin/seatunnel.sh -c job.conf -i region=eu -i rows=100 curl -X POST 'http://127.0.0.1:8080/submit-job?format=hocon&variables=region%3Deu%2Crows%3D100' \ -H 'Content-Type: text/plain' --data-binary @job.conf ``` The job runs as `sync_eu` with 100 rows. Proposed change: - `/submit-job` (json, hocon, sql) and `/submit-job/upload` accept a `variables` parameter written like `-i` (`region=eu,rows=100`; a quoted value may contain commas). - A JSON body can carry a top-level `variables` object instead, and `/submit-jobs` takes `params.variables` or a `variables` object per job. - Substitution reuses `ConfigBuilder`, so the same config and variables give the same job config as the CLI. - This runs inside the engine, so a few things differ from the CLI on purpose: - placeholders are never filled from the engine's system properties or environment variables; - a `[...]` / `{...}` value cannot `include` a file or URL; - sink table placeholders such as `${table_name}` are left for the sink. - Malformed entries, empty, reserved or duplicate keys, and placeholders that no variable fills return 400. The error message never includes a value. - Requests without `variables` behave exactly as they do today. - REST v2 docs (en/zh) get a short section with a CLI/REST example. ### Usage Scenario A scheduler (DolphinScheduler, Airflow) or an in-house platform keeps one config per pipeline and submits it over REST with a different date, table or connection setting on each run. ### Related issues Discussion #9060 asks for this. #11568 touches the same REST entry points (environment variable allowlist for HOCON) but is independent of this change. ### Are you willing to submit a PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
