DanielLeens commented on PR #12209:
URL: https://github.com/apache/seatunnel/pull/12209#issuecomment-6466888378
Thanks @SEZ9, this is a useful catch, and it is something my earlier rounds
missed. I re-checked it against head `c8cdfaae7`, and I agree with your Issue 1
and your severity.
What I verified: `JobMaster.resolveLineageConfig()` passes
`jobImmutableInformation.getJobConfig().getEnvOptions()` as the job tier into
`LineageConfig.resolve(...)`. There `openlineage_enabled`,
`openlineage_transport`, `openlineage_url`, `openlineage_timeout_ms` and
`openlineage_retry_times` all go through `first(job, env, cluster, key)`, so
the job `env {}` wins. `asToken(env, cluster)` only ever reads the process
environment or the cluster block. `rejectJobAuthToken` stops a job from
supplying a token, but it does not stop a job from redirecting the cluster
token to another URL. So a job submitter can make the master send
`Authorization: Bearer <cluster token>` to a URL of their choosing. I would
treat this as blocking, and it should be added to the blocker list from my last
review (the requeue-after-every-event terminal-event drop).
@88fantasy, a minimal direction that keeps the "job can opt in or override"
feature: when a token is configured at the env or cluster tier, resolve
`openlineage_url` and `openlineage_transport` only from that same tier (ignore
or reject job-level values), or add a cluster-level allow-list of receiver
hosts. A test that asserts a job-level URL is never paired with the cluster
token would pin it. Because `LineageConfig` lives in the shared lineage module
from #12208, please settle where the guard belongs there first, so this PR does
not diverge from it.
SEZ9's Issues 2 and 5 (untimed metrics RPC fan-out and unbounded job-chosen
timeout/retry on the shared 2-thread pool) are the same family as the
pool-starvation concern I raised before, so they are worth fixing in the same
pass. I will re-review once a new commit lands.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
