PDGGK opened a new pull request, #40944: URL: https://github.com/apache/beam/pull/40944
Fixes #40730. #40729 removed the `gcpopts` import from `gcsx`. This change handles the other regression listed in the issue. #40379 made `core/runtime/xlangx` import `options/jobopts` to read `--resource_hints`. The root `beam` package imports `xlangx`, so any binary that imports `beam` now registers all of jobopts' flags on `flag.CommandLine`: `--endpoint`, `--job_name`, `--async`, `--experiments`, `--parallelism` and the rest. A binary or test that defines a flag with one of those names panics at startup. The repro from the issue, run unchanged as a test in a separate module against each Beam version: ``` # github.com/apache/beam/sdks/v2 v2.77.0 ok repro 0.504s # github.com/apache/beam/sdks/v2 v2.78.0-RC1 panic: /var/folders/…/repro.test flag redefined: endpoint FAIL repro 0.367s # master at 99dbd880e07 panic: /var/folders/…/repro.test flag redefined: endpoint FAIL repro 0.377s # master with this change ok repro 0.359s ``` Both #40379 and #40729 are in `sdks/v2.78.0-RC1`. 2.78.0 therefore fixes the `--project` collision but still has this one. ### Change - `xlangx` now looks up `--resource_hints` by name with `flag.Lookup`, as `gcsx.GetGcsEndpoint` does since #40729. If the flag isn't registered, nothing could have set it, and no hints are forwarded. - The hint parsing moves from `jobopts.GetPipelineResourceHints` into a new `resource.ParseHints`, which both packages can use without registering flags. `jobopts.GetPipelineResourceHints` stays as a wrapper, so its API is unchanged. - `jobopts`' `stringSlice.Get()` returns a `[]string` rather than the unexported `stringSlice` type, so the value can be read through `flag.Getter` from outside the package. After this, `go list -deps ./go/pkg/beam` no longer lists `options/jobopts`, as in v2.77.0. The issue's long-term point, that runners register flags in `init()`, is out of scope here. ### Testing New tests: - `TestNoJobSubmissionFlags` (xlangx): asserts the xlangx test binary has no `--endpoint`, `--job_name` or `--resource_hints` flag. If the `jobopts` import is put back in `expand.go`, it fails with `flag --endpoint is registered (usage "Job service endpoint (required).")`, plus the same message for the other two flags. - `TestPipelineResourceHints` (xlangx): covers the lookup with the flag unregistered, and with two `--resource_hints` values. - `TestResourceHintsFlagGetter` (jobopts): pins the `[]string` result of `Get()` that xlangx relies on. If `Get()` returns `s` again, it fails. - `TestParseHints` (resource): the previous jobopts parsing test, moved with the same inputs and assertions. I also checked that the #40379 behaviour still works end to end, through exported API only. A test module imports `jobopts` (as a runner does), sets `--resource_hints=min_ram=2GB`, and calls `beam.TryCrossLanguage` against a handler registered with `xlangx.RegisterHandler`. The captured `ExpansionRequest` has `beam:option:resource_hints:v1` = `["min_ram=2000000000B"]`, both on v2.78.0-RC1 and with this change. Run locally: - `go test` for `options/...`, `core/runtime/xlangx/...`, `runners/universal/...`, `runners/dataflow` and the root `beam` package: all pass. - `gofmt -l`, `go vet --copylocks=false --unsafeptr=false` and `staticcheck` 2026.2 on the touched packages: all clean. The full `./go/pkg/...` run and the cross-language integration tests are left to CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
