tarun-google opened a new pull request, #40795:
URL: https://github.com/apache/beam/pull/40795
The Java and Go RunInference cross-language tests hardcode where the Python
SDK
harness puts staged files (`<semiPersistDir>/staged/sklearn_model`, or
`/tmp/staged/sklearn_model` in Go). That location belongs to the container's
boot program and is only known on the worker, so the tests break when it
changes (see #40699, which moves Python staging to
`<semi_persist_dir>/<worker id>/staged`).
The test expansion service (`expansion_service_test.py`) attaches the model
to
the Python environment as a staged file named `sklearn_model`. The tests now
pass that name and use a small `SklearnModelHandlerNumpy` subclass that
builds
the full path from `SEMI_PERSISTENT_DIRECTORY` in `load_model()`.
`load_model()`
runs on the worker after boot has set that variable. The subclass uses the
public `update_model_path()`, can be called repeatedly, and raises a clear
error if the variable is missing.
Test-only change. It works with both the current and the per-worker staging
layout, so it can merge ahead of #40699.
Testing (DirectRunner + Python expansion service, Python 3.11):
`./gradlew
:sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerJavaUsingPython
--tests '*RunInferenceTransformTest*' -PpythonVersion=3.11`
- current `boot.go` layout: 2/2 pass
- #40699 `boot.go` layout: 2/2 pass (before this change: 2/2 fail with
`FileNotFoundError: '/tmp/staged/sklearn_model'`)
- `spotlessApply` / `compileTestJava` clean; `gofmt` / `go vet` clean for
the Go
package (the Go test isn't run in CI because no suite provides its Python
expansion service).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
