shikhabhadoria opened a new pull request, #41424:
URL: https://github.com/apache/beam/pull/41424
## Summary
Addresses #41355.
This PR adds declarative `RunInference` support in Beam YAML
(`apache_beam.yaml.yaml_ml`) with lazy worker-side loading, schema-preserving
output, and per-element `error_handling` dead-letter routing with poison-pill
batch isolation:
1. **Declarative Providers & Lazy Worker-Side Loading**:
- Supports `provider: HuggingFacePipeline | VertexAI | PyTorch | Sklearn
| Custom` alongside `model_config` (while preserving backward compatibility
with `model_handler: {type: ..., config: ...}`).
- Wraps handlers in `_LazyYamlModelHandler` so framework imports
(`torch`, `transformers`, `sklearn`, `google.cloud.aiplatform`), model
downloads, and remote endpoint checks are deferred to worker `setup()` /
`load_model()`, keeping YAML graph construction (`--dry_run` and Runner API
serialization) offline and fast.
2. **Schema-Preserving Output & `input_field` / `output_field` /
`flatten_predictions`**:
- Supports declarative `input_field` extraction (automatically converting
to NumPy arrays for `Sklearn` or PyTorch tensors for `PyTorch` when needed),
`output_field`, `flatten_predictions: true` (e.g. expanding `output_field:
sentiment` into `sentiment_label: str` and `sentiment_score: float`), and
`output_type` with concrete `RowTypeConstraint` inference.
3. **Per-Element `error_handling` DLQ & Poison-Pill Isolation**:
- Integrates `run_inference` with
`@maybe_with_exception_handling_transform_fn` and
`_YamlRunInferenceTransform.with_exception_handling`.
- When a batch of `N` records fails during `run_inference`,
`_LazyYamlModelHandler` falls back to per-element retry against the
already-loaded in-memory model (without evicting or reloading it) so `N - 1`
valid records succeed and only the failing record is routed to the configured
`error_handling` output.
## Checklist
- [x] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
- [x] Replace these instructions with a description of your change and why
you have made it.
- [x] Mention the appropriate issue in your description (for example:
`addresses #123` or `fixes #456`).
- [x] Update `CHANGES.md` with noteworthy changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]