shikhabhadoria opened a new pull request, #41424:
URL: https://github.com/apache/beam/pull/41424

   ## Summary
   
   Addresses #41355.
   
   This PR adds declarative `RunInference` support in Beam YAML 
(`apache_beam.yaml.yaml_ml`) with lazy worker-side loading, schema-preserving 
output, and per-element `error_handling` dead-letter routing with poison-pill 
batch isolation:
   
   1. **Declarative Providers & Lazy Worker-Side Loading**:
      - Supports `provider: HuggingFacePipeline | VertexAI | PyTorch | Sklearn 
| Custom` alongside `model_config` (while preserving backward compatibility 
with `model_handler: {type: ..., config: ...}`).
      - Wraps handlers in `_LazyYamlModelHandler` so framework imports 
(`torch`, `transformers`, `sklearn`, `google.cloud.aiplatform`), model 
downloads, and remote endpoint checks are deferred to worker `setup()` / 
`load_model()`, keeping YAML graph construction (`--dry_run` and Runner API 
serialization) offline and fast.
   2. **Schema-Preserving Output & `input_field` / `output_field` / 
`flatten_predictions`**:
      - Supports declarative `input_field` extraction (automatically converting 
to NumPy arrays for `Sklearn` or PyTorch tensors for `PyTorch` when needed), 
`output_field`, `flatten_predictions: true` (e.g. expanding `output_field: 
sentiment` into `sentiment_label: str` and `sentiment_score: float`), and 
`output_type` with concrete `RowTypeConstraint` inference.
   3. **Per-Element `error_handling` DLQ & Poison-Pill Isolation**:
      - Integrates `run_inference` with 
`@maybe_with_exception_handling_transform_fn` and 
`_YamlRunInferenceTransform.with_exception_handling`.
      - When a batch of `N` records fails during `run_inference`, 
`_LazyYamlModelHandler` falls back to per-element retry against the 
already-loaded in-memory model (without evicting or reloading it) so `N - 1` 
valid records succeed and only the failing record is routed to the configured 
`error_handling` output.
   
   ## Checklist
   
    - [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
    - [x] Replace these instructions with a description of your change and why 
you have made it.
    - [x] Mention the appropriate issue in your description (for example: 
`addresses #123` or `fixes #456`).
    - [x] Update `CHANGES.md` with noteworthy changes.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to