jk-shah opened a new issue, #41355:
URL: https://github.com/apache/beam/issues/41355

   ### Description
   
   Beam YAML currently exposes `MLTransform` (`yaml_ml.py`) for 
embeddings/preprocessing and `Enrichment` (`yaml_enrichment.py`) for feature 
lookups, but has **no declarative `RunInference` provider** for scoring records 
in a YAML pipeline using local HuggingFace models, Vertex AI endpoints, or 
custom Python `ModelHandler`s.
   
   ### Proposed YAML Syntax
   
   ```yaml
   pipeline:
     transforms:
       - type: RunInference
         input: ReadFromPubSub
         config:
           provider: HuggingFacePipeline   # or VertexAI, PyTorch, Sklearn, 
Custom
           model_config:
             task: text-classification
             model: distilbert/distilbert-base-uncased-finetuned-sst-2-english
             large_model: true
           input_field: comment_text
           output_field: sentiment         # or flatten_predictions: true -> 
sentiment_label, sentiment_score
           min_batch_size: 8
           max_batch_size: 64
           error_handling:
             output: failed_inferences
   ```
   
   ### Key Capabilities
   1. **Lazy Worker-Side Loading**: Defer model downloads and endpoint client 
initialization to worker `setup()` / `load_model()` so YAML graph validation 
(`--dry_run`) remains offline and fast.
   2. **Schema-Preserving Output**: Attach predictions onto the input 
`beam.Row` (via `KeyedModelHandler`) with concrete output `RowTypeConstraint`s 
so downstream `Filter`, `Combine`, and `WriteToParquet` stages type-check 
cleanly.
   3. **Per-Element `error_handling` DLQ & Poison-Pill Isolation**: Integrate 
with Beam YAML's standard `error_handling` strip-and-route contract, falling 
back to per-element retry when a single malformed record fails a batch of `N` 
so `N-1` valid records succeed without evicting/reloading the in-memory model.
   
   ### Issue Priority
   
   Priority: 2 (default / most feature requests should be filed as P2/P3)
   
   ### Issue Components
   
   - [x] Component: Python SDK
   - [ ] Component: Java SDK
   - [ ] Component: Go SDK
   - [ ] Component: Typescript SDK
   - [ ] Component: IO connector
   - [x] Component: Beam YAML
   - [x] Component: Beam ML
   - [ ] Component: Beam examples
   - [ ] Component: Beam playground
   - [ ] Component: Beam katas
   - [ ] Component: Website
   - [ ] Component: Infrastructure
   - [ ] Component: Spark Runner
   - [ ] Component: Flink Runner
   - [ ] Component: Prism Runner
   - [ ] Component: Hazelcast Jet Runner
   - [ ] Component: Google Cloud Dataflow Runner
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to