Vamsi-klu opened a new pull request, #69749:
URL: https://github.com/apache/airflow/pull/69749
## Problem
apache-airflow-providers-google base depends on
google-cloud-aiplatform[evaluation] which pulls litellm, scikit-learn,
ruamel-yaml, tqdm, huggingface-hub, tokenizers. Users of GCS/BigQuery get ML
eval stack they never use. litellm has frequent proxy-server CVEs (auth bypass,
RCE) triaged by every user.
Root cause: providers/google/pyproject.toml:86
"google-cloud-aiplatform[evaluation]>=1.155.0"
Code needing evaluation: vertexai.preview.evaluation.EvalResult, EvalTask
used only in GenerativeModelHook.get_eval_task() and run_evaluation() and
RunEvaluationOperator.
## What I did
Part A: pyproject.toml
- Change base from "google-cloud-aiplatform[evaluation]>=1.155.0" to
"google-cloud-aiplatform>=1.155.0"
- Add optional extra "evaluation" =
["google-cloud-aiplatform[evaluation]>=1.155.0"]
Part B: Lazy import guard in hook
- Try/except ImportError for EvalResult/EvalTask, store
_evaluation_import_error
- In get_eval_task() and run_evaluation(), raise
AirflowOptionalProviderFeatureException with install instruction if import
failed
- Operator file doesn't import EvalTask at top-level, hook guard enough
Part C: Tests use pytest.importorskip("vertexai.preview.evaluation") to skip
gracefully when extra not installed
Part D: Changelog breaking change note directly edited per provider rules
Generated files via prek update-providers-dependencies --all-files: README,
docs/index.rst, provider info
## Impact
- Base install no longer pulls litellm/scikit-learn (security + image size +
dep hell improvement)
- Users needing RunEvaluationOperator must install
apache-airflow-providers-google[evaluation]
- Breaking change for Eval users, mitigated via changelog and clear error
message
- pandas remains base dep, ray remains
## Testing
- python -c "from airflow.providers.google.cloud.hooks.gcs import GCSHook"
works without evaluation extra
- Hook.get_eval_task() raises AirflowOptionalProviderFeatureException with
helpful msg when mocked missing
- uv run --project providers-google --extra evaluation pytest ... -k
evaluation → 1 passed
- ruff format/check passed
- Note: breeze ci-image rebuild needed
Fixes: #69323
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Muse Spark 1.1
Generated-by: Muse Spark 1.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
