CYarros10 opened a new pull request, #44891: URL: https://github.com/apache/airflow/pull/44891
This PR adds support for [Google Cloud Vertex AI Feature Store synchronization](https://cloud.google.com/vertex-ai/docs/featurestore/latest/sync-data) operations in Apache Airflow. [Vertex AI Feature Store](https://cloud.google.com/vertex-ai/docs/featurestore/latest/overview) is a managed, cloud-native feature store service that's integral to Vertex AI. It streamlines your ML feature management and online serving processes by letting you manage your feature data in a BigQuery table or view. You can then serve features online directly from the BigQuery data source. ## Components Added 1. **SyncFeatureViewOperator**: Triggers synchronization of a feature view, updating online serving data from the batch source 2. **GetFeatureViewSyncOperator**: Retrieves status and details of a feature view sync operation 3. **FeatureViewSyncSensor**: Monitors the progress of feature view sync operations 4. **FeatureStoreHook**: Core integration with the Vertex AI Feature Store API ## Usage Example ```python sync_task = SyncFeatureViewOperator( task_id="sync_task", project_id=PROJECT_ID, location=REGION, feature_online_store_id=FEATURE_ONLINE_STORE_ID, feature_view_id=FEATURE_VIEW_ID, ) wait_for_sync = FeatureViewSyncSensor( task_id="wait_for_sync", location=REGION, feature_view_sync_name="{{ task_instance.xcom_pull(task_ids='sync_task', key='return_value')}}", poke_interval=60, # Check every minute timeout=600, # Timeout after 10 minutes mode="reschedule", ) get_task = GetFeatureViewSyncOperator( task_id="get_task", location=REGION, feature_view_sync_name="{{ task_instance.xcom_pull(task_ids='sync_task', key='return_value')}}", ) sync_task >> wait_for_sync >> get_task ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
