The GitHub Actions job "Tests (AMD)" on airflow.git/iceberg-snapshot-trigger 
has failed.
Run started by GitHub user 1fanwang (triggered by 1fanwang).

Head commit for run:
9cb12ef41964944c2539125a9b46af5a877f8f83 / 1fanwang <[email protected]>
Add IcebergTableSnapshotTrigger for event-driven scheduling

The Iceberg provider ships a hook and nothing else, so there is no way to
schedule a DAG on 'this table has new data'. Anyone wanting it writes their own
trigger.

Add a trigger that polls a table's branch head through the existing IcebergHook
and emits an event when it advances, so a table commit can drive an 
AssetWatcher.
Triggers sharing a catalog connection, branch and poll interval report the same
shared_stream_key, so watching many tables in one catalog costs a single poll
rather than one per table.

Scoped to the trigger. Registering an iceberg:// asset URI scheme is left out:
that depends on how Iceberg datasets are named in OpenLineage, which is still
open, and a watcher works with any Asset name.

Signed-off-by: 1fanwang <[email protected]>

Report URL: https://github.com/apache/airflow/actions/runs/31432038609

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to