Hey all,

I was integrating our Java IcebergIO with Beam SQL (PR #34799
<https://github.com/apache/beam/pull/34799>) and got blocked on the fact
that Beam SQL currently lacks a "Catalog" concept. This is fundamental to
modern data architectures like Iceberg, where they are used to manage table
metadata and enable broad ecosystem integration.

To address this gap, I've opened a new PR (#35223
<https://github.com/apache/beam/pull/35223>), which introduces the *Catalog*
and *CatalogManager* interfaces, enabling support for:

   -

   CREATE CATALOG my_catalog TYPE 'local' PROPERTIES (...)
   -

   SET CATALOG my_catalog
   -

   DROP CATALOG my_catalog

I left a more detailed overview in the PR description.

My hope is that this foundational change will benefit not just IcebergIO,
but also other IOs and future Beam SQL integrations.

Please take a look and share any feedback, especially regarding major
architectural concerns. I'm working on a short timeline, so minor
enhancements can be noted for follow-up PRs.

Thank you!
Ahmed

Reply via email to