Thanks, Russell. I think this is a great idea. Super useful for reads for sure. Added some comments, and in particular I'd like to understand better what the write path challenges are. But agreed that reads are the thing that provide the most immediate value here.
-Tyler On Fri, Nov 22, 2024 at 2:32 PM Russell Spitzer <russell.spit...@gmail.com> wrote: > Hi Y'all, > > I've been thinking a lot about cross table compatibility as well as our > ability to govern other entity types within the Polaris catalog. I think a > good first step for the catalog would be to work on providing a system for > registering non-iceberg tables and providing an API / Service capability > for transforming those tables into Iceberg metadata when they are accessed > through Iceberg REST APIs. I think support for other apis may be something > we want to consider in the future as well since not all data-types can be > resolved as Iceberg metadata but for now I think this is the right > direction. > > I'd be glad to hear your thoughts, I plan to work on a POC of this concept > with Delta tables after Thanksgiving break. > > > > https://docs.google.com/document/d/1H2StuZ26LroibuQni3IJlErlKgrV9fEvYLHHqN7HWfE/edit?tab=t.01 > > > Thanks for your time, > Russ >
