Thanks, Jack! I agree that it would be great to get an idea of what catalogs are out there so we can add them to Iceberg. Always good to share those implementations. And not just for DynamoDB, too, if there are any others out there that aren't too specific to a company's internal metastore.
One thing I've thought about recently is defining a REST API for catalogs so that it's easier to build a service and not just directly on top of a database. That way you'd be able to scale out the service and not overwhelm your data store with connections. I know that's a concern for people running Hive metastores. If there's any interest in this, let's make some time to think about what we might want to include there. Ryan On Tue, Jun 8, 2021 at 11:25 PM Ye, Jack <yzhao...@amazon.com.invalid> wrote: > Hi everyone, > > > > Because of the many asks we have received for supporting AWS DynamoDB > Catalog implementation, I have put up PR > https://github.com/apache/iceberg/pull/2688 showing the DynamoDbCatalog > that we designed and implemented. > > > > I have heard a few internal implementations based on DynamoDB in different > companies and organizations so far. It would be great if we can share our > designs to compare the pros and cons of these implementations and arrive at > an optimal solution. > > > > Any suggestion or comment would be greatly appreciated! > > > > Thanks, > > Jack Ye > -- Ryan Blue
