Hi Folks: Here is a POC for persistence layer refactor. Please check it out and let me know what you think. Please note this is a POC, we still need a lot of effort to complete the refactor.
PR: https://github.com/apache/polaris/pull/1011. Design doc: https://docs.google.com/document/d/1Vuhw5b9-6KAol2vU3HUs9FJwcgWtiVVXMYhLtGmz53s/edit?tab=t.0 Experiment: - Added a DAO layer for the business entity namespace(except the read). - Integrated with existing DAO components (PolarisMetaStoreManager and PolarisMetaStoreSession). - All tests passed successfully, including a manual local run with Spark sql. Benefits: - Compatible with the existing backend(FDB), as we hide it behind the new DAO. - Adding new backends(Postgres/MongoDB) is much easier now, esp for Postgres, we could be able to use a similar model as Iceberg Jdbc catalog. - Allows gradual refactoring to remove old DAO dependencies (PolarisMetaStoreManager and PolarisMetaStoreSession). - Enables parallel development of new backend implementations. Next Steps: - Define business entities one by one to decouple them from FDB. - Create DAO interfaces for each entity to standardize operations (e.g., CRUD, ID generation). - Expand DAO implementations to support additional backends over time. Yufei
