Hey, Thanks for the proposal and I agree with Yufei. We had a backend synchronization CLI for projectNessie[1]. Maybe we can have something similar to that instead of taking a long path of the register table for migration between polaris instances.
[1] https://projectnessie.org/nessie-0-82-0/export_import/ - Ajantha On Sat, Apr 12, 2025 at 5:54 AM Yufei Gu <flyrain...@gmail.com> wrote: > Thanks, Mansehaj, for the proposal! This tool has potential, but I think we > should clarify its capabilities a bit more explicitly. Given its current > limitations, I'm not sure how broadly useful it would be. Have we explored > any alternative approaches—for example, performing synchronization on the > backend(FoundationDB, Postgres)? > > > Yufei > > On Thu, Apr 10, 2025 at 4:22 PM Mansehaj Singh > <mansehaj.si...@snowflake.com.invalid> wrote: > > > Hi all! Nice to meet you. > > > > I opened up https://github.com/apache/polaris-tools/pull/4 recently to > add > > a Polaris migration/synchronizer tool I've been working on to the > > polaris-tools repo. By request, I'm sharing a design document here > > detailing how the tool works and the roadmap for functionality that is in > > development. > > > > Here's the design doc giving a full overview: > > > > > https://docs.google.com/document/d/1AXKmzp3JaTuUS_FMNnxr_pHsBTs86rWRMborMi3deCw/edit?usp=sharing > > > > > > To summarize: > > > > We can think of this tool as a configurable mirroring/migration tool to > > migrate between two Polaris instances. I believe this would enable and > > support many use cases that are quite cumbersome to carry out manually > > today and break down barriers switching between open source and managed > > offerings of Polaris. The tool has been designed with goals in mind that > go > > beyond supporting just the CLI implementation. > > > > Please take a look at the design doc if you're interested! > > > > Thank you! > > - Sehaj > > >
