Hi,
I also support inviting Ajantha as a committer.
Still really prefer a separate repository, at least for now. We can always merge later.
It also makes the "migration to ASF" easier.
The "Nessie Iceberg-catalog-migrator tool" has been built to support migrations from any catalog to
any other catalog. This means, that the test matrix is quite complex and I expect it to become even more
complex and time consuming. IMHO there's no need to "bother" Polaris "main CI" for every
PR with catalog-migrator tests. I would also expect a different release cycle - no need to tie both together.
rename all packages to use org.apache.polaris
Better use org.apache.polaris.catalogmigrator as the base group ID.
refactore intTest to use Polaris instead of Nessie
This requires https://github.com/apache/polaris/pull/785. I'd also prefer to
keep the ITs against Nessie and add Polaris.
key component for features like federated catalogs.
We don't know yet how these (and other features) will look like and not how
it'll be related to the donated tool.
Please coordinate the migration from the source repo with me. We need to clean
some things up on the projectnessie Github org side first.
Robert
On 20.02.25 09:57, Jean-Baptiste Onofré wrote:
Hi Dmitri
About "evolution plan", I see the catalog migrator tool evolving as a
set of beans/providers that will be used in both CLI, and some server
features (like federated catalogs or "foreign catalogs").
We should not focus too much on catalog migrator as it is today but
more how it will be tomorrow.
That's why I'm more in favor of preparing the field and donating as a
module in the Polaris repo.
Regards
JB
On Thu, Feb 20, 2025 at 6:38 AM Dmitri Bourlatchkov <di...@apache.org> wrote:
+1 to accept the catalog migrator tool.
I support inviting Ajantha as a committer.
As to the source location, I tend to think that a separate repo makes sense
with the current state of the code, but I also agree that the overhead of
that may be too much, given that the codebase is small. I'm fine with
either a separate repo or a new module in the current Polaris repo.
What is the general plan for the evolution of the migrator tool? Are we
talking about integrating it into Polaris Servers or will it remain a
standalone tool as it is now?
Thanks,
Dmitri.
On Wed, Feb 19, 2025 at 11:39 AM Jean-Baptiste Onofré <j...@nanthrax.net>
wrote:
Hi folks,
Let me try to sum-up this topic.
1. Catalog Migration landing
It seems we have a preference to land catalog-migrator as a module on
the main polaris repo.
Robert expressed comments about CI, release cycle, dependencies.
My view on that is that the purpose of the catalog-migrator is to
evolve, and could become a key component for features like federated
catalogs.
Due to that, I think we can consider catalog-migrator as a module,
integrated in the Polaris CLI, or in the Polaris server,
Robert, does it work for you ?
2. Code/PR prep
I propose to work directly with Ajantha (main contributor of the
catalog-migrator) to prepare the code heading to a PR. We need:
- integrate in Polaris repo and gradle
- rename all packages to use org.apache.polaris
- add ASF header in all files
- refactore cli to use polaris style/naming
- refactore intTest to use Polaris instead of Nessie
- check the dependencies in the cli uber jar (hadoop, hive, ...) and
cleanup LICENSE/NOTICE there
- update README and cleanup other files
It should be pretty fast and we should be able to create a PR for
review/donation.
Thoughts ?
Regards
JB
On Tue, Feb 11, 2025 at 7:45 PM Jean-Baptiste Onofré <j...@nanthrax.net>
wrote:
Hi folks,
As discussed with some of you, we would like to propose donating the
Nessie Iceberg Catalog migrator tool
(https://github.com/projectnessie/iceberg-catalog-migrator) to Apache
Polaris.
A SGA has been already signed in case we accept the donation.
In terms of donation, I propose the following:
- the iceberg-catalog-migration can land in a separate Polaris repo
(as it is today for Nessie):
https://github.com/apache/polaris-catalog-migration or land as a
module in polaris repo directly
- we need to prepare the donation by changing the package names, etc
- I would suggest to consider inviting one of the main contributor of
Catalog Migrator (ajantha-bhat) as Polaris committer
WDYT ? Do we accept the iceberg-catalog-migrator tool in Polaris (we
can do a formal vote if we don't have obvious consensus) ?
Regards
JB
--
Robert Stupp
@snazy
