MonkeyCanCode commented on PR #225: URL: https://github.com/apache/polaris-tools/pull/225#issuecomment-4469233815
> > So this may not be 100% valid as people can be using namespace as tenant/workflow separator for a given writer. Assuming one app is writing to 1k tables and each of the table has their own namespace, the purposed solution may not be useable in this case. > > I can understand that namespace by namespace migration is not possible in this case. > > What about retrying again for failed tables like mentioned below? > > > Also, I would like to add that, if only 'K' tables migrated out of 'N' and you need to migrate remaining. You can call same cli for all 'N' tables. It will skip migration for registered tables (with the error that table already exist). But other tables will still be migrated in that process. It won't pause migration of other tables if current table already exists. Hello @ajantha-bhat , thanks for the info and sorry for the late response. For sure it is possible to just keep rerun the migrate to get all tables over. But this doesn't necessary cover the case where people who want to benchmark reads on two diff catalogs (point read-only operation to the new catalog). I do understand the concern with two tables registered on two catalogs can be dangerous as there can be split-brain problem, thus i didn't make it the default. Do you think it is too dangerous to have this option even if an admin had to implicitly add `--overwrite` to take effect of this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
