Hi Everyone, I am new to the community and love the tooling and had some questions regarding some issues we've faced with larger databases.
*As some background:* We are in the process of migrating from Heroku to AWS Aurora RDS. We initially tried out DMS but faced issues. We then took the WAL-E approach - whilst this worked this introduced some unnecessary risks for us such as a double step migration. Thus intro to Bucardo. We have successfully migrated several databases - but recently found some issues with larger databases e.g. we have one database in particular that has 5 tables consuming 750GB total with the biggest table over 350GB in size. Whilst the tasks run - we find after 24 hrs (not sure why it takes so long) the task's timeout and restart again. We run bucardo on an EC2 instance with Postgres installed (just empty shell) *Mitigations we are taking right now:* We have some apps that are 100% completely backed by Kafka and thus we can take them offline and recover later. For now, a pg_dump and pg_restore with parallel jobs allow us to achieve a full restore in approx 7 hrs. *My hopes:* Determine a way to get Bucardo to run the migration for larger tables without hitting timeouts - ideally parallelized to speed up time or settings that prevent the timeout from occurring. Thank you community and happy to provide more background Best, Wayne
_______________________________________________ Bucardo-general mailing list [email protected] https://bucardo.org/mailman/listinfo/bucardo-general
