Hi, First, it looks like the update-sequences was successful. That's what it's output looks like.
When you ran the "database migrate", it looks like that key error is the one about the non-existing "checksum_history" table. That table SHOULD EXIST in your DSpace 4 database (it's been in DSpace since v1.4, according to the code). So, it seems either you somehow removed it from your DSpace 4 database, or maybe you have some sort of permissions error where that table is not being "seen" by DSpace 7? It's hard to say, but you need to check to see if that table exists or not first. You could also try to do an upgrade to 5.x or 6.x first...but, I think you may hit this same error eventually until you figure out what happened to your "checksum_history" table. Tim ________________________________ From: [email protected] <[email protected]> on behalf of Night Librarian <[email protected]> Sent: Monday, August 29, 2022 5:39 PM To: DSpace Technical Support <[email protected]> Subject: Re: [dspace-tech] Re: Database migrate problem while upgrading from 4.0 to 7.2 Thank you, Tim. When I run the psql command, pointing to the v4 update-sequences file, I get the following list: $ psql -U dspace -f /temp/update-sequences.sql setval -------- 79 (1 row) setval -------- 90 (1 row) setval -------- 5274 (1 row) setval -------- 120 (1 row) setval -------- 591 (1 row) setval -------- 692 (1 row) setval -------- 162402 (1 row) setval -------- 2702 (1 row) setval -------- 4407 (1 row) setval -------- 4407 (1 row) setval -------- 5226 (1 row) setval -------- 49969 (1 row) setval -------- 86 (1 row) setval -------- 51 (1 row) setval -------- 200 (1 row) setval -------- 200 (1 row) setval -------- 2601 (1 row) setval -------- 132708 (1 row) setval -------- 229 (1 row) setval -------- 2711 (1 row) setval -------- 2083 (1 row) setval -------- 9488 (1 row) setval -------- 207 (1 row) setval -------- 8 (1 row) setval -------- (1 row) setval -------- (1 row) setval -------- 100 (1 row) setval -------- 49969 (1 row) setval -------- 5 (1 row) setval -------- (1 row) setval -------- (1 row) setval -------- 419 (1 row) setval -------- 6 (1 row) setval -------- 2216 (1 row) ========================================================================== Then, just to make sure, I check the database status, which looks the same as it was before I run the update-sequences: $ sudo ./dspace database status Database Type: postgres Database URL: jdbc:postgresql://localhost:5432/dspace Database Schema: public Database Username: dspace Database Software: PostgreSQL version 12.12 (Ubuntu 12.12-0ubuntu0.20.04.1) Database Driver: PostgreSQL JDBC Driver version 42.2.25 PostgreSQL 'pgcrypto' extension installed/up-to-date? true (version=1.3) FlywayDB Version: 6.5.7 +-----------+----------------+-----------------------------------------------------+------+--------------+---------+ | Category | Version | Description | Type | Installed On | State | +-----------+----------------+-----------------------------------------------------+------+--------------+---------+ | Versioned | 1.1 | Initial DSpace 1.1 database schema | SQL | | Pending | | Versioned | 1.2 | Upgrade to DSpace 1.2 schema | SQL | | Pending | | Versioned | 1.3 | Upgrade to DSpace 1.3 schema | SQL | | Pending | | Versioned | 1.3.9 | Drop constraint for DSpace 1 4 schema | JDBC | | Pending | | Versioned | 1.4 | Upgrade to DSpace 1.4 schema | SQL | | Pending | | Versioned | 1.5 | Upgrade to DSpace 1.5 schema | SQL | | Pending | | Versioned | 1.5.9 | Drop constraint for DSpace 1 6 schema | JDBC | | Pending | | Versioned | 1.6 | Upgrade to DSpace 1.6 schema | SQL | | Pending | | Versioned | 1.7 | Upgrade to DSpace 1.7 schema | SQL | | Pending | | Versioned | 1.8 | Upgrade to DSpace 1.8 schema | SQL | | Pending | | Versioned | 3.0 | Upgrade to DSpace 3.x schema | SQL | | Pending | | Versioned | 4.0 | Upgrade to DSpace 4.x schema | SQL | | Pending | | Versioned | 4.9.2015.10.26 | DS-2818 registry update | SQL | | Pending | | Versioned | 5.0.2014.08.08 | DS-1945 Helpdesk Request a Copy | SQL | | Pending | | Versioned | 5.0.2014.09.25 | DS 1582 Metadata For All Objects drop constraint | JDBC | | Pending | | Versioned | 5.0.2014.09.26 | DS-1582 Metadata For All Objects | SQL | | Pending | | Versioned | 5.0.2014.11.04 | Enable XMLWorkflow Migration | JDBC | | Pending | | Versioned | 5.6.2016.08.23 | DS-3097 | SQL | | Pending | | Versioned | 5.7.2017.04.11 | DS-3563 Index metadatavalue resource type id column | SQL | | Pending | | Versioned | 5.7.2017.05.05 | DS 3431 Add Policies for BasicWorkflow | JDBC | | Pending | | Versioned | 6.0.2015.03.06 | DS 2701 Dso Uuid Migration | JDBC | | Pending | | Versioned | 6.0.2015.03.07 | DS-2701 Hibernate migration | SQL | | Pending | | Versioned | 6.0.2015.08.31 | DS 2701 Hibernate Workflow Migration | JDBC | | Pending | | Versioned | 6.0.2015.09.01 | DS 2701 Enable XMLWorkflow Migration | JDBC | | Pending | | Versioned | 6.0.2016.01.03 | DS-3024 | SQL | | Pending | | Versioned | 6.0.2016.01.26 | DS 2188 Remove DBMS Browse Tables | JDBC | | Pending | | Versioned | 6.0.2016.02.25 | DS-3004-slow-searching-as-admin | SQL | | Pending | | Versioned | 6.0.2016.04.01 | DS-1955 Increase embargo reason | SQL | | Pending | | Versioned | 6.0.2016.04.04 | DS-3086-OAI-Performance-fix | SQL | | Pending | | Versioned | 6.0.2016.04.14 | DS-3125-fix-bundle-bitstream-delete-rights | SQL | | Pending | | Versioned | 6.0.2016.05.10 | DS-3168-fix-requestitem item id column | SQL | | Pending | | Versioned | 6.0.2016.07.21 | DS-2775 | SQL | | Pending | | Versioned | 6.0.2016.07.26 | DS-3277 fix handle assignment | SQL | | Pending | | Versioned | 6.0.2016.08.23 | DS-3097 | SQL | | Pending | | Versioned | 6.1.2017.01.03 | DS 3431 Add Policies for BasicWorkflow | JDBC | | Pending | | Versioned | 7.0.2017.10.12 | DS-3542-stateless-sessions | SQL | | Pending | | Versioned | 7.0.2018.04.03 | Upgrade Workflow Policy | JDBC | | Pending | | Versioned | 7.0.2018.04.16 | dspace-entities | SQL | | Pending | | Versioned | 7.0.2018.06.07 | DS-3851-permission | SQL | | Pending | | Versioned | 7.0.2019.05.02 | DS-4239-workflow-xml-migration | SQL | | Pending | | Versioned | 7.0.2019.06.14 | scripts-and-process | SQL | | Pending | | Versioned | 7.0.2019.07.31 | Retrieval of name variant | SQL | | Pending | | Versioned | 7.0.2019.11.13 | relationship type copy left right | SQL | | Pending | | Versioned | 7.0.2020.01.08 | DS-626-statistics-tracker | SQL | | Pending | | Versioned | 7.0.2020.10.31 | CollectionCommunity Metadata Handle | JDBC | | Pending | | Versioned | 7.0.2021.01.22 | Remove basic workflow | SQL | | Pending | | Versioned | 7.0.2021.02.08 | tilted rels | SQL | | Pending | | Versioned | 7.0.2021.03.18 | Move entity type to dspace schema | SQL | | Pending | | Versioned | 7.0.2021.09.24 | Move entity type from item template to collection | SQL | | Pending | | Versioned | 7.0.2021.10.04 | alter collection table drop workflow stem columns | SQL | | Pending | | Versioned | 7.1.2021.10.18 | Fix MDV place after migrating from DSpace 5 | SQL | | Pending | +-----------+----------------+-----------------------------------------------------+------+--------------+---------+ NOTE: This database is NOT yet initialized for auto-migrations (via Flyway). Your database looks to be compatible with DSpace version 4.0 All upgrades *after* version 4.0 will be run during the next migration. If you'd like to upgrade now, simply run 'dspace database migrate'. ========================================================================== And then, when I run database migrate ignored, I get an error about non-existing "checksum_history" table and "Migration V6.0_2015.03.07__DS-2701_Hibernate_migration.sql failed". The dspace.log (attached) shows (if I read it correctly), no errors until migration to version 6.0.2015.03.06, and then says: "ERROR unknown unknown org.flywaydb.core.internal.command.DbMigrate @ Migration of schema "public" to version 6.0.2015.03.07 - DS-2701 Hibernate migration failed! Changes successfully rolled back." $ sudo ./dspace database migrate ignored Database URL: jdbc:postgresql://localhost:5432/dspace Migrating database to latest version AND running previously "Ignored" migrations... (Check logs for details) Migration exception: java.sql.SQLException: Flyway migration error occurred at org.dspace.storage.rdbms.DatabaseUtils.updateDatabase(DatabaseUtils.java:775) at org.dspace.storage.rdbms.DatabaseUtils.updateDatabase(DatabaseUtils.java:662) at org.dspace.storage.rdbms.DatabaseUtils.main(DatabaseUtils.java:195) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:277) at org.dspace.app.launcher.ScriptLauncher.handleScript(ScriptLauncher.java:133) at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:98) Caused by: org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateException: Migration V6.0_2015.03.07__DS-2701_Hibernate_migration.sql failed ----------------------------------------------------------------- SQL State : 42P01 Error Code : 0 Message : ERROR: relation "checksum_history" does not exist Location : org/dspace/storage/rdbms/sqlmigration/postgres/V6.0_2015.03.07__DS-2701_Hibernate_migration.sql (/dspace/bin/file:/dspace/lib/dspace-api-7.2.jar!/org/dspace/storage/rdbms/sqlmigration/postgres/V6.0_2015.03.07__DS-2701_Hibernate_migration.sql) Line : 455 Statement : ALTER TABLE checksum_history RENAME COLUMN bitstream_id to bitstream_legacy_id at org.flywaydb.core.internal.command.DbMigrate.doMigrateGroup(DbMigrate.java:369) at org.flywaydb.core.internal.command.DbMigrate.access$200(DbMigrate.java:54) at org.flywaydb.core.internal.command.DbMigrate$3.call(DbMigrate.java:282) at org.flywaydb.core.internal.jdbc.TransactionalExecutionTemplate.execute(TransactionalExecutionTemplate.java:66) at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:279) at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:244) at org.flywaydb.core.internal.command.DbMigrate.access$100(DbMigrate.java:54) at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:162) at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:159) at org.flywaydb.core.internal.database.postgresql.PostgreSQLAdvisoryLockTemplate.execute(PostgreSQLAdvisoryLockTemplate.java:71) at org.flywaydb.core.internal.database.postgresql.PostgreSQLConnection.lock(PostgreSQLConnection.java:99) at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:140) at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:159) at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:137) at org.flywaydb.core.Flyway$1.execute(Flyway.java:206) at org.flywaydb.core.Flyway$1.execute(Flyway.java:158) at org.flywaydb.core.Flyway.execute(Flyway.java:527) at org.flywaydb.core.Flyway.migrate(Flyway.java:158) at org.dspace.storage.rdbms.DatabaseUtils.updateDatabase(DatabaseUtils.java:762) ... 9 more Caused by: org.flywaydb.core.internal.sqlscript.FlywaySqlScriptException: Migration V6.0_2015.03.07__DS-2701_Hibernate_migration.sql failed ----------------------------------------------------------------- SQL State : 42P01 Error Code : 0 Message : ERROR: relation "checksum_history" does not exist Location : org/dspace/storage/rdbms/sqlmigration/postgres/V6.0_2015.03.07__DS-2701_Hibernate_migration.sql (/dspace/bin/file:/dspace/lib/dspace-api-7.2.jar!/org/dspace/storage/rdbms/sqlmigration/postgres/V6.0_2015.03.07__DS-2701_Hibernate_migration.sql) Line : 455 Statement : ALTER TABLE checksum_history RENAME COLUMN bitstream_id to bitstream_legacy_id at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.handleException(DefaultSqlScriptExecutor.java:274) at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.executeStatement(DefaultSqlScriptExecutor.java:219) at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.execute(DefaultSqlScriptExecutor.java:127) at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.executeOnce(SqlMigrationExecutor.java:88) at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.access$000(SqlMigrationExecutor.java:33) at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor$1.call(SqlMigrationExecutor.java:77) at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor$1.call(SqlMigrationExecutor.java:74) at org.flywaydb.core.internal.database.DefaultExecutionStrategy.execute(DefaultExecutionStrategy.java:28) at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.execute(SqlMigrationExecutor.java:74) at org.flywaydb.core.internal.command.DbMigrate.doMigrateGroup(DbMigrate.java:366) ... 27 more Caused by: org.postgresql.util.PSQLException: ERROR: relation "checksum_history" does not exist at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2565) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2297) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:322) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:481) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:401) at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:322) at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:308) at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:284) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:279) at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:194) at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:194) at org.flywaydb.core.internal.jdbc.JdbcTemplate.executeStatement(JdbcTemplate.java:244) at org.flywaydb.core.internal.sqlscript.ParsedSqlStatement.execute(ParsedSqlStatement.java:111) at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.executeStatement(DefaultSqlScriptExecutor.java:207) ... 35 more ========================================================================== Maybe I need to try to upgrade to install DSpace 6.x first and upgrade to it, before upgrading to 7? Or is there something else I could try? On Monday, August 29, 2022 at 12:37:32 p.m. UTC-3 Tim Donohue wrote: Hi, You are running the wrong "update-sequences.sql" script (as that "handle_id_seq" is in the v7 script, not the v4 one)... likely because you tried to copy the v4 one into your v7 codebase. That won't work, because DSpace 7 will likely use the update-sequences.sql script that is embedded in dspace-api.jar. Instead, if you need to run the v4 update-sequences.sql manually by just doing something like this: psql -U dspace -f full/path/to/update-sequences.sql (This is assuming your DSpace database user account is named "dspace". If it isn't, then you need to change the "-U" param. For more information on psql see https://www.postgresql.org/docs/current/app-psql.html) Tim On Saturday, August 27, 2022 at 10:16:52 PM UTC-5 Night Librarian wrote: So, since I re-started the migration/upgrade process from the beginning, I re-downloaded the 4.x version update-sequences.sql from https://github.com/DSpace/DSpace/blob/dspace-4_x/dspace/etc/postgres/update-sequences.sql like I did in March and placed it in /temp/DSpace-dspace-7.2/dspace-api/src/main/resources/org/dspace/storage/rdbms/sqlmigration/postgres. What I don't understand, is that in March, after replacing the 7.x version by 4.x, it completed successfully: $ ./dspace database update-sequences Running org/dspace/storage/rdbms/sqlmigration/postgres/update-sequences.sql update-sequences complete Today, the terminal shows this: Running org/dspace/storage/rdbms/sqlmigration/postgres/update-sequences.sql Caught exception: org.postgresql.util.PSQLException: ERROR: relation "handle_id_seq" does not exist Position: 16 at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2565) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2297) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:322) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:481) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:401) at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:322) at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:308) at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:284) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:279) at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:194) at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:194) at org.dspace.storage.rdbms.DatabaseUtils.main(DatabaseUtils.java:353) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:277) at org.dspace.app.launcher.ScriptLauncher.handleScript(ScriptLauncher.java:133) at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:98) It's the exact error I got in March before I downloaded the 4.x update-sequences.sql and ran the command successfully. This time, it doesn't work. Am I using a wrong directory to store the file? Could there be a problem with permissions? What else am I missing? -- All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx --- You received this message because you are subscribed to a topic in the Google Groups "DSpace Technical Support" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/dspace-tech/UcAuyYjZiw4/unsubscribe. To unsubscribe from this group and all its topics, send an email to [email protected]<mailto:[email protected]>. To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/5288cb0c-9a18-494c-9d3a-6a999e404f2fn%40googlegroups.com<https://groups.google.com/d/msgid/dspace-tech/5288cb0c-9a18-494c-9d3a-6a999e404f2fn%40googlegroups.com?utm_medium=email&utm_source=footer>. -- All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/PH0PR22MB3274B9F04A73BC179CF22D80ED799%40PH0PR22MB3274.namprd22.prod.outlook.com.
