RE: [dspace-tech] Issue with database migration 6.3 to 7.4

[Cameron, Jacob]

That did help me make some progress.

I was able to import the database properly.  I also followed one of Tim’s 
suggestions so we could skip the 
V5.7_2017.04.11__DS-3563_Index_metadatavalue_resource_type_id_column.sql issue.

Now I’m getting a new error when trying to migrate the database on the 7.4 
server.  I found a suggestion from Tim to run ./dspace database migrate ignored 
on my old server to resolve the integer to UUID issue but it fails on my 
current 6.3 server.  The error is with one of the 7.0 sql queries, I’ve 
attached a copy of what I am receiving when trying to import on the new server 
and a copy of what I got when I tried to upgrade the database on my current 
server.

Jake
--
Jake Cameron, BCS(UNB)
Systems Support Specialist III
Information Systems and Technical Services University of Lethbridge Library
Phone:(403)329-2756
This e-mail, including any and all attachments, is only for the use of the 
intended recipient(s) and may contain information that is confidential or 
privileged. If you are not the intended recipient, you are advised that any 
dissemination, copying or other use of this e-mail is prohibited. Please notify 
the sender of the error in communication by return e-mail and destroy all 
copies of this e-mail. Thank you.

From: dspace-tech@googlegroups.com  On Behalf Of 
Mohammad S. AlMutairi
Sent: February 3, 2023 6:53 AM
To: DSpace Technical Support 
Subject: Re: [dspace-tech] Issue with database migration 6.3 to 7.4

Caution: This email was sent from someone outside of the University of 
Lethbridge. Do not click on links or open attachments unless you know they are 
safe. Suspicious emails should be forwarded to 
phish...@uleth.ca.

Hi Jake,

I don't think it's necessary to upgrade the old dspace PostgreSQL version 
because of this issue ( You already have version 13 installed on the new server 
). This is a common issue that can be caused by several causes. It's not 
related specifically to dspace. Just try the steps you see below and keep us 
updated here in the list.

On the old dspace Server execute (Double check the pg_dump path. 9.6?):
1) cd "C:\Program Files\PostgreSQL\9.6\bin"
2) .\pg_dump -U postgres -W -h localhost -Fc -v -C -c --if-exists -E UTF8 -d 
dspace -f dspace-63.sql
3) Move dspace-63.sql file to the new dspace server

On the new dspace server execute:
1) psql --username=postgres dspace
2) DROP EXTENSION pgcrypto;
3) DROP SCHEMA extensions;
4) \q
5) cd "C:\Program Files\PostgreSQL\13\bin"
6) .\pg_restore -U postgres -W -h localhost -v -e -c --if-exists -d dspace 
dspace-63.sql
7) psql --username=postgres dspace
8) \dn
9) \dx
10) \q

** The -e (--exit-on-error) you see in step 6 should give you a clue why the 
old or the new restoring was/is failing so watch for any error and post what it 
shows here to know exactly what's happening in your server (Hopefully not).

Regards,

On Friday, February 3, 2023 at 1:54:34 AM UTC+3 jacob.cameron wrote:

Hi Mohammad,



I’ve actually done that prior to this and still get a large number of errors no 
matter whether I import using psql or pg_restore for an sql or plain text dump.



I’m wondering if I need to upgrade from 9.6 to 13 in Postgres before it’ll fix 
it.  I’m sure I had to do that when I went from 1.8.3 to 6 originally. I could 
be wrong.  That was along time ago.  But I have something about that in old 
notes from that upgrade.



Jake



--

Jake Cameron, BCS(UNB)

Systems Support Specialist III

Information Systems and Technical Services University of Lethbridge Library

Phone:(403)329-2756

This e-mail, including any and all attachments, is only for the use of the 
intended recipient(s) and may contain information that is confidential or 
privileged. If you are not the intended recipient, you are advised that any 
dissemination, copying or other use of this e-mail is prohibited. Please notify 
the sender of the error in communication by return e-mail and destroy all 
copies of this e-mail. Thank you.



From: dspac...@googlegroups.com  On Behalf Of 
Mohammad S. AlMutairi
Sent: January 27, 2023 5:21 PM
To: DSpace Technical Support 
Subject: Re: [dspace-tech] Issue with database migration 6.3 to 7.4



Caution: This email was sent from someone outside of the University of 
Lethbridge. Do not click on links or open attachments unless you know they are 
safe. Suspicious emails should be forwarded to phis...@uleth.ca.



Hi all,



@Tim, I think Jacob issue is caused by the way he dumped his old database and 
the way he tried to restore it.  Those " thousands of invalid command /N" 
should've not shown up if the database was dumped in a custom binary format and 
the new server database was cleaned (emptied) or even recreated before 
restoring the old database int it using pg_restore (It's a must to use 
pg_restore for any binary format dumps). There is a good chance for him to able 
to migrate unless he is affected by the known issue you mentioned.

Re: [dspace-tech] Issue with database migration 6.3 to 7.4

[Mohammad S. AlMutairi]

Hi Jake,

I don't think it's necessary to upgrade the old dspace PostgreSQL version 
because of this issue ( You already have version 13 installed on the new 
server ). This is a common issue that can be caused by several causes. It's 
not related specifically to dspace. Just try the steps you see below and 
keep us updated here in the list.

On the old dspace Server execute (Double check the pg_dump path. 9.6?):
1) cd "C:\Program Files\PostgreSQL\9.6\bin"
2) .\pg_dump -U postgres -W -h localhost -Fc -v -C -c --if-exists -E UTF8 
-d dspace -f dspace-63.sql
3) Move dspace-63.sql file to the new dspace server

On the new dspace server execute:
1) psql --username=postgres dspace
2) DROP EXTENSION pgcrypto;
3) DROP SCHEMA extensions;
4) \q
5) cd "C:\Program Files\PostgreSQL\13\bin"
6) .\pg_restore -U postgres -W -h localhost -v -e -c --if-exists -d dspace 
dspace-63.sql
7) psql --username=postgres dspace
8) \dn
9) \dx
10) \q

** The -e (--exit-on-error) you see in step 6 should give you a clue why 
the old or the new restoring was/is failing so watch for any error and post 
what it shows here to know exactly what's happening in your server 
(Hopefully not).

Regards,
  

On Friday, February 3, 2023 at 1:54:34 AM UTC+3 jacob.cameron wrote:

Hi Mohammad,

 

I’ve actually done that prior to this and still get a large number of 
errors no matter whether I import using psql or pg_restore for an sql or 
plain text dump.

 

I’m wondering if I need to upgrade from 9.6 to 13 in Postgres before it’ll 
fix it.  I’m sure I had to do that when I went from 1.8.3 to 6 originally. 
I could be wrong.  That was along time ago.  But I have something about 
that in old notes from that upgrade.

 

Jake

 

--

Jake Cameron, BCS(UNB)

Systems Support Specialist III

Information Systems and Technical Services University of Lethbridge Library

Phone:(403)329-2756

This e-mail, including any and all attachments, is only for the use of the 
intended recipient(s) and may contain information that is confidential or 
privileged. If you are not the intended recipient, you are advised that any 
dissemination, copying or other use of this e-mail is prohibited. Please 
notify the sender of the error in communication by return e-mail and 
destroy all copies of this e-mail. Thank you.

 

*From:* dspac...@googlegroups.com  *On Behalf Of 
*Mohammad S. AlMutairi
*Sent:* January 27, 2023 5:21 PM
*To:* DSpace Technical Support 
*Subject:* Re: [dspace-tech] Issue with database migration 6.3 to 7.4

 

Caution: This email was sent from someone *outside of the University of 
Lethbridge*. Do not click on links or open attachments unless you know they 
are safe. Suspicious emails should be forwarded to phis...@uleth.ca.

 

Hi all, 

 

@Tim, I think Jacob issue is caused by the way he dumped his old database 
and the way he tried to restore it.  Those " thousands of invalid command 
/N" should've not shown up if the database was dumped in a custom binary 
format and the new server database was cleaned (emptied) or even recreated 
before restoring the old database int it using pg_restore (It's a must to 
use pg_restore for any binary format dumps). There is a good chance for him 
to able to migrate unless he is affected by the known issue you mentioned.

 

https://www.pgadmin.org/docs/pgadmin4/6.18/backup_dialog.html

 

Regards

On Friday, January 27, 2023 at 11:04:41 PM UTC+3 Tim Donohue wrote:

Hi Sean,

 

This issue is a known issue which some sites encounter during the upgrade 
to DSpace 7 *if they have previously ignored migrations which never ran 
during a past 5.x ->6.x upgrade*​.  Essentially, to summarize, there are 
two "5.7" migrations which can fail during the upgrade to 7.x... these 
migrations are *obsolete*​ and can be skipped over.

 

There's a thread discussing this on dspace-tech here, along with 
"workarounds" to skip the two problematic 5.7 migrations: 
https://groups.google.com/g/dspace-tech/c/PlOA1WMvd4M/m/eBVE5RfhBgAJ

 

There's also a fix for this coming in the 7.5 release.  In 7.5 (due Feb 20) 
we have a new "./dspace database skip" command which can be used to skip 
problematic migrations.  See the PR at 
https://github.com/DSpace/DSpace/pull/8611

 

So, you are not the first person to hit this migration bug. An easier fix 
is coming in 7.5... but in the meantime, you can use the dspace-tech link 
above to workaround the issue if you want to upgrade immediately to 7.4.


Tim

*From:* dspac...@googlegroups.com  on behalf of 
Cameron, Jacob 
*Sent:* Friday, January 27, 2023 12:46 PM
*To:* Sean Carte 


*Cc:* DSpace Technical Support 

*Subject:* RE: [dspace-tech] Issue with database migration 6.3 to 7.4 

 

I’ve tried importing with the postgres account, using an elevated command 
prompt, in pgadmin and they are all giving the same errors.  It’s still 
failing.  I’m at a loss now.  There’s a ton of tables missing when it 
finishes importing.

 

--

Jake Cameron, BCS(UNB)

Systems Support Specialist III

Information 

Re: [dspace-tech] Issue with database migration 6.3 to 7.4

[Mark H. Wood]

On Thu, Feb 02, 2023 at 10:54:25PM +, Cameron, Jacob wrote:
> I’m wondering if I need to upgrade from 9.6 to 13 in Postgres before it’ll 
> fix it.  I’m sure I had to do that when I went from 1.8.3 to 6 originally. I 
> could be wrong.  That was along time ago.  But I have something about that in 
> old notes from that upgrade.

Seeing that PostgreSQL v9 is EOL and v10 has had its last release,
upgrading is probably a good idea anyway.

-- 
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu

-- 
All messages to this mailing list should adhere to the Code of Conduct: 
https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
--- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dspace-tech/Y90LSI0MgYgafwup%40IUPUI.Edu.


signature.asc
Description: PGP signature