Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
On 2015-03-15, at 22:53 , Jayaraj JR jaya...@iisertvm.ac.in wrote:
As of now, we are already on Evergreen 2.6.4 version. In this context what
would be best way to upgrade to 2.7 series. Can we use
the upgrade script 2.6.3 to 2.7.0 here. If not what modifications are
necessary ?. If we upgrade to 2.6.7, will it cause further complexity
when upgrading to the 2.7 version.
Generally speaking, you need to examine your upgrade log and cross-reference
the major and minor schema upgrade scripts up to the version you’re upgrading
to. Then you should modify schema upgrade scripts to make custom versions that
include all the numbered schema upgrades that you need to apply and exclude all
the numbered schema updates that have already been applied.
Specifically for version 2.6.4, the 2.6.3-2.6.4 schema upgrade script includes
schema upgrades 0892, 0893, 0894, 0895.
The 2.6.3-2.7.0 upgrade script includes upgrades 0884, 0885, 0886, 0888, 0889,
but not any of the upgrades included in the 2.6.3-2.6.4. So, you should be able
to run the 2.6.3-2.7.0 schema upgrade script against your version without
trouble.
If you look at the 2.7.0-2.7.1 schema upgrade script, it has schema upgrades
0892, 0893, 0894, 0895, which is equivalent to the 2.6.3-2.6.4 script. So, you
will need to modify the 2.7.0-2.7.1 script to exclude all those upgrades, but
you probably want to keep the 2.7.1 version stamp, so your modified 2.7.0-2.7.1
script will look like this:
--Upgrade Script for 2.7.0 to 2.7.1
\set eg_version '''2.7.1'''
BEGIN;
INSERT INTO config.upgrade_log (version, applied_to) VALUES ('2.7.1',
:eg_version);
COMMIT;
From then on you should just be fine with the rest of the 2.7 series minor
version schema upgrade scripts, but check for yourself.
When you’re done upgrading, your goal is to make sure that the schema is in
accordance with it’s expected state for that particular version. If your
database ends up missing any of these numbered upgrades, you may later discover
the consequences in surprising ways.
Also, keep in mind what Galen recommended earlier in this thread: checking
through all of the point schema upgrades you plan to apply and seeing if there
are any redundant bib reingests that you can skip can save you some time.
Aleksey Lazar
IS Developer and Integrator - PALS
http://www.mnpals.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hello, We have been keenly observing the discussion about the upgrade of Evergreen ILS from version 2.6.4 to 2.7 series. The upgrade would be simple and easier if we are on the version 2.6.3 as the Evergreen 2.7 provides direct step for upgrade from 2.6.3 to 2.7.0. As of now, we are already on Evergreen 2.6.4 version. In this context what would be best way to upgrade to 2.7 series. Can we use the upgrade script 2.6.3 to 2.7.0 here. If not what modifications are necessary ?. If we upgrade to 2.6.7, will it cause further complexity when upgrading to the 2.7 version. Kindly advice us the options. Thanking You -- Jayaraj J R Library Information Assistant IISER Thiruvananthapuram
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hello. On 2015-03-07, at 12:20 , Ben Shum bs...@biblio.org wrote: So the upgrade scripts are intended to be done sequentially as written due to certain limitations in the build process of Evergreen currently (this is still under community discussion). So unfortunately, if you're already on 2.6.4, then you may have some small edits necessary to perform the upgrade from 2.6 series to 2.7 series. If I recall correctly, you should be able to the run the 2.6.3-2.7 script without problems, but when you try to do the 2.7.0-2.7.1 script, it might cause conflicts because some of the changes in the 2.7.0-2.7.1 might already have been done during the 2.6.3-2.6.4 script. Looking at the same situation, but with a 2.5.4 to 2.6.0 upgrade. (2.5.4+all announced security patches applied, that is). In situations like these, most system administrators will manually compare the differences between the conflicting upgrade scripts and make edits to the files as necessary to skip anything that's already been done and doesn't need to be re-done. Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? Long-term, the developer community needs to complete discussions about how to make the database upgrade process cleaner between major version shifts, but this is the present reality. Following up on my previous statement, could it help towards a cleaner solution if the version-upgrade scripts were “virtual” scripts that do not actually include the code, but call the scripts externally, with statements wrapped in IF EXISTS clauses probably? Thanks. Aleksey Lazar IS Developer and Integrator - PALS http://www.mnpals.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Without getting into too detailed or specifics, but While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I would say that this is the main reason our library system has stuck to master so that we have an unbroken chain of upgrade script by the numbers without any confusion of when to apply X or Y, etc. Perhaps not helpful, but some background thoughts to the mix. -- Ben On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/ -- Benjamin Shum Evergreen Systems Manager Bibliomation, Inc. 24 Wooster Ave. Waterbury, CT 06708 203-577-4070, ext. 113
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hi, On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? A couple reasons: - The monolithic scripts allow for consolidating multiple point updates into a set of statements that run more quickly. Arguably this is more of a potential for efficiency than something that has always been taken advantage of -- but sometimes, it has. - Of more import, since the monolithic scripts wrap all of the changes in one or two transactions, if something goes wrong, the transaction rolls back and the schema remains in a state that corresponds to a released version. In contrast, if something goes wrong while going through a sequence of point upgrades and one gets stuck, the database could be in a state that doesn't correspond with *any* released version -- which could be awkward if one is nearing the end of a maintenance window. Then again, if one hasn't done a dry run first... I should emphasize I'm listing some reasons why the monolithic scripts have existed -- I'm not suggesting that they're always superior to other approaches. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Mike is correct of course, and in the past several major versions (2.5, 2.6, 2.7 at least) anyways, we've avoided issues where backported database upgrade scripts contain any differences between the ones used in a previous branch vs. master's version of the same numbered scripts. So we seem safe for now. But as noted, it has happened before further back in Evergreen's past and we must maintain constant vigilance that it does not happen again to cause any difficulties upgrading between major branches. This is why I'm always wary of backporting any bug fixes that contain database changes via upgrade script. Knowing what you're applying and testing thoroughly is the only way to know for sure what you do to your system though, in the end. -- Ben On Fri, Mar 13, 2015 at 4:17 PM, Mike Rylander mrylan...@gmail.com wrote: On Fri, Mar 13, 2015 at 4:00 PM, Ben Shum bs...@biblio.org wrote: Without getting into too detailed or specifics, but While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. FWIW, if folks are reusing numbers for similar-but-different in back branches (as in, same effect, different code) then they're doing it wrong. It's happened a couple times in the past and is nothing but trouble. The rule is: every upgrade script number should be allocated in master, even if the script in master may be effectively empty (just a config.upgrade_log entry), as would be the case when the change is only needed in a back branch. --miker Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I would say that this is the main reason our library system has stuck to master so that we have an unbroken chain of upgrade script by the numbers without any confusion of when to apply X or Y, etc. Perhaps not helpful, but some background thoughts to the mix. -- Ben On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/ -- Benjamin Shum Evergreen Systems
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Mike, I was composing my response to Ben's message as yours came in, but it sounds like what you're proposing below is very much in line with what I was imagining. I'd be willing to assist in making this happen, since I think it would benefit us all. Thanks! Chris - Original Message - From: Mike Rylander mrylan...@gmail.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 4:07:51 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Chris, That's what I've advocated for in the past (see upgrade scripts 0526 and 0537), via a helper script that 1) only applies what's needed (based on config.upgrade_log) and 2) knows how to look up supersedes/deprecated-by info from within the scripts. Writing down a detailed plan keeps getting back-burnered, but there are at least four different scripts floating around that attempt to do various versions of that. tl;dr of such a thing: all that's really needed to make that work is 1) a concerted effort by devs to do things like supplying fine-grained db scripts, particularly reingest-only scripts when a reingest is needed 2) RM involvement in the creation of per-release scripts that do just what's needed for the upgrade proper (like, do one and only one reingest) 3) inclusion of supersedes/deprecated-by info in upgrade scripts that do, in fact, supersede (or are deprecated by) early scripts 4) a (say) perl script to read all the upgrade scripts and apply them in order, as needed, based on config.upgrade_log -- Mike Rylander | President | Equinox Software, Inc. / The Open Source Experts | phone: 1-877-OPEN-ILS (673-6457) | email: mi...@esilibrary.com | web: http://www.esilibrary.com -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
On Fri, Mar 13, 2015 at 4:00 PM, Ben Shum bs...@biblio.org wrote: Without getting into too detailed or specifics, but While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. FWIW, if folks are reusing numbers for similar-but-different in back branches (as in, same effect, different code) then they're doing it wrong. It's happened a couple times in the past and is nothing but trouble. The rule is: every upgrade script number should be allocated in master, even if the script in master may be effectively empty (just a config.upgrade_log entry), as would be the case when the change is only needed in a back branch. --miker Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I would say that this is the main reason our library system has stuck to master so that we have an unbroken chain of upgrade script by the numbers without any confusion of when to apply X or Y, etc. Perhaps not helpful, but some background thoughts to the mix. -- Ben On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/ -- Benjamin Shum Evergreen Systems Manager Bibliomation, Inc. 24 Wooster Ave. Waterbury, CT 06708 203-577-4070, ext. 113
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Chris, That's what I've advocated for in the past (see upgrade scripts 0526 and 0537), via a helper script that 1) only applies what's needed (based on config.upgrade_log) and 2) knows how to look up supersedes/deprecated-by info from within the scripts. Writing down a detailed plan keeps getting back-burnered, but there are at least four different scripts floating around that attempt to do various versions of that. tl;dr of such a thing: all that's really needed to make that work is 1) a concerted effort by devs to do things like supplying fine-grained db scripts, particularly reingest-only scripts when a reingest is needed 2) RM involvement in the creation of per-release scripts that do just what's needed for the upgrade proper (like, do one and only one reingest) 3) inclusion of supersedes/deprecated-by info in upgrade scripts that do, in fact, supersede (or are deprecated by) early scripts 4) a (say) perl script to read all the upgrade scripts and apply them in order, as needed, based on config.upgrade_log -- Mike Rylander | President | Equinox Software, Inc. / The Open Source Experts | phone: 1-877-OPEN-ILS (673-6457) | email: mi...@esilibrary.com | web: http://www.esilibrary.com On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
I should note for the mailing list that all the numbered examples I gave were entirely fictional and meant to be representative of the concept contained there-in, not any actual working values for any real versions of Evergreen. -- Ben On Fri, Mar 13, 2015 at 4:00 PM, Ben Shum bs...@biblio.org wrote: Without getting into too detailed or specifics, but While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I would say that this is the main reason our library system has stuck to master so that we have an unbroken chain of upgrade script by the numbers without any confusion of when to apply X or Y, etc. Perhaps not helpful, but some background thoughts to the mix. -- Ben On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/ -- Benjamin Shum Evergreen Systems Manager Bibliomation, Inc. 24 Wooster Ave. Waterbury, CT 06708 203-577-4070, ext. 113 -- Benjamin Shum Evergreen Systems Manager Bibliomation, Inc. 24 Wooster Ave. Waterbury, CT 06708 203-577-4070, ext. 113
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Thanks for the quick and detailed response, Ben. While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. I would be interested in seeing a system that did something where the Evergreen 2.5.1-2.7.0 upgrade script (for example) was really just a wrapper that said run scripts 0001, 0002, 0003, 0005, and 0010. It would also be nice if something in the database (maybe expanding config.upgrade_log?) would know which scripts constituted which version upgrade. People running sequentially through the scripts to stay on master are already out-of-band as far as releases go, so they would be left to manage the process on their own. Though it would be useful for the system to know that the system they are running is equivalent to 2.7.2 + scripts 0045 and 0046 for comparison purposes. Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I agree with this. The last three upgrades have been much smoother from my perspective. Thanks! Chris -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Galen, - The monolithic scripts allow for consolidating multiple point updates into a set of statements that run more quickly. Arguably this is more of a potential for efficiency than something that has always been taken advantage of -- but sometimes, it has. - Of more import, since the monolithic scripts wrap all of the changes in one or two transactions, if something goes wrong, the transaction rolls back and the schema remains in a state that corresponds to a released version. In contrast, if something goes wrong while going through a sequence of point upgrades and one gets stuck, the database could be in a state that doesn't correspond with *any* released version -- which could be awkward if one is nearing the end of a maintenance window. Then again, if one hasn't done a dry run first... Yeah, I can see the advantage of the full rollback. I wonder if the wrapper/controller script I'm imagining (and Mike mentions in his steps to make this actually happen response could have some sort of rollback option (though I know it's Postgres transactions that we depend on for rollback at this point). I should emphasize I'm listing some reasons why the monolithic scripts have existed -- I'm not suggesting that they're always superior to other approaches. Understood, thanks! -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hi, On Fri, Mar 13, 2015 at 4:22 PM, Chris Sharp csh...@georgialibraries.org wrote: Yeah, I can see the advantage of the full rollback. I wonder if the wrapper/controller script I'm imagining (and Mike mentions in his steps to make this actually happen response could have some sort of rollback option (though I know it's Postgres transactions that we depend on for rollback at this point). At first blush, I don't think that would be easy to do as such -- although one could envision a wrapper script that took out the begin/commit pairs in the individual point updates and ran the whole shebang in a single transaction, there are times when multiple transactions are required if one is both altering the structure of a table and altering data in it. Consequently, a rollback in case of a failure would be accomplished by restore from a base backup (or potentially, doing a PITR recovery to a specified time). But let's see if I can include a variation of this sentence in every single response I make in this thread: doing test runs of EG upgrades is a Good Thing. :) Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hello, So the upgrade scripts are intended to be done sequentially as written due to certain limitations in the build process of Evergreen currently (this is still under community discussion). So unfortunately, if you're already on 2.6.4, then you may have some small edits necessary to perform the upgrade from 2.6 series to 2.7 series. If I recall correctly, you should be able to the run the 2.6.3-2.7 script without problems, but when you try to do the 2.7.0-2.7.1 script, it might cause conflicts because some of the changes in the 2.7.0-2.7.1 might already have been done during the 2.6.3-2.6.4 script. In situations like these, most system administrators will manually compare the differences between the conflicting upgrade scripts and make edits to the files as necessary to skip anything that's already been done and doesn't need to be re-done. Long-term, the developer community needs to complete discussions about how to make the database upgrade process cleaner between major version shifts, but this is the present reality. -- Ben On Sat, Mar 7, 2015 at 9:23 AM, Jayaraj JR jaya...@iisertvm.ac.in wrote: Hello, We are currently working on Evergreen version 2.6.4. We would like to upgrade to 2.7 series. But the sql script found for upgrade is from 2.6.3 to 2.7 series. What script is to be done for upgrading from 2.6.4 or higher version to 2.7 series. Thanking You -- Jayaraj J R Library Information Assistant IISER Thiruvananthapuram -- Benjamin Shum Evergreen Systems Manager Bibliomation, Inc. 24 Wooster Ave. Waterbury, CT 06708 203-577-4070, ext. 113
