Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hello. On 2015-03-07, at 12:20 , Ben Shum bs...@biblio.org wrote: So the upgrade scripts are intended to be done sequentially as written due to certain limitations in the build process of Evergreen currently (this is still under community discussion). So unfortunately, if you're already on 2.6.4, then you may have some small edits necessary to perform the upgrade from 2.6 series to 2.7 series. If I recall correctly, you should be able to the run the 2.6.3-2.7 script without problems, but when you try to do the 2.7.0-2.7.1 script, it might cause conflicts because some of the changes in the 2.7.0-2.7.1 might already have been done during the 2.6.3-2.6.4 script. Looking at the same situation, but with a 2.5.4 to 2.6.0 upgrade. (2.5.4+all announced security patches applied, that is). In situations like these, most system administrators will manually compare the differences between the conflicting upgrade scripts and make edits to the files as necessary to skip anything that's already been done and doesn't need to be re-done. Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? Long-term, the developer community needs to complete discussions about how to make the database upgrade process cleaner between major version shifts, but this is the present reality. Following up on my previous statement, could it help towards a cleaner solution if the version-upgrade scripts were “virtual” scripts that do not actually include the code, but call the scripts externally, with statements wrapped in IF EXISTS clauses probably? Thanks. Aleksey Lazar IS Developer and Integrator - PALS http://www.mnpals.org/
[OPEN-ILS-GENERAL] Reminder: Ending soon! Early Bird Registration for the 2015 Evergreen International Conference ends 3/15
Hello, This is a reminder that early bird registration for the 2015 Evergreen International Conference in Hood River, Oregon ends Sunday, March 15th at 11:30 pm. After this Sunday the discounted conference rate of $200 will be increased to $250 and will be available through May 6, 2015. Be sure to lock in the discounted rate by visiting the conference registration https://www.eventbrite.com/e/evergreen-2015-international-conference-tickets-15029293020 page and signing up before the weekend is over. More information on the conference can be found at http://evergreen-ils.org/conference/eg15/ http://evergreen-ils.org/conference/eg15/ Hope to see you all in May! -Brent - Brent Mills Systems Librarian | Sage Library System
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Without getting into too detailed or specifics, but While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I would say that this is the main reason our library system has stuck to master so that we have an unbroken chain of upgrade script by the numbers without any confusion of when to apply X or Y, etc. Perhaps not helpful, but some background thoughts to the mix. -- Ben On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/ -- Benjamin Shum Evergreen Systems Manager Bibliomation, Inc. 24 Wooster Ave. Waterbury, CT 06708 203-577-4070, ext. 113
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hi, On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? A couple reasons: - The monolithic scripts allow for consolidating multiple point updates into a set of statements that run more quickly. Arguably this is more of a potential for efficiency than something that has always been taken advantage of -- but sometimes, it has. - Of more import, since the monolithic scripts wrap all of the changes in one or two transactions, if something goes wrong, the transaction rolls back and the schema remains in a state that corresponds to a released version. In contrast, if something goes wrong while going through a sequence of point upgrades and one gets stuck, the database could be in a state that doesn't correspond with *any* released version -- which could be awkward if one is nearing the end of a maintenance window. Then again, if one hasn't done a dry run first... I should emphasize I'm listing some reasons why the monolithic scripts have existed -- I'm not suggesting that they're always superior to other approaches. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Mike is correct of course, and in the past several major versions (2.5, 2.6, 2.7 at least) anyways, we've avoided issues where backported database upgrade scripts contain any differences between the ones used in a previous branch vs. master's version of the same numbered scripts. So we seem safe for now. But as noted, it has happened before further back in Evergreen's past and we must maintain constant vigilance that it does not happen again to cause any difficulties upgrading between major branches. This is why I'm always wary of backporting any bug fixes that contain database changes via upgrade script. Knowing what you're applying and testing thoroughly is the only way to know for sure what you do to your system though, in the end. -- Ben On Fri, Mar 13, 2015 at 4:17 PM, Mike Rylander mrylan...@gmail.com wrote: On Fri, Mar 13, 2015 at 4:00 PM, Ben Shum bs...@biblio.org wrote: Without getting into too detailed or specifics, but While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. FWIW, if folks are reusing numbers for similar-but-different in back branches (as in, same effect, different code) then they're doing it wrong. It's happened a couple times in the past and is nothing but trouble. The rule is: every upgrade script number should be allocated in master, even if the script in master may be effectively empty (just a config.upgrade_log entry), as would be the case when the change is only needed in a back branch. --miker Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I would say that this is the main reason our library system has stuck to master so that we have an unbroken chain of upgrade script by the numbers without any confusion of when to apply X or Y, etc. Perhaps not helpful, but some background thoughts to the mix. -- Ben On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/ -- Benjamin Shum Evergreen Systems
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Mike, I was composing my response to Ben's message as yours came in, but it sounds like what you're proposing below is very much in line with what I was imagining. I'd be willing to assist in making this happen, since I think it would benefit us all. Thanks! Chris - Original Message - From: Mike Rylander mrylan...@gmail.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 4:07:51 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Chris, That's what I've advocated for in the past (see upgrade scripts 0526 and 0537), via a helper script that 1) only applies what's needed (based on config.upgrade_log) and 2) knows how to look up supersedes/deprecated-by info from within the scripts. Writing down a detailed plan keeps getting back-burnered, but there are at least four different scripts floating around that attempt to do various versions of that. tl;dr of such a thing: all that's really needed to make that work is 1) a concerted effort by devs to do things like supplying fine-grained db scripts, particularly reingest-only scripts when a reingest is needed 2) RM involvement in the creation of per-release scripts that do just what's needed for the upgrade proper (like, do one and only one reingest) 3) inclusion of supersedes/deprecated-by info in upgrade scripts that do, in fact, supersede (or are deprecated by) early scripts 4) a (say) perl script to read all the upgrade scripts and apply them in order, as needed, based on config.upgrade_log -- Mike Rylander | President | Equinox Software, Inc. / The Open Source Experts | phone: 1-877-OPEN-ILS (673-6457) | email: mi...@esilibrary.com | web: http://www.esilibrary.com -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
On Fri, Mar 13, 2015 at 4:00 PM, Ben Shum bs...@biblio.org wrote: Without getting into too detailed or specifics, but While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. FWIW, if folks are reusing numbers for similar-but-different in back branches (as in, same effect, different code) then they're doing it wrong. It's happened a couple times in the past and is nothing but trouble. The rule is: every upgrade script number should be allocated in master, even if the script in master may be effectively empty (just a config.upgrade_log entry), as would be the case when the change is only needed in a back branch. --miker Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I would say that this is the main reason our library system has stuck to master so that we have an unbroken chain of upgrade script by the numbers without any confusion of when to apply X or Y, etc. Perhaps not helpful, but some background thoughts to the mix. -- Ben On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/ -- Benjamin Shum Evergreen Systems Manager Bibliomation, Inc. 24 Wooster Ave. Waterbury, CT 06708 203-577-4070, ext. 113
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Chris, That's what I've advocated for in the past (see upgrade scripts 0526 and 0537), via a helper script that 1) only applies what's needed (based on config.upgrade_log) and 2) knows how to look up supersedes/deprecated-by info from within the scripts. Writing down a detailed plan keeps getting back-burnered, but there are at least four different scripts floating around that attempt to do various versions of that. tl;dr of such a thing: all that's really needed to make that work is 1) a concerted effort by devs to do things like supplying fine-grained db scripts, particularly reingest-only scripts when a reingest is needed 2) RM involvement in the creation of per-release scripts that do just what's needed for the upgrade proper (like, do one and only one reingest) 3) inclusion of supersedes/deprecated-by info in upgrade scripts that do, in fact, supersede (or are deprecated by) early scripts 4) a (say) perl script to read all the upgrade scripts and apply them in order, as needed, based on config.upgrade_log -- Mike Rylander | President | Equinox Software, Inc. / The Open Source Experts | phone: 1-877-OPEN-ILS (673-6457) | email: mi...@esilibrary.com | web: http://www.esilibrary.com On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
I should note for the mailing list that all the numbered examples I gave were entirely fictional and meant to be representative of the concept contained there-in, not any actual working values for any real versions of Evergreen. -- Ben On Fri, Mar 13, 2015 at 4:00 PM, Ben Shum bs...@biblio.org wrote: Without getting into too detailed or specifics, but While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I would say that this is the main reason our library system has stuck to master so that we have an unbroken chain of upgrade script by the numbers without any confusion of when to apply X or Y, etc. Perhaps not helpful, but some background thoughts to the mix. -- Ben On Fri, Mar 13, 2015 at 3:47 PM, Chris Sharp csh...@georgialibraries.org wrote: Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/ -- Benjamin Shum Evergreen Systems Manager Bibliomation, Inc. 24 Wooster Ave. Waterbury, CT 06708 203-577-4070, ext. 113 -- Benjamin Shum Evergreen Systems Manager Bibliomation, Inc. 24 Wooster Ave. Waterbury, CT 06708 203-577-4070, ext. 113
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Thanks for the quick and detailed response, Ben. While the numbers for upgrade scripts are purely sequential in master, they are not complete and sequential for upgrades in release branches, at least once you start talking maintenance releases since we do not always backport every upgrade script to previous versions of Evergreen. So if an upgrade script went 0001, 0002, 0003, etc. through release 2.7.0, then we started developing for 2.8 series and the numbers continued, 0004, 0005, 0006, etc. then there might be gaps where things did not backport cleanly. So if 0005 was backported, but 0004 and 0006 was not, then the upgrade scripts in 2.7 might look like 0001, 0002, 0003, 0005, while the chain would remain unbroken in 2.8/master. If you then upgraded from 2.7 to 2.8, how would you know which you missed or didn't already get? What if 0005 applied a fix for something that was changed in 0004. So if you ran 0004 and not 0005 (a second time during the upgrade), your system breaks with an older bad upgrade script function. I would be interested in seeing a system that did something where the Evergreen 2.5.1-2.7.0 upgrade script (for example) was really just a wrapper that said run scripts 0001, 0002, 0003, 0005, and 0010. It would also be nice if something in the database (maybe expanding config.upgrade_log?) would know which scripts constituted which version upgrade. People running sequentially through the scripts to stay on master are already out-of-band as far as releases go, so they would be left to manage the process on their own. Though it would be useful for the system to know that the system they are running is equivalent to 2.7.2 + scripts 0045 and 0046 for comparison purposes. Creating a giant version-upgrade script is how things have been done to this point, but moving between major versions can still be fraught with strangeness in the upgrade scripts even if you face each one on its own. That said, we're getting better about making good upgrade scripts that don't stack against each other or cause disruptions. I agree with this. The last three upgrades have been much smoother from my perspective. Thanks! Chris -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Galen, - The monolithic scripts allow for consolidating multiple point updates into a set of statements that run more quickly. Arguably this is more of a potential for efficiency than something that has always been taken advantage of -- but sometimes, it has. - Of more import, since the monolithic scripts wrap all of the changes in one or two transactions, if something goes wrong, the transaction rolls back and the schema remains in a state that corresponds to a released version. In contrast, if something goes wrong while going through a sequence of point upgrades and one gets stuck, the database could be in a state that doesn't correspond with *any* released version -- which could be awkward if one is nearing the end of a maintenance window. Then again, if one hasn't done a dry run first... Yeah, I can see the advantage of the full rollback. I wonder if the wrapper/controller script I'm imagining (and Mike mentions in his steps to make this actually happen response could have some sort of rollback option (though I know it's Postgres transactions that we depend on for rollback at this point). I should emphasize I'm listing some reasons why the monolithic scripts have existed -- I'm not suggesting that they're always superior to other approaches. Understood, thanks! -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Since that's true... Couldn't we develop some sort of upgrade mechanism that just aggregates and runs each of the constituent scripts? What is the reasoning behind stringing them into the longer monolithic scripts since running through the numbered scripts provides the same outcome? (Asking the full list, not Galen specifically). - Original Message - From: Galen Charlton g...@esilibrary.com To: Evergreen Discussion Group open-ils-general@list.georgialibraries.org Sent: Friday, March 13, 2015 3:31:33 PM Subject: Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7 Hi, On Fri, Mar 13, 2015 at 1:51 PM, Lazar, Alexey Vladimirovich alexey.la...@mnsu.edu wrote: Since an Open-ILS/src/sql/Pg/version-upgrade script is a subset of several specific Open-ILS/src/sql/Pg/upgrade/ scripts, is there any harm in just applying the scripts separately, in the order they appear, to get the database up to a certain “version number? That approach will work just fine, though checking through all of the point schema upgrades you plan to apply and seeing if there are any redundant bib reingests that you can skip can save you some time. Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org -- Chris Sharp PINES System Administrator Georgia Public Library Service 1800 Century Place, Suite 150 Atlanta, Georgia 30345 (404) 235-7147 csh...@georgialibraries.org http://pines.georgialibraries.org/
Re: [OPEN-ILS-GENERAL] Upgrade Script for 2.6.4 to 2.7
Hi, On Fri, Mar 13, 2015 at 4:22 PM, Chris Sharp csh...@georgialibraries.org wrote: Yeah, I can see the advantage of the full rollback. I wonder if the wrapper/controller script I'm imagining (and Mike mentions in his steps to make this actually happen response could have some sort of rollback option (though I know it's Postgres transactions that we depend on for rollback at this point). At first blush, I don't think that would be easy to do as such -- although one could envision a wrapper script that took out the begin/commit pairs in the individual point updates and ran the whole shebang in a single transaction, there are times when multiple transactions are required if one is both altering the structure of a table and altering data in it. Consequently, a rollback in case of a failure would be accomplished by restore from a base backup (or potentially, doing a PITR recovery to a specified time). But let's see if I can include a variation of this sentence in every single response I make in this thread: doing test runs of EG upgrades is a Good Thing. :) Regards, Galen -- Galen Charlton Infrastructure and Added Services Manager Equinox Software, Inc. / The Open Source Experts email: g...@esilibrary.com direct: +1 770-709-5581 cell: +1 404-984-4366 skype: gmcharlt web:http://www.esilibrary.com/ Supporting Koha and Evergreen: http://koha-community.org http://evergreen-ils.org
Re: [OPEN-ILS-GENERAL] Serials and magazines icon in the hitlist
Hi Gislaine, That icon, along with several others, came with the Multi-Valued Fields / Composite Record Attribute definitions in 2.6. The icon is indeed derived from fixed fields. There is some documentation on how to configure these record attributes at http://docs.evergreen-ils.org/2.7/_marc_record_attributes.html. I hope this helps! Kathy On 03/13/2015 11:10 AM, gislaine.hame...@statcan.gc.ca wrote: Hello List, Could you provide some information regarding the icon shown in the screen capture? Is it derived from a fixed field or MARC tag? We’ve just migrated to 2.7.3 from 2.4.1, so not sure which version this came with or the functionality. Can it be turned on or off? Thanks, Gislaine Hamelin Integrated Library Systems Coordinator | Coordonnatrice du système integrée de la bibliotheque Library Services | Services bibliothèque R.H. Coats Building | Immeuble R.-H.-Coats / Floor | Étage 2 Q Statistics Canada | 100 Tunney's Pasture Driveway, Ottawa ON K1A 0T6 Statistique Canada | 100, promenade Tunney's Pasture, Ottawa ON K1A 0T6 gislaine.hame...@statcan.gc.ca Telephone | Téléphone 613-762-5443 Facsimile | Télécopieur 613-951-0939 Government of Canada | Gouvernement du Canada -- Kathy Lussier Project Coordinator Massachusetts Library Network Cooperative (508) 343-0128 kluss...@masslnc.org Twitter: http://www.twitter.com/kmlussier