Re: [ovirt-users] upgrade to 4.1
On Sun, Apr 23, 2017 at 2:53 PM, Fabrice Bacchella wrote: > >> Le 23 avr. 2017 à 07:59, Yedidyah Bar David a écrit : >> >> > >> The main reason we require this is for pg_dump/pg_restore which are ran >> during setup/rollback (if needed). pg_dump can't know for sure that all >> the changes in the db were done using a client of its own version (that >> is, current machine usually), and if indeed a newer client was used, it >> might have used features that pg_dump of the lower version doesn't know >> how to back up (and especially pg_restore does not know how to restore). >> See also [1]. I seem to have tested there (can't remember anymore, see >> comment 13) 9.2 client with 9.5 server and it didn't work. pg_dump(1) >> manpage says: >> >> Because pg_dump is used to transfer data to newer versions of >> PostgreSQL, the output of pg_dump can be expected to load into >> PostgreSQL server versions newer than pg_dump's version. pg_dump can >> also dump from PostgreSQL servers older than its own version. >> (Currently, servers back to version 7.0 are supported.) However, >> pg_dump cannot dump from PostgreSQL servers newer than its own major >> version; it will refuse to even try, rather than risk making an invalid >> dump. Also, it is not guaranteed that pg_dump's output can be loaded >> into a server of an older major version — not even if the dump was >> taken from a server of that version. Loading a dump file into an older >> server may require manual editing of the dump file to remove syntax not >> understood by the older server. Use of the --quote-all-identifiers >> option is recommended in cross-version cases, as it can prevent >> problems arising from varying reserved-word lists in different >> PostgreSQL versions. >> > > I don't get it, but I don't know pg so I might be wrong. > > You have a client application (like ovirt) written using features from V1 of > pg. Right. > > It's running on a server where version V2 is installing. For good reasons, V2 > >= V1 is needed. Indeed. > > The server is running a version V3. Again V3 >= V1 is needed. Except for > major version, does V3 => V2 is really needed ? For oVirt itself, no. > > And for backup the problem is the same. It must probably know every features > used in the application (so again being V1 or more). Why does it needs to > match both V2 and V3 ? It will probably fits V2 is installation is the same. > But that not mandatory. In a java application, the client library might be a > jar provided by the application and pg_dump a tool installed with native os > packaging. But how can complain against V3 ? pg_dump is indeed a tool installed from the OS. I copied above the relevant part of its manpage - if you think it does a wrong thing, perhaps better continue the discussion on postgresql lists. For our concern, the only relevant fact is that we decided to backup the db using pg_dump during setup, and to restore it on rollback. If you want to make this backup optional, please open an RFE, as already requested, and we can discuss this there. > > But with ovirt we have V1=V2=V3, Not exactly. Using your terminology, we have V1<=V2, V2=V3. > even for for patch level (9.4.8 against 9.4.11). What kind of feature that > ovirt don't know about might be missing ? I don't think ovirt might know > about any 9.4 since you talked about version 9.2 as the official supported > version. At the time, oVirt didn't check versions at all, and so failed on a certain combination, and it was decided to require V2=V3, as a simple and effective solution. This was also specifically discussed there, as you can see for yourself, open to the public for review/comments. Check especially comments 9 to 19 (in bz 1331168). If you think we need a more delicate test, please open an RFE, preferably providing the details I suggested that should be considered for one. Also please recall that you have a third, very simple, IMHO, option. If for any reason you decided to have your server with version V3, simply install V3 on your client machine (the oVirt engine). This will be simple and solve all problems. If your concern is that the OS does not supply pg V3, then it applies also to the OS of the server. If that is a different OS, you are welcome to port oVirt engine to that OS so you can have all you want at once. You also have a fourth option - patch oVirt to not check for the version, by reverting the patch that introduced this check (might require a bit more work than "git revert", but not much): https://gerrit.ovirt.org/59941 Of course, this will essentially be a fork, and I'd personally not recommend it. But it's still an option. Best, -- Didi ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] upgrade to 4.1
> Le 23 avr. 2017 à 07:59, Yedidyah Bar David a écrit : > > > The main reason we require this is for pg_dump/pg_restore which are ran > during setup/rollback (if needed). pg_dump can't know for sure that all > the changes in the db were done using a client of its own version (that > is, current machine usually), and if indeed a newer client was used, it > might have used features that pg_dump of the lower version doesn't know > how to back up (and especially pg_restore does not know how to restore). > See also [1]. I seem to have tested there (can't remember anymore, see > comment 13) 9.2 client with 9.5 server and it didn't work. pg_dump(1) > manpage says: > > Because pg_dump is used to transfer data to newer versions of > PostgreSQL, the output of pg_dump can be expected to load into > PostgreSQL server versions newer than pg_dump's version. pg_dump can > also dump from PostgreSQL servers older than its own version. > (Currently, servers back to version 7.0 are supported.) However, > pg_dump cannot dump from PostgreSQL servers newer than its own major > version; it will refuse to even try, rather than risk making an invalid > dump. Also, it is not guaranteed that pg_dump's output can be loaded > into a server of an older major version — not even if the dump was > taken from a server of that version. Loading a dump file into an older > server may require manual editing of the dump file to remove syntax not > understood by the older server. Use of the --quote-all-identifiers > option is recommended in cross-version cases, as it can prevent > problems arising from varying reserved-word lists in different > PostgreSQL versions. > I don't get it, but I don't know pg so I might be wrong. You have a client application (like ovirt) written using features from V1 of pg. It's running on a server where version V2 is installing. For good reasons, V2 >= V1 is needed. The server is running a version V3. Again V3 >= V1 is needed. Except for major version, does V3 => V2 is really needed ? And for backup the problem is the same. It must probably know every features used in the application (so again being V1 or more). Why does it needs to match both V2 and V3 ? It will probably fits V2 is installation is the same. But that not mandatory. In a java application, the client library might be a jar provided by the application and pg_dump a tool installed with native os packaging. But how can complain against V3 ? But with ovirt we have V1=V2=V3, even for for patch level (9.4.8 against 9.4.11). What kind of feature that ovirt don't know about might be missing ? I don't think ovirt might know about any 9.4 since you talked about version 9.2 as the official supported version. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] upgrade to 4.1
> Le 23 avr. 2017 à 10:56, Yedidyah Bar David a écrit : > > On Sun, Apr 23, 2017 at 10:59 AM, Fabrice Bacchella > wrote: >> >>> Le 23 avr. 2017 à 07:59, Yedidyah Bar David a écrit : > >> > > And it was not in the release notes, it's not funny to get this warning > after starting the upgrade >>> >>> This isn't a new test, see above bug. >>> >>> Are you sure it's the first time you see it? Perhaps you upgraded your pg >>> server only after the last upgrade of the engine? >>> >> >> That's might true about the test, but not the autovacuum_vacuum_scale_factor >> and others. >> >> And any way, about the test, having a different version of pg library and pg >> server is quite common when you have a centralized pg. You might plan an >> upgrade any time and don't expect every client to complain about a minor >> release, that's quite unexpected. > > Very well, so I detailed above a suggestion about what you can do. > > Either make backup (and therefore rollback) optional, or make the test > more delicate. Neither seems trivial to me in terms of risk (although > they might be quite simple in the amount of code to change). > >> >> The bug https://bugzilla.redhat.com/show_bug.cgi?id=1331168 is about major >> version, I got complain from ovirt about patch level. That's not the same >> thing. > > I didn't check PG's official terminology. The bug was about 9.2 > against 9.5. In general, "9" is the major version, and the bug still > applied. It might be that they consider "9.2" to be the major version, > and "9.5" a different major version. From the very bug description: > I'm testing migration from 3.6 EL6 with remote DBs (PG 8.x) to 4.0 EL7 while > still using remote DBs (which I restored from backup). > Problem is 4.0 supports PG 9.x but I was still original PG 8.x. engine-setup > from 4.0 should know it needs PG 9.x and should check PG version even for > remote DBs. In the extract from the man page, the main problems are also about major versions. You test 9.2 against 9.5. What I have failling is 9.4.8 against 9.4.11. That is very different from all the case described. > >> And the solution that ovirt propose is to have the server lying about >> version to all of it's client. > > Where did you see this? Please set: autovacuum_vacuum_scale_factor = 0.01 autovacuum_analyze_scale_factor = 0.075 autovacuum_max_workers = 6 server_version = 9.4.8 Are in the log from the upgrade command. > > The text you copied is: "Please use a Postgresql server of version '9.4.8'" > >> What can I do if every client I use require such a thing ? > > You mean: > 1. I want a single server with some version X > 2. I want different clients with different versions X1, X2, ... > 3. More than one of these clients requires the server to be the same > version as the client > ? > > I'd say - upgrade all such clients to the version of the server, or > make these clients not require that :-) Yes make these clients not require that. Ovirt is one of those complaining client. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] upgrade to 4.1
On Sun, Apr 23, 2017 at 10:59 AM, Fabrice Bacchella wrote: > >> Le 23 avr. 2017 à 07:59, Yedidyah Bar David a écrit : > And it was not in the release notes, it's not funny to get this warning after starting the upgrade >> >> This isn't a new test, see above bug. >> >> Are you sure it's the first time you see it? Perhaps you upgraded your pg >> server only after the last upgrade of the engine? >> > > That's might true about the test, but not the autovacuum_vacuum_scale_factor > and others. > > And any way, about the test, having a different version of pg library and pg > server is quite common when you have a centralized pg. You might plan an > upgrade any time and don't expect every client to complain about a minor > release, that's quite unexpected. Very well, so I detailed above a suggestion about what you can do. Either make backup (and therefore rollback) optional, or make the test more delicate. Neither seems trivial to me in terms of risk (although they might be quite simple in the amount of code to change). > > The bug https://bugzilla.redhat.com/show_bug.cgi?id=1331168 is about major > version, I got complain from ovirt about patch level. That's not the same > thing. I didn't check PG's official terminology. The bug was about 9.2 against 9.5. In general, "9" is the major version, and the bug still applied. It might be that they consider "9.2" to be the major version, and "9.5" a different major version. > And the solution that ovirt propose is to have the server lying about version > to all of it's client. Where did you see this? The text you copied is: "Please use a Postgresql server of version '9.4.8'" > What can I do if every client I use require such a thing ? You mean: 1. I want a single server with some version X 2. I want different clients with different versions X1, X2, ... 3. More than one of these clients requires the server to be the same version as the client ? I'd say - upgrade all such clients to the version of the server, or make these clients not require that :-) Best, -- Didi ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] upgrade to 4.1
> Le 23 avr. 2017 à 07:59, Yedidyah Bar David a écrit : >>> >>> >>> And it was not in the release notes, it's not funny to get this warning >>> after starting the upgrade > > This isn't a new test, see above bug. > > Are you sure it's the first time you see it? Perhaps you upgraded your pg > server only after the last upgrade of the engine? > That's might true about the test, but not the autovacuum_vacuum_scale_factor and others. And any way, about the test, having a different version of pg library and pg server is quite common when you have a centralized pg. You might plan an upgrade any time and don't expect every client to complain about a minor release, that's quite unexpected. The bug https://bugzilla.redhat.com/show_bug.cgi?id=1331168 is about major version, I got complain from ovirt about patch level. That's not the same thing. And the solution that ovirt propose is to have the server lying about version to all of it's client. What can I do if every client I use require such a thing ? ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] upgrade to 4.1
On Thu, Apr 20, 2017 at 4:50 PM, Fabrice Bacchella < fabrice.bacche...@orange.fr> wrote: > I tried to upgrade ovirt to version 4.1 from 4.0 and got: > > Found the following problems in PostgreSQL configuration for the > Engine database: >autovacuum_vacuum_scale_factor required to be at most 0.01 >autovacuum_analyze_scale_factor required to be at most 0.075 >autovacuum_max_workers required to be at least 6 >Postgresql client version is '9.4.8', whereas the version on > XXX is '9.4.11'. Please use a Postgresql server of version '9.4.8'. > I'm not sure we support 9.4.8, we use whatever comes with EL7, which AFAIR, is 9.2.x Y. > Please set: >autovacuum_vacuum_scale_factor = 0.01 >autovacuum_analyze_scale_factor = 0.075 >autovacuum_max_workers = 6 >server_version = 9.4.8 > in postgresql.conf on ''. Its location is usually > /var/lib/pgsql/data , or somewhere under /etc/postgresql* . > > I'm a little afraid about that. Does ovirt want pg to lies about it's > version ? It's a shared instance so what about other tools that access it ? > Is there some explanation about the meaning of those values ? > > And it was not in the release notes, it's not funny to get this warning > after starting the upgrade > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] upgrade to 4.1
On Thu, Apr 20, 2017 at 6:59 PM, Simone Tiraboschi wrote: > > On Thu, Apr 20, 2017 at 3:50 PM, Fabrice Bacchella > wrote: >> >> I tried to upgrade ovirt to version 4.1 from 4.0 and got: >> >> Found the following problems in PostgreSQL configuration for the >> Engine database: >>autovacuum_vacuum_scale_factor required to be at most 0.01 >>autovacuum_analyze_scale_factor required to be at most 0.075 >>autovacuum_max_workers required to be at least 6 >>Postgresql client version is '9.4.8', whereas the version on >> XXX is '9.4.11'. Please use a Postgresql server of version '9.4.8'. >> Please set: >>autovacuum_vacuum_scale_factor = 0.01 >>autovacuum_analyze_scale_factor = 0.075 >>autovacuum_max_workers = 6 >>server_version = 9.4.8 >> in postgresql.conf on ''. Its location is usually >> /var/lib/pgsql/data , or somewhere under /etc/postgresql* . >> >> I'm a little afraid about that. Does ovirt want pg to lies about it's >> version ? It's a shared instance so what about other tools that access it ? >> Is there some explanation about the meaning of those values ? > > > engine-setup it's comparing the version of the local psql client with the > version reported by the remote postgresql server as for: > https://bugzilla.redhat.com/show_bug.cgi?id=1331168 > > Currently it's a strict comparison; maybe we should be more flexibly > regarding .z versions; Didi? Is there a good reason to not have them exactly equal? The main reason we require this is for pg_dump/pg_restore which are ran during setup/rollback (if needed). pg_dump can't know for sure that all the changes in the db were done using a client of its own version (that is, current machine usually), and if indeed a newer client was used, it might have used features that pg_dump of the lower version doesn't know how to back up (and especially pg_restore does not know how to restore). See also [1]. I seem to have tested there (can't remember anymore, see comment 13) 9.2 client with 9.5 server and it didn't work. pg_dump(1) manpage says: Because pg_dump is used to transfer data to newer versions of PostgreSQL, the output of pg_dump can be expected to load into PostgreSQL server versions newer than pg_dump's version. pg_dump can also dump from PostgreSQL servers older than its own version. (Currently, servers back to version 7.0 are supported.) However, pg_dump cannot dump from PostgreSQL servers newer than its own major version; it will refuse to even try, rather than risk making an invalid dump. Also, it is not guaranteed that pg_dump's output can be loaded into a server of an older major version — not even if the dump was taken from a server of that version. Loading a dump file into an older server may require manual editing of the dump file to remove syntax not understood by the older server. Use of the --quote-all-identifiers option is recommended in cross-version cases, as it can prevent problems arising from varying reserved-word lists in different PostgreSQL versions. So personally I'd keep things as they are, unless someone can present a good reason to do a more delicate test. If someone indeed wants that, please: 1. Open a bz and state what you want (make backup optional, use some other less strict test on version numbers, something else?). 2. If you want more delicate tests, please verify that what you ask for indeed works, and provide details. E.g., try this: * setup an engine with 9.4.8 client and server (can be on same machine) * pg_dump this engine's db with 9.4.8 to dump1 * pg_restore dump1 to a 9.4.11 server. Did this work? * use pg_dump 9.4.8 to dump the db from the 9.4.11 server to dump2. Did this work? * use pg_restore 9.4.8 to restore dump2 to the 9.4.11 server. Did this work? * Compare dump1 and dump2. What are the differences? It might make sense to do similar tests with the opposite - client is newer than server. Also with "more far away" versions (say 9.4 and 9.5). Also perhaps other tests, especially if you manage to find new features in 9.4.11 compared to 9.4.8 (hopefully there aren't any) and in 9.5 compared to 9.4. [1] https://bugzilla.redhat.com/1331168 > > The other checks has been introduced for performance and scaling reasons. > >> >> >> And it was not in the release notes, it's not funny to get this warning >> after starting the upgrade This isn't a new test, see above bug. Are you sure it's the first time you see it? Perhaps you upgraded your pg server only after the last upgrade of the engine? Best, -- Didi ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] upgrade to 4.1
On Thu, Apr 20, 2017 at 3:50 PM, Fabrice Bacchella < fabrice.bacche...@orange.fr> wrote: > I tried to upgrade ovirt to version 4.1 from 4.0 and got: > > Found the following problems in PostgreSQL configuration for the > Engine database: >autovacuum_vacuum_scale_factor required to be at most 0.01 >autovacuum_analyze_scale_factor required to be at most 0.075 >autovacuum_max_workers required to be at least 6 >Postgresql client version is '9.4.8', whereas the version on > XXX is '9.4.11'. Please use a Postgresql server of version '9.4.8'. > Please set: >autovacuum_vacuum_scale_factor = 0.01 >autovacuum_analyze_scale_factor = 0.075 >autovacuum_max_workers = 6 >server_version = 9.4.8 > in postgresql.conf on ''. Its location is usually > /var/lib/pgsql/data , or somewhere under /etc/postgresql* . > > I'm a little afraid about that. Does ovirt want pg to lies about it's > version ? It's a shared instance so what about other tools that access it ? > Is there some explanation about the meaning of those values ? > engine-setup it's comparing the version of the local psql client with the version reported by the remote postgresql server as for: https://bugzilla.redhat.com/show_bug.cgi?id=1331168 Currently it's a strict comparison; maybe we should be more flexibly regarding .z versions; Didi? The other checks has been introduced for performance and scaling reasons. > > And it was not in the release notes, it's not funny to get this warning > after starting the upgrade > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
