Bug#984555: gavodachs2-server: fails to install/upgrade. breaks on executing SQL script.
On Fri, Jun 11, 2021 at 02:02:51PM +0200, Andreas Beckmann wrote: > Unfortunately, this is not the case currently because we have > > Package: postgresql-13-pgsphere > Conflicts: postgresql-pgsphere (<< 1.1.1+2020) > which causes the pg-11 extensions to be removed before pg_upgrade can be > performed. But I think these Conflicts+Replaces can be safely dropped since > all files in these two packages are in versioned paths. I agree the conflicts needs to be removed. This won't let us have a postgresql-pgsphere virtual package depending on "pgsphere for the current default postgres", though, will it? > It will only be dropped automatically if you enable autoremoval of unneeded > packages during the upgrade. But then postgresql-11 will probably go away as > well... Yes -- I suppose whoever runs postgres on Debian through a dist-upgrade has that off.
Bug#984555: gavodachs2-server: fails to install/upgrade. breaks on executing SQL script.
On Tue, 4 May 2021 11:32:40 +0200 Markus Demleitner wrote: Ah. That might be a solution. One thing that needs to be ensured, though, is that on upgrades, the old pgsphere packages do not get removed until the operator asks dpkg to explicitly. Perhaps stating the obvious, here's what needs to happen when upgrading from, say pg 11 to 13: (1) normal dist-upgrade pulls in (hopefully) postgres-13, keeps postgres-11 running as normal (hence, pgsphere-11 must still be there). (2) operator runs pg_upgrade (for which both pg-13 and pg-11 must be present with all necessary extensions) (3) the last step of pg_upgrade is to make pg-13 the default db server, and pg-11 isn't started any more (4) the operator can now purge postgres-11 together with the version 11 extensions. Thanks for making this clear ;-) Unfortunately, this is not the case currently because we have Package: postgresql-13-pgsphere Source: pgsphere Version: 1.1.1+2020-10-20-1 Replaces: postgresql-pgsphere (<< 1.1.1+2020) Provides: postgresql-pgsphere Depends: libc6 (>= 2.29), libgcc-s1 (>= 3.0), libhealpix-cxx2 (>= 3.60.0), libstdc++6 (>= 5.2), postgresql-13 Conflicts: postgresql-pgsphere (<< 1.1.1+2020) Package: postgresql-13-q3c Source: postgresql-q3c Version: 2.0.0-4 Replaces: postgresql-q3c (<< 2.0.0-2) Provides: postgresql-q3c Depends: postgresql-13, libc6 (>= 2.29) Conflicts: postgresql-q3c (<< 2.0.0-2) which causes the pg-11 extensions to be removed before pg_upgrade can be performed. But I think these Conflicts+Replaces can be safely dropped since all files in these two packages are in versioned paths. I'll file two RC bugs ... My hunch is that when pgsphere-11 is installed as an auto-dependency of pgsphere, it will be dropped in step (1), at which point pg-11 probably won't even start any more, or at least will be unable to dump for the pg_upgrade. It will only be dropped automatically if you enable autoremoval of unneeded packages during the upgrade. But then postgresql-11 will probably go away as well... Andreas
Bug#984555: gavodachs2-server: fails to install/upgrade. breaks on executing SQL script.
Hi Andreas, On Mon, May 03, 2021 at 07:19:52PM +0200, Andreas Beckmann wrote: > Followup-For: Bug #984555 > I think I have an idea what is happening: The upgrade from buster to > bullseye does not install postgresql-13-pgsphere and postgresql-13-q3c > but keeps postgresql-pgsphere and postgresql-q3c from buster installed. Yes, that's my analysis, too. > This is a problem of the postgresql-* packages which switched from real > packages postgresql-foo providing virtual postgresql-xx-foo to real > packages postgresql-xx-foo providing virtual postgresql-foo. > IMO it was wrong turning postgresql-foo into virtual packages, > especially if multiple postgresql-xx-foo are expected to be > co-installable, s.t. there are multiple packages providing > postgresql-foo at the same time. postgresql-foo should have stayed a > real package depending on the default version of postgresql-xx-foo. Ah. That might be a solution. One thing that needs to be ensured, though, is that on upgrades, the old pgsphere packages do not get removed until the operator asks dpkg to explicitly. Perhaps stating the obvious, here's what needs to happen when upgrading from, say pg 11 to 13: (1) normal dist-upgrade pulls in (hopefully) postgres-13, keeps postgres-11 running as normal (hence, pgsphere-11 must still be there). (2) operator runs pg_upgrade (for which both pg-13 and pg-11 must be present with all necessary extensions) (3) the last step of pg_upgrade is to make pg-13 the default db server, and pg-11 isn't started any more (4) the operator can now purge postgres-11 together with the version 11 extensions. My hunch is that when pgsphere-11 is installed as an auto-dependency of pgsphere, it will be dropped in step (1), at which point pg-11 probably won't even start any more, or at least will be unable to dump for the pg_upgrade. > and reducing that to > > Depends: adduser, members, postgresql-13-pgsphere, postgresql-13-q3c, > python3-gavo (= 2.3+dfsg-2) I've just done this, and I *think* that should do the right thing when upgrading to bullseye. I can't say I like it much, though, in particular because it's going to be a permanent chore in maintaining the explicit versions. What I'd *really* want to say is: "I depend on a postgres with the pgsphere and q3c extensions installed, listening on port 5432." How far we get there with the existing Depends logic would be something I'd probably ask the postgres folks. > > Perhaps also a > > Breaks: postgresql-pgsphere (<< 1.1.1+2020), postgresql-q3c (<< 2) > > would work for the upgrades to bullseye. Nah, that, I think, would be a disaster, because it'd throw out the extensions before step (2) above could run, no? Even though I think the problem is addressed for bullseye, I'm leaving this bug open because I'm hoping for some better way to solve this.
Bug#984555: gavodachs2-server: fails to install/upgrade.
On Tue, Mar 16, 2021 at 10:49:24AM +0100, Markus Demleitner wrote: > What is already making me nervous at a second glance: There's > postgresql-11-pgsphere, but on bullseye postgres will be postgres 13. > I'd hence suspect that the postgres DaCHS is talking to is not the > one that has pgsphere. Meanwhile, I'm pretty sure this is what happened here: You already had a postgres 11 on your boxes, but at some point a postgres 13 was put on the box as well, while postgres 11 lingered around. Now, when installing DaCHS, the system satisified the pgsphere dependency with postgres-11-pgsphere. This, however, means that there is no pgsphere on the operational pgsphere, which is what kills DaCHS (and, really, there is very little point running DaCHS without these extensions, which is why it doesn't even try to limp on). Summing up: If you un-install postgres 11, things should start to work. Is there a good way to prevent this confusing situation? I don't know, but I'll ask around. -- Markus
Bug#984555: gavodachs2-server: fails to install/upgrade.
> I tried your suggestion, but didn't get very far: > $ psql gavo > psql: error: FATAL: role "jrv" does not exist > > Let me know if there's something else you want to cross-check. This one is easy to fix: postgres wants to know who it talks to, and it doesn't know you. The easiest fix is to become the user dachsroot, which is recommended for DaCHS operations anyway. An alternative that would generally work in other situations on Debian: sudo -u postgres psql gavo But this bug report made actually try running DaCHS on unstable, and while I cannot reproduce the installation failure, that made me realise some additional adaptations to unstable are necessary. What is already making me nervous at a second glance: There's postgresql-11-pgsphere, but on bullseye postgres will be postgres 13. I'd hence suspect that the postgres DaCHS is talking to is not the one that has pgsphere. So, what would sudo -u postgres psql gavo -c "select version()" say? -- Markus
Bug#984555: Fwd: Bug#984555: gavodachs2-server: fails to install/upgrade. breaks on executing SQL script.
On Fri, 5 Mar 2021 12:08:06 +0100 Markus Demleitner <
msdem...@ari.uni-heidelberg.de> wrote:
> > createdb: database creation failed: ERROR: database "gavo" already
exists
> > Creation of gavo database failed; assuming it's already there
> > and carrying on.
>
> This means that there has been a DaCHS on that machine before, i.e.,
> that this is an upgrade. Is that right? If so, then this message
> is expected (and we ought to hide it one of these days).
>
> > /usr/lib/python3/dist-packages/gavo/user/initdachs.py:310: UserWarning:
SQL
> > script file for /usr/share/postgresql/contrib/pg_sphere.sql not found.
>
> This is where things go wrong. DaCHS should not even try to load the
> pgsphere SQL, as all pgspheres that can possibly in use now already
> use the new Postgres extension system.
>
> So, why does it try this? This:
>
> > Versions of packages gavodachs2-server depends on:
> > ii postgresql-pgsphere [postgresql-11-pgsphere] 1.1.1+2018.10.13-1
>
> is essentially as it should be. I wonder if it has been like this at
> the time of the DaCHS installation.
>
> But let's see if things are properly there now. Could you run
>
> psql gavo
>
> and then in psql
>
> select * from pg_available_extensions where name='pg_sphere';
>
> What does that say?
>
>
For what it's worth, I had the same error:
Setting up gavodachs2-server (2.3+dfsg-1) ...
createdb: database creation failed: ERROR: database "gavo" already exists
Creation of gavo database failed; assuming it's already there
and carrying on.
createuser: creation of new role failed: ERROR: role "dachsroot" already
exists
/usr/lib/python3/dist-packages/gavo/user/initdachs.py:310: UserWarning: SQL
script file for /usr/share/postgresql/contrib/pg_sphere.sql not found.
There are many reasons why that may be ok, but unless you know what you are
doing, you probably should install the corresponding postgres extension.
warnings.warn("SQL script file for %s not found. There are many"
/usr/lib/python3/dist-packages/gavo/user/initdachs.py:328: UserWarning: SQL
script file /usr/share/postgresql/contrib/pg_sphere.sql failed. Try
running manually using psql. While it hasn't run, the pgSphere extension
is not available.
warnings.warn("SQL script file %s failed. Try running manually"
/usr/lib/python3/dist-packages/gavo/user/initdachs.py:310: UserWarning: SQL
script file for /usr/share/postgresql/contrib/q3c.sql not found. There are
many reasons why that may be ok, but unless you know what you are doing,
you probably should install the corresponding postgres extension.
warnings.warn("SQL script file for %s not found. There are many"
/usr/lib/python3/dist-packages/gavo/user/initdachs.py:328: UserWarning: SQL
script file /usr/share/postgresql/contrib/q3c.sql failed. Try running
manually using psql. While it hasn't run, the q3c extension is not
available.
warnings.warn("SQL script file %s failed. Try running manually"
*** Error: While executing DB query:
select * from dc.rdmeta where sourceRD='__system__/dc_tables'
the database engine reported:
ERROR: schema "dc" does not exist
LINE 1: select * from dc.rdmeta where
sourceRD='__system__/dc_tables...
^
I tried your suggestion, but didn't get very far:
$ psql gavo
psql: error: FATAL: role "jrv" does not exist
Let me know if there's something else you want to cross-check.
(I am a fairly knowledgeable Debian user, but not an SQL user. )
- Jim Van Zandt
Bug#984555: Fwd: Bug#984555: gavodachs2-server: fails to install/upgrade. breaks on executing SQL script.
> createdb: database creation failed: ERROR: database "gavo" already exists > Creation of gavo database failed; assuming it's already there > and carrying on. This means that there has been a DaCHS on that machine before, i.e., that this is an upgrade. Is that right? If so, then this message is expected (and we ought to hide it one of these days). > /usr/lib/python3/dist-packages/gavo/user/initdachs.py:310: UserWarning: SQL > script file for /usr/share/postgresql/contrib/pg_sphere.sql not found. This is where things go wrong. DaCHS should not even try to load the pgsphere SQL, as all pgspheres that can possibly in use now already use the new Postgres extension system. So, why does it try this? This: > Versions of packages gavodachs2-server depends on: > ii postgresql-pgsphere [postgresql-11-pgsphere] 1.1.1+2018.10.13-1 is essentially as it should be. I wonder if it has been like this at the time of the DaCHS installation. But let's see if things are properly there now. Could you run psql gavo and then in psql select * from pg_available_extensions where name='pg_sphere'; What does that say?
Bug#984555: gavodachs2-server: fails to install/upgrade. breaks on executing SQL script.
Package: gavodachs2-server
Version: 2.3+dfsg-1
Severity: important
Dear Maintainer,
* What led up to the situation?
broken install
* What exactly did you do (or not do) that was effective (or
ineffective)?
tried to update system. apt breaks on gavodachs2-server
package is probably broken
* What was the outcome of this action?
from the output of apt:
A instalar gavodachs2-server (2.3+dfsg-1) ...
createdb: database creation failed: ERROR: database "gavo" already exists
Creation of gavo database failed; assuming it's already there
and carrying on.
createuser: creation of new role failed: ERROR: role "dachsroot" already exists
/usr/lib/python3/dist-packages/gavo/user/initdachs.py:310: UserWarning: SQL
script file for /usr/share/postgresql/contrib/pg_sphere.sql not found. There
are many reasons why that may be ok, but unless you know what you are doing,
you probably should install the corresponding postgres extension.
warnings.warn("SQL script file for %s not found. There are many"
/usr/lib/python3/dist-packages/gavo/user/initdachs.py:328: UserWarning: SQL
script file /usr/share/postgresql/contrib/pg_sphere.sql failed. Try running
manually using psql. While it hasn't run, the pgSphere extension is not
available.
warnings.warn("SQL script file %s failed. Try running manually"
/usr/lib/python3/dist-packages/gavo/user/initdachs.py:310: UserWarning: SQL
script file for /usr/share/postgresql/contrib/q3c.sql not found. There are
many reasons why that may be ok, but unless you know what you are doing, you
probably should install the corresponding postgres extension.
warnings.warn("SQL script file for %s not found. There are many"
/usr/lib/python3/dist-packages/gavo/user/initdachs.py:328: UserWarning: SQL
script file /usr/share/postgresql/contrib/q3c.sql failed. Try running manually
using psql. While it hasn't run, the q3c extension is not available.
warnings.warn("SQL script file %s failed. Try running manually"
*** Error: At /usr/lib/python3/dist-
packages/gavo/resources/inputs/__system__/adql.rd, line 166: Execution
of SQL script create_user_functions failed: type spoint does not exist
dpkg: erro ao processar o pacote gavodachs2-server (--configure):
o subprocesso instalado, do pacote gavodachs2-server, o script
post-installation retornou erro do status de saída 1
* What outcome did you expect instead?
a clean instalation
-- System Information:
Debian Release: bullseye/sid
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1,
'experimental')
Architecture: amd64 (x86_64)
Kernel: Linux 5.10.0-3-amd64 (SMP w/8 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE,
TAINT_UNSIGNED_MODULE
Locale: LANG=pt_PT.UTF-8, LC_CTYPE=pt_PT.UTF-8 (charmap=UTF-8),
LANGUAGE=pt:en_GB
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages gavodachs2-server depends on:
ii adduser 3.118
ii members 20080128.1+nmu1
ii postgresql-pgsphere [postgresql-11-pgsphere] 1.1.1+2018.10.13-1
ii postgresql-q3c [postgresql-11-q3c]1.6.0-1
ii python3-gavo 2.3+dfsg-1
gavodachs2-server recommends no packages.
gavodachs2-server suggests no packages.
-- no debconf information
