Re: [SOLVED] libpkg, sqlite and database problems prevent building any packages
On 07/08/2013 02:00, Thomas Mueller wrote: At first, with sqlite3 not listed as a dependency of pkg, I wouldn't have realized what needed fixing. That's deliberate. If pkg(8) had to install other packages as dependencies of itself, it would make bootstrapping pkg on a new system a bit difficult. So sqlite is bundled into the pkg source tree, along with a few other external bits. That pkg(8) uses sqlite is not exactly secret -- there's a whole man page for pkg-shell(8) which talks about using the sqlite3 console, and sqlite is clearly mentioned in pkg(8) and other man pages. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey signature.asc Description: OpenPGP digital signature
Re: [SOLVED] libpkg, sqlite and database problems prevent building any packages
On 07/08/2013 02:00, Thomas Mueller wrote: At first, with sqlite3 not listed as a dependency of pkg, I wouldn't have realized what needed fixing. That's deliberate. If pkg(8) had to install other packages as dependencies of itself, it would make bootstrapping pkg on a new system a bit difficult. So sqlite is bundled into the pkg source tree, along with a few other external bits. That pkg(8) uses sqlite is not exactly secret -- there's a whole man page for pkg-shell(8) which talks about using the sqlite3 console, and sqlite is clearly mentioned in pkg(8) and other man pages. Cheers, Matthew Dr Matthew J Seaman MA, D.Phil. man pkg-shell doesn't show much on using the sqlite3 console, mainly points why for the most part it should not be used. Maybe my rescue was the better way or at least the safer way? Tom ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
[SOLVED] libpkg, sqlite and database problems prevent building any packages
I solved the impasse by moving /var/db/pkg/local.sqlite to another place where it wouldn't be found, in this case /var/tmp . Then I went into $PORTSDIR/ports-mgmt/pkg and ran make deinstall install to update pkg. That produced a new, small, /var/db/pkg/local.sqlite which I renamed to local.sqlite.2 in preparation for restoring the older local.sqlite . Then, with the new pkg installed but the ghost of the old pkg remaining in local.sqlite, I ran, again, from $PORTSDIR/ports-mgmt/pkg make deinstall install to put local.sqlite in sync with the installation of updated pkg. I think the pkg database is restored now. At first, with sqlite3 not listed as a dependency of pkg, I wouldn't have realized what needed fixing. Tom ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
libpkg, sqlite and database problems prevent building any packages
I have smaller FreeBSD 9.1-STABLE and 9.2-BETA2 installations that I update
less frequently than the main hard-drive installation.
On the USB-stick installations, I find I can't upgrade or build any ports
because of a database mess. I get error messages mentioning sqlite even for
packages that don't depend on sqlite3.
I tried to build or upgrade devel/subversion, ports-mgmt/pkg, databases/sqlite3
and ports-mgmt/portmaster, and get error messages every time like
=== Starting check for runtime dependencies
=== Gathering dependency list for ports-mgmt/pkg from ports
pkg-static: warning: database version 19 is newer than libpkg(3) version 12,
but still compatible
pkg-static: sqlite: no such column: infos (pkgdb.c:1086)
=== No dependencies for ports-mgmt/pkg
pkg-static: warning: database version 19 is newer than libpkg(3) version 12,
but still compatible
pkg-static: sqlite: no such column: infos (pkgdb.c:1086)
pkg-static: warning: database version 19 is newer than libpkg(3) version 12,
but still compatible
pkg-static: sqlite: no such column: infos (pkgdb.c:1086)
=== Installing for pkg-1.1.4_1
=== Generating temporary packing list
=== Checking if ports-mgmt/pkg already installed
pkg-static: warning: database version 19 is newer than libpkg(3) version 12,
but still compatible
pkg-static: sqlite: no such column: infos (pkgdb.c:1086)
*** [check-already-installed] Error code 74
Stop in /BETA1/usr/ports/ports-mgmt/pkg.
*** [/BETA1/usr/ports/ports-mgmt/pkg/worki386/.install_done.pkg._usr_local]
Error code 1
Stop in /BETA1/usr/ports/ports-mgmt/pkg.
=== Installation of pkg-1.1.4_1 (ports-mgmt/pkg) failed
=== Aborting update
=== Update for ports-mgmt/pkg failed
=== Aborting update
=== Update for converters/libiconv failed
=== Aborting update
=== Update for devel/gettext failed
=== Aborting update
=== Update for devel/gmake failed
=== Aborting update
=== Update for devel/icu failed
=== Aborting update
=== Killing background jobs
Terminated
=== You can restart from the point of failure with this command line:
portmaster flags databases/sqlite3 devel/icu devel/gmake devel/gettext
converters/libiconv ports-mgmt/pkg
=== Exiting
I also get messages even when running pkg, this is the new pkgng.
You might want to see my /etc/make.conf .
Main hard drive installation:
PORTSDIR=/BETA1/usr/ports
PACKAGES=/usr/packages
WRKDIR=${.CURDIR}/work9amd64
# WITH_NEW_XORG=yes
# WITH_KMS=yes
WITH_PKGNG=yes
OPTIONS_SET=DOCS NLS
mpop_SET=GNUTLS NLS
# added by use.perl 2013-06-16 03:51:03
PERL_VERSION=5.16.3
for USB-stick installation with 9.2-BETA2 amd64:
PORTSDIR=/BETA1/usr/ports
PACKAGES=/usr/packages
WRKDIR=${.CURDIR}/workusb64
WITH_PKGNG=yes
OPTIONS_SET.mpop=GNUTLS NLS
# added by use.perl 2012-09-23 09:49:32
PERL_VERSION=5.14.2
for USB-stick installation with 9.1-STABLE i386:
PORTSDIR=/BETA1/usr/ports
PACKAGES=/usr/packages
WRKDIR=${.CURDIR}/worki386
# WITH_NEW_XORG=yes
# WITH_KMS=yes
WITH_PKGNG=yes
OPTIONS_SET=DOCS NLS
# WITHOUT_MODULES= usb/ulpt # This does not work
SRC_BASE=/STABLE1/usr/src
# added by use.perl 2013-06-17 12:56:10
PERL_VERSION=5.16.3
I know there was a major change in database format when subversion was updated
in ports from 1.7 to 1.8, but would this cause the problems that are driving me
crazy? There ought to be a way to resolve this, but I can't see what to do,
everything I try leads to the same error messages.
Tom
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: libpkg, sqlite and database problems prevent building any packages
On Mon, Aug 5, 2013 at 1:42 PM, Thomas Mueller
mueller6...@bellsouth.net wrote:
I have smaller FreeBSD 9.1-STABLE and 9.2-BETA2 installations that I update
less frequently than the main hard-drive installation.
On the USB-stick installations, I find I can't upgrade or build any ports
because of a database mess. I get error messages mentioning sqlite even for
packages that don't depend on sqlite3.
I tried to build or upgrade devel/subversion, ports-mgmt/pkg,
databases/sqlite3 and ports-mgmt/portmaster, and get error messages every
time like
=== Starting check for runtime dependencies
=== Gathering dependency list for ports-mgmt/pkg from ports
pkg-static: warning: database version 19 is newer than libpkg(3) version 12,
but still compatible
pkg-static: sqlite: no such column: infos (pkgdb.c:1086)
=== No dependencies for ports-mgmt/pkg
pkg-static: warning: database version 19 is newer than libpkg(3) version 12,
but still compatible
pkg-static: sqlite: no such column: infos (pkgdb.c:1086)
pkg-static: warning: database version 19 is newer than libpkg(3) version 12,
but still compatible
pkg-static: sqlite: no such column: infos (pkgdb.c:1086)
=== Installing for pkg-1.1.4_1
=== Generating temporary packing list
=== Checking if ports-mgmt/pkg already installed
pkg-static: warning: database version 19 is newer than libpkg(3) version 12,
but still compatible
pkg-static: sqlite: no such column: infos (pkgdb.c:1086)
*** [check-already-installed] Error code 74
Stop in /BETA1/usr/ports/ports-mgmt/pkg.
*** [/BETA1/usr/ports/ports-mgmt/pkg/worki386/.install_done.pkg._usr_local]
Error code 1
Stop in /BETA1/usr/ports/ports-mgmt/pkg.
=== Installation of pkg-1.1.4_1 (ports-mgmt/pkg) failed
=== Aborting update
=== Update for ports-mgmt/pkg failed
=== Aborting update
=== Update for converters/libiconv failed
=== Aborting update
=== Update for devel/gettext failed
=== Aborting update
=== Update for devel/gmake failed
=== Aborting update
=== Update for devel/icu failed
=== Aborting update
=== Killing background jobs
Terminated
=== You can restart from the point of failure with this command line:
portmaster flags databases/sqlite3 devel/icu devel/gmake
devel/gettext converters/libiconv ports-mgmt/pkg
=== Exiting
I also get messages even when running pkg, this is the new pkgng.
You might want to see my /etc/make.conf .
Main hard drive installation:
PORTSDIR=/BETA1/usr/ports
PACKAGES=/usr/packages
WRKDIR=${.CURDIR}/work9amd64
# WITH_NEW_XORG=yes
# WITH_KMS=yes
WITH_PKGNG=yes
OPTIONS_SET=DOCS NLS
mpop_SET=GNUTLS NLS
# added by use.perl 2013-06-16 03:51:03
PERL_VERSION=5.16.3
for USB-stick installation with 9.2-BETA2 amd64:
PORTSDIR=/BETA1/usr/ports
PACKAGES=/usr/packages
WRKDIR=${.CURDIR}/workusb64
WITH_PKGNG=yes
OPTIONS_SET.mpop=GNUTLS NLS
# added by use.perl 2012-09-23 09:49:32
PERL_VERSION=5.14.2
for USB-stick installation with 9.1-STABLE i386:
PORTSDIR=/BETA1/usr/ports
PACKAGES=/usr/packages
WRKDIR=${.CURDIR}/worki386
# WITH_NEW_XORG=yes
# WITH_KMS=yes
WITH_PKGNG=yes
OPTIONS_SET=DOCS NLS
# WITHOUT_MODULES= usb/ulpt # This does not work
SRC_BASE=/STABLE1/usr/src
# added by use.perl 2013-06-17 12:56:10
PERL_VERSION=5.16.3
I know there was a major change in database format when subversion was
updated in ports from 1.7 to 1.8, but would this cause the problems that are
driving me crazy? There ought to be a way to resolve this, but I can't see
what to do, everything I try leads to the same error messages.
Tom
This has nothing to do with subversion. Your pkg(8) client is out of
date compared to the /var/db/pkg/local.sqlite database.
-Kimmo
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: libpkg, sqlite and database problems prevent building any packages
I know there was a major change in database format when subversion was updated in ports from 1.7 to 1.8, but would this cause the problems that are driving me crazy? There ought to be a way to resolve this, but I can't see what to do, everything I try leads to the same error messages. Tom This has nothing to do with subversion. Your pkg(8) client is out of date compared to the /var/db/pkg/local.sqlite database. -Kimmo I could see my pkg client is out of date, but how do I update it? Attempts to update all gave me those error messages. Installation fails. Tom ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: libpkg, sqlite and database problems prevent building any packages
On 05/08/2013 14:30, Thomas Mueller wrote: I could see my pkg client is out of date, but how do I update it? Attempts to update all gave me those error messages. Installation fails. Your package database has got into an inconsistent state. pkg(8) is attempting to auto-update the schema to the latest version, but failing because it's trying to remove an 'infos' column from a table where that column has apparently already been removed. (Likely this situation has come about because pkg got killed in the middle of doing this update previously.) How happy are you to get down'n'dirty with the source code and running SQL from the command line? If you look at the updates pkg is attempting to run shown here: https://github.com/freebsd/pkg/blob/master/libpkg/private/db_upgrades.h would you be capable of looking at the DB schema, working out which of those updates had been applied, aplying any outstanding ones by hand and then setting the user version to 19 by: sql PRAGMA user_version = 19 ; ? If not, check in /var/backups for a good copy of your local.sqlite database and try and restore from there. Unfortunately, there's no guarantee that any backup copy doesn't have the same inconsistencies as your live copy. We need to make pkg(8) databases more resilient to the effects of SIGHUP or similar while they are elbows-deep in the bowels of the DB schema... Cheers, Matthew ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: libpkg, sqlite and database problems prevent building any packages
On Mon, Aug 5, 2013 at 5:03 PM, Matthew Seaman matt...@freebsd.org wrote: On 05/08/2013 14:30, Thomas Mueller wrote: I could see my pkg client is out of date, but how do I update it? Attempts to update all gave me those error messages. Installation fails. Your package database has got into an inconsistent state. pkg(8) is attempting to auto-update the schema to the latest version, but failing because it's trying to remove an 'infos' column from a table where that column has apparently already been removed. (Likely this situation has come about because pkg got killed in the middle of doing this update previously.) How happy are you to get down'n'dirty with the source code and running SQL from the command line? If you look at the updates pkg is attempting to run shown here: https://github.com/freebsd/pkg/blob/master/libpkg/private/db_upgrades.h would you be capable of looking at the DB schema, working out which of those updates had been applied, aplying any outstanding ones by hand and then setting the user version to 19 by: sql PRAGMA user_version = 19 ; ? If not, check in /var/backups for a good copy of your local.sqlite database and try and restore from there. Unfortunately, there's no guarantee that any backup copy doesn't have the same inconsistencies as your live copy. We need to make pkg(8) databases more resilient to the effects of SIGHUP or similar while they are elbows-deep in the bowels of the DB schema... Cheers, Matthew Ok this is then different problem than what I saw when I reverted back from pkg-devel to pkg, the error message was the same though. -Kimmo ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: libpkg, sqlite and database problems prevent building any packages
On 05/08/2013 15:24, Kimmo Paasiala wrote: Ok this is then different problem than what I saw when I reverted back from pkg-devel to pkg, the error message was the same though. Yeah -- we don't really have any support for down-reving the schema in local.sqlite at the moment, so going from pkg-devel to pkg can be traumatic. It should be OK right now as there's no difference in the DB schemas for what's currently in ports. repo.sqlite should have significantly more forwards and backwards compatibility, although with the latest versions of pkg, you aren't downloading a pre-built sqlite DB, but a collection of YAML documents containing selected package metadata. So all that up- and down- DB schema version changing code is pretty much redundant. C'est la vie. Cheers, Matthew ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: libpkg, sqlite and database problems prevent building any packages
On 05/08/2013 14:30, Thomas Mueller wrote: I could see my pkg client is out of date, but how do I update it? Attempts to update all gave me those error messages. Installation fails. Matthew Seaman responded: Your package database has got into an inconsistent state. pkg(8) is attempting to auto-update the schema to the latest version, but failing because it's trying to remove an 'infos' column from a table where that column has apparently already been removed. (Likely this situation has come about because pkg got killed in the middle of doing this update previously.) How happy are you to get down'n'dirty with the source code and running SQL from the command line? If you look at the updates pkg is attempting to run shown here: https://github.com/freebsd/pkg/blob/master/libpkg/private/db_upgrades.h would you be capable of looking at the DB schema, working out which of those updates had been applied, aplying any outstanding ones by hand and then setting the user version to 19 by: sql PRAGMA user_version = 19 ; ? If not, check in /var/backups for a good copy of your local.sqlite database and try and restore from there. Unfortunately, there's no guarantee that any backup copy doesn't have the same inconsistencies as your live copy. We need to make pkg(8) databases more resilient to the effects of SIGHUP or similar while they are elbows-deep in the bowels of the DB schema... Trying to do what you suggest would in all likelihood be an exercise in futility, especially when I am very tired from being unable to sleep. I didn't realize pkgng was so non-sturdy. I wonder what would happen if I delete /var/db/pkg/local.sqlite, or perhaps better, move it to another location and name so as not to burn my bridges. But anything so I can upgrade pkg. Does pkgng have no means of recovery? Can pkg regenerate a database, and from what? My /var/backups on the USB-stick 9.2-BETA2 amd64 installation shows total 280 -rw-r--r-- 1 root wheel1674 Apr 2 2012 aliases.bak -rw-r--r-- 1 root wheel 436 Apr 2 2012 group.bak -rw--- 1 root wheel1500 Apr 2 2012 master.passwd.bak -rw-r--r-- 1 root wheel 269144 Dec 13 2012 pkgdb.bak.tbz -rw-r--r-- 1 root wheel 129 Jul 16 2012 pkgdb.bak.tbz.2 pkg.bak.tbz looks like the database from the old format. My /var/backups on the USB-stick 9.1-STABLE i386 installation shows total 140 -rw-r--r-- 1 root wheel1688 Jan 20 2013 aliases.bak -rw-r--r-- 1 root wheel1674 Jun 6 2012 aliases.bak2 -rw-r--r-- 1 root wheel 481 Jan 17 2013 group.bak -rw-r--r-- 1 root wheel 449 Jul 13 2012 group.bak2 -rw--- 1 root wheel1761 Jan 17 2013 master.passwd.bak -rw--- 1 root wheel1500 Jun 6 2012 master.passwd.bak2 -rw-r--r-- 1 root wheel8524 May 28 03:01 pkgdb.bak.tbz -rw-r--r-- 1 root wheel 130 Jan 19 2013 pkgdb.bak.tbz.2 -rw-r--r-- 1 root wheel 100352 May 28 04:29 pkgng.db pkgng.db looks like the new format, but that was before I built wine and dependencies. Tom ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: libpkg, sqlite and database problems prevent building any packages
Bottom posting below; will reply in better format when/if I get another webmail client maybe... the text is below the next occurance of bottom posting From: Thomas Mueller mueller6...@bellsouth.net To: freebsd-ports@freebsd.org Sent: Monday, August 5, 2013 8:36 AM Subject: Re: libpkg, sqlite and database problems prevent building any packages On 05/08/2013 14:30, Thomas Mueller wrote: I could see my pkg client is out of date, but how do I update it? Attempts to update all gave me those error messages. Installation fails. Matthew Seaman responded: Your package database has got into an inconsistent state. pkg(8) is attempting to auto-update the schema to the latest version, but failing because it's trying to remove an 'infos' column from a table where that column has apparently already been removed. (Likely this situation has come about because pkg got killed in the middle of doing this update previously.) How happy are you to get down'n'dirty with the source code and running SQL from the command line? If you look at the updates pkg is attempting to run shown here: https://github.com/freebsd/pkg/blob/master/libpkg/private/db_upgrades.h would you be capable of looking at the DB schema, working out which of those updates had been applied, aplying any outstanding ones by hand and then setting the user version to 19 by: sql PRAGMA user_version = 19 ; ? If not, check in /var/backups for a good copy of your local.sqlite database and try and restore from there. Unfortunately, there's no guarantee that any backup copy doesn't have the same inconsistencies as your live copy. We need to make pkg(8) databases more resilient to the effects of SIGHUP or similar while they are elbows-deep in the bowels of the DB schema... Trying to do what you suggest would in all likelihood be an exercise in futility, especially when I am very tired from being unable to sleep. I didn't realize pkgng was so non-sturdy. I wonder what would happen if I delete /var/db/pkg/local.sqlite, or perhaps better, move it to another location and name so as not to burn my bridges. But anything so I can upgrade pkg. Does pkgng have no means of recovery? Can pkg regenerate a database, and from what? My /var/backups on the USB-stick 9.2-BETA2 amd64 installation shows total 280 -rw-r--r-- 1 root wheel 1674 Apr 2 2012 aliases.bak -rw-r--r-- 1 root wheel 436 Apr 2 2012 group.bak -rw--- 1 root wheel 1500 Apr 2 2012 master.passwd.bak -rw-r--r-- 1 root wheel 269144 Dec 13 2012 pkgdb.bak.tbz -rw-r--r-- 1 root wheel 129 Jul 16 2012 pkgdb.bak.tbz.2 pkg.bak.tbz looks like the database from the old format. My /var/backups on the USB-stick 9.1-STABLE i386 installation shows total 140 -rw-r--r-- 1 root wheel 1688 Jan 20 2013 aliases.bak -rw-r--r-- 1 root wheel 1674 Jun 6 2012 aliases.bak2 -rw-r--r-- 1 root wheel 481 Jan 17 2013 group.bak -rw-r--r-- 1 root wheel 449 Jul 13 2012 group.bak2 -rw--- 1 root wheel 1761 Jan 17 2013 master.passwd.bak -rw--- 1 root wheel 1500 Jun 6 2012 master.passwd.bak2 -rw-r--r-- 1 root wheel 8524 May 28 03:01 pkgdb.bak.tbz -rw-r--r-- 1 root wheel 130 Jan 19 2013 pkgdb.bak.tbz.2 -rw-r--r-- 1 root wheel 100352 May 28 04:29 pkgng.db pkgng.db looks like the new format, but that was before I built wine and dependencies. .. .. Bottom posting... Non-sturdy? , but this sort of showstopper is what IMHO should necessite a huge problem-resolving flowchart, before pkg(ng) fully obsoletes its legacy alternative... not to mention the possibility of pkg(ng) not having enough memory, on some installs where before /var/db/pkg would have enough. Just a 2c. As in, if I *have to* migrate to pkg/, I need something from which to copy the succint parts to a few post-its on the monitor... (think HUGE...FLOWCHART...) Thanks. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
