Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Andriy Gapon]

on 29/03/2011 23:29 Baptiste Daroussin said the following:
 ok let's try to say it simpler :) the main goal is to keep it simple
 for now, simple and rock solid, so that we can replace pkg_install and
 do some cleanup in the ports tree, add the must have features while
 doing that. And only when we will be ready for that and that portmgr
 have decided that it is mature enough to replace pkg_install, only
 after that we will start improving with new features and new changes.
 
 I thinks changing the package name scheme is not a must have
 feature, it for sure is and intresting feature, but what about pushing
 to after the first stable release? managing architecture as we plan to
 do it is enough imho.

Oh, yes, I realize all this and totally agree with it.
Given how huge and how visible our ports and packages systems are, it's better 
to
be slow and cautious.
All the ideas that I suggested were more for the next step than for now.

Thank you for the work!
-- 
Andriy Gapon
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Baptiste Daroussin]

2011/3/31 Andriy Gapon a...@freebsd.org:
 on 29/03/2011 23:29 Baptiste Daroussin said the following:
 ok let's try to say it simpler :) the main goal is to keep it simple
 for now, simple and rock solid, so that we can replace pkg_install and
 do some cleanup in the ports tree, add the must have features while
 doing that. And only when we will be ready for that and that portmgr
 have decided that it is mature enough to replace pkg_install, only
 after that we will start improving with new features and new changes.

 I thinks changing the package name scheme is not a must have
 feature, it for sure is and intresting feature, but what about pushing
 to after the first stable release? managing architecture as we plan to
 do it is enough imho.

 Oh, yes, I realize all this and totally agree with it.
 Given how huge and how visible our ports and packages systems are, it's 
 better to
 be slow and cautious.
 All the ideas that I suggested were more for the next step than for now.


And noted in my personnal TODO list :)

 Thank you for the work!
 --
 Andriy Gapon

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Julien Laffaye]

On Tue, Mar 29, 2011 at 5:15 AM, Tim Kientzle kient...@freebsd.org wrote:
 II. Package signing.

 That would be really nice.

 Right know we only planned to sign the repo database, so we can trust
 the sah256 of the packages stored in the database. Then if the package
 has the same sha256 as the one in the repo database it is considered
 trusted.
 If we want a per-package signing, we would have a tarball in a tarball.

 I really expected this to have been mentioned already, but this approach 
 (tarball in a tarball) is taken by Debian packages, and I don't remember 
 hearing of any issues related to it.  I don't think it's worth discounting 
 from the start without giving some considerationg, but I will defer to the 
 people actually doing the work.

 If you use libarchive-style streaming, it's even
 pretty straightforward to read and extract such
 things without having to create a bunch of
 temporary files.

 You just need to be careful about compression.

Agreed, if we dont want to verify the signature, we can extract the
tarball in the tarball efficiently.

But to verify the signature, we have to read the tarball in the
tarball twice: the first time to compute the digest and verify the
signature, the second time to do the real extraction.
So I guess that the tarball containing the real package archive and
the signature should be uncompressed. The real package archive would
be compressed, though.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Super Bisquit]

I'm just going to clarify a statement I made earlier on this thread in order
to remove some possible misconceptions. One can only boot 32bit PPC on a
32bit PPC machines and have it work properly. The same applies for 64bit ppc
machines.



On Tue, Mar 29, 2011 at 8:11 AM, Julien Laffaye jlaff...@freebsd.orgwrote:

 On Tue, Mar 29, 2011 at 5:15 AM, Tim Kientzle kient...@freebsd.org
 wrote:
  II. Package signing.
 
  That would be really nice.
 
  Right know we only planned to sign the repo database, so we can trust
  the sah256 of the packages stored in the database. Then if the package
  has the same sha256 as the one in the repo database it is considered
  trusted.
  If we want a per-package signing, we would have a tarball in a tarball.
 
  I really expected this to have been mentioned already, but this approach
 (tarball in a tarball) is taken by Debian packages, and I don't remember
 hearing of any issues related to it.  I don't think it's worth discounting
 from the start without giving some considerationg, but I will defer to the
 people actually doing the work.
 
  If you use libarchive-style streaming, it's even
  pretty straightforward to read and extract such
  things without having to create a bunch of
  temporary files.
 
  You just need to be careful about compression.

 Agreed, if we dont want to verify the signature, we can extract the
 tarball in the tarball efficiently.

 But to verify the signature, we have to read the tarball in the
 tarball twice: the first time to compute the digest and verify the
 signature, the second time to do the real extraction.
 So I guess that the tarball containing the real package archive and
 the signature should be uncompressed. The real package archive would
 be compressed, though.
 ___
 freebsd-hack...@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
 To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Andriy Gapon]

on 28/03/2011 21:22 Julien Laffaye said the following:
 On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper gcoo...@freebsd.org wrote:
 III. Package naming that includes architecture, major OS version (for 
 API/ABI),
 maybe more.

 This could be provided in the manifest. Doing it in the filename sort
 of turns into a mess, as I've discovered working at Cisco :).

 
 Actually, it *is* in the +MANIFEST of pkgng packages archives :-)

Well, by the package name I meant not only a package file name.
Let's imagine that we do support installing i386 packages on amd64 in parallel 
to
amd64 packages.  And for some reason I want to have both 32-bit and 64-bit
versions of, say, firefox; e.g. for benchmarking.  If the packages would have 
the
same name, then that would be impossible.

I think that having some thing in package name in addition to package metadata
could have certain benefits.

-- 
Andriy Gapon
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Baptiste Daroussin]

2011/3/29 Andriy Gapon a...@freebsd.org:
 on 28/03/2011 21:22 Julien Laffaye said the following:
 On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper gcoo...@freebsd.org wrote:
 III. Package naming that includes architecture, major OS version (for 
 API/ABI),
 maybe more.

 This could be provided in the manifest. Doing it in the filename sort
 of turns into a mess, as I've discovered working at Cisco :).


 Actually, it *is* in the +MANIFEST of pkgng packages archives :-)

 Well, by the package name I meant not only a package file name.
 Let's imagine that we do support installing i386 packages on amd64 in 
 parallel to
 amd64 packages.  And for some reason I want to have both 32-bit and 64-bit
 versions of, say, firefox; e.g. for benchmarking.  If the packages would have 
 the
 same name, then that would be impossible.

 I think that having some thing in package name in addition to package metadata
 could have certain benefits.

 --
 Andriy Gapon


I understand but I think pkgng is already quite radical changement.
More change is taking the risk that it would be rejected in the end,
we still do not have any reply from portmgr, there is no insurance
pkgng will in the end replace pkg_install. Currently pkgng requires
only very few changes from the ports infrastruture, I don't know the
cost of changing the name scheme.

If I'm not clear enough, supporting both 32bits and 64bits packages at
the same time on amd64 or arches that could support this kind of
installation, is a large change we don't want to take the
responsability of :) and implementing this in pkgng would significate
we already choose how it should work.

regards,
Bapt
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Baptiste Daroussin]

2011/3/29 Baptiste Daroussin b...@freebsd.org:
 2011/3/29 Andriy Gapon a...@freebsd.org:
 on 28/03/2011 21:22 Julien Laffaye said the following:
 On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper gcoo...@freebsd.org wrote:
 III. Package naming that includes architecture, major OS version (for 
 API/ABI),
 maybe more.

 This could be provided in the manifest. Doing it in the filename sort
 of turns into a mess, as I've discovered working at Cisco :).


 Actually, it *is* in the +MANIFEST of pkgng packages archives :-)

 Well, by the package name I meant not only a package file name.
 Let's imagine that we do support installing i386 packages on amd64 in 
 parallel to
 amd64 packages.  And for some reason I want to have both 32-bit and 64-bit
 versions of, say, firefox; e.g. for benchmarking.  If the packages would 
 have the
 same name, then that would be impossible.

 I think that having some thing in package name in addition to package 
 metadata
 could have certain benefits.

 --
 Andriy Gapon


 I understand but I think pkgng is already quite radical changement.
 More change is taking the risk that it would be rejected in the end,
 we still do not have any reply from portmgr, there is no insurance
 pkgng will in the end replace pkg_install. Currently pkgng requires
 only very few changes from the ports infrastruture, I don't know the
 cost of changing the name scheme.

 If I'm not clear enough, supporting both 32bits and 64bits packages at
 the same time on amd64 or arches that could support this kind of
 installation, is a large change we don't want to take the
 responsability of :) and implementing this in pkgng would significate
 we already choose how it should work.

 regards,
 Bapt


seems it was not clear :)

ok let's try to say it simpler :) the main goal is to keep it simple
for now, simple and rock solid, so that we can replace pkg_install and
do some cleanup in the ports tree, add the must have features while
doing that. And only when we will be ready for that and that portmgr
have decided that it is mature enough to replace pkg_install, only
after that we will start improving with new features and new changes.

I thinks changing the package name scheme is not a must have
feature, it for sure is and intresting feature, but what about pushing
to after the first stable release? managing architecture as we plan to
do it is enough imho.

But I can be wrong.

regards,
Bapt
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Andriy Gapon]

on 25/03/2011 12:11 Baptiste Daroussin said the following:
 Hi all,
 
 miwi@ launched the new thing called Experimental Call For Testing,
 it's our turn :)
 
 Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge
 contributor) have been working since the end of the last GSoC on a
 rewrite of pkg_install.
 
 pkgng is a binary package manager written from scratch for FreeBSD.
 
 After a long period of technology testing, (json, tinycdb, bdb, etc)
 and we now have achieved to implement the basic functionnality. We
 would greatly approciate to have some feedback, wider testing,
 patching, documenting etc, before implementing the higher level
 features.
 
 pkgng is built on top of a new libpkg, which allow to deal with the database 
 of
 installed packages, to deal with remote repositories, manage packages:
 creation, installation gathering informations, registering new ports.
 
 features supported are or will be :
 
 - smooth integration with bsd.port.mk (including bsd.pkg.mk line 2486)
 which allow  to have a bsd.port.mk which deal with both pkg_install
 and pkgng. (done in alpha)
 
 - the register command can analyse elf files when registering a new port to
 discover forgotten dependencies if necessary. (done in alpha using libelf)
 
 - the register command has two mode available : when dealing with old
 fashion ports it just registers the package, in new mode it does
 everything that would
 have been done by pkg add when installing the package : should display
 messages,  execute post-install, execute @exec etc. (old fashion done
 in the alpha)
 
 - pkg add supports two mode : the old fashion one (no real upgrade
 support) and  new one: upgrade scripts supported. (old fashion in the
 alpha)
 
 - new scripts supported +PREINSTALL +POSTINSTALL, +PREDEINSTALL,
 +POSTDEINSTALL, +PREUPGRADE, +POSTUPGRADE as well as the old fashion
 scripts : +INSTALL +DEINSTALL +UPGRADE (all supported *UPGRADES aren't
 supported in the alpha)
 
 - new +MANIFEST (plist-like format) with new metadatas : options, arch, os
 version, etc. (done in the alpha)
 
 - pkgng supports checking arch of the package which means that users
 won't be able to install sparc64 binary package into amd64 machines.
 (not done yet)
 
 - a special architecture all allows to specify when a package can be used
 on every architecture. (not done yet)
 
 - @dirrm and @dirrmtry are now deprecated, pkgng can discover itself
 which directory has to be removed. (done in the alpha but needs love
 :))
 
 - new repository (apt-like feature) (only the repository generation is done)
 
 - real support for reverse dependency (no ugly +REQUIRED_BY) (done in the 
 alpha)
 
 - test unit (libcheck) on libpkg. (done in the alpha needs some more love)
 
 - many more

Perhaps I am too impatient :) but I would like to inquire about the following
features:

I. A provides/requires interface for packages.
Each package specified a list of files (and perhaps other entities) that it
provides and requires.  At the initial stage, without ports modifications, these
could be: (1) a list of all files installed by package for provides; (2) for
requires - an auto-generated list of dependencies based on required shared
libraries plus dependency specifications in ports.
I think that this kind of interface should help with using alternatives that
provide the same interface (e.g. like gamin vs fam).

II. Package signing.

III. Package naming that includes architecture, major OS version (for API/ABI),
maybe more.

IV. Convenient support for i386 packages on amd64.
Distinct installation directories, proper installation conflict
detection/avoidance between i386 and amd64 packages, proper library paths, etc.

And finally some exotic ideas - support for multiple package sources (when
different people build packages in different ways (e.g. with different options, 
or
different optimizations) from the same ports tree; support for multiple ports
sources.(when people maintain different ports tree (e.g. kde or gnome 
development
ports tree)).  Perhaps, with some compatibility/hierarchy support for packages 
and
ports sources.  But that's almost a pipe dream, so don't take it seriously :)

-- 
Andriy Gapon
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Garrett Cooper]

On Mon, Mar 28, 2011 at 10:44 AM, Andriy Gapon a...@freebsd.org wrote:
 on 25/03/2011 12:11 Baptiste Daroussin said the following:
 Hi all,

 miwi@ launched the new thing called Experimental Call For Testing,
 it's our turn :)

 Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge
 contributor) have been working since the end of the last GSoC on a
 rewrite of pkg_install.

 pkgng is a binary package manager written from scratch for FreeBSD.

 After a long period of technology testing, (json, tinycdb, bdb, etc)
 and we now have achieved to implement the basic functionnality. We
 would greatly approciate to have some feedback, wider testing,
 patching, documenting etc, before implementing the higher level
 features.

 pkgng is built on top of a new libpkg, which allow to deal with the database 
 of
 installed packages, to deal with remote repositories, manage packages:
 creation, installation gathering informations, registering new ports.

 features supported are or will be :

 - smooth integration with bsd.port.mk (including bsd.pkg.mk line 2486)
 which allow  to have a bsd.port.mk which deal with both pkg_install
 and pkgng. (done in alpha)

 - the register command can analyse elf files when registering a new port to
 discover forgotten dependencies if necessary. (done in alpha using libelf)

 - the register command has two mode available : when dealing with old
 fashion ports it just registers the package, in new mode it does
 everything that would
 have been done by pkg add when installing the package : should display
 messages,  execute post-install, execute @exec etc. (old fashion done
 in the alpha)

 - pkg add supports two mode : the old fashion one (no real upgrade
 support) and  new one: upgrade scripts supported. (old fashion in the
 alpha)

 - new scripts supported +PREINSTALL +POSTINSTALL, +PREDEINSTALL,
 +POSTDEINSTALL, +PREUPGRADE, +POSTUPGRADE as well as the old fashion
 scripts : +INSTALL +DEINSTALL +UPGRADE (all supported *UPGRADES aren't
 supported in the alpha)

 - new +MANIFEST (plist-like format) with new metadatas : options, arch, os
 version, etc. (done in the alpha)

 - pkgng supports checking arch of the package which means that users
 won't be able to install sparc64 binary package into amd64 machines.
 (not done yet)

 - a special architecture all allows to specify when a package can be used
 on every architecture. (not done yet)

 - @dirrm and @dirrmtry are now deprecated, pkgng can discover itself
 which directory has to be removed. (done in the alpha but needs love
 :))

 - new repository (apt-like feature) (only the repository generation is done)

 - real support for reverse dependency (no ugly +REQUIRED_BY) (done in the 
 alpha)

 - test unit (libcheck) on libpkg. (done in the alpha needs some more love)

 - many more

 Perhaps I am too impatient :) but I would like to inquire about the following
 features:

 I. A provides/requires interface for packages.
 Each package specified a list of files (and perhaps other entities) that it
 provides and requires.  At the initial stage, without ports modifications, 
 these
 could be: (1) a list of all files installed by package for provides; (2) for
 requires - an auto-generated list of dependencies based on required shared
 libraries plus dependency specifications in ports.
 I think that this kind of interface should help with using alternatives that
 provide the same interface (e.g. like gamin vs fam).

 II. Package signing.

That would be really nice.

 III. Package naming that includes architecture, major OS version (for 
 API/ABI),
 maybe more.

This could be provided in the manifest. Doing it in the filename sort
of turns into a mess, as I've discovered working at Cisco :).

 IV. Convenient support for i386 packages on amd64.
 Distinct installation directories, proper installation conflict
 detection/avoidance between i386 and amd64 packages, proper library paths, 
 etc.

There are other architectures that would benefit from this as well,
like powerpc 32-bit on 64-bit, MIPs 32-bit on n32, etc.

This involves more work than pkgng could provide IIRC as build
infrastructure would need to be fixed to look at and link against
usr/lib32 instead of usr/lib, unless you mean to rewrite the linker
stuff at install-time.

 And finally some exotic ideas - support for multiple package sources (when
 different people build packages in different ways (e.g. with different 
 options, or
 different optimizations) from the same ports tree; support for multiple ports
 sources.(when people maintain different ports tree (e.g. kde or gnome 
 development
 ports tree)).  Perhaps, with some compatibility/hierarchy support for 
 packages and
 ports sources.  But that's almost a pipe dream, so don't take it seriously :)

It would be nice. That's a request in the same general area that
Gentoo portage's overlay goes into, but I think that would require
rewriting certain bits of ports infrastructure to be extensible, not
extend pkgng in this 

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Julien Laffaye]

On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper gcoo...@freebsd.org wrote:
 On Mon, Mar 28, 2011 at 10:44 AM, Andriy Gapon a...@freebsd.org wrote:
 on 25/03/2011 12:11 Baptiste Daroussin said the following:
 Hi all,

 miwi@ launched the new thing called Experimental Call For Testing,
 it's our turn :)

 Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge
 contributor) have been working since the end of the last GSoC on a
 rewrite of pkg_install.

 pkgng is a binary package manager written from scratch for FreeBSD.

 After a long period of technology testing, (json, tinycdb, bdb, etc)
 and we now have achieved to implement the basic functionnality. We
 would greatly approciate to have some feedback, wider testing,
 patching, documenting etc, before implementing the higher level
 features.

 pkgng is built on top of a new libpkg, which allow to deal with the 
 database of
 installed packages, to deal with remote repositories, manage packages:
 creation, installation gathering informations, registering new ports.

 features supported are or will be :

 - smooth integration with bsd.port.mk (including bsd.pkg.mk line 2486)
 which allow  to have a bsd.port.mk which deal with both pkg_install
 and pkgng. (done in alpha)

 - the register command can analyse elf files when registering a new port to
 discover forgotten dependencies if necessary. (done in alpha using libelf)

 - the register command has two mode available : when dealing with old
 fashion ports it just registers the package, in new mode it does
 everything that would
 have been done by pkg add when installing the package : should display
 messages,  execute post-install, execute @exec etc. (old fashion done
 in the alpha)

 - pkg add supports two mode : the old fashion one (no real upgrade
 support) and  new one: upgrade scripts supported. (old fashion in the
 alpha)

 - new scripts supported +PREINSTALL +POSTINSTALL, +PREDEINSTALL,
 +POSTDEINSTALL, +PREUPGRADE, +POSTUPGRADE as well as the old fashion
 scripts : +INSTALL +DEINSTALL +UPGRADE (all supported *UPGRADES aren't
 supported in the alpha)

 - new +MANIFEST (plist-like format) with new metadatas : options, arch, os
 version, etc. (done in the alpha)

 - pkgng supports checking arch of the package which means that users
 won't be able to install sparc64 binary package into amd64 machines.
 (not done yet)

 - a special architecture all allows to specify when a package can be used
 on every architecture. (not done yet)

 - @dirrm and @dirrmtry are now deprecated, pkgng can discover itself
 which directory has to be removed. (done in the alpha but needs love
 :))

 - new repository (apt-like feature) (only the repository generation is done)

 - real support for reverse dependency (no ugly +REQUIRED_BY) (done in the 
 alpha)

 - test unit (libcheck) on libpkg. (done in the alpha needs some more love)

 - many more

 Perhaps I am too impatient :) but I would like to inquire about the following
 features:

 I. A provides/requires interface for packages.
 Each package specified a list of files (and perhaps other entities) that it
 provides and requires.  At the initial stage, without ports modifications, 
 these
 could be: (1) a list of all files installed by package for provides; (2) for
 requires - an auto-generated list of dependencies based on required shared
 libraries plus dependency specifications in ports.
 I think that this kind of interface should help with using alternatives that
 provide the same interface (e.g. like gamin vs fam).

Adding require/provide support in pkgng is kind of trivial. But ports
have to support it first.


 II. Package signing.

 That would be really nice.

Right know we only planned to sign the repo database, so we can trust
the sah256 of the packages stored in the database. Then if the package
has the same sha256 as the one in the repo database it is considered
trusted.
If we want a per-package signing, we would have a tarball in a tarball.


 III. Package naming that includes architecture, major OS version (for 
 API/ABI),
 maybe more.

 This could be provided in the manifest. Doing it in the filename sort
 of turns into a mess, as I've discovered working at Cisco :).


Actually, it *is* in the +MANIFEST of pkgng packages archives :-)

 IV. Convenient support for i386 packages on amd64.
 Distinct installation directories, proper installation conflict
 detection/avoidance between i386 and amd64 packages, proper library paths, 
 etc.

 There are other architectures that would benefit from this as well,
 like powerpc 32-bit on 64-bit, MIPs 32-bit on n32, etc.

 This involves more work than pkgng could provide IIRC as build
 infrastructure would need to be fixed to look at and link against
 usr/lib32 instead of usr/lib, unless you mean to rewrite the linker
 stuff at install-time.

I dont think we are going that way right now...


 And finally some exotic ideas - support for multiple package sources (when
 different people build packages in different 

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Benjamin Kaduk]

On Mon, 28 Mar 2011, Julien Laffaye wrote:


On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper gcoo...@freebsd.org wrote:

On Mon, Mar 28, 2011 at 10:44 AM, Andriy Gapon a...@freebsd.org wrote:


II. Package signing.


That would be really nice.


Right know we only planned to sign the repo database, so we can trust
the sah256 of the packages stored in the database. Then if the package
has the same sha256 as the one in the repo database it is considered
trusted.
If we want a per-package signing, we would have a tarball in a tarball.


I really expected this to have been mentioned already, but this approach 
(tarball in a tarball) is taken by Debian packages, and I don't remember 
hearing of any issues related to it.  I don't think it's worth discounting 
from the start without giving some considerationg, but I will defer to the 
people actually doing the work.


-Ben Kaduk
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Tim Kientzle]

 II. Package signing.
 
 That would be really nice.
 
 Right know we only planned to sign the repo database, so we can trust
 the sah256 of the packages stored in the database. Then if the package
 has the same sha256 as the one in the repo database it is considered
 trusted.
 If we want a per-package signing, we would have a tarball in a tarball.
 
 I really expected this to have been mentioned already, but this approach 
 (tarball in a tarball) is taken by Debian packages, and I don't remember 
 hearing of any issues related to it.  I don't think it's worth discounting 
 from the start without giving some considerationg, but I will defer to the 
 people actually doing the work.

If you use libarchive-style streaming, it's even
pretty straightforward to read and extract such
things without having to create a bunch of
temporary files.

You just need to be careful about compression.

Tim

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Baptiste Daroussin]

2011/3/29 Tim Kientzle kient...@freebsd.org:
 II. Package signing.

 That would be really nice.

 Right know we only planned to sign the repo database, so we can trust
 the sah256 of the packages stored in the database. Then if the package
 has the same sha256 as the one in the repo database it is considered
 trusted.
 If we want a per-package signing, we would have a tarball in a tarball.

 I really expected this to have been mentioned already, but this approach 
 (tarball in a tarball) is taken by Debian packages, and I don't remember 
 hearing of any issues related to it.  I don't think it's worth discounting 
 from the start without giving some considerationg, but I will defer to the 
 people actually doing the work.

 If you use libarchive-style streaming, it's even
 pretty straightforward to read and extract such
 things without having to create a bunch of
 temporary files.

 You just need to be careful about compression.

 Tim



ok but what is the problem with signing only the repository then rely on digest?

I am not sure we need more that this.

second question howto sign? pgp? ssl?

First would be the easiest way to go but we don't have in base
anything to check signatures (maybe we should in that case
investigating to import netpgp), ssl why not? but which algorithm?
what security officer would prefer?

We are ok to investigate that part, but we need more information about
what is expected.

regards,
Bapt
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Baptiste Daroussin]

2011/3/25 Marcin Wisnicki mwisnicki+free...@gmail.com:
 On Fri, 25 Mar 2011 11:11:11 +0100, Baptiste Daroussin wrote:

 pkgng is a binary package manager written from scratch for FreeBSD.


 Fantastic!

 I know it is quite too early but I already have one feature request ;)
 Perhaps it could be added to the TODO as a post-1.0 goal.

 = Generic extraction filters =

 Allow registration of custom filters that can alter/exclude/add? files
 during package extraction (installation).

 Examples of possible filters:
  - strip debug info
  - exclude development files (headers, static libs)
  - exclude unused translations
  - exclude documentation (all or just unknown languages)
  - generic glob/regex path filters
  - optional file groups defined in package (install time OPTIONS) ?

 Some sort of configuration mechanism with list of enabled filters and
 their options (like a list of languages to keep).

 Most of this can be implemented as a simple glob/regex matching but there
 are edge cases where packages have some non-standard layout or have to
 keep certain file in which case a package metadata should contain a list
 of exclusions/additions from/to above categories.

 Package manager should register only actually installed files but list of
 alterations should be also kept somewhere in database.


 What do you think ?

 ___
 freebsd-ports@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-ports
 To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org


This can still be discussed but I don't really like the idea that
users may alter packages an installation/extraction time, that would
lead to lots of potential buggy installation and report.
If user aren't happy with the packaging, they can poke the maintainer,
send PR, patch etc.

regards,
Bapt
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Marin Atanasov Nikolov]

On Fri, Mar 25, 2011 at 12:11 PM, Baptiste Daroussin b...@freebsd.org wrote:
 Hi all,


Hi,

Great news!

I'll try to contribute with what I can for this project!

I've mirrored the pkgng Git repo here as well:

 - http://git.unix-heaven.org/cgit.cgi/pkgng/

And attached is my first patch :)

Could you please review and if possible, apply it?

Regards,
Marin

-- 
Marin Atanasov Nikolov

dnaeon AT gmail DOT com
daemon AT unix-heaven DOT org
http://www.unix-heaven.org/
From bd17ad1dfa336717fcc924217552dadd5236ed5f Mon Sep 17 00:00:00 2001
From: Marin Atanasov Nikolov dae...@unix-heaven.org
Date: Sat, 26 Mar 2011 14:19:04 +0200
Subject: [PATCH] Clarify a bit more the GOALS and some other minor changes

	- Clarify GOALS a bit more
	- Added main.h which contains CMD_MAX_LEN
	- Use strnlen() instead of strlen() in main.c
	- Use float numbers in fetch_status()
---
 docs/GOALS |  178 ++--
 pkg/add.c  |2 +-
 pkg/main.c |9 ++--
 pkg/main.h |7 +++
 4 files changed, 126 insertions(+), 70 deletions(-)
 create mode 100644 pkg/main.h

diff --git a/docs/GOALS b/docs/GOALS
index 1c381d2..ccf3414 100644
--- a/docs/GOALS
+++ b/docs/GOALS
@@ -1,104 +1,152 @@
-comamnds:
+pkgng - New version of FreeBSD's pkg_install
+
+Table of Contents:
+
+1. Available commands of pkgng
+2. Available subcommands of pkgng
+3. General behaviour
+4. New manifest format
+	4.1. During new install
+	4.2. During package deletion
+	4.3. During upgrade of a package
+5. Additional resources
+
+
+1. Available commands of pkgng:
+---
+
 - pkg:
-	- one command to rule them all
-	- each time pkg has to query the installed database it will try to first
-	  read the cache is the cache is too old (check but last mtime of the
-	  directory) it will regenerate the cache from installed package
+
+	- One command to rule them all
+	- Each time `pkg' has to query the package database it will try to first
+	  read the cache and if the cache is too old (check by last mtime of the
+	  directory) it will regenerate the cache for the installed packages.
 
 - pkg_*:
-	this will be shell script their goal is to be 100% compatible with
-	pkg_install they will call pkg which the good options (not sure it will
-	be really done in the end)
 
-subcommands:
+	- These will be shell scripts, which goal is to be 100% compatible with
+	  pkg_install. They will call `pkg' with the appropriate options (not sure
+	  it will be really done in the end).
+
+2. Available subcommands of pkgng:
+--
+
 - add:
-	- installs a package if local package is given
-	- manage http/ftp url to download throught libfetch a package and try to
-	  install it.
-	- in the two previous case, if the package as not installed dependencies
-	  it will try to fetch/find them in the same place the original package
-	  was.
-	- it will manage a completely new repository format pkg add bla will
-	  check in the database of the repository if the is a bla package and
-	  install it as well as it's dependencies but the dependencies will be
-	  compute before begining the fetch install process, to be sure
-	  everything is ok.
-	- should be able to only take a plist or a manifest in argument
+
+	- Installs a package if a local package is given
+	- Manages ftp/http downloads of packages through libfetch and
+	  tries to install them.
+	- In the two previous cases, if a package does not have it's 
+	  dependencies installed, `pkgng' will try to fetch/find them in the same
+	  place as the original package was.
+	- It will manage a completely new repository format. 
+	  `pkg add foo' will check in the database of the repository for 
+	  package foo and try to install it as well as it's dependencies.
+	  Dependencies will be computed before beginning the fetch/install
+	  process to ensure that everything will be OK.
+	- Should be able to only take a plist or a manifest in argument
 	  considering the files are already installed
 
 - repo:
-	- takes only one argument and generate two databases files (maybe more for
-	  the time we will be willing to work with UPDATING and MOVED) the first
-	  repo contain every informations concerning a package except files
-	  (which will be in the second one) + it will compute and add a sha256
-	  sum for each package.
-	  the database will be a sqlite file compressed with the xz format.
-	  the database will be signed so we can trust the sha256 of the
-	  packages, so if a package has the expected hash, it is considered
+
+	- Takes only one argument and generates two database files (maybe more for
+	  the time we will be willing to work with UPDATING and MOVED).
+	  The first repo contains all information about a package, except for
+	  the files (which will be in the second one) + it will compute and 
+	  add a sha256 sum for each package.
+	- The database will be a sqlite file compressed with the xz format.
+	- The database will be signed 

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Marcin Wisnicki]

On Sat, 26 Mar 2011 10:22:50 +, Baptiste Daroussin wrote:

 This can still be discussed but I don't really like the idea that users
 may alter packages an installation/extraction time, that would lead to
 lots of potential buggy installation and report.

That's why list of alterations have to be recorded and relevant tools 
would mark such packages appropriately. Using filters would be considered 
unsupported (unless maintainer of particular port/package says otherwise).
It's no different than doing unsupported things right now, just giving 
user more tools.
Besides, examples mentioned by me like removing docs and unused 
translations are rather safe and save significant amount of disk space.

 If user aren't happy
 with the packaging, they can poke the maintainer, send PR, patch etc.

That's not possible unless user is willing to build packages himself.
Actually I would rather not have build time options for things that can 
just as well be performed during installation.
Hacking makefiles to implement NOPORTDOCS is quite more complicated than 
setting a glob pattern.

There is also one very important use case I didn't mention before.
Such filters could serve as a hooking point for configuration management 
system. It makes it possible to capture config files as they are installed 
and perform backups, merges, check-in to vcs etc.

 regards,
 Bapt
 ___
 freebsd-ports@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe,
 send any mail to freebsd-ports-unsubscr...@freebsd.org


___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Michel Talon]

 This can still be discussed but I don't really like the idea that
 users may alter packages an installation/extraction time, that would
 lead to lots of potential buggy installation and report.
 If user aren't happy with the packaging, they can poke the maintainer,
 send PR, patch etc.

The whole point of *binary* packages is that people don't mess 
changing options, changing what is installed, etc. so that port
maintainers know exactly what people have on their machine, and be
able to test it. This is a prerequisite for a system which is not a
complete mess. Those you like to change things are on their own.


-- 

Michel TALON

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Julien Laffaye]

On Fri, Mar 25, 2011 at 10:11 AM, Baptiste Daroussin b...@freebsd.org wrote:
  Developpement site: http://git.etoilebsd.net/pkgng/

FYI, we moved to github[1] in order to have a bug tracker, pull
request and code review.
Also, I recommend to build from the HEAD of the git repository, to not
report fixed compilation warnings/bugs.

[1]: https://github.com/pkgng/pkgng
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Baptiste Daroussin]

2011/3/25 Alexander Leidinger alexan...@leidinger.net:
 Quoting Baptiste Daroussin b...@freebsd.org (from Fri, 25 Mar 2011
 11:11:11 +0100):

 pkgng is a binary package manager written from scratch for FreeBSD.

 I didn't had a look at it, just some comments about some parts you
 explained.

 features supported are or will be :

 - the register command can analyse elf files when registering a new port
 to
 discover forgotten dependencies if necessary. (done in alpha using libelf)

 This will probably fail if LD_LIBRARY_PATH is used, or if we are installing
 linuxulator ports.


this isn't activated by default, and if activated is only intended to
work on freebsd elf files.
This is done to workaround some bugguy ports not to be used in
production, pkg register shows in warning in that case so that
user/maintainers are warned they have something to fix.


 - a special architecture all allows to specify when a package can be
 used
 on every architecture. (not done yet)

 What if a package is able to install on a subset, e.g. the linuxulator ports
 are for amd64 and i386?


No clue for that at the moment but we are open to suggestions.

 What about DB corruption/loss? Do you keep the /var/db/pkg/package/xxx
 files even with pkgng and only use the DB as a way to speed up some work (so
 the DB corruption just requires to run pkg2ng), or are you lost of the DB is
 lost?


Nothing is done about DB corruption/loss, I am not sure we need to do something.
Maybe.
Currently a filesystem corruption/loss on /var/db/pkg would do the same.

but it is sqlite so we can perhaps provide a way to get compressed
dump so user can periodically backup their database.

regards,
Bapt
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Alexander Leidinger]

Quoting Baptiste Daroussin b...@freebsd.org (from Fri, 25 Mar 2011  
11:11:11 +0100):



pkgng is a binary package manager written from scratch for FreeBSD.


I didn't had a look at it, just some comments about some parts you explained.


features supported are or will be :



- the register command can analyse elf files when registering a new port to
discover forgotten dependencies if necessary. (done in alpha using libelf)


This will probably fail if LD_LIBRARY_PATH is used, or if we are  
installing linuxulator ports.



- new +MANIFEST (plist-like format) with new metadatas : options, arch, os
version, etc. (done in the alpha)

- pkgng supports checking arch of the package which means that users
won't be able to install sparc64 binary package into amd64 machines.
(not done yet)

- a special architecture all allows to specify when a package can be used
on every architecture. (not done yet)


What if a package is able to install on a subset, e.g. the linuxulator  
ports are for amd64 and i386?



 In term of technology we decided to use a sqlite3 database, and to
 prevent potential trolling, sqlite3 is used in it's amalgamation form
 which means it is incorporated in the code sources (as recommanded by
 sqlite developpers like a statically linked library) on build we only
 activate the features we need in sqlite.

 The alpha release come with an experimental tool pkg2ng to convert
 an existing package database to the new pkgng database format. So one
 can test pkgng without rebuild all its packages.


What about DB corruption/loss? Do you keep the  
/var/db/pkg/package/xxx files even with pkgng and only use the DB as  
a way to speed up some work (so the DB corruption just requires to run  
pkg2ng), or are you lost of the DB is lost?


Bye,
Alexander.

--
Real computer scientists don't comment their code.  The identifiers are
so long they can't afford the disk space.

http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org   netchild @ FreeBSD.org  : PGP ID = 72077137
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Alexander Leidinger]

Quoting Baptiste Daroussin b...@freebsd.org (from Fri, 25 Mar 2011  
15:14:52 +0100):



2011/3/25 Alexander Leidinger alexan...@leidinger.net:

Quoting Baptiste Daroussin b...@freebsd.org (from Fri, 25 Mar 2011
11:11:11 +0100):


pkgng is a binary package manager written from scratch for FreeBSD.


I didn't had a look at it, just some comments about some parts you
explained.


features supported are or will be :



- a special architecture all allows to specify when a package can be
used
on every architecture. (not done yet)


What if a package is able to install on a subset, e.g. the linuxulator ports
are for amd64 and i386?



No clue for that at the moment but we are open to suggestions.


The suggestion is easy, allow a way to specify a set of valid architectures.


What about DB corruption/loss? Do you keep the /var/db/pkg/package/xxx
files even with pkgng and only use the DB as a way to speed up some work (so
the DB corruption just requires to run pkg2ng), or are you lost of the DB is
lost?



Nothing is done about DB corruption/loss, I am not sure we need to  
do something.

Maybe.


I would say for sure. Info: In Solaris 10 sqlite is used for the  
service managenemt framework (SMF). It is possible that the DB is  
corrupt in some bad situations. In this case you have to rebuild the  
DB (script provided, been there, had to use it).



Currently a filesystem corruption/loss on /var/db/pkg would do the same.


Put a corruption of /var/db/pkg/xyz-1/+REQUIRED_BY would only affect a  
small part, and this part could be even recovered from (pkgdb from  
portupgrade is able to do it).



but it is sqlite so we can perhaps provide a way to get compressed
dump so user can periodically backup their database.


It needs to be automated. Maybe periodic daily... but maybe this is  
not often enough after a day of a lot of changes (think about it this  
way: do you want to lose a day of changes?). The current FS based DB  
is very robust, partly because there is redundant data, pertly because  
losing a file just means that the very limited subset of information  
is lost (and a reinstall of one port will fix it).


Bye,
Alexander.

--
Programming is an unnatural act.

http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org   netchild @ FreeBSD.org  : PGP ID = 72077137
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Julien Laffaye]

On Fri, Mar 25, 2011 at 2:38 PM, Alexander Leidinger
alexan...@leidinger.net wrote:
 Quoting Baptiste Daroussin b...@freebsd.org (from Fri, 25 Mar 2011
 15:14:52 +0100):

 2011/3/25 Alexander Leidinger alexan...@leidinger.net:

 Quoting Baptiste Daroussin b...@freebsd.org (from Fri, 25 Mar 2011
 11:11:11 +0100):

 pkgng is a binary package manager written from scratch for FreeBSD.

 I didn't had a look at it, just some comments about some parts you
 explained.

 features supported are or will be :

 - a special architecture all allows to specify when a package can be
 used
 on every architecture. (not done yet)

 What if a package is able to install on a subset, e.g. the linuxulator
 ports
 are for amd64 and i386?


 No clue for that at the moment but we are open to suggestions.

 The suggestion is easy, allow a way to specify a set of valid architectures.

That looks reasonable and easy to implement.


 What about DB corruption/loss? Do you keep the /var/db/pkg/package/xxx
 files even with pkgng and only use the DB as a way to speed up some work
 (so
 the DB corruption just requires to run pkg2ng), or are you lost of the DB
 is
 lost?


 Nothing is done about DB corruption/loss, I am not sure we need to do
 something.
 Maybe.

 I would say for sure. Info: In Solaris 10 sqlite is used for the service
 managenemt framework (SMF). It is possible that the DB is corrupt in some
 bad situations. In this case you have to rebuild the DB (script provided,
 been there, had to use it).

If sqlite is properly used with transactions, it is very hard to
corrupt the database. But if hardware lies to us and say that the data
is on disk whereas it isnt... what can we do?
Another potential problem is fsync(), but if it is broken on FreeBSD
we want to fix it!

BTW, the goal is to only have the database and not the flat files.
If you are paranoid about power outage, use something like zfs snapshots...


 Currently a filesystem corruption/loss on /var/db/pkg would do the same.

 Put a corruption of /var/db/pkg/xyz-1/+REQUIRED_BY would only affect a small
 part, and this part could be even recovered from (pkgdb from portupgrade is
 able to do it).

With sqlite we have atomicity! And locks!


 but it is sqlite so we can perhaps provide a way to get compressed
 dump so user can periodically backup their database.

 It needs to be automated. Maybe periodic daily... but maybe this is not
 often enough after a day of a lot of changes (think about it this way: do
 you want to lose a day of changes?). The current FS based DB is very robust,
 partly because there is redundant data, pertly because losing a file just
 means that the very limited subset of information is lost (and a reinstall
 of one port will fix it).

 Bye,
 Alexander.

Regards,
Julien
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Pietro Cerutti]

On 2011-Mar-25, 15:03, Julien Laffaye wrote:
  What about DB corruption/loss? Do you keep the /var/db/pkg/package/xxx
  files even with pkgng and only use the DB as a way to speed up some work
  (so
  the DB corruption just requires to run pkg2ng), or are you lost of the DB
  is
  lost?
 
 
  Nothing is done about DB corruption/loss, I am not sure we need to do
  something.
  Maybe.
 
  I would say for sure. Info: In Solaris 10 sqlite is used for the service
  managenemt framework (SMF). It is possible that the DB is corrupt in some
  bad situations. In this case you have to rebuild the DB (script provided,
  been there, had to use it).
 
 If sqlite is properly used with transactions, it is very hard to
 corrupt the database. But if hardware lies to us and say that the data
 is on disk whereas it isnt... what can we do?
 Another potential problem is fsync(), but if it is broken on FreeBSD
 we want to fix it!
 
 BTW, the goal is to only have the database and not the flat files.
 If you are paranoid about power outage, use something like zfs snapshots...

No need to look for strange scenarios, I'm surely going to sudo rm -f the file
more sooner than later, so... maybe just save a copy?

-- 
Pietro Cerutti
The FreeBSD Project
g...@freebsd.org

PGP Public Key:
http://gahr.ch/pgp


pgp4AYvylMp4W.pgp
Description: PGP signature

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Baptiste Daroussin]

2011/3/25 Pietro Cerutti g...@freebsd.org:
 On 2011-Mar-25, 15:03, Julien Laffaye wrote:
  What about DB corruption/loss? Do you keep the /var/db/pkg/package/xxx
  files even with pkgng and only use the DB as a way to speed up some work
  (so
  the DB corruption just requires to run pkg2ng), or are you lost of the DB
  is
  lost?
 
 
  Nothing is done about DB corruption/loss, I am not sure we need to do
  something.
  Maybe.
 
  I would say for sure. Info: In Solaris 10 sqlite is used for the service
  managenemt framework (SMF). It is possible that the DB is corrupt in some
  bad situations. In this case you have to rebuild the DB (script provided,
  been there, had to use it).

 If sqlite is properly used with transactions, it is very hard to
 corrupt the database. But if hardware lies to us and say that the data
 is on disk whereas it isnt... what can we do?
 Another potential problem is fsync(), but if it is broken on FreeBSD
 we want to fix it!

 BTW, the goal is to only have the database and not the flat files.
 If you are paranoid about power outage, use something like zfs snapshots...

 No need to look for strange scenarios, I'm surely going to sudo rm -f the file
 more sooner than later, so... maybe just save a copy?

 --
 Pietro Cerutti
 The FreeBSD Project
 g...@freebsd.org

 PGP Public Key:
 http://gahr.ch/pgp


I think we can provide a periodic script activable by users (I let
other decide if it has to be activated by default or not) that does a
pkg backup /path/to/file/backup
and xz it.

because copying can be a huge.

40Mo for the database here, corresponding to 70Mo in the old format
and to 600 packages. the dump xzed is only 3Mo

regards,
Bapt
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Andriy Gapon]

on 25/03/2011 17:35 Pietro Cerutti said the following:
 No need to look for strange scenarios, I'm surely going to sudo rm -f the file
 more sooner than later, so... maybe just save a copy?

I even can rm -rf / by accident.
What's your solution to this? :)

P.S. one solution would be a subcase of the other
-- 
Andriy Gapon
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Michel Talon]

Sorry i have just seen a table for dependencies in pkg_repo.c

CREATE TABLE deps (
origin TEXT,
name TEXT,
version TEXT,
package_id INTEGER REFERENCES packages(id),
PRIMARY KEY (package_id, origin)

So this seems fine.

-- 

Michel TALON

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Eitan Adler]

On Fri, Mar 25, 2011 at 11:35 AM, Andriy Gapon a...@freebsd.org wrote:
 on 25/03/2011 17:35 Pietro Cerutti said the following:
 No need to look for strange scenarios, I'm surely going to sudo rm -f the 
 file
 more sooner than later, so... maybe just save a copy?

 I even can rm -rf / by accident.
 What's your solution to this? :)

rm -rf /
rm: / may not be removed


-- 
Eitan Adler
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Michel Talon]

 Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge
 contributor) have been working since the end of the last GSoC on a
 rewrite of pkg_install.

Fantastic! it has been necessary since a looong time.

 - the register command can analyse elf files when registering a new port
   to discover forgotten dependencies if necessary. (done in alpha using
 libelf)

Fantastic! the dependencies as mentioned directly in Makefiles by
ports maintainers were not always perfect.

 In term of technology we decided to use a sqlite3 database, and to
  prevent potential trolling, sqlite3 is used in it's amalgamation form
  which means it is incorporated in the code sources (as recommanded by
  sqlite developpers like a statically linked library) on build we only
  activate the features we need in sqlite.

Fantastic! at least using something fast and tested instead of some
half-brewed solutions. I have just taken a look at the table packages,
it seems that it does not contain dependency information, but you can
discover it through analyze_elf, where do you store it?

This project is the thing i had dreamed about.


-- 

Michel TALON

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Thomas Dickey]

On Fri, Mar 25, 2011 at 11:54:35AM -0500, Eitan Adler wrote:
 On Fri, Mar 25, 2011 at 11:35 AM, Andriy Gapon a...@freebsd.org wrote:
  on 25/03/2011 17:35 Pietro Cerutti said the following:
  No need to look for strange scenarios, I'm surely going to sudo rm -f the 
  file
  more sooner than later, so... maybe just save a copy?
 
  I even can rm -rf / by accident.
  What's your solution to this? :)
 
 rm -rf /
 rm: / may not be removed

referring to the CVS, this should improve the approach.

cd /tmp
rm -rf ../*

ymmv

-- 
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net


pgp5D40nqAsVs.pgp
Description: PGP signature

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Alexander Leidinger]

On Fri, 25 Mar 2011 16:35:21 +0100 Pietro Cerutti g...@freebsd.org
wrote:


 On 2011-Mar-25, 15:03, Julien Laffaye wrote:
   What about DB corruption/loss? Do you keep
   the /var/db/pkg/package/xxx files even with pkgng and only
   use the DB as a way to speed up some work (so
   the DB corruption just requires to run pkg2ng), or are you lost
   of the DB is
   lost?
  
  
   Nothing is done about DB corruption/loss, I am not sure we need
   to do something.
   Maybe.
  
   I would say for sure. Info: In Solaris 10 sqlite is used for
   the service managenemt framework (SMF). It is possible that the
   DB is corrupt in some bad situations. In this case you have to
   rebuild the DB (script provided, been there, had to use it).
  
  If sqlite is properly used with transactions, it is very hard to
  corrupt the database. But if hardware lies to us and say that the

And as I told above, I even had such a case (more than once), and the
hardware was not buggy. What do you want to tell in this case, life
sucks, reinstall everything?

  data is on disk whereas it isnt... what can we do?

Sometimes you have to stay with broken hardware.

  Another potential problem is fsync(), but if it is broken on FreeBSD
  we want to fix it!
  
  BTW, the goal is to only have the database and not the flat files.
  If you are paranoid about power outage, use something like zfs
  snapshots...

There are more FS than only ZFS (personally I use ZFS, and I have
snapshots, but this is not a good solution for this problem).

As I told already, if it isn't automatic, nearly nobody will use it.
And the package management stuff has to be automatic, no freshman will
think about setting up a snapshot script when he starts to use
packages/ports.

 No need to look for strange scenarios, I'm surely going to sudo rm -f
 the file more sooner than later, so... maybe just save a copy?

A copy or two would be enough, but it has to be done automatically, and
once a day is not enough. A copy after each X modifications maybe
(for suitable definitions of X and 'modifications').

Bye,
Alexander.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Marcin Wisnicki]

On Fri, 25 Mar 2011 11:11:11 +0100, Baptiste Daroussin wrote:

 pkgng is a binary package manager written from scratch for FreeBSD.
 

Fantastic!

I know it is quite too early but I already have one feature request ;)
Perhaps it could be added to the TODO as a post-1.0 goal.

= Generic extraction filters =

Allow registration of custom filters that can alter/exclude/add? files 
during package extraction (installation).

Examples of possible filters:
 - strip debug info
 - exclude development files (headers, static libs)
 - exclude unused translations
 - exclude documentation (all or just unknown languages)
 - generic glob/regex path filters
 - optional file groups defined in package (install time OPTIONS) ?

Some sort of configuration mechanism with list of enabled filters and 
their options (like a list of languages to keep).

Most of this can be implemented as a simple glob/regex matching but there 
are edge cases where packages have some non-standard layout or have to 
keep certain file in which case a package metadata should contain a list 
of exclusions/additions from/to above categories.

Package manager should register only actually installed files but list of 
alterations should be also kept somewhere in database.


What do you think ?

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org