Re: [gentoo-dev] Moving our/portage stuff to var
On Wed, 19 Dec 2012, Brian Dolbec wrote: just to clarify, I'm voting for... /var/cache/distfiles /var/cache/packages Fine. /var/cache/repositories/ /var/cache/repositories/gentoo == the main portage tree /var/cache/repositories/local== the new location for a local overlay /var/cache/repositories/some-overlay == layman installed overlay So the Portage tree would move from the second level (/usr/portage) to the fourth level? IMHO this is unacceptable. Make it /var/cache/gentoo or /var/cache/portage; the /var/cache directory really isn't so overpopulated that there's a need for hiding things in subdirs. Also I wonder if local overlays should be in /var/cache? It might not always be possible to restore them. Ulrich
Re: [gentoo-dev] Moving our/portage stuff to var
On Thursday 20 December 2012 09:11:39 Ulrich Mueller wrote: /var/cache/repositories/local== the new location for a local overlay Also I wonder if local overlays should be in /var/cache? It might not always be possible to restore them. So, this is the present /usr/local/portage? What's wrong with it? According to FHS, /usr/local/ is reserved for local admin meddling, which fits the bill rather nicely fo local overlay. This is the one location that was actually making sense from the beginning. Now, trying to recall its history, I seem to remember that it was not there from the onset, but rather added when we first had this idea of overlays, and, faintly (remember), even having some discussion with some technical arguments. I guess this is how it ended up in the place that makes sense :). George
Re: [gentoo-dev] Moving our/portage stuff to var
On Wed, 19 Dec 2012 14:19:52 -0800
Zac Medico zmed...@gentoo.org wrote:
On 12/19/2012 02:01 PM, Alan McKinnon wrote:
On Wed, 19 Dec 2012 14:56:44 +0100
Diego Elio Pettenò flamee...@flameeyes.eu wrote:
Just mv /usr/portage /var/portage ? FFS no. Among other things, as
many said before, we should really take distfiles out of the tree
itself, and packages the same. And I don't want /var/packages
or /var/distfiles at all.
If we are going to move distfiles out of the tree into, what are the
odds of getting /some/path/portage/local to move somewhere else too?
What program uses this local directory? It's not used directly by
portage itself, though portage has an exclude for it in the default
PORTAGE_RSYNC_OPTS setting
(in /usr/share/portage/config/make.globals).
It goes back a long time, and is basically a poor man's local overlay
without having to use layman. As I understand it, portage will treat
the directory like any other when looking for ebuilds and resolving
deps, but exclude it from a sync.
That one has irked me for ages, its the one thing left on my systems
that stops the local tree dir being an exact replica of the upstream
master.
For portage's defaults, I won't settle for anything less than having
them all refer to separate directories which are *not* nested within
one other. These are the current default settings which violate my
requirements:
PORTDIR=/usr/portage
DISTDIR=${PORTDIR}/distfiles
PKGDIR=${PORTDIR}/packages
RPMDIR=${PORTDIR}/rpm
/usr/portage/local has the taste feel and smell of a hacky workaround:
shove a directory in the tree and exclude it from sync.
I suspect the best solution all round is to move all support for local
overlays into layman. I'd be happy with that. Probably make the portage
code cleaner too.
--
Alan McKinnon
alan.mckin...@gmail.com
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, Dec 20, 2012 at 4:25 AM, George Shapovalov geo...@gentoo.org wrote: On Thursday 20 December 2012 09:11:39 Ulrich Mueller wrote: /var/cache/repositories/local== the new location for a local overlay Also I wonder if local overlays should be in /var/cache? It might not always be possible to restore them. So, this is the present /usr/local/portage? What's wrong with it? According to FHS, /usr/local/ is reserved for local admin meddling, which fits the bill rather nicely fo local overlay. This is the one location that was actually making sense from the beginning. I actually like the /var/cache/repositories approach. You can always add a symlink to it if you want to for convenience (as I already do for /var/lib/portage/world). There is really nothing special about /usr/portage, other than it being the lowest-priority overlay. As far as the local one goes - again this is configurable in /etc/make.conf, and we don't really need to pre-create that directory either. Rich
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, 20 Dec 2012, Rich Freeman wrote: I actually like the /var/cache/repositories approach. You can always add a symlink to it if you want to for convenience (as I already do for /var/lib/portage/world). There is really nothing special about /usr/portage, other than it being the lowest-priority overlay. There's also nothing special about Earth amongst other planets, so please add Earth / Solar System when addressing your letters. SCNR, Ulrich
Re: [gentoo-dev] Moving our/portage stuff to var
Zac Medico zmed...@gentoo.org writes: On 12/19/2012 02:01 PM, Alan McKinnon wrote: If we are going to move distfiles out of the tree into, what are the odds of getting /some/path/portage/local to move somewhere else too? What program uses this local directory? It's not used directly by portage itself, though portage has an exclude for it in the default PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals). It is useful for 'site' local ebuilds. It allows a 'master' repository to sync with the main Gentoo one without disturbing local changes but allow other systems on-site to fetch the modified tree with a simple 'emerge --sync'.
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, 2012-12-20 at 09:11 +0100, Ulrich Mueller wrote:
On Wed, 19 Dec 2012, Brian Dolbec wrote:
just to clarify, I'm voting for...
/var/cache/distfiles
/var/cache/packages
Fine.
/var/cache/repositories/
/var/cache/repositories/gentoo == the main portage tree
/var/cache/repositories/local== the new location for a local overlay
/var/cache/repositories/some-overlay == layman installed overlay
So the Portage tree would move from the second level (/usr/portage) to
the fourth level? IMHO this is unacceptable. Make it /var/cache/gentoo
or /var/cache/portage; the /var/cache directory really isn't so
overpopulated that there's a need for hiding things in subdirs.
Also I wonder if local overlays should be in /var/cache? It might not
always be possible to restore them.
Ulrich
My idea for having all repos under one directory is to make it easier
for a pkg manager to simply scan the directory to know all installed
overlays. Currently each one has to be listed in a configured variable
in make.conf. So if you wanted your local overlay somewhere else, then
a symlink would work (provided the PM can/will autoscan repos), or add
it to the PORTDIR_OVERLAY variable (current behavior). I don't
otherwise have a strong desire for it to be there.
If and only if the tree and all overlays (not other directories) are not
under one directory, then an autoscan cannot easily happen.
For an example of what I am referring to. In layman-2.0.0 I have added
a /etc/layman/overlays directory. On start it will scan it for any
*.xml files and add those specifications to the overlays variable.
Layman-1.4 and previous required you add each entry to that overlays
variable in layman.cfg. Essentially this makes the overlays directory a
plug-in directory, so if you want to install an overlay not listed in
the main repositories list, just download the xml spec to
/etc/layman/overlays/${meaningful-name}.xml
then layman -f (adds it to it's cache)
no editing required :)
If repositories is too long a name, repos is shorter and still
meaningful, although still puts them at a 4th level.
--
Brian Dolbec dol...@gentoo.org
signature.asc
Description: This is a digitally signed message part
Re: [gentoo-dev] Moving our/portage stuff to var
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 20/12/12 10:37 AM, Brian Dolbec wrote: /var/cache/repositories/ /var/cache/repositories/gentoo == the main portage tree /var/cache/repositories/local== the new location for a local overlay /var/cache/repositories/some-overlay == layman installed overlay My idea for having all repos under one directory is to make it easier for a pkg manager to simply scan the directory to know all installed overlays. Currently each one has to be listed in a configured variable in make.conf. So if you wanted your local overlay somewhere else, then a symlink would work (provided the PM can/will autoscan repos), or add it to the PORTDIR_OVERLAY variable (current behavior). I don't otherwise have a strong desire for it to be there. If and only if the tree and all overlays (not other directories) are not under one directory, then an autoscan cannot easily happen. You could do this while not having the portage tree be in that directory. IE, portage goes in /var/cache/portage , and all the overlays go into /var/cache/repositories. The tree is separate enough IMO that autoscan can still happen easily, and also I believe that it can be assumed that the tree is in place. For instance, if the tree's location is defined to be elsewhere, it isn't done so via PORTDIR_OVERLAYS but rather PORTDIR. On an unrelated note, I would never treat my local overlays as cache. Ebuilds that (as a user) I wrote and installed by hand are not likely to be kept in a repository someplace, but rather the overlay dir would most likely be it's only location. IIRC the reason for /usr/portage/local/ was to have a path within the portage tree that rsync wouldn't kill; given that what you're suggesting is already not under the proposed portage tree location, emerge --sync couldn't touch it, and so I don't see a need at all to provide a 'local' repository destination by default. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) iF4EAREIAAYFAlDTNlQACgkQ2ugaI38ACPCxQAEApT/7CaIbuVTwnDQk93hhDjGu mXKPdCJg4h1iMECtdoABAJj2601LuRPUKFJ+BJa/FqrdRTsjSpBRiEd8pvO2042P =W3T9 -END PGP SIGNATURE-
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, Dec 20, 2012 at 11:01 AM, Ian Stakenvicius a...@gentoo.org wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On 20/12/12 10:37 AM, Brian Dolbec wrote:
/var/cache/repositories/ /var/cache/repositories/gentoo ==
the main portage tree /var/cache/repositories/local== the
new location for a local overlay
/var/cache/repositories/some-overlay == layman installed
overlay
My idea for having all repos under one directory is to make it
easier for a pkg manager to simply scan the directory to know all
installed overlays. Currently each one has to be listed in a
configured variable in make.conf. So if you wanted your local
overlay somewhere else, then a symlink would work (provided the PM
can/will autoscan repos), or add it to the PORTDIR_OVERLAY variable
(current behavior). I don't otherwise have a strong desire for it
to be there.
If and only if the tree and all overlays (not other directories)
are not under one directory, then an autoscan cannot easily
happen.
You could do this while not having the portage tree be in that
directory. IE, portage goes in /var/cache/portage , and all the
overlays go into /var/cache/repositories.
The tree is separate enough IMO that autoscan can still happen easily,
and also I believe that it can be assumed that the tree is in place.
For instance, if the tree's location is defined to be elsewhere, it
isn't done so via PORTDIR_OVERLAYS but rather PORTDIR.
On an unrelated note, I would never treat my local overlays as
cache. Ebuilds that (as a user) I wrote and installed by hand are not
likely to be kept in a repository someplace, but rather the overlay
dir would most likely be it's only location. IIRC the reason for
/usr/portage/local/ was to have a path within the portage tree that
rsync wouldn't kill; given that what you're suggesting is already not
under the proposed portage tree location, emerge --sync couldn't touch
it, and so I don't see a need at all to provide a 'local' repository
destination by default.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)
iF4EAREIAAYFAlDTNlQACgkQ2ugaI38ACPCxQAEApT/7CaIbuVTwnDQk93hhDjGu
mXKPdCJg4h1iMECtdoABAJj2601LuRPUKFJ+BJa/FqrdRTsjSpBRiEd8pvO2042P
=W3T9
-END PGP SIGNATURE-
It's sounding like the nearly the optimal solution would be:
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
Clearly, some data in question needs to be treated as persistent, and
others can be treated as cache. So it should probably be divided up
that way. The placement of tree and overlays as subfolders of the same
folder strikes me as appropriate, too.
The only thing I can't see an elegant workaround for are how to avoid
or handle repo name collisions between
/var/cache/portage/repositories/* and /var/db/portage/repositories/*
--
:wq
Re: [gentoo-dev] Moving our/portage stuff to var
On 20/12/2012 17:16, Michael Mol wrote:
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
+1
--
Diego Elio Pettenò — Flameeyes
flamee...@flameeyes.eu — http://blog.flameeyes.eu/
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, Dec 20, 2012 at 11:19 AM, Diego Elio Pettenò
flamee...@flameeyes.eu wrote:
On 20/12/2012 17:16, Michael Mol wrote:
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
+1
Agreed.
Look, we're not going to find any place that makes everybody happy.
This one seems to be logical from a design standpoint. As long as we
make sure that everything is set in configuration then individuals can
move it wherever they want to. Symlinks also work.
Rich
Re: [gentoo-dev] Moving our/portage stuff to var
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 12/20/2012 11:16 AM, Michael Mol wrote:
On Thu, Dec 20, 2012 at 11:01 AM, Ian Stakenvicius a...@gentoo.org wrote:
On 20/12/12 10:37 AM, Brian Dolbec wrote:
/var/cache/repositories/ /var/cache/repositories/gentoo ==
the main portage tree /var/cache/repositories/local== the
new location for a local overlay
/var/cache/repositories/some-overlay == layman installed
overlay
My idea for having all repos under one directory is to make it
easier for a pkg manager to simply scan the directory to know all
installed overlays. Currently each one has to be listed in a
configured variable in make.conf. So if you wanted your local
overlay somewhere else, then a symlink would work (provided the PM
can/will autoscan repos), or add it to the PORTDIR_OVERLAY variable
(current behavior). I don't otherwise have a strong desire for it
to be there.
If and only if the tree and all overlays (not other directories)
are not under one directory, then an autoscan cannot easily
happen.
You could do this while not having the portage tree be in that
directory. IE, portage goes in /var/cache/portage , and all the
overlays go into /var/cache/repositories.
The tree is separate enough IMO that autoscan can still happen easily,
and also I believe that it can be assumed that the tree is in place.
For instance, if the tree's location is defined to be elsewhere, it
isn't done so via PORTDIR_OVERLAYS but rather PORTDIR.
On an unrelated note, I would never treat my local overlays as
cache. Ebuilds that (as a user) I wrote and installed by hand are not
likely to be kept in a repository someplace, but rather the overlay
dir would most likely be it's only location. IIRC the reason for
/usr/portage/local/ was to have a path within the portage tree that
rsync wouldn't kill; given that what you're suggesting is already not
under the proposed portage tree location, emerge --sync couldn't touch
it, and so I don't see a need at all to provide a 'local' repository
destination by default.
It's sounding like the nearly the optimal solution would be:
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
Not to oversimplify but why exactly can't we leave /usr/local/portage
where it is? I'm not going to want to cd
/var/db/portage/repositories/local every time I want to edit a local
ebuild...
- -ZC
Clearly, some data in question needs to be treated as persistent, and
others can be treated as cache. So it should probably be divided up
that way. The placement of tree and overlays as subfolders of the same
folder strikes me as appropriate, too.
The only thing I can't see an elegant workaround for are how to avoid
or handle repo name collisions between
/var/cache/portage/repositories/* and /var/db/portage/repositories/*
--
:wq
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/
iQIcBAEBAgAGBQJQ0zvlAAoJEKXdFCfdEflKfiEP/0uLujqLlv1DVydqj3xXZUVq
t/c5mDsg3iJgt5T7Hm+ER949r2GUqju4veed4JQWFlVSaOoLEViL1Me/jPco5fC8
v064ktt2hOLPb+tR2IWaK3tR8i+LhcFEcIyANhl62ENPWgvOAR6V0KNFuudQLicS
QUFaJYKZkkYuPSTTqPld3QXzFwH1X6RCQaOtjCOqZKAZr9iW8HRNTTLpoa4bSMgr
VBswHyH+q0C9TzIVv5u8G8s8cYNdqHf1wrSTeMjq961tVzF3Tno5s1zk1MOyQ7cQ
MkQCxiMAum0d9PX87UkPuvHKgLdZ7e+tW26B9bS3M9yGu66lsHB7+sOTxFAJ9kqp
YKLuO2XPmpIMyDNc/5rQtTl5ygA9CmqSpUZEjMgwvCmemOHO3CsPXXQxfq6Ze7kK
/aNfCHJVEP/x8bY7PdWoexaScW/Qnqrqm6R+GCd6B3LGmTinGaDWYzJj+pAkpGUn
OAHcxATC9gX3AZr9atTFHRaPkD3L3FdYothVDZq5DDkW2qAmuhbqaEhzytDI3GLl
R+MEWYcMqvNLV5eYlNPe4OOaYfFTr/1mP0k/3ixjxJFwMDXxmIaJGTKoMOyoPu3O
diZlo0m2EPCH7Ggl9Fh0xf4P/wDSQB0AkfyldQhnVNbR5DRcTrkU6IfJBVLdcJ40
XljVqWuG27XmMXwjLGgV
=NLJg
-END PGP SIGNATURE-
Re: [gentoo-dev] Moving our/portage stuff to var
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On 20/12/12 11:25 AM, Rick Zero_Chaos Farina wrote:
On 12/20/2012 11:16 AM, Michael Mol wrote:
On Thu, Dec 20, 2012 at 11:01 AM, Ian Stakenvicius
a...@gentoo.org wrote: On 20/12/12 10:37 AM, Brian Dolbec
wrote:
/var/cache/repositories/ /var/cache/repositories/gentoo
== the main portage tree /var/cache/repositories/local
== the new location for a local overlay
/var/cache/repositories/some-overlay == layman
installed overlay
My idea for having all repos under one directory is to make
it easier for a pkg manager to simply scan the directory to
know all installed overlays. Currently each one has to be
listed in a configured variable in make.conf. So if you
wanted your local overlay somewhere else, then a symlink
would work (provided the PM can/will autoscan repos), or
add it to the PORTDIR_OVERLAY variable (current behavior).
I don't otherwise have a strong desire for it to be there.
If and only if the tree and all overlays (not other
directories) are not under one directory, then an autoscan
cannot easily happen.
You could do this while not having the portage tree be in that
directory. IE, portage goes in /var/cache/portage , and all the
overlays go into /var/cache/repositories.
The tree is separate enough IMO that autoscan can still happen
easily, and also I believe that it can be assumed that the tree
is in place. For instance, if the tree's location is defined to
be elsewhere, it isn't done so via PORTDIR_OVERLAYS but rather
PORTDIR.
On an unrelated note, I would never treat my local overlays as
cache. Ebuilds that (as a user) I wrote and installed by hand
are not likely to be kept in a repository someplace, but rather
the overlay dir would most likely be it's only location. IIRC
the reason for /usr/portage/local/ was to have a path within the
portage tree that rsync wouldn't kill; given that what you're
suggesting is already not under the proposed portage tree
location, emerge --sync couldn't touch it, and so I don't see a
need at all to provide a 'local' repository destination by
default.
It's sounding like the nearly the optimal solution would be:
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
Not to oversimplify but why exactly can't we leave
/usr/local/portage where it is? I'm not going to want to cd
/var/db/portage/repositories/local every time I want to edit a
local ebuild...
-ZC
IMO local user overlays would always end up being defined and placed
wherever the user decides them to be -- ie, /usr/local/portage as Rick
so nicely pointed out.
It may be a nice idea to try and enforce a structure to it/them, but
since PORTDIR_OVERLAY can be defined to include any path at all I
don't really see a point to it.
..I'm still with ulm about the tree not being in the repositories
subdir though. If I wanted to nuke all the overlays installed, rm
- -Rf /var/cache/portage/repos/* is very easy. If the tree is also in
that dir then it becomes less easy: find /var/cache/portage/repos
- -maxdepth 1 -mindepth 1 -not -name gentoo -exec rm -Rf {} \+
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)
iF4EAREIAAYFAlDTPlUACgkQ2ugaI38ACPBtPAD8DJ6BPjL6xY8alB5pbo7vQ5kb
dzRO9Z32F3r84RyVccABAIEu+k+ztw0ipoCwhmLlBHyiU6aEOsExixNvnMMLLu9X
=2gWT
-END PGP SIGNATURE-
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, 20 Dec 2012, Diego Elio Pettenò wrote:
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
+1
-1
The subdirs are too deeply nested. (Ebuilds would be at the eighth
level then...)
Let's put the tree in /var/cache/portage please, and distfiles in
/var/cache/distfiles. Layman overlays can stay where they are, or move
to /var/cache/layman.
Ulrich
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, Dec 20, 2012 at 12:05 PM, Ulrich Mueller u...@gentoo.org wrote:
On Thu, 20 Dec 2012, Diego Elio Pettenň wrote:
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
+1
-1
The subdirs are too deeply nested. (Ebuilds would be at the eighth
level then...)
Maybe I missed something...but what's wrong with that?
Let's put the tree in /var/cache/portage please, and distfiles in
/var/cache/distfiles. Layman overlays can stay where they are, or move
to /var/cache/layman.
--
:wq
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, Dec 20, 2012 at 06:58:11AM -0500, Rich Freeman wrote: On Thu, Dec 20, 2012 at 4:25 AM, George Shapovalov geo...@gentoo.org wrote: On Thursday 20 December 2012 09:11:39 Ulrich Mueller wrote: /var/cache/repositories/local== the new location for a local overlay Also I wonder if local overlays should be in /var/cache? It might not always be possible to restore them. So, this is the present /usr/local/portage? What's wrong with it? According to FHS, /usr/local/ is reserved for local admin meddling, which fits the bill rather nicely fo local overlay. This is the one location that was actually making sense from the beginning. I actually like the /var/cache/repositories approach. You can always add a symlink to it if you want to for convenience (as I already do for /var/lib/portage/world). There is really nothing special about /usr/portage, other than it being the lowest-priority overlay. If we do this, I don't like the name repositories -- what kind of repositories? should I put git repositories in there? As far as the local one goes - again this is configurable in /etc/make.conf, and we don't really need to pre-create that directory either. Right, there is nothing to pre-create for this; it should be left to the user. William pgpo5ORcGch97.pgp Description: PGP signature
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, 20 Dec 2012, Michael Mol wrote:
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
-1
The subdirs are too deeply nested. (Ebuilds would be at the eighth
level then...)
Maybe I missed something...but what's wrong with that?
There's no good reason for nesting it so deeply. As it is proposed
above, /var/cache/portage would contain only two subdirectories, and
/var/cache/portage/repositories only a single gentoo on a stable
system. Also /var/cache itself isn't overpopulated; I count about ten
entries on my systems.
We should go with a shorter (easier to remember, easier to type) path
and move things at least one level up. Two would be even better.
Let's put the tree in /var/cache/portage please, and distfiles in
/var/cache/distfiles. Layman overlays can stay where they are, or
move to /var/cache/layman.
Ulrich
Re: [gentoo-dev] Moving our/portage stuff to var
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
+1
-1
The subdirs are too deeply nested.
There is a practical advantage of having the main gentoo tree
and all overlays in the same directories which was not mentioned yet:
In this way it is easy to put them in a common filesystem appropriate
for this type of data (e.g. squashfs+aufs or whatever).
The same argument holds if you want to export the tree to several
systems: Probably all overlays should then be exported as well.
However, I have strong objections against using /var/cache at all:
FHS explicitly states:
Such data is locally generated ... The application must be able to
regenerate or restore the data.
In my opinion it is rather clear that regenerate does not mean
download. Not to speak about download-restricted data which
the application is certainly not able to regenerate.
(That locally maintained overlays do not fall into this category
is clear anyway; but again, it would be nice to have these possibly
on the same filesystem as all other ebuilds: Especially compressing
or other deduplicating filesystem would treat this better).
Therefore I would prefer another location.
Of course, on many systems, something under /srv might be the appropriate
place (especially if the tree should be exported); this should perhaps be
suggested to the user, although it is probably not a good idea to make
this the default, because any default can just be plain wrong for the
user's system.
What is really so bad about the current location under /usr as default?
Or maybe only split somewhat differently than currently like e.g.
/usr/portage/{gentoo,sunrise,...,local}
/usr/distiles
This would avoid the deep nesting, although it would allow a
common filesystem for the data of the same type.
As far as I can see, the argument against /usr/... is only that the
data is not constant. But actually, it is as constant as all
other data in /usr: Usually emerge --sync is only called to do
an emerge afterwards which means that the data only needs to be
modified if you intend to modify /usr anyway...
Martin
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, 20 Dec 2012 07:37:52 -0800 Brian Dolbec dol...@gentoo.org wrote: My idea for having all repos under one directory is to make it easier for a pkg manager to simply scan the directory to know all installed overlays. That's going to cause trouble, unless we start forcing overlays to contain enough information for a PM to configure them properly... -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Moving our/portage stuff to var
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 20/12/12 12:52 PM, Vaeth wrote: However, I have strong objections against using /var/cache at all: FHS explicitly states: Such data is locally generated ... The application must be able to regenerate or restore the data. emerge --sync -- regenerates/restores the portage tree layman -s -- regenerates/restores overlays emerge -f @world -- regenerates/restores any distfiles of consequence download-restricted data whose sole location is in a cache dir is a decision made by the user. Note that current portage has a PORTAGE_RO_DISTDIR var which could be used to specify the path to these fetch-restricted downloads rather than the user putting them directly into /var/cache/distfiles/. Maybe it would be pertinent to suggest and/or include a default path for this? What is really so bad about the current location under /usr as default? There were some suggestions about this in the initial thread posts. But /usr/portage is certainly OK to remain as-is, we do not *need* to make any changes. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) iF4EAREIAAYFAlDTU/AACgkQ2ugaI38ACPDmmwD/XIrHlMiDBU7WPUjp9DILJwuq fM/y5zUVZcoD2W7kcuEA/jPV9l1BH8qsw3UQxJKi6zbjBweD0uJU28jxiX2ybdjo =lBlR -END PGP SIGNATURE-
Re: [gentoo-dev] Moving our/portage stuff to var
On Thursday 20 December 2012 13:36:27 Alan McKinnon wrote: What program uses this local directory? It's not used directly by portage itself, though portage has an exclude for it in the default PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals). It goes back a long time, and is basically a poor man's local overlay without having to use layman. As I understand it, portage will treat the directory like any other when looking for ebuilds and resolving deps, but exclude it from a sync. Nope, he means /usr/portage/local, not /usr/local/portage. It is rarely (if ever, nowadays) present, as I understand, but you can find it mentioned in that config file. It may be just a legacy definition by now, looking at how only layman was mentioned with relation to it and even that one appears not to use it any more. BTW, /usr/local/portage is hardly poor man's. Where are you going to store your local changes that are of interest only to you and not present in any other overlays? (like, you want to keep some old version of some package after it has been cleaned, or your personal mods). The location even accords to FHS, which is, apparently, a rarity :). George
Re: [gentoo-dev] Moving our/portage stuff to var
On 20 December 2012 17:44, Ulrich Mueller u...@gentoo.org wrote:
On Thu, 20 Dec 2012, Michael Mol wrote:
/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}
-1
The subdirs are too deeply nested. (Ebuilds would be at the eighth
level then...)
Maybe I missed something...but what's wrong with that?
There's no good reason for nesting it so deeply. As it is proposed
above, /var/cache/portage would contain only two subdirectories, and
/var/cache/portage/repositories only a single gentoo on a stable
system. Also /var/cache itself isn't overpopulated; I count about ten
entries on my systems.
We should go with a shorter (easier to remember, easier to type) path
and move things at least one level up. Two would be even better.
Let's put the tree in /var/cache/portage please, and distfiles in
/var/cache/distfiles. Layman overlays can stay where they are, or
move to /var/cache/layman.
Ulrich
Yeah +1 to that. Makes more sense to me
--
Regards,
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
Re: [gentoo-dev] Moving our/portage stuff to var
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 20/12/12 01:18 PM, George Shapovalov wrote: On Thursday 20 December 2012 13:36:27 Alan McKinnon wrote: What program uses this local directory? It's not used directly by portage itself, though portage has an exclude for it in the default PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals). It goes back a long time, and is basically a poor man's local overlay without having to use layman. As I understand it, portage will treat the directory like any other when looking for ebuilds and resolving deps, but exclude it from a sync. Nope, he means /usr/portage/local, not /usr/local/portage. Alan's description *was* for /usr/portage/local -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) iF4EAREIAAYFAlDTVxYACgkQ2ugaI38ACPAEnQD/fo3/VbYD32yZMUuaEB0zZHES 71qGChzegSgxLqV01FoA/i583ha2AX+xLdw9/tyC7HUkzQ+9jXbqWKoT4Jay6bHc =dt7h -END PGP SIGNATURE-
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, 20 Dec 2012 18:44:06 +0100 Ulrich Mueller u...@gentoo.org wrote: There's no good reason for nesting it so deeply. As it is proposed above, /var/cache/portage would contain only two subdirectories, and /var/cache/portage/repositories only a single gentoo on a stable system. Also /var/cache itself isn't overpopulated; I count about ten entries on my systems. You really think most people don't have to use overlays? We should go with a shorter (easier to remember, easier to type) path and move things at least one level up. Two would be even better. You shouldn't ever be typing that path in... -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, Dec 20, 2012 at 12:31 PM, William Hubbs willi...@gentoo.org wrote: On Thu, Dec 20, 2012 at 06:58:11AM -0500, Rich Freeman wrote: On Thu, Dec 20, 2012 at 4:25 AM, George Shapovalov geo...@gentoo.org wrote: On Thursday 20 December 2012 09:11:39 Ulrich Mueller wrote: /var/cache/repositories/local== the new location for a local overlay Also I wonder if local overlays should be in /var/cache? It might not always be possible to restore them. So, this is the present /usr/local/portage? What's wrong with it? According to FHS, /usr/local/ is reserved for local admin meddling, which fits the bill rather nicely fo local overlay. This is the one location that was actually making sense from the beginning. I actually like the /var/cache/repositories approach. You can always add a symlink to it if you want to for convenience (as I already do for /var/lib/portage/world). There is really nothing special about /usr/portage, other than it being the lowest-priority overlay. If we do this, I don't like the name repositories -- what kind of repositories? should I put git repositories in there? I was pondering this, too. How about 'pms', for trees and overlays? -- :wq
Re: [gentoo-dev] Moving our/portage stuff to var
Ciaran McCreesh wrote: On Thu, 20 Dec 2012 18:44:06 +0100 Ulrich Mueller u...@gentoo.org wrote: There's no good reason for nesting it so deeply. As it is proposed above, /var/cache/portage would contain only two subdirectories, and /var/cache/portage/repositories only a single gentoo on a stable system. Also /var/cache itself isn't overpopulated; I count about ten entries on my systems. You really think most people don't have to use overlays? We should go with a shorter (easier to remember, easier to type) path and move things at least one level up. Two would be even better. You shouldn't ever be typing that path in... I was thinking tab completion myself. Just hit a couple letters and hit tab. That tab key types much faster and more accurately than I can. ;-) Dale :-) :-) -- I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, 20 Dec 2012, Ciaran McCreesh wrote:
We should go with a shorter (easier to remember, easier to type) path
and move things at least one level up. Two would be even better.
You shouldn't ever be typing that path in...
Ebuilds tell users to do so:
pkg_nofetch() {
einfo Please download ${foo} and place it in ${DISTDIR}
}
Re: [gentoo-dev] Moving our/portage stuff to var
On Thursday 20 December 2012 13:21:11 Ian Stakenvicius wrote: Nope, he means /usr/portage/local, not /usr/local/portage. Alan's description *was* for /usr/portage/local Really? It matches /usr/local/portage pretty well. How did it come around then? We had /usr/local/portage for ages for storing local changes..
Re: [gentoo-dev] Moving our/portage stuff to var
On 12/20/2012 06:12 AM, Graham Murray wrote: Zac Medico zmed...@gentoo.org writes: On 12/19/2012 02:01 PM, Alan McKinnon wrote: If we are going to move distfiles out of the tree into, what are the odds of getting /some/path/portage/local to move somewhere else too? What program uses this local directory? It's not used directly by portage itself, though portage has an exclude for it in the default PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals). It is useful for 'site' local ebuilds. It allows a 'master' repository to sync with the main Gentoo one without disturbing local changes but allow other systems on-site to fetch the modified tree with a simple 'emerge --sync'. This usage is slightly annoying, because it tends to give people the impression that it's safe to store random things inside $PORTDIR, while it's somewhat fragile given that it relies on special rsync options. Occasionally, we get bug reports from people who have lost files because of this sort of confusion: https://bugs.gentoo.org/show_bug.cgi?id=131030 https://bugs.gentoo.org/show_bug.cgi?id=392565 -- Thanks, Zac
Re: [gentoo-dev] Moving our/portage stuff to var
Ian Stakenvicius wrote: Such data is locally generated ... The application must be able to regenerate or restore the data. emerge --sync -- regenerates/restores the portage tree Perhaps my English is too poor, but IMHO download != regenerate/restore. Indeed, the FHS also emphasizes only the *time* which it costs to regenerate the data in /var/cache and that it can be deleted without data loss (e.g. if your filesystem runs full). However, if you would do this e.g. on a laptop because you blindly trusted FHS you may have a lot of problems: Not only can it be quite expensive to regenerate the data (e.g. if you have to pay a lot for the bandwidth where you currently are), it might even be impossible if you have no internet connection at all or e.g. if you realize only after the deletion that you would have to (re)emerge some package to make the internet connection work (e.g. because you did not revdep-rebuild before). (And, yes, if you deal with distdir carefully, there are several occassions when you might want to (re)emerge a package even if you have no or only very limited internet connection; not only for revdep-rebuild but also e.g. if you need to change certain USE-Flags.) Of course, an experienced user knows and can care about this all, but an experienced user will choose the location anyway as he needs: We are speaking about the default which should mainly help also the less experienced user. download-restricted data whose sole location is in a cache dir is a decision made by the user. If you make /var/cache/... the default, this suggests the wrong decision to the user. At least, it should be documented then in a prominent place that the default path where the data is required is probably not the appropriate path to keep it (which IMHO is a bit strange). [...] PORTAGE_RO_DISTDIR [...] Maybe it would be pertinent to suggest and/or include a default path for this? This would be a good idea anyway, also because of theoretically possible name collisions in distdir. Martin
Re: [gentoo-dev] Moving our/portage stuff to var
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 20/12/12 01:55 PM, George Shapovalov wrote: On Thursday 20 December 2012 13:21:11 Ian Stakenvicius wrote: Nope, he means /usr/portage/local, not /usr/local/portage. Alan's description *was* for /usr/portage/local Really? It matches /usr/local/portage pretty well. How did it come around then? We had /usr/local/portage for ages for storing local changes.. /usr/local/portage has always been a convention or recommendation; it's not a directory that portage (package or tree) ever created, enforced, or did anything in particular to support. /usr/portage/local/ came around (i think -- i was around at this time but was not a dev and was not privy to decision making) so that locally modified ebuilds could be stored and distributed (ie via netmount or manual rsync) along with the rest of the portage tree without worries of the changes being wiped out on the next --sync. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) iF4EAREIAAYFAlDTZhEACgkQ2ugaI38ACPBEXwD+IuFOgsHcQDNaqUCUfSZW53ca 7gsST6Prls/7rPmpGqcBAKnnUIH48UPcDYrwexlNbmPzRN9CjYaeR36/2qo/hC47 =r5C9 -END PGP SIGNATURE-
Re: [gentoo-dev] Moving our/portage stuff to var
On 12/20/2012 03:36 AM, Alan McKinnon wrote:
On Wed, 19 Dec 2012 14:19:52 -0800
Zac Medico zmed...@gentoo.org wrote:
On 12/19/2012 02:01 PM, Alan McKinnon wrote:
On Wed, 19 Dec 2012 14:56:44 +0100
Diego Elio Pettenò flamee...@flameeyes.eu wrote:
Just mv /usr/portage /var/portage ? FFS no. Among other things, as
many said before, we should really take distfiles out of the tree
itself, and packages the same. And I don't want /var/packages
or /var/distfiles at all.
If we are going to move distfiles out of the tree into, what are the
odds of getting /some/path/portage/local to move somewhere else too?
What program uses this local directory? It's not used directly by
portage itself, though portage has an exclude for it in the default
PORTAGE_RSYNC_OPTS setting
(in /usr/share/portage/config/make.globals).
It goes back a long time, and is basically a poor man's local overlay
without having to use layman. As I understand it, portage will treat
the directory like any other when looking for ebuilds and resolving
deps, but exclude it from a sync.
Portage doesn't have any special handling for this directory, aside from
the exclude in the default PORTAGE_RSYNC_OPTS setting. I would not
encourage people to use this directory for anything, because it tends to
give people the impression that it's safe to store random things inside
$PORTDIR, while it's somewhat fragile given that it relies on special
rsync options. Occasionally, we get bug reports from people who have
lost files because of this sort of confusion:
https://bugs.gentoo.org/show_bug.cgi?id=131030
https://bugs.gentoo.org/show_bug.cgi?id=392565
That one has irked me for ages, its the one thing left on my systems
that stops the local tree dir being an exact replica of the upstream
master.
For portage's defaults, I won't settle for anything less than having
them all refer to separate directories which are *not* nested within
one other. These are the current default settings which violate my
requirements:
PORTDIR=/usr/portage
DISTDIR=${PORTDIR}/distfiles
PKGDIR=${PORTDIR}/packages
RPMDIR=${PORTDIR}/rpm
/usr/portage/local has the taste feel and smell of a hacky workaround:
shove a directory in the tree and exclude it from sync.
Right.
I suspect the best solution all round is to move all support for local
overlays into layman. I'd be happy with that. Probably make the portage
code cleaner too.
As mentioned, portage doesn't have any special handling for this
directory (aside from the rsync exclude).
--
Thanks,
Zac
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, 20 Dec 2012 19:18:56 +0100 George Shapovalov geo...@gentoo.org wrote: It goes back a long time, and is basically a poor man's local overlay without having to use layman. As I understand it, portage will treat the directory like any other when looking for ebuilds and resolving deps, but exclude it from a sync. Nope, he means /usr/portage/local, not /usr/local/portage. It is rarely (if ever, nowadays) present, as I understand, but you can find it mentioned in that config file. It may be just a legacy definition by now, looking at how only layman was mentioned with relation to it and even that one appears not to use it any more. I thought about this some more, and now realise I've been using essentially the same set of config files since about 2004. I've never lost them and always had a current copy so each time I build a new host I just copy, tweak CFLAGS, maybe MAKEOPTS, and let 'er rip. My local is probably years out of date. Serves me right for not reading 50 screens of man page with every new host :-) This sub-thread is probably just noise, sorry for that. -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-dev] Moving our/portage stuff to var
On Thu, 20 Dec 2012 19:44:36 +0100
Ulrich Mueller u...@gentoo.org wrote:
On Thu, 20 Dec 2012, Ciaran McCreesh wrote:
We should go with a shorter (easier to remember, easier to type)
path and move things at least one level up. Two would be even
better.
You shouldn't ever be typing that path in...
Ebuilds tell users to do so:
pkg_nofetch() {
einfo Please download ${foo} and place it in ${DISTDIR}
}
I believe Unix has a facility for taking bits of text that are on the
screen and copying them without the need to type the whole thing in.
But in any case, DISTDIR shouldn't be under the repository dir at all.
--
Ciaran McCreesh
signature.asc
Description: PGP signature
Re: [gentoo-dev] Moving our/portage stuff to var
Am Dienstag, 18. Dezember 2012, 22:33:06 schrieb Diego Elio Pettenò: No /var/gentoo. No /var/repositories. /var/db/gentoo, /var/db/repositories, /var/cache/portage ... as long as Zac is fine with one whatever, but let's not invent any new top-level. Why not? From a FHS pov it seems ok... I would suggest /var/portage ... -Marc -- 0x35A64134 - 8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134 signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Moving our/portage stuff to var
On 19/12/2012 13:44, Marc Schiffbauer wrote: Why not? From a FHS pov it seems ok... I would suggest /var/portage ... Seriously, mine is going to be a huge veto here with as much power I can put. There is a _reason_ why stuff is added to /var/lib instead of having /var/postgres /var/mysql /var/foobar /var/wtf /var/wth /var/imtired ... As I said on other messages before (which you probably missed since you ask Why not?), putting it in /var/lib or /var/db or /var/cache makes it explicit how you should handle its backup. /var/portage ? I have to look it up manually. Let's to things properly instead of how it looks cooler. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/ signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
But, anyway, I think, /var/repositories/gentoo is very very nice idea ;) 19.12.2012 03:03, Rick Zero_Chaos Farina пишет: On 12/18/2012 02:49 PM, Rick Zero_Chaos Farina wrote: On 12/18/2012 01:38 PM, Zac Medico wrote: On 12/17/2012 02:19 AM, Tomáš Chvátal wrote: Currently we put portage into /usr/portage and all related stuff is to be in the subfolders there (distfiles, binpkg). I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). The only reason why we have this currently in usr is that bsd ports put their stuff in there and I suppose Daniel just did the same. With respect to reality how stuff is done in the linux land all the variable data should be in /var so we should adjust and move it in there too. What would you think? I like the idea. As noted in bug #378603 [1], I'd like the portage ebuild to ensure that the locations don't unexpectedly change for existing installs. Will it break catalyst? If so, we might begin the migration by fixing catalyst and having to set the new default locations in the make.conf that it generates. [1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1 Yes, it will break catalyst. However, if the folder that the portage tree goes in is still something/something/portage then it shouldn't be hard to fix. It should probably be mentioned (since most of us don't use the snapshots every day) that the snapshots actually contain a folder called portage in the tarball. Not that it would be impossible to change, but if we migrate from /usr/portage to /var/whatever/portage then the changes are trivial, if we migrate to /var/repositories/gentoo that makes things like unpacking the snapshots significantly non-trivial. I really don't care what everyone wants to do here (although I'm generally for sticking closer to FHS), but I warn that if the path doesn't end in portage the changes are going to be significantly non-trivial. Thanks, Zero -ZC signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
On 19/12/2012 14:03, Vadim A. Misbakh-Soloviov wrote: But, anyway, I think, /var/repositories/gentoo is very very nice idea ;) I'm going to repeat myself until this is shot down entirely. We're not going to create a new top-level directory in /var. Get over it. Stop. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/ signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
On Wed, 19 Dec 2012, Diego Elio Pettenò wrote: On 19/12/2012 13:44, Marc Schiffbauer wrote: I would suggest /var/portage ... Seriously, mine is going to be a huge veto here with as much power I can put. Why? The portage tree is of central importance for Gentoo, so IMHO a second-level directory would be acceptable for it. Besides, it currently is in /usr/portage, so it wouldn't be new but would only move from /usr to /var. There is a _reason_ why stuff is added to /var/lib instead of having /var/postgres /var/mysql /var/foobar /var/wtf /var/wth /var/imtired ... I don't understand how this is related to the discussion. None of the above have any relevance for Gentoo that would be comparable to Portage. This doesn't mean that /var/portage is the only possible choice. But IMHO it's better than some of the other suggestions that I've seen here, like /var/cache/portage/repositories/gentoo/tree and so on. As I said on other messages before (which you probably missed since you ask Why not?), putting it in /var/lib or /var/db or /var/cache makes it explicit how you should handle its backup. Yes, these are certainly fine, as long as we don't add additional useless subdirectory levels. /var/portage ? I have to look it up manually. Please, stay serious. ;-) Ulrich
Re: [gentoo-dev] Moving our/portage stuff to var
On 19/12/2012 14:43, Ulrich Mueller wrote: Why? The portage tree is of central importance for Gentoo, so IMHO a second-level directory would be acceptable for it. Besides, it currently is in /usr/portage, so it wouldn't be new but would only move from /usr to /var. I'm irked enough by /usr/portage that using that as a reason is just going to make me feel even more strongly that it should not be /var/portage. I don't understand how this is related to the discussion. None of the above have any relevance for Gentoo that would be comparable to Portage. See above. This doesn't mean that /var/portage is the only possible choice. But IMHO it's better than some of the other suggestions that I've seen here, like /var/cache/portage/repositories/gentoo/tree and so on. I'm not arguing that it should go 5 levels deep. But two or three deep is fine for me. Is it going to be /var/db/portage/master ? Fine. Is it going to be /var/cache/portage/tree? Fine. /var/cache/portage + /var/cache/distfiles ? Fine. Just mv /usr/portage /var/portage ? FFS no. Among other things, as many said before, we should really take distfiles out of the tree itself, and packages the same. And I don't want /var/packages or /var/distfiles at all. /var/portage ? I have to look it up manually. Please, stay serious. ;-) I am serious. If it's my first time backing up a system, and I encounter a directory /var/portage, it doesn't make it clear what it contains. Is it re-generable? Should it be backed up entirely? That's why my suggestion is to use /var/cache: it makes it clear that there is no definitive reason to back it up (as Justin said there is an issue with distfiles you can't re-download but that's a different story I'd say — maybe setting a default read-only distdir for said packages might make sense, but I don't want to get there at all). Also, I usually keep /var/cache in a more unsafe disk — I don't care if I lose cache because the drive dies, while /var/lib is fully backed up. I don't usually split /var/db but I can see what people were saying about having different allocation requirements for the tree compared to distfiles, and I guess that if we put the tree there we could gain something even for /var/db/pkg by splitting it. Tree hierarchies are there to make things more easily organized, not just to look nice. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/
Re: [gentoo-dev] Moving our/portage stuff to var
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On 19/12/12 08:56 AM, Diego Elio Pettenò wrote:
That's why my suggestion is to use /var/cache: it makes it clear
that there is no definitive reason to back it up (as Justin said
there is an issue with distfiles you can't re-download but that's a
different story I'd say — maybe setting a default read-only distdir
for said packages might make sense, but I don't want to get there
at all).
In terms of the fact that a current copy of the portage tree is always
available, no it isn't necessary to back it up. However, if one isn't
constantly maintaining their system via -uDN and doing say, updates on
a monthly cycle (ie, production systems), then it is very useful to
maintain the same portage tree snapshot as the system's last -uDN ...
As such I would argue that it is worthwhile to back it up.
Similarly, 'packages' should probably stay synchronized with the tree.
So in terms of the above, would that mean /var/lib is a better fit?
or would that mean /var/cache and it is up to the user to add their
own backup of /var/cache/portage ?
Distfiles, imo, are definitely just cache and can be discarded at any
time. There are issues if one has a very old tree that some distfiles
disappear from the mirrors (especially gentoo patchset tarballs) but
such is life -- personally I'd like to see all such files stored on a
dev's webspace in perpetuity so that SRC_URI could grab it from there
after it's dropped from the mirrors. As for special distfiles
(fetch-restricted etc), these would need to be downloaded manually
anyways and if they are of value they should be backed up elsewhere
(ie, not rely on the distfiles dir to keep them).
Either of i.e. /var/cache/{distfiles,packages,portage} or
/var/cache/portage/{distfiles,packages,tree} works for me; i can see
the extra directory level keeping all portage bits together as looking
nicer for the end user but meh.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)
iF4EAREIAAYFAlDRy8kACgkQ2ugaI38ACPAiLQD9HeENg+cPrkQcHhPF54h1AaPG
hvTvaq4GaghMNXCKV7sBAKz8cKR6LD8grvuTnftWVJiRYYbhYM+HANTaE5xWs6f+
=WPmW
-END PGP SIGNATURE-
Re: [gentoo-dev] Moving our/portage stuff to var
On 19/12/2012 15:14, Ian Stakenvicius wrote: So in terms of the above, would that mean /var/lib is a better fit? or would that mean /var/cache and it is up to the user to add their own backup of /var/cache/portage ? I would say it's up to the user. When I do that kind of setup I actually use a single rsync source and tar it up from there. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/ signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
On Tue, 2012-12-18 at 23:16 -0800, Joshua Saddler wrote: On Mon, 17 Dec 2012 11:19:20 +0100 Tomáš Chvátal tomas.chva...@gmail.com wrote: Currently we put portage into /usr/portage and all related stuff is to be in the subfolders there (distfiles, binpkg). I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). The only reason why we have this currently in usr is that bsd ports put their stuff in there and I suppose Daniel just did the same. With respect to reality how stuff is done in the linux land all the variable data should be in /var so we should adjust and move it in there too. What would you think? do it. stick it somewhere in /var. i have a small SLC SSD just for /var and /usr/portage partitions, since those consistently incur high writes. dropping to just one partition for all that i/o would be real nice. if this proposed change is made, please make sure to contact the GDP. while we don't update things like manpages or elog announcements, we would have a ton of stuff to fix in gentoo.org/doc/en/ . also, make sure stuff is sorted out on the catalyst/releng end well in advance, so users aren't stuck with bad stages. Yes, Catalyst and the portage defaults will be changed in a co-ordinated manner. I've started poking around catalyst, but it has paths hard-coded nearly everywhere, despite it having passed config variables around most places. So, It is going to need code cleanup first. It will take me some time to get familiar enough with the code before I make many changes to clean it up. For the documentation, primarily the install handbook, perhaps it would be better to mention both locations, at least mention the old location so existing and old users won't be thrown for a loop. It will also need some attention in the forums where tons of threads will be referencing /usr/portage. Perhaps the docs team could start preparing the docs changes in a manner that will be easy to search/replace with the final correct locations when that decision is made. And YES Diego, it won't be /var/portage or /var/repositories, we heard you. From my rough tracking, I believe somewhere under /var/cache was majority vote. Anyway, once catalyst is ready it will be easy to set it to whatever is finally decided. -- Brian Dolbec dol...@gentoo.org signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Moving our/portage stuff to var
On 19/12/2012 16:54, Brian Dolbec wrote: And YES Diego, it won't be /var/portage or /var/repositories, we heard you. Thanks, it's appreciated :) -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/ signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
Am Mittwoch, 19. Dezember 2012, 14:43:56 schrieb Ulrich Mueller: On Wed, 19 Dec 2012, Diego Elio Pettenò wrote: On 19/12/2012 13:44, Marc Schiffbauer wrote: I would suggest /var/portage ... Seriously, mine is going to be a huge veto here with as much power I can put. Why? The portage tree is of central importance for Gentoo, so IMHO a second-level directory would be acceptable for it. Besides, it currently is in /usr/portage, so it wouldn't be new but would only move from /usr to /var. +1 FHS 2.3: Applications must generally not add directories to the top level of /var. Such directories should only be added __if they have some system-wide implication__, and in consultation with the FHS mailing list. -Marc -- 0x35A64134 - 8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134 signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Moving our/portage stuff to var
On Wed, 19 Dec 2012 14:56:44 +0100 Diego Elio Pettenò flamee...@flameeyes.eu wrote: Just mv /usr/portage /var/portage ? FFS no. Among other things, as many said before, we should really take distfiles out of the tree itself, and packages the same. And I don't want /var/packages or /var/distfiles at all. If we are going to move distfiles out of the tree into, what are the odds of getting /some/path/portage/local to move somewhere else too? That one has irked me for ages, its the one thing left on my systems that stops the local tree dir being an exact replica of the upstream master. -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-dev] Moving our/portage stuff to var
On 12/19/2012 02:01 PM, Alan McKinnon wrote:
On Wed, 19 Dec 2012 14:56:44 +0100
Diego Elio Pettenò flamee...@flameeyes.eu wrote:
Just mv /usr/portage /var/portage ? FFS no. Among other things, as
many said before, we should really take distfiles out of the tree
itself, and packages the same. And I don't want /var/packages
or /var/distfiles at all.
If we are going to move distfiles out of the tree into, what are the
odds of getting /some/path/portage/local to move somewhere else too?
What program uses this local directory? It's not used directly by
portage itself, though portage has an exclude for it in the default
PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals).
That one has irked me for ages, its the one thing left on my systems
that stops the local tree dir being an exact replica of the upstream
master.
For portage's defaults, I won't settle for anything less than having
them all refer to separate directories which are *not* nested within one
other. These are the current default settings which violate my requirements:
PORTDIR=/usr/portage
DISTDIR=${PORTDIR}/distfiles
PKGDIR=${PORTDIR}/packages
RPMDIR=${PORTDIR}/rpm
--
Thanks,
Zac
Re: [gentoo-dev] Moving our/portage stuff to var
On Wed, Dec 19, 2012 at 5:19 PM, Zac Medico zmed...@gentoo.org wrote: On 12/19/2012 02:01 PM, Alan McKinnon wrote: On Wed, 19 Dec 2012 14:56:44 +0100 Diego Elio Pettenņ flamee...@flameeyes.eu wrote: Just mv /usr/portage /var/portage ? FFS no. Among other things, as many said before, we should really take distfiles out of the tree itself, and packages the same. And I don't want /var/packages or /var/distfiles at all. If we are going to move distfiles out of the tree into, what are the odds of getting /some/path/portage/local to move somewhere else too? What program uses this local directory? It's not used directly by portage itself, though portage has an exclude for it in the default PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals). layman used to use /usr/portage/local for storing overlays. See code listing 2.2. http://www.gentoo.org/proj/en/overlays/userguide.xml
Re: [gentoo-dev] Moving our/portage stuff to var
Am 19.12.2012 17:25, schrieb Diego Elio Pettenò: On 19/12/2012 17:19, Marc Schiffbauer wrote: FHS 2.3: Applications must generally not add directories to the top level of /var. Such directories should only be added __if they have some system-wide implication__, and in consultation with the FHS mailing list. Since you like adding emphasis: FHS 2.3: Applications must generally not add directories to the top level of /var. Such directories should only be added if they have some system-wide implication, *and in consultation with the FHS mailing list.* It's always nice to cherry-pick half a phrase from a standard, right? No. That would be the next step. But I do not see any problem here. Do you? -- 8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134
Re: [gentoo-dev] Moving our/portage stuff to var
On Wed, 2012-12-19 at 17:33 -0500, Mike Gilbert wrote: On Wed, Dec 19, 2012 at 5:19 PM, Zac Medico zmed...@gentoo.org wrote: On 12/19/2012 02:01 PM, Alan McKinnon wrote: On Wed, 19 Dec 2012 14:56:44 +0100 Diego Elio Pettenņ flamee...@flameeyes.eu wrote: Just mv /usr/portage /var/portage ? FFS no. Among other things, as many said before, we should really take distfiles out of the tree itself, and packages the same. And I don't want /var/packages or /var/distfiles at all. If we are going to move distfiles out of the tree into, what are the odds of getting /some/path/portage/local to move somewhere else too? I am already making this configurable in catalyst which builds the stages and install media. It is my intention to move it to the new repositories location alongside the gentoo tree directory and any layman installed overlays. What program uses this local directory? It's not used directly by portage itself, though portage has an exclude for it in the default PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals). layman used to use /usr/portage/local for storing overlays. See code listing 2.2. http://www.gentoo.org/proj/en/overlays/userguide.xml I plan to migrate layman to the new portage/catalyst defaults when they are done. The current default is /var/lib/layman As I will not be the sole dictator as to what these new defaults will be. I am working on the code to make it easy to set to whatever the final decision is made to be. just to clarify, I'm voting for... /var/cache/distfiles /var/cache/packages /var/cache/repositories/ /var/cache/repositories/gentoo == the main portage tree /var/cache/repositories/local== the new location for a local overlay /var/cache/repositories/some-overlay == layman installed overlay -- Brian Dolbec dol...@gentoo.org signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/2012 13:15, Dirkjan Ochtman wrote: On Mon, Dec 17, 2012 at 11:23 AM, Diego Elio Pettenò flamee...@flameeyes.eu wrote: I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. +1. Cheers, Dirkjan +1 Ultra
Re: [gentoo-dev] Moving our/portage stuff to var
On 12/17/2012 02:19 AM, Tomáš Chvátal wrote: Currently we put portage into /usr/portage and all related stuff is to be in the subfolders there (distfiles, binpkg). I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). The only reason why we have this currently in usr is that bsd ports put their stuff in there and I suppose Daniel just did the same. With respect to reality how stuff is done in the linux land all the variable data should be in /var so we should adjust and move it in there too. What would you think? I like the idea. As noted in bug #378603 [1], I'd like the portage ebuild to ensure that the locations don't unexpectedly change for existing installs. Will it break catalyst? If so, we might begin the migration by fixing catalyst and having to set the new default locations in the make.conf that it generates. [1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1 -- Thanks, Zac
Re: [gentoo-dev] Moving our/portage stuff to var
On 12/17/2012 02:19 AM, Tomáš Chvátal wrote: Currently we put portage into /usr/portage and all related stuff is to be in the subfolders there (distfiles, binpkg). I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). The only reason why we have this currently in usr is that bsd ports put their stuff in there and I suppose Daniel just did the same. With respect to reality how stuff is done in the linux land all the variable data should be in /var so we should adjust and move it in there too. What would you think? I like the idea. As noted in bug #378603 [1], I'd like the portage ebuild to ensure that the locations don't unexpectedly change for existing installs. Will it break catalyst? If so, we might begin the migration by fixing catalyst and having to set the new default locations in the make.conf that it generates. [1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1 -- Thanks, Zac
Re: [gentoo-dev] Moving our/portage stuff to var
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/18/2012 01:38 PM, Zac Medico wrote: On 12/17/2012 02:19 AM, Tomáš Chvátal wrote: Currently we put portage into /usr/portage and all related stuff is to be in the subfolders there (distfiles, binpkg). I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). The only reason why we have this currently in usr is that bsd ports put their stuff in there and I suppose Daniel just did the same. With respect to reality how stuff is done in the linux land all the variable data should be in /var so we should adjust and move it in there too. What would you think? I like the idea. As noted in bug #378603 [1], I'd like the portage ebuild to ensure that the locations don't unexpectedly change for existing installs. Will it break catalyst? If so, we might begin the migration by fixing catalyst and having to set the new default locations in the make.conf that it generates. [1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1 Yes, it will break catalyst. However, if the folder that the portage tree goes in is still something/something/portage then it shouldn't be hard to fix. - -ZC -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBAgAGBQJQ0MjSAAoJEKXdFCfdEflKTZ4P/2h5iKIGbNWhxAGkWEDRez31 YiCVUf5lJeDGAYpJ0hM20aWebzrR+HITqcKENKfGwerr+2/VeKVKsyYR8M1tNbQs 9jCNTMBfoYOEm8ElOu0Vzny5uWV3LflP0BMsnOjBDUpxw9ESXuz+54tkknsyEaLD yBoVoe4zj4GfY/B3aXO08gqig/w7sGR3iLPyhy3iJimznK/qMszyMJ9LzCFVSmwz zXVMoaM1VKTwVI7E/CZmxBWIF4KVoQ1fl4xJoqt4l82HK/5IDoRLu+s5c6rJLDhp SPDsFQa+XtdHOZvaXhn+15zAcWFJm8UBdXtnfi8tJmpoukEJr3BscC9DRUCdRzbR mzE0ptqDayoJ8tKuxUR+ppU987PPIAJ03h/X3yTWsIc7QPB1TtFI5W6mk4/iVPSy FF9LrcxYvAM/+Cqu+LlhKIapqK+XAA7nw6txpEjiw3HcTZ+0O/GGHTlKM0gxbx0i kMGe00zwodoN7DwGYenmulDk3eX71moZ/ioXMMNisDAazmcbOKOEXC/pmuUk342H 9oUtgYCH/PJGjdZHAlQnBwyfIP2YCNRZEnL6lN5t6V+p0XPzMB97tJCYsB2qEifx m1qOPZhQX5dXovKJZyrs93SbpkUhMF8TXi6IsosVtAQFldRd2LJOawRBEkZzF2Yn mn0gQl0ORyorOMEYsxbL =lZrh -END PGP SIGNATURE-
Re: [gentoo-dev] Moving our/portage stuff to var
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/18/2012 02:49 PM, Rick Zero_Chaos Farina wrote: On 12/18/2012 01:38 PM, Zac Medico wrote: On 12/17/2012 02:19 AM, Tomáš Chvátal wrote: Currently we put portage into /usr/portage and all related stuff is to be in the subfolders there (distfiles, binpkg). I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). The only reason why we have this currently in usr is that bsd ports put their stuff in there and I suppose Daniel just did the same. With respect to reality how stuff is done in the linux land all the variable data should be in /var so we should adjust and move it in there too. What would you think? I like the idea. As noted in bug #378603 [1], I'd like the portage ebuild to ensure that the locations don't unexpectedly change for existing installs. Will it break catalyst? If so, we might begin the migration by fixing catalyst and having to set the new default locations in the make.conf that it generates. [1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1 Yes, it will break catalyst. However, if the folder that the portage tree goes in is still something/something/portage then it shouldn't be hard to fix. It should probably be mentioned (since most of us don't use the snapshots every day) that the snapshots actually contain a folder called portage in the tarball. Not that it would be impossible to change, but if we migrate from /usr/portage to /var/whatever/portage then the changes are trivial, if we migrate to /var/repositories/gentoo that makes things like unpacking the snapshots significantly non-trivial. I really don't care what everyone wants to do here (although I'm generally for sticking closer to FHS), but I warn that if the path doesn't end in portage the changes are going to be significantly non-trivial. Thanks, Zero -ZC -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBAgAGBQJQ0MwqAAoJEKXdFCfdEflKF0YQAJ1mBkAPgDcIdXL+2eIdc5GO yk7t2qSOtSaU159C8d2xI2AdB3b5jm5AXWrrvpAovNIAMTN/fFa781oKQVkFprsS AZsT3UFluLep3ngLVTvpv8nry2t0AhT+/k1u1GAeurnTT5b4GDGXu7EF+QHzeaLC +8VF8/N0TvRp+PwZ9lEJGXd+RVXxQ6AJXo1XsIbgKnA/L5mCKCfYn3o+9oQCqHbW fPVeGyqEAKVYFG8rNUHwxPHYUvxhIC7xxuBl81ErXv5VqhcgNNjX4SMnKXeT3Z/V FRxtnLJMz69mlX9C+CMf9boFVfveMjGj7pZOzopJGGhJM3d8Q/rAbgHejEmtrlAd n9FBLc7ihGGBGGcTHrmhG+1DalmY/c2WaR63NfR+Le5mIpPpC0mibhK5TYeWRAcF zFJhjmLwBfyAlBLnQk6gd4x/c8jMH68wgvRqPNMJbWETkT3tntgl7cUcoesv6TdT fGNuISGCqMSEHHTKxX9XvqBl5EDSNEIS4I7rbFwO3qtuHuR2+88H506E9RQU4Xdd +/bV6EW3Z7s/9hp7LhvK5CqEXAfJwN278aJOKAuyNX/YmwKa1VXy/ZDMXEUrTa6x r+LcgyNHUkY3Sr4OFR7yoIY/oH4FrP1Y5H45GQUNje/XeL7bnLoN9h4QfeIwwt/L 6KZZtj3hmJa7I/m5hBwv =VeR7 -END PGP SIGNATURE-
Re: [gentoo-dev] Moving our/portage stuff to var
Am Mon, 17 Dec 2012 13:45:35 +0100 schrieb Diego Elio Pettenò flamee...@flameeyes.eu: On 17/12/2012 13:42, Ulrich Mueller wrote: If we change the location, can we then move distfiles to some place outside of the tree? Something like: /var/cache/portage /var/cache/distfiles What I do on my systems is /var/cache/portage/tree /var/cache/portage/distfiles +1 for moving the location my suggestion: /var/gentoo/distfiles /var/gentoo/packages /var/gentoo/repositories /var/gentoo/repositories/layman /var/gentoo/repositories/portage . . this could be extended, like: /var/gentoo/repositories/sth_paludis_related It would be nice, if you'd adopted my scheme, then I don't need to change anything.. ;)
Re: [gentoo-dev] Moving our/portage stuff to var
No /var/gentoo. No /var/repositories. /var/db/gentoo, /var/db/repositories, /var/cache/portage ... as long as Zac is fine with one whatever, but let's not invent any new top-level. Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/ On Tue, Dec 18, 2012 at 10:20 PM, Florian D. flockm...@gmail.com wrote: Am Mon, 17 Dec 2012 13:45:35 +0100 schrieb Diego Elio Pettenò flamee...@flameeyes.eu: On 17/12/2012 13:42, Ulrich Mueller wrote: If we change the location, can we then move distfiles to some place outside of the tree? Something like: /var/cache/portage /var/cache/distfiles What I do on my systems is /var/cache/portage/tree /var/cache/portage/distfiles +1 for moving the location my suggestion: /var/gentoo/distfiles /var/gentoo/packages /var/gentoo/repositories /var/gentoo/repositories/layman /var/gentoo/repositories/portage . . this could be extended, like: /var/gentoo/repositories/sth_paludis_related It would be nice, if you'd adopted my scheme, then I don't need to change anything.. ;)
Re: [gentoo-dev] Moving our/portage stuff to var
On 12/18/2012 01:33 PM, Diego Elio Pettenò wrote: No /var/gentoo. No /var/repositories. /var/db/gentoo, /var/db/repositories, /var/cache/portage ... as long as Zac is fine with one whatever, but let's not invent any new top-level. Yeah, /var/db or /var/cache sounds good to me. I would encourage all those interested to coordinate with the catalyst developers to have the new defaults written in the make.conf that it generates. Having the new defaults explicitly recorded there does not require any changes to portage (though I will bring portage's internal defaults into sync as soon as possible). Also, I think it might help avoid confusion for users if they are able to see the new defaults explicitly recorded in make.conf (as opposed to /usr/share/portage/config/make.globals which is somewhat obscure). -- Thanks, Zac
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, 17 Dec 2012 11:19:20 +0100 Tomáš Chvátal tomas.chva...@gmail.com wrote: Currently we put portage into /usr/portage and all related stuff is to be in the subfolders there (distfiles, binpkg). I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). The only reason why we have this currently in usr is that bsd ports put their stuff in there and I suppose Daniel just did the same. With respect to reality how stuff is done in the linux land all the variable data should be in /var so we should adjust and move it in there too. What would you think? do it. stick it somewhere in /var. i have a small SLC SSD just for /var and /usr/portage partitions, since those consistently incur high writes. dropping to just one partition for all that i/o would be real nice. if this proposed change is made, please make sure to contact the GDP. while we don't update things like manpages or elog announcements, we would have a ton of stuff to fix in gentoo.org/doc/en/ . also, make sure stuff is sorted out on the catalyst/releng end well in advance, so users aren't stuck with bad stages. signature.asc Description: PGP signature
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/
Re: [gentoo-dev] Moving our/portage stuff to var
2012/12/17 Diego Elio Pettenò flamee...@flameeyes.eu: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. Thats right, it should be even better location. :-)
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, 17 Dec 2012 11:23:00 +0100 Diego Elio Pettenò flamee...@flameeyes.eu wrote: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. +1 on /var/cache. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Moving our/portage stuff to var
On 17 December 2012 18:27, Tomáš Chvátal tomas.chva...@gmail.com wrote: 2012/12/17 Diego Elio Pettenò flamee...@flameeyes.eu: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. Thats right, it should be even better location. :-) I prefer some location in /var/ as well. -- Cheers, Ben | yngwin Gentoo developer Gentoo Qt project lead, Gentoo Wiki admin
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/12 11:23, Diego Elio Pettenò wrote: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. fetch-restricted files are to be considered critical here. Do we want to force the user to keep them twice? So an additional location which is not a cache? Of course PORTAGE_RO_DISTDIRS and friends are nice here, but they are not part of a default setup. justin signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
On 12/17/12 11:30 AM, Michał Górny wrote: On Mon, 17 Dec 2012 11:23:00 +0100 Diego Elio Pettenò flamee...@flameeyes.eu wrote: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. +1 on /var/cache. Agreed. Bonus points if we consider suggesting to move it on a dedicated file system ^^; lu
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/2012 12:06, justin wrote: fetch-restricted files are to be considered critical here. Do we want to force the user to keep them twice? So an additional location which is not a cache? Of course PORTAGE_RO_DISTDIRS and friends are nice here, but they are not part of a default setup. I would still think they are cache. I can re-download Oracle's JRE binaries; Portage's copy is a cache because I don't need to back it up. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/ signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, Dec 17, 2012 at 11:23 AM, Diego Elio Pettenò flamee...@flameeyes.eu wrote: I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. +1. Cheers, Dirkjan
Re: [gentoo-dev] Moving our/portage stuff to var
+1 /var/cache Derek Dai On Mon, Dec 17, 2012 at 6:30 PM, Michał Górny mgo...@gentoo.org wrote: On Mon, 17 Dec 2012 11:23:00 +0100 Diego Elio Pettenò flamee...@flameeyes.eu wrote: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. +1 on /var/cache. -- Best regards, Michał Górny
Re: [gentoo-dev] Moving our/portage stuff to var
On 17 December 2012 10:30, Michał Górny mgo...@gentoo.org wrote: On Mon, 17 Dec 2012 11:23:00 +0100 Diego Elio Pettenò flamee...@flameeyes.eu wrote: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. +1 on /var/cache. -- Best regards, Michał Górny +1 sounds good to me. -- Regards, Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
Re: [gentoo-dev] Moving our/portage stuff to var
On Monday 17 December 2012 11:19:20 Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). Finally! And, while we are at it, lets more distfiles out of portage. That is the only (prominent) dir inside that has drastically different storage requirements. Having portage on a separate partition requires now changing defaults, bind mounts or symlinks. It is better to have it done right from the onset then to workaround every time. Oh, and +1 on /var/cache/portage for the location. George
Re: [gentoo-dev] Moving our/portage stuff to var
On 17.12.2012 11:23, Diego Elio Pettenò wrote:
On 17/12/2012 11:19, Tomáš Chvátal wrote:
I've always myself override these defaults in make.conf to point for
/var/portage/ (not /var/lib because I never bothered enough how to
make world and config files to be put elsewhere :P).
I would say let's work on that so that portage can keep them there.
Although I'm more for /var/cache/portage myself, as both distfiles and
tree can be re-generated.
With this change distfiles and packages could be moved to a directory
which is not a subdir of portage. Something like
/var/cache/{portage,distfiles,packages}
or
/var/cache/portage/{tree,distfiles,packages}
since the file types and storage requirements are so different. At least
I prefer not to have too many filesystems mounted inside each other.
Philipp
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/12 12:17, Diego Elio Pettenò wrote: On 17/12/2012 12:06, justin wrote: fetch-restricted files are to be considered critical here. Do we want to force the user to keep them twice? So an additional location which is not a cache? Of course PORTAGE_RO_DISTDIRS and friends are nice here, but they are not part of a default setup. I would still think they are cache. I can re-download Oracle's JRE binaries; Portage's copy is a cache because I don't need to back it up. I am more thinking about packages which are not as easy accessible as JRE. There are a couple sci packages which are distributed on request by mail other inconvenient methods. Sometimes even not by your own, but by your PI or other seniors. And even sometimes upstreams doesn't distribute them anymore at all. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, 17 Dec 2012, Michał Górny wrote: On Mon, 17 Dec 2012 11:23:00 +0100 Diego Elio Pettenò flamee...@flameeyes.eu wrote: I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. +1 on /var/cache. If we change the location, can we then move distfiles to some place outside of the tree? Something like: /var/cache/portage /var/cache/distfiles Rationale for this is that the tree with many small files and distfiles (which tend to be large) have very different parameters for filesystem optimisation, when one of them (or both) are kept on a separate partition. Ulrich
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/2012 13:42, Ulrich Mueller wrote: If we change the location, can we then move distfiles to some place outside of the tree? Something like: /var/cache/portage /var/cache/distfiles What I do on my systems is /var/cache/portage/tree /var/cache/portage/distfiles -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/2012 13:39, justin wrote: I am more thinking about packages which are not as easy accessible as JRE. There are a couple sci packages which are distributed on request by mail other inconvenient methods. Sometimes even not by your own, but by your PI or other seniors. And even sometimes upstreams doesn't distribute them anymore at all. In which case you should keep them somewhere safer anyway, which is what I do for that kind of files. Remember that eclean acts on distfiles as well. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/ signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
Am Montag, den 17.12.2012, 11:19 +0100 schrieb Tomáš Chvátal: Currently we put portage into /usr/portage and all related stuff is to be in the subfolders there (distfiles, binpkg). I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I always move the stuff as well: * /var/cache/distfiles * /var/cache/packages ... may not be the best choice since they can't always be regenerated * /var/db/repositories/portage * /var/db/repositories/... (other portage repositories) * /var/db/paludis/repositories/... (for paludis-specific repositories, like layman) Tiziano
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, 17 Dec 2012 11:19:20 +0100 Tomáš Chvátal tomas.chva...@gmail.com wrote: The only reason why we have this currently in usr is that bsd ports put their stuff in there and I suppose Daniel just did the same. +1 on /var/cache. Agreed. Bonus points if we consider suggesting to move it on a dedicated file system ^^; /var sounds right but if /usr is still huge it may annoy some (like apt can) with smaller drives who now need lots of free space for new programs in both /usr and /var. Of course there is LVM. On OpenBSD the Auto partition map suggests /usr/ports /usr/src as seperate partitions as long as you have a fair amount of space. It possibly even suggests a seperate obj partition. The benefit being you can mkfs/newfs much quicker than deleting many many files. Security (DAC permission avoidance) and nuking more than what you wanted obviously needs consideration for that kind of function. So it's probably a user exercise?
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/2012 14:40, Kevin Chadwick wrote: /var sounds right but if /usr is still huge it may annoy some (like apt can) with smaller drives who now need lots of free space for new programs in both /usr and /var. Of course there is LVM. Changing our defaults is unlikely to force users to change their settings. On OpenBSD the Auto partition map suggests /usr/ports /usr/src as seperate partitions as long as you have a fair amount of space. It possibly even suggests a seperate obj partition. The benefit being you can mkfs/newfs much quicker than deleting many many files. Security (DAC permission avoidance) and nuking more than what you wanted obviously needs consideration for that kind of function. Honestly I would never take what OpenBSD does to face value. But in general this is a call for users — myself I have been keeping them split on the tinderbox host but merged into the rootfs for the laptops. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, Dec 17, 2012 at 8:40 AM, Kevin Chadwick ma1l1i...@yahoo.co.uk wrote: So it's probably a user exercise? It already is a user exercise. A stage3 doesn't even contain the /usr/portage directory - you manually create it per the handbook (or more likely let tar/etc do it for you. I also would like to see distfiles moved. Ideally the package tree should be a perfect copy of what is on the rsync mirrors. It seems a bit odd to stick other stuff in there, which needs special treatment as a result. To the extent that this isn't already supported, portage should simply let you set the location in make.conf. I'd also suggest at least considering how paludis handles this. They just have a directory containing config file per repository, with a priority setting. The portage tree is just another overlay, which is a good way to handle it. The sync mechanism handles the main tree identically to overlays as a result, though you can specify what to sync. Rich
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/2012 14:49, Rich Freeman wrote: I'd also suggest at least considering how paludis handles this. They just have a directory containing config file per repository, with a priority setting. The portage tree is just another overlay, which is a good way to handle it. The sync mechanism handles the main tree identically to overlays as a result, though you can specify what to sync. Let's not conflate two completely different changes with two completely different work required to deal with them. Changing our default locations, and the documentation is quick. Changing the way the syncing/handling is done is a nightmare. If we conflate the issue, we can stop discussing as we're never ever going to go anywhere. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/
Re: [gentoo-dev] Moving our/portage stuff to var
Am Montag, 17. Dezember 2012, 11:23:00 schrieb Diego Elio Pettenò: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. What about setups where portage tree is mounted via NFS to reduce traffic and disk space? FHS states[1] that /var/cache is *locally* generated. Which aould not be the case for such setups... I prefer /var as well, but I am not such if /var/cache would be the right place... [1] From FHS 2.3: /var/cache is intended for cached data from applications. Such data is locally generated as a result of time-consuming I/O or calculation. -Marc -- 0x35A64134 - 8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134 signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/2012 15:49, Marc Schiffbauer wrote: What about setups where portage tree is mounted via NFS to reduce traffic and disk space? Since nothing in Gentoo and/or other distributions _enforces_ FHS, you're allowed to do as you prefer FHS states[1] that /var/cache is *locally* generated. Which aould not be the case for such setups... Again, you can do as you prefer. Do you wish to violate FHS to just keep in the usual place? You can. You want to move it somewhere else to adhere to FHS? You can. I prefer /var as well, but I am not such if /var/cache would be the right place... Any other suggestions on where to place it? And please don't say /var/lib because that would usually be backed up. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/ signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
Am Montag, 17. Dezember 2012, 15:56:11 schrieb Diego Elio Pettenò: On 17/12/2012 15:49, Marc Schiffbauer wrote: What about setups where portage tree is mounted via NFS to reduce traffic and disk space? Since nothing in Gentoo and/or other distributions _enforces_ FHS, you're allowed to do as you prefer FHS states[1] that /var/cache is *locally* generated. Which aould not be the case for such setups... Again, you can do as you prefer. Do you wish to violate FHS to just keep in the usual place? You can. You want to move it somewhere else to adhere to FHS? You can. I prefer /var as well, but I am not such if /var/cache would be the right place... Any other suggestions on where to place it? And please don't say /var/lib because that would usually be backed up. FHS also states: [...] Other portions may be shared [between systems], notably /var/mail, /var/cache/man, /var/cache/fonts, and /var/spool/news. So I think this might indeed be interpreted like that /var/cache/portage would be perfectly ok. Another place I could imagine is /var/portage because of its fundamental importance in gentoo. FHS about that: Applications must generally not add directories to the top level of /var. Such directories should only be added if they have some system- wide implication, and in consultation with the FHS mailing list. I think this would also be ok because portage can be counted as system-wide implication ... -Marc -- 0x35A64134 - 8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134 signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Moving our/portage stuff to var
On 12/17/2012 05:30 AM, Michał Górny wrote: On Mon, 17 Dec 2012 11:23:00 +0100 Diego Elio Pettenòflamee...@flameeyes.eu wrote: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. +1 on /var/cache. +1 on the idea of moving portage and +1 on the idea of /var/cache -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 8040 5A4D 8709 21B1 1A88 33CE 979C AF40 D045 5535 GnuPG ID : D0455535
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, 2012-12-17 at 11:23 +0100, Diego Elio Pettenò wrote: On 17/12/2012 11:19, Tomáš Chvátal wrote: I've always myself override these defaults in make.conf to point for /var/portage/ (not /var/lib because I never bothered enough how to make world and config files to be put elsewhere :P). I would say let's work on that so that portage can keep them there. Although I'm more for /var/cache/portage myself, as both distfiles and tree can be re-generated. I've been pushing for the portage tree to move somewhere in /var for awhile. Also I think the tree and layman overlays should be under the same master directory, then portage, pkgcore,... could just quickly scan the master directory sub directories to auto-add the valid overlays there. No need to add them to PORTDIR_OVERLAY something like /var/cache/repositories/ /var/cache/repositories/gentoo /var/cache/repositories/x11== overlay /var/cache/repositories/local== overlay /var/cache/repositories/distfiles== move it out of the tree dir. /var/cache/repositories/packages== move it out of the tree dir. -- Brian Dolbec dol...@gentoo.org signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, 2012-12-17 at 13:47 +0100, Diego Elio Pettenò wrote: On 17/12/2012 13:39, justin wrote: I am more thinking about packages which are not as easy accessible as JRE. There are a couple sci packages which are distributed on request by mail other inconvenient methods. Sometimes even not by your own, but by your PI or other seniors. And even sometimes upstreams doesn't distribute them anymore at all. In which case you should keep them somewhere safer anyway, which is what I do for that kind of files. Remember that eclean acts on distfiles as well. which is why eclean has a config file where you can add files/pkgs, patterns to exclude from being cleaned. -- Brian Dolbec dol...@gentoo.org signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, 2012-12-17 at 15:56 +0100, Diego Elio Pettenò wrote: On 17/12/2012 15:49, Marc Schiffbauer wrote: What about setups where portage tree is mounted via NFS to reduce traffic and disk space? Since nothing in Gentoo and/or other distributions _enforces_ FHS, you're allowed to do as you prefer FHS states[1] that /var/cache is *locally* generated. Which aould not be the case for such setups... Again, you can do as you prefer. Do you wish to violate FHS to just keep in the usual place? You can. You want to move it somewhere else to adhere to FHS? You can. I prefer /var as well, but I am not such if /var/cache would be the right place... Any other suggestions on where to place it? And please don't say /var/lib because that would usually be backed up. then /var/repositories/ similar to my previous reply. It is very clear by the name what it's purpose is. Also name the portage tree dir gentoo like it's repo_name and all but one of the layman overlays available to install. -- Brian Dolbec dol...@gentoo.org signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Moving our/portage stuff to var
On 17/12/2012 16:51, Brian Dolbec wrote: then /var/repositories/ similar to my previous reply. It is very clear by the name what it's purpose is. Also name the portage tree dir gentoo like it's repo_name and all but one of the layman overlays available to install. Erm, why should we invent something new on top of var which is something that about everywhere it's suggested NOT to do? Using /var/lib clearly spells back this up yourself. Using /var/cache clearly spells can be lost without consequences. Using /var/tmp clearly spells things will be deleted. /var/repositories sounds just like NIH, seriously. Somebody already proposed /var/db — that probably makes more sense if you want to go that route, although I wouldn't put distfiles or packages there. -- Diego Elio Pettenò — Flameeyes flamee...@flameeyes.eu — http://blog.flameeyes.eu/ signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, 2012-12-17 at 15:02 +0100, Diego Elio Pettenò wrote: On 17/12/2012 14:49, Rich Freeman wrote: I'd also suggest at least considering how paludis handles this. They just have a directory containing config file per repository, with a priority setting. The portage tree is just another overlay, which is a good way to handle it. The sync mechanism handles the main tree identically to overlays as a result, though you can specify what to sync. Let's not conflate two completely different changes with two completely different work required to deal with them. Changing our default locations, and the documentation is quick. Changing the way the syncing/handling is done is a nightmare. If we conflate the issue, we can stop discussing as we're never ever going to go anywhere. I agree. But for the purpose of answering Rich, layman is also capable of syncing the portage tree, although, not using the round robin and other user sync variations portage is capable of. Once I finish the python 3 compatibility code changes. I will re-visit getting layman's api use into portage, pkgcore for them to use to sync the overlays from within the package manager. I have already demonstrated how easy it is for portage to run layman's api. An in tree example is app-portage/esearch which adding the -l parameter will use the layman api if available or fall back to running layman in a subprocess to sync the overlays as well as the portage tree. -- Brian Dolbec dol...@gentoo.org signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Moving our/portage stuff to var
On Mon, 17 Dec 2012 15:56:11 +0100 Diego Elio Pettenò flamee...@flameeyes.eu wrote: Any other suggestions on where to place it? And please don't say /var/lib because that would usually be backed up. /var/db -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Moving our/portage stuff to var
On 12/17/12 2:19 AM, Tomáš Chvátal wrote: With respect to reality how stuff is done in the linux land all the variable data should be in /var so we should adjust and move it in there too. What would you think? Fully seconded. +1 to /var/cache/portage and having distfiles outside of the portage tree directory. Paweł signature.asc Description: OpenPGP digital signature
