Re: [gentoo-dev] [News item review] Portage rsync tree verification (v4)

2018-01-28 Thread Michał Górny
Hopefully the final version. --- Title: Portage rsync tree verification Author: Michał Górny Posted: 2018-01-xx Revision: 1 News-Item-Format: 2.0 Display-If-Installed: sys-apps/portage Starting with sys-apps/portage-2.3.22, Portage will verify the Gentoo repository after rsync by default. The n

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Michał Górny
W dniu nie, 28.01.2018 o godzinie 15∶01 +0800, użytkownik Jason Zaman napisał: > On Sat, Jan 27, 2018 at 12:24:39AM +0100, Michał Górny wrote: > > Migrating mirrors to the hashed structure > > - > > The hard link solution allows us to save space on the master

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Ulrich Mueller
> On Sat, 27 Jan 2018, Michał Górny wrote: > This specification currently defines one section: ``[structure]``. > This section defines one or more repository structure definitions > using sequential integer keys. The definition keyed as ``0`` > is the most preferred structure. The package ma

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Michał Górny
W dniu nie, 28.01.2018 o godzinie 11∶14 +0100, użytkownik Ulrich Mueller napisał: > > > > > > On Sat, 27 Jan 2018, Michał Górny wrote: > > This specification currently defines one section: ``[structure]``. > > This section defines one or more repository structure definitions > > using sequential in

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Ulrich Mueller
> On Sun, 28 Jan 2018, Michał Górny wrote: >> > This specification currently defines one section: ``[structure]``. >> > This section defines one or more repository structure definitions >> > using sequential integer keys. The definition keyed as ``0`` >> > is the most preferred structure. Th

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Michał Górny
W dniu nie, 28.01.2018 o godzinie 11∶22 +0100, użytkownik Ulrich Mueller napisał: > > > > > > On Sun, 28 Jan 2018, Michał Górny wrote: > > > > This specification currently defines one section: ``[structure]``. > > > > This section defines one or more repository structure definitions > > > > using s

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Ulrich Mueller
> On Sun, 28 Jan 2018, Michał Górny wrote: > How about this then: > | This specification currently defines one section: ``[structure]``. > | This section defines one or more repository structure definitions > | using non-negative sequential integer keys. The definition with > | the ``0`` key

[gentoo-dev] Re: [News item review] Portage rsync tree verification (v4)

2018-01-28 Thread Duncan
Michał Górny posted on Sun, 28 Jan 2018 09:58:37 +0100 as excerpted: > The new verification is intended for users who are syncing via rsync. > Verification mechanisms for other methods of sync will be provided in > the future. > > This does not affect users syncing using git and other methods. >

Re: [gentoo-dev] [News item review] Portage rsync tree verification (v4)

2018-01-28 Thread Andrew Barchuk
Hi everyone, > three possible solutions for splitting distfiles were listed: > > a. using initial portion of filename, > > b. using initial portion of file hash, > > c. using initial portion of filename hash. > > The significant advantage of the filename option was simplicity. With > that sol

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Andrew Barchuk
[my apologies for posting the message to a wrong thread before] Hi everyone, > three possible solutions for splitting distfiles were listed: > > a. using initial portion of filename, > > b. using initial portion of file hash, > > c. using initial portion of filename hash. > > The significant adva

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Gordon Pettey
On Sun, Jan 28, 2018 at 2:43 PM, Andrew Barchuk wrote: > There's another option to use character ranges for each directory > computed in a way to have the files distributed evenly. One way to do > that is to use filename prefix of dynamic length so that each range > holds the same number of files.

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Andrew Barchuk
> To the contrary, that would not remain balanced, because your > boundaries are entirely dependent on exactly what is in the tree at > the moment you run your script. Now the package manager has to perform > directory listing, sort and find the file name that's closest, open > that directory, find

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Gordon Pettey
On Sun, Jan 28, 2018 at 4:00 PM, Andrew Barchuk wrote: > I don't see a reason to organize distfiles in a > multi-level hierarchy: e.g. if the goal is to keep no more than 1000 > files in a folder than the limit of single level hierarchy is a million > which is more than enough for foreseeable futu

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Zac Medico
On 01/28/2018 02:00 PM, Andrew Barchuk wrote: >> To the contrary, that would not remain balanced, because your >> boundaries are entirely dependent on exactly what is in the tree at >> the moment you run your script. Now the package manager has to perform >> directory listing, sort and find the fil

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Andrew Barchuk
> In order to use that for distfiles mirrors, such that clients could know > where to fetch the files from, you'd need the mirror's http server to > redirect the request to the appropriate location (since the location > would not be predictable from the client side). That's not necessary: the cli

[gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2018-01-28 23:59 UTC

2018-01-28 Thread Robin H. Johnson
The attached list notes all of the packages that were added or removed from the tree, for the week ending 2018-01-28 23:59 UTC. Removals: app-crypt/nistp224 20180128-00:45 zlogeneb51aa320750 app-doc/linux-gazette 20180127-22:57 zlogeneeb20a9691bf app

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Michał Górny
W dniu nie, 28.01.2018 o godzinie 21∶43 +0100, użytkownik Andrew Barchuk napisał: > [my apologies for posting the message to a wrong thread before] > > Hi everyone, > > > three possible solutions for splitting distfiles were listed: > > > > a. using initial portion of filename, > > > > b. using

Re: [gentoo-dev] [News item review] Portage rsync tree verification (v4)

2018-01-28 Thread Robin H. Johnson
On Sun, Jan 28, 2018 at 09:30:31PM +0100, Andrew Barchuk wrote: > Hi everyone, > > > three possible solutions for splitting distfiles were listed: > There's another option to use character ranges for each directory > computed in a way to have the files distributed evenly. One way to do > that is t

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-28 Thread Robin H. Johnson
On Sun, Jan 28, 2018 at 03:01:11PM +0800, Jason Zaman wrote: > Another thing im wondering is if we can just use the same dir layout as > the packages themselves. that would fix texlive since it has a whole lot > of separate packages. eg /usr/portage/distfiles/app-cat/pkg/pkg-1.0.tgz Texlive is wors