Re: [gentoo-portage-dev] [PATCH] emirrordist: Report all stat() errors instead of first one

2019-10-21 Thread Zac Medico
On 10/20/19 2:54 AM, Michał Górny wrote: > When DeletionIterator fails, report all stat() errors. Reporting > just the first one results in confusing logs, suggesting that one > of the location did not exist but the other existed and was removed. > --- >

[gentoo-portage-dev] [PATCH v2] emirrordist: Clean dangling symlinks up

2019-10-21 Thread Michał Górny
Bug: https://bugs.gentoo.org/697906 Signed-off-by: Michał Górny --- lib/portage/_emirrordist/DeletionIterator.py | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/lib/portage/_emirrordist/DeletionIterator.py b/lib/portage/_emirrordist/DeletionIterator.py index

Re: [gentoo-portage-dev] [PATCH 1/2] fetch: Use real os.walk() to avoid unicode issues with Portage

2019-10-21 Thread Zac Medico
On 10/21/19 2:16 AM, Michał Górny wrote: > On Mon, 2019-10-21 at 02:10 -0700, Zac Medico wrote: >> On 10/21/19 1:43 AM, Michał Górny wrote: >>> Use real os.walk() when getting filenames for FlatLayout. Unlike >>> the wrapped Portage module, it return str output for str path parameter, >>> so we

Re: [gentoo-portage-dev] [PATCH v2] emirrordist: Clean dangling symlinks up

2019-10-21 Thread Zac Medico
On 10/21/19 1:02 AM, Michał Górny wrote: > Bug: https://bugs.gentoo.org/697906 > Signed-off-by: Michał Górny > --- > lib/portage/_emirrordist/DeletionIterator.py | 11 --- > 1 file changed, 8 insertions(+), 3 deletions(-) > > diff --git a/lib/portage/_emirrordist/DeletionIterator.py >

Re: [gentoo-portage-dev] [PATCH 1/2] fetch: Use real os.walk() to avoid unicode issues with Portage

2019-10-21 Thread Zac Medico
On 10/21/19 1:43 AM, Michał Górny wrote: > Use real os.walk() when getting filenames for FlatLayout. Unlike > the wrapped Portage module, it return str output for str path parameter, > so we don't have to recode it back and forth. > > Signed-off-by: Michał Górny > --- >

Re: [gentoo-portage-dev] [PATCH 1/2] fetch: Use real os.walk() to avoid unicode issues with Portage

2019-10-21 Thread Michał Górny
On Mon, 2019-10-21 at 02:10 -0700, Zac Medico wrote: > On 10/21/19 1:43 AM, Michał Górny wrote: > > Use real os.walk() when getting filenames for FlatLayout. Unlike > > the wrapped Portage module, it return str output for str path parameter, > > so we don't have to recode it back and forth. > >

Re: [gentoo-portage-dev] [PATCH] emirrordist: Clean dangling symlinks up

2019-10-21 Thread Michał Górny
On Mon, 2019-10-21 at 00:44 -0700, Zac Medico wrote: > On 10/20/19 3:23 AM, Michał Górny wrote: > > Bug: https://bugs.gentoo.org/697906 > > Signed-off-by: Michał Górny > > --- > > lib/portage/_emirrordist/DeletionIterator.py | 21 +--- > > 1 file changed, 18 insertions(+), 3

Re: [gentoo-portage-dev] [PATCH] emirrordist: Clean dangling symlinks up

2019-10-21 Thread Zac Medico
On 10/20/19 3:23 AM, Michał Górny wrote: > Bug: https://bugs.gentoo.org/697906 > Signed-off-by: Michał Górny > --- > lib/portage/_emirrordist/DeletionIterator.py | 21 +--- > 1 file changed, 18 insertions(+), 3 deletions(-) > > diff --git

Re: [gentoo-portage-dev] [PATCH 2/2] emirrordist: Pass path from DeletionIterator to DeletionTask

2019-10-21 Thread Zac Medico
On 10/21/19 1:43 AM, Michał Górny wrote: > Since DeletionIterator needs to stat the distfile and therefore find > one working path for it, pass it to DeletionTask instead of recomputing > it there. This also fixes wrongly assuming that first layout will > always be correct. > > Bug:

Re: [gentoo-portage-dev] [PATCH] make.globals: Change FETCHCOMMAND_RSYNC to --copy-links

2019-10-21 Thread Ulrich Mueller
> On Mon, 21 Oct 2019, Michał Górny wrote: > This also removes most of the other options that are irrelevant or even > undesirable to distfile fetching, that is: > - '-r' since we always fetch a single file, so recursive operation is > unnecessary > - '-p', '-o', '-g' since we want to

[gentoo-portage-dev] [PATCH] make.globals: Change FETCHCOMMAND_RSYNC to --copy-links

2019-10-21 Thread Michał Górny
Change FETCHCOMMAND_RSYNC to use '-Lt' over '-a'. Notably, this replaces --links with --copy-links option, i.e. makes rsync copy underlying files when symlinks are met. This is important since we do not transfer symlink targets, therefore '-l' ends up creating dangling symlinks. This also

Re: [gentoo-portage-dev] [PATCH] make.globals: Change FETCHCOMMAND_RSYNC to --copy-links

2019-10-21 Thread Michał Górny
On Mon, 2019-10-21 at 17:25 +0200, Ulrich Mueller wrote: > > > > > > On Mon, 21 Oct 2019, Michał Górny wrote: > > This also removes most of the other options that are irrelevant or even > > undesirable to distfile fetching, that is: > > - '-r' since we always fetch a single file, so recursive

[gentoo-portage-dev] [PATCH 1/2] fetch: Use real os.walk() to avoid unicode issues with Portage

2019-10-21 Thread Michał Górny
Use real os.walk() when getting filenames for FlatLayout. Unlike the wrapped Portage module, it return str output for str path parameter, so we don't have to recode it back and forth. Signed-off-by: Michał Górny --- lib/portage/package/ebuild/fetch.py | 3 ++- 1 file changed, 2 insertions(+),

[gentoo-portage-dev] [PATCH 2/2] emirrordist: Pass path from DeletionIterator to DeletionTask

2019-10-21 Thread Michał Górny
Since DeletionIterator needs to stat the distfile and therefore find one working path for it, pass it to DeletionTask instead of recomputing it there. This also fixes wrongly assuming that first layout will always be correct. Bug: https://bugs.gentoo.org/697890 Signed-off-by: Michał Górny ---

Re: [gentoo-portage-dev] [PATCH] make.globals: Change FETCHCOMMAND_RSYNC to --copy-links

2019-10-21 Thread Zac Medico
On 10/21/19 8:06 AM, Michał Górny wrote: > Change FETCHCOMMAND_RSYNC to use '-Lt' over '-a'. Notably, this > replaces --links with --copy-links option, i.e. makes rsync copy > underlying files when symlinks are met. This is important since > we do not transfer symlink targets, therefore '-l'