Re: [gentoo-portage-dev] [PATCH] man/make.conf.5: Mention eclean in PKGDIR section

2020-04-26 Thread Zac Medico
On 4/26/20 9:54 AM, Matt Turner wrote:
> And remove obsolete text from binpkg-multi-instance. I believe this was
> obsolete even before my gentoolkit patches that add the --changed-deps
> flag to eclean packages.
> 
> Signed-off-by: Matt Turner 
> ---
>  man/make.conf.5 | 7 +++
>  1 file changed, 3 insertions(+), 4 deletions(-)
> 
> diff --git a/man/make.conf.5 b/man/make.conf.5
> index f82fed65a..a0aa5f129 100644
> --- a/man/make.conf.5
> +++ b/man/make.conf.5
> @@ -310,10 +310,6 @@ running older portage, since the file format is 
> identical, the
>  per\-package PATH attribute in the 'Packages' index directs them to
>  download the file from the correct URI, and they automatically use
>  BUILD_TIME metadata to select the latest builds.
> -
> -There is currently no automated way to prune old builds from PKGDIR,
> -although it is possible to remove packages manually, and then run
> -\(aqemaint \-\-fix binhost' to update the ${PKGDIR}/Packages index.

The emaint --fix binhost part could still be a useful hint in case
someone wants to delete something manually.

>  .TP
>  .B buildpkg
>  Binary packages will be created for all packages that are merged. Also see
> @@ -838,6 +834,9 @@ to it's category. However, for backward compatibility 
> with the layout
>  used by older versions of portage, if the \fI${PKGDIR}/All\fR directory
>  exists then all packages will be stored inside of it and symlinks to
>  the packages will be created in the category subdirectories.
> +
> +If you would like to selectively prune obsolete files from this directory, 
> see
> +\fBeclean\fR(1) from the gentoolkit package.
>  .br
>  Defaults to /var/cache/binpkgs.
>  .TP
> 
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Michael Orlitzky
On 4/26/20 3:25 PM, Ulrich Mueller wrote:
>> On Sun, 26 Apr 2020, Michael Orlitzky wrote:
> 
>> Fuel for the fire:
> 
>>  * https://www.nongnu.org/lzip/lzip_benchmark.html
>>  * https://www.nongnu.org/lzip/xz_inadequate.html
> 
> Yep. That's why lzip is the dominant compression format now, and nobody
> is using xz any more.
> 

If you believed that argument, you would be using Windows =P

Dude has GRAPHS, it must be true.



Re: [gentoo-portage-dev] [PATCH] make.globals: add default BINPKG_COMPRESS setting (bug 715108)

2020-04-26 Thread Brian Dolbec
On Sun, 26 Apr 2020 14:57:32 -0700
Zac Medico  wrote:

> The ebuild will have a default enabled USE=zstd which changes the
> default to zstd here.
> 
> Bug: https://bugs.gentoo.org/715108
> Bug: https://bugs.gentoo.org/719456
> Signed-off-by: Zac Medico 
> ---
>  cnf/make.globals | 5 -
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/cnf/make.globals b/cnf/make.globals
> index 139e1ce97..4a59dbe3c 100644
> --- a/cnf/make.globals
> +++ b/cnf/make.globals
> @@ -1,4 +1,4 @@
> -# Copyright 1999-2019 Gentoo Authors
> +# Copyright 1999-2020 Gentoo Authors
>  # Distributed under the terms of the GNU General Public License v2
>  # System-wide defaults for the Portage system
>  
> @@ -34,6 +34,9 @@ RPMDIR="/var/cache/rpm"
>  # Temporary build directory
>  PORTAGE_TMPDIR="/var/tmp"
>  
> +# The compression used for binary packages. Defaults to zstd when
> USE=zstd is enabled. +BINPKG_COMPRESS="bzip2"
> +
>  # Fetching command (3 tries, passive ftp for firewall compatibility)
>  FETCHCOMMAND="wget -t 3 -T 60 --passive-ftp -O
> \"\${DISTDIR}/\${FILE}\" \"\${URI}\"" RESUMECOMMAND="wget -c -t 3 -T
> 60 --passive-ftp -O \"\${DISTDIR}/\${FILE}\" \"\${URI}\""

works for me :)



[gentoo-portage-dev] [PATCH] make.globals: add default BINPKG_COMPRESS setting (bug 715108)

2020-04-26 Thread Zac Medico
The ebuild will have a default enabled USE=zstd which changes the
default to zstd here.

Bug: https://bugs.gentoo.org/715108
Bug: https://bugs.gentoo.org/719456
Signed-off-by: Zac Medico 
---
 cnf/make.globals | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/cnf/make.globals b/cnf/make.globals
index 139e1ce97..4a59dbe3c 100644
--- a/cnf/make.globals
+++ b/cnf/make.globals
@@ -1,4 +1,4 @@
-# Copyright 1999-2019 Gentoo Authors
+# Copyright 1999-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 # System-wide defaults for the Portage system
 
@@ -34,6 +34,9 @@ RPMDIR="/var/cache/rpm"
 # Temporary build directory
 PORTAGE_TMPDIR="/var/tmp"
 
+# The compression used for binary packages. Defaults to zstd when USE=zstd is 
enabled.
+BINPKG_COMPRESS="bzip2"
+
 # Fetching command (3 tries, passive ftp for firewall compatibility)
 FETCHCOMMAND="wget -t 3 -T 60 --passive-ftp -O \"\${DISTDIR}/\${FILE}\" 
\"\${URI}\""
 RESUMECOMMAND="wget -c -t 3 -T 60 --passive-ftp -O \"\${DISTDIR}/\${FILE}\" 
\"\${URI}\""
-- 
2.25.3




Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Zac Medico
On 4/26/20 1:09 PM, Matt Turner wrote:
> On Sun, Apr 26, 2020 at 12:29 PM Michał Górny  wrote:
>>
>> On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
>>> Bug: https://bugs.gentoo.org/715108
>>> Signed-off-by: Matt Turner 
>>> ---
>>> Strawman patch. Bikeshed away.
>>>
>>
>> xz is generally slow and doesn't do parallel good.  If we want to change
>> this, we should go for something cool like zstd that scales better.
> 
> That works for me.
> 

Let's make the default conditional on zstd USE flag as suggested in bug
719456. It will be similar to how the ebuild enables FEATURES=xattr for
USE=xattr.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Matt Turner
On Sun, Apr 26, 2020 at 12:29 PM Michał Górny  wrote:
>
> On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
> > Bug: https://bugs.gentoo.org/715108
> > Signed-off-by: Matt Turner 
> > ---
> > Strawman patch. Bikeshed away.
> >
>
> xz is generally slow and doesn't do parallel good.  If we want to change
> this, we should go for something cool like zstd that scales better.

That works for me.



Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Fabian Groffen
On 26-04-2020 21:29:42 +0200, Michał Górny wrote:
> On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
> > Bug: https://bugs.gentoo.org/715108
> > Signed-off-by: Matt Turner 
> > ---
> > Strawman patch. Bikeshed away.
> > 
> 
> xz is generally slow and doesn't do parallel good.  If we want to change
> this, we should go for something cool like zstd that scales better.

I'd go for zstd too.  It seems to be the best of both worlds, good
compression at a good speed.

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Michał Górny
On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
> Bug: https://bugs.gentoo.org/715108
> Signed-off-by: Matt Turner 
> ---
> Strawman patch. Bikeshed away.
> 

xz is generally slow and doesn't do parallel good.  If we want to change
this, we should go for something cool like zstd that scales better.

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Ulrich Mueller
> On Sun, 26 Apr 2020, Michael Orlitzky wrote:

> Fuel for the fire:

>  * https://www.nongnu.org/lzip/lzip_benchmark.html
>  * https://www.nongnu.org/lzip/xz_inadequate.html

Yep. That's why lzip is the dominant compression format now, and nobody
is using xz any more.

SCNR,
Ulrich


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Michael Orlitzky
On 4/26/20 12:55 PM, Matt Turner wrote:
> Bug: https://bugs.gentoo.org/715108
> Signed-off-by: Matt Turner 
> ---
> Strawman patch. Bikeshed away.
> 

Fuel for the fire:

 * https://www.nongnu.org/lzip/lzip_benchmark.html
 * https://www.nongnu.org/lzip/xz_inadequate.html



[gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Matt Turner
Bug: https://bugs.gentoo.org/715108
Signed-off-by: Matt Turner 
---
Strawman patch. Bikeshed away.

 bin/quickpkg   | 2 +-
 lib/portage/package/ebuild/doebuild.py | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/bin/quickpkg b/bin/quickpkg
index df8c1a8e8..1feb01fa1 100755
--- a/bin/quickpkg
+++ b/bin/quickpkg
@@ -115,7 +115,7 @@ def quickpkg_atom(options, infos, arg, eout):
binpkg_tmpfile = os.path.join(bintree.pkgdir,
cpv + ".tbz2." + str(os.getpid()))
ensure_dirs(os.path.dirname(binpkg_tmpfile))
-   binpkg_compression = settings.get("BINPKG_COMPRESS", 
"bzip2")
+   binpkg_compression = settings.get("BINPKG_COMPRESS", 
"xz")
try:
compression = _compressors[binpkg_compression]
except KeyError as e:
diff --git a/lib/portage/package/ebuild/doebuild.py 
b/lib/portage/package/ebuild/doebuild.py
index 2bff94cb1..5020b46fe 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -539,7 +539,7 @@ def doebuild_environment(myebuild, mydo, myroot=None, 
settings=None,
mysettings["KV"] = ""
mysettings.backup_changes("KV")
 
-   binpkg_compression = mysettings.get("BINPKG_COMPRESS", "bzip2")
+   binpkg_compression = mysettings.get("BINPKG_COMPRESS", "xz")
try:
compression = _compressors[binpkg_compression]
except KeyError as e:
-- 
2.25.3




[gentoo-portage-dev] [PATCH] man/make.conf.5: Mention eclean in PKGDIR section

2020-04-26 Thread Matt Turner
And remove obsolete text from binpkg-multi-instance. I believe this was
obsolete even before my gentoolkit patches that add the --changed-deps
flag to eclean packages.

Signed-off-by: Matt Turner 
---
 man/make.conf.5 | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/man/make.conf.5 b/man/make.conf.5
index f82fed65a..a0aa5f129 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -310,10 +310,6 @@ running older portage, since the file format is identical, 
the
 per\-package PATH attribute in the 'Packages' index directs them to
 download the file from the correct URI, and they automatically use
 BUILD_TIME metadata to select the latest builds.
-
-There is currently no automated way to prune old builds from PKGDIR,
-although it is possible to remove packages manually, and then run
-\(aqemaint \-\-fix binhost' to update the ${PKGDIR}/Packages index.
 .TP
 .B buildpkg
 Binary packages will be created for all packages that are merged. Also see
@@ -838,6 +834,9 @@ to it's category. However, for backward compatibility with 
the layout
 used by older versions of portage, if the \fI${PKGDIR}/All\fR directory
 exists then all packages will be stored inside of it and symlinks to
 the packages will be created in the category subdirectories.
+
+If you would like to selectively prune obsolete files from this directory, see
+\fBeclean\fR(1) from the gentoolkit package.
 .br
 Defaults to /var/cache/binpkgs.
 .TP
-- 
2.25.3




Re: [gentoo-portage-dev] Need backup mentor for FUSE-based sandbox project

2020-04-26 Thread Michał Górny
On Sat, 2020-04-25 at 21:40 -0700, Alec Warner wrote:
> On Thu, Apr 23, 2020 at 12:09 PM Michał Górny  wrote:
> 
> > Hi, everyone.
> > 
> > It seems that we *urgently* (read: in 6 days) need to find backup
> > mentors for this year's GSoC projects.  I'm mentoring the project to
> > develop a FUSE-based sandbox alternative that's going to work reliably
> > with more packages than the LD_PRELOAD hack [1].
> > 
> > Would anyone volunteer to be the backup maintainer for this?
> > 
> > [1]
> > https://summerofcode.withgoogle.com/dashboard/organization/5749981018849280/proposal/5572241732927488/
> 
> I get a 403 for that.
> 
> https://wiki.gentoo.org/wiki/Google_Summer_of_Code/2020/Ideas#FUSE-powered_sandbox
> is
> your writeup; I assume the above is supposed to be a link to the student's
> proposal?

Yes and yes.  I suppose idea of stable links didn't reach Google yet. 
However, I should have noted that we have a backup mentor now but
there's no harm in having a second backup ;-).

> 
> 
> > 
> > --
> > Best regards,
> > Michał Górny
> > 
> > 

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part