Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-05-09 Thread Zac Medico
On 4/26/20 2:12 PM, Zac Medico wrote:
> On 4/26/20 1:09 PM, Matt Turner wrote:
>> On Sun, Apr 26, 2020 at 12:29 PM Michał Górny  wrote:
>>>
>>> On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
 Bug: https://bugs.gentoo.org/715108
 Signed-off-by: Matt Turner 
 ---
 Strawman patch. Bikeshed away.

>>>
>>> xz is generally slow and doesn't do parallel good.  If we want to change
>>> this, we should go for something cool like zstd that scales better.
>>
>> That works for me.
>>
> 
> Let's make the default conditional on zstd USE flag as suggested in bug
> 719456. It will be similar to how the ebuild enables FEATURES=xattr for
> USE=xattr.

In https://github.com/gentoo/gentoo/pull/15520 the issue was raised that
it's probably not a good idea to change this default via the zstd USE
flags, since changing the format of binary packages is kind of a big
deal (it affects consumers of any binary packages that you build).

I think it might be a good idea to make this new default apply only to
new installations, like we did when changing default locations for bug
378603.

If we're changing the default, then all architectures need keywords, so
I've opened https://bugs.gentoo.org/721960.
-- 
Thanks,
Zac



Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-27 Thread Michał Górny
On Sun, 2020-04-26 at 18:23 -0400, Michael Orlitzky wrote:
> On 4/26/20 3:25 PM, Ulrich Mueller wrote:
> > > > > > > On Sun, 26 Apr 2020, Michael Orlitzky wrote:
> > > Fuel for the fire:
> > >  * https://www.nongnu.org/lzip/lzip_benchmark.html
> > >  * https://www.nongnu.org/lzip/xz_inadequate.html
> > 
> > Yep. That's why lzip is the dominant compression format now, and nobody
> > is using xz any more.
> > 
> 
> If you believed that argument, you would be using Windows =P
> 
> Dude has GRAPHS, it must be true.
> 

It is true but nobody cares.  People managed to shove a lot of unused
code into the kernel and nobody cares.

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Michael Orlitzky
On 4/26/20 3:25 PM, Ulrich Mueller wrote:
>> On Sun, 26 Apr 2020, Michael Orlitzky wrote:
> 
>> Fuel for the fire:
> 
>>  * https://www.nongnu.org/lzip/lzip_benchmark.html
>>  * https://www.nongnu.org/lzip/xz_inadequate.html
> 
> Yep. That's why lzip is the dominant compression format now, and nobody
> is using xz any more.
> 

If you believed that argument, you would be using Windows =P

Dude has GRAPHS, it must be true.



Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Zac Medico
On 4/26/20 1:09 PM, Matt Turner wrote:
> On Sun, Apr 26, 2020 at 12:29 PM Michał Górny  wrote:
>>
>> On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
>>> Bug: https://bugs.gentoo.org/715108
>>> Signed-off-by: Matt Turner 
>>> ---
>>> Strawman patch. Bikeshed away.
>>>
>>
>> xz is generally slow and doesn't do parallel good.  If we want to change
>> this, we should go for something cool like zstd that scales better.
> 
> That works for me.
> 

Let's make the default conditional on zstd USE flag as suggested in bug
719456. It will be similar to how the ebuild enables FEATURES=xattr for
USE=xattr.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Matt Turner
On Sun, Apr 26, 2020 at 12:29 PM Michał Górny  wrote:
>
> On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
> > Bug: https://bugs.gentoo.org/715108
> > Signed-off-by: Matt Turner 
> > ---
> > Strawman patch. Bikeshed away.
> >
>
> xz is generally slow and doesn't do parallel good.  If we want to change
> this, we should go for something cool like zstd that scales better.

That works for me.



Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Fabian Groffen
On 26-04-2020 21:29:42 +0200, Michał Górny wrote:
> On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
> > Bug: https://bugs.gentoo.org/715108
> > Signed-off-by: Matt Turner 
> > ---
> > Strawman patch. Bikeshed away.
> > 
> 
> xz is generally slow and doesn't do parallel good.  If we want to change
> this, we should go for something cool like zstd that scales better.

I'd go for zstd too.  It seems to be the best of both worlds, good
compression at a good speed.

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Michał Górny
On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
> Bug: https://bugs.gentoo.org/715108
> Signed-off-by: Matt Turner 
> ---
> Strawman patch. Bikeshed away.
> 

xz is generally slow and doesn't do parallel good.  If we want to change
this, we should go for something cool like zstd that scales better.

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Ulrich Mueller
> On Sun, 26 Apr 2020, Michael Orlitzky wrote:

> Fuel for the fire:

>  * https://www.nongnu.org/lzip/lzip_benchmark.html
>  * https://www.nongnu.org/lzip/xz_inadequate.html

Yep. That's why lzip is the dominant compression format now, and nobody
is using xz any more.

SCNR,
Ulrich


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Michael Orlitzky
On 4/26/20 12:55 PM, Matt Turner wrote:
> Bug: https://bugs.gentoo.org/715108
> Signed-off-by: Matt Turner 
> ---
> Strawman patch. Bikeshed away.
> 

Fuel for the fire:

 * https://www.nongnu.org/lzip/lzip_benchmark.html
 * https://www.nongnu.org/lzip/xz_inadequate.html



[gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Matt Turner
Bug: https://bugs.gentoo.org/715108
Signed-off-by: Matt Turner 
---
Strawman patch. Bikeshed away.

 bin/quickpkg   | 2 +-
 lib/portage/package/ebuild/doebuild.py | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/bin/quickpkg b/bin/quickpkg
index df8c1a8e8..1feb01fa1 100755
--- a/bin/quickpkg
+++ b/bin/quickpkg
@@ -115,7 +115,7 @@ def quickpkg_atom(options, infos, arg, eout):
binpkg_tmpfile = os.path.join(bintree.pkgdir,
cpv + ".tbz2." + str(os.getpid()))
ensure_dirs(os.path.dirname(binpkg_tmpfile))
-   binpkg_compression = settings.get("BINPKG_COMPRESS", 
"bzip2")
+   binpkg_compression = settings.get("BINPKG_COMPRESS", 
"xz")
try:
compression = _compressors[binpkg_compression]
except KeyError as e:
diff --git a/lib/portage/package/ebuild/doebuild.py 
b/lib/portage/package/ebuild/doebuild.py
index 2bff94cb1..5020b46fe 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -539,7 +539,7 @@ def doebuild_environment(myebuild, mydo, myroot=None, 
settings=None,
mysettings["KV"] = ""
mysettings.backup_changes("KV")
 
-   binpkg_compression = mysettings.get("BINPKG_COMPRESS", "bzip2")
+   binpkg_compression = mysettings.get("BINPKG_COMPRESS", "xz")
try:
compression = _compressors[binpkg_compression]
except KeyError as e:
-- 
2.25.3