On 3/25/19 12:13 AM, Robin Broda via arch-dev-public wrote:
> Given that low-end systems can simply change the thread allocation to 1 or 2
> to slash the compressor memory usage as a trade-off on speed,
> i don't think that is a problem.
>
> New changeset:
> PKGEXT='.pkg.tar.zst'
>
On 3/24/19 8:38 PM, Evangelos Foutras via arch-dev-public wrote:
> On Mon, 25 Mar 2019 at 01:22, Jan Alexander Steffens via
> arch-dev-public wrote:
>>
>> On Sun, Mar 24, 2019 at 7:35 PM Robin Broda via arch-dev-public
>> wrote:
>>> The required changeset is, i think:
>>> PKGEXT='.pkg.tar.zst'
Hello again,
after archange and Baptiste mentioned that the numbers look a little odd, I
took some more time and re-ran the tests with additional parameters.
Most notably, this includes -T2 - to show behavior on lower-spec machines, and
it fixes the higher compression levels by appending
On Mon, 25 Mar 2019 at 02:47, Robin Broda via arch-dev-public
wrote:
> What's unclear to me is the difference between zstd -T0 and zstdmt, however.
zstdmt is an alias/shortcut for "zstd -T0".
> I do think that at -19+, memory usage becomes a bigger issue.
> The difference between -18 and -19 on
On 3/25/19 1:38 AM, Evangelos Foutras via arch-dev-public wrote:
> On Mon, 25 Mar 2019 at 01:22, Jan Alexander Steffens via
> arch-dev-public wrote:
>>
>> When we implement this, I would say we go with "zstd -c -T0 -" in
>> pacman's makepkg.conf and "zstd -C -T0 -18 -" in the configs shipped
>>
Given the suggestion of using -18-, I decided to calculate how much
bigger our packages would be with the numbers given:
cuda-10.0.130-2-x86_64.pkg.tar 58.5M 104.40%
gcc-8.2.1+20181127-1-x86_64.pkg.tar 2.8M108.30%
go-2:1.12.1-1-x86_64.pkg.tar
On Mon, 25 Mar 2019 at 01:22, Jan Alexander Steffens via
arch-dev-public wrote:
>
> On Sun, Mar 24, 2019 at 7:35 PM Robin Broda via arch-dev-public
> wrote:
> > The required changeset is, i think:
> > PKGEXT='.pkg.tar.zst'
> > COMPRESSZST=(zstd -c -T0 -18 -)
>
> When we implement this, I would
On 3/25/19 1:33 AM, Gaetan Bisson via arch-dev-public wrote:
> - Prepare a static build of libarchive-3.3.3 compressed with xz and
> write a wiki page with detailed instructions on how to manually switch
> from an old system (for users who might want to switch even later).
>
Users that may
[2019-03-25 00:46:15 +0100] Morten Linderud via arch-dev-public:
> On Sun, Mar 24, 2019 at 04:39:54PM -0700, Andrew Gregory via arch-dev-public
> wrote:
> > I don't consider hoping that libarchive doesn't need a rebuild in the
> > near future a great strategy. That being said, this is really
> >
On Mon, Mar 25, 2019 at 12:46:15AM +0100, Public mailing list for Arch Linux
development wrote:
> On Sun, Mar 24, 2019 at 04:39:54PM -0700, Andrew Gregory via arch-dev-public
> wrote:
> > I don't consider hoping that libarchive doesn't need a rebuild in the
> > near future a great strategy.
On Sun, Mar 24, 2019 at 04:39:54PM -0700, Andrew Gregory via arch-dev-public
wrote:
> I don't consider hoping that libarchive doesn't need a rebuild in the
> near future a great strategy. That being said, this is really
> a question of how long of a period we need between libarchive v3.3.3
> and
On 03/25/19 at 12:28am, Robin Broda via arch-dev-public wrote:
> On 3/25/19 12:22 AM, Andrew Gregory wrote:
> > On 03/25/19 at 12:15am, Robin Broda via arch-dev-public wrote:
> >> On 3/24/19 11:20 PM, Evangelos Foutras via arch-dev-public wrote:
> >>> On Sun, 24 Mar 2019 at 23:45, Allan McRae via
On 25/3/19 9:28 am, Robin Broda via arch-dev-public wrote:
> On 3/25/19 12:22 AM, Andrew Gregory wrote:
>> On 03/25/19 at 12:15am, Robin Broda via arch-dev-public wrote:
>>> On 3/24/19 11:20 PM, Evangelos Foutras via arch-dev-public wrote:
On Sun, 24 Mar 2019 at 23:45, Allan McRae via
On Sun, Mar 24, 2019 at 04:22:55PM -0700, Andrew Gregory via arch-dev-public
wrote:
> On 03/25/19 at 12:15am, Robin Broda via arch-dev-public wrote:
> > Thus i don't think we need a hold-off period like this, Allan.
>
> We still need a hold-off period, we're just waiting to make sure
> people
On 3/25/19 12:22 AM, Andrew Gregory wrote:
> On 03/25/19 at 12:15am, Robin Broda via arch-dev-public wrote:
>> On 3/24/19 11:20 PM, Evangelos Foutras via arch-dev-public wrote:
>>> On Sun, 24 Mar 2019 at 23:45, Allan McRae via arch-dev-public
>>> wrote:
We need to assume every system
On 03/25/19 at 12:15am, Robin Broda via arch-dev-public wrote:
> On 3/24/19 11:20 PM, Evangelos Foutras via arch-dev-public wrote:
> > On Sun, 24 Mar 2019 at 23:45, Allan McRae via arch-dev-public
> > wrote:
> >>
> >> On 25/3/19 4:34 am, Robin Broda via arch-dev-public wrote:
> >>> This change
On Sun, Mar 24, 2019 at 7:35 PM Robin Broda via arch-dev-public
wrote:
> The required changeset is, i think:
> PKGEXT='.pkg.tar.zst'
> COMPRESSZST=(zstd -c -T0 -18 -)
When we implement this, I would say we go with "zstd -c -T0 -" in
pacman's makepkg.conf and "zstd -C -T0 -18 -" in the configs
On 3/24/19 9:18 PM, Baptiste Jonglez wrote:
> Just one detail: your results for -19, -20 and -21 are identical because
> apparently zstd needs an additional flag (--ultra) to "unlock" the higher
> compression levels:
>
> zstd -c -T0 -20 -
> Warning : compression level higher than max,
On 3/24/19 11:20 PM, Evangelos Foutras via arch-dev-public wrote:
> On Sun, 24 Mar 2019 at 23:45, Allan McRae via arch-dev-public
> wrote:
>>
>> On 25/3/19 4:34 am, Robin Broda via arch-dev-public wrote:
>>> This change requires a new pacman release, as as of writing this, zstd
>>> support is in
On Sun, 24 Mar 2019 at 23:45, Allan McRae via arch-dev-public
wrote:
>
> On 25/3/19 4:34 am, Robin Broda via arch-dev-public wrote:
> > This change requires a new pacman release, as as of writing this, zstd
> > support is in master but hasn't landed in a release yet.
>
> Which is a complete
On 25/3/19 4:34 am, Robin Broda via arch-dev-public wrote:
> This change requires a new pacman release, as as of writing this, zstd
> support is in master but hasn't landed in a release yet.
Which is a complete blocker for quite a period of time.
We need to assume every system has a copy of
Hi,
On 24-03-19, Robin Broda via arch-dev-public wrote:
> So, TL;DR, the benefits of `zstd -c -T0 -18 -` over `xz -c -z -` are:
> - Massive speed gain in compression
> - Massive speed gain in decompression
> - Stable, reproducible multithreading
> The speed gain in decompression substantially
Attached here is the script i wrote to make most of these measurements, for
anyone interested in reproducing these results - and the raw results of the
benchmark.
Read it before running it. You may need to make adjustments.
Rob
bench.sh
Description: application/shellscript
On Sun, 24 Mar 2019 at 19:35, Robin Broda via arch-dev-public <
arch-dev-public@archlinux.org> wrote:
> Hello all,
>
> in the past few weeks, some TUs and Developers have compared different
> compression algorithms to potentially replace the default compression
> method used in devtools.
> The
Hello all,
in the past few weeks, some TUs and Developers have compared different
compression algorithms to potentially replace the default compression method
used in devtools.
The current method is `xz -c -z -` which is single-threaded and rather slow, so
we are looking to replace it with
25 matches
Mail list logo
