On Tue, Oct 17, 2023 at 1:59 PM Mattia Verga via devel
wrote:
>
> Is there any chance we can have this drafted as a change request for F40?
>
It is being drafted, don't worry:
https://fedoraproject.org/wiki/Changes/ZlibNGTransition
We'll hopefully submit it soon.
--
真実はいつも一つ！/ Always,
Is there any chance we can have this drafted as a change request for F40?
Inviato da Proton Mail mobile
Messaggio originale
Il 19 Set 2023, 18:26, Daniel Alley ha scritto:
> Will this be posted soon? ___
> devel mailing list --
Will this be posted soon?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
Thank you.
Leslie Satenstein
On Saturday, September 2, 2023 at 09:29:36 a.m. EDT, Daniel Alley
wrote:
Yeah, the Zstd default of 3 is tuned to be roughly on par with alternatives
but much faster. To get better compression than gzip you generally need to go
to higher levels.
Yeah, the Zstd default of 3 is tuned to be roughly on par with alternatives but
much faster. To get better compression than gzip you generally need to go to
higher levels.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send
On Fri, Sep 01, 2023 at 06:45:22PM +0200, Florian Weimer wrote:
> * Richard W. M. Jones:
>
> > I tested the speed of decompression using:
> >
> > $ hyperfine 'qemu-img convert -W -m 16 -f qcow2 test.qcow2.XXX -O raw
> > test.out'
> > (qemu 8.0.0-4.fc39.x86_64)
> >
> > $ hyperfine 'nbdkit
* Richard W. M. Jones:
> I tested the speed of decompression using:
>
> $ hyperfine 'qemu-img convert -W -m 16 -f qcow2 test.qcow2.XXX -O raw
> test.out'
> (qemu 8.0.0-4.fc39.x86_64)
>
> $ hyperfine 'nbdkit -U - --filter=qcow2dec file test.qcow2.XXX --run
> '\''nbdcopy --request-size
On Fri, Sep 1, 2023 at 9:02 AM Richard W.M. Jones wrote:
> On Fri, Sep 01, 2023 at 07:10:59AM -0500, Richard Shaw wrote:
> > Drive by comment because I found this thread interesting...
> >
> > What compression level was used for zstd? My understanding is that higher
> > levels take longer to
On Fri, Sep 01, 2023 at 07:10:59AM -0500, Richard Shaw wrote:
> Drive by comment because I found this thread interesting...
>
> What compression level was used for zstd? My understanding is that higher
> levels take longer to compress but decompression time remains virtually the
> same.
Seems to
Drive by comment because I found this thread interesting...
What compression level was used for zstd? My understanding is that higher
levels take longer to compress but decompression time remains virtually the
same.
Thanks,
Richard
___
devel mailing
On Fri, Sep 01, 2023 at 01:10:16PM +0200, Kevin Wolf wrote:
> And nbdkit seems to get worse instead of better with larger cluster
> size, no matter whether zlib or zstd is used.
It's caused by nbdcopy's default request size being 256k. Increasing
it to 2M cures the scaling problem - see updated
Am 01.09.2023 um 12:03 hat Richard W.M. Jones geschrieben:
> On Fri, Sep 01, 2023 at 10:55:50AM +0100, Daniel P. Berrangé wrote:
> > On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote:
> > > On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote:
> > > > I understand the
On Fri, Sep 01, 2023 at 11:06:20AM +0100, Daniel P. Berrangé wrote:
> On Fri, Sep 01, 2023 at 11:03:54AM +0100, Richard W.M. Jones wrote:
> > I forgot to say that nbdkit is using zlib-ng, since I made the source
> > level changes a few weeks back (but most of the nbdkit performance
> > improvement
On Fri, Sep 01, 2023 at 11:03:54AM +0100, Richard W.M. Jones wrote:
> On Fri, Sep 01, 2023 at 10:55:50AM +0100, Daniel P. Berrangé wrote:
> > On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote:
> > > On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote:
> > > > Am 31.08.2023
On Fri, Sep 01, 2023 at 10:55:50AM +0100, Daniel P. Berrangé wrote:
> On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote:
> > On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote:
> > > Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben:
> > > > On Thu, Aug 31, 2023 at
On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote:
> Results:
>
> Cluster Compression Compressed size Prog Decompression speed
>
> 4k zlib 3228811264 qemu 5.921 s ± 0.074 s
> 4k zstd 3258097664 qemu 5.189 s ± 0.158 s
>
>
On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote:
> On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote:
> > Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben:
> > > On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote:
> > > > [ Cc: qemu-block ]
> > > >
> >
On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote:
> $ hyperfine 'nbdkit -U - --filter=qcow2dec file test.qcow2.XXX --run
> '\''nbdcopy --request-size "$uri" test.out'\'' '
Sorry, copy and paste error, the command is:
hyperfine 'nbdkit -U - --filter=qcow2dec file
On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote:
> Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben:
> > On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote:
> > > [ Cc: qemu-block ]
> > >
> > > Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben:
> > > > On Tue,
Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben:
> On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote:
> > [ Cc: qemu-block ]
> >
> > Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben:
> > > On Tue, Aug 29, 2023 at 05:49:24PM -, Daniel Alley wrote:
> > > > > The
* Richard W. M. Jones:
> On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote:
>> Unfortunately, we seem to build the RHEL packages with --disable-zstd
>> (I suppose just because we tend to disable everything nobody explicitly
>> asked for). Maybe we should check other distros. If zstd is
On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote:
> [ Cc: qemu-block ]
>
> Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben:
> > On Tue, Aug 29, 2023 at 05:49:24PM -, Daniel Alley wrote:
> > > > The background to this is I've spent far too long trying to optimize
> > > >
On Tue, Aug 29, 2023 at 05:49:24PM -, Daniel Alley wrote:
> > The background to this is I've spent far too long trying to optimize
> > the conversion of qcow2 files to raw files. Most existing qcow2 files
> > that you can find online are zlib compressed, including the qcow2
> > images
> The background to this is I've spent far too long trying to optimize
> the conversion of qcow2 files to raw files. Most existing qcow2 files
> that you can find online are zlib compressed, including the qcow2
> images provided by Fedora. Each cluster in the file is separately
> compressed as a
I went ahead and started to modify Jacek's initial proposal [1] in order
to integrate the feedback from this thread as well as some changes I had
in mind.
I have created a RFC pull request [2]. Hopefully this will help answer
some of the questions we've seen here. Feedback is appreciated.
I have
One practical example - the process of creating an OCI container image of
the Fedora Flatpak runtime does a *lot* of zlib compression and
decompression. (https://pagure.io/flatpak-module-tools/issue/22). A lot of
that is temporary to save disk space and IO bandwidth and could be switched
to zstd
As far as I can tell it's mostly legacy removal and adoption of (now)
standardized types, such as "z_size_t" being replaced by "size_t"
There is a document here that tries to describe the different considerations:
https://github.com/zlib-ng/zlib-ng/blob/develop/PORTING.md
"The zlib-ng native
@Tulio Magno Quites Machado Filho there is one
question I have due to the zlib-ng...
What is the difference between zlib-ng compiled with and without the
`ZLIB_COMPAT=ON` option? Are there any differences in the performance, or
is it only the names of the functions?
I'm interested in this
>
> In the end most of the patches were dropped the uplift wasn't worth the
> effort of maintaining them downstream, when the effort can be better
> spent getting a 10X uplift using a more modern compression
> implementation (ex zstd/lz4/lzo/etc) that isn't written with so many
> byte oriented
Hi,
On 8/6/23 08:33, John Reiser wrote:
On 8/6/23 02:00, Peter Robinson wrote:
We tried to pull some of the optimisations in some time ago to the
Fedora package and they caused some issues with compatibility.
Please provide *any* documentation! Such as: the dates the work was
performed,
Sorry, resending because the original message was rejected by the
mailing list.
Hi Lukas,
Lukas Javorsky writes:
> Hi,
>
> I'm currently maintaining the zlib package across Fedora and Red Hat products.
>
> I like the proposal for the zlib-ng package, there are just a few questions
> for @Tulio
On Mon, Aug 07, 2023 at 09:29:08AM -0300, Tulio Magno Quites Machado Filho
wrote:
> "Richard W.M. Jones" writes:
>
> > The background to this is I've spent far too long trying to optimize
> > the conversion of qcow2 files to raw files. Most existing qcow2 files
> > that you can find online are
Hi,
I'm currently maintaining the zlib package across Fedora and Red Hat
products.
I like the proposal for the zlib-ng package, there are just a few questions
for @Tulio Magno Quites Machado Filho :
1) Just to clarify, do you want to have two separate packages (zlib-ng and
e.g. zlib-ng-compat)
The zlib-ng 2.1 beta, apparently, has some significant further enhancements
coming down the pipe. So the potential is there for users to see improvements
much greater than 40% eventually.
https://www.phoronix.com/news/Zlib-ng-2.1-Beta
"With zlib-ng 2.1 beta there is upwards of 56% faster
On 07/08/2023 15:29, Tulio Magno Quites Machado Filho wrote:
"Richard W.M. Jones" writes:
The background to this is I've spent far too long trying to optimize
the conversion of qcow2 files to raw files. Most existing qcow2 files
that you can find online are zlib compressed, including the
"Richard W.M. Jones" writes:
> The background to this is I've spent far too long trying to optimize
> the conversion of qcow2 files to raw files. Most existing qcow2 files
> that you can find online are zlib compressed, including the qcow2
> images provided by Fedora. Each cluster in the file
On Sat, Aug 05, 2023 at 04:45:36PM +0100, Neal Gompa wrote:
> I think it'd be worth us moving to zlib-ng as the sole zlib provider.
+1. zlib-ng is already widely used, so if it has issues, then we'd have a
major problem. So we're not really saving much by having two implementations.
One library
Also to that point, the compatibility issues aren't always compatibility
issues, but rather poorly written tests. For example, some tests might
hardcode an expected checksum [0] for the compressed output, which could be
broken by any number of changes even if the compressed output is entirely
On 8/6/23 02:00, Peter Robinson wrote:
We tried to pull some of the optimisations in some time ago to the
Fedora package and they caused some issues with compatibility.
Please provide *any* documentation! Such as: the dates the work was performed,
the participants, the nature of the issues,
On Sun, Aug 06, 2023 at 04:34:46AM -, Daniel Alley wrote:
> >In my test zlib-ng is about 40% faster.
>
> I did some testing with zlib-ng and createrepo-c a few months ago
> [0], and I also found that the compression portion of the workload
> was about 40% faster. So this matches my
On Sun, Aug 6, 2023 at 5:35 AM Daniel Alley wrote:
>
> >In my test zlib-ng is about 40% faster.
>
> I did some testing with zlib-ng and createrepo-c a few months ago [0], and I
> also found that the compression portion of the workload was about 40% faster.
> So this matches my experience, too.
>In my test zlib-ng is about 40% faster.
I did some testing with zlib-ng and createrepo-c a few months ago [0], and I
also found that the compression portion of the workload was about 40% faster.
So this matches my experience, too.
(BTW, if that github comment [0] sounds negative, it isn't
On Sat, Aug 5, 2023 at 4:12 PM Richard W.M. Jones wrote:
>
> The background to this is I've spent far too long trying to optimize
> the conversion of qcow2 files to raw files. Most existing qcow2 files
> that you can find online are zlib compressed, including the qcow2
> images provided by
The background to this is I've spent far too long trying to optimize
the conversion of qcow2 files to raw files. Most existing qcow2 files
that you can find online are zlib compressed, including the qcow2
images provided by Fedora. Each cluster in the file is separately
compressed as a zlib
44 matches
Mail list logo
