Re: [Qemu-block] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

2018-02-22 Thread Alberto Garcia
On Wed 21 Feb 2018 05:59:58 PM CET, Eric Blake wrote: > But as Berto has convinced me that an externally produced image can > convince us to read up to 4M (even though we don't need that much to > decompress), A (harmless but funny) consequence of the way this works is that for any valid

Re: [Qemu-block] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

2018-02-21 Thread Eric Blake
On 02/21/2018 11:39 AM, Kevin Wolf wrote: See my commit message comment - we have other spots in the code base that blindly g_malloc(2 * s->cluster_size). Though is that a reason to do the same in new code or to phase out such allocations whenever you touch them? Touché. And I intended

Re: [Qemu-block] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

2018-02-21 Thread Kevin Wolf
Am 21.02.2018 um 17:59 hat Eric Blake geschrieben: > On 02/21/2018 10:51 AM, Kevin Wolf wrote: > > Am 20.02.2018 um 23:24 hat Eric Blake geschrieben: > > > When reading a compressed image, we were allocating s->cluster_data > > > to 32*cluster_size + 512 (possibly over 64 megabytes, for an image >

Re: [Qemu-block] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

2018-02-21 Thread Eric Blake
On 02/21/2018 10:51 AM, Kevin Wolf wrote: Am 20.02.2018 um 23:24 hat Eric Blake geschrieben: When reading a compressed image, we were allocating s->cluster_data to 32*cluster_size + 512 (possibly over 64 megabytes, for an image with 2M clusters). Let's check out the history: Much later, in

Re: [Qemu-block] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

2018-02-21 Thread Kevin Wolf
Am 20.02.2018 um 23:24 hat Eric Blake geschrieben: > When reading a compressed image, we were allocating s->cluster_data > to 32*cluster_size + 512 (possibly over 64 megabytes, for an image > with 2M clusters). Let's check out the history: > > Back when qcow2 was first written, we used

Re: [Qemu-block] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

2018-02-21 Thread Alberto Garcia
On Wed 21 Feb 2018 04:00:54 PM CET, Eric Blake wrote: >> - Solution b: the width of the 'compressed cluster size' field is >>(cluster_bits - 8), that's (cluster_size / 256) sectors. > > Not true. It is (cluster_bits - 9) or (cluster_size / 512). It's not, it's (cluster_bits - 8), the

Re: [Qemu-block] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

2018-02-21 Thread Eric Blake
On 02/21/2018 04:04 AM, Alberto Garcia wrote: On Tue 20 Feb 2018 11:24:59 PM CET, Eric Blake wrote: I was also preparing a patch to change this, but you arrived first :-) So, it's time to cut back on the waste. A compressed cluster will NEVER occupy more than an uncompressed cluster (okay,

Re: [Qemu-block] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

2018-02-21 Thread Alberto Garcia
On Tue 20 Feb 2018 11:24:59 PM CET, Eric Blake wrote: I was also preparing a patch to change this, but you arrived first :-) > So, it's time to cut back on the waste. A compressed cluster > will NEVER occupy more than an uncompressed cluster (okay, gzip > DOES document that because the