On 07/25/2015 10:50 AM, Michael Paquier wrote:
On Sat, Jul 25, 2015 at 11:09 PM, Marc Mamin <m.ma...@intershop.de> wrote:
On Sat, Jul 25, 2015 at 9:56 PM, Andrew Dunstan <and...@dunslane.net> wrote:
On 07/25/2015 03:20 AM, Andrew Dunstan wrote:
On 07/25/2015 02:34 AM, Marc Mamin wrote:
Andrew Dunstan <and...@dunslane.net> writes:
Hmm. Yeah. It looks like commit
a7ad5cf0cfcfab8418000d652fa4f0c6ad6c8911
changed from using the default compression for libz to using the
compression set in pg_dump options, which defaults to 0. This actually
seems like the right thing to do, but it certainly should have been
called out much more forcefully in release notes, and arguably should
not have been changed in stable releases. Not sure what we do about it
now.
really 0? wouldn't that mean no compression at all?
No, that's not right either. The default should be Z_DEFAULT_COMPRESSION,
so we shouldn't actually see a difference in the default case. And it is
definitely compressing some. So I'm now puzzled by what you're seeing.
OK, I have got this worked out. I'll have a bug fix shortly.
So you are basically planning to switch to Z_BEST_SPEED instead of
Z_DEFAULT_COMPRESSION where needed for the default code path?
It looks like Z_DEFAULT_COMPRESSION was the used value before, and from the
source code I guess this should still be the case.
From a quick testing, it now behaves as if the minus sign from
Z_DEFAULT_COMPRESSION is lost on the way, resulting in -Z1. this might indicate
that Z_DEFAULT_COMPRESSION is parsed like the command line argument.
gzopen only allows a digit, so -1 results in the - being ignored and 1
being the compression value. It's a pity gzopen is so liberal about
accepting modes with values it ignores, or we'd have noticed this ages ago.
I think I understand what is happening... With a quick test using the
default compression level in directory mode, gzopen is called with
w-1. By decrypting the docs of zlib (http://www.zlib.net/manual.html),
section File Access Functions, it seems to me that calling gzopen like
that will cause the file to not be compressed at all, which is
actually why you are seeing an increase in your dump files, while we
should call it with a compression mode of 6 actually based on what
Z_DEFAULT_COMPRESSION means for zlib now, or simply without a
compression level specified such as the default is used by zlib. Prior
to a7ad5cf0, what we did was actually gzopen with always "w" or "wb"
that caused the default compression level of 6 to be used. Hence I
think that we should definitely use "w" or "wb" without specifying a
level number when the compression level is Z_DEFAULT_COMPRESSION, and
"wbN" (0<=N<=9) when a compression level is given by the user. A patch
is attached for this purpose. Marc, does it work for you?
e.g. pg_dump -Z-2 is like pg_dump -Z2
The compression needs to be included in range [0,9].
I propose to tighten pg_dump's rules so that only 0..9 are accepted as
arguments for -Z, and in compress_io.c:cfopen(), if compression is equal
to Z_DEFAULT_COMPRESSION, not add any explicit compression value to the
mode, thus using the zlib default.
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
