(2013/10/22 12:52), Fujii Masao wrote:
On Tue, Oct 22, 2013 at 12:47 PM, Amit Kapila <amit.kapil...@gmail.com> wrote:
On Mon, Oct 21, 2013 at 4:40 PM, KONDO Mitsumasa
<kondo.mitsum...@lab.ntt.co.jp> wrote:
(2013/10/19 14:58), Amit Kapila wrote:
On Tue, Oct 15, 2013 at 11:41 AM, KONDO Mitsumasa
<kondo.mitsum...@lab.ntt.co.jp> wrote:
In general, my thinking is that we should prefer compression to reduce
IO (WAL volume), because reducing WAL volume has other benefits as
well like sending it to subscriber nodes. I think it will help cases
where due to less n/w bandwidth, the disk allocated for WAL becomes
full due to high traffic on master and then users need some
alternative methods to handle such situations.
Do you talk about archiving WAL file?

    One of the points what I am talking about is sending data over
network to subscriber nodes for streaming replication and another is
WAL in pg_xlog. Both scenario's get benefited if there is is WAL

It can easy to reduce volume that we
set and add compression command with copy command at archive_command.


I think many users would like to use a method which can reduce WAL
volume and the users which don't find it enough useful in their
environments due to decrease in TPS or not significant reduction in
WAL have the option to disable it.
I favor to select compression algorithm for higher performance. If we need
to compress WAL file more, in spite of lessor performance, we can change
archive copy command with high compression algorithm and add documents that
how to compress archive WAL files at archive_command. Does it wrong?

  No, it is not wrong, but there are scenario's as mentioned above
where less WAL volume can be beneficial.

actual, many of NoSQLs use snappy for purpose of higher performance.

Okay, you can also check the results with snappy algorithm, but don't
just rely completely on snappy for this patch, you might want to think
of another alternative for this patch.

So, our consensus is to introduce the hooks for FPW compression so that
users can freely select their own best compression algorithm?
Yes, it will be also good for future improvement. But I think WAL compression for disaster recovery system should be need in walsender and walreceiver proccess, and it is propety architecture for DR system. Higher compression ratio with high CPU usage algorithm in FPW might affect bad for perfomance in master server. If we can set compression algorithm in walsender and walreciever, performance is same as before or better, and WAL send performance will be better.

Mitsumasa KONDO
NTT Open Source Software Center

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to