Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-05-21 Thread Koichi Suzuki
added information on btree strip log records, which anables to produce corresponding incremental logs from the full page writes. 2007/5/21, Tom Lane [EMAIL PROTECTED]: Koichi Suzuki [EMAIL PROTECTED] writes: As replied to Patch queue triage by Tom, here's simplified patch to mark WAL record

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-05-07 Thread Koichi Suzuki
be considered again. Best Regards; -- - Koichi Suzuki diff -cr pgsql_org/src/backend/access/transam/xlog.c pgsql/src/backend/access/transam/xlog.c *** pgsql_org/src/backend/access/transam/xlog.c 2007-05-02 15:56:38.0 +0900 --- pgsql/src/backend/access/transam/xlog.c 2007-05

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-26 Thread Koichi Suzuki
will find it even more so. Now that I have Koichi's explanation of the problem, I vote for simply slaving this to the PITR settings and not having a separate option at all. Could I have more specific suggestion on this? Regards; -- - Koichi Suzuki ---(end

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Koichi Suzuki
in WAL, and making the recovery able to skip dummy records. This I would do unconditionally. Andreas ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq -- - Koichi

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-23 Thread Koichi Suzuki
, compression ratio, CPU usage and I/O by pg_compresslog are all quite better than those in gzip. Please let me know if you intended defferently. Regards; -- - Koichi Suzuki ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-22 Thread Koichi Suzuki
. So far, it works fine. Yes, Tom didn't like LSN replacing eighter. I withdraw my concern regarding pg_decompresslog. Your work in this area is extremely valuable and I hope my comments are not discouraging. Thank you Andreas -- - Koichi Suzuki ---(end

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-20 Thread Koichi Suzuki
short WAL records in pg_compresslog like: advance LSN by 8192 bytes. Andreas -- - Koichi Suzuki ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-20 Thread Koichi Suzuki
about doing this for at least 3 years now, so I don't see any reason to baulk at it now. I'm happy with Koichi-san's patch as-is, assuming further extensive testing will be carried out on it during beta. -- - Koichi Suzuki ---(end of broadcast

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-20 Thread Koichi Suzuki
before it could be merged (eg, I noted a distinct lack of I/O error checking). regards, tom lane -- - Koichi Suzuki ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-12 Thread Koichi Suzuki
produces. For recovery, we have to restore LSN as the original WAL. Pg_decompresslog restores removed full page writes as a dumm records so that recovery redo functions won't be confused. Regards; Andreas -- - Koichi Suzuki ---(end of broadcast

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-11 Thread Koichi Suzuki
The score below was taken based on 8.2 code, not 8.3 code. So I don't think the below measure is introduced only in 8.3 code. Tom Lane wrote: Koichi Suzuki [EMAIL PROTECTED] writes: For more information, when checkpoint interval is one hour, the amount of the archived log size was as follows

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-11 Thread Koichi Suzuki
not degrade performance. -- Koichi Suzuki ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-10 Thread Koichi Suzuki
reasonable measure. Again, in my proposal, it is not the issue to increase run time performance. Issue is to decrease the size of archive log to save the storage. Regards; Tom Lane wrote: Koichi Suzuki [EMAIL PROTECTED] writes: My proposal is to remove unnecessary full page writes (they are needed

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-05 Thread Koichi Suzuki
. Regards; Simon Riggs wrote: On Tue, 2007-04-03 at 19:45 +0900, Koichi Suzuki wrote: Bruce Momjian wrote: Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches Thank you very much for including. Attached is an update

Re: [PATCHES] [HACKERS] Full page writes improvement, code update again.

2007-04-03 Thread Koichi Suzuki
Here's third revision of WAL archival optimization patch. GUC parameter name was changed to wal_add_optimization_info. Regards; -- Koichi Suzuki 20070403_pg_lesslog.tar.gz Description: application/gzip ---(end of broadcast)--- TIP 1

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-03 Thread Koichi Suzuki
Bruce Momjian wrote: Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches Thank you very much for including. Attached is an update of the patch according to Simon Riggs's comment about GUC name. Regards; -- Koichi

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-01 Thread Koichi Suzuki
bit is used to mark that full page writes must not be removed at offline optimization by pg_compresslog. Regards; -- -- Koichi Suzuki -- Koichi Suzuki ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating

Re: [PATCHES] Full page writes improvement, code update

2007-03-30 Thread Koichi Suzuki
to continue my post. Sorry for confusion. Please refer to the original thread about this discussion. Best Regards; -- -- Koichi Suzuki ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-29 Thread Koichi Suzuki
Hi, Here're some feedback to the comment: Simon Riggs wrote: On Wed, 2007-03-28 at 10:54 +0900, Koichi Suzuki wrote: As written below, full page write can be categolized as follows: 1) Needed for crash recovery: first page update after each checkpoint. This has to be kept in WAL. 2) Needed

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-29 Thread Koichi Suzuki
have any suggestion to reflect what it really means. Regards; -- Koichi Suzuki ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-29 Thread Koichi Suzuki
Hi, Here's a patch reflected some of Simon's comments. 1) Removed an elog call in a critical section. 2) Changed the name of the commands, pg_complesslog and pg_decompresslog. 3) Changed diff option to make a patch. -- Koichi Suzuki pg_lesslog.tgz Description: Binary data

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-27 Thread Koichi Suzuki
Simon; Thanks a lot for your comments/advices. I'd like to write some feedback. Simon Riggs wrote: On Tue, 2007-03-27 at 11:52 +0900, Koichi Suzuki wrote: Here's an update of a code to improve full page writes as proposed in http://archives.postgresql.org/pgsql-hackers/2007-01/msg01491.php

[PATCHES] Full page writes improvement, code update

2007-03-26 Thread Koichi Suzuki
but still would like to hear comments/opinions/advices. Regards; -- Koichi Suzuki pg_lesslog.tgz Description: Binary data ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

Re: [HACKERS] [PATCHES] Full page writes improvement

2007-02-08 Thread Koichi Suzuki
as contrib and the patch itself does work if WAL is simply copied to the archive directory. Regards; Koichi Suzuki Tom Lane wrote: Koichi Suzuki [EMAIL PROTECTED] writes: Tom Lane wrote: Doesn't this break crash recovery on PITR slaves? Compressed archive log contains the same data

[PATCHES] Full page writes improvement

2007-01-31 Thread Koichi Suzuki
. Included is a patch for this as well as archive and restore command source. Patch is very small and I hope this to be included in 8.3. -- Koichi Suzuki pg_lesslog.tar.gz Description: application/gzip ---(end of broadcast)--- TIP 4: Have you

Re: [PATCHES] A couple of patches for PostgreSQL 64bit support

2005-07-19 Thread Koichi Suzuki
if it would be more interesting to try them against CVS rather than 8.0.3 (and if it would be easy to port :)? Mark -- --- Koichi Suzuki Open Source Engineeering Departmeent, NTT DATA Intellilink Corporation Phone: +81-3-5566-9628 WWW: http

Re: [PATCHES] A couple of patches for PostgreSQL 64bit support

2005-07-14 Thread Koichi Suzuki
with 64bit patch PostgreSQL (base is 8.0.1). Benchmark machine is dual opteron (1.4GHz, 1MB cache each) with 8GB of memory and 120GB of IDE hard disk. Koichi Suzuki wrote: I have some experimeltal data about this extension. I will gather it and post hopefully this weekend

Re: [PATCHES] A couple of patches for PostgreSQL 64bit support

2005-07-12 Thread Koichi Suzuki
Hi, Attached is a result of pgbench with 64bit patch PostgreSQL (base is 8.0.1). Benchmark machine is dual opteron (1.4GHz, 1MB cache each) with 8GB of memory and 120GB of IDE hard disk. Koichi Suzuki wrote: I have some experimeltal data about this extension. I will gather it and post

[PATCHES] A couple of patches for PostgreSQL 64bit support

2005-07-07 Thread Koichi Suzuki
. -- --- Koichi Suzuki Open Source Engineeering Departmeent, NTT DATA Intellilink Corporation Phone: +81-3-5566-9628 WWW: http://www.intellilink.co.jp -- Proposal: 64bit Extension in PostgreSQL 8.0.x

[PATCHES] A couple of patches for PostgreSQL 64bit support

2005-07-07 Thread Koichi Suzuki
Hi, all, I have posted a couple of patches with regard to 64bit environment support to PATCHES ml. It expands size of shared memory to 64bit space and extends XID to 64bit. Please take a look at it. -- --- Koichi Suzuki Open Source Engineeering

Re: [PATCHES] A couple of patches for PostgreSQL 64bit support

2005-07-07 Thread Koichi Suzuki
I have some experimeltal data about this extension. I will gather it and post hopefully this weekend. Tom Lane wrote: Koichi Suzuki [EMAIL PROTECTED] writes: Here're a couple of patches for PostgreSQL 64bit support. There're just two extension to 64bit, size of shared memory and transaction