Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Jeff Janes
On Thursday, April 11, 2013, Kevin Grittner wrote: I also log the number of pages truncated at the time it gave up, as it would be nice to know if it is completely starving or making some progress. If we're going to have the message, we should make it useful. My biggest question here

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-12 Thread Hannu Krosing
On 04/11/2013 07:29 PM, Fujii Masao wrote: On Thu, Apr 11, 2013 at 10:25 PM, Hannu Krosing ha...@2ndquadrant.com wrote: You just shut down the old master and let the standby catch up (takas a few microseconds ;) ) before you promote it. After this you can start up the former master with

[HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Sameer Thakur
Hello, The current process of transferring data files from one cluster to another by using pg_dump and pg_restore is time consuming. The proposed tool tries to make migration faster for tables and indices only by copying their binary data files. This is like pg_upgrade but used for migration of

Re: [HACKERS] [COMMITTERS] pgsql: Add sql_drop event for event triggers

2013-04-12 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Yeah, I was just looking at the IfSupported variant. In the structure I just suggested (separate ProcessSlowUtility function), we could make that work by having switch cases for some statements in both functions, I've done it the way you propose here, and

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-12 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Alvaro Herrera alvhe...@2ndquadrant.com writes: This idea doesn't work because of back-patch considerations (i.e. we would not be able to create the functions in back branches, and so this new style of pg_dump would only work with future server versions).

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-12 Thread Dimitri Fontaine
Michael Paquier michael.paqu...@gmail.com writes: I recall discussions about reverse engineering of a parsed query tree in the event trigger threads but nothing has been committed I think. Also, you Yes. The name used in there was Normalized Command String. need to consider that implementing

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-12 Thread Shuai Fan
On 04/11/2013 11:48 PM, Andrew Dunstan wrote: It could be interesting to have a library that would output database metadata in some machine readable and manipulatable format such as JSON or XML. One thing that's annoying about the text output pg_dump produces is that it's not at all

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-12 Thread Andres Freund
On 2013-04-12 02:29:01 +0900, Fujii Masao wrote: On Thu, Apr 11, 2013 at 10:25 PM, Hannu Krosing ha...@2ndquadrant.com wrote: You just shut down the old master and let the standby catch up (takas a few microseconds ;) ) before you promote it. After this you can start up the former

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-12 Thread Andres Freund
On 2013-04-12 11:18:01 +0530, Pavan Deolasee wrote: On Thu, Apr 11, 2013 at 8:39 PM, Ants Aasma a...@cybertec.at wrote: On Thu, Apr 11, 2013 at 5:33 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 04/11/2013 03:52 PM, Ants Aasma wrote: On Thu, Apr 11, 2013 at 4:25 PM, Hannu

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-12 Thread Pavan Deolasee
On Fri, Apr 12, 2013 at 4:29 PM, Andres Freund and...@2ndquadrant.comwrote: I don't think that holds true at all. If you look at pg_stat_bgwriter in any remotely bugs cluster with a hot data set over shared_buffers you'll notice that a large percentage of writes will have been done by

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-12 Thread Andres Freund
On 2013-04-12 16:58:44 +0530, Pavan Deolasee wrote: On Fri, Apr 12, 2013 at 4:29 PM, Andres Freund and...@2ndquadrant.comwrote: I don't think that holds true at all. If you look at pg_stat_bgwriter in any remotely bugs cluster with a hot data set over shared_buffers you'll notice that

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-12 Thread Hannu Krosing
On 04/11/2013 12:17 AM, Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Hannu Krosing wrote: Natural solution to this seems to move most of pg_dump functionality into backend as functions, so we have pg_dump_xxx() for everything we want to dump plus a topological sort function

Re: [HACKERS] [PATCH] pg_regress and non-default unix socket path

2013-04-12 Thread Christoph Berg
Re: To PostgreSQL Hackers 2013-04-09 20130409120807.gd26...@msgid.df7cb.de If the patch looks too intrusive at this stage of the release, it would be enough if the last chunk got included, which should really be painless: diff --git a/src/test/regress/pg_regress.c

Re: [HACKERS] [sepgsql 2/3] Add db_schema:search permission checks

2013-04-12 Thread Robert Haas
On Mon, Apr 8, 2013 at 12:28 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Thanks. I could find two obvious wording stuffs here, please see smaller one of the attached patches. I didn't fixup manner to use XXX in source code comments. Committed. Also, the attached function-execute-permission

Re: [HACKERS] [ADMIN] after 9.2.4 patch vacuumdb -avz not analyzing all tables

2013-04-12 Thread Mike Broers
On further review this particular server skipped from 9.2.2 to 9.2.4. This is my most busy and downtime sensitive server and I was waiting on a maintenance window to patch to 9.2.3 when 9.2.4 dropped and bumped up the urgency. However, I have 3 other less busy production servers that were all

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Tom Lane
Sameer Thakur samthaku...@gmail.com writes: The proposed tool tries to make migration faster for tables and indices only by copying their binary data files. There's 0 chance of making that work, because the two databases wouldn't have the same notions of committed XIDs. You apparently don't

Re: [HACKERS] [sepgsql 2/3] Add db_schema:search permission checks

2013-04-12 Thread Alvaro Herrera
Robert Haas escribió: On Mon, Apr 8, 2013 at 12:28 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Also, the attached function-execute-permission patch is a rebased version. I rethought its event name should be OAT_FUNCTION_EXECUTE, rather than OAT_FUNCTION_EXEC according to the manner without

Re: [HACKERS] Analyzing bug 8049

2013-04-12 Thread Robert Haas
On Thu, Apr 11, 2013 at 1:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: This idea needs more fleshing out, but it's seeming awfully attractive right now. The big problem with it is that it's going to be a more invasive patch than I feel terribly comfortable about back-patching. However, I'm not

Re: [HACKERS] [sepgsql 2/3] Add db_schema:search permission checks

2013-04-12 Thread Robert Haas
On Fri, Apr 12, 2013 at 10:42 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas escribió: On Mon, Apr 8, 2013 at 12:28 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Also, the attached function-execute-permission patch is a rebased version. I rethought its event name should be

Re: [HACKERS] [ADMIN] after 9.2.4 patch vacuumdb -avz not analyzing all tables

2013-04-12 Thread Scott Marlowe
Does this behavior only affect the 9.2 branch? Or was it ported to 9.1 or 9.0 or 8.4 as well? On Thu, Apr 11, 2013 at 7:48 PM, Kevin Grittner kgri...@ymail.com wrote: Tom Lane t...@sss.pgh.pa.us wrote: However I've got to say that both of those side-effects of exclusive-lock abandonment

Re: [HACKERS] Analyzing bug 8049

2013-04-12 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Apr 11, 2013 at 1:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: The plan I'm considering is to get this written and committed to HEAD in the next week, so that it can go out in 9.3beta1. After the patch has survived a reasonable amount of beta

Re: [HACKERS] [PATCH] pg_regress and non-default unix socket path

2013-04-12 Thread Robert Haas
On Tue, Apr 9, 2013 at 8:08 AM, Christoph Berg c...@df7cb.de wrote: Debian has been patching pg_regress for years because our default unix socket directory is /var/run/postgresql, but that is not writable by the build user at build time. This used to be a pretty ugly make- patch-make

[HACKERS] How to use EXPLAIN (TIMING)

2013-04-12 Thread Robins Tharakan
Hi, While creating regression tests for EXPLAIN I am (somehow) unable to get (TIMING) option to work with EXPLAIN! I must be doing something stupid but all these options below just didn't work. Could someone point me to the right direction? mpf2=# explain (TIMING) SELECT 1; ERROR: EXPLAIN

Re: [HACKERS] How to use EXPLAIN (TIMING)

2013-04-12 Thread Andres Freund
On 2013-04-12 20:44:02 +0530, Robins Tharakan wrote: Hi, While creating regression tests for EXPLAIN I am (somehow) unable to get (TIMING) option to work with EXPLAIN! I must be doing something stupid but all these options below just didn't work. Could someone point me to the right

Re: [HACKERS] [ADMIN] after 9.2.4 patch vacuumdb -avz not analyzing all tables

2013-04-12 Thread Kevin Grittner
Scott Marlowe scott.marl...@gmail.com wrote: Does this behavior only affect the 9.2 branch? Or was it ported to 9.1 or 9.0 or 8.4 as well? After leaving it on master for a while to see if anyone reported problems in development, I back-patched as far as 9.0 in time for the 9.2.3 (and related)

Re: [HACKERS] [PATCH] pg_regress and non-default unix socket path

2013-04-12 Thread Alvaro Herrera
Robert Haas escribió: On Tue, Apr 9, 2013 at 8:08 AM, Christoph Berg c...@df7cb.de wrote: Debian has been patching pg_regress for years because our default unix socket directory is /var/run/postgresql, but that is not writable by the build user at build time. This used to be a pretty ugly

Re: [HACKERS] Analyzing bug 8049

2013-04-12 Thread Dickson S. Guedes
Em Sex, 2013-04-12 às 10:58 -0400, Tom Lane escreveu: Robert Haas robertmh...@gmail.com writes: On Thu, Apr 11, 2013 at 1:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: The plan I'm considering is to get this written and committed to HEAD in the next week, so that it can go out in 9.3beta1.

Re: [HACKERS] Patch to make pgindent work cleanly

2013-04-12 Thread Bruce Momjian
On Tue, Feb 19, 2013 at 04:50:45PM -0500, Gurjeet Singh wrote: Please find attached the patch for some cleanup and fix bit rot in pgindent script. There were a few problems with the script. .) It failed to use the $ENV{PGENTAB} even if it was set. .) The file it tries to download from

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Andrew Dunstan
On 04/12/2013 10:15 AM, Tom Lane wrote: Sameer Thakur samthaku...@gmail.com writes: The proposed tool tries to make migration faster for tables and indices only by copying their binary data files. There's 0 chance of making that work, because the two databases wouldn't have the same notions

Re: [HACKERS] Analyzing bug 8049

2013-04-12 Thread Tom Lane
Dickson S. Guedes lis...@guedesoft.net writes: In my tests, after ANALYZE _bug_header and _bug_line, the query plan changes and query results returns as expected. Is this a chance that things isn't too bad? No, it just means that in this particular scenario, the bug only manifests if a

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Kevin Grittner
Jeff Janes jeff.ja...@gmail.com wrote: If we're going to have the message, we should make it useful. My biggest question here is not whether we should add this info, but whether it should be DEBUG instead of LOG I like it being LOG.  If it were DEBUG, I don't think anyone would be likely to

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Alvaro Herrera
Andrew Dunstan escribió: On 04/12/2013 10:15 AM, Tom Lane wrote: Sameer Thakur samthaku...@gmail.com writes: The proposed tool tries to make migration faster for tables and indices only by copying their binary data files. There's 0 chance of making that work, because the two databases

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 04/12/2013 10:15 AM, Tom Lane wrote: There's 0 chance of making that work, because the two databases wouldn't have the same notions of committed XIDs. Yeah. Trying to think way outside the box, could we invent some sort of fixup mechanism that

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Andres Freund
On 2013-04-12 12:14:24 -0400, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 04/12/2013 10:15 AM, Tom Lane wrote: There's 0 chance of making that work, because the two databases wouldn't have the same notions of committed XIDs. Yeah. Trying to think way outside the box,

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Pavan Deolasee
On Fri, Apr 12, 2013 at 9:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 04/12/2013 10:15 AM, Tom Lane wrote: There's 0 chance of making that work, because the two databases wouldn't have the same notions of committed XIDs. Yeah. Trying to think

Re: [HACKERS] [PATCH] pg_regress and non-default unix socket path

2013-04-12 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: The hunk that changes the messages might need some thought so that it doesn't cause a translation regression. But in general I see no reason not to do this before we release beta1. It seems safe enough, and changes that reduce the need for packagers

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Bruce Momjian
On Fri, Apr 12, 2013 at 10:22:38PM +0530, Pavan Deolasee wrote: On Fri, Apr 12, 2013 at 9:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 04/12/2013 10:15 AM, Tom Lane wrote: There's 0 chance of making that work, because the two

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Tom Lane
Kevin Grittner kgri...@ymail.com writes: OK, will review that to confirm;but assuming that's right, and nobody else is already working on a fix, I propose to do the following: (1)  Restore the prior behavior of the VACUUM command.  This was only ever intended to be a fix for a serious

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Andres Freund
On 2013-04-12 13:09:02 -0400, Tom Lane wrote: Kevin Grittner kgri...@ymail.com writes: OK, will review that to confirm;but assuming that's right, and nobody else is already working on a fix, I propose to do the following: (1)� Restore the prior behavior of the VACUUM command.� This was

[HACKERS] Object oriented programming language native to EAV

2013-04-12 Thread Sergei Sheinin
Programming language environment whose parser and interpreter are written with plpgsql. Proof of concept prototype has been tested. An object oriented programming language implemented with self-describing Entity Attribute Value model that stores objects and object metadata descriptions. The

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: OK, will review that to confirm;but assuming that's right, and nobody else is already working on a fix, I propose to do the following: (1)  Restore the prior behavior of the VACUUM command.  This was only ever

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-04-12 13:09:02 -0400, Tom Lane wrote: However, we're still thinking too small. I've been wondering whether we couldn't entirely remove the dirty, awful kluges that were installed in the lock manager to kill autovacuum when somebody blocked

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Bruce Momjian
On Wed, Apr 10, 2013 at 11:19:56AM -0700, Jeff Davis wrote: On Wed, 2013-04-10 at 11:01 +0300, Ants Aasma wrote: I think we should first deal with using it for page checksums and if future versions want to reuse some of the code for WAL checksums then we can rearrange the code. Sounds

Re: [HACKERS] [ADMIN] after 9.2.4 patch vacuumdb -avz not analyzing all tables

2013-04-12 Thread Mike Broers
Looks like psql vacuum (verbose, analyze) is not reflecting in pg_stat_user_tables as well in some cases. In this scenario I run the command, it outputs all the deleted pages etc (unlike the vacuumdb -avz analyze that seemed to be skipped in the log), but it does not update pg_stat_user_tables.

Re: [HACKERS] Patch to make pgindent work cleanly

2013-04-12 Thread Gurjeet Singh
On Fri, Apr 12, 2013 at 11:44 AM, Bruce Momjian br...@momjian.us wrote: On Tue, Feb 19, 2013 at 04:50:45PM -0500, Gurjeet Singh wrote: Please find attached the patch for some cleanup and fix bit rot in pgindent script. There were a few problems with the script. .) It failed to use

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Tom Lane
Kevin Grittner kgri...@ymail.com writes: Tom Lane t...@sss.pgh.pa.us wrote: I think that the minimum appropriate fix here is to revert the hunk I quoted, ie take out the suppression of stats reporting and analysis. I'm not sure I understand -- are you proposing that is all we do for both the

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Alvaro Herrera
Tom Lane escribió: Are you saying you intend to revert that whole concept? That'd be okay with me, I think. Otherwise we need some thought about how to inform the stats collector what's really happening. Maybe what we need is to consider table truncation as a separate activity from

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Well, it wouldn't be that hard to replace XIDs with FrozenXID or InvalidXID as appropriate, if you had access to the source database's clog while you did the copying. It just wouldn't be very fast. If you're doing that in a streaming method, it strikes

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: I suppose it would still be faster than a COPY transfer, but I'm not sure it'd be enough faster to justify the work and the additional portability hits you'd be taking. The big win here over a binary COPY is

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Kevin Grittner
[some relevant dropped bits of the thread restored] Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: Jeff Janes jeff.ja...@gmail.com wrote: I propose to do the following: (1)  Restore

Re: [HACKERS] [sepgsql 2/3] Add db_schema:search permission checks

2013-04-12 Thread Kohei KaiGai
2013/4/12 Robert Haas robertmh...@gmail.com: On Fri, Apr 12, 2013 at 10:42 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas escribió: On Mon, Apr 8, 2013 at 12:28 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Also, the attached function-execute-permission patch is a rebased

Re: [HACKERS] [sepgsql 2/3] Add db_schema:search permission checks

2013-04-12 Thread Kohei KaiGai
A problem regarding to validation of sepgsql-regtest policy module is originated by semodule commands that takes root privilege to list up installed policy modules. So, I avoided to use this command in the test_sepgsql script. However, I have an idea that does not raise script fail even if

Re: [HACKERS] Detach/attach table and index data files from one cluster to another

2013-04-12 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: The big win here over a binary COPY is pulling through the indexes as-is as well- without having to rebuild them. [... lots of reasons this is hard ...] I agree that it's quite a bit more difficult, to the point

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Jeff Davis
On Thu, 2013-04-11 at 20:12 +0100, Simon Riggs wrote: So, if we apply a patch like the one attached, we then end up with the WAL checksum using the page checksum as an integral part of its calculation. (There is no increase in code inside WALInsertLock, nothing at all touched in that area).

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-12 Thread Tom Lane
Kevin Grittner kgri...@ymail.com writes: For now what I'm suggesting is generating statistics in all the cases it did before, plus the case where it starts truncation but does not complete it.  The fact that before this patch there were cases where the autovacuum worker was killed, resulting

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Bruce Momjian
On Fri, Apr 12, 2013 at 12:07:36PM -0700, Jeff Davis wrote: (Attached patch is discussion only. Checking checksum in recovery isn't coded at all.) I like it. A few points: * Given that setting the checksum is unconditional in a backup block, do we want to zero the checksum field when

Re: [HACKERS] Patch to make pgindent work cleanly

2013-04-12 Thread Bruce Momjian
On Fri, Apr 12, 2013 at 01:34:49PM -0400, Gurjeet Singh wrote: Can you also improve the output when it dies upon failure to fetch something? Currently the only error message it emits is fetching xyz, and leaves the user confused as to what really the problem was. The only indication of a

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Andres Freund
On 2013-04-11 20:12:59 +0100, Simon Riggs wrote: On 11 April 2013 04:27, Jeff Davis pg...@j-davis.com wrote: On Wed, 2013-04-10 at 20:17 +0100, Simon Riggs wrote: OK, so we have a single combined calculate a checksum for a block function. That uses Jeff's zeroing trick and Ants'

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Bruce Momjian
On Fri, Apr 12, 2013 at 09:28:42PM +0200, Andres Freund wrote: Only point worth discussing is that this change would make backup blocks be covered by a 16-bit checksum, not the CRC-32 it is now. i.e. the record header is covered by a CRC32 but the backup blocks only by 16-bit. That means

Re: [HACKERS] Patch to make pgindent work cleanly

2013-04-12 Thread Gurjeet Singh
On Fri, Apr 12, 2013 at 3:26 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Apr 12, 2013 at 01:34:49PM -0400, Gurjeet Singh wrote: Can you also improve the output when it dies upon failure to fetch something? Currently the only error message it emits is fetching xyz, and leaves the

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Andres Freund
On 2013-04-12 15:31:36 -0400, Bruce Momjian wrote: On Fri, Apr 12, 2013 at 09:28:42PM +0200, Andres Freund wrote: Only point worth discussing is that this change would make backup blocks be covered by a 16-bit checksum, not the CRC-32 it is now. i.e. the record header is covered by a

[HACKERS] Small reduction in memory usage of index relcache entries

2013-04-12 Thread Heikki Linnakangas
While debugging an out-of-memory situation, I noticed that the relcache entry of each index consumes 2k of memory. From a memory dump: ... pg_database_datname_index: 2048 total in 1 blocks; 712 free (0 chunks); 1336 used pg_trigger_tgrelid_tgname_index: 2048 total in 1 blocks; 576

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Heikki Linnakangas
On 12.04.2013 22:31, Bruce Momjian wrote: On Fri, Apr 12, 2013 at 09:28:42PM +0200, Andres Freund wrote: Only point worth discussing is that this change would make backup blocks be covered by a 16-bit checksum, not the CRC-32 it is now. i.e. the record header is covered by a CRC32 but the

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Simon Riggs
On 12 April 2013 21:03, Heikki Linnakangas hlinnakan...@vmware.com wrote: No, the patch has to compute the 16-bit checksum for the page when the full-page image is added to the WAL record. There would otherwise be no need to calculate the page checksum at that point, but only later when the

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Jeff Davis
On Fri, 2013-04-12 at 15:21 -0400, Bruce Momjian wrote: * When we do PageSetChecksumInplace(), we need to be 100% sure that the hole is empty; otherwise the checksum will fail when we re-expand it. It might be worth a memset beforehand just to be sure. Do we write the page holes to the

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Jeff Davis
On Fri, 2013-04-12 at 21:28 +0200, Andres Freund wrote: That means we will have to do the verification for this in ValidXLogRecord() *not* in RestoreBkpBlock or somesuch. Otherwise we won't always recognize the end of WAL correctly. And I am a bit wary of reducing the likelihood of noticing

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Jeff Davis
On Fri, 2013-04-12 at 23:03 +0300, Heikki Linnakangas wrote: I think this is a bad idea. It complicates the WAL format significantly. Simon's patch didn't include the changes to recovery to validate the checksum, but I suspect it would be complicated. And it reduces the error-detection

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Ants Aasma
On Sat, Apr 13, 2013 at 12:38 AM, Jeff Davis pg...@j-davis.com wrote: On Fri, 2013-04-12 at 23:03 +0300, Heikki Linnakangas wrote: I think this is a bad idea. It complicates the WAL format significantly. Simon's patch didn't include the changes to recovery to validate the checksum, but I

Re: [HACKERS] Small reduction in memory usage of index relcache entries

2013-04-12 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: Looking closer at where that memory is spent, a lot of it goes into the FmgrInfo structs in RelationAmInfo. But some of them are outright unused (ambuild, ambuildempty, amcostestimate, amoptions), and a few others hardly are called so

Re: [HACKERS] Hash Join cost estimates

2013-04-12 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: I'm certainly curious about those, but I'm also very interested in the possibility of making NTUP_PER_BUCKET much smaller, or perhaps variable depending on the work_mem setting. Not sure about that. That