Re: [HACKERS] New replication mode: write

2012-01-24 Thread Fujii Masao
On Wed, Jan 25, 2012 at 5:28 AM, Simon Riggs wrote: > On Tue, Jan 24, 2012 at 11:00 AM, Simon Riggs wrote: > >> Yes, it might be too hard, but lets look. > > Your committer has timed out ;-) > > committed write mode only Thanks for the commit! The apply mode is attractive, but I need more t

Re: [HACKERS] [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

2012-01-24 Thread Fujii Masao
On Wed, Jan 25, 2012 at 5:35 AM, Jaime Casanova wrote: > On Tue, Jan 24, 2012 at 3:22 PM, Simon Riggs wrote: >> Add new replication mode synchronous_commit = 'write'. >> Replication occurs only to memory on standby, not to disk, >> so provides additional performance if user wishes to >> reduce du

Re: [HACKERS] PgNext: CFP

2012-01-24 Thread Dave Page
On Tuesday, January 24, 2012, Stefan Kaltenbrunner wrote: > On 01/24/2012 07:36 PM, Dave Page wrote: >> What date & venue? >> >> On Tuesday, January 24, 2012, Joshua D. Drake wrote: >>> >>> Hello, >>> >>> The call for papers for PgNext (the old PgWest/PgEast) is now open: >>> >>> January 19th: Ta

Re: [HACKERS] some longer, larger pgbench tests with various performance-related patches

2012-01-24 Thread Pavan Deolasee
On Wed, Jan 25, 2012 at 2:23 AM, Robert Haas wrote: > Early yesterday morning, I was able to use Nate Boley's test machine > do a single 30-minute pgbench run at scale factor 300 using a variety > of trees built with various patches, and with the -l option added to > track latency on a per-transac

Re: [HACKERS] Group commit, revised

2012-01-24 Thread Simon Riggs
On Fri, Jan 20, 2012 at 10:30 PM, Heikki Linnakangas wrote: > I spent some time cleaning this up. Details below, but here are the > highlights: > > * Reverted the removal of wal_writer_delay > * Removed heuristic on big flushes No contested viewpoints on anything there. > * Doesn't rely on WAL

Re: [HACKERS] pg_upgrade with plpython is broken

2012-01-24 Thread Bruce Momjian
On Fri, Jan 20, 2012 at 07:01:46AM +0200, Peter Eisentraut wrote: > On tor, 2012-01-19 at 17:04 -0500, Bruce Momjian wrote: > > For that reason, I wonder if I should just hard-code the plpython > > rename into the pg_upgrade test in check_loadable_libraries(). > > Yes, I haven't come up with a bet

[HACKERS] Fix for pg_upgrade tablespace function usage

2012-01-24 Thread Bruce Momjian
I have applied the attached patch to git head to fix the new SQL tablespace location function usage in pg_upgrade to properly check cluster version numbers, and a fix missing table alias. I found this problem during testing. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-24 Thread Robert Haas
On Tue, Jan 24, 2012 at 8:13 PM, Tom Lane wrote: >> Well, the bytea experience was IMNSHO a complete disaster (It was >> earlier mentioned that jdbc clients were silently corrupting bytea >> datums) and should be held up as an example of how *not* to do things; > > Yeah.  In both cases, the (propo

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-24 Thread Tom Lane
Merlin Moncure writes: > On Tue, Jan 24, 2012 at 11:55 AM, Robert Haas wrote: >> I do wonder whether we are making a mountain out of a mole-hill here, >> though. If I properly understand the proposal on the table, which >> it's possible that I don't, but if I do, the new format is >> self-identi

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-24 Thread Tatsuo Ishii
> My test was run with synchronous_commit=off, so I didn't expect the > group commit patch to have much of an impact. I included it mostly to > see whether by chance it helped anyway (since it also helps other WAL > flushes, not just commits) or whether it caused any regression. Oh, I see. > One

[HACKERS] Different error messages executing CREATE TABLE or ALTER TABLE to create a column "xmin"

2012-01-24 Thread Vik Reykja
I took my first stab at hacking the sources to fix the bug reported here: http://archives.postgresql.org/pgsql-bugs/2012-01/msg00152.php It's such a simple patch but it took me several hours with Google and IRC and I'm sure I did many things wrong. It seems to work as advertised, though, so I'm

Re: [HACKERS] some longer, larger pgbench tests with various performance-related patches

2012-01-24 Thread Simon Riggs
On Tue, Jan 24, 2012 at 8:53 PM, Robert Haas wrote: > > do a single 30-minute pgbench run at scale factor 300 using a variety Nice A minor but necessary point: Repeated testing of the Group commit patch when you have synch commit off is clearly pointless, so publishing numbers for that without s

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-24 Thread Merlin Moncure
On Tue, Jan 24, 2012 at 11:55 AM, Robert Haas wrote: > I do wonder whether we are making a mountain out of a mole-hill here, > though.  If I properly understand the proposal on the table, which > it's possible that I don't, but if I do, the new format is > self-identifying: when the optimization i

[HACKERS] some longer, larger pgbench tests with various performance-related patches

2012-01-24 Thread Robert Haas
Early yesterday morning, I was able to use Nate Boley's test machine do a single 30-minute pgbench run at scale factor 300 using a variety of trees built with various patches, and with the -l option added to track latency on a per-transaction basis. All tests were done using 32 clients and permane

Re: [HACKERS] New replication mode: write

2012-01-24 Thread Simon Riggs
On Tue, Jan 24, 2012 at 11:00 AM, Simon Riggs wrote: > Yes, it might be too hard, but lets look. Your committer has timed out ;-) committed write mode only --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL Development, 24x7 Support, Training & Services -- Sent vi

Re: [HACKERS] PgNext: CFP

2012-01-24 Thread Stefan Kaltenbrunner
On 01/24/2012 07:36 PM, Dave Page wrote: > What date & venue? > > On Tuesday, January 24, 2012, Joshua D. Drake wrote: >> >> Hello, >> >> The call for papers for PgNext (the old PgWest/PgEast) is now open: >> >> January 19th: Talk submission opens >> April 15th: Talk submission closes >> April 30

Re: [HACKERS] controlling the location of server-side SSL files

2012-01-24 Thread Peter Eisentraut
On tor, 2012-01-19 at 15:44 -0500, Robert Haas wrote: > On Sat, Jan 14, 2012 at 8:40 AM, Peter Eisentraut wrote: > > On mån, 2012-01-02 at 06:32 +0200, Peter Eisentraut wrote: > >> I think I would like to have a set of GUC parameters to control the > >> location of the server-side SSL files. > > >

Re: [HACKERS] PgNext: CFP

2012-01-24 Thread Dave Page
What date & venue? On Tuesday, January 24, 2012, Joshua D. Drake wrote: > > Hello, > > The call for papers for PgNext (the old PgWest/PgEast) is now open: > > January 19th: Talk submission opens > April 15th: Talk submission closes > April 30th: Speaker notification > > Submit: https://www.postgr

Re: [HACKERS] lots of unused variable warnings in assert-free builds

2012-01-24 Thread Tom Lane
Robert Haas writes: > Yes, that's what I meant when I suggested it originally. I'm just not > sure it's any nicer than adding ifdefs for USE_ASSERT_CHECKING. I'm inclined to think that it probably is nicer, just because of less vertical space used. But again, this opinion is contingent on what

Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace

2012-01-24 Thread Robert Haas
On Sun, Jan 22, 2012 at 11:04 AM, Julien Tachoires wrote: > 2011/12/15 Alvaro Herrera : >> >> Uhm, surely you could compare the original toast tablespace to the heap >> tablespace, and if they differ, handle appropriately when creating the >> new toast table?  Just pass down the toast tablespace i

Re: [HACKERS] lots of unused variable warnings in assert-free builds

2012-01-24 Thread Robert Haas
On Tue, Jan 24, 2012 at 1:03 PM, Tom Lane wrote: > I wrote: >> Also, it occurs to me that an intermediate macro >> "PG_USED_FOR_ASSERTS_ONLY" would be a good idea, first because it >> documents *why* you want to mark the variable as possibly unused, >> and second because changing the macro definit

Re: [HACKERS] lots of unused variable warnings in assert-free builds

2012-01-24 Thread Tom Lane
I wrote: > Also, it occurs to me that an intermediate macro > "PG_USED_FOR_ASSERTS_ONLY" would be a good idea, first because it > documents *why* you want to mark the variable as possibly unused, > and second because changing the macro definition would provide an easy way > to check for totally-unu

Re: [HACKERS] lots of unused variable warnings in assert-free builds

2012-01-24 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 24, 2012 at 12:12 PM, Tom Lane wrote: >> Ouch. Is it really true that __attribute__((unused)) disables detection >> of use of uninitialized variables? > Oh, I think maybe I am confused. The downsides of disabling *unused* > variable detection are obviously much

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-24 Thread Robert Haas
On Tue, Jan 24, 2012 at 11:16 AM, Merlin Moncure wrote: >> Our current protocol allocates a 2-byte integer for the purposes of >> specifying the type of each parameter, and another 2-byte integer for >> the purpose of specifying the result type... but only one bit is >> really needed at present: t

[HACKERS] PgNext: CFP

2012-01-24 Thread Joshua D. Drake
Hello, The call for papers for PgNext (the old PgWest/PgEast) is now open: January 19th: Talk submission opens April 15th: Talk submission closes April 30th: Speaker notification Submit: https://www.postgresqlconference.org/talk_types Sincerely, JD -- Command Prompt, Inc. - http://www.comma

Re: [HACKERS] Page Checksums

2012-01-24 Thread Jim Nasby
On Jan 24, 2012, at 9:15 AM, Simon Riggs wrote: > On Tue, Jan 24, 2012 at 2:49 PM, Robert Treat wrote: >>> And yes, I would for sure turn such functionality on if it were present. >>> >> >> That's nice to say, but most people aren't willing to take a 50% >> performance hit. Not saying what we en

Re: [HACKERS] lots of unused variable warnings in assert-free builds

2012-01-24 Thread Robert Haas
On Tue, Jan 24, 2012 at 12:12 PM, Tom Lane wrote: > Robert Haas writes: >> Spraying the code with __attribute__((unused)) is somewhat undesirable >> because it could mask a failure to properly initialize the variable in >> an assert-enabled build. > > Ouch.  Is it really true that __attribute__((

Re: [HACKERS] lots of unused variable warnings in assert-free builds

2012-01-24 Thread Tom Lane
Robert Haas writes: > Spraying the code with __attribute__((unused)) is somewhat undesirable > because it could mask a failure to properly initialize the variable in > an assert-enabled build. Ouch. Is it really true that __attribute__((unused)) disables detection of use of uninitialized variabl

Re: [HACKERS] lots of unused variable warnings in assert-free builds

2012-01-24 Thread Robert Haas
On Sat, Jan 21, 2012 at 1:06 PM, Peter Eisentraut wrote: > So, here are three patches that could solve this issue. > > pg-cassert-unused-attribute.patch, the one I already showed, using > __attribute__((unused). > > pg-cassert-unused-ifdef.patch, using only additional #ifdef > USE_ASSERT_CHECKING.

Re: [HACKERS] Multithread Query Planner

2012-01-24 Thread Robert Haas
On Tue, Jan 24, 2012 at 11:25 AM, Tom Lane wrote: > Robert Haas writes: >> I doubt it.  Almost nothing in the backend is thread-safe.  You can't >> acquire a heavyweight lock, a lightweight lock, or a spinlock. You >> can't do anything that might elog() or ereport().  None of those >> things are

Re: [HACKERS] Multithread Query Planner

2012-01-24 Thread Tom Lane
Robert Haas writes: > I doubt it. Almost nothing in the backend is thread-safe. You can't > acquire a heavyweight lock, a lightweight lock, or a spinlock. You > can't do anything that might elog() or ereport(). None of those > things are reentrant. Not to mention palloc, another extremely fund

Re: [HACKERS] Measuring relation free space

2012-01-24 Thread Jaime Casanova
On Mon, Jan 23, 2012 at 7:18 PM, Noah Misch wrote: > On Mon, Jan 23, 2012 at 04:56:24PM -0300, Alvaro Herrera wrote: >> >> Hm.  Leaf pages hold as much tuples as non-leaf pages, no?  I mean >> for each page element there's a value and a CTID.  In non-leaf those >> CTIDs point to other index pages,

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-24 Thread Merlin Moncure
On Tue, Jan 24, 2012 at 8:26 AM, Robert Haas wrote: > On Mon, Jan 23, 2012 at 5:49 PM, Merlin Moncure wrote: >> I'm not sure that you're getting anything with that user facing >> complexity.  The only realistic case I can see for explicit control of >> wire formats chosen is to defend your applic

Re: [HACKERS] Multithread Query Planner

2012-01-24 Thread Robert Haas
On Mon, Jan 23, 2012 at 2:45 PM, Merlin Moncure wrote: > Yes, but OP is proposing to use multiple threads inside the forked > execution process.  That's a completely different beast.  Many other > databases support parallel execution of a single query and it might > very well be better/easier to d

Re: [HACKERS] Page Checksums

2012-01-24 Thread Simon Riggs
On Tue, Jan 24, 2012 at 2:49 PM, Robert Treat wrote: >> And yes, I would for sure turn such functionality on if it were present. >> > > That's nice to say, but most people aren't willing to take a 50% > performance hit. Not saying what we end up with will be that bad, but > I've seen people get up

Re: Removing freelist (was Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?)

2012-01-24 Thread Simon Riggs
On Mon, Jan 23, 2012 at 4:01 PM, Tom Lane wrote: > The real > problem there is that BufFreelistLock is also used to protect the > clock sweep pointer. Agreed > I think basically we gotta find a way to allow > multiple backends to run clock sweeps concurrently.  Or else fix > things so that the

Re: Removing freelist (was Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?)

2012-01-24 Thread Jeff Janes
On Mon, Jan 23, 2012 at 5:06 AM, Robert Haas wrote: > On Mon, Jan 23, 2012 at 12:12 AM, Tom Lane wrote: >> Robert Haas writes: >>> On Sat, Jan 21, 2012 at 5:29 PM, Jim Nasby wrote: We should also look at having the freelist do something useful, instead of just dropping it completely.

Re: [HACKERS] Page Checksums

2012-01-24 Thread Robert Treat
On Tue, Jan 24, 2012 at 3:02 AM,   wrote: >> * Robert Treat: >> >>> Would it be unfair to assert that people who want checksums but aren't >>> willing to pay the cost of running a filesystem that provides >>> checksums aren't going to be willing to make the cost/benefit trade >>> off that will be a

Re: Removing freelist (was Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?)

2012-01-24 Thread Jeff Janes
On Sat, Jan 21, 2012 at 2:29 PM, Jim Nasby wrote: > On Jan 20, 2012, at 11:54 AM, Heikki Linnakangas wrote: >> So, you're proposing that we remove freelist altogether? Sounds reasonable, >> but that needs to be performance tested somehow. I'm not sure what exactly >> the test should look like,

Re: [HACKERS] Inline Extension

2012-01-24 Thread Robert Haas
On Mon, Jan 23, 2012 at 7:59 PM, Daniel Farina wrote: > Things are still a bit ugly in the more complex cases: consider > PostGIS's linkage against libproj and other libraries.  In order to > cover all cases, I feel that what I need is an optional hook (for the > same of argument, let's say it's a

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-24 Thread Robert Haas
On Mon, Jan 23, 2012 at 5:49 PM, Merlin Moncure wrote: > I'm not sure that you're getting anything with that user facing > complexity.  The only realistic case I can see for explicit control of > wire formats chosen is to defend your application from format changes > in the server when upgrading t

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-24 Thread Robert Haas
On Tue, Jan 24, 2012 at 1:26 AM, Tatsuo Ishii wrote: >> ** pgbench, permanent tables, scale factor 100, 300 s ** >> 1 group-commit-2012-01-21 614.425851 -10.4% >> 8 group-commit-2012-01-21 4705.129896 +6.3% >> 16 group-commit-2012-01-21 7962.131701 +2.0% >> 24 group-commit-2012-01-21 13074.939290

Re: GiST for range types (was Re: [HACKERS] Range Types - typo + NULL string constructor)

2012-01-24 Thread Alexander Korotkov
Hi! New version of patch is attached. On Thu, Dec 22, 2011 at 11:46 AM, Jeff Davis wrote: > A few comments: > > * In range_gist_picksplit, it would be nice to have a little bit more > intuitive description of what's going on with the nonEmptyCount and > nonInfCount numbers. For instance, it appe

Re: [HACKERS] New replication mode: write

2012-01-24 Thread Simon Riggs
On Tue, Jan 24, 2012 at 10:47 AM, Fujii Masao wrote: >>> I'm afraid I could not understand your idea. Could you explain it in >>> more detail? >> >> We either tell libpqwalreceiver about the latch, or we tell >> walreceiver about the socket used by libpqwalreceiver. >> >> In either case we share

Re: [HACKERS] Online base backup from the hot-standby

2012-01-24 Thread Simon Riggs
On Tue, Jan 24, 2012 at 10:54 AM, Simon Riggs wrote: > On Tue, Jan 24, 2012 at 9:51 AM, Fujii Masao wrote: > >>> I'll proceed to commit for this now. >> >> Thanks a lot! > > Can I just check a few things? Just to clarify, not expecting another patch version, just reply here and I can edit. --

Re: [HACKERS] Online base backup from the hot-standby

2012-01-24 Thread Simon Riggs
On Tue, Jan 24, 2012 at 9:51 AM, Fujii Masao wrote: >> I'll proceed to commit for this now. > > Thanks a lot! Can I just check a few things? You say /* +* Update full_page_writes in shared memory and write an +* XLOG_FPW_CHANGE record before resource manager writes cleanup +

Re: [HACKERS] New replication mode: write

2012-01-24 Thread Fujii Masao
On Mon, Jan 23, 2012 at 9:53 PM, Simon Riggs wrote: > On Mon, Jan 23, 2012 at 10:03 AM, Fujii Masao wrote: > To make the walreceiver call WaitLatchOrSocket(), we would need to merge it and libpq_select() into one function. But the former is the backend function and the latter

Re: [HACKERS] WAL Restore process during recovery

2012-01-24 Thread Fujii Masao
On Tue, Jan 24, 2012 at 6:49 PM, Simon Riggs wrote: > On Tue, Jan 24, 2012 at 9:43 AM, Fujii Masao wrote: >> On Tue, Jan 24, 2012 at 12:23 AM, Simon Riggs wrote: >>> On Mon, Jan 23, 2012 at 12:23 PM, Fujii Masao wrote: Why does walrestore need to be invoked even when restore_command is >>>

Re: [HACKERS] Online base backup from the hot-standby

2012-01-24 Thread Fujii Masao
On Mon, Jan 23, 2012 at 10:11 PM, Simon Riggs wrote: > On Mon, Jan 23, 2012 at 10:29 AM, Fujii Masao wrote: >> On Fri, Jan 20, 2012 at 11:34 PM, Simon Riggs wrote: >>> On Fri, Jan 20, 2012 at 12:54 PM, Fujii Masao wrote: Thanks for the review! On Fri, Jan 20, 2012 at 8:15 PM, Sim

Re: [HACKERS] WAL Restore process during recovery

2012-01-24 Thread Simon Riggs
On Tue, Jan 24, 2012 at 9:43 AM, Fujii Masao wrote: > On Tue, Jan 24, 2012 at 12:23 AM, Simon Riggs wrote: >> On Mon, Jan 23, 2012 at 12:23 PM, Fujii Masao wrote: >>> Why does walrestore need to be invoked even when restore_command is >>> not specified? It seems to be useless. We invoke walrecei

Re: [HACKERS] WAL Restore process during recovery

2012-01-24 Thread Fujii Masao
On Tue, Jan 24, 2012 at 12:23 AM, Simon Riggs wrote: > On Mon, Jan 23, 2012 at 12:23 PM, Fujii Masao wrote: >> Why does walrestore need to be invoked even when restore_command is >> not specified? It seems to be useless. We invoke walreceiver only when >> primary_conninfo is specified now. Simila

Re: [HACKERS] Page Checksums

2012-01-24 Thread Florian Weimer
> I would chip in and say that I would prefer sticking to well-known proved > filesystems like xfs/ext4 and let the application do the checksumming. Yes, that's a different way of putting my concern. If you want a proven file system with checksumming (and an fsck), options are really quite limite

Re: [HACKERS] Page Checksums

2012-01-24 Thread jesper
> * Robert Treat: > >> Would it be unfair to assert that people who want checksums but aren't >> willing to pay the cost of running a filesystem that provides >> checksums aren't going to be willing to make the cost/benefit trade >> off that will be asked for? Yes, it is unfair of course, but it's