Re: [HACKERS] Scaling shared buffer eviction

2014-10-09 Thread Amit Kapila
On Fri, Oct 10, 2014 at 1:08 AM, Andres Freund wrote: > On 2014-10-09 16:01:55 +0200, Andres Freund wrote: > > > > I don't think OLTP really is the best test case for this. Especially not > > pgbench with relatilvely small rows *and* a uniform distribution of > > access. > > > > Try parallel COPY

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-09 Thread Kyotaro HORIGUCHI
Hmm.. Sorry for my stupidity. > Why is that necessary? It seems really rather wrong to make > BIO_set_retry_write() dependant on ProcDiePending? Especially as, at > least in my testing, it's not even required because the be_tls_write() > can just check the error properly? I mistook the previous c

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread Craig Ringer
On 10/10/2014 04:16 AM, MauMau wrote: > From: "Craig Ringer" >> It'd be interesting and useful to run this test on a debug build of >> PostgreSQL, i.e. one compiled against the debug version of the C library >> and with full debuginfo not just minimal .pdb. > > Although I'm not sure the user can

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-09 Thread Amit Kapila
On Wed, Oct 8, 2014 at 7:05 PM, Andres Freund wrote: > > Hi, > > Attached you can find the next version of my LW_SHARED patchset. Now > that atomics are committed, it seems like a good idea to also add their > raison d'ĂȘtre. > > Since the last public version I have: > * Addressed lots of Amit's co

Re: [HACKERS] schema-only -n option in pg_restore fails

2014-10-09 Thread Erik Rijkers
On Thu, October 9, 2014 23:19, Josh Berkus wrote: > All, [dump/restore -n bug] Perhaps this (from five years ago) can be fixed too (esp. if only a doc-fix): http://www.postgresql.org/message-id/4833.156.83.1.81.1240955642.squir...@webmail.xs4all.nl It's not the same problem but also a failu

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 1:13 PM, Peter Geoghegan wrote: >> Can you write >> a separate patch to use SortSupport for B-tree index builds, please? >> Eliminating the FunctionCallInfoData overhead should shave off some some >> cycles from every index build. > > I'll look into it. Hopefully an effort t

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-09 Thread Gavin Flower
On 10/10/14 12:38, Jim Nasby wrote: On 10/8/14, 5:51 PM, Peter Geoghegan wrote: On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittner wrote: >Although the last go-around does suggest that there is at least one >point of difference on the semantics. You seem to want to fire the >BEFORE INSERT trigger

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-09 Thread Jim Nasby
On 10/8/14, 5:51 PM, Peter Geoghegan wrote: On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittner wrote: >Although the last go-around does suggest that there is at least one >point of difference on the semantics. You seem to want to fire the >BEFORE INSERT triggers before determining whether this wil

Re: [HACKERS] Build (definition?) errors - in bootstrap

2014-10-09 Thread Michael Paquier
On Fri, Oct 10, 2014 at 8:11 AM, Tom Lane wrote: > Lou Picciano writes: >> Having just git pulled from orgin/master: >> $ ./configure >> $ make >> [ fails ] > > The buildfarm doesn't seem unhappy, so I doubt there's anything wrong > with the code as such. Try "make clean" or even "make distclean

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Jim Nasby
On 10/9/14, 4:03 PM, Alvaro Herrera wrote: Bruce Momjian wrote: I agree this is a serious problem. We have discussed various options, but have not decided on anything. The TODO list has: https://wiki.postgresql.org/wiki/Todo Improve setting of visibility map bits for read-on

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-09 Thread Jim Nasby
On 10/9/14, 4:57 PM, Andres Freund wrote: If you modify either, you better grep for them... I don't think that's going to happen anyway. Requiring it during startup would mean exposing SHARED_LOCK_MASK outside of lwlock.c which'd be ugly. We could possibly stick a StaticAssert() someplace in lwlo

Re: [HACKERS] Build (definition?) errors - in bootstrap

2014-10-09 Thread Tom Lane
Lou Picciano writes: > Having just git pulled from orgin/master: > $ ./configure > $ make > [ fails ] The buildfarm doesn't seem unhappy, so I doubt there's anything wrong with the code as such. Try "make clean" or even "make distclean" and rebuild. Also, if your computer's clock is or was badl

[HACKERS] Obsolete reference to _bt_tuplecompare() within tuplesort.c

2014-10-09 Thread Peter Geoghegan
I found a reference made obsolete by commit 9e85183b, which is from way back in 2000. comparetup_index_btree() says: /* * This is similar to _bt_tuplecompare(), but we have already done the * index_getattr calls for the first column, and we need to keep track of * whether any null fields are pres

[HACKERS] Build (definition?) errors - in bootstrap

2014-10-09 Thread Lou Picciano
Having just git pulled from orgin/master: $ ./configure $ make Mileage: ... make -C bootstrap all make[3]: Entering directory `/path/to/postgresql/src/backend/bootstrap' gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-09 Thread Andres Freund
On 2014-10-09 16:52:46 -0500, Jim Nasby wrote: > On 10/8/14, 8:35 AM, Andres Freund wrote: > >+#define EXCLUSIVE_LOCK (((uint32) 1) << (31 - 1)) > >+ > >+/* Must be greater than MAX_BACKENDS - which is 2^23-1, so we're fine. */ > >+#define SHARED_LOCK_MASK (~EXCLUSIVE_LOCK) > > There should at lea

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-09 Thread Jim Nasby
On 10/8/14, 8:35 AM, Andres Freund wrote: +#define EXCLUSIVE_LOCK (((uint32) 1) << (31 - 1)) + +/* Must be greater than MAX_BACKENDS - which is 2^23-1, so we're fine. */ +#define SHARED_LOCK_MASK (~EXCLUSIVE_LOCK) There should at least be a comment where we define MAX_BACKENDS about the relati

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Andres Freund
On 2014-10-09 18:16:46 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2014-10-09 18:03:00 -0300, Alvaro Herrera wrote: > > > Bruce Momjian wrote: > > > > > > > I agree this is a serious problem. We have discussed various options, > > > > but have not decided on anything. The TODO lis

Re: [HACKERS] schema-only -n option in pg_restore fails

2014-10-09 Thread Josh Berkus
All, Crossing this over to -hackers because it's stopped being a bug and is now a TODO item. See below. For those not on pgsql-bugs, I've quoted the full bug report below my proposal. On 10/09/2014 12:36 PM, Josh Berkus wrote: > Summary: pg_restore -n attempts to restore objects to pg_catalog s

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Alvaro Herrera
Kevin Grittner wrote: > Wouldn't we get substantially the same thing just by counting tuple > inserts toward the autovacuum vacuum threshold? I mean, it unless > the table is due for wraparound prevention autovacuum, it will only > visit pages that don't have the all-visible bit set, right? And

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-10-09 18:03:00 -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > I agree this is a serious problem. We have discussed various options, > > > but have not decided on anything. The TODO list has: > > > > > > https://wiki.postgresql.org/wiki/Todo > >

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Kevin Grittner
Alvaro Herrera wrote: > Bruce Momjian wrote: >> I agree this is a serious problem. We have discussed various options, >> but have not decided on anything. The TODO list has: >> >> https://wiki.postgresql.org/wiki/Todo >> >> Improve setting of visibility map bits for read-only and insert

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Andres Freund
On 2014-10-09 18:03:00 -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > I agree this is a serious problem. We have discussed various options, > > but have not decided on anything. The TODO list has: > > > > https://wiki.postgresql.org/wiki/Todo > > > > Improve setting of visib

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Andres Freund
On 2014-10-09 17:02:02 -0400, Robert Haas wrote: > On Thu, Oct 9, 2014 at 3:53 PM, Andres Freund wrote: > >> OK, here's an attempt at a real patch for that. I haven't perf-tested > >> this. > > > > Neato. With a really trivial SELECT: > > > > before: > > tps = 28150.794776 (excluding connections

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Alvaro Herrera
Bruce Momjian wrote: > I agree this is a serious problem. We have discussed various options, > but have not decided on anything. The TODO list has: > > https://wiki.postgresql.org/wiki/Todo > > Improve setting of visibility map bits for read-only and insert-only > workloads >

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Robert Haas
On Thu, Oct 9, 2014 at 3:53 PM, Andres Freund wrote: >> OK, here's an attempt at a real patch for that. I haven't perf-tested this. > > Neato. With a really trivial SELECT: > > before: > tps = 28150.794776 (excluding connections establishing) > after: > tps = 29978.767703 (excluding connections e

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Bruce Momjian
On Thu, Oct 9, 2014 at 02:34:17PM +0400, Alexey Bashtanov wrote: > Hello! > > Autovacuum daemon performs vacuum when the number of rows > updated/deleted (n_dead_tuples) reaches some threshold. > Similarly it performs analyze when the number of rows changed in any > way (incl. inserted). > When a

Re: [HACKERS] pg_upgrade, locale and encoding

2014-10-09 Thread Bruce Momjian
On Tue, Oct 7, 2014 at 03:52:24PM +0300, Heikki Linnakangas wrote: > While looking at bug #11431, I noticed that pg_upgrade still seems > to think that encoding and locale are cluster-wide properties. We > got per-database locale support in 8.4, and encoding has been > per-database much longer tha

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-10-09 Thread Kevin Grittner
Tomas Vondra wrote: > On 9.10.2014 16:55, Kevin Grittner wrote: >> I've tried various other tests using \timing rather than EXPLAIN, and >> the patched version looks even better in those cases. I have seen up >> to 4x the performance for a query using the patched version, higher >> variability in

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread MauMau
From: "Andres Freund" What precisely do you mean with "Intel64"? 64bit x86 or Itanium? 64-bit x86, i.e. x86-64. Also, what's the precise workload? Can you reproduce the problem? IIUC, each client inserts 1000 records into one table, then repeats updating all those records. I'll ask him

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 12:51 PM, Heikki Linnakangas wrote: > Oh, I didn't realize we don't do that already! I'm surprised, I would've > expected index build to have been the first thing we'd use the SortSupport > stuff in. The thing is that the most compelling numbers for sortsupport (plus the re

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread MauMau
From: "Craig Ringer" It'd be interesting and useful to run this test on a debug build of PostgreSQL, i.e. one compiled against the debug version of the C library and with full debuginfo not just minimal .pdb. Although I'm not sure the user can do this now, I'll ask him anyway. How were the s

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Andres Freund
On 2014-10-09 15:01:19 -0400, Robert Haas wrote: > On Thu, Oct 9, 2014 at 8:20 AM, Andres Freund wrote: > > On 2014-10-09 08:18:18 -0400, Robert Haas wrote: > >> On Thu, Oct 9, 2014 at 5:34 AM, Andres Freund > >> wrote: > >> > Interesting - in my local profile AtStart_Inval() is more pronounced

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Heikki Linnakangas
On 10/09/2014 10:25 PM, Peter Geoghegan wrote: On Thu, Oct 9, 2014 at 12:14 PM, Heikki Linnakangas wrote: B-tree index build uses tuplesort.c. What's missing? I don't think that all that much is missing. Tuplesort expects to work with an index scankey when sorting B-Tree tuples. There needs t

Re: [HACKERS] pgbench throttling latency limit

2014-10-09 Thread Fabien COELHO
One thing bothers me with the log format. Here's an example: 0 81 4621 0 1412881037 912698 3005 0 82 6173 0 1412881037 914578 4304 0 83 skipped 0 1412881037 914578 5217 0 83 skipped 0 1412881037 914578 5099 0 83 4722 0 1412881037 916203 3108 0 84 4142 0 1412881037 918023 2333 0 85 2465

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-09 Thread Robert Haas
On Mon, Oct 6, 2014 at 8:15 PM, Peter Eisentraut wrote: > On Thu, 2014-10-02 at 21:18 -0400, Robert Haas wrote: >> > If none of this gets us closer to an answer, I can try to produce a >> > patch that produces more details for such failures. >> >> A test that fails for no reason that can be gleane

Re: [HACKERS] Scaling shared buffer eviction

2014-10-09 Thread Andres Freund
On 2014-10-09 16:01:55 +0200, Andres Freund wrote: > On 2014-10-09 18:17:09 +0530, Amit Kapila wrote: > > On Fri, Sep 26, 2014 at 7:04 PM, Robert Haas wrote: > > > > > > On another point, I think it would be a good idea to rebase the > > > bgreclaimer patch over what I committed, so that we have a

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 12:14 PM, Heikki Linnakangas wrote: > B-tree index build uses tuplesort.c. What's missing? I don't think that all that much is missing. Tuplesort expects to work with an index scankey when sorting B-Tree tuples. There needs to be something like a reverse lookup of the sorts

Re: [HACKERS] pgbench throttling latency limit

2014-10-09 Thread Heikki Linnakangas
On 10/05/2014 10:43 AM, Fabien COELHO wrote: Hello Heikki, Here are new patches, again the first one is just refactoring, and the second one contains this feature. I'm planning to commit the first one shortly, and the second one later after people have had a chance to look at it. I looked at

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Heikki Linnakangas
On 10/09/2014 09:59 PM, Peter Geoghegan wrote: My concern is that if we only get it committed in the last commitfest, we may run out of time to make sortsupport work for B-Tree index builds. That's where the sortsupport for text stuff will be really useful. B-tree index build uses tuplesort.c.

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Robert Haas
On Thu, Oct 9, 2014 at 8:20 AM, Andres Freund wrote: > On 2014-10-09 08:18:18 -0400, Robert Haas wrote: >> On Thu, Oct 9, 2014 at 5:34 AM, Andres Freund wrote: >> > Interesting - in my local profile AtStart_Inval() is more pronounced >> > than AfterTriggerBeginQuery(). I've quickly and in a ugly

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-09 Thread Peter Geoghegan
On Mon, Oct 6, 2014 at 11:53 AM, Peter Geoghegan wrote: > On Mon, Oct 6, 2014 at 11:27 AM, Robert Haas wrote: >> Well, really, I was just suggesting that I can spend more time on the >> patch, but not immediately. > > We haven't really talked about the idea of the HyperLogLog-based abort > mechan

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-09 Thread Peter Geoghegan
On Wed, Oct 8, 2014 at 6:29 AM, Simon Riggs wrote: > Lets see the results of running a COPY please. Not exactly sure what you mean here. A concurrent COPY? -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://ww

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-10-09 Thread Tomas Vondra
On 9.10.2014 16:55, Kevin Grittner wrote: > > I've tried various other tests using \timing rather than EXPLAIN, and > the patched version looks even better in those cases. I have seen up > to 4x the performance for a query using the patched version, higher > variability in run time without the patc

Re: [HACKERS] Expose options to explain? (track_io_timing)

2014-10-09 Thread Jeff Janes
On Thu, Oct 9, 2014 at 10:17 AM, Joshua D. Drake wrote: > > Salut! > > Fellow volunteers, I request assistance in understanding the following: > > When I explain a query I can get the following information: > > >| "I/O Read Time": 0.000, >| "I/O Write Time": 0.

Re: [HACKERS] replicating DROP commands across servers

2014-10-09 Thread Alvaro Herrera
Jim Nasby wrote: > On 10/6/14, 11:24 PM, Robert Haas wrote: > > Offlist. > > >>FWIW, I've run into situations more than once in userspace where I need a > >>way to properly separate schema and object name. Generally I can make do > >>using reg* casts and then hitting catalog tables, but it'd be n

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > For embedded devices and similar small-scale systems, I can see Tom's > > point. At the same time, I would expect those to require sufficient > > configuration that also setting log_checkpoints to 'off' wouldn't be a > > huge deal

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Tom Lane
Stephen Frost writes: > * Andres Freund (and...@2ndquadrant.com) wrote: >> On 2014-10-09 09:44:09 -0400, Tom Lane wrote: >>> -1. Every time we've turned on default logging of routine events, >>> there's been pushback and it was eventually turned off again as log spam. >> We're talking about 2 lo

Re: [HACKERS] replicating DROP commands across servers

2014-10-09 Thread Jim Nasby
On 10/6/14, 11:24 PM, Robert Haas wrote: Offlist. FWIW, I've run into situations more than once in userspace where I need a way to properly separate schema and object name. Generally I can make do using reg* casts and then hitting catalog tables, but it'd be nice if there was an easier way. S

Re: [HACKERS] alter user set local_preload_libraries.

2014-10-09 Thread Fujii Masao
On Mon, Sep 15, 2014 at 1:33 AM, Tom Lane wrote: > Peter Eisentraut writes: >> On 9/1/14 7:51 AM, Kyotaro HORIGUCHI wrote: >>> The attached patch simply changes the context for local_... to >>> PGC_USERSET and edits the doc. > >> I had this ready to commit, but then > >> Invent PGC_SU_BACKEND

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2014-10-09 09:44:09 -0400, Tom Lane wrote: > > Stephen Frost writes: > > > Yeah, I agree with this- it's extremely useful information and it's > > > really not that verbose in general.. > > > > -1. Every time we've turned on default logging of

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Andres Freund
On 2014-10-09 09:44:09 -0400, Tom Lane wrote: > Stephen Frost writes: > > * Andres Freund (and...@2ndquadrant.com) wrote: > >> How about flipping the default for log_checkpoints instead? There really > >> isn't a good reason for having it disabled by default. > > > Yeah, I agree with this- it's e

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Marti Raudsepp
On Thu, Oct 2, 2014 at 4:21 PM, Michael Banck wrote: > we have seen repeatedly that users can be confused about why PostgreSQL > is not shutting down even though they requested it. Usually, this is > because `log_checkpoints' is not enabled and the final checkpoint is > being written, delaying sh

[HACKERS] Expose options to explain? (track_io_timing)

2014-10-09 Thread Joshua D. Drake
Salut! Fellow volunteers, I request assistance in understanding the following: When I explain a query I can get the following information: | "I/O Read Time": 0.000, | "I/O Write Time": 0.000 I know why it is 0. My question is this, can we expose it to explai

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Jeff Janes
On Wed, Oct 8, 2014 at 10:52 AM, Michael Banck wrote: > Hi, > > Am Samstag, den 04.10.2014, 15:05 -0500 schrieb Jim Nasby: > > On 10/4/14, 1:21 PM, Jeff Janes wrote: > > > On Thu, Oct 2, 2014 at 6:21 AM, Michael Banck wrote: > > > we have seen repeatedly that users can be confused about why >

Re: [HACKERS] Corporate and Individual Contributor License Agreements (CLAs)

2014-10-09 Thread Joshua D. Drake
Arcadiy, You may want to refer them to the license itself which will make it very easy for them to understand why the Contributor License Agreement is not required: PostgreSQL is released under the PostgreSQL License, a liberal Open Source license, similar to the BSD or MIT licenses. Post

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-10-09 Thread Rahila Syed
Hello, Thank you for review. >1) I don't think it's a good idea to put the full page write compression into struct XLogRecord. Full page write compression information can be stored in varlena struct of compressed blocks as done for toast data in pluggable compression support patch. If I unde

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread Andres Freund
On 2014-10-09 22:47:48 +0900, MauMau wrote: > Hello, > > One user reported a hang problem with 9.4 beta2 on Windows. The PostgreSQL > is 64-bit version. I couldn't find the cause, but want to solve the > problem. Could you help with this? > > I heard that the user had run 16 concurrent psql se

Re: [HACKERS] Proposal for better support of time-varying timezone abbreviations

2014-10-09 Thread Tom Lane
Chris Bandy writes: > On Tue, Oct 7, 2014 at 5:05 PM, Tom Lane wrote: >> + /* use strncmp so that we match truncated tokens */ >> result = strncmp(key, position->token, TOKMAXLEN); > In your proposal you wanted to remove "crufty code that deals with > non-null-terminated token strings". Is this

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread Craig Ringer
On 10/09/2014 09:47 PM, MauMau wrote: > > I heard that the user had run 16 concurrent psql sessions which executes > INSERT and UPDATE statements, which is a write-intensive stress test. > He encountered the hang phenomenon twice, one of which occured several > hours after the start of the test,

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-10-09 Thread Kevin Grittner
Heikki Linnakangas wrote: > On 10/02/2014 03:20 AM, Kevin Grittner wrote: >> My only concern from the benchmarks is that it seemed like there >> was a statistically significant increase in planning time: >> >> unpatched plan time average: 0.450 ms >> patched plan time average: 0.536 ms >> >> That

Re: [HACKERS] Proposal for better support of time-varying timezone abbreviations

2014-10-09 Thread Chris Bandy
On Tue, Oct 7, 2014 at 5:05 PM, Tom Lane wrote: typedef struct > { > ! char token[TOKMAXLEN + 1]; /* now always null-terminated */ > char type; > ! int32 value; > } datetkn; Being entirely new to this code, "now" makes me think of the "current timestamp". I think this word can be remove

Re: [HACKERS] Scaling shared buffer eviction

2014-10-09 Thread Amit Kapila
On Thu, Oct 9, 2014 at 7:31 PM, Andres Freund wrote: > > On 2014-10-09 18:17:09 +0530, Amit Kapila wrote: > > On Fri, Sep 26, 2014 at 7:04 PM, Robert Haas wrote: > > > > > > On another point, I think it would be a good idea to rebase the > > > bgreclaimer patch over what I committed, so that we h

Re: [HACKERS] Scaling shared buffer eviction

2014-10-09 Thread Andres Freund
On 2014-10-09 18:17:09 +0530, Amit Kapila wrote: > On Fri, Sep 26, 2014 at 7:04 PM, Robert Haas wrote: > > > > On another point, I think it would be a good idea to rebase the > > bgreclaimer patch over what I committed, so that we have a > > clean patch against master to test with. > > Please fin

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Tom Lane
Stephen Frost writes: > * Andres Freund (and...@2ndquadrant.com) wrote: >> How about flipping the default for log_checkpoints instead? There really >> isn't a good reason for having it disabled by default. > Yeah, I agree with this- it's extremely useful information and it's > really not that ver

[HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-09 Thread MauMau
Hello, One user reported a hang problem with 9.4 beta2 on Windows. The PostgreSQL is 64-bit version. I couldn't find the cause, but want to solve the problem. Could you help with this? I heard that the user had run 16 concurrent psql sessions which executes INSERT and UPDATE statements, w

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2014-10-02 15:21:48 +0200, Michael Banck wrote: > > I've attached a trivial patch for this, should it be added to the next > > commitfest? > > How about flipping the default for log_checkpoints instead? There really > isn't a good reason for hav

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-10-09 Thread MauMau
From: "Simon Riggs" I hope we can get pgAudit in as a module for 9.5. I also hope that it will stimulate the requirements/funding of further work in this area, rather than squash it. My feeling is we have more examples of feature sets that grow over time (replication, view handling, hstore/JSONB

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Andres Freund
On 2014-10-02 15:21:48 +0200, Michael Banck wrote: > Hi, > > we have seen repeatedly that users can be confused about why PostgreSQL > is not shutting down even though they requested it. Usually, this is > because `log_checkpoints' is not enabled and the final checkpoint is > being written, delay

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Robert Haas
On Wed, Oct 8, 2014 at 1:52 PM, Michael Banck wrote: > Looking at it from a DBA perspective, this would indeed be better, yes. > > However, I see a few issues with that: > > 1. If you are using an init script (or another wrapper around pg_ctl), > you don't get any of its output it seems. > > 2. Ha

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Andres Freund
On 2014-10-09 08:18:18 -0400, Robert Haas wrote: > On Thu, Oct 9, 2014 at 5:34 AM, Andres Freund wrote: > > Interesting - in my local profile AtStart_Inval() is more pronounced > > than AfterTriggerBeginQuery(). I've quickly and in a ugly fashion hacked > > AtStart_Inval() out of readonly queries

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Robert Haas
On Thu, Oct 9, 2014 at 5:34 AM, Andres Freund wrote: > Interesting - in my local profile AtStart_Inval() is more pronounced > than AfterTriggerBeginQuery(). I've quickly and in a ugly fashion hacked > AtStart_Inval() out of readonly queries ontop of your patch. Together > that yields a ~3.5% perfo

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-09 Thread Andres Freund
On 2014-10-09 14:06:35 +0900, Kyotaro HORIGUCHI wrote: > Hello, simplly inhibit set retry flag when ProcDiePending in > my_sock_write seems enough. > > But it returns with SSL_ERROR_SYSCALL not SSL_ERROR_WANT_WRITE so > I modified the patch 4 as the attached patch. Why is that necessary? It seems

[HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-09 Thread Alexey Bashtanov
Hello! Autovacuum daemon performs vacuum when the number of rows updated/deleted (n_dead_tuples) reaches some threshold. Similarly it performs analyze when the number of rows changed in any way (incl. inserted). When a table is mostly insert-only, its visibility map is not updated as vacuum th

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-10-09 Thread furuyao
> >>> If we remove --fsync-interval, resoponse time to user will not be > delay. > >>> Although, fsync will be executed multiple times in a short period. > >>> And there is no way to solve the problem without --fsync-interval, > >>> what > >> should we do about it? > >> > >> I'm sorry, I didn't und

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-09 Thread Andres Freund
On 2014-10-08 13:52:14 -0400, Robert Haas wrote: > On Sun, Jun 29, 2014 at 9:12 PM, Tom Lane wrote: > > Meh. Even "SELECT 1" is going to be doing *far* more pallocs than that to > > get through raw parsing, parse analysis, planning, and execution startup. > > If you can find a few hundred pallocs

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 1:56 AM, Peter Geoghegan wrote: > My point is that people are not really inclined to use an alias in > UPDATEs in general when referring to the target. The thing that seems > special (and worthy of special qualification) is the reference to what > you call the "incoming data

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 1:41 AM, Simon Riggs wrote: > YES, which is why I specifically requested the ability to reference > "the incoming data". My point is that people are not really inclined to use an alias in UPDATEs in general when referring to the target. The thing that seems special (and wor

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 1:33 AM, Marti Raudsepp wrote: > ROW and COALESCE behave almost like functions: they operate on any > expression or value you pass to them. Okay, then like CONFLICTING() is like many of the XML expressions. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Simon Riggs
On 9 October 2014 09:11, Peter Geoghegan wrote: >> You may also wish to support the AS keyword, as MERGE does to make the >> above even more clear. >> >> e.g. SET col = EXISTING.col + NEW.col > > That's less clear, IMV. EXISTING.col is col - the very same Var. So > why qualify that it's the exist

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Marti Raudsepp
On Thu, Oct 9, 2014 at 11:11 AM, Peter Geoghegan wrote: > On Thu, Oct 9, 2014 at 12:38 AM, Simon Riggs wrote: >> Do not use CONFLICTING() which looks like it is a function. > > So is ROW(). Or COALESCE(). ROW and COALESCE behave almost like functions: they operate on any expression or value you

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Peter Geoghegan
On Thu, Oct 9, 2014 at 12:38 AM, Simon Riggs wrote: > Do not use CONFLICTING() which looks like it is a function. So is ROW(). Or COALESCE(). > Instead, use a row qualifier, such as NEW, OLD etc to reference values > from the incoming data > e.g. CONFLICTING.value rather than CONFLICTING(value)

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-09 Thread Michael Paquier
On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas wrote: > 1. Where do the FF files come from? In 9.2, FF-segments are not supposed to > created, ever. > > I think we should add a check in walreceiver, to throw an error if the > master sends an invalid WAL pointer, pointing to an FF segment. Atta

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-09 Thread Simon Riggs
On 9 October 2014 07:27, Peter Geoghegan wrote: > Please be more specific. Do not use CONFLICTING() which looks like it is a function. Instead, use a row qualifier, such as NEW, OLD etc to reference values from the incoming data e.g. CONFLICTING.value rather than CONFLICTING(value) Do not use