Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Heikki Linnakangas
On 09/28/2014 11:31 PM, Peter Geoghegan wrote: On Sun, Sep 28, 2014 at 1:17 PM, Simon Riggs si...@2ndquadrant.com wrote: MERGE INTO tab USING VALUES ('foo') WHEN NOT MATCHED THEN INSERT (colB) WHEN MATCHED THEN UPDATE SET colB = NEW.p1 and throwing ERROR: full syntax for MERGE not

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Andres Freund
On 2014-09-29 09:51:45 +0300, Heikki Linnakangas wrote: That said, it would be handy if the syntax was closer to MERGE. Aside from the concurrency issues, it does the same thing, right? So how about making the syntax identical to MERGE, except for swapping the MERGE keyword with e.g. UPSERT?

Re: [HACKERS] Add generate_series(numeric, numeric)

2014-09-29 Thread Michael Paquier
Hi, Nice patch! And welcome here. On Mon, Sep 29, 2014 at 12:42 PM, Платон Малюгин malugi...@gmail.com wrote: Could you help to find mistakes? This implementation is rather broken, particularly when thinking that this code could be used with a negative step... I also see no point in saving

Re: [HACKERS] Collation-aware comparisons in GIN opclasses

2014-09-29 Thread Heikki Linnakangas
On 09/15/2014 06:28 PM, Alexander Korotkov wrote: Hackers, some GIN opclasses uses collation-aware comparisons while they don't need to do especially collation-aware comparison. Examples are text[] and hstore opclasses. Hmm. It would be nice to use the index for inequality searches, at least

Re: [HACKERS] Collation-aware comparisons in GIN opclasses

2014-09-29 Thread Oleg Bartunov
On Mon, Sep 29, 2014 at 11:48 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 09/15/2014 06:28 PM, Alexander Korotkov wrote: Hackers, some GIN opclasses uses collation-aware comparisons while they don't need to do especially collation-aware comparison. Examples are text[] and

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Simon Riggs
On 28 September 2014 21:31, Peter Geoghegan p...@heroku.com wrote: On Sun, Sep 28, 2014 at 1:17 PM, Simon Riggs si...@2ndquadrant.com wrote: MERGE INTO tab USING VALUES ('foo') WHEN NOT MATCHED THEN INSERT (colB) WHEN MATCHED THEN UPDATE SET colB = NEW.p1 and throwing ERROR: full syntax

Re: [HACKERS] Replication identifiers, take 3

2014-09-29 Thread Andres Freund
On 2014-09-27 12:11:16 -0400, Steve Singer wrote: On 09/26/2014 06:05 PM, Andres Freund wrote: On 2014-09-26 14:57:12 -0400, Robert Haas wrote: Sure, it'll possibly not be trivial to move them elsewhere. On the other hand, the padding bytes have been unused for 8+ years without somebody

[HACKERS] initdb -S and tablespaces

2014-09-29 Thread Abhijit Menon-Sen
Hi. I just noticed that initdb -S (Safely write all database files to disk and exit) does (only) the following in perform_fsync: pre_sync_fname(pdir, true); walkdir(pg_data, pre_sync_fname); fsync_fname(pdir, true); walkdir(pg_data, fsync_fname); walkdir() reads the directory

Re: [HACKERS] [v9.5] Custom Plan API

2014-09-29 Thread Kouhei Kaigai
On Wed, Sep 17, 2014 at 7:40 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: At this moment, I revised the above portion of the patches. create_custom_plan() was modified to call PlanCustomPath callback next to the initialization of tlist and clauses. It's probably same as what you

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Simon Riggs
On 29 September 2014 02:20, Peter Geoghegan p...@heroku.com wrote: On Sun, Sep 28, 2014 at 6:15 PM, Gavin Flower gavinflo...@archidevsys.co.nz wrote: What I have a problem with is using the MERGE syntax to match people's preexisting confused ideas about what MERGE does. If we do that, it'll

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-09-29 Thread Andres Freund
On 2014-09-28 19:51:36 +0100, Simon Riggs wrote: On 27 September 2014 09:29, Andres Freund and...@anarazel.de wrote: On 2014-09-27 10:23:33 +0300, Heikki Linnakangas wrote: This patch has gotten a fair amount of review, and has been rewritten once during the commitfest. I think it's pretty

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Simon Riggs
On 29 September 2014 08:02, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-29 09:51:45 +0300, Heikki Linnakangas wrote: That said, it would be handy if the syntax was closer to MERGE. Aside from the concurrency issues, it does the same thing, right? So how about making the syntax

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Craig Ringer
On 09/29/2014 05:10 PM, Simon Riggs wrote: Please go to some trouble to tidy things up so we have clarity that *we* can see and decide for ourselves whether or not you are correct. Are you suggesting a wiki page to document the issues, discussions around each issue, etc? A summary mail?

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-09-29 Thread Andres Freund
On 2014-09-29 11:13:43 +0200, Andres Freund wrote: I doubt it'll be hard to construct a case where it'll show. My first try of using a pgbench scale 100, -M prepared, -cj8 with a custom file with 1 write and 5 read transaction yielded the following on my laptop: Hm. On second thought that

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-29 Thread David Rowley
On Mon, Sep 29, 2014 at 2:41 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-28 17:32:21 +1300, David Rowley wrote: My understanding of foreign keys is that any pending foreign key triggers will be executed just before the query completes, so we should only ever encounter

Re: [HACKERS] initdb -S and tablespaces

2014-09-29 Thread Andres Freund
On 2014-09-29 14:09:01 +0530, Abhijit Menon-Sen wrote: Hi. I just noticed that initdb -S (Safely write all database files to disk and exit) does (only) the following in perform_fsync: pre_sync_fname(pdir, true); walkdir(pg_data, pre_sync_fname); fsync_fname(pdir, true);

Re: [HACKERS] initdb -S and tablespaces

2014-09-29 Thread Abhijit Menon-Sen
At 2014-09-29 11:54:10 +0200, and...@2ndquadrant.com wrote: Note that the perform_fsync() *was* ok for its original purpose in initdb. At the end of initdb there's no relevant tablespaces. But if used *after* pg_upgrade, that's not necessarily the case. Right. So, since I'm writing a

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-09-29 Thread furuyao
On 09/05/2014 08:51 AM, furu...@pm.nttdata.co.jp wrote: Thanks for the review! I understand the attention message wasn't appropriate. To report the write location, even If you do not specify a replication slot. So the fix only appended messages. There was a description of the

Re: [HACKERS] initdb -S and tablespaces

2014-09-29 Thread Andres Freund
On 2014-09-29 15:43:32 +0530, Abhijit Menon-Sen wrote: At 2014-09-29 11:54:10 +0200, and...@2ndquadrant.com wrote: Note that the perform_fsync() *was* ok for its original purpose in initdb. At the end of initdb there's no relevant tablespaces. But if used *after* pg_upgrade, that's not

Re: [HACKERS] initdb -S and tablespaces

2014-09-29 Thread Abhijit Menon-Sen
At 2014-09-29 12:59:09 +0200, and...@2ndquadrant.com wrote: So I'm afraid at least in a first patch it'll need to be a bit of duplication. Fixing initdb's code back to 9.3 and the backend all the way back to 9.0. OK, thanks, I'll submit two separate patches then. -- Abhijit -- Sent via

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-09-29 Thread Andres Freund
On 2014-09-29 11:31:11 +0200, Andres Freund wrote: On 2014-09-29 11:13:43 +0200, Andres Freund wrote: I doubt it'll be hard to construct a case where it'll show. My first try of using a pgbench scale 100, -M prepared, -cj8 with a custom file with 1 write and 5 read transaction yielded the

Re: [HACKERS] [v9.5] Custom Plan API

2014-09-29 Thread Thom Brown
On 29 September 2014 09:48, Kouhei Kaigai kai...@ak.jp.nec.com wrote: On Wed, Sep 17, 2014 at 7:40 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: At this moment, I revised the above portion of the patches. create_custom_plan() was modified to call PlanCustomPath callback next to the

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-29 Thread Andres Freund
On 2014-09-29 22:42:57 +1300, David Rowley wrote: On Mon, Sep 29, 2014 at 2:41 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-28 17:32:21 +1300, David Rowley wrote: My understanding of foreign keys is that any pending foreign key triggers will be executed just before the

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-29 Thread Andres Freund
Hi, On 2014-09-22 10:39:32 +, Syed, Rahila wrote: Please find attached the patch to compress FPW. I've given this a quick look and noticed some things: 1) I don't think it's a good idea to put the full page write compression into struct XLogRecord. 2) You've essentially removed a lot of

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-29 Thread Andres Freund
On 2014-09-22 15:40:41 +0900, Michael Paquier wrote: Updated patches attached. These are patches about pg_dump. I don't think these were the ones you wanted to attach... :) Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] Options OUTPUT_PLUGIN_* controlling format are confusing (Was: Misleading error message in logical decoding)

2014-09-29 Thread Michael Paquier
On Mon, Sep 29, 2014 at 7:50 PM, Andres Freund and...@2ndquadrant.com wrote: What about the attached patch then? Thanks for this update. This looks good. Here are a couple of small comments: 1) This sentence is correct English, but I don't recall seeing in the docs such a formulation: +

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-29 Thread Michael Paquier
On Mon, Sep 29, 2014 at 9:42 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-22 15:40:41 +0900, Michael Paquier wrote: Updated patches attached. These are patches about pg_dump. I don't think these were the ones you wanted to attach... :) Wah, sorry (bakabakashii...). Here they

Re: [HACKERS] Options OUTPUT_PLUGIN_* controlling format are confusing (Was: Misleading error message in logical decoding)

2014-09-29 Thread Andres Freund
On 2014-09-29 21:48:08 +0900, Michael Paquier wrote: On Mon, Sep 29, 2014 at 7:50 PM, Andres Freund and...@2ndquadrant.com wrote: What about the attached patch then? Thanks for this update. This looks good. Here are a couple of small comments: 1) This sentence is correct English, but

Re: [HACKERS] [v9.5] Custom Plan API

2014-09-29 Thread Kohei KaiGai
2014-09-29 20:26 GMT+09:00 Thom Brown t...@linux.com: On 29 September 2014 09:48, Kouhei Kaigai kai...@ak.jp.nec.com wrote: On Wed, Sep 17, 2014 at 7:40 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: At this moment, I revised the above portion of the patches. create_custom_plan() was modified

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-29 Thread Heikki Linnakangas
On 09/27/2014 11:50 PM, Marko Tiikkaja wrote: Hi, On 9/25/14, 3:56 PM, I wrote: On 9/25/14 3:50 PM, Heikki Linnakangas wrote: Are you planning to post the main patch rebased on top of this soon? As in the next day or two? Otherwise I'll mark this as Returned with feedback for this commitfest.

Re: [HACKERS] Missing newlines in verbose logs of pg_dump, introduced by RLS patch

2014-09-29 Thread Stephen Frost
* Michael Paquier (michael.paqu...@gmail.com) wrote: On Mon, Sep 29, 2014 at 10:07 AM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: The schema name is missing... attached patch add it. Ah, right, thanks. It didn't occur to me immediately :) Your patch looks good to me, and you

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Simon Riggs
On 29 September 2014 10:27, Craig Ringer cr...@2ndquadrant.com wrote: On 09/29/2014 05:10 PM, Simon Riggs wrote: Please go to some trouble to tidy things up so we have clarity that *we* can see and decide for ourselves whether or not you are correct. Are you suggesting a wiki page to

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-29 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-09-28 10:41:56 -0400, Tom Lane wrote: If this optimization only works in that scenario, it's dead in the water, because that assumption is unsupportable. The planner does not in general use the same query snapshot as the executor, so even

Re: [HACKERS] Last Commitfest patches waiting review

2014-09-29 Thread Heikki Linnakangas
On 09/27/2014 05:12 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: [ unreviewed patches ] Grouping Sets There has been a lot of discussion, but no decisions. See http://www.postgresql.org/message-id/87vbodhtvb@news-spur.riddles.org.uk. Would a committer be

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Simon Riggs
On 28 September 2014 08:40, Peter Geoghegan p...@heroku.com wrote: On Sat, Sep 27, 2014 at 11:21 PM, Simon Riggs si...@2ndquadrant.com wrote: My request was for the following... Agree command semantics by producing these things * Explanatory documentation (Ch6.4 Data Manipulation - Upsert)

Re: [HACKERS] WITH CHECK and Column-Level Privileges

2014-09-29 Thread Robert Haas
On Sat, Sep 27, 2014 at 1:19 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Also attached is a patch for 9.4 which does the same, but also removes the row reporting (completely) from the WITH CHECK case. It could be argued that it would be helpful to have it there also, perhaps, but I'm not

Re: [HACKERS] WITH CHECK and Column-Level Privileges

2014-09-29 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Sat, Sep 27, 2014 at 1:19 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Also attached is a patch for 9.4 which does the same, but also removes the row reporting (completely) from the WITH CHECK case. It could be argued that it would be

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-29 Thread Andres Freund
On 2014-09-29 10:12:25 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-09-28 10:41:56 -0400, Tom Lane wrote: If this optimization only works in that scenario, it's dead in the water, because that assumption is unsupportable. The planner does not in general use

Re: [HACKERS] WITH CHECK and Column-Level Privileges

2014-09-29 Thread Robert Haas
On Mon, Sep 29, 2014 at 10:26 AM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Sat, Sep 27, 2014 at 1:19 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Also attached is a patch for 9.4 which does the same, but also removes the row reporting

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Robert Haas
On Sat, Sep 27, 2014 at 11:00 PM, Andrew Dunstan and...@dunslane.net wrote: On 09/27/2014 10:52 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 09/27/2014 06:27 PM, Tom Lane wrote: So my vote is for a separate function and no optional arguments. You mean like

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-29 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-09-29 10:12:25 -0400, Tom Lane wrote: I see. So why aren't we simply ignoring deferrable FKs when making the optimization? That pushes it back from depending on execution-time state (unsafe) to depending on table DDL (safe). IIRC there's

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-29 Thread Marko Tiikkaja
On 9/29/14 3:02 PM, Heikki Linnakangas wrote: Thanks! I found the pgp_extract_armor_headers()'s signature quite weird, so I simplified that by making it always return arrays of keys and values. The callers is now responsible for returning all the keys (pgp_armor_header_keys) or finding the

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: +1. I am sort of surprised that anyone things this null-stripping behavior is something that ought to be part of core PostgreSQL instead of, I don't know, relegated to an extension somewhere far from the bright center of the galaxy. I've never

Re: [HACKERS] Replication identifiers, take 3

2014-09-29 Thread Robert Haas
On Sat, Sep 27, 2014 at 12:11 PM, Steve Singer st...@ssinger.info wrote: If we were now increasing the WAL record size anyway for some unrelated reason, would we be willing to increase it by a further 2 bytes for the node identifier? Obviously not. Otherwise Andres would be proposing to put

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-29 Thread Robert Haas
On Mon, Sep 29, 2014 at 8:36 AM, Andres Freund and...@anarazel.de wrote: 1) I don't think it's a good idea to put the full page write compression into struct XLogRecord. Why not, and where should that be put? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise

Re: [HACKERS] Options OUTPUT_PLUGIN_* controlling format are confusing (Was: Misleading error message in logical decoding)

2014-09-29 Thread Robert Haas
On Mon, Sep 29, 2014 at 9:03 AM, Andres Freund and...@2ndquadrant.com wrote: -except that changes are returned as typebytea/type. +except that changes are returned as typebytea/type and that it can +be used on slots using output plugins that only support binary output.

Re: [HACKERS] [v9.5] Custom Plan API

2014-09-29 Thread Robert Haas
On Mon, Sep 29, 2014 at 9:04 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Do we need to match the prototype of wrapper function with callback? Yes. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-29 Thread Andres Freund
On 2014-09-29 11:02:49 -0400, Robert Haas wrote: On Mon, Sep 29, 2014 at 8:36 AM, Andres Freund and...@anarazel.de wrote: 1) I don't think it's a good idea to put the full page write compression into struct XLogRecord. Why not, and where should that be put? Hah. I knew that somebody

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Andrew Dunstan
On 09/29/2014 10:38 AM, Robert Haas wrote: On Sat, Sep 27, 2014 at 11:00 PM, Andrew Dunstan and...@dunslane.net wrote: On 09/27/2014 10:52 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 09/27/2014 06:27 PM, Tom Lane wrote: So my vote is for a separate function and no

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-29 Thread Andres Freund
On 2014-09-29 18:27:01 +0300, Heikki Linnakangas wrote: On 09/29/2014 06:02 PM, Robert Haas wrote: On Mon, Sep 29, 2014 at 8:36 AM, Andres Freund and...@anarazel.de wrote: 1) I don't think it's a good idea to put the full page write compression into struct XLogRecord. Why not, and where

Re: [HACKERS] KNN-GiST with recheck

2014-09-29 Thread Alexander Korotkov
On Mon, Sep 29, 2014 at 6:16 AM, Bruce Momjian br...@momjian.us wrote: On Fri, Sep 26, 2014 at 10:49:42AM +0400, Alexander Korotkov wrote: Does this also fix the identical PostGIS problem or is there something PostGIS needs to do? This patch provides general infrastructure for

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Robert Haas
On Fri, Sep 26, 2014 at 5:40 PM, Peter Geoghegan p...@heroku.com wrote: I will be frank. Everyone knows that the nbtree locking parts of this are never going to be committed over your objections. It cannot happen. And yet, I persist in proposing that we go that way. I may be stubborn, but I am

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Stephen Frost
Merlin, * Merlin Moncure (mmonc...@gmail.com) wrote: Are you defining 'core' as 'supported by the core project' (in which case I agree) or 'not an extension' (in which case I disagree). Which means you're suggesting it as an extension which lives in contrib..? Otherwise, I'm not following.

Re: [HACKERS] WITH CHECK and Column-Level Privileges

2014-09-29 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Well, I think that's an acceptable approach from the point of view of fixing the security exposure, but it's far from ideal. Good error messages are important for usability. I can live with this as a short-term fix, but in the long run I strongly

Re: [HACKERS] open items for 9.4

2014-09-29 Thread Robert Haas
On Sun, Sep 28, 2014 at 11:36 PM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: So, can we get Beta3 out now? If nobody else steps up and says they want to do some performance testing, I'll push the latest lengths+offsets patch tomorrow. Are any of the other open

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Stephen Frost
* Andrew Dunstan (and...@dunslane.net) wrote: That said, doing this as an extension is probably a good way to go, as I suggested upthread, since we could then make it available for 9.4, rather than making people wait until 9.5. Two points on this- having it in 9.5 doesn't preclude someone from

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-29 Thread Heikki Linnakangas
On 09/29/2014 06:02 PM, Robert Haas wrote: On Mon, Sep 29, 2014 at 8:36 AM, Andres Freund and...@anarazel.de wrote: 1) I don't think it's a good idea to put the full page write compression into struct XLogRecord. Why not, and where should that be put? It should be a flag in BkpBlock. -

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Merlin Moncure
On Mon, Sep 29, 2014 at 9:45 AM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: +1. I am sort of surprised that anyone things this null-stripping behavior is something that ought to be part of core PostgreSQL instead of, I don't know, relegated to an

Re: [HACKERS] open items for 9.4

2014-09-29 Thread Magnus Hagander
On Mon, Sep 29, 2014 at 5:53 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-29 11:50:19 -0400, Robert Haas wrote: The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-29 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: Attached is a contrib module that lets you launch arbitrary command in a background worker, and supporting infrastructure patches for core. Very cool! Started looking into this while waiting on a few CLOBBER_CACHE_ALWAYS runs to finish

Re: [HACKERS] open items for 9.4

2014-09-29 Thread Andres Freund
On 2014-09-29 11:50:19 -0400, Robert Haas wrote: The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep the GUC, but if we're going to

Re: [HACKERS] open items for 9.4

2014-09-29 Thread Alvaro Herrera
Andres Freund wrote: On 2014-09-29 11:50:19 -0400, Robert Haas wrote: - pg_dump fails with --if-exists and blobs This looks like a 9.4 regression. Alvaro, IIRC you were looking at this one? I am. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Andrew Dunstan (and...@dunslane.net) wrote: That said, doing this as an extension is probably a good way to go, as I suggested upthread, since we could then make it available for 9.4, rather than making people wait until 9.5. Two points on this-

Re: [HACKERS] open items for 9.4

2014-09-29 Thread Pavel Stehule
Dne 29.9.2014 18:00 Magnus Hagander mag...@hagander.net napsal(a): On Mon, Sep 29, 2014 at 5:53 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-29 11:50:19 -0400, Robert Haas wrote: The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-29 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: BTW, it seems like there is consensus that we ought to reorder the items in a jsonb object to have keys first and then values, independently of the other issues under discussion. This means we

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread David E. Wheeler
On Sep 29, 2014, at 9:09 AM, Tom Lane t...@sss.pgh.pa.us wrote: I seem to recall that we've run into practical difficulties with moving extensions into core. It might be OK for a functions-only extension though. It does make upgrading difficult, though, as I’ve learned the hard way with

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Bruce Momjian br...@momjian.us writes: On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: BTW, it seems like there is consensus that we ought to reorder the items in a jsonb object to have keys first and then values, independently of the other

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Peter Geoghegan
On Mon, Sep 29, 2014 at 8:31 AM, Robert Haas robertmh...@gmail.com wrote: I'll be frank, too. Heikki doesn't need to persuade you to go his way, because everyone other than yourself who has looked at this problem has come up with a design that looks like his. Andres suggested something that

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-29 Thread Robert Haas
On Mon, Sep 29, 2014 at 12:05 PM, Stephen Frost sfr...@snowman.net wrote: Perhaps I'm just being a bit over the top, but all this per-character work feels a bit ridiculous.. When we're using MAXIMUM_ALIGNOF, I suppose it's not so bad, but is there no hope to increase that and make this whole

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Peter Geoghegan
On Mon, Sep 29, 2014 at 10:37 AM, Peter Geoghegan p...@heroku.com wrote: But more likely, somebody will go find some other bit space that can be used to do this. My concerns have nothing to do with the availability of bit space, obviously. -- Peter Geoghegan -- Sent via pgsql-hackers

[HACKERS] effective_io_concurrency documentation

2014-09-29 Thread Peter Eisentraut
The documentation of effective_io_concurrency says Asynchronous I/O depends on an effective functionposix_fadvise/ function, which some operating systems lack. If the function is not present then setting this parameter to anything but zero will result in an error. This is not

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Peter Geoghegan
On Mon, Sep 29, 2014 at 7:21 AM, Simon Riggs si...@2ndquadrant.com wrote: If you were an ORM developer reading the PostgreSQL Release Notes for 9.5, which URL would you visit to see a complete description of the new feature, including how it works concurrently, locking and other aspects. How

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: Done --- 201409291 is the cutover point. Just to clarify- the commit bumped the catversion to 201409292, so version = 201409291 has the old format while version

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Peter Geoghegan
On Mon, Sep 29, 2014 at 12:02 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-29 09:51:45 +0300, Heikki Linnakangas wrote: That said, it would be handy if the syntax was closer to MERGE. Aside from the concurrency issues, it does the same thing, right? So how about making the syntax

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Peter Geoghegan
On Mon, Sep 29, 2014 at 2:27 AM, Craig Ringer cr...@2ndquadrant.com wrote: Please go to some trouble to tidy things up so we have clarity that *we* can see and decide for ourselves whether or not you are correct. Are you suggesting a wiki page to document the issues, discussions around each

Re: [HACKERS] open items for 9.4

2014-09-29 Thread Andres Freund
On 2014-09-29 11:28:07 -0700, Josh Berkus wrote: On 09/29/2014 08:53 AM, Andres Freund wrote: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep

Re: [HACKERS] open items for 9.4

2014-09-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep the GUC, but if we're going to remove

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-09-29 Thread Andres Freund
On 2014-09-29 14:46:20 -0400, Robert Haas wrote: On Fri, May 9, 2014 at 10:18 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, May 3, 2014 at 4:31 AM, Dave Page dave.p...@enterprisedb.com wrote: Hamid@EDB; Can you please have someone configure anole to build git head as well as the

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-29 Thread Josh Berkus
On 09/29/2014 11:49 AM, Arthur Silva wrote: What's the call on the stride length? Are we going to keep it hardcoded? Please, yes. The complications caused by a variable stride length would be horrible. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-29 Thread Tom Lane
Arthur Silva arthur...@gmail.com writes: What's the call on the stride length? Are we going to keep it hardcoded? At the moment it's 32, but we could change it without forcing a new initdb. I ran a simple test that seemed to show 32 was a good choice, but if anyone else wants to try other

Re: [HACKERS] trivial patch for dynahash logging

2014-09-29 Thread Jeff Janes
On Sat, Sep 27, 2014 at 3:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: under HASH_STATISTICS, the output reporting this HTAB upon destruction is pretty useless. Which HTAB would this one be? It is not necessarily the most recently created one. This

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Peter Geoghegan
On Mon, Sep 29, 2014 at 2:14 AM, Simon Riggs si...@2ndquadrant.com wrote: 1. SQL Standard MERGE (or a subset) 2. MySQL Compatible syntax 3. Something completely different If we go for (3), I would like to see a long and detailed explanation of what is wrong with (1) and (2) before we do (3).

Re: [HACKERS] effective_io_concurrency documentation

2014-09-29 Thread Peter Geoghegan
On Mon, Sep 29, 2014 at 10:41 AM, Peter Eisentraut pete...@gmx.net wrote: I think this is a mistake. For example, we allow setting ssl to false even if SSL support is not compiled in. +1 -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Pavel Stehule
2014-09-28 18:35 GMT+02:00 Andrew Dunstan and...@dunslane.net: On 09/27/2014 11:58 PM, Stephen Frost wrote: All, On Saturday, September 27, 2014, Andrew Dunstan and...@dunslane.net mailto:and...@dunslane.net wrote: On 09/27/2014 10:52 PM, Tom Lane wrote: Andrew Dunstan

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Pavel Stehule
2014-09-29 17:38 GMT+02:00 Stephen Frost sfr...@snowman.net: * Andrew Dunstan (and...@dunslane.net) wrote: That said, doing this as an extension is probably a good way to go, as I suggested upthread, since we could then make it available for 9.4, rather than making people wait until 9.5.

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Pavel Stehule
2014-09-27 14:00 GMT+02:00 Stephen Frost sfr...@snowman.net: Andrew, all, * Andrew Dunstan (and...@dunslane.net) wrote: I should have been paying a bit more attention to the recent work on adding an ignore_nulls option to row_to_json(). Here are some belated thought. I apologize to Pavel

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-09-29 Thread Robert Haas
On Mon, Sep 29, 2014 at 2:52 PM, Andres Freund and...@2ndquadrant.com wrote: This happened again, and I investigated further. It looks like the postmaster knows full well that it's supposed to start more bgworkers: the ones that never get started are in the postmaster's BackgroundWorkerList,

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Pavel Stehule
2014-09-29 21:23 GMT+02:00 Pavel Stehule pavel.steh...@gmail.com: 2014-09-27 14:00 GMT+02:00 Stephen Frost sfr...@snowman.net: Andrew, all, * Andrew Dunstan (and...@dunslane.net) wrote: I should have been paying a bit more attention to the recent work on adding an ignore_nulls option

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-09-29 Thread Andres Freund
On 2014-09-29 15:24:55 -0400, Robert Haas wrote: On Mon, Sep 29, 2014 at 2:52 PM, Andres Freund and...@2ndquadrant.com wrote: If that theory is true, wouldn't things get unstuck everytime a new connection comes in? Or 60 seconds have passed? That's not to say this isn't wrong, but still?

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-09-29 Thread Andres Freund
On 2014-09-29 14:46:20 -0400, Robert Haas wrote: On Fri, May 9, 2014 at 10:18 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, May 3, 2014 at 4:31 AM, Dave Page dave.p...@enterprisedb.com wrote: Hamid@EDB; Can you please have someone configure anole to build git head as well as the

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-09-29 Thread Robert Haas
On Mon, Sep 29, 2014 at 3:37 PM, Andres Freund and...@2ndquadrant.com wrote: Yea :(. Note how signals are blocked in all the signal handlers and only unblocked for a very short time (the sleep). (stare at random shit for far too long) Ah. DetermineSleepTime(), which is called while signals

Re: [HACKERS] open items for 9.4

2014-09-29 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-09-29 14:44:42 -0400, Tom Lane wrote: Personally I think a hardwired #define should be plenty. What's the argument that users will need to tune this at runtime? That right now it can make quite noticeable differences in scalability. And

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-09-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Sep 29, 2014 at 3:37 PM, Andres Freund and...@2ndquadrant.com wrote: Ah. DetermineSleepTime(), which is called while signals are unblocked!, modifies BackgroundWorkerList. Previously that only iterated the list, without modifying it. That's

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-09-29 Thread Andres Freund
On 2014-09-29 16:16:24 -0400, Robert Haas wrote: If you can manually run stuff on that machine, it'd be rather helpful if you could put a PG_SETMASK(BlockSig);...PG_SETMASK(UnBlockSig); in the HaveCrashedWorker() loop. I'd do it the other way around, and adjust ServerLoop to put the

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-09-29 Thread Robert Haas
On Mon, Sep 29, 2014 at 4:20 PM, Andres Freund and...@2ndquadrant.com wrote: Let's just check in the fix. It'll either fix anole or not, but we should fix the bug you found either way. Right. Are you going to do it? I can, but it'll be tomorrow. I'm neck deep in another bug right now. I

Re: [HACKERS] json (b) and null fields

2014-09-29 Thread Andrew Dunstan
On 09/29/2014 04:32 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 09/29/2014 04:14 PM, Tom Lane wrote: More to the point, the way to fix any concerns about double parsing is to create row_to_jsonb(), not to plaster a bunch of options on row_to_json(). row_to_jsonb would

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Kevin Grittner
Peter Geoghegan p...@heroku.com wrote: As I've mentioned, it isn't the MERGE syntax because that is quite a different thing. There is a place for it, but it's not strategically important in the same way as upsert is. I think that the subset of the MERGE syntax that would be needed for UPSERT

Re: [HACKERS] open items for 9.4

2014-09-29 Thread Andres Freund
On 2014-09-29 16:35:12 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-09-29 16:16:38 -0400, Tom Lane wrote: I wonder why it's a fixed constant at all, and not something like wal_buffers / 8. Because that'd be horrible performancewise on a system with many

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Peter Geoghegan
On Mon, Sep 29, 2014 at 1:40 PM, Kevin Grittner kgri...@ymail.com wrote: I think that the subset of the MERGE syntax that would be needed for UPSERT behavior would be as follows. For one row as literals: MERGE INTO tab t USING (VALUES ('foo', 'p1')) new(id, colB) ON (t.id =

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-29 Thread Kevin Grittner
Peter Geoghegan p...@heroku.com wrote: Heikki, Andres and I are against using MERGE for this, fwiw. Tom seemed to think so too, on previous occasions. It isn't a matter of alternative syntaxes. I have described in detail why I think it's a bad idea - I have linked to that about 3 times in

  1   2   >