Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-02 Thread Kyotaro HORIGUCHI
Sorry, I missed this message and only cought up when reading your CF status mail. I've attached three patches: Could let me know how to get the CF status mail? I think he meant this email I sent last weekend: http://www.postgresql.org/message-id/542672d2.3060...@vmware.com I see,

[HACKERS] Typo fixes in src/backend/rewrite/rewriteHandler.c

2014-10-02 Thread Etsuro Fujita
Here is the comments in process_matched_tle() in rewriteHandler.c. 883 * such nodes; consider 884 * UPDATE tab SET col.fld1.subfld1 = x, col.fld2.subfld2 = y 885 * The two expressions produced by the parser will look like 886 * FieldStore(col, fld1,

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-02 Thread Heikki Linnakangas
On 10/02/2014 02:52 AM, Peter Geoghegan wrote: Having been surprisingly successful at advancing our understanding of arguments for and against various approaches to value locking, I decided to try the same thing out elsewhere. I have created a general-purpose UPSERT wiki page. The page is:

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-10-02 Thread Heikki Linnakangas
On 10/02/2014 03:20 AM, Kevin Grittner wrote: My only concern from the benchmarks is that it seemed like there was a statistically significant increase in planning time: unpatched plan time average: 0.450 ms patched plan time average: 0.536 ms That *might* just be noise, but it seems likely

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-10-02 Thread Tomas Vondra
Dne 2 Říjen 2014, 2:20, Kevin Grittner napsal(a): Tomas Vondra t...@fuzzy.cz wrote: On 12.9.2014 23:22, Robert Haas wrote: My first thought is to revert to NTUP_PER_BUCKET=1, but it's certainly arguable. Either method, though, figures to be better than doing nothing, so let's do something.

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 12:07 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Could you write down all of the discussed syntaxes, using a similar notation we use in the manual, with examples on how to use them? And some examples on what is possible with some syntaxes, and not with others?

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-02 Thread Kyotaro HORIGUCHI
Hello, I propose the attached patch. It adds a new flag ImmediateDieOK, which is a weaker form of ImmediateInterruptOK that only allows handling a pending die-signal in the signal handler. Robert, others, do you see a problem with this? Per se I don't have a problem with it. There

Re: [HACKERS] Replication identifiers, take 3

2014-10-02 Thread Heikki Linnakangas
On 09/23/2014 09:24 PM, Andres Freund wrote: I've previously started two threads about replication identifiers. Check http://archives.postgresql.org/message-id/20131114172632.GE7522%40alap2.anarazel.de and http://archives.postgresql.org/message-id/20131211153833.GB25227%40awork2.anarazel.de .

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-02 Thread Andres Freund
On 2014-10-02 17:47:39 +0900, Kyotaro HORIGUCHI wrote: Hello, I propose the attached patch. It adds a new flag ImmediateDieOK, which is a weaker form of ImmediateInterruptOK that only allows handling a pending die-signal in the signal handler. Robert, others, do you see a

Re: [HACKERS] Replication identifiers, take 3

2014-10-02 Thread Andres Freund
On 2014-10-02 11:49:31 +0300, Heikki Linnakangas wrote: On 09/23/2014 09:24 PM, Andres Freund wrote: I've previously started two threads about replication identifiers. Check http://archives.postgresql.org/message-id/20131114172632.GE7522%40alap2.anarazel.de and

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Andres Freund
On 2014-10-01 18:19:05 +0200, Ilya Kosmodemiansky wrote: I have a patch which is actually not commitfest-ready now, but it always better to start discussing proof of concept having some patch instead of just an idea. That's a good way to start work on a topic like this. From an Oracle DBA's

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-02 Thread Kyotaro HORIGUCHI
Hi, But this is imo prohibitive. Yes, we're doing it for a long while. But no, that's not ok. It actually prompoted me into prototyping the latch thing (in some other thread). I don't think existing practice justifies expanding it further. I see, in that case, this approach seems

Re: [HACKERS] pgbench throttling latency limit

2014-10-02 Thread Heikki Linnakangas
On 09/15/2014 08:46 PM, Fabien COELHO wrote: I'm not sure I like the idea of printing a percentage. It might be unclear what the denominator was if somebody feels the urge to work back to the actual number of skipped transactions. I mean, I guess it's probably just the value you passed to

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Ilya Kosmodemiansky
On Thu, Oct 2, 2014 at 11:50 AM, Andres Freund and...@2ndquadrant.com wrote: Not just from a oracle DBA POV ;). Generally. sure Saying that, principally they mean an Oracle Wait Interface analogue. The Basic idea is to have counters or sensors all around database kernel to measure what a

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Ilya Kosmodemiansky
On Thu, Oct 2, 2014 at 5:25 AM, Craig Ringer cr...@2ndquadrant.com wrote: It's not at all clear to me that a DTrace-like (or perf-based, rather) approach is unsafe, slow, or unsuitable for production use. With appropriate wrapper tools I think we could have quite a useful library of

[HACKERS] proposal: doc: simplify examples of dynamic SQL

2014-10-02 Thread Pavel Stehule
Hi There are few less readable examples of dynamic SQL in plpgsql doc like: EXECUTE 'SELECT count(*) FROM ' || tabname::regclass || ' WHERE inserted_by = $1 AND inserted = $2' INTO c USING checked_user, checked_date; or EXECUTE 'UPDATE tbl SET ' || quote_ident(colname)

Re: [HACKERS] pgcrypto: PGP signatures

2014-10-02 Thread Marko Tiikkaja
On 10/2/14 1:47 PM, Heikki Linnakangas wrote: I looked at this briefly, and was surprised that there is no support for signing a message without encrypting it. Is that intentional? Instead of adding a function to encrypt and sign a message, I would have expected this to just add a new function

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: The patch https://commitfest.postgresql.org/action/patch_view?id=885 (discussion starts here I hope - http://www.postgresql.org/message-id/4fe8ca2c.3030...@uptime.jp) demonstrates performance problems; LWLOCK_STAT, LOCK_DEBUG and DTrace-like

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: 1. I've decided to put pg_stat_lwlock into extension pg_stat_lwlock (simply for test purposes). Is it OK, or better to implement it somewhere inside pg_catalog or in another extension (for example pg_stat_statements)? I personally am

Re: [HACKERS] Replication identifiers, take 3

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 4:49 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: An origin column in the table itself helps tremendously to debug issues with the replication system. In many if not most scenarios, I think you'd want to have that extra column, even if it's not strictly required.

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-02 Thread Robert Haas
On Wed, Oct 1, 2014 at 4:56 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Mon, Sep 29, 2014 at 12:05 PM, Stephen Frost sfr...@snowman.net wrote: Perhaps I'm just being a bit over the top, but all this per-character work feels a bit ridiculous..

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-02 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Oct 1, 2014 at 4:56 PM, Stephen Frost sfr...@snowman.net wrote: Was just thinking that we might be able to work out what needs to be done without having to actually do it on a per-character basis. That said, I'm not sure it's

[HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-02 Thread Michael Banck
Hi, we have seen repeatedly that users can be confused about why PostgreSQL is not shutting down even though they requested it. Usually, this is because `log_checkpoints' is not enabled and the final checkpoint is being written, delaying shutdown. As no message besides shutting down is written

Re: [HACKERS] Time measurement format - more human readable

2014-10-02 Thread Bogdan Pilch
On 9/29/14, 1:08 AM, Andres Freund wrote: On 2014-09-28 20:32:30 -0400, Gregory Smith wrote: There are already a wide range of human readable time interval output formats available in the database; see the list at

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 9:54 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera wrote: So in essence what we're going to do is that the balance mechanism considers only tables that don't have per-table configuration options; for those that do, we will use the values configured

[HACKERS] port/atomics/arch-*.h are missing from installation

2014-10-02 Thread Kohei KaiGai
I got the following error when I try to build my extension towards the latest master branch. Is the port/atomics/*.h files forgotten on make install? [kaigai@magro pg_strom]$ make gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Andres Freund
On 2014-09-26 10:28:21 -0400, Robert Haas wrote: On Fri, Sep 26, 2014 at 8:55 AM, Oskari Saarenmaa o...@ohmu.fi wrote: So you think a read barrier is the same thing as an acquire barrier and a write barrier is the same as a release barrier? That would be surprising. It's certainly not

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Andres Freund
On 2014-09-25 10:42:29 -0400, Robert Haas wrote: On Thu, Sep 25, 2014 at 10:24 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-25 10:22:47 -0400, Robert Haas wrote: On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund and...@2ndquadrant.com wrote: That leads me to wonder: Have

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund and...@2ndquadrant.com wrote: OK. Given that the results look good, do you plan to push this? By this, you mean the increase in the number of buffer mapping partitions to 128, and a corresponding increase in MAX_SIMUL_LWLOCKS? If so, and if you

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Andres Freund
On 2014-10-02 10:40:30 -0400, Robert Haas wrote: On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund and...@2ndquadrant.com wrote: OK. Given that the results look good, do you plan to push this? By this, you mean the increase in the number of buffer mapping partitions to 128, and a

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 10:34 AM, Andres Freund and...@2ndquadrant.com wrote: It's actually more complex than that :( Simple things first: Oracle's definition seems pretty iron clad: http://docs.oracle.com/cd/E18659_01/html/821-1383/gjzmf.html __machine_acq_barrier is a clear superset of

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 10:44 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-02 10:40:30 -0400, Robert Haas wrote: On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund and...@2ndquadrant.com wrote: OK. Given that the results look good, do you plan to push this? By this, you mean

Re: [HACKERS] port/atomics/arch-*.h are missing from installation

2014-10-02 Thread Andres Freund
Hi, On 2014-10-02 23:33:36 +0900, Kohei KaiGai wrote: I got the following error when I try to build my extension towards the latest master branch. Is the port/atomics/*.h files forgotten on make install? You're right. The attached patch is probably right remedy. I've changed the order to

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Thu, Oct 2, 2014 at 9:54 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera wrote: So in essence what we're going to do is that the balance mechanism considers only tables that don't have per-table configuration options; for

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-02 Thread David G Johnston
Michael Banck-2 wrote Hi, we have seen repeatedly that users can be confused about why PostgreSQL is not shutting down even though they requested it. Usually, this is because `log_checkpoints' is not enabled and the final checkpoint is being written, delaying shutdown. As no message

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Andres Freund
On 2014-10-02 10:55:06 -0400, Robert Haas wrote: On Thu, Oct 2, 2014 at 10:34 AM, Andres Freund and...@2ndquadrant.com wrote: It's actually more complex than that :( Simple things first: Oracle's definition seems pretty iron clad:

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Andres Freund
On 2014-10-02 10:56:05 -0400, Robert Haas wrote: On Thu, Oct 2, 2014 at 10:44 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-02 10:40:30 -0400, Robert Haas wrote: On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund and...@2ndquadrant.com wrote: OK. Given that the results

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 11:18 AM, Andres Freund and...@2ndquadrant.com wrote: So let's use those, then. Right, I've never contended that. OK, cool. A fully barrier on x86 should be an mfence, right? Right. I've not talked about changing full barrier semantics. What I was referring to is

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Alvaro Herrera
Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: I agree with both of those arguments. I have run into very few customers who have used the autovacuum settings to customize behavior for particular tables, and anyone who hasn't should see no change (right?), so my guess

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-02 Thread Michael Banck
Hi, Am Donnerstag, den 02.10.2014, 08:17 -0700 schrieb David G Johnston: Michael Banck-2 wrote I've attached a trivial patch for this, should it be added to the next commitfest? Peeking at this provokes a couple of novice questions: While apparently it is impossible to have a

[HACKERS] DDL Damage Assessment

2014-10-02 Thread Dimitri Fontaine
Hi fellow hackers, I would like to work on a new feature allowing our users to assess the amount of trouble they will run into when running a DDL script on their production setups, *before* actually getting their services down. The main practical example I can offer here is the ALTER TABLE

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Fabrízio de Royes Mello
On Thu, Oct 2, 2014 at 1:30 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Hi fellow hackers, I would like to work on a new feature allowing our users to assess the amount of trouble they will run into when running a DDL script on their production setups, *before* actually getting their

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 1:46 PM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: On Thu, Oct 2, 2014 at 1:30 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Hi fellow hackers, I would like to work on a new feature allowing our users to assess the amount of trouble they will run

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Harold Giménez
I think the main issue is when a table rewrite is triggered on a DDL command on a large table, as this is what frequently leads to unavailability. The idea of introducing a NOREWRITE keyword to DDL commands then came up (credit: Peter Geoghegan). When the NOREWRITE keyword is used and the DDL

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Heikki Linnakangas
On 10/02/2014 05:40 PM, Robert Haas wrote: On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund and...@2ndquadrant.com wrote: OK. Given that the results look good, do you plan to push this? By this, you mean the increase in the number of buffer mapping partitions to 128, and a corresponding

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Andres Freund
On 2014-10-02 20:04:58 +0300, Heikki Linnakangas wrote: On 10/02/2014 05:40 PM, Robert Haas wrote: On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund and...@2ndquadrant.com wrote: OK. Given that the results look good, do you plan to push this? By this, you mean the increase in the number of

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/02/2014 11:30 AM, Dimitri Fontaine wrote: Questions: 1. Do you agree that a systematic way to report what a DDL command (or script, or transaction) is going to do on your production database is a feature we should provide to our growing

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Andres Freund
On 2014-10-02 11:35:32 -0400, Robert Haas wrote: On Thu, Oct 2, 2014 at 11:18 AM, Andres Freund and...@2ndquadrant.com wrote: Which is why these acquire/release fences, in contrast to acquire/release operations, have more guarantees... You put your finger right onto the spot. But,

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Joshua D. Drake
On 10/02/2014 09:30 AM, Dimitri Fontaine wrote: Questions: 1. Do you agree that a systematic way to report what a DDL command (or script, or transaction) is going to do on your production database is a feature we should provide to our growing user base? I would say it is late to

[HACKERS] TAP test breakage on MacOS X

2014-10-02 Thread Robert Haas
make check-world dies ingloriously for me, like this: /bin/sh ../../../config/install-sh -c -d tmp_check/log make -C ../../.. DESTDIR='/Users/rhaas/pgsql/src/bin/initdb'/tmp_check/install install '/Users/rhaas/pgsql/src/bin/initdb'/tmp_check/log/install.log 21 cd .

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 1:07 PM, Andres Freund and...@2ndquadrant.com wrote: Do a make check-world and it'll hopefully fail ;). Check pg_buffercache_pages.c. Yep. Committed, with an update to the comments in lwlock.c to allude to the pg_buffercache issue. -- Robert Haas EnterpriseDB:

Re: [HACKERS] Proper query implementation for Postgresql driver

2014-10-02 Thread Robert Haas
On Tue, Sep 30, 2014 at 1:20 AM, Craig Ringer cr...@2ndquadrant.com wrote: Frankly, I suggest dropping simple entirely and using only the parse/bind/describe/execute flow in the v3 protocol. The last time I checked, that was significantly slower.

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: 1. Do you agree that a systematic way to report what a DDL command (or script, or transaction) is going to do on your production database is a feature we should provide to our growing user base? I definitely like the idea of such a

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Harold Giménez (har...@heroku.com) wrote: I think the main issue is when a table rewrite is triggered on a DDL command on a large table, as this is what frequently leads to unavailability. The idea of introducing a NOREWRITE keyword to DDL commands then came up (credit: Peter Geoghegan).

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Joshua D. Drake (j...@commandprompt.com) wrote: 2. What do you think such a feature should look like? I liked the other post that said: EXPLAIN ALTER TABLE or whatever. Heck it could even be useful to have EXPLAIN ANALZYE ALTER TABLE in case people want to run it on staging/test/dev

Re: [HACKERS] NEXT VALUE FOR sequence

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 7:27 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: SQL:2003 introduced the function NEXT VALUE FOR sequence. Google tells me that at least DB2, SQL Server and a few niche databases understand it so far. As far as I can tell there is no standardised equivalent of

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Josh Berkus
Questions: 1. Do you agree that a systematic way to report what a DDL command (or script, or transaction) is going to do on your production database is a feature we should provide to our growing user base? Yes. 2. What do you think such a feature should look like? As with

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 12:54 AM, Peter Geoghegan p...@heroku.com wrote: I've started off by adding varied examples of the use of the existing proposed syntax. I'll expand on this soon. I spent some time today expanding on the details, and commenting on the issues around the custom syntax

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread José Luis Tallón
On 10/02/2014 06:30 PM, Dimitri Fontaine wrote: Hi fellow hackers, [snip] Questions: 1. Do you agree that a systematic way to report what a DDL command (or script, or transaction) is going to do on your production database is a feature we should provide to our growing user base?

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-02 Thread Bruce Momjian
On Tue, Sep 30, 2014 at 02:57:43PM -0700, Josh Berkus wrote: I don't know that that is the *expectation*. However, I personally would find it *acceptable* if it meant that we could get efficient merge semantics on other aspects of the syntax, since my primary use for MERGE is bulk loading.

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Steven Lembark
EXPLAIN ALTER TABLE ? Good thing: People recognize it. Bad thing: People might not be able to tell the difference between a DDL and DML result. What about EXPLAIN DDL ...? The extra keyword (DDL) makes it a bit more explicit that the results are not comparable to the standard

Re: [HACKERS] PL/pgSQL 2

2014-10-02 Thread Steven Lembark
On Mon, 01 Sep 2014 12:00:48 +0200 Marko Tiikkaja ma...@joh.to wrote: create a new language. There are enough problems with SQL in general, enough alternatives proposed over time that it might be worth coming up with something that Just Works. -- Steven Lembark

Re: [HACKERS] PL/pgSQL 2

2014-10-02 Thread Steven Lembark
Python2 - Python3 would've been a lot less painful if you could mark, on a module-by-module basis, whether a module was python2 or python3 code. It wasn't very practical for Python because python code can reach deep into the guts of unrelated objects discovered at runtime - it can

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 4:40 PM, Stephen Frost sfr...@snowman.net wrote: * Joshua D. Drake (j...@commandprompt.com) wrote: 2. What do you think such a feature should look like? I liked the other post that said: EXPLAIN ALTER TABLE or whatever. Heck it could even be useful to have EXPLAIN

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 12:40 PM, Stephen Frost sfr...@snowman.net wrote: The downside of the 'explain' approach is that the script then has to be modified to put 'explain' in front of everything and then you have to go through each statement and consider it. Having a 'dry-run' transaction

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 1:37 PM, Peter Geoghegan p...@heroku.com wrote: And yet, in theory session 2's impact on production should not be minimal, if we consider something like EXPLAIN output. Should have been minimal, I mean. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 5:37 PM, Peter Geoghegan p...@heroku.com wrote: Session 3 is an innocent bystander. It goes to query the same table in an ordinary, routine way - a SELECT statement. Even though session 2's lock is not granted yet, session 3 is not at liberty to skip the queue and get

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 6:00 PM, Peter Geoghegan p...@heroku.com wrote: Granted, it's something that's not easily automatable, whereas a nowait is. However, rather than nowait, I'd prefer cancellable semantics, that would cancel voluntarily if any other transaction requests a conflicting lock,

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Claudio Freire (klaussfre...@gmail.com) wrote: On Thu, Oct 2, 2014 at 4:40 PM, Stephen Frost sfr...@snowman.net wrote: The downside of the 'explain' approach is that the script then has to be modified to put 'explain' in front of everything and then you have to go through each statement

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 1:52 PM, Claudio Freire klaussfre...@gmail.com wrote: The explain would show the AccessExclusiveLock, so it would be enough for a heads-up to kill all idle-in-transaction holding locks on the target relation (if killable, or just wait). I think that there are very few

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 6:03 PM, Stephen Frost sfr...@snowman.net wrote: * Claudio Freire (klaussfre...@gmail.com) wrote: On Thu, Oct 2, 2014 at 4:40 PM, Stephen Frost sfr...@snowman.net wrote: The downside of the 'explain' approach is that the script then has to be modified to put 'explain'

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 1:10 PM, Bruce Momjian br...@momjian.us wrote: I think if we use the MERGE command for this feature we would need to use a non-standard keyword to specify that we want OLTP/UPSERT functionality. That would allow us to mostly use the MERGE standard syntax without having

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-02 Thread Peter Eisentraut
On 10/2/14 3:19 PM, Robert Haas wrote: 1..2 ok 1 - initdb with invalid option nonzero exit code ok 2 - initdb with invalid option prints error message # Looks like your test exited with 256 just after 2. not ok 3 - initdb options handling # Failed test 'initdb options

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 2:04 PM, Claudio Freire klaussfre...@gmail.com wrote: I've done that manually (throw the DDL, and cancel if it takes more than a couple of seconds) on modest but relatively busy servers with quite some success. Fair enough, but that isn't the same as NOWAIT. It's

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-02 Thread Andres Freund
On 2014-10-02 17:09:43 -0400, Peter Eisentraut wrote: On 10/2/14 3:19 PM, Robert Haas wrote: 1..2 ok 1 - initdb with invalid option nonzero exit code ok 2 - initdb with invalid option prints error message # Looks like your test exited with 256 just after 2. not ok 3 -

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: On Thu, Oct 2, 2014 at 12:40 PM, Stephen Frost sfr...@snowman.net wrote: The downside of the 'explain' approach is that the script then has to be modified to put 'explain' in front of everything and then you have to go through each statement and

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Andres Freund
On 2014-10-02 17:03:59 -0400, Stephen Frost wrote: * Claudio Freire (klaussfre...@gmail.com) wrote: On Thu, Oct 2, 2014 at 4:40 PM, Stephen Frost sfr...@snowman.net wrote: The downside of the 'explain' approach is that the script then has to be modified to put 'explain' in front of

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Andres Freund
On 2014-10-02 13:49:36 -0300, Claudio Freire wrote: EXPLAIN ALTER TABLE ? I don't think that'll work - there's already EXPLAIN for some CREATE. At least CREATE TABLE ... AS, CREATE VIEW ... AS and SELECT INTO. Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Claudio Freire (klaussfre...@gmail.com) wrote: On Thu, Oct 2, 2014 at 6:03 PM, Stephen Frost sfr...@snowman.net wrote: That sounds extremely complex. You'd have to implement the fake columns, foreign keys, indexes, etc on most execution nodes, the planner, and even system views. Eh?

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2014-10-02 17:03:59 -0400, Stephen Frost wrote: That sounds extremely complex. You'd have to implement the fake columns, foreign keys, indexes, etc on most execution nodes, the planner, and even system views. Eh? We have MVCC

Re: [HACKERS] Assertion failure in syncrep.c

2014-10-02 Thread Simon Riggs
On 18 September 2014 07:32, Pavan Deolasee pavan.deola...@gmail.com wrote: 564 /* 565 * Set state to complete; see SyncRepWaitForLSN() for discussion of 566 * the various states. 567 */ 568 thisproc-syncRepState = SYNC_REP_WAIT_COMPLETE; 569 570

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-02 Thread Bruce Momjian
On Thu, Oct 2, 2014 at 02:08:30PM -0700, Peter Geoghegan wrote: On Thu, Oct 2, 2014 at 1:10 PM, Bruce Momjian br...@momjian.us wrote: I think if we use the MERGE command for this feature we would need to use a non-standard keyword to specify that we want OLTP/UPSERT functionality. That

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 6:19 PM, Stephen Frost sfr...@snowman.net wrote: * Claudio Freire (klaussfre...@gmail.com) wrote: On Thu, Oct 2, 2014 at 6:03 PM, Stephen Frost sfr...@snowman.net wrote: That sounds extremely complex. You'd have to implement the fake columns, foreign keys, indexes, etc

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Dimitri Fontaine
Alvaro Herrera alvhe...@2ndquadrant.com writes: - will the table have to be rewritten? the indexes? Please give my DDL deparsing patch a look. There is a portion there about deparsing ALTER TABLE specifically; what it does is save a list of subcommands, and for each of them we either

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Jan Wieck
On 10/02/2014 01:15 PM, Joe Conway wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/02/2014 11:30 AM, Dimitri Fontaine wrote: Questions: 1. Do you agree that a systematic way to report what a DDL command (or script, or transaction) is going to do on your production database is a

Re: [HACKERS] Proposal for updating src/timezone

2014-10-02 Thread Tom Lane
John Cochran j69coch...@gmail.com writes: As it is, I've finished checking the differences between the postgres and IANA code for zic.c after editing both to eliminate non-functional style differences such as indentation, function prototypes, comparing strchr results against NULL or 0, etc. It

Re: [HACKERS] NEXT VALUE FOR sequence

2014-10-02 Thread Thomas Munro
On 3 October 2014 00:01, Thomas Munro mu...@ip9.org wrote: On 2 October 2014 14:48, Tom Lane t...@sss.pgh.pa.us wrote: Thomas Munro mu...@ip9.org writes: SQL:2003 introduced the function NEXT VALUE FOR sequence. Google tells me that at least DB2, SQL Server and a few niche databases

Re: [HACKERS] CREATE IF NOT EXISTS INDEX

2014-10-02 Thread Marti Raudsepp
On Wed, Oct 1, 2014 at 2:42 PM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: So, what's the correct/best grammar? CREATE [ IF NOT EXISTS ] [ UNIQUE ] INDEX index_name or CREATE [ UNIQUE ] INDEX [ IF NOT EXISTS ] index_name I've elected myself as the reviewer for this patch. Here are

Re: [HACKERS] NEXT VALUE FOR sequence

2014-10-02 Thread Tom Lane
Thomas Munro mu...@ip9.org writes: On 2 October 2014 14:48, Tom Lane t...@sss.pgh.pa.us wrote: Have you checked the archives about this? My recollection is that one reason it's not in there (aside from having to reserve NEXT) is that the standard-mandated semantics are not the same as

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/02/2014 06:43 PM, Jan Wieck wrote: The real question is at what level of information, returned to the user, does this feature become user friendly? It is one thing to provide information of the kind of TAKE ACCECSS EXCLUSIVE LOCK ON

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 5:08 PM, Greg Smith greg.sm...@crunchydatasolutions.com wrote: When 9.4 is already giving a more than 100% gain on this targeted test case, I can't see that chasing after maybe an extra 10% is worth having yet another GUC around. Especially when it will probably take

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Alvaro Herrera wrote: Basically, if you are on 9.3.5 or earlier any per-table options for autovacuum cost delay will misbehave (meaning: any such table will be processed with settings flattened according to balancing of the standard

Re: [HACKERS] Patch to add support of IF NOT EXISTS to others CREATE statements

2014-10-02 Thread Marti Raudsepp
On Tue, Aug 26, 2014 at 4:20 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 04/14/2014 10:31 PM, Fabrízio de Royes Mello wrote: The attached patch contains CINE for sequences. I just strip this code from the patch rejected before. Committed with minor changes Hmm, the CommitFest

Re: [HACKERS] CREATE IF NOT EXISTS INDEX

2014-10-02 Thread Marti Raudsepp
On Fri, Oct 3, 2014 at 2:15 AM, Marti Raudsepp ma...@juffo.org wrote: + ereport(NOTICE, + (errcode(ERRCODE_DUPLICATE_TABLE), + errmsg(relation \%s\ already exists, skipping, + indexRelationName))); 1. Clearly relation should be

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 5:09 PM, Peter Eisentraut pete...@gmx.net wrote: On 10/2/14 3:19 PM, Robert Haas wrote: 1..2 ok 1 - initdb with invalid option nonzero exit code ok 2 - initdb with invalid option prints error message # Looks like your test exited with 256 just after 2.

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-02 Thread Michael Paquier
On Thu, Oct 2, 2014 at 12:44 AM, Andres Freund and...@2ndquadrant.com wrote: I pushed the first part. Thanks. Attached is a rebased version of patch 2, implementing the actual feature. One thing I noticed with more testing is that if --create is used and that the destination folder does not

[HACKERS] GiST splitting on empty pages

2014-10-02 Thread Andrew Gierth
This is from Bug #11555, which is still in moderation as I type this (analysis was done via IRC). The GiST insertion code appears to have no length checks at all on the inserted entry. index_form_tuple checks for length = 8191, with the default blocksize, but obviously a tuple less than 8191

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-02 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: make check-world dies ingloriously for me, like this: FWIW, it works fine for me on my Mac laptop, using the Perl 5.16.2 that comes standard with OSX 10.9.5. I did have to install IPC::Run from CPAN though. # Failed test 'initdb options handling' #

Re: [HACKERS] Patch to add support of IF NOT EXISTS to others CREATE statements

2014-10-02 Thread Fabrízio de Royes Mello
On Thu, Oct 2, 2014 at 9:38 PM, Marti Raudsepp ma...@juffo.org wrote: On Tue, Aug 26, 2014 at 4:20 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 04/14/2014 10:31 PM, Fabrízio de Royes Mello wrote: The attached patch contains CINE for sequences. I just strip this code from the

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Jim Nasby
On 10/2/14, 2:43 PM, Josh Berkus wrote: Questions: 1. Do you agree that a systematic way to report what a DDL command (or script, or transaction) is going to do on your production database is a feature we should provide to our growing user base? Yes. +1 2. What do you think

  1   2   >