Re: [HACKERS] Assert's vs elog ERROR vs elog FATAL

2013-04-26 Thread Tom Lane
Daniel Wood writes: > Is the main difference between: > if (s->blockState != TBLOCK_SUBINPROGESS) > elog(*FATAL*, ... > vs > Assert(s->blockState == TBLOCK_SUBINPROGRESS); > the fact that in both cases: > a) the situation is unexpected, as in no user code can create this;

[HACKERS] Assert's vs elog ERROR vs elog FATAL

2013-04-26 Thread Daniel Wood
These two questions are about the correct coding practice in Postgresql vs the specifics of xact.c Is the main difference between: if (s->blockState != TBLOCK_SUBINPROGESS) elog(*FATAL*, ... vs Assert(s->blockState == TBLOCK_SUBINPROGRESS); the fact that in both cases: a) t

[HACKERS] exactly what is COPY BOTH mode supposed to do in case of an error?

2013-04-26 Thread Robert Haas
It seems the backend and libpq don't agree. The backend makes no special provision to wait for a CopyDone message if an error occurs during copy-both. It simply sends an ErrorResponse and that's it. libpq, on the other hand, treats either CopyDone or ErrorResponse as a cue to transition to PGASYN

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix collation assignment for aggregates with ORDER BY.

2013-04-26 Thread David Fetter
On Fri, Apr 26, 2013 at 05:52:03PM -0700, Jeff Davis wrote: > On Fri, 2013-04-26 at 16:46 -0700, David Fetter wrote: > > On Fri, Apr 26, 2013 at 07:49:47PM +, Tom Lane wrote: > > > Given this risk and the lack of field complaints about the issue, it > > > doesn't seem prudent to back-patch. >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix collation assignment for aggregates with ORDER BY.

2013-04-26 Thread Jeff Davis
On Fri, 2013-04-26 at 16:46 -0700, David Fetter wrote: > On Fri, Apr 26, 2013 at 07:49:47PM +, Tom Lane wrote: > > Given this risk and the lack of field complaints about the issue, it > > doesn't seem prudent to back-patch. ... > This needs back-patching to 9.1, where the bug was introduced.

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Jeff Davis
On Sat, 2013-04-27 at 00:20 +0200, Andres Freund wrote: > CFLAGS_VECTORIZATION? EXTRA sounds to generic to me. I went with CFLAGS_VECTOR to be a little shorter while still keeping some meaning. > I think it would be better to have a PGAC_PROG_CC_VAR_OPT or so which > assigns the flag to some pass

[HACKERS] Re: [COMMITTERS] pgsql: Fix collation assignment for aggregates with ORDER BY.

2013-04-26 Thread David Fetter
On Fri, Apr 26, 2013 at 07:49:47PM +, Tom Lane wrote: > Fix collation assignment for aggregates with ORDER BY. > > ORDER BY expressions were being treated the same as regular aggregate > arguments for purposes of collation determination, but really they should > not affect the aggregate's coll

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Andres Freund
On 2013-04-26 12:57:09 -0700, Jeff Davis wrote: > I updated the patch and split it into two parts (attached). > The second patch adds the configure-time check for the right > compilation flags, and uses them when compiling checksum.c. I > called the new variable CFLAGS_EXTRA, for lack of a better

[HACKERS] pg_ctl non-idempotent behavior change

2013-04-26 Thread Jeff Janes
After 87306184580c9c49717, if the postmaster dies without cleaning up (i.e. power outage), running "pg_ctl start" just gives this message and then exits: pg_ctl: another server might be running Under the old behavior, it would try to start the server anyway, and succeed, then go through recovery

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Jeff Davis
On Fri, 2013-04-26 at 16:40 -0400, Greg Smith wrote: > I think I need to do two baselines: master without checksums, and > master with extra optimizations but still without checksums. It may be > the case that using better compile time optimizations gives a general > speedup that's worth consi

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Greg Smith
On 4/26/13 3:57 PM, Jeff Davis wrote: The second patch adds the configure-time check for the right compilation flags, and uses them when compiling checksum.c. I called the new variable CFLAGS_EXTRA, for lack of a better idea, so feel free to come up with a new name. It doesn't check for, or use,

Re: [HACKERS] libpq COPY handling

2013-04-26 Thread Gavin Flower
On 27/04/13 02:48, Tom Lane wrote: Robert Haas writes: On Thu, Apr 25, 2013 at 1:56 PM, Tom Lane wrote: However, the documentation in libpq.sgml is a bit bogus too, because it counsels trying the PQputCopyEnd() call again, which will not work (since we already changed the asyncStatus). We co

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Jeff Davis
r use, -msse4.1, but that can be specified by the user by configuring with CFLAGS_EXTRA="-msse4.1". I don't know of any more required changes, aside from documentation improvements. Regards, Jeff Davis fnv-jeff-20130426.patch Description: Binary data fnv-jeff-20130426-cfl

Re: [HACKERS] Bug Fix: COLLATE with multiple ORDER BYs in aggregates

2013-04-26 Thread Tom Lane
David Fetter writes: > While testing the upcoming FILTER clause for aggregates, Erik Rijkers > uncovered a long-standing bug in $subject, namely that this case > wasn't handled. Please find attached a patch by Andrew Gierth and > myself which fixes this issue and adds a regression test to ensure

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-04-26 Thread Dimitri Fontaine
Joe Conway writes: > Committed back to 9.1 Thanks, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-ha

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-04-26 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/08/2013 08:34 AM, Dimitri Fontaine wrote: > Joe Conway writes: >> OK, maybe I'll try to take a look in the meantime. > > That would be awesome :) > >> Did you have any comment on the other pg_dump patch (reviewed by >> Vibhor)? > > This whole

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 19:50, Magnus Hagander wrote: On Fri, Apr 26, 2013 at 6:43 PM, Simon Riggs wrote: On 26 April 2013 17:25, Heikki Linnakangas wrote: Actually, from a usability point of view I think would be nice to have just one setting, "recovery_target". It's already somewhat confusing to have

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Magnus Hagander
On Fri, Apr 26, 2013 at 6:43 PM, Simon Riggs wrote: > On 26 April 2013 17:25, Heikki Linnakangas wrote: >> On 26.04.2013 19:05, Simon Riggs wrote: >>> >>> On 26 April 2013 16:38, Robert Haas wrote: On Fri, Apr 26, 2013 at 11:35 AM, Simon Riggs wrote: > > Given that I was d

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 17:25, Heikki Linnakangas wrote: > On 26.04.2013 19:05, Simon Riggs wrote: >> >> On 26 April 2013 16:38, Robert Haas wrote: >>> >>> On Fri, Apr 26, 2013 at 11:35 AM, Simon Riggs >>> wrote: Given that I was describing how we might implement Heikki's suggestion, I fi

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Robert Haas
On Fri, Apr 26, 2013 at 12:25 PM, Heikki Linnakangas wrote: >> Doing it the other way means you need to add a new kind of recovery >> target to the API just for this. >> recovery_target_immediate = on > > Sounds good to me. Yeah, I don't have a problem with that, at all. > Actually, from a usabi

Re: [HACKERS] Functional dependencies and GROUP BY - for subqueries

2013-04-26 Thread Tom Lane
Ashutosh Bapat writes: > On Fri, Apr 26, 2013 at 7:54 PM, Tom Lane wrote: >> A larger point is that the patch as proposed doesn't fix the stated >> problem, because it only descends into written-out subqueries. It >> would only succeed at looking into views if we applied it after >> rewriting, r

Re: [HACKERS] pg_controldata gobbledygook

2013-04-26 Thread Jeff Janes
On Fri, Apr 26, 2013 at 2:08 AM, Andres Freund wrote: > On 2013-04-25 23:07:02 -0400, Peter Eisentraut wrote: > > I'm not sure who is supposed to be able to read this sort of stuff: > > > > Latest checkpoint's NextXID: 0/7575 > > Latest checkpoint's NextOID: 49152 > > Latest chec

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 19:05, Simon Riggs wrote: On 26 April 2013 16:38, Robert Haas wrote: On Fri, Apr 26, 2013 at 11:35 AM, Simon Riggs wrote: Given that I was describing how we might implement Heikki's suggestion, I find this comment confusing. Please explain. Heikki's suggestion is simply to ha

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 16:38, Robert Haas wrote: > On Fri, Apr 26, 2013 at 11:35 AM, Simon Riggs wrote: >> Given that I was describing how we might implement Heikki's >> suggestion, I find this comment confusing. >> >> Please explain. > > Heikki's suggestion is simply to have a mode that stops as soon

Re: [HACKERS] [ADMIN] Simultaneous index creates on different schemas cause deadlock?

2013-04-26 Thread Paul Hinze
On Thu, Apr 25, 2013 at 12:17 PM, Tom Lane wrote: > The cause is that each one will wait for all older snapshots to be > gone --- and it does that before dropping its own snapshot, so that the > other ones will see it as something to be waited out too. This makes sense. Thank you for explaining.

Re: [HACKERS] danger of stats_temp_directory = /dev/shm

2013-04-26 Thread Robert Haas
On Thu, Apr 25, 2013 at 12:09 AM, Tom Lane wrote: > Alvaro Herrera writes: >> Jeff Janes escribió: >>> With the stats file split patch 187492b6c2e8cafc5 introduced in 9.3dev, now >>> after a crash the postmaster will try to delete all files in the directory >>> stats_temp_directory. When that is

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Robert Haas
On Fri, Apr 26, 2013 at 11:35 AM, Simon Riggs wrote: > Given that I was describing how we might implement Heikki's > suggestion, I find this comment confusing. > > Please explain. Heikki's suggestion is simply to have a mode that stops as soon as consistency is reached. The server already knows

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 15:38, Robert Haas wrote: > On Fri, Apr 26, 2013 at 10:05 AM, Simon Riggs wrote: >> Restore points are definitely the way to go here, this is what they >> were created for. Stopping at a labelled location has a defined >> meaning for the user, which is much better than just "stop

Re: [HACKERS] Functional dependencies and GROUP BY - for subqueries

2013-04-26 Thread Ashutosh Bapat
On Fri, Apr 26, 2013 at 7:54 PM, Tom Lane wrote: > Ashutosh Bapat writes: > > The reason being, it doesn't look into the subqueries (in FROM clause) to > > infer that p.product_id is essentially product.product_id which is a > > primary key. > > Right. > > > Attached find a crude patch to infer

Re: [HACKERS] libpq COPY handling

2013-04-26 Thread Robert Haas
On Fri, Apr 26, 2013 at 10:48 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Apr 25, 2013 at 1:56 PM, Tom Lane wrote: >>> However, the documentation in libpq.sgml is a bit bogus too, because it >>> counsels trying the PQputCopyEnd() call again, which will not work >>> (since we already cha

Re: [HACKERS] Failing start-up archive recovery at Standby mode in PG9.2.4

2013-04-26 Thread Mitsumasa KONDO
I explain more detail about this problem. This problem was occurred by RestartPoint create illegal WAL file in during archive recovery. But I cannot recognize why illegal WAL file was created in CreateRestartPoint(). My attached patch is really plain… In problem case at XLogFileReadAnyTLI(), fir

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Magnus Hagander
On Apr 26, 2013 4:38 PM, "Robert Haas" wrote: > > On Fri, Apr 26, 2013 at 10:05 AM, Simon Riggs wrote: > > Restore points are definitely the way to go here, this is what they > > were created for. Stopping at a labelled location has a defined > > meaning for the user, which is much better than ju

Re: [HACKERS] libpq COPY handling

2013-04-26 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 25, 2013 at 1:56 PM, Tom Lane wrote: >> However, the documentation in libpq.sgml is a bit bogus too, because it >> counsels trying the PQputCopyEnd() call again, which will not work >> (since we already changed the asyncStatus). We could make that say "a >> zero

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Robert Haas
On Fri, Apr 26, 2013 at 10:05 AM, Simon Riggs wrote: > Restore points are definitely the way to go here, this is what they > were created for. Stopping at a labelled location has a defined > meaning for the user, which is much better than just "stop anywhere > convenient", which I found so frighte

Re: [HACKERS] libpq COPY handling

2013-04-26 Thread Robert Haas
On Thu, Apr 25, 2013 at 1:56 PM, Tom Lane wrote: > However, the documentation in libpq.sgml is a bit bogus too, because it > counsels trying the PQputCopyEnd() call again, which will not work > (since we already changed the asyncStatus). We could make that say "a > zero result is informational, y

Re: [HACKERS] Functional dependencies and GROUP BY - for subqueries

2013-04-26 Thread Tom Lane
Ashutosh Bapat writes: > The reason being, it doesn't look into the subqueries (in FROM clause) to > infer that p.product_id is essentially product.product_id which is a > primary key. Right. > Attached find a crude patch to infer the same by traversing subqueries. I think this is probably a ba

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Simon Riggs
On 26 April 2013 14:40, Tom Lane wrote: > Andres Freund writes: >>> On 2013-04-26 13:11:00 +0200, Florian Pflug wrote: The unresolved code issue that I know of is moving the compiler flags behind a configure check. I would greatly appreciate it if you could take a look at that. M

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 14:48, Tom Lane wrote: > Magnus Hagander writes: >> That said, maybe the easier choice for a *system* (such as v-thingy) >> would be to simply to the full backup using pg_basebackup -x (or >> similar), therefor not needing the log archive at all when restoring. >> Yes, it makes t

Re: [HACKERS] Functional dependencies and GROUP BY - for subqueries

2013-04-26 Thread Ashutosh Bapat
Hi All, If group by clause has primary key, the targetlist may have columns which are not part of the aggregate and not part of group by clause. The relevant commit is e49ae8d3bc588294d07ce1a1272b31718cfca5ef and relevant mail thread has subject Functional dependencies and GROUP BY. As a result, f

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Tom Lane
Magnus Hagander writes: > That said, maybe the easier choice for a *system* (such as v-thingy) > would be to simply to the full backup using pg_basebackup -x (or > similar), therefor not needing the log archive at all when restoring. > Yes, it makes the base backup slightly larger, but also much >

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Tom Lane
Andres Freund writes: >> On 2013-04-26 13:11:00 +0200, Florian Pflug wrote: >>> The unresolved code issue that I know of is moving the compiler flags >>> behind a configure check. I would greatly appreciate it if you could take a >>> look at that. My config-fu is weak and it would take me some t

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 14:54, Magnus Hagander wrote: On Fri, Apr 26, 2013 at 1:47 PM, Simon Riggs wrote: On 26 April 2013 11:29, Heikki Linnakangas wrote: But there is also an operation to simply "restore a backup". Just because a tool supports an imprecise definition of a restore, doesn't mean Pos

Re: [HACKERS] pg_controldata gobbledygook

2013-04-26 Thread Robert Haas
On Fri, Apr 26, 2013 at 5:08 AM, Andres Freund wrote: > I have to admit I don't see the point. None of those values is particularly > interesting to anybody without implementation level knowledge and those > will likely deal with them just fine. And I find the version with the > shorter names far

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 12:54, Magnus Hagander wrote: > That said, maybe the easier choice for a *system* (such as v-thingy) > would be to simply to the full backup using pg_basebackup -x (or > similar), therefor not needing the log archive at all when restoring. > Yes, it makes the base backup slightly

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Magnus Hagander
On Fri, Apr 26, 2013 at 1:47 PM, Simon Riggs wrote: > On 26 April 2013 11:29, Heikki Linnakangas wrote: > >> But there is also an operation to simply "restore a backup". > > Just because a tool supports an imprecise definition of a restore, > doesn't mean Postgres should encourage and support tha

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 26 April 2013 11:29, Heikki Linnakangas wrote: > But there is also an operation to simply "restore a backup". Just because a tool supports an imprecise definition of a restore, doesn't mean Postgres should encourage and support that. "Restore a backup" is more suited to filesystems where mos

Re: [HACKERS] pg_controldata gobbledygook

2013-04-26 Thread Bernd Helmle
--On 25. April 2013 23:19:14 -0400 Tom Lane wrote: I think I've heard of scripts grepping the output of pg_controldata for this that or the other. Any rewording of the labels would break that. While I'm not opposed to improving the labels, I would vote against your second, abbreviated scheme

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Andres Freund
On 2013-04-26 13:11:00 +0200, Florian Pflug wrote: > > The unresolved code issue that I know of is moving the compiler flags > > behind a configure check. I would greatly appreciate it if you could take a > > look at that. My config-fu is weak and it would take me some time to figure > > out how

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Florian Pflug
On Apr26, 2013, at 10:28 , Ants Aasma wrote: > On Apr 25, 2013 10:38 PM, "Jeff Davis" wrote: > > > > On Tue, 2013-04-23 at 11:44 +0300, Ants Aasma wrote: > > > I will try to reword. > > > > Did you have a chance to clarify this, as well as some of the other > > documentation issues Simon mentione

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 12:16, Simon Riggs wrote: On 18 April 2013 19:11, Heikki Linnakangas wrote: I just found out that if you use continuous archiving and online backups, it's surprisingly difficult to restore a backup, without replaying any more WAL than necessary. I didn't add it myself because I

Re: [HACKERS] Recovery target 'immediate'

2013-04-26 Thread Simon Riggs
On 18 April 2013 19:11, Heikki Linnakangas wrote: > I just found out that if you use continuous archiving and online backups, > it's surprisingly difficult to restore a backup, without replaying any more > WAL than necessary. I didn't add it myself because I don't see the need, if we think more

Re: [HACKERS] pg_controldata gobbledygook

2013-04-26 Thread Andres Freund
On 2013-04-25 23:07:02 -0400, Peter Eisentraut wrote: > I'm not sure who is supposed to be able to read this sort of stuff: > > Latest checkpoint's NextXID: 0/7575 > Latest checkpoint's NextOID: 49152 > Latest checkpoint's NextMultiXactId: 7 > Latest checkpoint's NextMultiOffset

Re: [HACKERS] pg_controldata gobbledygook

2013-04-26 Thread Gavin Flower
On 26/04/13 18:53, Daniel Farina wrote: On Thu, Apr 25, 2013 at 9:34 PM, Tom Lane wrote: Alvaro Herrera writes: Tom Lane wrote: I think I've heard of scripts grepping the output of pg_controldata for this that or the other. Any rewording of the labels would break that. While I'm not opposed

Re: [HACKERS] Failing start-up archive recovery at Standby mode in PG9.2.4

2013-04-26 Thread KONDO Mitsumasa
Hi, I discavered the problem cause. I think taht horiguchi's discovery is another problem... Problem has CreateRestartPoint. In recovery mode, PG should not WAL record. Because PG does not know latest WAL file location. But in this problem case, PG(standby) write WAL file at RestartPoint in arch

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Ants Aasma
On Apr 25, 2013 10:38 PM, "Jeff Davis" wrote: > > On Tue, 2013-04-23 at 11:44 +0300, Ants Aasma wrote: > > I will try to reword. > > Did you have a chance to clarify this, as well as some of the other > documentation issues Simon mentioned here? > > http://www.postgresql.org/message-id/CA+U5nMKVEu

Re: [HACKERS] Failing start-up archive recovery at Standby mode in PG9.2.4

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 07:47, Amit Langote wrote: How would code after applying this patch behave if a recycled segment gets renamed using the newest timeline (say 3) while we are still recovering from a lower timeline (say 2)? In that case, since XLogFileReadAnyTLI returns that recycled segment as the

Re: [HACKERS] Failing start-up archive recovery at Standby mode in PG9.2.4

2013-04-26 Thread Amit Langote
What would happen if a recycled segment gets assigned a newer timeline than the one we are currently recovering from? In the reported erroneous behavior, that is what happens causing XLogFileReadAnyTLI() to return such bogus segment causing the error. Since, expectedTLIs contains a newer timeline a

Re: [HACKERS] event trigger API documentation?

2013-04-26 Thread Dimitri Fontaine
Peter Eisentraut writes: > It seems pretty straightforward and useful, so I'm not sure where your > hesitation is coming from. If you're talking about my hesitation to consider what we have now as marketing material worthy, it comes from the fact that I don't have a use case where I don't need sp

Re: [HACKERS] Failing start-up archive recovery at Standby mode in PG9.2.4

2013-04-26 Thread Heikki Linnakangas
On 26.04.2013 07:02, Kyotaro HORIGUCHI wrote: I am uncertain a bit weather it is necessary to move curFileTLI to anywhere randomly read . On a short glance, the random access occurs also for reading checkpoint-related records. I didn't understand that. Also I don't have clear distinction bet