Re: [HACKERS] MultiXact truncation, startup et al.

2013-11-29 Thread Andres Freund
Hi, Thanks, looks saner than my version ;) On 2013-11-29 01:00:35 -0300, Alvaro Herrera wrote: ! /* ! * FIXME this calls TransactionIdDidAbort() internally, ! * falsifying the claim in the comment at the

Re: [HACKERS] Todo item: Support amgettuple() in GIN

2013-11-29 Thread Antonin Houska
On 11/29/2013 01:13 AM, Andreas Karlsson wrote: When doing partial matching the code need to be able to return the union of all TIDs in all the matching posting trees in TID order (to be able to do AND and OR operations with multiple search keys later). It does this by traversing them

Re: [HACKERS] docbook-xsl version for release builds

2013-11-29 Thread Magnus Hagander
On Fri, Nov 29, 2013 at 4:06 AM, Peter Eisentraut pete...@gmx.net wrote: On Mon, 2013-11-25 at 16:32 +0100, Magnus Hagander wrote: Thanks for the reminder - done (installed the DEB from sid, version 1.78.1). This will somehow show up in the snapshot builds, correct? So we (you? :P) can

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-29 Thread Amit Khandekar
On 27 November 2013 09:59, Rajeev rastogi rajeev.rast...@huawei.com wrote: On 26 November 2013, Amit Khandelkar wrote: On 26 November 2013 18:59, Amit Khandekar amit.khande...@enterprisedb.com wrote: On 25 November 2013 15:25, Rajeev rastogi rajeev.rast...@huawei.com wrote: OK. I

Re: [HACKERS] Todo item: Support amgettuple() in GIN

2013-11-29 Thread Andreas Karlsson
On 11/29/2013 09:54 AM, Antonin Houska wrote: On 11/29/2013 01:13 AM, Andreas Karlsson wrote: When doing partial matching the code need to be able to return the union of all TIDs in all the matching posting trees in TID order (to be able to do AND and OR operations with multiple search keys

[HACKERS] commit fest 2013-11 week 2 report

2013-11-29 Thread Peter Eisentraut
Last week's status: Fri Nov 22 Status Summary. Needs Review: 47, Waiting on Author: 28, Ready for Committer: 10, Committed: 18, Returned with Feedback: 3, Rejected: 3. Total: 109. Current status: Fri Nov 29 Status Summary. Needs Review: 29, Waiting on Author: 33, Ready for Committer: 13,

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-29 Thread Haribabu kommi
On 29 November 2013 12:00 Amit Kapila wrote: On Tue, Nov 26, 2013 at 7:26 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 25 November 2013 10:43 Amit Kapila wrote: On Fri, Nov 22, 2013 at 12:12 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 19 November 2013 10:33 Amit

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-29 Thread Rajeev rastogi
On 26 November 2013, Amit Khandelkar wrote: Can you please submit the \COPY patch as a separate patch ? Since these are two different issues, I would like to have these two fixed and committed separately. You can always test the \COPY issue using \COPY TO followed by INSERT. Please find the

Re: [HACKERS] Todo item: Support amgettuple() in GIN

2013-11-29 Thread Antonin Houska
On 11/29/2013 01:57 PM, Andreas Karlsson wrote: On 11/29/2013 09:54 AM, Antonin Houska wrote: On 11/29/2013 01:13 AM, Andreas Karlsson wrote: When doing partial matching the code need to be able to return the union of all TIDs in all the matching posting trees in TID order (to be able to do

Re: [HACKERS] SSL: better default ciphersuite

2013-11-29 Thread Peter Eisentraut
On Fri, 2013-11-15 at 01:11 +0200, Marko Kreen wrote: Attached patch changes the default ciphersuite to HIGH:!aNULL instead of old DEFAULT:!LOW:!EXP:!MD5:@STRENGTH where DEFAULT is a shortcut for ALL:!aNULL:!eNULL. Main goal is to leave low-level ciphersuite details to

Re: [HACKERS] [PATCH 1/2] SSL: GUC option to prefer server cipher order

2013-11-29 Thread Peter Eisentraut
On Thu, 2013-11-14 at 11:45 +0100, Magnus Hagander wrote: I think the default behaviour should be the one we recommend (which would be to have the server one be preferred). But I do agree with the requirement to have a GUC to be able to remove it Is there a reason why you would want to turn

Re: [HACKERS] Status of FDW pushdowns

2013-11-29 Thread Atri Sharma
Sent from my iPad On 28-Nov-2013, at 16:13, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Tom Lane t...@sss.pgh.pa.us writes: I'm not real sure what this'd buy us that wouldn't be done as well or better by creating a view on the remote side. (IOW, there's nothing that says that the

Re: [HACKERS] [PATCH 1/2] SSL: GUC option to prefer server cipher order

2013-11-29 Thread Marko Kreen
On Fri, Nov 29, 2013 at 09:25:02AM -0500, Peter Eisentraut wrote: On Thu, 2013-11-14 at 11:45 +0100, Magnus Hagander wrote: I think the default behaviour should be the one we recommend (which would be to have the server one be preferred). But I do agree with the requirement to have a GUC to

Re: [HACKERS] [PATCH 1/2] SSL: GUC option to prefer server cipher order

2013-11-29 Thread Heikki Linnakangas
On 11/29/2013 05:43 PM, Marko Kreen wrote: On Fri, Nov 29, 2013 at 09:25:02AM -0500, Peter Eisentraut wrote: On Thu, 2013-11-14 at 11:45 +0100, Magnus Hagander wrote: I think the default behaviour should be the one we recommend (which would be to have the server one be preferred). But I do

Re: [HACKERS] MultiXact truncation, startup et al.

2013-11-29 Thread Andres Freund
On 2013-11-29 12:49:32 -0300, Alvaro Herrera wrote: + * - xidFullScanLimit (also known as the table freeze age) represents the + * minimum Xid age past which a vacuum will be turned into a full-table one, + * to freeze tuples across the whole table. Vacuuming a table younger than + *

Re: [HACKERS] [GENERAL] pg_upgrade ?deficiency

2013-11-29 Thread Bruce Momjian
On Fri, Nov 29, 2013 at 12:46:01AM +0100, Karsten Hilbert wrote: On Thu, Nov 28, 2013 at 10:39:18AM -0500, Bruce Momjian wrote: Well, then we are actually using two different reasons for patching pg_dumpall and not pg_dump. Your reason is based on the probability of failure, while

Re: [HACKERS] SSL: better default ciphersuite

2013-11-29 Thread Marko Kreen
On Fri, Nov 29, 2013 at 09:18:49AM -0500, Peter Eisentraut wrote: On Fri, 2013-11-15 at 01:11 +0200, Marko Kreen wrote: Attached patch changes the default ciphersuite to HIGH:!aNULL instead of old DEFAULT:!LOW:!EXP:!MD5:@STRENGTH where DEFAULT is a shortcut for

Re: [HACKERS] [PATCH 1/2] SSL: GUC option to prefer server cipher order

2013-11-29 Thread Marko Kreen
On Fri, Nov 29, 2013 at 05:51:28PM +0200, Heikki Linnakangas wrote: On 11/29/2013 05:43 PM, Marko Kreen wrote: On Fri, Nov 29, 2013 at 09:25:02AM -0500, Peter Eisentraut wrote: On Thu, 2013-11-14 at 11:45 +0100, Magnus Hagander wrote: I think the default behaviour should be the one we

Re: [HACKERS] Todo item: Support amgettuple() in GIN

2013-11-29 Thread Tom Lane
Andreas Karlsson andr...@proxel.se writes: I decided to look into how much work implementing the todo item about supporting amgettuple in GIN would be, since exclusion constraints on GIN would be neat. Robert Haas suggested a solution[1], but to fix it we also need to look into why the

Re: [HACKERS] [RFC] overflow checks optimized away

2013-11-29 Thread Tom Lane
Greg Stark st...@mit.edu writes: b) I'm concerned these checks depend on INT_MIN/MAX and SHRT_MIN/MAX which may not be exactly the right length. I'm kind of confused why c.h assumes long is 32 bits and short is 16 bits though so I don't think I'm making it any worse. I think it's something we

Re: [HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-29 Thread Bruce Momjian
On Fri, Nov 29, 2013 at 12:27:49AM -0500, Robert Haas wrote: I wish we'd just left this whole thing well enough alone. It wasn't broken, and didn't need fixing. Well, this started with a complaint that one SET command outside of a transaction had no effect, and expanded to other SET commands

Re: [HACKERS] Todo item: Support amgettuple() in GIN

2013-11-29 Thread Heikki Linnakangas
On 11/29/2013 07:13 PM, Tom Lane wrote: Andreas Karlsson andr...@proxel.se writes: I decided to look into how much work implementing the todo item about supporting amgettuple in GIN would be, since exclusion constraints on GIN would be neat. Robert Haas suggested a solution[1], but to fix it we

Re: [HACKERS] Todo item: Support amgettuple() in GIN

2013-11-29 Thread Andreas Karlsson
On 11/29/2013 06:13 PM, Tom Lane wrote: Note that Robert's proposed solution is no solution, because it just puts you right back in the bind of needing guaranteed non-lossy storage of a TID set that might be too big to fit in memory. The solution should work if we could guarantee that a

Re: [HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-29 Thread Bruce Momjian
On Fri, Nov 29, 2013 at 12:27:49AM -0500, Robert Haas wrote: On Thu, Nov 28, 2013 at 11:04 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: David Johnston wrote: In all of these cases we are assuming that the user understands that emitting a warning means that something is being logged

Re: [HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-29 Thread Bruce Momjian
On Fri, Nov 29, 2013 at 01:05:20PM -0500, Bruce Momjian wrote: On Fri, Nov 29, 2013 at 12:27:49AM -0500, Robert Haas wrote: On Thu, Nov 28, 2013 at 11:04 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: David Johnston wrote: In all of these cases we are assuming that the user

Re: [HACKERS] GIN improvements part 1: additional information

2013-11-29 Thread Heikki Linnakangas
On 11/29/2013 11:41 AM, Heikki Linnakangas wrote: On 11/28/2013 09:19 AM, Alexander Korotkov wrote: On Wed, Nov 27, 2013 at 1:14 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 11/26/13 15:34, Alexander Korotkov wrote: What's your plans about GIN now? I tried to rebase packed

Re: [HACKERS] [RFC] overflow checks optimized away

2013-11-29 Thread Greg Stark
On Fri, Nov 29, 2013 at 5:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: c) I want to add regression tests that will ensure that the overflow checks are all working. So far I haven't been able to catch any being optimized away even with -fno-wrapv and -fstrict-overflow. This does not leave me

Re: [HACKERS] MultiXact truncation, startup et al.

2013-11-29 Thread Andres Freund
On 2013-11-29 16:30:08 -0300, Alvaro Herrera wrote: New versions of all these patches, plus one more patch which removes the behavior that HeapTupleGetUpdateXid checks for aborted updates. From 0dce0b75da2732ca93f4c451b9bae6d4416794c3 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera

Re: [HACKERS] [RFC] overflow checks optimized away

2013-11-29 Thread Greg Stark
On Fri, Nov 29, 2013 at 7:39 PM, Greg Stark st...@mit.edu wrote: Just as an update I did get gcc to do the wrong thing on purpose. The only overflow check that the regression tests find missing is the one for int8abs() ie: Also, one of the places GCC warns about optimizing away an overflow

Re: [HACKERS] lock on object is already held

2013-11-29 Thread Tom Lane
Daniel Wood dw...@salesforce.com writes: ... Presuming your fix is putting PG_SETMASK(UnBlockSig) immediately before each of the 6 calls to ereport(ERROR,...) I've been running the stress test with both this fix and the lock already held fix. I'm now planning to put it in error cleanup

Re: [HACKERS] fe-secure.c and SSL/TLS

2013-11-29 Thread Marko Kreen
Reply to mails in pgsql-bugs: http://www.postgresql.org/message-id/CAH8yC8mc_2J2UY0Q42WQdWFyaoqT3onG+83Fr=vn46j5+ml...@mail.gmail.com and http://www.postgresql.org/message-id/CAH8yC8nZVUyCQznkQd8=ELMM4k_=uxjrjt8yf9v22cy2x_d...@mail.gmail.com * Default ciphersuite I would argue nothing

Re: [HACKERS] Time-Delayed Standbys

2013-11-29 Thread Fabrízio de Royes Mello
On Fri, Nov 29, 2013 at 5:49 AM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: Hi Royes, I'm sorry for my late review... No problem... I feel potential of your patch in PG replication function, and it might be something useful for all people. I check your patch and have some

Re: [HACKERS] [RFC] overflow checks optimized away

2013-11-29 Thread Heikki Linnakangas
On 11/29/2013 10:06 PM, Greg Stark wrote: On Fri, Nov 29, 2013 at 7:39 PM, Greg Stark st...@mit.edu wrote: Just as an update I did get gcc to do the wrong thing on purpose. The only overflow check that the regression tests find missing is the one for int8abs() ie: Also, one of the places

Re: [HACKERS] MultiXact truncation, startup et al.

2013-11-29 Thread Andres Freund
On 2013-11-29 16:30:08 -0300, Alvaro Herrera wrote: As a second bug, heap_freeze_tuple() didn't properly handle multixacts that need to be frozen according to cutoff_multi, but whose updater xid is still alive. Instead of preserving the update Xid, it just set Xmax invalid, which leads to

Re: [HACKERS] MultiXact bugs

2013-11-29 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-11-27 15:42:11 -0800, Kevin Grittner wrote: What back-patching will be needed for a fix?  It sounds like 9.3? Yep. In going over this, I found pre-existing bugs when a tuple was both inserted and deleted by concurrent transactions, but

Re: [HACKERS] pg_upgrade segfaults when given an invalid PGSERVICE value

2013-11-29 Thread Bruce Momjian
On Thu, Mar 28, 2013 at 03:06:30PM -0400, Steve Singer wrote: So to summarise: Plan A: The first patch I attached for pg_upgrade + documentation changes, and changing the other places that call PQconndefaults() to accept failures on either out of memory, or an invalid PGSERVICE Plan B:

Re: [HACKERS] Request for Patch Feedback: Lag Lead Window Functions Can Ignore Nulls

2013-11-29 Thread Bruce Momjian
On Fri, Jul 5, 2013 at 02:36:10PM -0700, Jeff Davis wrote: On Fri, 2013-06-28 at 13:26 -0400, Robert Haas wrote: I haven't really reviewed the windowing-related code in depth; I thought Jeff might jump back in for that part of it. Jeff, is that something you're planning to do? Yes,

Re: [HACKERS] Request for Patch Feedback: Lag Lead Window Functions Can Ignore Nulls

2013-11-29 Thread Jeff Davis
On Fri, 2013-11-29 at 16:18 -0500, Bruce Momjian wrote: On Fri, Jul 5, 2013 at 02:36:10PM -0700, Jeff Davis wrote: On Fri, 2013-06-28 at 13:26 -0400, Robert Haas wrote: I haven't really reviewed the windowing-related code in depth; I thought Jeff might jump back in for that part of it.

Re: [HACKERS] Request for Patch Feedback: Lag Lead Window Functions Can Ignore Nulls

2013-11-29 Thread Bruce Momjian
On Fri, Nov 29, 2013 at 01:21:27PM -0800, Jeff Davis wrote: On Fri, 2013-11-29 at 16:18 -0500, Bruce Momjian wrote: On Fri, Jul 5, 2013 at 02:36:10PM -0700, Jeff Davis wrote: On Fri, 2013-06-28 at 13:26 -0400, Robert Haas wrote: I haven't really reviewed the windowing-related code in

Re: [HACKERS] MultiXact truncation, startup et al.

2013-11-29 Thread Kevin Grittner
Alvaro Herrera alvhe...@2ndquadrant.com wrote: New versions of all these patches, plus one more patch which removes the behavior that HeapTupleGetUpdateXid checks for aborted updates. (Turns out this was necessary to get freezing right.)  My previous patch to avoid InvalidXid as page prune

Re: [HACKERS] MultiXact bugs

2013-11-29 Thread Andres Freund
On 2013-11-29 13:14:06 -0800, Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: On 2013-11-27 15:42:11 -0800, Kevin Grittner wrote: What back-patching will be needed for a fix?  It sounds like 9.3? Yep. In going over this, I found pre-existing bugs when a tuple was

Re: [HACKERS] MultiXact truncation, startup et al.

2013-11-29 Thread Andres Freund
On 2013-11-29 13:26:00 -0800, Kevin Grittner wrote: Alvaro Herrera alvhe...@2ndquadrant.com wrote: New versions of all these patches, plus one more patch which removes the behavior that HeapTupleGetUpdateXid checks for aborted updates. (Turns out this was necessary to get freezing

Re: [HACKERS] MultiXact bugs

2013-11-29 Thread Andres Freund
On 2013-11-29 22:27:16 +0100, Andres Freund wrote: Looking at predicate.c I think I see a bigger problem though: Isn't its usage of HeapTupleSatisfiesVacuum() quite dangerous? It passes TransactionXmin to HeapTupleSatisfiesVacuum(). But since that's just the transaction's own cutoff, not the

Re: [HACKERS] [RFC] overflow checks optimized away

2013-11-29 Thread Tom Lane
Greg Stark st...@mit.edu writes: Also, one of the places GCC warns about optimizing away an overflow check (with -fno-wrapv) is inside the localtime.c file from the tz library. I fixed it in my patch but in fact I checked and it's already fixed upstream so I'm wondering whether you expect to

Re: [HACKERS] MultiXact bugs

2013-11-29 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: Looking at predicate.c I think I see a bigger problem though: Isn't its usage of HeapTupleSatisfiesVacuum() quite dangerous? It passes TransactionXmin to HeapTupleSatisfiesVacuum(). But since that's just the transaction's own cutoff, not the global

Re: [HACKERS] fe-secure.c and SSL/TLS

2013-11-29 Thread Jeffrey Walton
Hi Marko, Forgive me for cherry picking two of these... I think Postgres uses SSL_VERIFY_PEER + SSL_set_verify() callback instead. At least for me, the psql -d dbname=foo sslmode=verify-ca fails when cert does not match. I can't comment on the use of psql. My apologies for my ignorance.

Re: [HACKERS] PostgreSQL Service on Windows does not start. ~ is not a valid Win32 application

2013-11-29 Thread Tom Lane
Rajeev rastogi rajeev.rast...@huawei.com writes: OK. Then I am moving it to ready for committer. I've committed this patch. I added a make_native_path() call to fix the slashes-versus-backslashes issue noted by Christian Ullrich, since that was an easy one-line addition. I didn't do anything

Re: [HACKERS] fe-secure.c and SSL/TLS

2013-11-29 Thread Marko Kreen
On Fri, Nov 29, 2013 at 06:01:01PM -0500, Jeffrey Walton wrote: I know of no other ways to check the result of OpenSSL's chain validation. The open question (for me) is where are SSL_get_verify_result/X509_V_OK checked? Neither show up in the Postgres sources. According to SSL_set_verify

Re: [HACKERS] PostgreSQL Service on Windows does not start. ~ is not a valid Win32 application

2013-11-29 Thread Andrew Dunstan
On 11/29/2013 06:43 PM, Tom Lane wrote: Rajeev rastogi rajeev.rast...@huawei.com writes: OK. Then I am moving it to ready for committer. I've committed this patch. I added a make_native_path() call to fix the slashes-versus-backslashes issue noted by Christian Ullrich, since that was an easy

Re: [HACKERS] PostgreSQL Service on Windows does not start. ~ is not a valid Win32 application

2013-11-29 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 11/29/2013 06:43 PM, Tom Lane wrote: I've committed this patch. I added a make_native_path() call to fix the slashes-versus-backslashes issue noted by Christian Ullrich, since that was an easy one-line addition. I don't mind changing this, but

Re: [HACKERS] MultiXact truncation, startup et al.

2013-11-29 Thread Alvaro Herrera
Okay, I have pushed all these patches, including the fixes suggested here and then some. Hats off to Andres, who handled all the bug analysis, figured out the test cases that tickled things in all the wrong ways, and came up with the patches. Whenever we meet again, let's make sure to find a

Re: [HACKERS] Incomplete freezing when truncating a relation during vacuum

2013-11-29 Thread Noah Misch
On Wed, Nov 27, 2013 at 02:14:53PM +0100, Andres Freund wrote: With regard to fixing things up, ISTM the best bet is heap_prune_chain() so far. That's executed b vacuum and by opportunistic pruning and we know we have the appropriate locks there. Looks relatively easy to fix up