Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Peter Geoghegan
On Thu, Apr 30, 2015 at 6:54 AM, Robert Haas wrote: > The other, related problem is that the ordering operator might start > to return different results than it did at index creation time. For > example, consider a btree index built on a text column. Now consider > 'yum update'. glibc gets upda

Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Simon Riggs
On 23 April 2015 at 17:24, Andres Freund wrote: > Split into a new thread, the other one is already growing fast > enough. This discussion started at > http://archives.postgresql.org/message-id/55391469.5010506%40iki.fi > > On April 23, 2015 6:48:57 PM GMT+03:00, Heikki Linnakangas < > hlinn...@i

Re: [HACKERS] [Proposal] More Vacuum Statistics

2015-06-07 Thread Tomas Vondra
Hi, On 06/05/15 14:10, Naoya Anzai wrote: Thank you for quick feedback, and I'm sorry for slow response. All of your opinions were very helpful for me. I have confirmed Greg's Idea "Timing events". http://www.postgresql.org/message-id/509300f7.5000...@2ndquadrant.com Greg said at first, "Parsi

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-07 Thread Kevin Grittner
Joshua D. Drake wrote: > On 06/06/2015 07:14 PM, Peter Geoghegan wrote: >> On Sat, Jun 6, 2015 at 7:07 PM, Robert Haas wrote: >>> Perhaps we're honoring this more in the breech than in the >>> observance, but I'm not making up what Tom has said about this: >>> >>> http://www.postgresql.org/messa

Re: [HACKERS] checkpointer continuous flushing

2015-06-07 Thread Fabien COELHO
Hello Andres, They pretty much can't if you flush things frequently. That's why I think this won't be acceptable without the sorting in the checkpointer. * VERSION 2 "WORK IN PROGRESS". The implementation is more a proof-of-concept for having feedback than clean code. What it does: - as v

Re: [HACKERS] PoC: Partial sort

2015-06-07 Thread Andreas Karlsson
On 09/15/2014 01:58 PM, Alexander Korotkov wrote: On Sun, Sep 14, 2014 at 9:32 AM, Peter Geoghegan mailto:p...@heroku.com>> wrote: I think we might be better off if a tuplesort function was called shortly after tuplesort_begin_heap() is called. How top-n heap sorts work is something

Re: [HACKERS] PoC: Partial sort

2015-06-07 Thread Peter Geoghegan
On Sun, Jun 7, 2015 at 8:10 AM, Andreas Karlsson wrote: > Are you planning to work on this patch for 9.6? FWIW I hope so. It's a nice patch. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.or

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-07 Thread Jeff Janes
On Sat, Jun 6, 2015 at 7:35 AM, Geoff Winkless wrote: > On 6 June 2015 at 13:41, Sehrope Sarkuni wrote: > >> >> > It's much easier to work into dev/test setups if there are system >> packages as it's just a config change to an existing script. Building >> from source would require a whole new wo

Re: [HACKERS] Resource Owner reassign Locks

2015-06-07 Thread Jeff Janes
On Thu, Jun 21, 2012 at 5:32 AM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 18.06.2012 13:59, Heikki Linnakangas wrote: > >> On 10.06.2012 23:39, Jeff Janes wrote: >> I found the interface between resowner.c and lock.c a bit confusing. >> resowner.c would sometimes call

Re: [HACKERS] Resource Owner reassign Locks

2015-06-07 Thread Andres Freund
On 2015-06-07 13:44:08 -0700, Jeff Janes wrote: > I'd like to advocate for back-patching this to 9.0, 9.1, and 9.2. It has > run without problems for a while now, and it can be considered a bug that > systems with a very large number of objects cannot be upgraded in a > reasonable time. +1 Greet

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-07 Thread Andrew Dunstan
On 06/05/2015 11:08 PM, Amit Kapila wrote: On Fri, Jun 5, 2015 at 10:51 AM, Amit Kapila > wrote: On Fri, Jun 5, 2015 at 9:57 AM, Andrew Dunstan mailto:and...@dunslane.net>> wrote: On 06/04/2015 11:35 PM, Amit Kapila wrote: Theoreti

Re: [HACKERS] could not truncate directory "pg_subtrans": apparent wraparound

2015-06-07 Thread Thomas Munro
On Sat, Jun 6, 2015 at 4:51 PM, Thomas Munro wrote: > On Sat, Jun 6, 2015 at 1:25 PM, Alvaro Herrera > wrote: >> Thomas Munro wrote: >> >>> My idea was that if I could get oldestXact == next XID in >>> TruncateSUBSTRANS, then TransactionIdToPage(oldestXact) for a value of >>> oldestXact that hap

Re: [HACKERS] could not truncate directory "pg_subtrans": apparent wraparound

2015-06-07 Thread Thomas Munro
On Mon, Jun 8, 2015 at 12:29 PM, Thomas Munro wrote: > Here's a repro script and a suggested patch. Argh... I realised immediately after sending this that subtransaction truncation doesn't even use the oldest XID computed by vacuum, it uses GetOldestXmin (the "oldest transaction that was running

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-07 Thread Alvaro Herrera
Joshua D. Drake wrote: > > On 06/05/2015 08:07 PM, Bruce Momjian wrote: > > >> From my side, it is only recently I got some clear answers to my questions > >>about how it worked. I think it is very important that major features have > >>extensive README type documentation with them so the underly

Re: [HACKERS] Memory leak with XLogFileCopy since de768844 (WAL file with .partial)

2015-06-07 Thread Michael Paquier
On Fri, Jun 5, 2015 at 10:45 PM, Fujii Masao wrote: > Why don't we call InstallXLogFileSegment() at the end of XLogFileCopy()? > If we do that, the risk of memory leak you're worried will disappear at all. Yes, that looks fine, XLogFileCopy() would copy to a temporary file, then install it definit

[HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-07 Thread Abhijit Menon-Sen
Hi. This is a followup to a 2014-02 discussion that led to pg_stats_temp being excluded from pg_basebackup. At the time, it was discussed to exclude pg_log as well, but nothing eventually came of that. Recently, one of our customers has had a basebackup fail because pg_log contained files that we

Re: [HACKERS] pg_stat_archiver issue with aborted archiver

2015-06-07 Thread Michael Paquier
On Sun, Jun 7, 2015 at 1:11 AM, Julien Rouhaud wrote: > I just noticed that if the archiver aborts (for instance if the > archive_command exited with a return code > 127), pg_stat_archiver won't > report those failed attempts. This happens with both 9.4 and 9.5 branches. > > Please find attached a

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-07 Thread Amit Kapila
On Mon, Jun 8, 2015 at 5:52 AM, Andrew Dunstan wrote: > > On 06/05/2015 11:08 PM, Amit Kapila wrote: >> >> >> Okay, I think I can understand why you want to be cautious for >> having a different check for this path, but in that case there is a >> chance that recovery might fail when it

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-07 Thread Michael Paquier
On Mon, Jun 8, 2015 at 12:42 PM, Abhijit Menon-Sen wrote: > This is a followup to a 2014-02 discussion that led to pg_stats_temp > being excluded from pg_basebackup. At the time, it was discussed to > exclude pg_log as well, but nothing eventually came of that. It seems to be that: http://www.pos

Re: [HACKERS] Restore-reliability mode

2015-06-07 Thread Peter Geoghegan
On Sat, Jun 6, 2015 at 12:58 PM, Noah Misch wrote: > - Call VALGRIND_MAKE_MEM_NOACCESS() on a shared buffer when its local pin > count falls to zero. Under CLOBBER_FREED_MEMORY, wipe a shared buffer > when its global pin count falls to zero. Did a patch for this ever materialize? --

Re: [HACKERS] skipping pg_log in basebackup

2015-06-07 Thread Abhijit Menon-Sen
At 2015-06-08 13:09:02 +0900, michael.paqu...@gmail.com wrote: > > It seems to be that: > http://www.postgresql.org/message-id/cahgqgwh0okz6ckpjkcwojga3ejwffm1enrmro3dkdoteaai...@mail.gmail.com (Note that this is about calculating the wrong size, whereas my bug is about the file being too large to

[HACKERS] Collection of memory leaks for ECPG driver

2015-06-07 Thread Michael Paquier
Hi all, Please find attached a patch aimed at fixing a couple of memory leaks in the ECPG driver. Coverity (and sometimes I after reading some other code paths) found them. Regards, -- Michael diff --git a/src/interfaces/ecpg/compatlib/informix.c b/src/interfaces/ecpg/compatlib/informix.c index d

Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Amit Kapila
On Sun, Jun 7, 2015 at 3:02 PM, Simon Riggs wrote: > > On 23 April 2015 at 17:24, Andres Freund wrote: >> > > > It's hard to see how to save space there without reference to a specific use case. I see different solutions depending upon whether we assume a low number of transactions or a high numb

Re: [HACKERS] amcheck prototype

2015-06-07 Thread Peter Geoghegan
On Wed, Nov 19, 2014 at 2:09 PM, Peter Geoghegan wrote: > Attached is a revision of what I previously called btreecheck, which > is now renamed to amcheck. This never really went anywhere, because as a project I don't think that it has very crisp goals. My sense is that it could be developed in a

[HACKERS] Memory leak fixes for pg_dump, pg_dumpall, initdb and pg_upgrade

2015-06-07 Thread Michael Paquier
Hi all, Please find attached a set of fixes for a couple of things in src/bin: - pg_dump/pg_dumpall: -- getFormattedTypeName, convertTSFunction and myFormatType return strdup'd results that are never free'd. -- convertTSFunction returns const char. I fail to see the point of that... In my opinion

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-07 Thread Andres Freund
On 2015-06-05 20:47:33 +0200, Andres Freund wrote: > On 2015-06-05 14:33:12 -0400, Tom Lane wrote: > > Robert Haas writes: > > > 1. The problem that we might truncate an SLRU members page away when > > > it's in the buffers, but not drop it from the buffers, leading to a > > > failure when we try