Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-19 Thread Robert Haas
On Mon, May 19, 2014 at 7:58 PM, Andrew Dunstan wrote: > Well, the original code was put in for a reason, presumably that we were > getting some stale data and wanted to exclude it. So I'm unwilling to throw > it out altogether. If someone can propose a reasonable sanity check then I'm > prepared

Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-19 Thread Tom Lane
Andrew Dunstan writes: > Well, the original code was put in for a reason, presumably that we were > getting some stale data and wanted to exclude it. So I'm unwilling to > throw it out altogether. If someone can propose a reasonable sanity > check then I'm prepared to implement it. Would it be

[HACKERS] Negative imact of maintenance_work_mem to GIN size

2014-05-19 Thread Alexander Korotkov
Hi! I found that sometimes larger maintenance_work_mem leads to larger GIN index. That is quite strange. ISTM that it's related to posting lists compression but I can't figure out how exactly it is. See example on delicious bookmarks dataset. http://mira.sai.msu.ru/~megera/tmp/js.copy.gz set ma

Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-19 Thread Andrew Dunstan
On 05/19/2014 05:37 PM, Tomas Vondra wrote: IMHO the problem is that d6a97674 was the last revision in the REL9_3_STABLE branch when the test started (00:14), but at 06:06 777d07d7 got committed. So the check at the end failed, because the tested revision was suddenly ~2 days over the limit. T

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-19 Thread Tom Lane
Andres Freund writes: > On 2014-05-19 23:40:32 +0200, Tomas Vondra wrote: >> I was however wondering if this might be related to OOM errors a few >> local users reported to us. IIRC they've been using temporary tables >> quite heavily - not sure if that could be related. > I've significant doubts

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-05-19 Thread Thomas Munro
Hello As a simple example for people wondering what the point of this feature is, I created a table work (id, data, status) and then create 10,000 items with status 'NEW' and then started a number of worker threads that did the following pair of transactions, with and without SKIP LOCKED DATA on t

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-19 Thread Andres Freund
On 2014-05-19 23:40:32 +0200, Tomas Vondra wrote: > On 19.5.2014 22:11, Tom Lane wrote: > > Tomas Vondra writes: > > I intentionally didn't do that, first because I have only a limited > > amount of confidence in the patch, and second because I don't think > > it matters for anything except CLOBBE

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-19 Thread Tomas Vondra
On 19.5.2014 22:11, Tom Lane wrote: > Tomas Vondra writes: >> On 18.5.2014 20:49, Tom Lane wrote: >>> With both of these things fixed, I'm not seeing any significant memory >>> bloat during the first parallel group of the regression tests. I don't >>> think I'll have the patience to let it run mu

Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-19 Thread Tomas Vondra
On 19.5.2014 23:04, Andrew Dunstan wrote: > > On 05/19/2014 03:40 PM, Tomas Vondra wrote: >> On 17.5.2014 22:35, Tomas Vondra wrote: >>> On 17.5.2014 19:58, Andrew Dunstan wrote: On 05/15/2014 07:47 PM, Tomas Vondra wrote: > On 15.5.2014 22:07, Andrew Dunstan wrote: >> Yes, I've seen

Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-19 Thread Andrew Dunstan
On 05/19/2014 03:40 PM, Tomas Vondra wrote: On 17.5.2014 22:35, Tomas Vondra wrote: On 17.5.2014 19:58, Andrew Dunstan wrote: On 05/15/2014 07:47 PM, Tomas Vondra wrote: On 15.5.2014 22:07, Andrew Dunstan wrote: Yes, I've seen that. Frankly, a test that takes something like 500 hours is a bi

Re: [HACKERS] Minor jsonb comment fix

2014-05-19 Thread Tom Lane
Peter Geoghegan writes: > Attached patch makes minor adjustment to a comment that is no longer > accurate (as of commit ff7bbb01). Applied, thanks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

[HACKERS] Minor jsonb comment fix

2014-05-19 Thread Peter Geoghegan
Attached patch makes minor adjustment to a comment that is no longer accurate (as of commit ff7bbb01). -- Peter Geoghegan *** a/src/backend/utils/adt/jsonb_util.c --- b/src/backend/utils/adt/jsonb_util.c *** convertJsonbScalar(StringInfo buffer, JE *** 1462,1468 /* * Compare

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-19 Thread Tom Lane
Tomas Vondra writes: > On 18.5.2014 20:49, Tom Lane wrote: >> With both of these things fixed, I'm not seeing any significant memory >> bloat during the first parallel group of the regression tests. I don't >> think I'll have the patience to let it run much further than that >> (the uuid and enum

Re: [HACKERS] 9.4 checksum error in recovery with btree index

2014-05-19 Thread Jeff Janes
On Mon, May 19, 2014 at 3:32 AM, Heikki Linnakangas wrote: > On 05/18/2014 06:30 AM, Jeff Janes wrote: > >> On Saturday, May 17, 2014, Heikki Linnakangas >> wrote: >> >>> >>> The seems to have fixed it. >> > > Okay, thanks, committed. > > Your torn-page generator seems to be very good at fi

Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-19 Thread Tomas Vondra
On 17.5.2014 22:35, Tomas Vondra wrote: > On 17.5.2014 19:58, Andrew Dunstan wrote: >> >> On 05/15/2014 07:47 PM, Tomas Vondra wrote: >>> On 15.5.2014 22:07, Andrew Dunstan wrote: Yes, I've seen that. Frankly, a test that takes something like 500 hours is a bit crazy. >>> Maybe. It certai

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-19 Thread Tomas Vondra
On 18.5.2014 20:49, Tom Lane wrote: > With both of these things fixed, I'm not seeing any significant memory > bloat during the first parallel group of the regression tests. I don't > think I'll have the patience to let it run much further than that > (the uuid and enum tests are still running aft

Re: [HACKERS] uuid-ossp (Re: [pgsql-packagers] Postgresapp 9.4 beta build ready)

2014-05-19 Thread Peter Eisentraut
On 5/18/14, 12:28 AM, Tom Lane wrote: > So, having seen that proof-of-concept, I'm wondering if we shouldn't make > an effort to support contrib/uuid-ossp with a choice of UUID libraries > underneath it. There is a non-OSSP set of UUID library functions > available on Linux ("libuuid" from util-li

Re: [HACKERS] vacuumdb --all --analyze-in-stages - wrong order?

2014-05-19 Thread Peter Eisentraut
On 5/18/14, 3:52 AM, Pavel Stehule wrote: > Hello > > I am looking on --analyze-in-stages option. If I understand well, > motivation for this option is a get some minimal statistic for databases > in minimal time. But when I tested, I found so iterations are per > databases, not per stages - some

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-19 Thread Tom Lane
Andres Freund writes: > On 2014-05-19 11:25:04 -0400, Tom Lane wrote: >> No, we'd have two independent entries, each with its own correct refcount. >> When the refcount on the no-longer-linked-in-the-hashtable entry goes to >> zero, it'd be leaked, same as it's always been. (The refcount presumab

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-19 Thread Tom Lane
Christoph Berg writes: > FTR, I've just had a look at armhf (arm-linux-gnueabihf), the address > layout looks exactly the same there, and 9.3 crashes easily, so it's > really a problem of all Linux 32bit archs. I'm puzzled the regression > tests passed there [1], but anyway, we'll probably get rid

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-19 Thread Christoph Berg
Re: To Tom Lane 2014-05-19 <20140519144717.gg7...@msgid.df7cb.de> > Disabling -pie for all 32bit archs seems to be the way to go for us > now. FTR, I've just had a look at armhf (arm-linux-gnueabihf), the address layout looks exactly the same there, and 9.3 crashes easily, so it's really a problem

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-19 Thread Andres Freund
On 2014-05-19 11:25:04 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-05-18 14:49:10 -0400, Tom Lane wrote: > >> if (RelationHasReferenceCountZero(oldrel)) > >> RelationDestroyRelation(oldrel, false); > >> else > >> elog(WARNING, "leaking still-referenced duplicate relation");

Re: [HACKERS] 9.4 release notes

2014-05-19 Thread David Johnston
On Mon, May 19, 2014 at 12:36 AM, Bruce Momjian wrote: > On Thu, May 15, 2014 at 06:08:47PM -0700, David G Johnston wrote: > > Some errors and suggestions - my apologizes for the format as I do not > have > > a proper patching routine setup. > > > > Patch Review - Top to Bottom (mostly, I think..

Re: [HACKERS] 9.4 release notes

2014-05-19 Thread David Johnston
On Mon, May 19, 2014 at 10:23 AM, Bruce Momjian wrote: > On Thu, May 15, 2014 at 06:08:47PM -0700, David G Johnston wrote: > > Some errors and suggestions - my apologizes for the format as I do not > have > > a proper patching routine setup. > > > > Sorry, let me address some items I skipped on y

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-19 Thread Tom Lane
Andres Freund writes: > On 2014-05-18 14:49:10 -0400, Tom Lane wrote: >> if (RelationHasReferenceCountZero(oldrel)) >> RelationDestroyRelation(oldrel, false); >> else >> elog(WARNING, "leaking still-referenced duplicate relation"); > If that happens we'd essentially have a too low referen

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-19 Thread Andres Freund
Hi, On 2014-05-18 14:49:10 -0400, Tom Lane wrote: > AFAICT, the inner invocation's Relation should always have zero reference > count by the time we get back to the outer invocation. Therefore it > should be possible for RelationCacheInsert to just delete the > about-to-be-unreachable Relation st

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-19 Thread Christoph Berg
Re: Andres Freund 2014-05-19 <20140519141221.gc5...@alap3.anarazel.de> > On 2014-05-19 09:53:11 -0400, Tom Lane wrote: > > I think throwing an error out of a SIGBUS handler is right out. There > > would be no way to know exactly what code we were interrupting. It's > > the same reason we don't le

Re: [HACKERS] 9.4 release notes

2014-05-19 Thread Bruce Momjian
On Thu, May 15, 2014 at 06:08:47PM -0700, David G Johnston wrote: > Some errors and suggestions - my apologizes for the format as I do not have > a proper patching routine setup. > Sorry, let me address some items I skipped on your list: > IIUC: Logical decoding allows for streaming of statement

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-19 Thread Andres Freund
On 2014-05-19 09:53:11 -0400, Tom Lane wrote: > I think throwing an error out of a SIGBUS handler is right out. There > would be no way to know exactly what code we were interrupting. It's > the same reason we don't let, eg, the SIGALRM handler throw a timeout > error directly (in most places any

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-19 Thread Tom Lane
Andres Freund writes: > Isn't the far more obvious thing ot just not build postgres with -pie on > 32bit? It's hardly a security benefit if it allows plain user to crash > the server. Yeah, that's what I was doing when I was at Red Hat --- PIE mode would be nice, but not when it breaks basic func

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-19 Thread Andres Freund
Hi, On 2014-05-19 13:53:18 +0200, Christoph Berg wrote: > * PostgreSQL allocates lots of heap using brk() instead of mmap() It doesn't really do that, btw. It's the libc's mmap that makes those decisions, not postgres. Greetings, Andres Freund -- Andres Freund http://www.

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-19 Thread Andres Freund
On 2014-05-19 13:53:18 +0200, Christoph Berg wrote: > I've done some more digging. The problem exists also on plain 32bit > kernels, not only 64bit running a 32bit userland. (Tested on Debian > Wheezy's 3.2.57 kernel.) Too bad. > Debian/Ubuntu have been using hardened PostgreSQL builds for years

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-19 Thread Christoph Berg
Re: To Tom Lane 2014-05-19 <20140519091808.ga7...@msgid.df7cb.de> > Re: Tom Lane 2014-05-18 <26862.1400449...@sss.pgh.pa.us> > > OK, so the problem is that getrlimit(RLIMIT_STACK) is lying to us about > > the available stack depth. I'd classify that as a kernel bug. I wonder > > if it's a differe

Re: [HACKERS] pg_isready --username seems an empty promise

2014-05-19 Thread Heikki Linnakangas
On 05/19/2014 01:37 PM, Erik Rijkers wrote: pg_isready has --username: -U, --username=USERNAME user name to connect as so is replying when given a non-existent user not a bug? pg_isready --username= -p 6544 /tmp:6544 - accepting connections There is no user . (PG envvars are rem

Re: [HACKERS] cosmetic fixes

2014-05-19 Thread Heikki Linnakangas
On 05/19/2014 02:36 AM, Euler Taveira wrote: Hi, Here are some more trivial fixes in pg_recvlogical message style. Thanks, applied. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-h

[HACKERS] pg_isready --username seems an empty promise

2014-05-19 Thread Erik Rijkers
pg_isready has --username: -U, --username=USERNAME user name to connect as so is replying when given a non-existent user not a bug? pg_isready --username= -p 6544 /tmp:6544 - accepting connections There is no user . (PG envvars are removed) Thanks, Erik Rijkers -- Sent

Re: [HACKERS] 9.4 checksum error in recovery with btree index

2014-05-19 Thread Heikki Linnakangas
On 05/18/2014 06:30 AM, Jeff Janes wrote: On Saturday, May 17, 2014, Heikki Linnakangas wrote: On 05/17/2014 12:28 AM, Jeff Janes wrote: More fun with my torn page injection test program on 9.4. 24171 2014-05-16 14:00:44.934 PDT:WARNING: 01000: page verification failed, calculated checksu

Re: [HACKERS] Allowing join removals for more join types

2014-05-19 Thread Dilip kumar
On 19 May 2014 12:15 David Rowley Wrote, >I think you are right here, it would be correct to remove that join, but I >also think that the query in question could be quite easily be written as: >select t1.a from t1 left join t2 on t1.a=t2.b; >Where the join WILL be removed. The distinct clause h

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-19 Thread Christoph Berg
Re: Tom Lane 2014-05-18 <26862.1400449...@sss.pgh.pa.us> > OK, so the problem is that getrlimit(RLIMIT_STACK) is lying to us about > the available stack depth. I'd classify that as a kernel bug. I wonder > if it's a different manifestation of this issue: > https://bugzilla.redhat.com/show_bug.cgi

Re: [HACKERS] Re: [GENERAL] Is it typo in pg_stat_replication column name in PG 9.4 ?

2014-05-19 Thread Raghavendra
​ On Mon, May 19, 2014 at 6:35 AM, Fabrízio de Royes Mello < fabri...@timbira.com.br> wrote: > On 18-05-2014 05:40, Raghavendra wrote: > > Hi, > > > > PostgreSQL 9.4 document for pg_stat_replication view mentions column name > > as "backend_xid", whereas when a view described it shows column name

Re: [HACKERS] Re: [GENERAL] Is it typo in pg_stat_replication column name in PG 9.4 ?

2014-05-19 Thread Fujii Masao
On Mon, May 19, 2014 at 10:05 AM, Fabrízio de Royes Mello wrote: > On 18-05-2014 05:40, Raghavendra wrote: >> Hi, >> >> PostgreSQL 9.4 document for pg_stat_replication view mentions column name >> as "backend_xid", whereas when a view described it shows column name as >> "backend_xmin". >> http://