Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-31 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera <alvhe...@2ndquadrant.com> writes: > > ... However, when you create an index, you can > > indicate which operator class to use, and it may not be the default one. > > If a different one is chosen at index creation time, then a que

Re: [HACKERS] [patch] pg_dump/pg_restore zerror() and strerror() mishap

2017-07-28 Thread Alvaro Herrera
Kunshchikov Vladimir wrote: > Hello Alvaro, > > here goes v4 version: removed unused header. > > Compilation of this code snippet with -Wall -Wexter -std=c89 doesn't produce > any warnings. Great, thanks. +const char * +get_cfp_error(cfp* fp) +{ +#ifdef HAVE_LIBZ + if(fp->compressedfp){

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-29 Thread Alvaro Herrera
Mark Rofail wrote: > These are limitations of the patch ordered by importance: > > ✗ presupposes that count(distinct y) has exactly the same notion of > equality that the PK unique index has. In reality, count(distinct) will > fall back to the default btree opclass for the array element type.

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-07-31 Thread Alvaro Herrera
Robert Haas wrote: > An alternative approach is to have some kind of other identifier, > let's call it a distributed transaction ID (DXID) which is mapped by > each node onto a local XID. Postgres-XL seems to manage this problem by using a transaction manager node, which is in charge of

Re: [HACKERS] [patch] pg_dump/pg_restore zerror() and strerror() mishap

2017-08-02 Thread Alvaro Herrera
Kunshchikov Vladimir wrote: > Hello Alvaro, > > here goes v4 version: removed unused header. > > Compilation of this code snippet with -Wall -Wexter -std=c89 doesn't produce > any warnings. Great, thanks! I have pushed this to all branches since 9.4. Would you please give it a look? Please

Re: [HACKERS] Subscription code improvements

2017-08-02 Thread Alvaro Herrera
Petr Jelinek wrote: > I split it into several patches: I wish you'd stop splitting error message strings across multiple lines. I've been trapped by a faulty grep not matching a split error message a number of times :-( I know by now to remove words until I get a match, but it seems an

[HACKERS] fixing pg_upgrade strings (was Re: pgsql: Add new files to nls.mk and add translation)

2017-08-02 Thread Alvaro Herrera
Peter Eisentraut wrote: > Add new files to nls.mk and add translation markers This reminds me that I noticed a few days ago another really serious broken piece in pg_upgrade where check_required_directory() is incurring in the ugliest case of string building I've ever seen. I didn't have the

Re: [HACKERS] [COMMITTERS] pgsql: Fix pg_dump's errno checking for zlib I/O

2017-08-02 Thread Alvaro Herrera
Alvaro Herrera wrote: > Fix pg_dump's errno checking for zlib I/O So this broke a few buildfarm members. I'll into it tomorrow. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-

Re: [HACKERS] Macros bundling RELKIND_* conditions

2017-08-03 Thread Alvaro Herrera
Joe Conway wrote: > On 08/02/2017 10:52 PM, Ashutosh Bapat wrote: > > On Wed, Aug 2, 2017 at 11:15 PM, Alvaro Herrera > > <alvhe...@2ndquadrant.com> wrote: > >> Alvaro Herrera wrote: > >>> I think pg_class is a reasonable place to put more generic relk

Re: [HACKERS] Better way to handle suppression of CASCADE detail messages

2017-08-01 Thread Alvaro Herrera
Andres Freund wrote: > On 2017-08-01 13:48:34 -0400, Robert Haas wrote: > > On Tue, Aug 1, 2017 at 1:39 PM, Andres Freund wrote: > > > Oid is probably not good enough - with parallel tests and such it's not > > > necessarily predicable. Even less so when the tests are run

Re: [HACKERS] elog vs errmsg_internal

2017-08-03 Thread Alvaro Herrera
Peter Eisentraut wrote: > Is there a preferred method to select between using elog() and > errmsg_internal()? > > Attached is a patch that converts some DEBUG messages to one of those > two to remove them from translation, but I'm not sure which one to pick > other than by random aesthetics. I

Re: [HACKERS] Macros bundling RELKIND_* conditions

2017-08-03 Thread Alvaro Herrera
Tom Lane wrote: > I think Peter's got the error and the detail backwards. It should be > more like > > ERROR: "someview" cannot have constraints > DETAIL: "someview" is a view. > > If we do it like that, we need one ERROR message per error reason, > and one DETAIL per relkind, which should be

Re: [HACKERS] Log LDAP "diagnostic messages"?

2017-08-15 Thread Alvaro Herrera
Christoph Berg wrote: > Re: Thomas Munro 2017-08-10 >

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-15 Thread Alvaro Herrera
Tom Lane wrote: > Robert Haas writes: > > I really, really strongly encourage you to rip the use of DSA out here > > entirely. It is reducing the reliability of a critical part of the > > system for no actual benefit other than speculation that this is going > > to be

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Aug 14, 2017 at 1:12 PM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: > > Yeah, the problem that lwlocks aren't released is because the launcher > > is not in a transaction at that point, so AbortCurrentTransaction() > > doesn't

Re: [HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-14 Thread Alvaro Herrera
Michael Paquier wrote: > On Tue, Aug 15, 2017 at 10:27 AM, Masahiko Sawada > wrote: > > Currently vacuum verbose outputs vacuum logs as follows. The first log > > message INFO: vacuuming "public.hoge" writes the relation name with > > schema name but subsequent vacuum logs

Re: [HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-14 Thread Alvaro Herrera
Michael Paquier wrote: > On Tue, Aug 15, 2017 at 11:14 AM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: > >> In vacuum_rel()@vacuum.c, there are a couple of logs that could be > >> improved as well with the schema name. > > > > I agree

Re: [HACKERS] Comment in snapbuild.c file

2017-08-10 Thread Alvaro Herrera
Masahiko Sawada wrote: > Hi all, > > In snapbuild.c file, there is a comment as follows. > >* NB: Because of that xmax can be lower than xmin, because we only >* increase xmax when a catalog modifying transaction commits. While odd >* looking, it's correct and actually more efficient

Re: [HACKERS] Comment in snapbuild.c file

2017-08-12 Thread Alvaro Herrera
Alvaro Herrera wrote: > Masahiko Sawada wrote: > > Hi all, > > > > In snapbuild.c file, there is a comment as follows. > > > >* NB: Because of that xmax can be lower than xmin, because we only > >* increase xmax when a catalog modifying transact

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Alvaro Herrera
intend to give these patches further thought before pushing anything, will update this thread no later than tomorrow 19:00 UTC-0300. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From d761ef2c2fef336fae908d802237

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Alvaro Herrera
Robert Haas wrote: > Actually, now that you mention it, I think it *is* broken already, or > more to the point, if you configure that value, the autovacuum > launcher hangs up, because of the AutovacuumWorkItem stuff that Alvaro > added. When I just tested it, the AV launcher somehow ended up >

Re: [HACKERS] Replacing lfirst() with lfirst_node() appropriately in planner.c

2017-07-13 Thread Alvaro Herrera
Ashutosh Bapat wrote: > Happened to stumble across some instances of lfirst() which could use > lfirst_node() in planner.c. Here's patch which replaces calls to > lfirst() extracting node pointers by lfirst_node() in planner.c. Sounds good. > Are we carrying out such replacements in master or

Re: [HACKERS] PostgreSQL - Weak DH group

2017-07-13 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Jul 13, 2017 at 5:32 PM, Heikki Linnakangas wrote: > > Objections to committing this now, instead of waiting for v11? > > But I am -1 for the sneak part. It is not the time to have a new > feature in 10, the focus is to stabilize. But if we

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-07-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > Here is the query: > > with recursive index_details as ( > select > 'pgbench_accounts_pkey'::text idx > ), [...] Hmm, this seems potentially very useful. Care to upload it to https://wiki.postgresql.org/wiki/Category:Snippets ? -- Álvaro Herrera

Re: [HACKERS] More race conditions in logical replication

2017-07-07 Thread Alvaro Herrera
Petr Jelinek wrote: > So best idea I could come up with is to make use of the new condition > variable API. That lets us wait for variable which can be set per slot. > > It's not backportable however, I am not sure how big problem that is > considering the lack of complaints until now (maybe we

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-11 Thread Alvaro Herrera
Amit Kapila wrote: > Yes, I also think the same idea can be used, in fact, I have mentioned > it [1] as soon as you have committed that patch. Do we want to do > anything at this stage for PG-10? I don't think we should attempt > something this late unless people feel this is a show-stopper

Re: [HACKERS] More race conditions in logical replication

2017-07-10 Thread Alvaro Herrera
Alvaro Herrera wrote: > I'll next update this on or before Monday 10th at 19:00 CLT. I couldn't get to this today as I wanted. Next update on Wednesday 12th, same time. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Train

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote > wrote: > > Actually, if \d had shown RELKIND_PARTITIONED_TABLE tables as of Type > > "partitioned table", we wouldn't need a separate flag for marking a table > > as having partitions. > > I

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-10 Thread Alvaro Herrera
Amit Kapila wrote: > On Tue, Jul 11, 2017 at 6:51 AM, AP wrote: > > On Fri, Jul 07, 2017 at 05:58:25PM +0530, Amit Kapila wrote: > >> I can understand your concerns. To address first concern we need to > >> work on one or more of following work items: (a) work on vacuums that >

Re: [HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Alvaro Herrera
Claudio Freire wrote: > A missing optimization is that having tid unification allows VACUUM to > implement a different strategy when it needs to clean up only a tiny > fraction of the index. It can do the lookup by key-tid instead of > scanning the whole index, which can be a win if the index is

Re: [HACKERS] [COMMITTERS] pgsql: pg_test_timing: Add NLS

2017-07-06 Thread Alvaro Herrera
Peter Eisentraut wrote: > pg_test_timing: Add NLS > > Also straighten out use of time unit abbreviations a bit. We (well, Carlos Chapi, who's doing the translation work now) just noticed that this has a bug in this line + printf("%6s %10s %10s\n", _("< us"), _("% of total"), _("count"));

[HACKERS] bug in locking an update tuple chain

2017-07-14 Thread Alvaro Herrera
in the referenced table. It worked fine for them on 9.3.4. After some research, we determined that the problem disappeared if commit this commit was reverted: Author: Alvaro Herrera <alvhe...@alvh.no-ip.org> Branch: master Release: REL9_6_BR [533e9c6b0] 2016-07-15 14:17:20 -0400 Branch: REL9_5_STABLE R

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-14 Thread Alvaro Herrera
Mark Rofail wrote: > On Wed, Jul 12, 2017 at 2:30 PM, Mark Rofail <markm.rof...@gmail.com> wrote: > > > On Wed, Jul 12, 2017 at 12:53 AM, Alvaro Herrera <alvhe...@2ndquadrant.com > > > wrote: > >> > >> We have one opclass for each type combination

Re: [HACKERS] More race conditions in logical replication

2017-07-15 Thread Alvaro Herrera
After going over this a bunch more times, I found other problems. For example, I noticed that if I create a temporary slot in one session, then another session would rightly go to sleep if it tries to drop that slot. But the slot goes away when the first session disconnects, so I'd expect the

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-18 Thread Alvaro Herrera
Mark Rofail wrote: > On Tue, 18 Jul 2017 at 7:43 pm, Alexander Korotkov > wrote: > > > separate operators for int4[] @>> int8, int4[] @>> int4, int4[] @>> int2, > > int4[] @>> numeric. > > > > My only comment on the separate operators is its high maintenance. Any new >

Re: [HACKERS] More race conditions in logical replication

2017-07-18 Thread Alvaro Herrera
Alvaro Herrera wrote: > After going over this a bunch more times, I found other problems. For > example, I noticed that if I create a temporary slot in one session, > then another session would rightly go to sleep if it tries to drop that > slot. But the slot goes away when the f

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-18 Thread Alvaro Herrera
Alexander Korotkov wrote: > The problem is that you need to have not only opclass entries for the > operators, but also operators themselves. I.e. separate operators for > int4[] @>> int8, int4[] @>> int4, int4[] @>> int2, int4[] @>> numeric. You > tried to add multiple pg_amop rows for single

Re: [HACKERS] Cache lookup errors with functions manipulation object addresses

2017-07-20 Thread Alvaro Herrera
Michael Paquier wrote: > - getObjectDescription and getObjectIdentity are called in quite a > couple of places. We could have those have a kind of missing_ok, but > as the status is just for adding cache lookup errors I have kept the > interface simple as this keeps the code in objectaddress.c

Re: [HACKERS] Cache lookup errors with functions manipulation object addresses

2017-07-20 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Jul 20, 2017 at 4:04 PM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: > > I think the addition of checks everywhere for NULL return is worse. > > Let's add a missing_ok flag instead, so that most callers can just trust > >

Re: [HACKERS] PgFDW connection invalidation by ALTER SERVER/ALTER USER MAPPING

2017-07-20 Thread Alvaro Herrera
Kyotaro HORIGUCHI wrote: > Finally, I added a new TAP test library PsqlSession. It offers > interactive psql sessions. Then added a simple test to > postgres_fdw using it. Hmm, I think this can be very useful for other things. Let's keep this in mind to use in the future, even if we find

Re: [HACKERS] Locale dependency in new postgres_fdw test

2017-07-21 Thread Alvaro Herrera
Tom Lane wrote: > We could stabilize this test result by forcing lc_messages = C in > the foreign server options. However, that would lose regression > coverage of situations where the remote locale is different, which > doesn't seem like a terribly good thing. I don't understand what is the

Re: [HACKERS] More race conditions in logical replication

2017-07-21 Thread Alvaro Herrera
I'm back at looking into this again, after a rather exhausting week. I think I have a better grasp of what is going on in this code now, and it appears to me that we should change the locking so that active_pid is protected by ReplicationSlotControlLock instead of the slot's spinlock; but I

Re: [HACKERS] More race conditions in logical replication

2017-07-25 Thread Alvaro Herrera
Petr Jelinek wrote: > On 25/07/17 01:33, Alvaro Herrera wrote: > > Alvaro Herrera wrote: > >> I'm back at looking into this again, after a rather exhausting week. I > >> think I have a better grasp of what is going on in this code now, > > > > Here's a

Re: [HACKERS] [patch] pg_dump/pg_restore zerror() and strerror() mishap

2017-07-25 Thread Alvaro Herrera
Kunshchikov Vladimir wrote: > Errors should be like this: > pg_restore: [compress_io] could not read from input file: d3/2811.dat.gz: > invalid distance too far back > > Attached small fix for this issue. After looking at this patch, I don't like it very much. In particular, I don't like the

Re: [HACKERS] TAP: allow overriding PostgresNode in get_new_node

2017-07-25 Thread Alvaro Herrera
Chapman Flack wrote: > Any takers if I propose this amendment in the form of a patch? > > Relying on the perl idiom instead of a $node->isa() test shortens > the patch; does that ameliorate at all the concern about complicating > core for the benefit of modules? Yeah, I like this (I also got

Re: [HACKERS] More race conditions in logical replication

2017-07-24 Thread Alvaro Herrera
Alvaro Herrera wrote: > I'm back at looking into this again, after a rather exhausting week. I > think I have a better grasp of what is going on in this code now, Here's an updated patch, which I intend to push tomorrow. > and it > appears to me that we should change th

Re: [HACKERS] More race conditions in logical replication

2017-07-27 Thread Alvaro Herrera
Alvaro Herrera wrote: > Alvaro Herrera wrote: > > > Hmm, yeah, that's not good. However, I didn't like the idea of putting > > it inside the locked area, as it's too much code. Instead I added just > > before acquiring the spinlock. We cancel the sleep unconditionally &

Re: [HACKERS] Increase Vacuum ring buffer.

2017-07-27 Thread Alvaro Herrera
Claudio Freire wrote: > On Thu, Jul 27, 2017 at 1:46 PM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: > > Claudio Freire wrote: > > > >> > The vacuuming the very large table with no index could also take a > >> > long time, and it sc

Re: [HACKERS] Increase Vacuum ring buffer.

2017-07-27 Thread Alvaro Herrera
Claudio Freire wrote: > > The vacuuming the very large table with no index could also take a > > long time, and it scans and vacuums blocks one by one. So I imagined > > that we can vacuum the FSM once vacuumed a certain amount of blocks. > > And that can avoid bloating table during the long-time

Re: [HACKERS] Remove old comments in dependencies.c and README.dependencies

2017-07-26 Thread Alvaro Herrera
atorikoshi wrote: > Attached patch removes the comments about min_group_size. Agreed. Removed those comments. Thanks for the patch. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers

Re: [HACKERS] bug in locking an update tuple chain

2017-07-26 Thread Alvaro Herrera
Amit Kapila wrote: > On Sat, Jul 15, 2017 at 2:30 AM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: > > a transaction wants to lock the > > updated version of some tuple, and it does so; and some other > > transaction is also locking the same tuple concurrentl

Re: [HACKERS] expand_dbname in postgres_fdw

2017-07-26 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Jul 26, 2017 at 5:38 AM, Ashutosh Bapat > wrote: > > According to F.34.1.1 at [1] passing connection string as dbname > > option should work, so your question is valid. I am not aware of any > > discussion around this on hackers.

Re: [HACKERS] [patch] pg_dump/pg_restore zerror() and strerror() mishap

2017-07-26 Thread Alvaro Herrera
Kunshchikov Vladimir wrote: > Hello Alvaro, thanks for the feedback, fixed all of your points. > Attached new version of patch. Looks great -- it's a lot smaller than the original even. One final touch -- see cfread(), where we have an #ifdef where we test for fp->compressedfp; the "#else"

Re: [HACKERS] bug in locking an update tuple chain

2017-07-26 Thread Alvaro Herrera
> The attached patch fixes the problem. When locking some old tuple version of > the chain, if we detect that we already hold that lock > (test_lockmode_for_conflict returns HeapTupleSelfUpdated), do not try to lock > it again but instead skip ahead to the next version. This fixes the synthetic

Re: [HACKERS] More race conditions in logical replication

2017-07-26 Thread Alvaro Herrera
Alvaro Herrera wrote: > Hmm, yeah, that's not good. However, I didn't like the idea of putting > it inside the locked area, as it's too much code. Instead I added just > before acquiring the spinlock. We cancel the sleep unconditionally > later on if we didn't need to sleep after

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-19 Thread Alvaro Herrera
Mark Rofail wrote: > On Tue, Jul 18, 2017 at 11:14 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > > > > Why did we add an operator and not a support > > procedure? > > I thought the support procedures were constant within an opclass. Uhh ... I

Re: [HACKERS] [TRAP: FailedAssertion] causing server to crash

2017-07-19 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Jul 19, 2017 at 8:26 AM, Neha Sharma > wrote: > > I am getting FailedAssertion while executing the attached script.However,I > > am not able to produce the core dump for the same,the script runs in > > background and takes around a day

Re: [HACKERS] autovacuum can't keep up, bloat just continues to rise

2017-07-19 Thread Alvaro Herrera
Peter Geoghegan wrote: > Index bloat is a general problem that B-Trees have in all other major > systems, but I think that PostgreSQL has a tendency to allow indexes > to become progressively more bloated over time, in a way that it often > can never recover from [1]. Interesting assertion.

Re: [HACKERS] New partitioning - some feedback

2017-07-12 Thread Alvaro Herrera
Amit Langote wrote: > On 2017/07/11 13:34, Alvaro Herrera wrote: > > Robert Haas wrote: > >> On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote > >> <langote_amit...@lab.ntt.co.jp> wrote: > > > >>> Actually, if \d had shown RELKIND_PARTITIONED_TA

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-07-12 Thread Alvaro Herrera
Peter Geoghegan wrote: > Now, that might not seem like that much of a difference, but if you > consider how duplicates are handled in the B-Tree code, and how unique > index enforcement works, I think it could be. It could lead to heavy > buffer lock contention, because we sometimes do a lot of

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-11 Thread Alvaro Herrera
Mark Rofail wrote: >- now the RI checks utilise the @>(anyarray, anyelement) > - however there's a small problem: > operator does not exist: integer[] @> smallint > I assume that external casting would be required here. But how can I > downcast smallint to integer or

Re: [HACKERS] More race conditions in logical replication

2017-07-12 Thread Alvaro Herrera
te DBA, Training & Services >From 55533aa3cdc2fecbf7b6b6c661649342a204e73b Mon Sep 17 00:00:00 2001 From: Alvaro Herrera <alvhe...@alvh.no-ip.org> Date: Wed, 12 Jul 2017 18:38:33 -0400 Subject: [PATCH v3 1/1] Wait for slot to become free in when dropping it --- src/backend/

Re: [HACKERS] [COMMITTERS] pgsql: Add a Gather executor node.

2017-07-21 Thread Alvaro Herrera
Robert Haas wrote: > I think those top-of-file lists are just annoying, and would prefer to > rip them out entirely rather than spend time maintaining them. +1 -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Reducing pg_ctl's reaction time

2017-06-28 Thread Alvaro Herrera
Tom Lane wrote: > Andres Freund writes: > So when I removed the miscadmin.h include, I found out that pg_ctl is > also relying on PG_BACKEND_VERSIONSTR from that file. > > There are at least three things we could do here: > > 1. Give this up as not worth this much trouble.

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-29 Thread Alvaro Herrera
Noah Misch wrote: > On Wed, Jun 28, 2017 at 06:52:14PM -0400, Alvaro Herrera wrote: > > I think I'm done with the walsender half of this patch; I still need to > > review the walreceiver part. I will report back tomorrow 19:00 CLT. > > This PostgreSQL 10 open item is pas

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-28 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Jun 8, 2017 at 3:31 AM, Robert Haas wrote: > > I think if you're going to fix it so that we take spinlocks on > > MyWalSnd in a bunch of places that we didn't take them before, it > > would make sense to fix all the places where we're

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-30 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Jun 29, 2017 at 7:52 AM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: > > I think I'm done with the walsender half of this patch; I still need to > > review the walreceiver part. I will report back tomorrow 19:00 CLT. > >

Re: [HACKERS] Race-like failure in recovery/t/009_twophase.pl

2017-07-02 Thread Alvaro Herrera
Tom Lane wrote: > I'd kind of like to fix it now, so I can reason in a less confused way > about the actual problem. OK, no objections here. > Last night I didn't have a clear idea of how > to make it better, but what I'm thinking this morning is: > > * Naming the underlying server objects

Re: [HACKERS] Race-like failure in recovery/t/009_twophase.pl

2017-07-02 Thread Alvaro Herrera
Tom Lane wrote: > Part of the reason I'm confused is that the programming technique > being used in 009_twophase.pl, namely doing > > ($node_master, $node_slave) = ($node_slave, $node_master); > > and then working with the reversed variable names, is ENTIRELY TOO CUTE > FOR ITS OWN GOOD.

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-03 Thread Alvaro Herrera
Mark Rofail wrote: > On Mon, Jun 26, 2017 at 6:44 PM, Alexander Korotkov > wrote: > > > Have you met any particular problem here? Or is it just a lot of > > mechanical work? > > > > Just A LOT of mechanictal work, thankfully. The patch is now rebased and > all regress

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

2017-07-03 Thread Alvaro Herrera
Tom Lane wrote: > (Other unfinished work: teaching the MSVC scripts to use this, > and teaching pg_upgrade's test script to use it.) Maybe it'd be simpler to rewrite pg_upgrade tests using PostgresNode instead? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-07-03 Thread Alvaro Herrera
Michael Paquier wrote: > Thanks Álvaro for pushing the patch. I had a second look and the > result looks good to me. > > - SpinLockAcquire(>mutex); > + } > + pid = walsnd->pid; > The WAL receiver code used a cast to (int) in > pg_stat_get_wal_receiver(). I don't recall adding

Re: [HACKERS] Suspicious place in heap_prepare_freeze_tuple()

2017-07-05 Thread Alvaro Herrera
Teodor Sigaev wrote: > Playing around freezing tuple I found suspicious piece of code: > > heap_prepare_freeze_tuple(): > ... > frz->t_infomask = tuple->t_infomask; > ... > frz->t_infomask &= ~HEAP_XMAX_BITS; > frz->xmax = newxmax; > if (flags &

Re: [HACKERS] More race conditions in logical replication

2017-07-05 Thread Alvaro Herrera
Noah Misch wrote: > The above-described topic is currently a PostgreSQL 10 open item. Peter, > since you committed the patch believed to have created it, you own this open > item. I volunteer to own this item. My next update is going to be on or before Friday 7th at 19:00 Chilean time, though

Re: [HACKERS] Suspicious place in heap_prepare_freeze_tuple()

2017-07-05 Thread Alvaro Herrera
Masahiko Sawada wrote: > On Thu, Jul 6, 2017 at 1:36 AM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > > Teodor Sigaev wrote: > > > >> Playing around freezing tuple I found suspicious piece of code: > >> > >> heap_prepare_freeze_tuple()

Re: [HACKERS] Incorrect use of ERRCODE_UNDEFINED_COLUMN in extended stats

2017-04-24 Thread Alvaro Herrera
David Rowley wrote: > The attached small patched fixes an incorrect usage of an error code > in the extended stats code. Thanks for the report. I'm on vacation starting today until May 2nd. If nobody commits this in the meantime, I'll get to it when I'm back. -- Álvaro Herrera

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-04 Thread Alvaro Herrera
Robert Haas wrote: > I think this approach is actually better anyway. There's no guarantee > that VACUUM can be responsive enough to get the job done in time, work > items or no work items, Yeah, autovacuum work items don't have a guaranteed response time. They're okay for things that "ought to

Re: [HACKERS] Small code improvement for btree

2017-08-04 Thread Alvaro Herrera
Masahiko Sawada wrote: > While hacking the btree code I found two points we can improve in nbtxlog.c. > > @@ -135,7 +135,7 @@ _bt_clear_incomplete_split(XLogReaderState > *record, uint8 block_id) > Pagepage = (Page) BufferGetPage(buf); > BTPageOpaque

Re: [HACKERS] [patch] pg_dump/pg_restore zerror() and strerror() mishap

2017-07-28 Thread Alvaro Herrera
Vladimir Kunschikov wrote: > >This "maxlen" business and the fallback error message are > >strange. We have roughly equivalent code in pg_basebackup.c > >which has been working since 2011 > >Perhaps you can drop the memchr/fallback tricks and adopt the > >pg_basebackup coding? Or is there a

[HACKERS] Re: [COMMITTERS] pgsql: Fix inadequacies in recently added wait events

2017-08-08 Thread Alvaro Herrera
Thread moved to -hackers. Thomas Munro wrote: > On Wed, Aug 9, 2017 at 7:39 AM, Alvaro Herrera <alvhe...@alvh.no-ip.org> > wrote: > > While at it, fix numerous other problems in the vicinity: > All of the above seem like good candidates for a checker script in > src/t

Re: [HACKERS] Macros bundling RELKIND_* conditions

2017-08-02 Thread Alvaro Herrera
I think pg_class is a reasonable place to put more generic relkind lists alongside a matching error message for each, rather than specialized "does this relkind have storage" macros. What about something like a struct list in pg_class.h, { { relkinds_r_i_T, { 'r', 'i', 'T' },

Re: [HACKERS] Macros bundling RELKIND_* conditions

2017-08-02 Thread Alvaro Herrera
Alvaro Herrera wrote: > I think pg_class is a reasonable place to put more generic relkind lists > alongside a matching error message for each, rather than specialized > "does this relkind have storage" macros. What about something like a > struct list in pg_cl

Re: [HACKERS] Macros bundling RELKIND_* conditions

2017-08-02 Thread Alvaro Herrera
Peter Eisentraut wrote: > I don't find this style of error message optimal anyway. If I do, for > example > > ALTER TABLE someview ADD CONSTRAINT ... > ERROR: "someview" is not a table, foreign table, whatever > > then this information is not helpful. It's not like I'm going to turn > my view

Re: [HACKERS] More race conditions in logical replication

2017-08-08 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Aug 7, 2017 at 8:14 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > > BTW, I noticed that the PG_WAIT_LOCK value that we're using for wait > > event here (and in the replication slot case) is bogus. We probably > > need

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix inadequacies in recently added wait events

2017-08-09 Thread Alvaro Herrera
Tom Lane wrote: > I think generating whatever we can from a single authoritative file > is indeed a good idea. Yay. > But I had the impression that people also wanted to enforce a rule > about "only one use of each wait event name", which'd require a > checker script, no? (I'm not really

Re: [HACKERS] dubious error message from partition.c

2017-08-09 Thread Alvaro Herrera
Dean Rasheed wrote: > On 9 August 2017 at 13:03, Robert Haas wrote: > > On Tue, Aug 8, 2017 at 11:34 PM, Tom Lane wrote: > >> A small suggestion is that it'd be better to write it like "Specified > >> upper bound \"%s\" precedes lower bound \"%s\"." I

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix inadequacies in recently added wait events

2017-08-09 Thread Alvaro Herrera
This thread is surprising. If we generate the few lines of code being in trouble, we don't need any checker script, so I don't see why we'd go the route of the checker script instead. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA,

Re: [HACKERS] More race conditions in logical replication

2017-08-07 Thread Alvaro Herrera
Alvaro Herrera wrote: > Alvaro Herrera wrote: > > I just noticed that Jacana failed again in the subscription test, and it > > looks like it's related to this. I'll take a look tomorrow if no one > > beats me to it. > > https://buildfarm.postgresql.org/cgi-bin/show_

Re: [HACKERS] Adding hook in BufferSync for backup purposes

2017-08-07 Thread Alvaro Herrera
Андрей Бородин wrote: > ==What== > I propose to add hook inside BufferSync() function it inform extensions that > we > are going to write pages to disk. Please see patch attached. I pass a > timestamp > of the checkpoint, but it would be good if we could also pass there number of > checkpoint

[HACKERS] Re: [GSOC][weekly report 9] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-08-07 Thread Alvaro Herrera
Mengxing Liu wrote: > In the last week, I focus on: > > > 1) Continuing on optimization of skip list. Let me state once again that I'm certainly not an expert in the predicate locks module and that I hope Kevin will chime in with more useful feedback than mine. Some random thoughts: * I

Re: [HACKERS] Function to move the position of a replication slot

2017-08-16 Thread Alvaro Herrera
Andres Freund wrote: > On 2017-08-16 17:06:42 -0400, Tom Lane wrote: > > If I understand what this is meant to do, maybe better > > pg_move_replication_slot_lsn() or pg_change_replication_slot_lsn() ? > > The point being that you're adjusting the LSN pointer contained > > in the slot, which is

Re: [HACKERS] Proposal: global index

2017-08-19 Thread Alvaro Herrera
Ildar Musin wrote: > I found the thread about indirect indexes > (https://www.postgresql.org/message-id/20161018182843.xczrxsa2yd47pnru%40alvherre.pgsql), > but it seems that it haven't been updated for some time and I couldn't > find InMemoryIndexTuple in the latest patch. Is there a newer

Re: [HACKERS] Proposal: global index

2017-08-18 Thread Alvaro Herrera
Erikjan Rijkers wrote: > On 2017-08-18 11:12, Ildar Musin wrote: > > Hi hackers, > > > > While we've been developing pg_pathman extension one of the most > > frequent questions we got from our users was about global index > > support. We cannot provide it within an extension. And I couldn't find

Re: [HACKERS] Proposal: global index

2017-08-18 Thread Alvaro Herrera
Ildar Musin wrote: > While we've been developing pg_pathman extension one of the most frequent > questions we got from our users was about global index support. We cannot > provide it within an extension. And I couldn't find any recent discussion > about someone implementing it. So I'm thinking

Re: [HACKERS] [PATCH] Call RelationDropStorage() for broader range of object drops.

2017-09-14 Thread Alvaro Herrera
Hadi Moshayedi wrote: > To provide more context, in cstore_fdw creating the storage is easy, we > only need to hook into CREATE FOREIGN TABLE using event triggers. Removing > the storage is not that easy, for DROP FOREIGN TABLE we can use event > triggers. This all sounds a little more

Re: [HACKERS] psql: new help related to variables are not too readable

2017-09-13 Thread Alvaro Herrera
Most of the time I suppose you'd search (using the pager's search function) whatever you're looking for, rather than read the whole page from top to bottom. Why is it that we're not opening the pager automatically when this help is invoked via psql --help=variables? "\? variables" already does

Re: [HACKERS] DROP SUBSCRIPTION hangs if sub is disabled in the same transaction

2017-09-14 Thread Alvaro Herrera
Peter Eisentraut wrote: > On 9/13/17 09:56, Alvaro Herrera wrote: > > Tom Lane wrote: > >> Peter Eisentraut <peter.eisentr...@2ndquadrant.com> writes: > > > >>> - Disallow DROP SUBSCRIPTION in a transaction under certain > >>> circumstances, f

Re: [HACKERS] psql: new help related to variables are not too readable

2017-09-14 Thread Alvaro Herrera
Tom Lane wrote: > "David G. Johnston" writes: > >​If I was going to try and read it like a book I'd want the extra > > white-space to make doing so easier (white-space gives the eye a breather > > when done with a particular concept) - and the length wouldn't really >

Re: [HACKERS] Log LDAP "diagnostic messages"?

2017-09-14 Thread Alvaro Herrera
BTW I added --with-ldap and --with-pam to the configure line for the reports in coverage.postgresql.org and the % covered in auth.c went from 24% to 18.9% (from very bad to terribly sad). -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] Log LDAP "diagnostic messages"?

2017-09-14 Thread Alvaro Herrera
I think the ldap_unbind() changes should be in a separate preliminary patch to be committed separately and backpatched. The other bits looks fine, with nitpicks 1. please move the new support function to the bottom of the section dedicated to LDAP, and include a prototype 2. please wrap lines

  1   2   3   4   5   6   7   8   9   10   >