Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-12 Thread Atri Sharma
On Wed, Nov 12, 2014 at 1:24 PM, David Rowley dgrowle...@gmail.com wrote: On Tue, Nov 11, 2014 at 9:29 PM, Simon Riggs si...@2ndquadrant.com wrote: This plan type is widely used in reporting queries, so will hit the mainline of BI applications and many Mat View creations. This will allow

[HACKERS] Unintended restart after recovery error

2014-11-12 Thread Antonin Houska
While looking at postmaster.c:reaper(), one problematic case occurred to me. 1. Startup process signals PMSIGNAL_RECOVERY_STARTED. 2. Checkpointer process is forked and immediately dies. 3. reaper() catches this failure, calls HandleChildCrash() and thus sets FatalError to true. 4. Startup

Re: [HACKERS] Unintended restart after recovery error

2014-11-12 Thread Antonin Houska
Antonin Houska a...@cybertec.at wrote: While looking at postmaster.c:reaper(), one problematic case occurred to me. 1. Startup process signals PMSIGNAL_RECOVERY_STARTED. 2. Checkpointer process is forked and immediately dies. 3. reaper() catches this failure, calls HandleChildCrash()

Re: [HACKERS] inherit support for foreign tables

2014-11-12 Thread Ashutosh Bapat
Hi Fujita-san, I reviewed fdw-chk-3 patch. Here are my comments Sanity 1. The patch applies on the latest master using patch but not by git apply 2. it compiles clean 3. Regression run is clean, including the contrib module regressions Tests --- 1. The tests added in file_fdw module

[HACKERS] how to determine clause for new keyword, how to add new clause in gram.y

2014-11-12 Thread Pankaj Bagul
Please tell me, if want to add 'selectivity' keyword, which clause will it categorize ? Can you please tell me in detail, how should i modify gram.y, as I am new to parser writing ? and use it thereafter. Thank you. -- Pankaj A. Bagul

Re: PENDING_LIST_CLEANUP_SIZE - maximum size of GIN pending list Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-11-12 Thread Fujii Masao
On Wed, Nov 12, 2014 at 12:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Fujii Masao masao.fu...@gmail.com writes: On Tue, Nov 11, 2014 at 10:14 PM, Robert Haas robertmh...@gmail.com wrote: Not to kibitz too much after-the-fact, but wouldn't it be better to give this a name that has GIN in it

[HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Heikki Linnakangas
There's a race condition between a backend running queries in hot standby mode, and restoring a full-page image from a WAL record. It's present in all supported versions. RestoreBackupBlockContents does this: buffer = XLogReadBufferExtended(bkpb.node, bkpb.fork, bkpb.block,

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-12 Thread Robert Haas
On Tue, Nov 11, 2014 at 7:48 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Isn't provolatile = PROVOLATILE_IMMUTABLE sufficient? There are certainly things that are parallel-safe that are not immutable. It might be the case that everything immutable is parallel-safe. -- Robert Haas

Re: [HACKERS] tracking commit timestamps

2014-11-12 Thread Petr Jelinek
On 10/11/14 14:53, Robert Haas wrote: On Mon, Nov 10, 2014 at 8:39 AM, Petr Jelinek p...@2ndquadrant.com wrote: I did the calculation above wrong btw, it's actually 20 bytes not 24 bytes per record, I am inclined to just say we can live with that. If you do it as 20 bytes, you'll have to do

Re: [HACKERS] Failback to old master

2014-11-12 Thread Ants Aasma
On Tue, Nov 11, 2014 at 11:52 PM, Maeldron T. maeld...@gmail.com wrote: As far as I remember (I can’t test it right now but I am 99% sure) promoting the slave makes it impossible to connect the old master to the new one without making a base_backup. The reason is the timeline change. It

[HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-12 Thread Peter Eisentraut
Could someone translate this detail message to English: ereport(LOG, (errmsg(logical decoding found consistent point at %X/%X, (uint32) (lsn 32), (uint32) lsn), errdetail(running xacts with xcnt == 0))); (or downgrade to debug

Re: [HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-12 Thread Andres Freund
Hi, On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: Could someone translate this detail message to English: ereport(LOG, (errmsg(logical decoding found consistent point at %X/%X, (uint32) (lsn 32), (uint32) lsn),

Re: [HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-12 Thread Alvaro Herrera
Andres Freund wrote: Hi, On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: Could someone translate this detail message to English: ereport(LOG, (errmsg(logical decoding found consistent point at %X/%X, (uint32) (lsn 32), (uint32)

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: There's a race condition between a backend running queries in hot standby mode, and restoring a full-page image from a WAL record. It's present in all supported versions. I can think of two ways to fix this: 1. Have ReadBufferExtended

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 7:39 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 2. When ReadBufferExtended doesn't find the page in cache, it returns the buffer in !BM_VALID state (i.e. still in I/O in-progress state). Require the caller to call a second function, after locking the page, to

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-11-12 Thread Robert Haas
On Tue, Nov 11, 2014 at 4:27 AM, Andres Freund and...@2ndquadrant.com wrote: The more important thing here is that I see little chance of this getting in before Heikki's larger rework of the wal format gets in. Since that'll change everything around anyay I'm unsure how much point there is to

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Heikki Linnakangas
On 11/12/2014 04:56 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: There's a race condition between a backend running queries in hot standby mode, and restoring a full-page image from a WAL record. It's present in all supported versions. I can think of two ways to fix

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-11-12 Thread Andres Freund
On 2014-11-12 10:13:18 -0500, Robert Haas wrote: On Tue, Nov 11, 2014 at 4:27 AM, Andres Freund and...@2ndquadrant.com wrote: The more important thing here is that I see little chance of this getting in before Heikki's larger rework of the wal format gets in. Since that'll change everything

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 11/12/2014 04:56 PM, Tom Lane wrote: Not great either. What about an RBM_NOERROR mode that is like RBM_ZERO in terms of handling error conditions, but does not forcibly zero the page if it's already valid? Anyway, you don't want to read

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Heikki Linnakangas
On 11/12/2014 05:08 PM, Robert Haas wrote: On Wed, Nov 12, 2014 at 7:39 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 2. When ReadBufferExtended doesn't find the page in cache, it returns the buffer in !BM_VALID state (i.e. still in I/O in-progress state). Require the caller to call a

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Heikki Linnakangas
On 11/12/2014 05:20 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 11/12/2014 04:56 PM, Tom Lane wrote: Not great either. What about an RBM_NOERROR mode that is like RBM_ZERO in terms of handling error conditions, but does not forcibly zero the page if it's already

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 11/12/2014 05:20 PM, Tom Lane wrote: On reconsideration I think the RBM_ZERO returns page already locked alternative may be the less ugly. That has the advantage that any code that doesn't get updated will fail clearly and reliably.

Re: [HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-12 Thread Andres Freund
On 2014-11-12 11:56:01 -0300, Alvaro Herrera wrote: Andres Freund wrote: Hi, On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: Could someone translate this detail message to English: ereport(LOG, (errmsg(logical decoding found consistent point at

Re: [HACKERS] Unintended restart after recovery error

2014-11-12 Thread Fujii Masao
On Wed, Nov 12, 2014 at 6:52 PM, Antonin Houska a...@cybertec.at wrote: While looking at postmaster.c:reaper(), one problematic case occurred to me. 1. Startup process signals PMSIGNAL_RECOVERY_STARTED. 2. Checkpointer process is forked and immediately dies. 3. reaper() catches this

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Robert Haas
On Mon, Nov 10, 2014 at 4:58 PM, Andres Freund and...@2ndquadrant.com wrote: No, that's not what I was thinking of. Consider: export pg_libdir=$(pg_config --libdir) mkdir $pg_libdir/plugins ln -s $pg_libdir/auto_explain.so $pg_libdir/plugins/ PG_OPTIONS='-c

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Andres Freund
On 2014-11-12 11:15:26 -0500, Robert Haas wrote: On Mon, Nov 10, 2014 at 4:58 PM, Andres Freund and...@2ndquadrant.com wrote: No, that's not what I was thinking of. Consider: export pg_libdir=$(pg_config --libdir) mkdir $pg_libdir/plugins ln -s $pg_libdir/auto_explain.so

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 11:19 AM, Andres Freund and...@2ndquadrant.com wrote: The question is whether the library is actually loaded in that case? Because that normally only happens early during startup - which is why it's a PGC_BACKEND guc. It looks like that does not work. [rhaas pgsql]$

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-11-12 Thread Fujii Masao
On Mon, Nov 10, 2014 at 7:19 PM, furu...@pm.nttdata.co.jp wrote: On Fri, Oct 31, 2014 at 5:46 PM, furu...@pm.nttdata.co.jp wrote: We seem to be going in circles. You suggested having two options, --feedback, and --fsync, which is almost exactly what Furuya posted originally. I objected

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-12 Thread Josh Berkus
On 11/07/2014 02:03 PM, Josh Berkus wrote: But, like I said, there's a serviceable workaround. Some update on this. We've seen a problem in production with this setup which I can't reproduce as a test case, but which may jog Heikki's memory for something to fix. 1. Recover master to 2014-11-10

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Andres Freund
On 2014-11-12 11:36:14 -0500, Robert Haas wrote: On Wed, Nov 12, 2014 at 11:19 AM, Andres Freund and...@2ndquadrant.com wrote: The question is whether the library is actually loaded in that case? Because that normally only happens early during startup - which is why it's a PGC_BACKEND

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-12 Thread Jim Nasby
On 11/11/14, 2:01 AM, Andres Freund wrote: On 2014-11-10 19:36:18 -0600, Jim Nasby wrote: On 11/10/14, 12:56 PM, Andres Freund wrote: On 2014-11-10 12:37:29 -0600, Jim Nasby wrote: On 11/10/14, 12:15 PM, Andres Freund wrote: If what we want is to quantify the extent of the issue, would it be

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-12 Thread Andres Freund
On 2014-11-12 11:34:04 -0600, Jim Nasby wrote: On 11/11/14, 2:01 AM, Andres Freund wrote: On 2014-11-10 19:36:18 -0600, Jim Nasby wrote: Towards that simple end, I'm a bit torn. My preference would be to simply log, and throw a warning if it's over some threshold. I believe that would give

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 11:36 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Nov 12, 2014 at 11:19 AM, Andres Freund and...@2ndquadrant.com wrote: The question is whether the library is actually loaded in that case? Because that normally only happens early during startup - which is why

Re: [HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 10:59 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-12 11:56:01 -0300, Alvaro Herrera wrote: Andres Freund wrote: Hi, On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: Could someone translate this detail message to English:

Re: [HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-12 Thread Andres Freund
On 2014-11-12 12:40:41 -0500, Robert Haas wrote: On Wed, Nov 12, 2014 at 10:59 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-12 11:56:01 -0300, Alvaro Herrera wrote: Andres Freund wrote: Hi, On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: Could someone

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 12:37 PM, Andres Freund and...@2ndquadrant.com wrote: Stop overdesigning this. Add it to the existing mesage and let us be done with this. This thread has already wasted far too much time. That's a little harsh, but I agree. Producing a warning here is just going to

Re: [HACKERS] alter user set local_preload_libraries.

2014-11-12 Thread Fujii Masao
On Wed, Nov 5, 2014 at 1:22 AM, Peter Eisentraut pete...@gmx.net wrote: On 10/9/14 1:58 PM, Fujii Masao wrote: Also I think that it's useful to allow ALTER ROLE/DATABASE SET to set PGC_BACKEND and PGC_SU_BACKEND parameters. So, what about applying the attached patch? This patch allows that and

Re: [HACKERS] what does this mean: running xacts with xcnt == 0

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 12:47 PM, Andres Freund and...@2ndquadrant.com wrote: So maybe 'Encountered xl_running_xacts record with xcnt = 0.'? That's not very user-facing, is it -- I mean, why bother the user with the names of structs and members thereof? It seems better to describe what

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 12:12 PM, Josh Berkus j...@agliodbs.com wrote: On 11/07/2014 02:03 PM, Josh Berkus wrote: But, like I said, there's a serviceable workaround. Some update on this. We've seen a problem in production with this setup which I can't reproduce as a test case, but which may

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-11-12 Thread Alvaro Herrera
Fujii Masao wrote: --- 127,152 When this option is used, applicationpg_receivexlog/ will report a flush position to the server, indicating when each segment has been synchronized to disk so that the server can remove that segment if it !

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 2:57 AM, Jeff Davis pg...@j-davis.com wrote: Trying to catch up on this thread, please excuse me if these questions are already covered. Welcome to the party. The more, the merrier. :-) You mention the possibility of undetected deadlocks, which is surely

[HACKERS] Reverse Engineering - search constraints are not explicitly stated in the tables from the VIEW

2014-11-12 Thread nill
I am analyzing query plans generated by the view in the database PostgreSQL 8.3, looking for missing information constraints not explicitly registrants in the tables. In nested queries, (ex. IN clause, ...), the query plan consist in the evaluation of the subplane derived from clause (SELECT *

Re: [HACKERS] pg_prewarm really needs some CHECK_FOR_INTERRUPTS

2014-11-12 Thread Andres Freund
On 2014-11-11 12:17:11 +0100, Andres Freund wrote: pg_prewarm() currently can't be cannot be interrupted - which seems odd given that it's intended to read large amounts of data from disk. A rather slow process. Unless somebody protests I'm going to add a check to the top of each of the

[HACKERS] Error building the EnterpriseDB mysql_fdw on OSX

2014-11-12 Thread Kirk Roybal
I'm running 10.9.5 of OSX. I got the MySQL and PostgreSQL dependencies installed (I think). Checked out the git repo for mysql_fdw from git://github.com/EnterpriseDB/mysql_fdw USE_PGXS=1 make and got the error: mysql_fdw.c mysql_fdw.c:153:56: error: use of undeclared identifier

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-12 Thread Robert Haas
On Sun, Nov 9, 2014 at 11:48 PM, Peter Geoghegan p...@heroku.com wrote: On Fri, Nov 7, 2014 at 12:57 PM, Robert Haas robertmh...@gmail.com wrote: Based on this review from a month ago, I'm going to mark this Waiting on Author. If nobody updates the patch in a few days, I'll mark it Returned

Re: [HACKERS] Error building the EnterpriseDB mysql_fdw on OSX

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 3:48 PM, Kirk Roybal k...@webfinish.com wrote: I'm running 10.9.5 of OSX. I got the MySQL and PostgreSQL dependencies installed (I think). Checked out the git repo for mysql_fdw from git://github.com/EnterpriseDB/mysql_fdw USE_PGXS=1 make and got the error:

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Robert Haas
On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut pete...@gmx.net wrote: If REINDEX cannot work without an exclusive lock, we should invent some other qualifier, like WITH FEWER LOCKS. What he said. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 4:10 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut pete...@gmx.net wrote: If REINDEX cannot work without an exclusive lock, we should invent some other qualifier, like WITH FEWER LOCKS. What he said. But more to the

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-12 Thread Peter Geoghegan
On Wed, Nov 12, 2014 at 12:59 PM, Robert Haas robertmh...@gmail.com wrote: I agree with your proposed approach to moving Levenshtein into core. However, I think this should be separated into two patches, one of them moving the Levenshtein functionality into core, and the other adding the new

Re: [HACKERS] Add CREATE support to event triggers

2014-11-12 Thread Robert Haas
On Mon, Nov 10, 2014 at 9:02 PM, Jim Nasby jim.na...@bluetreble.com wrote: +1. Adding columns is a PITA, you have to manually ensure you do it on all slaves first. Drop is somewhat worse, because you have to do it on the master first, opposite of the (more usual) case of adding a column.

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Andres Freund
On 2014-11-12 16:11:58 -0500, Robert Haas wrote: On Wed, Nov 12, 2014 at 4:10 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut pete...@gmx.net wrote: If REINDEX cannot work without an exclusive lock, we should invent some other qualifier, like

Re: [HACKERS] Reverse Engineering - search constraints are not explicitly stated in the tables from the VIEW

2014-11-12 Thread Tom Lane
nill nill...@hotmail.com writes: I am analyzing query plans generated by the view in the database PostgreSQL 8.3, looking for missing information constraints not explicitly registrants in the tables. You realize of course that 8.3 is nearly 7 years old and has been out of support for awhile.

Re: [HACKERS] On partitioning

2014-11-12 Thread Robert Haas
On Mon, Nov 10, 2014 at 8:53 PM, Amit Langote langote_amit...@lab.ntt.co.jp wrote: Above commands are merely transformed into ALTER TABLE subcommands that arrange partitioned table and partitions into inheritance hierarchy, but with extra information, that is, allowed values for the partition

Re: [HACKERS] Unintended restart after recovery error

2014-11-12 Thread Antonin Houska
Fujii Masao masao.fu...@gmail.com wrote: On Wed, Nov 12, 2014 at 6:52 PM, Antonin Houska a...@cybertec.at wrote: While looking at postmaster.c:reaper(), one problematic case occurred to me. 1. Startup process signals PMSIGNAL_RECOVERY_STARTED. 2. Checkpointer process is forked and

Re: [HACKERS] Add CREATE support to event triggers

2014-11-12 Thread Andres Freund
On 2014-11-12 16:36:30 -0500, Robert Haas wrote: On Mon, Nov 10, 2014 at 9:02 PM, Jim Nasby jim.na...@bluetreble.com wrote: +1. Adding columns is a PITA, you have to manually ensure you do it on all slaves first. Drop is somewhat worse, because you have to do it on the master first,

Re: [HACKERS] On partitioning

2014-11-12 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I thought putting the partition boundaries into pg_inherits was a strange choice. I'd put it in pg_class, or in pg_partition if we decide to create that. Yeah. I rather doubt that we want this mechanism to be very closely tied to the existing

Re: [HACKERS] tracking commit timestamps

2014-11-12 Thread Jim Nasby
On 11/12/14, 7:06 AM, Petr Jelinek wrote: - if the xid passed to get interface is out of range -infinity timestamp is returned (I think it's bad idea to throw errors here as the valid range is not static and same ID can start throwing errors between calls theoretically) Wouldn't NULL be

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Jim Nasby
On 11/12/14, 9:47 AM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 11/12/2014 05:20 PM, Tom Lane wrote: On reconsideration I think the RBM_ZERO returns page already locked alternative may be the less ugly. That has the advantage that any code that doesn't get updated

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 4:39 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-12 16:11:58 -0500, Robert Haas wrote: On Wed, Nov 12, 2014 at 4:10 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut pete...@gmx.net wrote: If REINDEX cannot

Re: [HACKERS] On partitioning

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 5:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I thought putting the partition boundaries into pg_inherits was a strange choice. I'd put it in pg_class, or in pg_partition if we decide to create that. Yeah. I rather doubt that

Re: [HACKERS] Unintended restart after recovery error

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 4:52 PM, Antonin Houska a...@cybertec.at wrote: Fujii Masao masao.fu...@gmail.com wrote: On Wed, Nov 12, 2014 at 6:52 PM, Antonin Houska a...@cybertec.at wrote: While looking at postmaster.c:reaper(), one problematic case occurred to me. 1. Startup process

Re: [HACKERS] On partitioning

2014-11-12 Thread Jim Nasby
On 11/12/14, 5:27 PM, Robert Haas wrote: Maybe as anyarray, but I think pg_node_tree might even be better. That can also represent data of some arbitrary type, but it doesn't enforce that everything is uniform. Of course, the more general you make it, the more likely that it'll be impossible

Re: [HACKERS] pg_multixact not getting truncated

2014-11-12 Thread Jim Nasby
On 11/10/14, 12:16 AM, Josh Berkus wrote: On 11/09/2014 08:00 PM, Josh Berkus wrote: On 11/08/2014 01:46 PM, Andres Freund wrote: I'm these days suggesting that people should add manual vacuuming for older relations during off peak hours on busy databases. There's too many sites which service

Re: [HACKERS] Teaching pg_dump to use NOT VALID constraints

2014-11-12 Thread Jim Nasby
On 11/10/14, 12:00 PM, Simon Riggs wrote: On 10 November 2014 17:33, Alvaro Herrera alvhe...@2ndquadrant.com wrote: pg_dump --no-revalidaton will add NOT VALID onto the recreation SQL for any FKs, but only for ones that were already known to be valid. Well. Constraints that haven't been

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-12 Thread Michael Paquier
On Wed, Nov 12, 2014 at 9:49 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 11, 2014 at 7:48 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Isn't provolatile = PROVOLATILE_IMMUTABLE sufficient? There are certainly things that are parallel-safe that are not immutable. It might be the

[HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-12 Thread Tom Lane
I noticed that the recent custom-path commit completely ignored my advice about not including executor headers into planner headers or vice versa. On the way to fixing that, I was dismayed to discover that the RLS patch has utterly bollixed all semblance of modularization of the headers.

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Andres Freund
On 2014-11-12 18:23:38 -0500, Robert Haas wrote: On Wed, Nov 12, 2014 at 4:39 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-12 16:11:58 -0500, Robert Haas wrote: On Wed, Nov 12, 2014 at 4:10 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Nov 6, 2014 at 9:50 AM, Peter

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-12 Thread Peter Geoghegan
On Wed, Nov 12, 2014 at 1:13 PM, Peter Geoghegan p...@heroku.com wrote: On Wed, Nov 12, 2014 at 12:59 PM, Robert Haas robertmh...@gmail.com wrote: I agree with your proposed approach to moving Levenshtein into core. However, I think this should be separated into two patches, one of them moving

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-12 Thread Peter Geoghegan
On Wed, Nov 12, 2014 at 4:54 PM, Peter Geoghegan p...@heroku.com wrote: Attached patch moves the Levenshtein distance implementation into core. Oops. Somehow managed to send a *.patch.swp file. :-) Here is the actual patch. -- Peter Geoghegan From b7df918f1a52107637600f3b22d1cff18bd07ae1 Mon

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-12 Thread Andreas Karlsson
Hi, Here is version 2 of the patch which detects the presence of gcc/clang style 128-bit integers and has been cleaned up to a reviewable state. I have not added support for any other compilers since I found no documentation 128-bit support with icc or MSVC. I do not have access to any

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Alvaro Herrera
Andres Freund wrote: On 2014-11-12 18:23:38 -0500, Robert Haas wrote: The problem is that it's very hard to avoid the wrong index's relfilenode being used when swapping the relfilenodes between two indexes. How about storing both the old and new relfilenodes in the same pg_class

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Michael Paquier
On Thu, Nov 13, 2014 at 9:31 AM, Andres Freund and...@2ndquadrant.com wrote: I don't recall what the problem with just swapping the names was - but I'm pretty sure there was one... Hm. The index relation oids are referred to by constraints and dependencies. That's somewhat solvable. But I

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Michael Paquier
On Thu, Nov 13, 2014 at 10:26 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Nov 13, 2014 at 9:31 AM, Andres Freund and...@2ndquadrant.com wrote: I don't recall what the problem with just swapping the names was - but I'm pretty sure there was one... Hm. The index relation oids are

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-12 Thread Andreas Karlsson
On 11/13/2014 02:03 AM, Andreas Karlsson wrote: Here is version 2 of the patch which detects the presence of gcc/clang style 128-bit integers and has been cleaned up to a reviewable state. I have not added support for any other compilers since I found no documentation 128-bit support with icc or

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-12 Thread Alvaro Herrera
Tom Lane wrote: I noticed that the recent custom-path commit completely ignored my advice about not including executor headers into planner headers or vice versa. On the way to fixing that, I was dismayed to discover that the RLS patch has utterly bollixed all semblance of modularization of

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-12 Thread Alvaro Herrera
Andreas Karlsson wrote: On 11/13/2014 02:03 AM, Andreas Karlsson wrote: Here is version 2 of the patch which detects the presence of gcc/clang style 128-bit integers and has been cleaned up to a reviewable state. I have not added support for any other compilers since I found no documentation

Re: [HACKERS] pg_multixact not getting truncated

2014-11-12 Thread Alvaro Herrera
Jim Nasby wrote: On 11/10/14, 12:16 AM, Josh Berkus wrote: On 11/09/2014 08:00 PM, Josh Berkus wrote: On 11/08/2014 01:46 PM, Andres Freund wrote: I'm these days suggesting that people should add manual vacuuming for older relations during off peak hours on busy databases. There's too many

Re: PENDING_LIST_CLEANUP_SIZE - maximum size of GIN pending list Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-11-12 Thread Fujii Masao
On Wed, Nov 12, 2014 at 9:30 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Nov 12, 2014 at 12:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Fujii Masao masao.fu...@gmail.com writes: On Tue, Nov 11, 2014 at 10:14 PM, Robert Haas robertmh...@gmail.com wrote: Not to kibitz too much

Re: [HACKERS] tracking commit timestamps

2014-11-12 Thread Michael Paquier
On Thu, Nov 13, 2014 at 7:56 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 11/12/14, 7:06 AM, Petr Jelinek wrote: - if the xid passed to get interface is out of range -infinity timestamp is returned (I think it's bad idea to throw errors here as the valid range is not static and same ID

Re: [HACKERS] Race in tablespace test on Windows

2014-11-12 Thread Amit Kapila
On Thu, Nov 13, 2014 at 8:46 AM, Noah Misch n...@leadboat.com wrote: On Tue, Nov 11, 2014 at 10:21:26AM +0530, Amit Kapila wrote: On Sat, Nov 8, 2014 at 10:34 AM, Noah Misch n...@leadboat.com wrote: Here is a briefer command sequence exhibiting the same problem: To make this work as

Re: [HACKERS] tracking commit timestamps

2014-11-12 Thread Michael Paquier
On Wed, Nov 12, 2014 at 10:06 PM, Petr Jelinek p...@2ndquadrant.com wrote: Brief list of changes: - the commit timestamp record now stores timestamp, lsn and nodeid Now that not only the commit timestamp is stored, calling that commit timestamp, committs or commit_timestamp is strange, no? If

Re: [HACKERS] inherit support for foreign tables

2014-11-12 Thread Ashutosh Bapat
Hi Fujita-san, I tried to apply fdw-inh-3.patch on the latest head from master branch. It failed to apply using both patch and git apply. patch failed to apply because of rejections in contrib/file_fdw/output/file_fdw.source and doc/src/sgml/ref/create_foreign_table.sgml On Fri, Nov 7, 2014 at

Re: [HACKERS] On partitioning

2014-11-12 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Wed, Nov 12, 2014 at 5:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Maybe as anyarray, but I think pg_node_tree might even be better. That can also represent data of some arbitrary type, but it doesn't

Re: [HACKERS] inherit support for foreign tables

2014-11-12 Thread Etsuro Fujita
Hi Ashutosh, Thanks for the review! (2014/11/13 15:23), Ashutosh Bapat wrote: I tried to apply fdw-inh-3.patch on the latest head from master branch. It failed to apply using both patch and git apply. patch failed to apply because of rejections in contrib/file_fdw/output/file_fdw.source and

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-12 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Tom Lane wrote: I noticed that the recent custom-path commit completely ignored my advice about not including executor headers into planner headers or vice versa. On the way to fixing that, I was dismayed to discover that the RLS patch

Re: [HACKERS] inherit support for foreign tables

2014-11-12 Thread Ashutosh Bapat
On Thu, Nov 13, 2014 at 12:20 PM, Etsuro Fujita fujita.ets...@lab.ntt.co.jp wrote: Hi Ashutosh, Thanks for the review! (2014/11/13 15:23), Ashutosh Bapat wrote: I tried to apply fdw-inh-3.patch on the latest head from master branch. It failed to apply using both patch and git apply.

Re: [HACKERS] On partitioning

2014-11-12 Thread Amit Langote
From: Stephen Frost [mailto:sfr...@snowman.net] Sent: Thursday, November 13, 2014 3:40 PM The point for me is just that range and list partitioning probably need different structure, and hash partitioning, if we want to support that, needs something else again. Range partitioning needs

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-12 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: src/include/rewrite/rowsecurity.h, which one would reasonably think to be a rewriter header (nevermind its header comment to the contrary), nonetheless includes execnodes.h (executor stuff) I'll fix the header comment. The include of execnodes.h was a

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-12 Thread Jeff Davis
On Wed, 2014-11-12 at 14:16 -0500, Robert Haas wrote: Detected deadlocks are fine. Improving the deadlock detector is the heart of what needs to be done here. OK, great. As you say, the lock requests we're talking about will rarely wait, so deadlocks won't be frequent. The issue is making

Re: [HACKERS] On partitioning

2014-11-12 Thread Amit Langote
ow...@postgresql.org] On Behalf Of Amit Langote Sent: Thursday, November 13, 2014 3:50 PM Greenplum uses a single table for this purpose with separate columns for range and list cases, for example. They store allowed values per partition though. They have 6 partitioning related

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-11-12 Thread Amit Kapila
On Mon, Oct 27, 2014 at 5:26 PM, Amit Kapila amit.kapil...@gmail.com wrote: Going further with verification of this patch, I found below issue: Run the testcase.sql file at below link: http://www.postgresql.org/message-id/4205e661176a124faf891e0a6ba9135266347...@szxeml509-mbs.china.huawei.com

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-12 Thread Josh Berkus
On 11/12/2014 10:06 AM, Robert Haas wrote: hat *appears* to be happening is that the pause_at_recovery_target, followed by the restart, on the replica causes it to advance one commit on timeline 1. But *not all the time*; this doesn't happen in my pgbench-based tests. There's a