Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-02-26 Thread pokurev
Hello, Thank you for your comments. Please find attached patch addressing following comments. >As I might have written upthread, transferring the whole string >as a progress message is useless at least in this scenario. Since >they are a set of fixed messages, each of them can be represented

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Konstantin Knizhnik
On 02/26/2016 09:30 PM, Alvaro Herrera wrote: Konstantin Knizhnik wrote: Yes, it is certainly possible to develop cluster by cloning PostgreSQL. But it cause big problems both for developers, which have to permanently synchronize their branch with master, and, what is more important, for

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Bruce Momjian
On Fri, Feb 26, 2016 at 03:30:29PM -0300, Alvaro Herrera wrote: > That's not the point, though. I don't think a Postgres clone with a GTM > solves any particular problem that's not already solved by the existing > forks. However, if you have a clone at home and you make a GTM work on > it, then

Re: [HACKERS] [COMMITTERS] pgsql: Add a test framework for recovery

2016-02-26 Thread Alvaro Herrera
Alvaro Herrera wrote: > Add a test framework for recovery > > This long-awaited framework is an expansion of the existing PostgresNode > stuff to support additional features for recovery testing; the recovery > tests included in this commit are a starting point that cover some of > the recovery

Re: [HACKERS] [PATCH] fix DROP OPERATOR to reset links to itself on commutator and negator

2016-02-26 Thread Roma Sokolov
Thanks for comments, Euler! > ... is hard to understand. Instead, you could separate the conditional > expression into a variable. Fixed the patch to be more descriptive and to avoid repeating same computation over and over again. See v2 of the patch attached. > I don't think those are

Re: [HACKERS] pg_filedump patch for 9.5

2016-02-26 Thread Pavel Raiskup
On Saturday 08 of August 2015 20:38:38 Satoshi Nagayasu wrote: > I have created a patch for pg_filedump to work with 9.5. > Here is a list of changes. > > * Fix to rename CRC32 macros to work with 9.5. > * Fix to add missing DBState: DB_SHUTDOWNED_IN_RECOVERY. > * Fix to add missing page flags

Re: [HACKERS] Prepared Statement support for Parallel query

2016-02-26 Thread Robert Haas
On Thu, Feb 25, 2016 at 1:09 PM, Robert Haas wrote: > On Thu, Feb 25, 2016 at 8:53 AM, Amit Kapila wrote: >>> But if the user says >>> they want to PREPARE the query, they are probably not going to fetch >>> all rows. >> >> After PREPARE, user

Re: [HACKERS] Prepared Statement support for Parallel query

2016-02-26 Thread Amit Kapila
On Fri, Feb 26, 2016 at 4:37 PM, Robert Haas wrote: > > On Thu, Feb 25, 2016 at 1:09 PM, Robert Haas wrote: > > On Thu, Feb 25, 2016 at 8:53 AM, Amit Kapila wrote: > >>> But if the user says > >>> they want to PREPARE the

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-02-26 Thread Amit Langote
Hi Vinayak, Thanks for updating the patch! A quick comment: On 2016/02/26 17:28, poku...@pm.nttdata.co.jp wrote: >> CREATE VIEW pg_stat_vacuum_progress AS >> SELECT S.s[1] as pid, >> S.s[2] as relid, >> CASE S.s[3] >>WHEN 1 THEN 'Scanning Heap' >>

Re: [HACKERS] get current log file

2016-02-26 Thread Robert Haas
On Fri, Feb 26, 2016 at 8:31 AM, Armor wrote: > I think I know what you are concerned about. May be I did not explain my > solution very clearly. > (i) Using a variable named last_syslogger_file_time replace > first_syslogger_file_time in syslogger.c. When postmaster

Re: [HACKERS] POC: Cache data in GetSnapshotData()

2016-02-26 Thread Robert Haas
On Thu, Feb 25, 2016 at 12:57 PM, Mithun Cy wrote: > I have fixed all of the issues reported by regress test. Also now when > backend try to cache the snapshot we also try to store the self-xid and sub > xid, so other backends can use them. > > I also did some

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

2016-02-26 Thread Robert Haas
On Wed, Feb 24, 2016 at 12:59 PM, Thomas Munro wrote: > On Wed, Feb 24, 2016 at 5:48 PM, Thomas Munro > wrote: >> Here is a first pass at that. [...] > > On Wed, Feb 24, 2016 at 1:23 AM, Robert Haas wrote: >>

Re: [HACKERS] Sanity checking for ./configure options?

2016-02-26 Thread Robert Haas
On Wed, Feb 24, 2016 at 4:01 AM, David Fetter wrote: > I'm thinking that both the GUC check and the configure one should > restrict it to [1024..65535]. Doesn't sound like a good idea to me. If somebody has a reason they want to do that, they shouldn't have to hack the source

Re: [HACKERS] get current log file

2016-02-26 Thread Euler Taveira
On 26-02-2016 08:03, Robert Haas wrote: > I don't think we're going to accept this feature if it might fail in > corner cases. And that design seems awfully complex. > Agree. > The obvious way to implement this, to me at least, seems to be for the > syslogger to write a file someplace in the

Re: [HACKERS] Sanity checking for ./configure options?

2016-02-26 Thread Tom Lane
David Fetter writes: > On Fri, Feb 26, 2016 at 04:55:23PM +0530, Robert Haas wrote: >> On Wed, Feb 24, 2016 at 4:01 AM, David Fetter wrote: >>> I'm thinking that both the GUC check and the configure one should >>> restrict it to [1024..65535]. >> Doesn't

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Konstantin Knizhnik
The reason of the problem is that invalidation messages are not delivered to replica after the end of concurrent create index. Invalidation messages are included in xlog as part of transaction commit record. Concurrent index create is split into three transaction, last of which is just

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-02-26 Thread Valery Popov
26.02.2016 01:10, Michael Paquier пишет: On Fri, Feb 26, 2016 at 1:38 AM, Valery Popov wrote: Hi, Michael 23.02.2016 10:17, Michael Paquier пишет: Attached is a set of patches implementing a couple of things that have been discussed, so let's roll in. Those 4

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Robert Haas
On Wed, Feb 24, 2016 at 3:05 PM, Oleg Bartunov wrote: > I already several times pointed, that we need XTM to be able to continue > development in different directions, since there is no clear winner. > Moreover, I think there is no fits-all solution and while I agree we need

Re: [HACKERS] Sanity checking for ./configure options?

2016-02-26 Thread David Fetter
On Fri, Feb 26, 2016 at 04:55:23PM +0530, Robert Haas wrote: > On Wed, Feb 24, 2016 at 4:01 AM, David Fetter wrote: > > I'm thinking that both the GUC check and the configure one should > > restrict it to [1024..65535]. > > Doesn't sound like a good idea to me. If somebody has

Re: [HACKERS] [PATH] Correct negative/zero year in to_date/to_timestamp

2016-02-26 Thread Shulgin, Oleksandr
On Fri, Feb 26, 2016 at 3:24 PM, Ivan Kartyshov wrote: > The following review has been posted through the commitfest application: > > make installcheck-world: tested, failed > Implements feature: tested, failed > Spec compliant: tested, failed >

Re: [HACKERS] get current log file

2016-02-26 Thread Tom Lane
Euler Taveira writes: > On 26-02-2016 08:03, Robert Haas wrote: >> But there's one thing I'm slightly baffled about: why would you >> actually need this? > The use case I have in mind is consume log file by using a tool like > logstash. In this case, logstash accepts

Re: [HACKERS] get current log file

2016-02-26 Thread Euler Taveira
On 26-02-2016 11:50, Tom Lane wrote: > This needs to be explained a lot more clearly than it has been so far, > else we are going to reject this proposed feature as being more code and > more overhead than is justified. Exactly why would you need a pointer to > the current log file, rather than

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-02-26 Thread Simon Riggs
On 25 February 2016 at 18:42, Amit Kapila wrote: > On Thu, Feb 25, 2016 at 11:38 PM, Simon Riggs > wrote: > >> On 24 February 2016 at 23:26, Amit Kapila >> wrote: >> >>> From past few weeks, we were facing some

Re: [HACKERS] [PATH] Correct negative/zero year in to_date/to_timestamp

2016-02-26 Thread Ivan Kartyshov
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: tested, failed Documentation:tested, failed Applied this patch, it works well, make what it expected

Re: [HACKERS] get current log file

2016-02-26 Thread Tom Lane
Euler Taveira writes: > Those are good concerns. Also, we already have emit_log_hook that could > grab server log messages. A small extension using the hook (there are > some out there) could be use with a log consuming tool. Hmmm ... emit_log_hook runs in the process

Re: [HACKERS] [PATCH] fix DROP OPERATOR to reset links to itself on commutator and negator

2016-02-26 Thread Euler Taveira
On 26-02-2016 17:51, Roma Sokolov wrote: > Fixed the patch to be more descriptive and to avoid repeating same > computation over and over again. See v2 of the patch attached. > Oh, much better. > Why do you think that? Should I remove them or maybe send as separate > patch? > Because it is not

Re: [HACKERS] [REVIEW] In-core regression tests for replication, cascading, archiving, PITR, etc.

2016-02-26 Thread Michael Paquier
On Sat, Feb 27, 2016 at 4:30 AM, Alvaro Herrera wrote: > Craig Ringer wrote: >> Should be committed ASAP IMO. > > Finally pushed it. Let's see how it does in the buildfarm. Now let's > get going and add more tests, I know there's no shortage of people with > test

Re: [HACKERS] [COMMITTERS] pgsql: Add a test framework for recovery

2016-02-26 Thread Michael Paquier
On Sat, Feb 27, 2016 at 4:59 AM, Alvaro Herrera wrote: > Alvaro Herrera wrote: >> Add a test framework for recovery >> >> This long-awaited framework is an expansion of the existing PostgresNode >> stuff to support additional features for recovery testing; the recovery

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Simon Riggs
On 27 February 2016 at 01:23, Andres Freund wrote: > On 2016-02-27 01:16:34 +, Simon Riggs wrote: > > If the above is true, then the proposed fix wouldn't work either. > > > > No point in sending a cache invalidation message on the standby if you > > haven't also written

Re: [HACKERS] Sanity checking for ./configure options?

2016-02-26 Thread Peter Eisentraut
On 2/22/16 6:24 PM, Jim Nasby wrote: > On 2/5/16 10:08 AM, David Fetter wrote: >> On Wed, Feb 03, 2016 at 06:02:57PM -0600, Jim Nasby wrote: >>> I just discovered that ./configure will happily accept >>> '--with-pgport=' (I >>> was actually doing =$PGPORT, and didn't realize $PGPORT was empty).

[HACKERS] syslog configurable line splitting behavior

2016-02-26 Thread Peter Eisentraut
Writing log messages to syslog caters to ancient syslog implementations in two ways: - sequence numbers - line splitting While these are arguably reasonable defaults, I would like a way to turn them off, because they get in the way of doing more interesting things with syslog (e.g., logging

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Robert Haas
On Sat, Feb 27, 2016 at 1:49 AM, Konstantin Knizhnik wrote: > pg_tsdtm is based on another approach: it is using system time as CSN and > doesn't require arbiter. In theory there is no limit for scalability. But > differences in system time and necessity to use more

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-02-26 Thread Andres Freund
On February 26, 2016 7:55:18 PM PST, Amit Kapila wrote: >On Sat, Feb 27, 2016 at 12:41 AM, Andres Freund >wrote: >> >> Hi, >> >> On 2016-02-25 12:56:39 +0530, Amit Kapila wrote: >> > From past few weeks, we were facing some performance degradation in

Re: [HACKERS] [PATH] Correct negative/zero year in to_date/to_timestamp

2016-02-26 Thread Vitaly Burovoy
On 2/26/16, Shulgin, Oleksandr wrote: > On Fri, Feb 26, 2016 at 3:24 PM, Ivan Kartyshov > > wrote: > >> The following review has been posted through the commitfest application: > >> make installcheck-world: tested, failed >> Implements

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Andres Freund
On 2016-02-27 01:45:57 +, Simon Riggs wrote: > Surely then the fix is to make heap_inplace_update() assign an xid? That > way any catalog change will always generate a commit record containing the > invalidation that goes with the change. No need to fix up the breakage > later. Well, we

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Robert Haas
On Fri, Feb 26, 2016 at 10:56 PM, Konstantin Knizhnik wrote: > We do not have formal prove that proposed XTM is "general enough" to handle > all possible transaction manager implementations. > But there are two general ways of dealing with isolation: snapshot based and

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-02-26 Thread Amit Kapila
On Sat, Feb 27, 2016 at 12:41 AM, Andres Freund wrote: > > Hi, > > On 2016-02-25 12:56:39 +0530, Amit Kapila wrote: > > From past few weeks, we were facing some performance degradation in the > > read-only performance bench marks in high-end machines. My colleague > > Mithun,

[HACKERS] Re: [COMMITTERS] pgsql: Respect TEMP_CONFIG when running contrib regression tests.

2016-02-26 Thread Robert Haas
On Sat, Feb 27, 2016 at 9:00 AM, Andrew Dunstan wrote: >> Sure. Saving three lines of Makefile duplication is hardly a >> world-shattering event, so I thought there might be some other >> purpose. But I'm not against saving three lines of duplication >> either, if it won't

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-02-26 Thread Andres Freund
Hi, I'm not really convinced by RegisterStandbyMsgPrefix() et al. There's not much documentation about what it actually is supposed to acomplish. Afaics you're basically forced to use shared_preload_libraries with it right now? Also, iterating through a linked list everytime something is logged

Re: [HACKERS] proposal: get oldest LSN - function

2016-02-26 Thread Andres Freund
On 2016-02-26 18:45:14 +0300, Kartyshov Ivan wrote: > Function pg_oldest_xlog_location gets us the oldest LSN (Log Sequence > Number) in xlog. > > It is useful additional tool for DBA (we can get replicationSlotMinLSN, so > why not in master), it can show us, if xlog replication or wal-sender is

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Kevin Grittner
On Fri, Feb 26, 2016 at 2:19 PM, Konstantin Knizhnik wrote: > pg_tsdtm is based on another approach: it is using system time > as CSN Which brings up an interesting point, if we want logical replication to be free of serialization anomalies for those using

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Andres Freund
On 2016-02-26 18:05:55 +0300, Konstantin Knizhnik wrote: > The reason of the problem is that invalidation messages are not delivered to > replica after the end of concurrent create index. > Invalidation messages are included in xlog as part of transaction commit > record. > Concurrent index create

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Simon Riggs
On 27 February 2016 at 00:33, Simon Riggs wrote: > On 27 February 2016 at 00:29, Andres Freund wrote: > >> On 2016-02-26 18:05:55 +0300, Konstantin Knizhnik wrote: >> > The reason of the problem is that invalidation messages are not >> delivered to >>

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Simon Riggs
On 26 February 2016 at 22:48, Kevin Grittner wrote: > On Fri, Feb 26, 2016 at 2:19 PM, Konstantin Knizhnik > wrote: > > > pg_tsdtm is based on another approach: it is using system time > > as CSN > > Which brings up an interesting point, if we want

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-02-26 Thread Andres Freund
On 2016-02-02 13:12:50 +0300, Alexander Korotkov wrote: > On Tue, Feb 2, 2016 at 12:43 AM, Andres Freund wrote: > > > On 2016-02-01 13:06:57 +0300, Alexander Korotkov wrote: > > > On Mon, Feb 1, 2016 at 11:34 AM, Alexander Korotkov < > > > a.korot...@postgrespro.ru> wrote: >

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Simon Riggs
On 27 February 2016 at 00:29, Andres Freund wrote: > On 2016-02-26 18:05:55 +0300, Konstantin Knizhnik wrote: > > The reason of the problem is that invalidation messages are not > delivered to > > replica after the end of concurrent create index. > > Invalidation messages are

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Andres Freund
On 2016-02-27 01:16:34 +, Simon Riggs wrote: > If the above is true, then the proposed fix wouldn't work either. > > No point in sending a cache invalidation message on the standby if you > haven't also written WAL, since the catalog re-read would just see the old > row. > >

Re: [HACKERS] proposal: get oldest LSN - function

2016-02-26 Thread Kartyshov Ivan
On 27.02.2016 03:07, Andres Freund wrote How does it help with any of that? Hi, thank you for fast answer. Maybe i wasn't too accurate in terms, because I newbie, but: We can get information about xlog, using big amout of support function (pg_current_xlog_location(),

Re: [HACKERS][PATCH] Supporting +-Infinity values by to_timestamp(float8)

2016-02-26 Thread Vitaly Burovoy
Added to the CF 2016-03: https://commitfest.postgresql.org/9/546/ -- Best regards, Vitaly Burovoy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] raw output from copy

2016-02-26 Thread Pavel Stehule
Hi 2015-08-06 10:37 GMT+02:00 Pavel Stehule : > Hi, > > Psql based implementation needs new infrastructure (more than few lines) > > Missing: > > * binary mode support > * parametrized query support, > > I am not against, but both points I proposed, and both was

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-02-26 Thread Amit Kapila
On Tue, Feb 23, 2016 at 7:06 PM, Robert Haas wrote: > On Sun, Feb 21, 2016 at 7:45 PM, Amit Kapila > wrote: > >> I mean, my basic feeling is that I would not accept a 2-3% regression in >>> the single client case to get a 10% speedup in the case

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Konstantin Knizhnik
On 02/27/2016 06:57 AM, Robert Haas wrote: On Sat, Feb 27, 2016 at 1:49 AM, Konstantin Knizhnik wrote: pg_tsdtm is based on another approach: it is using system time as CSN and doesn't require arbiter. In theory there is no limit for scalability. But differences in

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Konstantin Knizhnik
On 02/27/2016 04:16 AM, Simon Riggs wrote: On 27 February 2016 at 00:33, Simon Riggs > wrote: On 27 February 2016 at 00:29, Andres Freund > wrote: On 2016-02-26 18:05:55 +0300,

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-02-26 Thread Amit Kapila
On Sat, Feb 27, 2016 at 10:03 AM, Amit Kapila wrote: > > > Here, we can see that there is a gain of ~15% to ~38% at higher client > count. > > The attached document (perf_write_clogcontrollock_data_v6.ods) contains > data, mainly focussing on single client performance.

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-02-26 Thread Vinayak Pokale
Hello, On Fri, Feb 26, 2016 at 6:19 PM, Amit Langote wrote: > > Hi Vinayak, > > Thanks for updating the patch! A quick comment: > > On 2016/02/26 17:28, poku...@pm.nttdata.co.jp wrote: > >> CREATE VIEW pg_stat_vacuum_progress AS > >> SELECT S.s[1] as pid, > >>

[HACKERS][PATCH] Supporting +-Infinity values by to_timestamp(float8)

2016-02-26 Thread Vitaly Burovoy
Hello, Hackers! I worked on a patch[1] allows "EXTRACT(epoch FROM +-Inf::timestamp[tz])" to return "+-Inf::float8". There is an opposite function "to_timestamp(float8)" which now defined as: SELECT ('epoch'::timestamptz + $1 * '1 second'::interval) Since intervals do not support infinity values,

Re: [HACKERS] PATCH: index-only scans with partial indexes

2016-02-26 Thread Robert Haas
On Fri, Feb 26, 2016 at 6:16 PM, Michael Paquier wrote: > On Fri, Feb 26, 2016 at 4:18 PM, Kyotaro HORIGUCHI > wrote: >> I marked this as "ready for commiter" and tried to add me as the >> *second* author. But the CF app forces certain

Re: [HACKERS] Sanity checking for ./configure options?

2016-02-26 Thread Ivan Kartyshov
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: tested, failed Documentation:tested, failed Tested, I think it`s rather important to make cleanup work

[HACKERS] [PATCH] fix DROP OPERATOR to reset links to itself on commutator and negator

2016-02-26 Thread Roma Sokolov
Hello, hackers! As was mentioned in that message [1], and earlier in [2], DROPping OPERATOR with negator or commutator doesn't update respective fields (oprnegate and oprcom) on them. While this issue is probably not very frequent, this behaviour leaves database in inconsistent state and prevents

Re: [HACKERS] [REVIEW] In-core regression tests for replication, cascading, archiving, PITR, etc.

2016-02-26 Thread Alvaro Herrera
Victor Wagner wrote: > I'll second Stas' suggestion about psql_ok/psql_fail functions. > > 1. psql_ok instead of just psql would provide visual feedback for the > reader of code. One would see 'here condition is tested, here is > something ended with _ok/_fail'. > > It would be nice that seeing

Re: [HACKERS] get current log file

2016-02-26 Thread Armor
I think I know what you are concerned about. May be I did not explain my solution very clearly. (i) Using a variable named last_syslogger_file_time replace first_syslogger_file_time in syslogger.c. When postmaster initialize logger process, last_syslogger_file_time will be assign the time

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-02-26 Thread Robert Haas
On Fri, Feb 26, 2016 at 8:41 PM, Simon Riggs wrote: > Don't understand this. If a problem is caused by one of two things, first > you check one, then the other. I don't quite understand how you think that patch can be decomposed into multiple, independent changes. It was

[HACKERS] proposal: get oldest LSN - function

2016-02-26 Thread Kartyshov Ivan
Hello, I want to suggest a client-side little function, implemented in the attached patch. Function pg_oldest_xlog_location gets us the oldest LSN (Log Sequence Number) in xlog. It is useful additional tool for DBA (we can get replicationSlotMinLSN, so why not in master), it can show us, if

Re: [HACKERS] [REVIEW] In-core regression tests for replication, cascading, archiving, PITR, etc.

2016-02-26 Thread Alvaro Herrera
Michael Paquier wrote: > Attached are rebased patches, split into 3 parts doing the following: > - 0001, fix default configuration of MSVC builds ignoring TAP tests BTW you keep submitting this one and I keep ignoring it. I think you should start a separate thread for this one, so that some

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Robert Haas
On Fri, Feb 26, 2016 at 7:21 PM, Oleg Bartunov wrote: > Right now tm is hardcoded and it's doesn't matter "if other people might > need" at all. We at least provide developers ("other people") ability to > work on their implementations and the patch is safe and doesn't

Re: [HACKERS] FDW handling count(*) through AnalyzeForeignTable or other constant time push-down

2016-02-26 Thread Simon Riggs
On 25 February 2016 at 09:48, Gabe F. Rudy wrote: > Hey all, > > > > I’m building a FDW around a column-store backend (similar to CStore but > for genomic data!). > > > > I have tables in the billions of rows, and have a common query pattern of > asking for the table size

[HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Васильев Дмитрий
Session opened on replica doesn't see concurrently created indexes at this time on master. We have master and replica: 1. master: pgbench -i -s 10 2. replica: explain (analyze,verbose) select * from pgbench_accounts where abalance = 1; 3. master: ALTER INDEX pgbench_accounts_abalance_idx

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Alexander Korotkov
On Fri, Feb 26, 2016 at 3:41 PM, Васильев Дмитрий wrote: > Session opened on replica doesn't see concurrently created indexes at this > time on master. > As I get, on standby index is visible when you run SQL queries on catalog tables (that is what \d+ does), but

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Васильев Дмитрий
This is obviously a bug because without "concurrently" create index this do not reproduce. --- Dmitry Vasilyev Postgres Professional: http://www.postgrespro.com Russian Postgres Company 2016-02-26 16:36 GMT+03:00 Alexander Korotkov : > On Fri, Feb 26, 2016 at 3:41 PM,

Re: [HACKERS] PATCH: index-only scans with partial indexes

2016-02-26 Thread Michael Paquier
On Fri, Feb 26, 2016 at 4:18 PM, Kyotaro HORIGUCHI wrote: > I marked this as "ready for commiter" and tried to add me as the > *second* author. But the CF app forces certain msyterious order > for listed names. Is there any means to arrange the author names > in

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Oleg Bartunov
On Fri, Feb 26, 2016 at 3:50 PM, Robert Haas wrote: > On Wed, Feb 24, 2016 at 3:05 PM, Oleg Bartunov > wrote: > > I already several times pointed, that we need XTM to be able to continue > > development in different directions, since there is no clear

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Konstantin Knizhnik
We do not have formal prove that proposed XTM is "general enough" to handle all possible transaction manager implementations. But there are two general ways of dealing with isolation: snapshot based and CSN based. pg_dtm and pg_tsdtm prove that both of them can be implemented using XTM. If you

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Joshua D. Drake
On 02/26/2016 08:06 AM, Robert Haas wrote: On Fri, Feb 26, 2016 at 7:21 PM, Oleg Bartunov wrote: Right now tm is hardcoded and it's doesn't matter "if other people might need" at all. We at least provide developers ("other people") ability to work on their

Re: [HACKERS] FDW handling count(*) through AnalyzeForeignTable or other constant time push-down

2016-02-26 Thread Gabe F. Rudy
Ok, I get that. Really what I am *rooting* for is Aggregate (and Sort By) Push-Down to FDW plugins. I can already internalize conditional filters for most cases, and doing a count on the filtered results would be considerably faster in my FDW back-end before all the records and Datums have to

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Robert Haas
On Fri, Feb 26, 2016 at 10:00 PM, Joshua D. Drake wrote: > Robert, this is all a game. It is a game of who wins the intellectual prize > to whatever problem. Who gets the market or mind share and who gets to > pretend they win the Oscar for coolest design. JD, I don't

Re: [HACKERS] [PATCH] fix DROP OPERATOR to reset links to itself on commutator and negator

2016-02-26 Thread Euler Taveira
On 26-02-2016 12:46, Roma Sokolov wrote: > Regression tests are added to check DROP OPERATOR behaves as intended > (including > case with self-commutator and unlikely case with operator being both negator > and > commutator). > I don't think those are mandatory. > Should this patch be added to

Re: [HACKERS] [PATH] Correct negative/zero year in to_date/to_timestamp

2016-02-26 Thread Ivan Kartyshov
> Why does it say "tested, failed" for all points above there? ;-) Hi, I just used Web reviewer form on https://commitfest.postgresql.org to make review on patch, but form doesn't work properly unlike the patch.))

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-02-26 Thread Andres Freund
Hi, On 2016-02-25 12:56:39 +0530, Amit Kapila wrote: > From past few weeks, we were facing some performance degradation in the > read-only performance bench marks in high-end machines. My colleague > Mithun, has tried by reverting commit ac1d794 which seems to degrade the > performance in HEAD

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Alvaro Herrera
Konstantin Knizhnik wrote: > Yes, it is certainly possible to develop cluster by cloning PostgreSQL. > But it cause big problems both for developers, which have to permanently > synchronize their branch with master, > and, what is more important, for customers, which can not use standard >

Re: [HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc.

2016-02-26 Thread Alvaro Herrera
Kyotaro HORIGUCHI wrote: > So, I'd like to propose four (or five) changes to this harness. > > - prove_check to remove all in tmp_check > > - TestLib to preserve temporary directories/files if the current >test fails. > > - PostgresNode::get_new_node to create data directory with >

Re: [HACKERS] [REVIEW] In-core regression tests for replication, cascading, archiving, PITR, etc.

2016-02-26 Thread Alvaro Herrera
Craig Ringer wrote: > Should be committed ASAP IMO. Finally pushed it. Let's see how it does in the buildfarm. Now let's get going and add more tests, I know there's no shortage of people with test scripts waiting for this. Thanks, Michael, for the persistency, and thanks to all reviewers.