Re: [HACKERS] Horizontal scalability/sharding

On 2015-09-02 PM 04:07, Albe Laurenz wrote: > Amit Langote wrote: >> On 2015-09-02 PM 03:25, Amit Kapila wrote: >>> Will it handle deadlocks across different table partitions. Consider >>> a case as below: >>> >>> T1 >>> 1. Updates row R1 of T1 on shard S1 >>> 2. Updates row R2 of T2 on shard S2

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On Wed, Sep 2, 2015 at 6:21 AM, Michael Paquier wrote: > Yes, that's what I have been looking at actually by having some markers in > relcache.c. The reference count of this relation get incremented here: So, I have been playing more with this code, and as mentioned by Andres this test case goes

Re: [HACKERS] Horizontal scalability/sharding

Etsuro Fujita wrote: > On 2015/09/02 16:40, Amit Langote wrote: >> On 2015-09-02 PM 04:07, Albe Laurenz wrote: >>> Amit Langote wrote: On 2015-09-02 PM 03:25, Amit Kapila wrote: > Will it handle deadlocks across different table partitions. Consider > a case as below: > > T1

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Tue, Sep 1, 2015 at 8:12 PM, Andres Freund wrote: > On 2015-09-01 14:07:19 -0400, Robert Haas wrote: > > But I think it's quite wrong to assume that the infrastructure for > > this is available and usable everywhere, because in my experience, > > that's far from the case.

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule wrote: > >> But do we really need the slots mechanism? Would it not be OK to just >> let the LWLock do the sequencing of concurrent requests? Given that we >> only going to use one message queue per cluster, there's not

Re: [HACKERS] Horizontal scalability/sharding

On 2015/09/02 16:40, Amit Langote wrote: On 2015-09-02 PM 04:07, Albe Laurenz wrote: Amit Langote wrote: On 2015-09-02 PM 03:25, Amit Kapila wrote: Will it handle deadlocks across different table partitions. Consider a case as below: T1 1. Updates row R1 of T1 on shard S1 2. Updates row R2

Re: [HACKERS] ON CONFLICT DO UPDATE using EXCLUDED.column gives an error about mismatched types

Peter, On 2015-08-11 AM 07:37, Peter Geoghegan wrote: > What I'm going to do is roll this into my own pending patch to fix the > issue with wholerow vars, which is also down to a problem with the > excluded targetlist initially generated by calling expandRelAttrs(). > Andres might want to take

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-02 11:01 GMT+02:00 Shulgin, Oleksandr : > On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule > wrote: > >> >>> But do we really need the slots mechanism? Would it not be OK to just >>> let the LWLock do the sequencing of concurrent

Re: [HACKERS] Horizontal scalability/sharding

On 2015/09/02 15:40, Amit Langote wrote: On 2015-09-02 PM 03:25, Amit Kapila wrote: On Wed, Sep 2, 2015 at 11:35 AM, Etsuro Fujita The UPDATE/DELETE pushdown, which I've proposed, would ensure the sane behaviour for inherited UPDATEs/DELETEs, as existing

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/1/15 11:59 PM, Michael Paquier wrote: On Wed, Sep 2, 2015 at 12:59 PM, Jim Nasby wrote: On 9/1/15 8:42 PM, Michael Paquier wrote: The crash is triggered by having an exception raised in this particular call stack. Since there's no syntax error in master/0.2.1, there's no assert failure

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On Wed, Sep 2, 2015 at 6:13 AM, Jim Nasby wrote: > On 9/1/15 11:59 PM, Michael Paquier wrote: >> >> On Wed, Sep 2, 2015 at 12:59 PM, Jim Nasby wrote: >>> >>> On 9/1/15 8:42 PM, Michael Paquier wrote: >>> The crash is triggered by having an exception raised in this

Re: [HACKERS] Horizontal scalability/sharding

On 2015-09-02 PM 03:25, Amit Kapila wrote: > On Wed, Sep 2, 2015 at 11:35 AM, Etsuro Fujita >> >> The UPDATE/DELETE pushdown, which I've proposed, would ensure the sane >> behaviour for inherited UPDATEs/DELETEs, as existing non-pushed-down >> UPDATE/DELETE does,

Re: [HACKERS] Horizontal scalability/sharding

On 2015/09/02 14:28, Amit Langote wrote: On 2015-09-02 PM 01:28, Amit Kapila wrote: On Tue, Sep 1, 2015 at 9:48 PM, Robert Haas wrote: I'm not averse to making the "connect to the remote nodes" part of this solution use something other than the FDW infrastructure at

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 11:35 AM, Etsuro Fujita wrote: > On 2015/09/02 14:28, Amit Langote wrote: > >> On 2015-09-02 PM 01:28, Amit Kapila wrote: >> >>> On Tue, Sep 1, 2015 at 9:48 PM, Robert Haas wrote: > I'm not averse to making

Re: [HACKERS] Multi-tenancy with RLS

On Fri, Aug 14, 2015 at 12:00 PM, Haribabu Kommi wrote: > > Here I attached the proof concept patch. Here I attached an updated patch by adding policies to the most of the system catalog tables, except the following. AggregateRelationId AccessMethodRelationId

Re: [HACKERS] Horizontal scalability/sharding

Amit Langote wrote: > On 2015-09-02 PM 03:25, Amit Kapila wrote: >> Will it handle deadlocks across different table partitions. Consider >> a case as below: >> >> T1 >> 1. Updates row R1 of T1 on shard S1 >> 2. Updates row R2 of T2 on shard S2 >> >> T2 >> 1. Updates row R2 of T2 on shard S2 >> 2.

Re: [HACKERS] creating extension including dependencies

Hi, I'm looking at committing this patch. I found some nitpick-level things that I can easily fixup. But I dislike two things: 1) Passing the list of parents through the cascade DefElem strikes me as incredibly ugly. For one the cascade option really should take a true/false type option on the

[HACKERS] PSA: Upcoming Linux scheduler changes

Folks, This is something we should be watching for and if people have time, testing to see how it affects us: http://lkml.iu.edu/hypermail/linux/kernel/1508.3/04818.html -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

On Wed, Sep 2, 2015 at 10:30 AM, Tom Lane wrote: > But if you have in mind that typical FDWs would actually create join paths > at that point, consider that > > 1. The FDW would have to find all the combinations of its supplied > relations (unless you are only intending to

Re: [HACKERS] Improving test coverage of extensions with pg_dump

Andres Freund wrote: > > As I recall, Andrew Dunstan has a module that > > tests cross-version pg_upgrade and one thing he does is dump both and > > compare; the problem is that there are differences, so he keeps a count > > of how many lines he expect to differ between any two releases. > > I'm

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 11:18 AM, Robert Haas wrote: > It would be a bad idea to cling blindly to the FDW infrastructure if > it's fundamentally inadequate to do what we want. On the other hand, > it would also be a bad idea to set about recreating it without a > really

Re: [HACKERS] creating extension including dependencies

On 2015-09-02 17:27:38 +0200, Andres Freund wrote: > 1) Passing the list of parents through the cascade DefElem strikes me as > incredibly ugly. > > For one the cascade option really should take a true/false type option > on the C level (so you can do defGetBoolean()), for another passing >

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

Hi On 09/02/2015 03:53 PM, Andres Freund wrote: Hi, On 2015-07-18 12:17:39 +0200, Julien Rouhaud wrote: I didn't know that the thread must exists on -hackers to be able to add a commitfest entry, so I transfer the thread here. Please, in the future, also update the title of the thread to

Re: [HACKERS] Improving test coverage of extensions with pg_dump

Andres Freund wrote: > Isn't a full test with a separate initdb, create extension etc. a really > heavyhanded way to test this? I mean that's a test where the setup takes > up to 10s, whereas the actual runtime is in the millisecond range? I spent some time looking over this patch yesterday, and

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 7:08 PM, Robert Haas wrote: > On Tue, Sep 1, 2015 at 12:00 AM, Pavan Deolasee > wrote: > > My worry is that if we start implementing them again from scratch, it > will > > take a few years before we get them in a usable

Re: [HACKERS] WIP: About CMake v2

On Tue, Sep 01, 2015 at 11:46:05AM -0400, Tom Lane wrote: > Andres Freund writes: > > On 2015-09-01 10:32:39 -0400, Noah Misch wrote: > >> A monolithic patch replacing the GNU make build system with a CMake build > >> system sounds far too hard to write and review; we should

Re: [HACKERS] pgbench stats per script & other stuff

On 2015-07-30 18:03:56 +0200, Fabien COELHO wrote: > > >v6 is just a rebase after a bug fix by Andres Freund. > > > >Also a small question: The patch currently displays pgbench scripts > >starting numbering at 0. Probably a little too geek... should start at 1? > > v7 is a rebase after another

Re: [HACKERS] Improving test coverage of extensions with pg_dump

On 2015-09-02 14:30:33 -0300, Alvaro Herrera wrote: > I was thinking in having this be renamed src/test/modules/extensions/ > and then the extension contained here would be renamed ext001_fk_tables > or something like that; later we could ext002_something for testing some > other angle of

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

Robert Haas writes: > On Wed, Sep 2, 2015 at 10:30 AM, Tom Lane wrote: >> But if you have in mind that typical FDWs would actually create join paths >> at that point, consider that >> >> 1. The FDW would have to find all the combinations of its

Re: [HACKERS] Improving test coverage of extensions with pg_dump

On 2015-08-02 19:15:58 -0700, Michael Paquier wrote: > +psql 'postgres', 'CREATE EXTENSION tables_fk'; > + > +# Insert some data before running the dump, this is needed to check > +# consistent data dump of tables with foreign key dependencies > +psql 'postgres', 'INSERT INTO cc_tab_fkey VALUES

Re: [HACKERS] WIP: Make timestamptz_out less slow.

On 2015-08-09 12:47:53 +1200, David Rowley wrote: > I took a bit of weekend time to finish this one off. Patch attached. > > A quick test shows a pretty good performance increase: > > create table ts (ts timestamp not null); > insert into ts select generate_series('2010-01-01 00:00:00',

Re: [HACKERS] [PATCH] Microvacuum for gist.

Hi, I don't know too much about gist, but did a quick read through. Mostly spotting some stylistic issues. Please fix those making it easier for the next reviewer. > *** a/src/backend/access/gist/gist.c > --- b/src/backend/access/gist/gist.c > *** > *** 36,42 static bool

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 04:14 PM, Petr Jelinek wrote: > On 2015-09-02 00:09, Josh Berkus wrote: >> On 09/01/2015 02:29 PM, Tomas Vondra wrote: >>> So while you may be right in single-DC deployments, with multi-DC >>> deployments the situation is quite different - not only that the network >>> bandwidth is

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 2015-09-02 18:06:54 +0200, Tomas Vondra wrote: > Maybe the best thing we can do is just completely abandon the "number of > spindles" idea, and just say "number of I/O requests to prefetch". Possibly > with an explanation of how to estimate it (devices * queue length). I think that'd be a lot

Re: [HACKERS] GIN pending clean up is not interruptable

On 2015-08-12 11:59:48 -0700, Jeff Janes wrote: > Attached patch does it that way. There was also a free-standing > CHECK_FOR_INTERRUPTS() which had no reason that I could see not be a > vacuum_delay_point, so I changed that one as well. I think we should backpatch this - any arguments against?

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On Tue, Sep 1, 2015 at 6:43 PM, and...@anarazel.de wrote: > Why a new tranche for each of these? And it can't be correct that each > has the same base? I complained about the same-base problem before. Apparently, that got ignored. > I don't really like the tranche model as

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 11:41 AM, Robert Haas wrote: > On Wed, Sep 2, 2015 at 1:57 PM, Josh Berkus wrote: >> Even if it's only on paper, any new sharding design needs to address >> these questions: >> >> 1. How do we ensure no/minimal data is lost if we lose a node? >> 2. How do we

Re: [HACKERS] src/test/ssl broken on HEAD

On Wed, Aug 26, 2015 at 3:37 AM, Michael Paquier wrote: > On Wed, Aug 26, 2015 at 4:35 PM, Michael Paquier > wrote: >> Only HEAD is impacted, and attached is a patch to fix the problem. > > Actually this version is better, I forgot to update

Re: [HACKERS] Allow replication roles to use file access functions

On Wed, Sep 2, 2015 at 11:32 PM, Fujii Masao wrote: > Did you confirm that replication user can complete pg_rewind > after this patch is applied? Yes. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] pgbench stats per script & other stuff

On Wed, Sep 2, 2015 at 5:55 PM, Andres Freund wrote: > On 2015-09-02 14:36:51 -0400, Robert Haas wrote: >> On Wed, Sep 2, 2015 at 2:20 PM, Fabien COELHO wrote: >> >> I'm wondering if percentages instead of weights would be a better >> >> idea. That'd mean

Re: [HACKERS] Allow replication roles to use file access functions

Micahel, * Michael Paquier (michael.paqu...@gmail.com) wrote: > On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera > wrote: > > Michael Paquier wrote: > >> As of now, file access functions in genfile.c can only be used by > >> superusers. This proposal is to relax those

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 1:59 PM, Merlin Moncure wrote: > On Tue, Sep 1, 2015 at 11:18 AM, Robert Haas wrote: >> It would be a bad idea to cling blindly to the FDW infrastructure if >> it's fundamentally inadequate to do what we want. On the other hand,

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 7:12 AM, Andres Freund wrote: > On 2015-07-19 16:34:52 -0700, Peter Geoghegan wrote: > Hm. Is a compiler test actually test anything reliably here? Won't this > just throw a warning during compile time about an unknown function? I'll need to look into

Re: [HACKERS] Allow replication roles to use file access functions

On Thu, Sep 3, 2015 at 8:59 AM, Andres Freund wrote: > On 2015-09-02 19:48:15 -0400, Tom Lane wrote: >> Just on general principles, this seems like a pretty horrid idea. >> To me replication privilege means the ability to transfer data out of >> the master, not to cause

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 2015-09-03 01:59:13 +0200, Tomas Vondra wrote: > That's a bit surprising, especially considering that e_i_c=30 means ~100 > pages to prefetch if I'm doing the math right. > > AFAIK queue depth for SATA drives generally is 32 (so prefetching 100 pages > should not make a difference), 256 for

Re: [HACKERS] exposing pg_controldata and pg_config as functions

On Wed, Sep 2, 2015 at 5:31 PM, Joe Conway wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 09/02/2015 05:25 PM, Tom Lane wrote: >> Robert Haas writes: >>> But I'm not sure I like the idea of adding a server dependency on >>> the ability

Re: [HACKERS] Improving test coverage of extensions with pg_dump

On Thu, Sep 3, 2015 at 2:30 AM, Alvaro Herrera wrote: > Andres Freund wrote: >> Isn't a full test with a separate initdb, create extension etc. a really >> heavyhanded way to test this? I mean that's a test where the setup takes >> up to 10s, whereas the actual runtime is in the millisecond range?

Re: [HACKERS] Improving test coverage of extensions with pg_dump

On Thu, Sep 3, 2015 at 12:38 AM, Andres Freund wrote: > Isn't a full test with a separate initdb, create extension etc. a really > heavyhanded way to test this? I mean that's a test where the setup takes > up to 10s, whereas the actual runtime is in the millisecond range? > > Adding tests in this

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/2/15 2:56 PM, Jim Nasby wrote: On 9/2/15 2:17 PM, Alvaro Herrera wrote: Michael Paquier wrote: I haven't written yet a test case but I think that we could reproduce that simply by having a relation referenced in the exception block of a first function, calling a second function that

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On 2015-09-02 16:23:13 -0700, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 4:12 PM, Andres Freund wrote: > >> Well, still needs to work for tuplestore, which does not have a SortTuple. > > > > Isn't it even more trivial there? It's just an array of void*'s? So > >

Re: [HACKERS] Allow replication roles to use file access functions

On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera wrote: > Michael Paquier wrote: >> Hi all, >> >> As of now, file access functions in genfile.c can only be used by >> superusers. This proposal is to relax those functions so as >> replication users can use them as well.

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 4:26 PM, Andres Freund wrote: > I'm not following? Just write pg_read_prefetch(state->memtuples + 3 + > readptr->current) and the corresponding version for tuplesort in the > callsites? Oh, I see. Maybe I'll do it that way when I pick this up in a

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 09/03/2015 12:23 AM, Andres Freund wrote: On 2015-09-02 14:31:35 -0700, Josh Berkus wrote: On 09/02/2015 02:25 PM, Tomas Vondra wrote: As I explained, spindles have very little to do with it - you need multiple I/O requests per device, to get the benefit. Sure, the DBAs should know how

Re: [HACKERS] Allow replication roles to use file access functions

On 2015-09-02 19:48:15 -0400, Tom Lane wrote: > Just on general principles, this seems like a pretty horrid idea. > To me replication privilege means the ability to transfer data out of > the master, not to cause arbitrary state changes on the master. It's not about the permission to trigger

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On 2015-09-02 17:14:12 -0700, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 12:24 PM, Peter Geoghegan wrote: > > On Wed, Sep 2, 2015 at 7:12 AM, Andres Freund wrote: > >> On 2015-07-19 16:34:52 -0700, Peter Geoghegan wrote: > >> Hm. Is a compiler test

Re: [HACKERS] src/test/ssl broken on HEAD

On Thu, Sep 3, 2015 at 5:22 AM, Robert Haas wrote: > Still, that's not a reason not commit this, so done. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 6:56 PM, Bruce Momjian wrote: > On Wed, Sep 2, 2015 at 02:41:46PM -0400, Robert Haas wrote: >> 4. Therefore, I think that we should instead use logical replication, >> which might be either synchronous or asynchronous. When you modify >> one copy of the

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

That doesn't match any of the empirical tests I did at the time. I posted graphs of the throughput for varying numbers of spindles with varying amount of prefetch. In every case more prefetching increases throuput up to N times the single platter throuput where N was the number of spindles. There

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 4:12 PM, Andres Freund wrote: >> Well, still needs to work for tuplestore, which does not have a SortTuple. > > Isn't it even more trivial there? It's just an array of void*'s? So > prefetch(state->memtuples + 3 + readptr->current)? All I meant is that

Re: [HACKERS] Allow replication roles to use file access functions

Michael Paquier writes: > On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera > wrote: >> Michael Paquier wrote: >>> The main argument for this change is that pg_rewind makes use of those >>> functions, forcing users to use a superuser role when

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 12:24 PM, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 7:12 AM, Andres Freund wrote: >> On 2015-07-19 16:34:52 -0700, Peter Geoghegan wrote: >> Hm. Is a compiler test actually test anything reliably here? Won't this >> just throw a

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

, On Wed, Aug 26, 2015 at 4:05 AM, Kouhei Kaigai wrote: >> On 2015/08/26 16:07, Kouhei Kaigai wrote: >> I wrote: >> >> Maybe I'm missing something, but why do we need such a flexiblity for >> >> the columnar-stores? >> >> > Even if we enforce them a new interface

Re: [HACKERS] Allow replication roles to use file access functions

On Thu, Sep 3, 2015 at 10:05 AM, Stephen Frost wrote: > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera wrote: > The replication role already has read-only access to everything > (nearly?) in the PGDATA directory. The specific issue in this

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 03:56 PM, Bruce Momjian wrote: On Wed, Sep 2, 2015 at 02:41:46PM -0400, Robert Haas wrote: 4. Therefore, I think that we should instead use logical replication, which might be either synchronous or asynchronous. When you modify one copy of the data, that change will then be

Re: [HACKERS] WIP: Make timestamptz_out less slow.

On 3 September 2015 at 05:10, Andres Freund wrote: > On 2015-08-09 12:47:53 +1200, David Rowley wrote: > > I took a bit of weekend time to finish this one off. Patch attached. > > > > A quick test shows a pretty good performance increase: > > > > create table ts (ts timestamp

Re: [HACKERS] Allow replication roles to use file access functions

* Michael Paquier (michael.paqu...@gmail.com) wrote: > On Thu, Sep 3, 2015 at 10:05 AM, Stephen Frost wrote: > > * Michael Paquier (michael.paqu...@gmail.com) wrote: > >> On Thu, Sep 3, 2015 at 4:52 AM, Alvaro Herrera wrote: > > The replication role already has read-only access to everything > >

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 07:50:25PM -0700, Joshua Drake wrote: > >Can you explain why logical replication is better than binary > >replication for this use-case? > > > > Selectivity? I was assuming you would just create identical slaves to handle failure, rather than moving selected data around.

Re: [HACKERS] GIN pending clean up is not interruptable

On Thu, Sep 3, 2015 at 2:15 AM, Andres Freund wrote: > On 2015-08-12 11:59:48 -0700, Jeff Janes wrote: >> Attached patch does it that way. There was also a free-standing >> CHECK_FOR_INTERRUPTS() which had no reason that I could see not be a >> vacuum_delay_point, so I

Re: [HACKERS] Horizontal scalability/sharding

On Thu, Sep 3, 2015 at 3:41 AM, Robert Haas wrote: > 3. IIUC, Postgres-XC handles this problem by reducing at least > volatile functions, maybe all functions, to constants. Then it > generates an SQL statement to be sent to the data node to make the > appropriate change. If there's more than one

Re: [HACKERS] Allow replication roles to use file access functions

On Thu, Sep 3, 2015 at 11:20 AM, Stephen Frost wrote: > * Michael Paquier wrote: >> 1) Use a differential backup to me, or the possibility to fetch a set >> of data block diffs from a source node using an LSN and then re-apply >> them on the target node. The major disadvantage of this approach is

Re: [HACKERS] pg_stat_statements query jumbling question

On 2015/09/01 14:39, Satoshi Nagayasu wrote: > On 2015/09/01 14:01, Tom Lane wrote: >> Satoshi Nagayasu writes: >>> On 2015/09/01 13:41, Peter Geoghegan wrote: If you want to use the queryId field directly, which I recall you mentioning before, then that's harder. There

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 9:43 PM, David Rowley wrote: > Peter, would you be able to share the test case which you saw the speedup > on. So far I've been unable to see much of an improvement. The case I tested was an internal sort CREATE INDEX. I don't recall the exact

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 09:03:25PM -0400, Robert Haas wrote: > > Can you explain why logical replication is better than binary > > replication for this use-case? > > Uh, well, for the same reasons it is better in many other cases. > Particularly, you probably don't want to replicate all the data

Re: [HACKERS] Horizontal scalability/sharding

On Thu, Sep 3, 2015 at 8:28 AM, Bruce Momjian wrote: > > On Wed, Sep 2, 2015 at 07:50:25PM -0700, Joshua Drake wrote: > > >Can you explain why logical replication is better than binary > > >replication for this use-case? > > > > > > > Selectivity? > > I was assuming you would

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On 3 September 2015 at 07:24, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 7:12 AM, Andres Freund wrote: > > What worries me about adding explicit prefetching is that it's *highly* > > platform and even micro-architecture dependent. Why is looking three >

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/09/03 9:41, Robert Haas wrote: That having been said, I don't entirely like Fujita-san's patch either. Much of the new code is called immediately adjacent to an FDW callback which could pretty trivially do the same thing itself, if needed. Another idea about that code is to call that

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 9:04 PM, Oleg Bartunov wrote: > > > > One lesson from XL we got is that we need testing framework for cluster, > so any cluster project should at least pass functional and performance > testing. > +1. In early XC days, we focused a lot on adding newer

Re: [HACKERS] Horizontal scalability/sharding

>> One lesson from XL we got is that we need testing framework for cluster, >> so any cluster project should at least pass functional and performance >> testing. >> > > +1. In early XC days, we focused a lot on adding newer features and > supporting as many PG features as possible. That took its

Re: [HACKERS] [PATCH] SQL function to report log message

On Mon, Aug 31, 2015 at 10:08 PM, Pavel Stehule wrote: > > > 2015-09-01 6:59 GMT+02:00 Pavel Stehule : > >> >> >> 2015-08-31 20:43 GMT+02:00 dinesh kumar : >> >>> Hi, >>> >>> On Sat, Aug 29, 2015 at 4:22 PM, Pavel Stehule

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/2/15 2:17 PM, Alvaro Herrera wrote: Michael Paquier wrote: I haven't written yet a test case but I think that we could reproduce that simply by having a relation referenced in the exception block of a first function, calling a second function that itself raises an exception, causing the

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

On Tue, Sep 1, 2015 at 10:58 PM, Stefan Kaltenbrunner < ste...@kaltenbrunner.cc> wrote: > On 07/25/2015 03:38 AM, dinesh kumar wrote: > > > > > > On Fri, Jul 24, 2015 at 10:22 AM, Robert Haas > > wrote: > > > > On Thu, Jul 23, 2015 at

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 08:27 PM, Robert Haas wrote: On Wed, Sep 2, 2015 at 1:59 PM, Merlin Moncure wrote: This strikes me as a bit of a conflict of interest with FDW which seems to want to hide the fact that it's foreign; the FDW implementation makes it's own optimization

Re: [HACKERS] psql tabcomplete - minor bugfix - tabcomplete for SET ROLE TO xxx

Hi 2015-09-02 15:23 GMT+02:00 Andres Freund : > Hi, > > On 2015-07-08 14:50:37 +0200, Pavel Stehule wrote: > > - static const char *const my_list[] = > > - {"DEFAULT", NULL}; > > + /* fallback for GUC settings */ > >

Re: [HACKERS] pgbench stats per script & other stuff

Hello Andres, Maybe add --builtin list to show them? Yep, easy enough. [...] +Shorthand for -b simple-update@1. +Shorthand for -b select-only@1. I'm a bit inclined to remove these options. Hm... This is really backward compatibility, and people may find reference to

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Wed, Sep 2, 2015 at 4:52 AM, Shulgin, Oleksandr wrote: > On Tue, Sep 1, 2015 at 8:12 PM, Andres Freund wrote: >> >> On 2015-09-01 14:07:19 -0400, Robert Haas wrote: >> > But I think it's quite wrong to assume that the infrastructure for >> >

Re: [HACKERS] src/test/ssl broken on HEAD

On 09/02/2015 04:22 PM, Robert Haas wrote: On Wed, Aug 26, 2015 at 3:37 AM, Michael Paquier wrote: On Wed, Aug 26, 2015 at 4:35 PM, Michael Paquier wrote: Only HEAD is impacted, and attached is a patch to fix the problem. Actually this

Re: [HACKERS] pgbench stats per script & other stuff

On Wed, Sep 2, 2015 at 2:20 PM, Fabien COELHO wrote: >> I'm wondering if percentages instead of weights would be a better >> idea. That'd mean you'd be forced to be more careful when adding another >> script (having to adjust the percentages of other scripts) but arguably >>

Re: [HACKERS] FSM versus GIN pending list bloat

On Mon, Aug 10, 2015 at 1:16 PM, Jeff Janes wrote: > I have a simple test case that inserts an array of 101 md5 digests into each > row. With 10_000 of these rows inserted into an already indexed table, I > get 40MB for the table and 80MB for the index unpatched. With the

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

Michael Paquier wrote: > I haven't written yet a test case but I think that we could reproduce > that simply by having a relation referenced in the exception block of > a first function, calling a second function that itself raises an > exception, causing the referencing error. Hm, so function 2

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

I wrote: > ... I imagine that we should allow FDWs to > store some data within RelOptInfo structs that represent foreign joins > belonging entirely to them, so that there'd be a handy place to keep that > data till later. Actually, if we do that (ie, provide a "void *fdw_state" field in join

Re: [HACKERS] Allow replication roles to use file access functions

Michael Paquier wrote: > Hi all, > > As of now, file access functions in genfile.c can only be used by > superusers. This proposal is to relax those functions so as > replication users can use them as well. Here are the functions aimed > by this patch: > - pg_stat_file > - pg_read_binary_file > -

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

On 09/02/2015 10:10 PM, dinesh kumar wrote: > On Tue, Sep 1, 2015 at 10:58 PM, Stefan Kaltenbrunner > > wrote: > > On 07/25/2015 03:38 AM, dinesh kumar wrote: > > > > > > On Fri, Jul 24, 2015 at 10:22 AM, Robert Haas

Re: [HACKERS] ON CONFLICT DO UPDATE using EXCLUDED.column gives an error about mismatched types

On Wed, Sep 2, 2015 at 1:18 AM, Amit Langote wrote: > Did you get around to making a patch for this? I've worked on it inconsistently. I'll pick this up again soon. I may take the opportunity to talk this over with Andres in person when we meet at Postgres Open

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 1:57 PM, Josh Berkus wrote: > Even if it's only on paper, any new sharding design needs to address > these questions: > > 1. How do we ensure no/minimal data is lost if we lose a node? > 2. How do we replace a lost node (without taking the cluster down)?

Re: [HACKERS] [PATCH] SQL function to report log message

On Tue, Sep 1, 2015 at 6:13 PM, Jim Nasby wrote: > My thought is that there's a fair amount of places where we do string > comparison for not a great reason. Perhaps a better example is data that > comes back from a trigger; AFTER/BEFORE, INSERT/UPDATE/..., which is more

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 2 Sep 2015 14:54, "Andres Freund" wrote: > > > > + /*-- > > + * The user-visible GUC parameter is the number of drives (spindles), > > + * which we need to translate to a number-of-pages-to-prefetch target. > > + * The target value is stashed in

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, On 02/09/2015 18:06, Tomas Vondra wrote: > Hi > > On 09/02/2015 03:53 PM, Andres Freund wrote: >> >> Hi, >> >> On 2015-07-18 12:17:39 +0200, Julien Rouhaud wrote: >>> I didn't know that the thread must exists on -hackers to be >>> able to add

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 3:03 PM, Josh Berkus wrote: >> 4. Therefore, I think that we should instead use logical replication, >> which might be either synchronous or asynchronous. When you modify >> one copy of the data, that change will then be replicated to all other >> nodes.

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/09/2015 15:53, Andres Freund wrote: > On 2015-07-18 12:17:39 +0200, Julien Rouhaud wrote: > > You also didn't touch /* * How many buffers PrefetchBuffer callers > should try to stay ahead of their * ReadBuffer calls by. This is > maintained by

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 12:30 PM, Robert Haas wrote: > On Wed, Sep 2, 2015 at 3:03 PM, Josh Berkus wrote: >>> 4. Therefore, I think that we should instead use logical replication, >>> which might be either synchronous or asynchronous. When you modify >>> one copy of the data, that

  1   2   >