Re: [HACKERS] Horizontal scalability/sharding

On 2015/09/02 15:40, Amit Langote wrote: On 2015-09-02 PM 03:25, Amit Kapila wrote: On Wed, Sep 2, 2015 at 11:35 AM, Etsuro Fujita The UPDATE/DELETE pushdown, which I've proposed, would ensure the sane behaviour for inherited UPDATEs/DELETEs, as existing non-pushed-down UPDATE/DELETE does, bec

Re: [HACKERS] Horizontal scalability/sharding

Amit Langote wrote: > On 2015-09-02 PM 03:25, Amit Kapila wrote: >> Will it handle deadlocks across different table partitions. Consider >> a case as below: >> >> T1 >> 1. Updates row R1 of T1 on shard S1 >> 2. Updates row R2 of T2 on shard S2 >> >> T2 >> 1. Updates row R2 of T2 on shard S2 >> 2. U

Re: [HACKERS] Horizontal scalability/sharding

On 2015-09-02 PM 04:07, Albe Laurenz wrote: > Amit Langote wrote: >> On 2015-09-02 PM 03:25, Amit Kapila wrote: >>> Will it handle deadlocks across different table partitions. Consider >>> a case as below: >>> >>> T1 >>> 1. Updates row R1 of T1 on shard S1 >>> 2. Updates row R2 of T2 on shard S2 >>

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On Wed, Sep 2, 2015 at 6:21 AM, Michael Paquier wrote: > Yes, that's what I have been looking at actually by having some markers in > relcache.c. The reference count of this relation get incremented here: So, I have been playing more with this code, and as mentioned by Andres this test case goes

Re: [HACKERS] Horizontal scalability/sharding

On 2015/09/02 16:40, Amit Langote wrote: On 2015-09-02 PM 04:07, Albe Laurenz wrote: Amit Langote wrote: On 2015-09-02 PM 03:25, Amit Kapila wrote: Will it handle deadlocks across different table partitions. Consider a case as below: T1 1. Updates row R1 of T1 on shard S1 2. Updates row R2 of

Re: [HACKERS] ON CONFLICT DO UPDATE using EXCLUDED.column gives an error about mismatched types

Peter, On 2015-08-11 AM 07:37, Peter Geoghegan wrote: > What I'm going to do is roll this into my own pending patch to fix the > issue with wholerow vars, which is also down to a problem with the > excluded targetlist initially generated by calling expandRelAttrs(). > Andres might want to take an

Re: [HACKERS] Horizontal scalability/sharding

Etsuro Fujita wrote: > On 2015/09/02 16:40, Amit Langote wrote: >> On 2015-09-02 PM 04:07, Albe Laurenz wrote: >>> Amit Langote wrote: On 2015-09-02 PM 03:25, Amit Kapila wrote: > Will it handle deadlocks across different table partitions. Consider > a case as below: > > T1 >>>

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Tue, Sep 1, 2015 at 8:12 PM, Andres Freund wrote: > On 2015-09-01 14:07:19 -0400, Robert Haas wrote: > > But I think it's quite wrong to assume that the infrastructure for > > this is available and usable everywhere, because in my experience, > > that's far from the case. > > Especially when t

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule wrote: > >> But do we really need the slots mechanism? Would it not be OK to just >> let the LWLock do the sequencing of concurrent requests? Given that we >> only going to use one message queue per cluster, there's not much >> concurrency you can g

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-02 11:01 GMT+02:00 Shulgin, Oleksandr : > On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule > wrote: > >> >>> But do we really need the slots mechanism? Would it not be OK to just >>> let the LWLock do the sequencing of concurrent requests? Given that we >>> only going to use one message q

Re: [HACKERS] Horizontal scalability/sharding

On 2015-09-02 PM 05:07, Etsuro Fujita wrote: > On 2015/09/02 16:40, Amit Langote wrote: >> On 2015-09-02 PM 04:07, Albe Laurenz wrote: >>> >>> That would only hold for a single query, right? >>> >>> If 1. and 2. in the above example come from different queries within one >>> transaction, you cannot

Re: [HACKERS] pageinspect patch, for showing tuple data

В письме от 3 августа 2015 15:35:23 пользователь Alvaro Herrera написал: > Nikolay Shaplov wrote: > > This patch adds several new functions, available from SQL queries. All > > these functions are based on heap_page_items, but accept slightly > > different arguments and has one additional column at

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 12:49 AM, Josh Berkus wrote: > On 09/01/2015 11:36 AM, Tomas Vondra wrote: > >> We want multiple copies of shards created by the sharding system itself. > >> Having a separate, and completely orthagonal, redundancy system to the > >> sharding system is overly burdensome o

Re: [HACKERS] Horizontal scalability/sharding

On 2015-09-02 PM 06:41, Amit Langote wrote: > > I think Albe may have a point here... > > Even inherited updates case appears to cause a deadlock if they are in > different queries. Demonstrated below: > > -- setup > CREATE TABLE t(a int); > CREATE TABLE t1() INHERITS(t); > CREATE TABLE t2() INH

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 11:16 AM, Pavel Stehule wrote: > > > 2015-09-02 11:01 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule >> wrote: >> >>> But do we really need the slots mechanism? Would it not be OK to just let th

Re: PENDING_LIST_CLEANUP_SIZE - maximum size of GIN pending list Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

On Tue, Aug 11, 2015 at 2:58 AM, Jeff Janes wrote: > On Thu, Oct 30, 2014 at 5:30 AM, Fujii Masao wrote: >> >> On Thu, Oct 30, 2014 at 7:30 PM, Etsuro Fujita >> wrote: >> >> > + { >> > + {"pending_list_cleanup_size", PGC_USERSET, >> > CLIENT_CONN_STATEMENT, >> > +

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 3:55 PM, Amit Langote wrote: > On 2015-09-02 PM 06:41, Amit Langote wrote: > > > > I think Albe may have a point here... > > > > Even inherited updates case appears to cause a deadlock if they are in > > different queries. Demonstrated below: > > > > -- setup > > CREATE TAB

Re: [HACKERS] FSM versus GIN pending list bloat

On Wed, Aug 5, 2015 at 5:50 AM, Simon Riggs wrote: > On 4 August 2015 at 21:04, Jeff Janes wrote: > >>> >>> Couple of questions here... >>> >>> * the docs say "it's desirable to have pending-list cleanup occur in the >>> background", but there is no way to invoke that, except via VACUUM. I think

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 4:19 PM, Pavan Deolasee wrote: > On Wed, Sep 2, 2015 at 3:55 PM, Amit Langote < langote_amit...@lab.ntt.co.jp> wrote: >> >> On 2015-09-02 PM 06:41, Amit Langote wrote: >> >> Which, I now realize, is not the worry Amit Kapila's expresses. >> >> The deadlock was *indeed detect

[HACKERS] about fsync in CLOG buffer write

Hi Hackers, We host many PG instances on each of our machines, and we want to minimize the calls to fsync within each instance, in order to minimize possible impact on other instances. We found there is a fsync call when CLOG buffer is written out in SlruPhysicalWritePage(). It is often called wh

Re: [HACKERS] synchronous_commit = apply

On Wed, Sep 2, 2015 at 10:25 AM, Thomas Munro wrote: > Hi > > Do you think it's reasonable to want to COMMIT a particular transaction on a > master node, and then immediately run a read-only query on a hot standby > node that is guaranteed to see that transaction? > > A friend of mine who works wi

Re: [HACKERS] about fsync in CLOG buffer write

On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: > We found there is a fsync call when CLOG buffer > is written out in SlruPhysicalWritePage(). It is often called when a backend > needs to check transaction status with SimpleLruReadPage(). That's when there's not enough buffers available some other, a

[HACKERS] commitfest does not see my real latest patch

Hi! I've added a patch to commitfest https://commitfest.postgresql.org/7/363/ but in my mail thread I've committed two version of patch. Old one (pageinspect.diff) and a new one (pageinspect_show_tuple_data.diff) and commitfest software for some reason sees only old patch, not new. I think t

Re: [HACKERS] synchronous_commit = apply

On Tue, Sep 1, 2015 at 9:25 PM, Thomas Munro wrote: > The next problem is that the master can be waiting quite a long time for a > reply from the remote walreceiver containing the desired apply LSN: in the > best case it learns of apply progress from replies to subsequent unrelated > records (whic

Re: [HACKERS] commitfest does not see my real latest patch

On Wed, Sep 2, 2015 at 2:34 PM, Nikolay Shaplov wrote: > Hi! > > I've added a patch to commitfest > > https://commitfest.postgresql.org/7/363/ > > > but in my mail thread I've committed two version of patch. Old one > (pageinspect.diff) and a new one (pageinspect_show_tuple_data.diff) > > and com

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-02 12:36 GMT+02:00 Shulgin, Oleksandr : > On Wed, Sep 2, 2015 at 11:16 AM, Pavel Stehule > wrote: > >> >> >> 2015-09-02 11:01 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule >>> wrote: >>> > But do we really

Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

On 2015/08/01 23:25, Tom Lane wrote: > Robert Haas writes: >> The problem that was bothering us (or at least what was bothering me) >> is that the PlannerInfo provides only a list of SpecialJoinInfo >> structures, which don't directly give you the original join order. In >> fact, min_righthand an

Re: [HACKERS] Horizontal scalability/sharding

On 2015/09/02 20:42, Amit Kapila wrote: On Wed, Sep 2, 2015 at 4:19 PM, Pavan Deolasee mailto:pavan.deola...@gmail.com>> wrote: > On Wed, Sep 2, 2015 at 3:55 PM, Amit Langote mailto:langote_amit...@lab.ntt.co.jp>> wrote: >> On 2015-09-02 PM 06:41, Amit Langote wrote: >> Which, I now realize, i

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 2:56 PM, Pavel Stehule wrote: > > > 2015-09-02 12:36 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Wed, Sep 2, 2015 at 11:16 AM, Pavel Stehule >> wrote: >> >>> >>> >>> 2015-09-02 11:01 GMT+02:00 Shulgin, Oleksandr < >>> oleksandr.shul...@zalando.d

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-02 15:00 GMT+02:00 Shulgin, Oleksandr : > On Wed, Sep 2, 2015 at 2:56 PM, Pavel Stehule > wrote: > >> >> >> 2015-09-02 12:36 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> On Wed, Sep 2, 2015 at 11:16 AM, Pavel Stehule >>> wrote: >>> 2015-09-02 11

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 3:04 PM, Pavel Stehule wrote: > > >> Well, maybe I'm missing something, but sh_mq_create() will just overwrite >> the contents of the struct, so it doesn't care about sender/receiver: only >> sh_mq_set_sender/receiver() do. >> > > if you create sh_mq from scratch, then you c

Re: [HACKERS] [POC] FETCH limited by bytes.

On 2015-02-27 13:50:22 -0500, Corey Huinker wrote: > +static DefElem* > +get_option(List *options, char *optname) > +{ > + ListCell *lc; > + > + foreach(lc, options) > + { > + DefElem *def = (DefElem *) lfirst(lc); > + > + if (strcmp(def->defname, optname) == 0)

Re: [HACKERS] psql tabcomplete - minor bugfix - tabcomplete for SET ROLE TO xxx

Hi, On 2015-07-08 14:50:37 +0200, Pavel Stehule wrote: > - static const char *const my_list[] = > - {"DEFAULT", NULL}; > + /* fallback for GUC settings */ > > - COMPLETE_WITH_LIST(my_list); > + ch

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

Hi, On 2015-07-18 12:17:39 +0200, Julien Rouhaud wrote: > I didn't know that the thread must exists on -hackers to be able to add > a commitfest entry, so I transfer the thread here. Please, in the future, also update the title of the thread to something fitting. > @@ -539,6 +541,9 @@ ExecInitB

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 06:11:45PM -0400, Bruce Momjian wrote: > Let me clearer about what the Citus Data paper shows. I said originally > that the data was sent to the coordinator, sorted, then resent to the > shards, but the document: > > https://goo.gl/vJWF85 > > https://www.citus

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On 2015-07-19 16:34:52 -0700, Peter Geoghegan wrote: > +# PGAC_C_BUILTIN_PREFETCH > +# - > +# Check if the C compiler understands __builtin_prefetch(), > +# and define HAVE__BUILTIN_PREFETCH if so. > +AC_DEFUN([PGAC_C_BUILTIN_PREFETCH], > +[AC_CACHE_CHECK(for __builtin_prefe

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

On Wed, Aug 5, 2015 at 2:16 AM, Robert Haas wrote: > On Mon, Aug 3, 2015 at 10:31 AM, Fujii Masao wrote: >> track_commit_timestamp tracks COMMIT PREPARED as expected in standby server, >> but not in master server. Is this intentional? It should track COMMIT >> PREPARED >> even in master? Otherwi

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

Etsuro Fujita writes: > On 2015/08/01 23:25, Tom Lane wrote: >> In my vision you probably really only want one call per build_join_rel >> event (that is, per construction of a new RelOptInfo), not per >> make_join_rel event. > I'd vote for hooking at standard_join_search. I think that method wou

Re: [HACKERS] Allow replication roles to use file access functions

On Thu, Aug 27, 2015 at 10:50 AM, Michael Paquier wrote: > Hi all, > > As of now, file access functions in genfile.c can only be used by > superusers. This proposal is to relax those functions so as > replication users can use them as well. Here are the functions aimed > by this patch: > - pg_stat

Re: [HACKERS] WIP: About CMake v2

On Tue, Sep 01, 2015 at 11:46:05AM -0400, Tom Lane wrote: > Andres Freund writes: > > On 2015-09-01 10:32:39 -0400, Noah Misch wrote: > >> A monolithic patch replacing the GNU make build system with a CMake build > >> system sounds far too hard to write and review; we should expect to > >> mainta

Re: [HACKERS] creating extension including dependencies

Hi, I'm looking at committing this patch. I found some nitpick-level things that I can easily fixup. But I dislike two things: 1) Passing the list of parents through the cascade DefElem strikes me as incredibly ugly. For one the cascade option really should take a true/false type option on the C

[HACKERS] PSA: Upcoming Linux scheduler changes

Folks, This is something we should be watching for and if people have time, testing to see how it affects us: http://lkml.iu.edu/hypermail/linux/kernel/1508.3/04818.html -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and

Re: [HACKERS] creating extension including dependencies

On 2015-09-02 17:27:38 +0200, Andres Freund wrote: > 1) Passing the list of parents through the cascade DefElem strikes me as > incredibly ugly. > > For one the cascade option really should take a true/false type option > on the C level (so you can do defGetBoolean()), for another passing > throug

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 7:08 PM, Robert Haas wrote: > On Tue, Sep 1, 2015 at 12:00 AM, Pavan Deolasee > wrote: > > My worry is that if we start implementing them again from scratch, it > will > > take a few years before we get them in a usable state. What XC/XL lacked > is > > probably a Robert H

Re: [HACKERS] Improving test coverage of extensions with pg_dump

On 2015-08-02 19:15:58 -0700, Michael Paquier wrote: > +psql 'postgres', 'CREATE EXTENSION tables_fk'; > + > +# Insert some data before running the dump, this is needed to check > +# consistent data dump of tables with foreign key dependencies > +psql 'postgres', 'INSERT INTO cc_tab_fkey VALUES (1)

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

Hi On 09/02/2015 03:53 PM, Andres Freund wrote: Hi, On 2015-07-18 12:17:39 +0200, Julien Rouhaud wrote: I didn't know that the thread must exists on -hackers to be able to add a commitfest entry, so I transfer the thread here. Please, in the future, also update the title of the thread to so

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

On Wed, Sep 2, 2015 at 10:30 AM, Tom Lane wrote: > But if you have in mind that typical FDWs would actually create join paths > at that point, consider that > > 1. The FDW would have to find all the combinations of its supplied > relations (unless you are only intending to generate one path for th

Re: [HACKERS] [PATCH] Microvacuum for gist.

Hi, I don't know too much about gist, but did a quick read through. Mostly spotting some stylistic issues. Please fix those making it easier for the next reviewer. > *** a/src/backend/access/gist/gist.c > --- b/src/backend/access/gist/gist.c > *** > *** 36,42 static bool gistinse

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On 2 September 2015 at 04:52, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > > On Tue, Sep 1, 2015 at 8:12 PM, Andres Freund wrote: >> >> On 2015-09-01 14:07:19 -0400, Robert Haas wrote: >> > But I think it's quite wrong to assume that the infrastructure for >> > this is available and

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 2015-09-02 18:06:54 +0200, Tomas Vondra wrote: > Maybe the best thing we can do is just completely abandon the "number of > spindles" idea, and just say "number of I/O requests to prefetch". Possibly > with an explanation of how to estimate it (devices * queue length). I think that'd be a lot b

[HACKERS] about fsync in CLOG buffer write

Hi Hackers, We host many PG instances on each of our machines, and we want to minimize the calls to fsync within each instance, in order to minimize possible impact on other instances. We found there is a fsync call when CLOG buffer is written out in SlruPhysicalWritePage(). It is often called whe

Re: [HACKERS] WIP: Make timestamptz_out less slow.

On 2015-08-09 12:47:53 +1200, David Rowley wrote: > I took a bit of weekend time to finish this one off. Patch attached. > > A quick test shows a pretty good performance increase: > > create table ts (ts timestamp not null); > insert into ts select generate_series('2010-01-01 00:00:00', '2011-01-

Re: [HACKERS] GIN pending clean up is not interruptable

On 2015-08-12 11:59:48 -0700, Jeff Janes wrote: > Attached patch does it that way. There was also a free-standing > CHECK_FOR_INTERRUPTS() which had no reason that I could see not be a > vacuum_delay_point, so I changed that one as well. I think we should backpatch this - any arguments against?

Re: [HACKERS] pgbench stats per script & other stuff

On 2015-07-30 18:03:56 +0200, Fabien COELHO wrote: > > >v6 is just a rebase after a bug fix by Andres Freund. > > > >Also a small question: The patch currently displays pgbench scripts > >starting numbering at 0. Probably a little too geek... should start at 1? > > v7 is a rebase after another sm

Re: [HACKERS] Improving test coverage of extensions with pg_dump

Andres Freund wrote: > Isn't a full test with a separate initdb, create extension etc. a really > heavyhanded way to test this? I mean that's a test where the setup takes > up to 10s, whereas the actual runtime is in the millisecond range? I spent some time looking over this patch yesterday, and

Re: [HACKERS] Improving test coverage of extensions with pg_dump

On 2015-09-02 14:30:33 -0300, Alvaro Herrera wrote: > I was thinking in having this be renamed src/test/modules/extensions/ > and then the extension contained here would be renamed ext001_fk_tables > or something like that; later we could ext002_something for testing some > other angle of extension

Re: [HACKERS] Improving test coverage of extensions with pg_dump

Andres Freund wrote: > > As I recall, Andrew Dunstan has a module that > > tests cross-version pg_upgrade and one thing he does is dump both and > > compare; the problem is that there are differences, so he keeps a count > > of how many lines he expect to differ between any two releases. > > I'm

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

Robert Haas writes: > On Wed, Sep 2, 2015 at 10:30 AM, Tom Lane wrote: >> But if you have in mind that typical FDWs would actually create join paths >> at that point, consider that >> >> 1. The FDW would have to find all the combinations of its supplied >> relations (unless you are only intendin

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 04:14 PM, Petr Jelinek wrote: > On 2015-09-02 00:09, Josh Berkus wrote: >> On 09/01/2015 02:29 PM, Tomas Vondra wrote: >>> So while you may be right in single-DC deployments, with multi-DC >>> deployments the situation is quite different - not only that the network >>> bandwidth is no

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 11:18 AM, Robert Haas wrote: > It would be a bad idea to cling blindly to the FDW infrastructure if > it's fundamentally inadequate to do what we want. On the other hand, > it would also be a bad idea to set about recreating it without a > really good reason, and - just to

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

I wrote: > ... I imagine that we should allow FDWs to > store some data within RelOptInfo structs that represent foreign joins > belonging entirely to them, so that there'd be a handy place to keep that > data till later. Actually, if we do that (ie, provide a "void *fdw_state" field in join RelOp

Re: [HACKERS] pgbench stats per script & other stuff

Hello Andres, Maybe add --builtin list to show them? Yep, easy enough. [...] +Shorthand for -b simple-update@1. +Shorthand for -b select-only@1. I'm a bit inclined to remove these options. Hm... This is really backward compatibility, and people may find reference to th

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 1:59 PM, Merlin Moncure wrote: > On Tue, Sep 1, 2015 at 11:18 AM, Robert Haas wrote: >> It would be a bad idea to cling blindly to the FDW infrastructure if >> it's fundamentally inadequate to do what we want. On the other hand, >> it would also be a bad idea to set about

Re: [HACKERS] pgbench stats per script & other stuff

On Wed, Sep 2, 2015 at 2:20 PM, Fabien COELHO wrote: >> I'm wondering if percentages instead of weights would be a better >> idea. That'd mean you'd be forced to be more careful when adding another >> script (having to adjust the percentages of other scripts) but arguably >> that's a good thing? >

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 1:57 PM, Josh Berkus wrote: > Even if it's only on paper, any new sharding design needs to address > these questions: > > 1. How do we ensure no/minimal data is lost if we lose a node? > 2. How do we replace a lost node (without taking the cluster down)? >2. a. how do we

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On Tue, Sep 1, 2015 at 6:43 PM, and...@anarazel.de wrote: > Why a new tranche for each of these? And it can't be correct that each > has the same base? I complained about the same-base problem before. Apparently, that got ignored. > I don't really like the tranche model as in the patch right no

Re: [HACKERS] [PATCH] SQL function to report log message

On Tue, Sep 1, 2015 at 6:13 PM, Jim Nasby wrote: > My thought is that there's a fair amount of places where we do string > comparison for not a great reason. Perhaps a better example is data that > comes back from a trigger; AFTER/BEFORE, INSERT/UPDATE/..., which is more > expensive to setup the v

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 2 Sep 2015 14:54, "Andres Freund" wrote: > > > > + /*-- > > + * The user-visible GUC parameter is the number of drives (spindles), > > + * which we need to translate to a number-of-pages-to-prefetch target. > > + * The target value is stashed in *extra and then assign

Re: [HACKERS] FSM versus GIN pending list bloat

On Mon, Aug 10, 2015 at 1:16 PM, Jeff Janes wrote: > I have a simple test case that inserts an array of 101 md5 digests into each > row. With 10_000 of these rows inserted into an already indexed table, I > get 40MB for the table and 80MB for the index unpatched. With the patch, I > get 7.3 MB f

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Wed, Sep 2, 2015 at 4:52 AM, Shulgin, Oleksandr wrote: > On Tue, Sep 1, 2015 at 8:12 PM, Andres Freund wrote: >> >> On 2015-09-01 14:07:19 -0400, Robert Haas wrote: >> > But I think it's quite wrong to assume that the infrastructure for >> > this is available and usable everywhere, because in

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 11:41 AM, Robert Haas wrote: > On Wed, Sep 2, 2015 at 1:57 PM, Josh Berkus wrote: >> Even if it's only on paper, any new sharding design needs to address >> these questions: >> >> 1. How do we ensure no/minimal data is lost if we lose a node? >> 2. How do we replace a lost node (with

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, On 02/09/2015 18:06, Tomas Vondra wrote: > Hi > > On 09/02/2015 03:53 PM, Andres Freund wrote: >> >> Hi, >> >> On 2015-07-18 12:17:39 +0200, Julien Rouhaud wrote: >>> I didn't know that the thread must exists on -hackers to be >>> able to add a

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

Michael Paquier wrote: > I haven't written yet a test case but I think that we could reproduce > that simply by having a relation referenced in the exception block of > a first function, calling a second function that itself raises an > exception, causing the referencing error. Hm, so function 2

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

On Wed, Sep 2, 2015 at 7:12 AM, Andres Freund wrote: > On 2015-07-19 16:34:52 -0700, Peter Geoghegan wrote: > Hm. Is a compiler test actually test anything reliably here? Won't this > just throw a warning during compile time about an unknown function? I'll need to look into that. >> +/* >> + * P

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 3:03 PM, Josh Berkus wrote: >> 4. Therefore, I think that we should instead use logical replication, >> which might be either synchronous or asynchronous. When you modify >> one copy of the data, that change will then be replicated to all other >> nodes. If you are OK with

Re: [HACKERS] [PATCH] SQL function to report log message

On Mon, Aug 31, 2015 at 10:08 PM, Pavel Stehule wrote: > > > 2015-09-01 6:59 GMT+02:00 Pavel Stehule : > >> >> >> 2015-08-31 20:43 GMT+02:00 dinesh kumar : >> >>> Hi, >>> >>> On Sat, Aug 29, 2015 at 4:22 PM, Pavel Stehule >>> wrote: >>> Hi I am starting to work review of this patc

Re: [HACKERS] Pg_upgrade remote copy

Bruce Momjian wrote: > On Fri, Aug 28, 2015 at 10:34:38PM -0700, AI Rumman wrote: > > In pg_upgrade, how about adding a feature to copy data directory over > > network. > > That is, we can run pg_upgrade from our new host, where old host will be a > > remote machine. > I think it is much simpler

Re: [HACKERS] Allow replication roles to use file access functions

Michael Paquier wrote: > Hi all, > > As of now, file access functions in genfile.c can only be used by > superusers. This proposal is to relax those functions so as > replication users can use them as well. Here are the functions aimed > by this patch: > - pg_stat_file > - pg_read_binary_file > -

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/2/15 2:17 PM, Alvaro Herrera wrote: Michael Paquier wrote: I haven't written yet a test case but I think that we could reproduce that simply by having a relation referenced in the exception block of a first function, calling a second function that itself raises an exception, causing the re

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 08:27 PM, Robert Haas wrote: On Wed, Sep 2, 2015 at 1:59 PM, Merlin Moncure wrote: This strikes me as a bit of a conflict of interest with FDW which seems to want to hide the fact that it's foreign; the FDW implementation makes it's own optimization decisions which might make s

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

On Tue, Sep 1, 2015 at 10:58 PM, Stefan Kaltenbrunner < ste...@kaltenbrunner.cc> wrote: > On 07/25/2015 03:38 AM, dinesh kumar wrote: > > > > > > On Fri, Jul 24, 2015 at 10:22 AM, Robert Haas > > wrote: > > > > On Thu, Jul 23, 2015 at 8:15 PM, dinesh kumar > >

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

On 09/02/2015 10:10 PM, dinesh kumar wrote: > On Tue, Sep 1, 2015 at 10:58 PM, Stefan Kaltenbrunner > mailto:ste...@kaltenbrunner.cc>> wrote: > > On 07/25/2015 03:38 AM, dinesh kumar wrote: > > > > > > On Fri, Jul 24, 2015 at 10:22 AM, Robert Haas >

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/09/2015 15:53, Andres Freund wrote: > On 2015-07-18 12:17:39 +0200, Julien Rouhaud wrote: > > You also didn't touch /* * How many buffers PrefetchBuffer callers > should try to stay ahead of their * ReadBuffer calls by. This is > maintained by

Re: [HACKERS] src/test/ssl broken on HEAD

On Wed, Aug 26, 2015 at 3:37 AM, Michael Paquier wrote: > On Wed, Aug 26, 2015 at 4:35 PM, Michael Paquier > wrote: >> Only HEAD is impacted, and attached is a patch to fix the problem. > > Actually this version is better, I forgot to update a comment. For so long as this test suite is not run b

Re: [HACKERS] ON CONFLICT DO UPDATE using EXCLUDED.column gives an error about mismatched types

On Wed, Sep 2, 2015 at 1:18 AM, Amit Langote wrote: > Did you get around to making a patch for this? I've worked on it inconsistently. I'll pick this up again soon. I may take the opportunity to talk this over with Andres in person when we meet at Postgres Open shortly. -- Peter Geoghegan --

Re: [HACKERS] src/test/ssl broken on HEAD

On 09/02/2015 04:22 PM, Robert Haas wrote: On Wed, Aug 26, 2015 at 3:37 AM, Michael Paquier wrote: On Wed, Aug 26, 2015 at 4:35 PM, Michael Paquier wrote: Only HEAD is impacted, and attached is a patch to fix the problem. Actually this version is better, I forgot to update a comment. For

Re: [HACKERS] psql tabcomplete - minor bugfix - tabcomplete for SET ROLE TO xxx

Hi 2015-09-02 15:23 GMT+02:00 Andres Freund : > Hi, > > On 2015-07-08 14:50:37 +0200, Pavel Stehule wrote: > > - static const char *const my_list[] = > > - {"DEFAULT", NULL}; > > + /* fallback for GUC settings */ > > > > -

Re: [HACKERS] src/test/ssl broken on HEAD

On Wed, Sep 2, 2015 at 4:50 PM, Andrew Dunstan wrote: > Tell me what's needed and I'll look at creating a buildfarm test module for > it. You run the tests via: make -C src/test/ssl check But nota bene security caveats: commit e39250c644ea7cd3904e4e24570db21a209cf97f Author: Heikki Linnakangas

Re: [HACKERS] src/test/ssl broken on HEAD

Andrew Dunstan wrote: > > > On 09/02/2015 04:22 PM, Robert Haas wrote: > >For so long as this test suite is not run by 'make check-world' or by > >the buildfarm, it's likely to keep getting broken, and we're likely to > >keep not noticing. I realize that the decision to exclude this from > >'ma

Re: [HACKERS] exposing pg_controldata and pg_config as functions

On Mon, Aug 31, 2015 at 9:47 PM, Peter Eisentraut wrote: > On 8/25/15 11:32 PM, Joe Conway wrote: >> 1.) pg_controldata() function and pg_controldata view added > > I don't think dumping out whatever pg_controldata happens to print as a > bunch of text fields is very sophisticated. We have functi

Re: [HACKERS] Improving replay of XLOG_BTREE_VACUUM records

25 авг. 2015 г., в 16:03, Michael Paquier написал(а):On Sun, Jul 26, 2015 at 9:46 PM, Andres Freund wrote:On 2015-07-24 09:53:49 +0300, Heikki Linnakangas wrote:To me it sounds like this shouldn't go through the full ReadBuffer()rigamarole. That code is already complex e

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

Hi, On 09/02/2015 08:49 PM, Greg Stark wrote: On 2 Sep 2015 14:54, "Andres Freund" wrote: + /*-- + * The user-visible GUC parameter is the number of drives (spindles), + * which we need to translate to a number-of-pages-to-prefetch target. + * The target value is

Re: [HACKERS] exposing pg_controldata and pg_config as functions

Robert Haas writes: > But I'm not sure I like the idea of adding a server dependency on the > ability to exec pg_controldata. That seems like it could be > unreliable at best, and a security vulnerability at worst. I hadn't been paying attention --- the proposed patch actually depends on exec'in

Re: [HACKERS] exposing pg_controldata and pg_config as functions

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/02/2015 05:25 PM, Tom Lane wrote: > Robert Haas writes: >> But I'm not sure I like the idea of adding a server dependency on >> the ability to exec pg_controldata. That seems like it could be >> unreliable at best, and a security vulnerability

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

On 09/02/2015 02:25 PM, Tomas Vondra wrote: > > As I explained, spindles have very little to do with it - you need > multiple I/O requests per device, to get the benefit. Sure, the DBAs > should know how many spindles they have and should be able to determine > optimal IO depth. But we actually sa

Re: [HACKERS] exposing pg_controldata and pg_config as functions

Tom Lane wrote: > Robert Haas writes: > > But I'm not sure I like the idea of adding a server dependency on the > > ability to exec pg_controldata. That seems like it could be > > unreliable at best, and a security vulnerability at worst. > > I hadn't been paying attention --- the proposed patch

Re: [HACKERS] Horizontal scalability/sharding

On 09/02/2015 12:30 PM, Robert Haas wrote: > On Wed, Sep 2, 2015 at 3:03 PM, Josh Berkus wrote: >>> 4. Therefore, I think that we should instead use logical replication, >>> which might be either synchronous or asynchronous. When you modify >>> one copy of the data, that change will then be repli

Re: [HACKERS] exposing pg_controldata and pg_config as functions

On 09/02/2015 02:34 PM, Alvaro Herrera wrote: > Tom Lane wrote: >> Robert Haas writes: >>> But I'm not sure I like the idea of adding a server dependency on the >>> ability to exec pg_controldata. That seems like it could be >>> unreliable at best, and a security vulnerability at worst. >> >> I h

Re: [HACKERS] Improving replay of XLOG_BTREE_VACUUM records

On Sun, Jul 26, 2015 at 1:46 PM, Andres Freund wrote: > To me it sounds like this shouldn't go through the full ReadBuffer() > rigamarole. That code is already complex enough, and here it's really > not needed. I think it'll be much easier to review - and actually faster > in many cases to simply

Re: [HACKERS] exposing pg_controldata and pg_config as functions

Josh Berkus wrote: > On 09/02/2015 02:34 PM, Alvaro Herrera wrote: > > I think trying to duplicate the exact strings isn't too nice an > > interface. > > Well, for pg_controldata, no, but what else would you do for pg_config? I was primarily looking at pg_controldata, so we agree there. As for

  1   2   >