Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

On Wed, Apr 13, 2016 at 11:24 AM, Etsuro Fujita wrote: > On 2016/04/13 3:14, Robert Haas wrote: >> >> I'm wondering why we are fixing this specific case and not any of the >> other calls to PQexec() or PQexecParams() in postgres_fdw.c. >> >> I mean, many of those

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On 2016-04-12 16:49:25 +, Kevin Grittner wrote: > On a big NUMA machine with 1000 connections in saturation load > there was a performance regression due to spinlock contention, for > acquiring values which were never used. Just fill with dummy > values if we're not going to use them. FWIW,

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 12:08 PM, Alvaro Herrera wrote: > Kevin Grittner wrote: >> Avoid extra locks in GetSnapshotData if old_snapshot_threshold < 0 >> >> On a big NUMA machine with 1000 connections in saturation load >> there was a performance regression due to

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

On Mon, Apr 11, 2016 at 9:45 PM, Michael Paquier wrote: > On Mon, Apr 11, 2016 at 5:16 PM, Etsuro Fujita > wrote: >> On 2016/04/11 12:30, Michael Paquier wrote: >>> >>> + if ((cancel = PQgetCancel(entry->conn))) >>> +

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 12 Apr 2016, at 17:23, Merlin Moncure wrote: > On Mon, Apr 11, 2016 at 11:39 AM, Justin Clift wrote: >> Moving over a conversation from the pgsql-advocacy mailing list. In it >> Simon (CC'd) raised the issue of potentially creating a >>

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Kevin Grittner wrote: > Avoid extra locks in GetSnapshotData if old_snapshot_threshold < 0 > > On a big NUMA machine with 1000 connections in saturation load > there was a performance regression due to spinlock contention, for > acquiring values which were never used. Just fill with dummy >

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Tue, Apr 12, 2016 at 9:36 AM, Craig Ringer wrote: > Repeating the mapping at each checkpoint sounds pretty reasonable and means > we always know what we need. There's no need to bloat each record with an > extension name and no need for any kind of ugly global

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

There are only 3 fundamental options I see, the cleaner can wait, "help", or move on. "Helping" is what it does now and is dangerous. Moving on gives the above-discussed unthrottling problem. Waiting has two problems. The act of waiting will cause autovacuums to be canceled, unless ugly hacks

Re: [HACKERS] Speedup twophase transactions

> On 12 Apr 2016, at 15:47, Michael Paquier wrote: > > On Mon, Apr 11, 2016 at 7:16 PM, Stas Kelvich wrote: >> Michael, it looks like that you are the only one person who can reproduce >> that bug. I’ve tried on bunch of OS’s and didn’t observe that behaviour, >>

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Mon, Apr 11, 2016 at 11:39 AM, Justin Clift wrote: > Moving over a conversation from the pgsql-advocacy mailing list. In it > Simon (CC'd) raised the issue of potentially creating a > backwards-compatibility > breaking release at some point in the future, to deal with

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 12:32 PM, Justin Clift wrote: > Yeah. Moving the discussion here was more to determine which items > really would need a backwards compatible break. eg no other approach can > be found. > > Seems I started it off badly, as no-one's yet jumped in to

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

This restricts the memory used by ordinary backends when doing the cleanup to be work_mem. Shouldn't we let them use maintenance_work_mem? Only one backend can be doing this clean up of a given index at any given time, so we don't need to worry about many concurrent allocations of

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Kevin Grittner writes: > On Tue, Apr 12, 2016 at 12:08 PM, Alvaro Herrera > wrote: >> old_snapshot_threshold is PGC_POSTMASTER, so this is okay AFAICS, but >> perhaps it'd be a good idea to add a oneline comment to guc.c indicating >> to verify this

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Kevin Grittner wrote: > FWIW, I spent a fair amount of time trying to make it PGC_SIGHUP, > since it would be very nice to allow that; but I kept running into > one problem after another with it, some of which were very hard to > see how to fix. My inclination is that trying to comment all the >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Hi, On 2016-04-12 14:17:12 -0500, Kevin Grittner wrote: > Well, something is different between your environment and mine, > since I saw no difference at scale 100 and 2.2% at scale 200. In a readonly test or r/w? A lot of this will be different between single-socket and multi-socket servers; as

Re: [HACKERS] Remove unused function from walsender.c

On Tue, Apr 12, 2016 at 6:36 AM, Simon Riggs wrote: > On 11 April 2016 at 08:05, Fujii Masao wrote: > >> >> There is an unused function GetOldestWALSendPointer() in walsender.c. >> Per comment, it was introduced because we may need it in the future

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

On 2016-04-12 14:29:10 -0400, Robert Haas wrote: > On Wed, Apr 6, 2016 at 6:57 AM, Andres Freund wrote: > > While benchmarking on hydra > > (c.f. > > http://archives.postgresql.org/message-id/20160406104352.5bn3ehkcsceja65c%40alap3.anarazel.de), > > which has quite slow IO, I

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 1:56 PM, Andres Freund wrote: > On 2016-04-12 13:44:00 -0500, Kevin Grittner wrote: >> On Tue, Apr 12, 2016 at 12:38 PM, Andres Freund wrote: >>> On 2016-04-12 16:49:25 +, Kevin Grittner wrote: On a big NUMA machine with

Re: [HACKERS] Updated backup APIs for non-exclusive backups

On Tue, Apr 12, 2016 at 8:39 AM, Noah Misch wrote: > On Mon, Apr 11, 2016 at 11:22:27AM +0200, Magnus Hagander wrote: > > On Thu, Apr 7, 2016 at 5:15 AM, Noah Misch wrote: > > > Unless you especially want to self-impose the same tight resolution > > >

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 9:25 PM, Josh berkus wrote: > On 04/12/2016 10:43 AM, Robert Haas wrote: > > 1. Large backward compatibility breaks are bad. Therefore, if any of > > these things are absolutely impossible to do without major > > compatibility breaks, we shouldn't do

Re: [HACKERS] pgsql: Add the "snapshot too old" feature

On 2016-04-12 13:57:26 -0500, Kevin Grittner wrote: > I have booked two large NUMA machines for Friday to look for any > remaining performance regressions on high-concurrency loads on such > machines. Anyone with ideas on particular tests that they feel > should be included, please let me know

Re: [HACKERS] Support for N synchronous standby servers - take 2

On Tue, Apr 12, 2016 at 9:04 AM, Masahiko Sawada wrote: > On Mon, Apr 11, 2016 at 8:47 PM, Fujii Masao wrote: >> On Mon, Apr 11, 2016 at 5:52 PM, Masahiko Sawada >> wrote: >>> On Mon, Apr 11, 2016 at 1:31 PM, Fujii Masao

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 2:28 PM, Andres Freund wrote: > On 2016-04-12 14:17:12 -0500, Kevin Grittner wrote: >> Well, something is different between your environment and mine, >> since I saw no difference at scale 100 and 2.2% at scale 200. > > In a readonly test or r/w?

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 04/12/2016 10:43 AM, Robert Haas wrote: > 1. Large backward compatibility breaks are bad. Therefore, if any of > these things are absolutely impossible to do without major > compatibility breaks, we shouldn't do them at all. +1 > 2. Small backward compatibility breaks are OK, but don't

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

On Wed, Apr 6, 2016 at 6:57 AM, Andres Freund wrote: > While benchmarking on hydra > (c.f. > http://archives.postgresql.org/message-id/20160406104352.5bn3ehkcsceja65c%40alap3.anarazel.de), > which has quite slow IO, I was once more annoyed by how incredibly long > the vacuum

Re: [HACKERS] WIP: Upper planner pathification

Robert Haas writes: > On Thu, Mar 17, 2016 at 2:31 PM, Tom Lane wrote: >> So what I would now propose is >> >> typedef void (*create_upper_paths_hook_type) (PlannerInfo *root, >> UpperRelationKind stage, >>

[HACKERS] Problems with huge_pages and IBM Power8

Hi We have been doing some testing with Postgres (9.5.2) compiled on a Power8 running Centos 7 When working with huge_pages, we initially got this error. munmap(0x3efbe400) failed: Invalid argument after a bit of investigation we noticed that hugepagesize is har coded to 2MB

Re: [HACKERS] pgsql: Add the "snapshot too old" feature

On Mon, Apr 11, 2016 at 12:31 PM, Kevin Grittner wrote: > On Mon, Apr 11, 2016 at 8:20 AM, Alexander Korotkov >> So, for read-only benchmark this patch introduces more than 5 times >> regression on big machine. > > I did not schedule a saturation test on a big machine. I

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

On 2016-04-12 07:00, Andres Freund wrote: On 2016-04-12 00:32:13 -0400, Tom Lane wrote: I wrote: This commit has broken buildfarm member gaur, and no doubt pademelon will be equally unhappy once it catches up to HEAD. Spoke too soon, actually: pademelon did not get as far as initdb. cc:

Re: [HACKERS] raw output from copy

2016-04-12 12:22 GMT+02:00 Ants Aasma : > On 8 Apr 2016 9:14 pm, "Pavel Stehule" wrote: > > 2016-04-08 20:54 GMT+02:00 Andrew Dunstan : > >> I should add that I've been thinking about this some more, and that I > now agree that

Re: [HACKERS] Problems with huge_pages and IBM Power8

On 2016-04-12 21:58:14 +0200, reiner peterke wrote: > Hi > > We have been doing some testing with Postgres (9.5.2) compiled on a Power8 > running Centos 7 > > When working with huge_pages, we initially got this error. > > munmap(0x3efbe400) failed: Invalid argument *munmap*, not mmap

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 04/12/2016 01:07 PM, Oleg Bartunov wrote: > > Our roadmap http://www.postgresql.org/developer/roadmap/ is the problem. > We don't have clear roadmap and that's why we cannot plan future feature > full release. As someone who's worked at multiple proprietary software companies, having a

Re: [HACKERS] Parallel Aggregate costs don't consider combine/serial/deserial funcs

On Sun, Apr 10, 2016 at 8:47 AM, David Rowley wrote: > I realised a few days ago that the parallel aggregate code does not > cost for the combine, serialisation and deserialisation functions at > all. Oops. > I've attached a patch which fixes this. I've committed

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 2016-04-12 11:25:21 -0700, Josh berkus wrote: > Here's the features I can imagine being worth major backwards > compatibility breaks: > > 1. Fully pluggable storage with a clean API. > > 2. Total elimination of VACUUM or XID freezing > > 3. Fully transparent-to-the user MM

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On 2016-04-12 13:44:00 -0500, Kevin Grittner wrote: > On Tue, Apr 12, 2016 at 12:38 PM, Andres Freund wrote: > > On 2016-04-12 16:49:25 +, Kevin Grittner wrote: > >> On a big NUMA machine with 1000 connections in saturation load > >> there was a performance regression due

Re: [HACKERS] Problems with huge_pages and IBM Power8

Andres Freund writes: > On 2016-04-12 21:58:14 +0200, reiner peterke wrote: >> Looking for some insight into this issue. the error from the postgres >> log on ubuntu is below. It apperas to be related to semephores. > I've a bit of a hard time believing that this is related

Re: [HACKERS] Problems with huge_pages and IBM Power8

> On Apr 12, 2016, at 10:20 PM, Andres Freund wrote: > > On 2016-04-12 21:58:14 +0200, reiner peterke wrote: >> Hi >> >> We have been doing some testing with Postgres (9.5.2) compiled on a Power8 >> running Centos 7 >> >> When working with huge_pages, we initially got

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 2:27 PM, Andres Freund wrote: > none but 2) seem likely to require a substantial compatibility break. And even that doesn't require one, if you keep the only system around and make the new system optional via some sort of pluggable storage API. Which,

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 12:38 PM, Andres Freund wrote: > On 2016-04-12 16:49:25 +, Kevin Grittner wrote: >> On a big NUMA machine with 1000 connections in saturation load >> there was a performance regression due to spinlock contention, for >> acquiring values which were

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 2:53 PM, Kevin Grittner wrote: > Readonly with client and job counts matching scale. Single-socket i7, BTW. >> A lot of this will be different between >> single-socket and multi-socket servers; as soon as you have the latter >> the likelihood of

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 1:07 PM, Oleg Bartunov wrote: > > >> * we don't *know* that any of the above items will require a backwards >> compatibility break. >> >> People keep talking about "we might want to break compatibility/file >> format one day". But nobody is working

Re: [HACKERS] Problems with huge_pages and IBM Power8

reiner peterke writes: >> On Apr 12, 2016, at 10:20 PM, Andres Freund wrote: >> I've a bit of a hard time believing that this is related to huge pages. > Well all i have at the moment is that when we disabled huge pages on the > kernel level and then

Re: [HACKERS] raw output from copy

Pavel Stehule writes: > I had a idea about additional options of COPY RAW statements. One can be > CAST function. These CAST functions can be used to any for any format. Uh, what? CAST() is not about external representations of values, and overloading it for that

Re: [HACKERS] fd.c: flush data problems on osx

Michael Paquier writes: > On Mon, Mar 21, 2016 at 9:09 PM, Stas Kelvich > wrote: >> On 21 Mar 2016, at 14:53, Andres Freund wrote: >>> I think we still need to fix the mmap() implementation to support the >>> offset = 0,

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 4:07 PM, Oleg Bartunov wrote: > Our roadmap http://www.postgresql.org/developer/roadmap/ is the problem. We > don't have clear roadmap and that's why we cannot plan future feature full > release. There are several postgres-centric companies, which have

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 4:32 PM, David G. Johnston wrote: > I give a solid +10 to Robert's opinions on the matter and aside from > figuring out if and how to fit first-number versioning dynamics into our > release policies I think the community is doing a sufficient

Re: [HACKERS] Parallel Aggregate costs don't consider combine/serial/deserial funcs

On 13 April 2016 at 08:52, Robert Haas wrote: > On Sun, Apr 10, 2016 at 8:47 AM, David Rowley > wrote: >> I realised a few days ago that the parallel aggregate code does not >> cost for the combine, serialisation and deserialisation functions

[HACKERS] Pglogical questions and problems

Hello, Alright -- here we go PostgreSQL 9.5.2, Ubuntu Trusty, All packages from apt.postgresql.org except PgLogical which is from 2Q: I have the following setup: Origin0->Replica0 Table: logical_test(id bigserial primary key) Origin0: SELECT pglogical.create_node( node_name := 'origin',

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

Robert, Andres, * Andres Freund (and...@anarazel.de) wrote: > On 2016-04-12 14:29:10 -0400, Robert Haas wrote: > > On Wed, Apr 6, 2016 at 6:57 AM, Andres Freund wrote: > > That does not seem like a good idea from here. One of the ideas I > > still want to explore at some

Re: [HACKERS] Choosing parallel_degree

On 11/04/2016 22:53, Julien Rouhaud wrote: > On 11/04/2016 17:44, Robert Haas wrote: >> >> We should probably add the number of workers actually obtained to the >> EXPLAIN ANALYZE output. That's been requested before. >> > > If it's not too late for 9.6, it would be very great. > Just in case

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 2:04 PM, Robert Haas wrote: > On Tue, Apr 12, 2016 at 4:32 PM, David G. Johnston > wrote: > > I give a solid +10 to Robert's opinions on the matter and aside from > > figuring out if and how to fit first-number

Re: [HACKERS] Performance degradation in commit 6150a1b0

On Thu, Mar 31, 2016 at 10:13 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Mar 31, 2016 at 6:45 AM, Andres Freund wrote: >>> On 2016-03-31 06:43:19 -0400, Robert Haas wrote: To which proposal are you referring? > >>> 1)

Re: [HACKERS] Speedup twophase transactions

On Wed, Apr 13, 2016 at 1:53 AM, Stas Kelvich wrote: >> On 12 Apr 2016, at 15:47, Michael Paquier wrote: >> >> It looks to be the case... The PREPARE phase replayed after the >> standby is restarted in recovery creates a series of exclusive

Re: [HACKERS] Updated backup APIs for non-exclusive backups

On Tue, Apr 12, 2016 at 10:08:23PM +0200, Magnus Hagander wrote: > On Tue, Apr 12, 2016 at 8:39 AM, Noah Misch wrote: > > On Mon, Apr 11, 2016 at 11:22:27AM +0200, Magnus Hagander wrote: > > > Well, if we *don't* do the rewrite before we release it, then we have to > > >

Re: [HACKERS] Parallel Queries and PostGIS

On 1 April 2016 at 17:12, David Rowley wrote: > On 30 March 2016 at 09:14, Robert Haas wrote: >> On Tue, Mar 29, 2016 at 3:48 PM, Paul Ramsey >> wrote: I have no idea why the cost adjustments that you need are

Re: [HACKERS] Performance degradation in commit 6150a1b0

On Tue, Apr 12, 2016 at 10:30 PM, Noah Misch wrote: > That sounds like this open item is ready for CLOSE_WAIT status; is it? I just retested this on power2. Here are the results. I retested 3fed4174 and 6150a1b0 plus master as of deb71fa9. 5-minute pgbench -S runs, scale

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On 2016-04-12 23:52:14 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > I'm kinda inclined to apply that portion (or just the whole patch with > > the spurious #ifdef 0 et al fixed) into 9.6; and add the necessary > > checks in a few places. Because I really think this is likely to hit >

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

Magnus Hagander writes: > On Mon, Jan 11, 2016 at 6:19 AM, Amit Kapila > wrote: >> On Sun, Jan 10, 2016 at 11:55 PM, Tom Lane wrote: >>> I think the reason why we're getting "No error" instead of a useful >>> strerror report is

Re: [HACKERS] Performance degradation in commit 6150a1b0

On Tue, Apr 12, 2016 at 05:36:07PM -0400, Robert Haas wrote: > So the current status of this issue is: > > 1. Andres committed a patch (008608b9d51061b1f598c197477b3dc7be9c4a64) > to reduce the size of an LWLock by an amount equal to the size of a > mutex (modulo alignment). > > 2. Andres also

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Andres Freund wrote: > I'm kinda inclined to apply that portion (or just the whole patch with > the spurious #ifdef 0 et al fixed) into 9.6; and add the necessary > checks in a few places. Because I really think this is likely to hit > unsuspecting users. !!! Be sure to consult with the RMT

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

On 2016/04/13 3:14, Robert Haas wrote: I'm wondering why we are fixing this specific case and not any of the other calls to PQexec() or PQexecParams() in postgres_fdw.c. I mean, many of those instances are cases where the query isn't likely to run for very long, but certainly "FETCH %d FROM

Re: [HACKERS] raw output from copy

2016-04-12 22:48 GMT+02:00 Tom Lane : > Pavel Stehule writes: > > I had a idea about additional options of COPY RAW statements. One can be > > CAST function. These CAST functions can be used to any for any format. > > Uh, what? CAST() is not about

Re: [HACKERS] Parser extensions (maybe for 10?)

On 12 April 2016 at 13:51, Tom Lane wrote: > Craig Ringer writes: > > The other area where there's room for extension without throwing out the > > whole thing and rebuilding is handling of new top-level statements. We > can > > probably dispatch the

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

On Mon, Apr 11, 2016 at 7:33 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Sun, Apr 10, 2016 at 2:24 PM, Amit Kapila > wrote: >> >> I also tried to run perf top during pgbench and get some interesting >>> results. >>> >>> Without 5364b357: >>>5,69%

Re: [HACKERS] Updated backup APIs for non-exclusive backups

On Mon, Apr 11, 2016 at 11:22:27AM +0200, Magnus Hagander wrote: > On Thu, Apr 7, 2016 at 5:15 AM, Noah Misch wrote: > > Unless you especially want to self-impose the same tight resolution > > schedule > > that 9.6 regressions experience, let's move this to the "Non-bugs"

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

On Thu, Apr 07, 2016 at 05:53:54PM -0700, Jeff Janes wrote: > On Thu, Apr 7, 2016 at 4:33 PM, Tom Lane wrote: > > Jeff Janes writes: > >> To summarize the behavior change: > > > >> In the released code, an inserting backend that violates the pending > >>

Re: [HACKERS] Choosing parallel_degree

On Tue, Apr 12, 2016 at 3:15 AM, Simon Riggs wrote: > On 8 April 2016 at 17:49, Robert Haas wrote: > > >> With the patch, you can - if you wish - substitute >> some other number for the one the planner comes up with. > > > I saw you're using

[HACKERS] Re: Parallel Aggregate costs don't consider combine/serial/deserial funcs

On Mon, Apr 11, 2016 at 12:47:29AM +1200, David Rowley wrote: > Hi, > > I realised a few days ago that the parallel aggregate code does not > cost for the combine, serialisation and deserialisation functions at > all. > > I've attached a patch which fixes this. > > One small point which I was a

Re: [HACKERS] Preprocessor condition fix

* Tom Lane wrote: Christian Ullrich writes: According to git grep, this is the only place where WIN64 is used without the leading underscore. Hm, my grep found another one ... Oh, sorry. I saw that one, but thought it was intentional because _WIN64 is defined

[HACKERS] Update copyright in genericdesc.c

Hi, Attached fixes copyright in file mentioned in $subject. Thanks, Amit diff --git a/src/backend/access/rmgrdesc/genericdesc.c b/src/backend/access/rmgrdesc/genericdesc.c index caa9a03..0796bb8 100644 --- a/src/backend/access/rmgrdesc/genericdesc.c +++

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

On Tue, Apr 12, 2016 at 12:40 AM, Andres Freund wrote: > I did get access to the machine (thanks!). My testing shows that > performance is sensitive to various parameters influencing memory > allocation. E.g. twiddling with max_connections changes > performance. With

Re: [HACKERS] raw output from copy

On 8 Apr 2016 9:14 pm, "Pavel Stehule" wrote: > 2016-04-08 20:54 GMT+02:00 Andrew Dunstan : >> I should add that I've been thinking about this some more, and that I now >> agree that something should be done to support this at the SQL level, mainly

Re: [HACKERS] Choosing parallel_degree

On 04/11/2016 09:14 PM, Robert Haas wrote: postgres=# explain analyze verbose select * from abd where n<=1; >>ERROR: requested shared memory size overflows size_t >> >>if we remove the analyze keyword then query running successfully. >> >>Expected = Is it not better to throw the error at the

Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

Hi 2016-02-13 15:54 GMT+01:00 Tom Lane : > Simon Riggs writes: > > On 10 February 2016 at 16:36, Tom Lane wrote: > >> FWIW, I wasn't paying attention either, but I'm convinced by Robert's > >> argument. Avoiding coupling between

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

Hello Andres, I can think of a number of relatively easy ways to address this: 1) Just zap (or issue?) all pending flush requests when getting an smgrinval/smgrclosenode 2) Do 1), but filter for the closed relnode 3) Actually handle the case of the last open segment not being RELSEG_SIZE

Re: [HACKERS] Choosing parallel_degree

On 12 April 2016 at 07:58, Amit Kapila wrote: > On Tue, Apr 12, 2016 at 3:15 AM, Simon Riggs > wrote: > >> On 8 April 2016 at 17:49, Robert Haas wrote: >> >> >>> With the patch, you can - if you wish - substitute >>> some

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Mon, Apr 11, 2016 at 9:54 PM, Craig Ringer wrote: > If you rely on shared_preload_libraries, then "oops, I forgot to put > myextension in shared_preload_libraries on the downstream" becomes a Bad > Thing. There's no way to go back and redo the work you've passed over.

Re: [HACKERS] Choosing parallel_degree

On Mon, Apr 11, 2016 at 5:45 PM, Simon Riggs wrote: > On 8 April 2016 at 17:49, Robert Haas wrote: >> With the patch, you can - if you wish - substitute >> some other number for the one the planner comes up with. > > I saw you're using

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

On Mon, Apr 11, 2016 at 1:17 PM, Andres Freund wrote: > I can think of a number of relatively easy ways to address this: > 1) Just zap (or issue?) all pending flush requests when getting an >smgrinval/smgrclosenode > 2) Do 1), but filter for the closed relnode > 3)

Re: [HACKERS] Choosing parallel_degree

On 04/11/2016 08:57 PM, Julien Rouhaud wrote: >Expected = Expecting worker8 information , also loops=10 (including the >Master) > Even if you set a per-table parallel_degree higher than max_parallel_degree, it'll be maxed at max_parallel_degree. Then, the explain shows that the planner assumed

Re: [HACKERS] Update copyright in genericdesc.c

* Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > Attached fixes copyright in file mentioned in $subject. Fixed. Thanks! Stephen signature.asc Description: Digital signature

Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

2016-04-12 14:51 GMT+02:00 Robert Haas : > On Tue, Apr 12, 2016 at 4:18 AM, Pavel Stehule > wrote: > > If there will be simple way, how to fix it, then I'll fix my extensions. > But > > new API is working only when the extension has own share

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

Justin Clift wrote: Simon included a short starter list of potentials which might be in that category: * SQL compliant identifiers * Remove RULEs * Change recovery.conf * Change block headers * Retire template0, template1 * Optimise FSM * Add heap metapage * Alter tuple headers

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Tue, Apr 12, 2016 at 3:08 AM, Michael Paquier wrote: > What I thought we should be able to do with that should not be only > limited to indexes, aka to: > - Be able to register and log full pages > - Be able to log data associated to a block > - Be able to have

Re: [HACKERS] Speedup twophase transactions

On Mon, Apr 11, 2016 at 7:16 PM, Stas Kelvich wrote: > Michael, it looks like that you are the only one person who can reproduce > that bug. I’ve tried on bunch of OS’s and didn’t observe that behaviour, also > looking at your backtraces I can’t get who is holding this lock (and all of > that

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Sun, Apr 10, 2016 at 7:49 AM, Tom Lane wrote: > 1. It doesn't seem like generic_xlog.c has thought very carefully about > the semantics of the "hole" between pd_lower and pd_upper. The mainline > XLOG code goes to some lengths to ensure that the hole stays all-zeroes; >

Re: [HACKERS] Choosing parallel_degree

On 12 April 2016 at 13:53, Robert Haas wrote: > On Mon, Apr 11, 2016 at 5:45 PM, Simon Riggs > wrote: > > On 8 April 2016 at 17:49, Robert Haas wrote: > >> With the patch, you can - if you wish - substitute > >> some other

Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

On Tue, Apr 12, 2016 at 4:18 AM, Pavel Stehule wrote: > If there will be simple way, how to fix it, then I'll fix my extensions. But > new API is working only when the extension has own share memory segment. For > some complex extensions like Orafce it means expensive

Re: [HACKERS] Preprocessor condition fix

Christian Ullrich writes: > * Tom Lane wrote: >> Hm, my grep found another one ... > Oh, sorry. I saw that one, but thought it was intentional because _WIN64 > is defined automatically anyway. Oh? Then we should not need that one (the /D switch in win32.mak) at all.

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Tue, Apr 12, 2016 at 3:33 AM, Tom Lane wrote: > ... BTW, with respect to the documentation angle, it seems to me > that it'd be better if GenericXLogRegister were renamed to > GenericXLogRegisterBuffer, or perhaps GenericXLogRegisterPage. > I think this would make the

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 12 Apr 2016, at 14:12, Yury Zhuravlev wrote: > Justin Clift wrote: >> Simon included a short starter list of potentials which might be in >> that category: >> >> * SQL compliant identifiers >> * Remove RULEs >> * Change recovery.conf >> * Change block headers

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

On Tue, Apr 12, 2016 at 3:48 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Tue, Apr 12, 2016 at 12:40 AM, Andres Freund > wrote: > >> I did get access to the machine (thanks!). My testing shows that >> performance is sensitive to various parameters

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On 12 April 2016 at 20:48, Robert Haas wrote: > > So how do we look at a generic log record, say "ok, the upstream that > wrote > > this says it needs to invoke registered generic xlog hook 42, which is > > from at " ? > > Record enough information in WAL that the rmgrs

Re: [HACKERS] Preprocessor condition fix

* From: Tom Lane [mailto:t...@sss.pgh.pa.us] > Christian Ullrich writes: > > * Tom Lane wrote: > >> Hm, my grep found another one ... > > > Oh, sorry. I saw that one, but thought it was intentional because _WIN64 > > is defined automatically anyway. > > Oh? Then we

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

Andres Freund writes: > On 2016-04-12 00:32:13 -0400, Tom Lane wrote: >> Apparently, assigning the result of init_spin_delay() is not as portable >> as you thought. > Hm. I'm not sure why not though? Is it because delayStatus isn't static > or global scope? It looks like

Re: [HACKERS] Preprocessor condition fix

Christian Ullrich writes: > * Tom Lane wrote: >> Oh? Then we should not need that one (the /D switch in win32.mak) at all. >> Should we just remove it? > We have both confirmed several times that nothing depends on it. I think it > can go. Done.

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

Alexander Korotkov writes: > Attached patch is intended to fix this. It zeroes "hole" in both > GenericXLogFinish() and generic_redo(). Pushed. I rewrote the comments a bit and modified GenericXLogFinish so that it doesn't copy data only to overwrite it with zeroes.

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

Alexander Korotkov writes: > I agree with both of these ideas. Patch is attached. Pushed with minor cleanup. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

I agree with both of these ideas. Patch is attached. Hmm, you accidentally forget to change flag type of GenericXLogRegister in contrib/bloom signature. -- Teodor Sigaev E-mail: teo...@sigaev.ru WWW:

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Tue, Apr 12, 2016 at 6:34 PM, Teodor Sigaev wrote: > I agree with both of these ideas. Patch is attached. >> > > Hmm, you accidentally forget to change flag type of GenericXLogRegister in > contrib/bloom signature. Fixed. -- Alexander Korotkov Postgres Professional:

  1   2   >