Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

On 2017-04-06 14:57:56 -0700, Peter Geoghegan wrote: > On Thu, Apr 6, 2017 at 2:50 PM, Andres Freund <and...@anarazel.de> wrote: > > Pushed with very minor wording changes. > > This had a typo: > > + * If copy is true, the slot receives a copied tuple that'll that

Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

Hi, On 2017-05-04 18:24:47 -0700, Andres Freund wrote: > Hi, > > On 2016-12-22 19:33:30 +0000, Andres Freund wrote: > > Skip checkpoints, archiving on idle systems. > > As part of an independent bugfix I noticed that Michael & I appear to > have introduced an off-by-

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

Hi, On 2017-05-05 15:29:40 -0400, Robert Haas wrote: > On Thu, May 4, 2017 at 9:37 PM, Andres Freund <and...@anarazel.de> wrote: > It's pretty easy (but IMHO not very interesting) to measure internal > contention in the Parallel Seq Scan node. As David points out > downth

Re: [HACKERS] snapbuild woes

On 2017-05-05 13:53:16 +0200, Petr Jelinek wrote: > On 05/05/17 02:42, Andres Freund wrote: > > On 2017-05-04 17:00:04 -0700, Andres Freund wrote: > >> Attached is a prototype patch for that. > > > > I am not sure I understand the ABI comment for started_collect

Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

Hi, On 2017-05-05 11:50:12 +0530, Amit Kapila wrote: > I see that EndPos can be updated in one of the cases after releasing > the lock (refer below code). Won't that matter? I can't see how it'd in the cases that'd matter for GetLastImportantRecPtr() - but it'd probably good to note it in the

Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

On 2017-05-05 11:04:14 +0530, Amit Kapila wrote: > On Fri, May 5, 2017 at 6:54 AM, Andres Freund <and...@anarazel.de> wrote: > > Hi, > > > > On 2016-12-22 19:33:30 +, Andres Freund wrote: > >> Skip checkpoints, archiving on idle systems. > > >

Re: [HACKERS] CTE inlining

enefit of that? With FDWs there's not even a common set of things pushed down between different FDW users. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

On 2017-05-04 19:45:33 -0700, Andres Freund wrote: > Increment phs_cblock without checking rs_nblocks, but outside of the > lock do a % scan->rs_nblocks, to get the "actual" position. Finish if > (phs_cblock - phs_startblock) / scan->rs_nblocks >= 1. Err, as I've bee

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

On 2017-05-05 14:40:43 +1200, David Rowley wrote: > On 5 May 2017 at 14:36, Andres Freund <and...@anarazel.de> wrote: > > I wonder how much doing the atomic ops approach alone can help, that > > doesn't have the issue that the work might be unevenly distributed > > bet

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

Hi, On 2017-05-05 14:20:48 +1200, David Rowley wrote: > Yeah, I did get some time to look over the contention in Parallel Seq > Scan a while back and I discovered that on the machine that I was > testing on. the lock obtained in heap_parallelscan_nextpage() was > causing workers to have to wait

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

lem, but without trying to figure them out, we're going to have a hard time to develop a better costing model... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

On 2017-05-04 18:23:38 -0700, Peter Geoghegan wrote: > On Thu, May 4, 2017 at 6:08 PM, Robert Haas wrote: > >> E.g. to power amazon's data migration service (yes, that scares me). > > > > If you recall, I did predict prior to commit that test_decoding would > > get put into

[HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

Hi, On 2016-12-22 19:33:30 +, Andres Freund wrote: > Skip checkpoints, archiving on idle systems. As part of an independent bugfix I noticed that Michael & I appear to have introduced an off-by-one here. A few locations do comparisons like: /* * Only log if

Re: [HACKERS] PG 10 release notes

On 2017-05-04 21:08:41 -0400, Robert Haas wrote: > On Thu, May 4, 2017 at 8:09 PM, Andres Freund <and...@anarazel.de> wrote: > > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > >> Even test_decoding is (perhaps regrettably) being used to build production >

Re: [HACKERS] PG 10 release notes

On 2017-05-04 20:46:24 -0400, Bruce Momjian wrote: > On Thu, May 4, 2017 at 05:09:40PM -0700, Andres Freund wrote: > > > > I would not in any way refer to logical decoding as being only a proof > > > > of concept, even before logical replication. > > >

Re: [HACKERS] snapbuild woes

On 2017-05-04 17:00:04 -0700, Andres Freund wrote: > Attached is a prototype patch for that. Oops. Andres >From b6eb46e376e40f3e2e9a55d16b1b37b27904564b Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Thu, 4 May 2017 16:40:52 -0700 Subject: [PATCH 1/2] WIP: Fi

Re: [HACKERS] PG 10 release notes

On 2017-05-04 19:56:21 -0400, Bruce Momjian wrote: > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > This came up from discussion related to this item: > > the ability of logical decoding to follow timeline switches > > My point was that based on the text it is test_decoding

Re: [HACKERS] PG 10 release notes

On 2017-05-04 19:56:21 -0400, Bruce Momjian wrote: > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian wrote: > > >> Or the ability of logical decoding to follow timeline switches. > > > > > > I didn't think logical

Re: [HACKERS] snapbuild woes

Hi, On 2017-05-02 08:55:53 +0200, Petr Jelinek wrote: > Aah, now I understand we talked about slightly different things, I > considered the running thing to be first step towards tracking aborted > txes everywhere. > I think > we'll have to revisit tracking of aborted transactions in PG11 then >

Re: [HACKERS] PG 10 release notes

On 2017-05-04 17:33:13 -0500, Merlin Moncure wrote: > On Mon, May 1, 2017 at 7:02 AM, Robert Haas wrote: > > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian wrote: > >> I didn't think logical decoding was really more than a proof-of-concept > >> until

Re: [HACKERS] PG 10 release notes

On 2017-04-25 15:29:01 -0400, Bruce Momjian wrote: > Uh, the only logical decoding code that I know we ship pre-PG 10 is > contrib/test_decoding/. That's completely wrong. src/backend/replication/logical/ is a a bit bigger than that... - Andres -- Sent via pgsql-hackers mailing list

Re: [HACKERS] CTE inlining

Set node, and handle the column accesses via the nromal projection. > ​Except I suspect we at least have a chance to detect the above and not > de-optimize it by evaluating "json_populate_record" once for every column > in mytype. That's what happens with the non CTE version - don't

Re: [HACKERS] PROVE_FLAGS

On 2017-05-03 15:14:27 -0400, Andrew Dunstan wrote: > > Can someone please explain to me why we have this in Makefile.global.in? > (from commit e9c81b60 ) > > > PROVE_FLAGS = > > > ISTM it's unnecessary, and prevents us from using the same named value > in the environment. I want to be

[HACKERS] Re: Potential hot-standby bug around xacts committed but in xl_running_xacts

On 2017-05-02 07:12:41 +0200, Simon Riggs wrote: > /* > * The running-xacts snapshot can contain xids that were still visible > * in the procarray when the snapshot was taken, but were already > * WAL-logged as completed. They're not running anymore, so ignore > * them. > */ > if

Re: [HACKERS] Logical replication in the same cluster

On 2017-05-02 09:17:27 +0200, Petr Jelinek wrote: > Yes because otherwise we risk leaving slot on the upstream if the > command fails downstream. Shouldn't temporary slots be able to solve that concern? Create it as temporary at the beginning, mark it as permanent at the end? Greetings,

[HACKERS] Potential hot-standby bug around xacts committed but in xl_running_xacts

ime, fixed by ExpireOldKnownAssignedTransactionIds(). Am I missing something that protects against the above scenario? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] snapbuild woes

On 2017-05-01 11:09:44 +0200, Petr Jelinek wrote: > On 01/05/17 10:03, Andres Freund wrote: > > On 2017-05-01 03:54:49 +0200, Petr Jelinek wrote: > >> But, I still think we need to restart the tracking after new > >> xl_running_xacts. Reason for that is afaics a

Re: [HACKERS] Logical replication in the same cluster

On 2017-05-01 11:31:53 -0700, Peter Geoghegan wrote: > On Mon, May 1, 2017 at 11:24 AM, Andres Freund <and...@anarazel.de> wrote: > > Doing catalog changes in recovery is frought with problems. Essentially > > requires starting one worker per database, before allowing acce

Re: [HACKERS] Logical replication in the same cluster

On 2017-05-01 11:22:47 -0700, Peter Geoghegan wrote: > On Fri, Apr 28, 2017 at 9:28 AM, Robert Haas wrote: > > On Thu, Apr 27, 2017 at 4:08 AM, Petr Jelinek > > wrote: > >> Back when writing the original patch set, I was also playing with the

Re: [HACKERS] snapbuild woes

On 2017-05-01 12:32:07 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-05-01 08:46:47 -0400, Tom Lane wrote: > >> 30sec is kind of a big lump from a buildfarm standpoint, especially if > >> you mean "it runs for 30s on m

Re: [HACKERS] snapbuild woes

On 2017-05-01 08:46:47 -0400, Tom Lane wrote: > Noah Misch <n...@leadboat.com> writes: > > On Fri, Apr 21, 2017 at 10:34:58PM -0700, Andres Freund wrote: > >> ... I was wondering about adding > >> a loop that simply runs for like 30s and then

Re: [HACKERS] snapbuild woes

On 2017-05-01 03:54:49 +0200, Petr Jelinek wrote: > I agree with adding running, I think that's good thing even for the per > transaction tracking and snapshot exports - we could use the newly added > field to get rid of the issue we have with 'snapshot too large' when > there were many aborted

Re: [HACKERS] snapbuild woes

On 2017-04-30 17:59:21 -0700, Andres Freund wrote: > ISTM, we need a xip_status array in SnapBuild->running. Then, whenever > a xl_running_xacts is encountered while builder->running.xcnt > 0, loop > for i in 0 .. xcnt_space, and end SnapBuildEndTxn() if xip_statu

Re: [HACKERS] Logical replication in the same cluster

On 2017-04-26 23:41:51 +0200, Petr Jelinek wrote: > Yes that's result of how logical replication slots work, the transaction > that needs to finish is your transaction. It can be worked around by > creating the slot manually via the SQL interface for example and create > the subscription using

Re: [HACKERS] snapbuild woes

Hi, On 2017-04-15 05:18:49 +0200, Petr Jelinek wrote: > + /* > + * c) we already seen the xl_running_xacts and tried to do the above. > + * However because of race condition in LogStandbySnapshot() there might > + * have been transaction reported as running but in reality has

Re: [HACKERS] CTE inlining

On 2017-04-30 07:19:21 +0200, Pavel Stehule wrote: > why we cannot to introduce GUC option - enable_cteoptfence ? Doesn't really solve the issue, and we've generally shied away from GUCs that influence behaviour after a few bad experiences. What if you want one CTE inlined, but another one not?

Re: [HACKERS] CTE inlining

'll surprise people forever, whereas using something as an intentional barrier usually is, well, intentional. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] CTE inlining

> SELECT target list I really think it's just something that needs to be > done. With help from Tom, luckily... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] CTE inlining

On 2017-04-30 00:28:46 -0400, Tom Lane wrote: > There's already a pretty large hill to climb here in the way of > breaking peoples' expectations about CTEs being optimization > fences. Breaking the documented semantics about CTEs being > single-evaluation seems to me to be an absolute

Re: [HACKERS] snapbuild woes

On April 27, 2017 9:34:44 PM PDT, Noah Misch <n...@leadboat.com> wrote: >On Fri, Apr 21, 2017 at 10:36:21PM -0700, Andres Freund wrote: >> On 2017-04-17 21:16:57 -0700, Andres Freund wrote: >> > I've since the previous update reviewed Petr's patch, which he >

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

On 2017-04-27 17:22:25 -0400, Tom Lane wrote: > > Yep, and I've seen that turn into a serious problem in production. > > How so? Shouldn't the indexscan go back and mark such tuples dead in > the index, such that they'd be visited this way only once? If that's > not happening, maybe we should

Re: [HACKERS] Unportable implementation of background worker start

On 2017-04-27 16:35:29 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-26 17:05:39 -0400, Tom Lane wrote: > >> I went ahead and changed the call to epoll_create into epoll_create1. > >> I'm not too concerned about loss

Re: [HACKERS] frogmouth failures

On 2017-04-27 16:30:35 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > I've been trying to track down the cause of recent failures at the "make > > check" stage on frogmouth, a 32-bit Windows/Mingw instance running on XP. > > I've been wondering about that

Re: [HACKERS] Unportable implementation of background worker start

oo. Cool. Change looks good to me. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Unportable implementation of background worker start

On 2017-04-26 11:42:38 -0400, Tom Lane wrote: > 1. Let HEAD stand as it is. We have a problem with slow response to > bgworker start requests that arrive while ServerLoop is active, but that's > a pretty tight window usually (although I believe I've seen it hit at > least once in testing). > >

Re: [HACKERS] PG 10 release notes

On 2017-04-25 22:13:00 -0400, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote: > > On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote: > > > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote: > > > > Or the ability

Re: [HACKERS] PG 10 release notes

has advantages on its own, it's imo more of a stepping stone. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Separation walsender & normal backends

On 2017-04-26 08:41:46 +0800, Craig Ringer wrote: > I'd very much like to reduce the amount of magic global juggling done > by the walsender, unify the XLogRead paths, unify the timeline > following logic for physical walsenders, logical walsenders and > logical decoding on normal backends, allow

Re: [HACKERS] Cached plans and statement generalization

earch_path. Consider e.g. cases like CREATE SCHEMA a; CREATE SCHEMA b; CREATE TABLE a.foobar(somecol int); SET search_patch = 'b,a'; SELECT * FROM foobar; CREATE TABLE b.foobar(anothercol int); SELECT * FROM foobar; -- may not be cached plan from before! it sounds - my memory of DB2 is very faint, and

Re: [HACKERS] Separation walsender & normal backends

On 2017-04-25 23:24:40 +0200, Petr Jelinek wrote: > On 25/04/17 17:13, Fujii Masao wrote: > > On Tue, Apr 25, 2017 at 11:34 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> Andres Freund <and...@anarazel.de> writes: > >>> I've for a while s

Re: [HACKERS] Cached plans and statement generalization

On 2017-04-25 21:11:08 +, Doug Doole wrote: > When I did this in DB2, I didn't use the parser - it was too expensive. I > just tokenized the statement and used some simple rules to bypass the > invalid cases. For example, if I saw the tokens "ORDER" and "BY" then I'd > disallow replacement

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

On 2017-04-25 21:22:44 +0100, Simon Riggs wrote: > On 24 April 2017 at 19:59, Andres Freund <and...@anarazel.de> wrote: > > > I don't think that's generally true. > > If you think that, from a risk perspective it is enough for me to > continue to investigate and I

Re: [HACKERS] Separation walsender & normal backends

On 2017-04-25 10:34:20 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > I've for a while suspected that the separation & duplication of > > infrastructure between walsenders and normal backends isn't nice. > > I think we should consider

Re: [HACKERS] PG 10 release notes

sponsibility to be consistent what what they have > wanted in the past. Why? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

On 2017-04-25 13:11:32 -0400, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 01:37:13PM -0300, Claudio Freire wrote: > > The truncate scan has been measured to be five times faster than without > > this patch (that was on a slow disk, but it shouldn't hurt on fast > > disks.) > > > >

Re: [HACKERS] PG 10 release notes

On 2017-04-25 10:10:07 -0400, Bruce Momjian wrote: > On Mon, Apr 24, 2017 at 08:52:05PM -0700, Andres Freund wrote: > > On 2017-04-24 23:45:06 -0400, Tom Lane wrote: > > Oh, I completely agree with accumulating related changes, and that > > code-level details aren't useful

[HACKERS] Separation walsender & normal backends

ast parts of it) into parsenodes.h - do a lot of other minor cleanup Greetings, Andres Freund >From 63866c2c8f9c968957dfb0d5b8c5ceca2c4787ed Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Mon, 24 Apr 2017 22:57:49 -0700 Subject: [PATCH] WIP: POC: Prototype: Un

Re: [HACKERS] Patch - Tcl 8.6 version support for PostgreSQL

$solution->{options}->{tcl} . '/lib/tcl85.lib'); Any chance of formulating these in a version agnostic way, instead of copying the same stanza for every version? E.g. using a wildcard or such... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Link to commits in PG 10 release notes

Hi, I wonder if there's a reasonable way that allows to add links to the more crucial commits for changelog entries. The source e.g. has Support parallel bitmap heap scans (Dilip Kumar) This allows a single index scan to dispatch parallel workers to process different areas of the heap.

Re: [HACKERS] PG 10 release notes

On 2017-04-24 23:45:06 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: > >> I remember seeing those and those are normally details I do not put in > >> the release notes as there isn't a cl

Re: [HACKERS] PG 10 release notes

On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: > On Mon, Apr 24, 2017 at 08:36:00PM -0700, Andres Freund wrote: > > On 2017-04-24 21:31:44 -0400, Bruce Momjian wrote: > > > I have committed the first draft of the Postgres 10 release notes. They > > > are curren

Re: [HACKERS] PG 10 release notes

On 2017-04-24 21:31:44 -0400, Bruce Momjian wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. > > The only unusual thing is that this release has ~180 items

Re: [HACKERS] walsender & parallelism

doesn't strike me as something that should go into a/src/Makefile.global.in - I'd rather put it in b/src/test/regress/GNUmakefile. check is in .global because it's used in a lot of different makefiles, but that's not true for this target, right? Greetings, Andres Freund -- Sent via pgsql-ha

Re: [HACKERS] snapbuild woes

2001 From: Andres Freund <and...@anarazel.de> Date: Sun, 23 Apr 2017 20:41:29 -0700 Subject: [PATCH] Preserve required !catalog tuples while computing initial decoding snapshot. The logical decoding machinery already preserved all the required catalog tuples, which is sufficient in the c

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

On 2017-04-24 15:41:25 -0700, Mark Dilger wrote: > The recent fix in 546c13e11b29a5408b9d6a6e3cca301380b47f7f has local variable > overwriteOK > assigned but not used in twophase.c RecoverPreparedTransactions(void). I'm > not sure if that's > future-proofing or an oversight. It seems to be

Re: [HACKERS] Unportable implementation of background worker start

On 2017-04-24 18:14:41 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-24 17:33:39 -0400, Tom Lane wrote: > >> coypu's problem is unrelated: > > > Note I was linking the 9.6 report form coypu, not HEAD. Afaics the 9.6 > >

Re: [HACKERS] Unportable implementation of background worker start

On 2017-04-24 17:33:39 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-24 13:16:44 -0700, Andres Freund wrote: > >> Unclear if related, but > >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=gharial=2017-04-24%2019%3A30%

Re: [HACKERS] to-do item for explain analyze of hash aggregates?

On 2017-04-24 21:13:16 +0200, Tomas Vondra wrote: > On 04/24/2017 08:52 PM, Andres Freund wrote: > > On 2017-04-24 11:42:12 -0700, Jeff Janes wrote: > > > The explain analyze of the hash step of a hash join reports something like > > > this: > > > > > >

Re: [HACKERS] to-do item for explain analyze of hash aggregates?

On 2017-04-24 13:55:57 -0700, Jeff Janes wrote: > On Mon, Apr 24, 2017 at 12:13 PM, Tomas Vondra I've added it to the wiki Todo page. (Hopefully that has not doomed it to > be forgotten about) The easiest way to avoid that fate is to implement it yourself ;) --

Re: [HACKERS] Patch - Tcl 8.6 version support for PostgreSQL

On 2017-04-24 16:18:30 -0400, Robert Haas wrote: > On Sat, Apr 22, 2017 at 1:58 PM, Sandeep Thakkar < > sandeep.thak...@enterprisedb.com> wrote: > > > Tcl8.6 is already supported in PostgreSQL. > > > > What commit added support for it? I don't think the main build mechanism requires explicit

Re: [HACKERS] Unportable implementation of background worker start

On 2017-04-24 13:16:44 -0700, Andres Freund wrote: > Unclear if related, but > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=gharial=2017-04-24%2019%3A30%3A42 > has a suspicious timing of failing in a weird way. Given that gharial is also failing on 9.6 (same set of commits)

Re: [HACKERS] Unportable implementation of background worker start

On 2017-04-21 23:50:41 -0400, Tom Lane wrote: > I wrote: > > Attached is a lightly-tested draft patch that converts the postmaster to > > use a WaitEventSet for waiting in ServerLoop. I've got mixed emotions > > about whether this is the direction to proceed, though. > > Attached are a couple of

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

On 2017-04-24 13:29:11 +0100, Simon Riggs wrote: > On 24 April 2017 at 00:25, Andres Freund <and...@anarazel.de> wrote: > > if the subxid->xid mapping doesn't actually exist - as it's the case > > with this bug afaics - we'll not get the correct toplevel > >

Re: [HACKERS] to-do item for explain analyze of hash aggregates?

On 2017-04-24 11:42:12 -0700, Jeff Janes wrote: > The explain analyze of the hash step of a hash join reports something like > this: > >-> Hash (cost=458287.68..458287.68 rows=24995368 width=37) (actual > rows=24995353 loops=1) > Buckets: 33554432 Batches: 1 Memory Usage:

Re: [HACKERS] OK, so culicidae is *still* broken

On 2017-04-24 11:08:48 -0700, Andres Freund wrote: > On 2017-04-24 23:14:40 +0800, Craig Ringer wrote: > > In the long run we'll probably be forced toward threading or far pointers. > > I'll vote for removing the windows port, before going for that. And I'm > not even joking

Re: [HACKERS] OK, so culicidae is *still* broken

On 2017-04-24 14:43:11 -0400, Tom Lane wrote: > (We have accepted that kind of overhead for DSM segments, but the > intention I think is to allow only very trivial data structures in > the DSM segments. Losing compiler pointer type checking for data > structures like the lock or PGPROC tables

Re: [HACKERS] Cached plans and statement generalization

- Migrating to parameters can actually cause significant slowdowns, not nice if that happens implicitly. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] OK, so culicidae is *still* broken

t. On 2017-04-24 23:14:40 +0800, Craig Ringer wrote: > In the long run we'll probably be forced toward threading or far pointers. I'll vote for removing the windows port, before going for that. And I'm not even joking. Greetings, Andres Freund -- Sent via pgsql-hackers mailing lis

Re: [HACKERS] walsender & parallelism

On 2017-04-24 18:29:51 +0200, Petr Jelinek wrote: > On 24/04/17 07:42, Andres Freund wrote: > > > > > > On April 23, 2017 10:31:18 PM PDT, Petr Jelinek > > <petr.jeli...@2ndquadrant.com> wrote: > >> On 24/04/17 04:31, Petr Jelinek wrote: > >>

Re: [HACKERS] walsender & parallelism

On April 23, 2017 10:31:18 PM PDT, Petr Jelinek wrote: >On 24/04/17 04:31, Petr Jelinek wrote: >So actually maybe running regression tests through it might be >reasonable approach if we add new make target for it. That sounds like a good plan. >Note that the

Re: [HACKERS] walsender & parallelism

On 2017-04-24 04:27:58 +0200, Petr Jelinek wrote: > On 24/04/17 01:43, Andres Freund wrote: > > > >> BTW while looking at the code, I don't understand why we call > >> latch_sigusr1_handler after calling SetLatch(MyLatch), shouldn't just > >> the SetLatch

Re: [HACKERS] walsender & parallelism

re got_SIGHUP (with a bit less generic name name) between different types of processes (i.e. putting it in miscadmin.h or such). It's not exactly pretty, but there's also no benefit in duplicating it everywhere, and without it you run into the issue presented here. Greetings, Andres Freund

Re: [HACKERS] walsender & parallelism

Hi, On 2017-04-23 16:59:41 -0700, Andres Freund wrote: > Hi, > > On 2017-04-22 17:53:19 +0200, Petr Jelinek wrote: > > Here is patch. I changed both SIGINT and SIGUSR1 handlers, afaics it > > does not break anything for existing walsender usage. > > > diff -

Re: [HACKERS] A note about debugging TAP failures

ou'd just accumulate cruft forever without manual intervention. So that's not too surprising imo... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] walsender & parallelism

y ignore SIGHUPS unless in WalSndLoop, WalSndWriteData,WalSndWaitForWal. That actually seems like an active bug to me? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] walsender & parallelism

On 2017-04-21 04:20:26 +0200, Petr Jelinek wrote: > Looks like SIGUSR1 being different is problem here - it's normally used > to . I also noticed that we don't handle SIGINT (query cancel). I think we really need to unify the paths between walsender and normal backends to a much larger degree.

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

op level or subxact * but it doesn't matter. If it's present, the xid is visible. */ for (j = 0; j < snapshot->subxcnt; j++) { if (TransactionIdEquals(xid, snapshot->subxip[j])) return true

Re: [HACKERS] Removing select(2) based latch (was Unportable implementation of background worker start)

Hi, On 2017-04-20 17:27:42 -0400, Tom Lane wrote: > In short: yeah, let's nuke the WAIT_USE_SELECT implementation. > It's dead code and it's unlikely to get resurrected. Done. > BTW, noting that SUSv2 specifies not , I wonder > whether we couldn't drop configure's test for the latter along

Re: [HACKERS] valgrind error in subscription code

On 2017-04-22 21:08:18 +0200, Petr Jelinek wrote: > On 22/04/17 20:31, Andres Freund wrote: > > Hi, > > > > I enabled skink / the valgrind animal to run the tap tests too (hugely > > increasing the test time :(), and that paid of: > > https://buildfarm.postgresql

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

On 2017-04-22 19:55:18 -0400, Tom Lane wrote: > Now that we've got consistent failure reports about the 009_twophase.pl > recovery test, I set out to find out why it's failing. It looks to me > like the reason is that this (twophase.c:2145): > > SubTransSetParent(xid, subxid,

Re: [HACKERS] A note about debugging TAP failures

On 2017-04-22 23:59:11 +0200, Daniel Gustafsson wrote: > Since we have the name of the running testscript, can’t we just add that to > the > tempdir to make the name more descriptive? With the attached patch I get > tmp_check/001_pgbench_data_main_ItEm when running tests in src/bin/pgbench for >

Re: [HACKERS] A note about debugging TAP failures

On 2017-04-22 17:25:49 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-22 16:22:59 -0400, Tom Lane wrote: > >> In the particular case I was interested in here, the test script thought > >> everything was successful :-(. I'm work

Re: [HACKERS] A note about debugging TAP failures

On 2017-04-22 16:22:59 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-22 13:51:30 -0400, Tom Lane wrote: > >> I think we need to fix TestLib and/or PostgresNode so that there's a way > >> to make TAP tests not auto-clean the

Re: [HACKERS] A note about debugging TAP failures

he test's path or such, to create the relevant temp directory. Before a start a test could then clean that up, that way the amount of remaining cruft would be bound by the number of tests. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To mak

Re: [HACKERS] valgrind error in subscription code

Hi, On 2017-04-22 21:08:18 +0200, Petr Jelinek wrote: > Thanks, here is patch to fix that - I also removed the individual > settings of everything to NULL/0/InvalidOid etc and just replaced it all > with memset. Cool. > diff --git a/src/backend/replication/logical/relation.c >

[HACKERS] valgrind error in subscription code

that looks more like something that needs to be silenced, based on the 3s look I gave it. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Unportable implementation of background worker start

h I'd still advocate for > keeping the change in maybe_start_bgworker. Not yet having looked at your patches, that sounds like a reasonable plan. I'd still like to get something like your CLOEXEC patch applied independently however. < patches > Looks reasonable on a quick skim.

Re: [HACKERS] snapbuild woes

On 2017-04-17 21:16:57 -0700, Andres Freund wrote: > I've since the previous update reviewed Petr's patch, which he since has > updated over the weekend. I'll do another round tomorrow, and will see > how it looks. I think we might need some more tests for this to be > committable,

Re: [HACKERS] snapbuild woes

On 2017-04-20 13:32:10 +0200, Petr Jelinek wrote: > On 20/04/17 02:09, Andres Freund wrote: > > On 2017-04-17 21:16:57 -0700, Andres Freund wrote: > > I'm working on some infrastructure around this. Not sure if it needs to > > be committed, but it's certainly useful for e

Re: [HACKERS] multithreading in Batch/pipelining mode for libpq

On 2017-04-22 09:14:50 +0800, Craig Ringer wrote: > 2) if the answer to the previous question is negative, is it possible to > send asynchronous queries in one thread while reading results in another > thread? > > > Not right now. libpq's state tracking wouldn't cope. > > I imagine it could be

<    2   3   4   5   6   7   8   9   10   11   >