Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 10:51 PM, Heikki Linnakangas wrote: > I still don't get it. When building the initial runs, we don't need buffer > space for maxTapes yet, because we're only writing to a single tape at a > time. An unused tape shouldn't take much memory. In inittapes(),

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On 09/07/2016 12:46 AM, Peter Geoghegan wrote: On Tue, Sep 6, 2016 at 12:34 AM, Heikki Linnakangas wrote: Why do we reserve the buffer space for all the tapes right at the beginning? Instead of the single USEMEM(maxTapes * TAPE_BUFFER_OVERHEAD) callin inittapes(), couldn't we

Re: [HACKERS] patch: function xmltable

2016-09-07 5:03 GMT+02:00 Craig Ringer : > On 7 September 2016 at 04:13, Pavel Stehule > wrote: > > >> Overall, I think this needs to be revised with appropriate comments. > >> Whitespace/formatting needs fixing since it's all over the place. > >>

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 10:36 PM, Peter Geoghegan wrote: > Well, maybe, but the whole idea behind replacement_sort_tuples (by > which I mean the continued occasional use of replacement selection by > Postgres) was that we hope to avoid a merge step *entirely*. This new > merge

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 10:28 PM, Heikki Linnakangas wrote: >> BTW, the way that k-way merging is made more efficient by this >> approach makes the case for replacement selection even weaker than it >> was just before we almost killed it. > > > This also makes the replacement

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On 09/06/2016 10:42 PM, Peter Geoghegan wrote: On Tue, Sep 6, 2016 at 12:39 PM, Peter Geoghegan wrote: On Tue, Sep 6, 2016 at 12:08 AM, Heikki Linnakangas wrote: I attach a patch that changes how we maintain the heap invariant during tuplesort merging.

Re: [HACKERS] [sqlsmith] Failed assertion in joinrels.c

On Wed, Sep 7, 2016 at 8:52 AM, Haribabu Kommi wrote: > I reviewed and tested the patch. The changes are fine. > This patch provides better error message compared to earlier. > > Marked the patch as "Ready for committer" in commit-fest. Thanks for the review ! --

Re: [HACKERS] Push down more UPDATEs/DELETEs in postgres_fdw

Thanks Fujita-san for working on this. > * with the patch: > postgres=# explain verbose delete from ft1 using ft2 where ft1.a = ft2.a; > QUERY PLAN > >

Re: [HACKERS] [PATCH] COPY vs \copy HINT

Craig Ringer writes: > On 7 September 2016 at 04:19, Christoph Berg wrote: >> I like your new version, it's crisp and transports the right message. > OK, updated with Tom's tweaked version of Christoph's wording per > discussion. Thanks all. Pushed with

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On 7 September 2016 at 11:37, Corey Huinker wrote: > On Tue, Sep 6, 2016 at 11:24 PM, Craig Ringer > wrote: >> >> On 7 September 2016 at 11:21, Corey Huinker >> wrote: >> > On Tue, Sep 6, 2016 at 6:53 PM, Craig

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On Tue, Sep 6, 2016 at 11:24 PM, Craig Ringer wrote: > On 7 September 2016 at 11:21, Corey Huinker > wrote: > > On Tue, Sep 6, 2016 at 6:53 PM, Craig Ringer < > craig.rin...@2ndquadrant.com> > > > And the TAP test would detect the operating

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On Tue, Sep 6, 2016 at 9:46 PM, Amit Langote wrote: > On 2016/09/07 3:12, Corey Huinker wrote: > > On Fri, Sep 2, 2016 at 5:07 AM, Amit Langote wrote: > >> I am not familiar with win32 stuff too, so I don't have much to say > about > >> that. Maybe someone else

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On 7 September 2016 at 11:21, Corey Huinker wrote: > On Tue, Sep 6, 2016 at 6:53 PM, Craig Ringer > And the TAP test would detect the operating system and know to create an FDW > that has the PROGRAM value 'cat test_data.csv' on Unix, 'type

Re: [HACKERS] [sqlsmith] Failed assertion in joinrels.c

On Fri, Aug 26, 2016 at 7:11 PM, Dilip Kumar wrote: > I have changed this in attached patch.. > I reviewed and tested the patch. The changes are fine. This patch provides better error message compared to earlier. Marked the patch as "Ready for committer" in commit-fest.

Re: [HACKERS] Suggestions for first contribution?

On 7 September 2016 at 10:54, Noah Misch wrote: > On Tue, Sep 06, 2016 at 01:32:10PM -0400, Christian Convey wrote: >> It sounds like the most useful thing I can do at the moment is perform >> code reviews. I assumed I'd need more experience with the PG code >> base, but I

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On Tue, Sep 6, 2016 at 6:53 PM, Craig Ringer wrote: > On 7 Sep. 2016 02:14, "Corey Huinker" wrote: > > > > > Having regression tests for this is extremely problematic, because the > program invoked would need to be an invokable command on

Re: [HACKERS] Optimization for lazy_scan_heap

On Wed, Sep 7, 2016 at 1:47 AM, Robert Haas wrote: > On Mon, Sep 5, 2016 at 8:57 PM, Masahiko Sawada wrote: >>> What performance difference does this make, in a realistic use case? >> >> I have yet to measure performance effect but it would be effect

Re: [HACKERS] patch: function xmltable

On 7 September 2016 at 04:13, Pavel Stehule wrote: >> Overall, I think this needs to be revised with appropriate comments. >> Whitespace/formatting needs fixing since it's all over the place. >> Documentation is insufficient (per notes below). > > > I am not able to

Re: [HACKERS] improved DefElem list processing

On 9/6/16 7:23 PM, Tom Lane wrote: > I'm curious where you are on that? I find myself needing to whack around > this processing in CREATE EXTENSION, but I don't want to create a merge > problem for you if you're close to committing. I have committed what I have for now. Thanks. -- Peter

Re: [HACKERS] Suggestions for first contribution?

On Tue, Sep 06, 2016 at 01:32:10PM -0400, Christian Convey wrote: > It sounds like the most useful thing I can do at the moment is perform > code reviews. I assumed I'd need more experience with the PG code > base, but I keep on reading that newcomers' reviews are welcome. > Unless someone has a

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

On Wed, Sep 7, 2016 at 12:16 AM, David Steele wrote: > On 9/1/16 9:53 AM, Peter Eisentraut wrote: >> >> On 8/15/16 3:39 PM, David Steele wrote: >>> >>> That patch got me thinking about what else could be excluded and after >>> some investigation I found the following:

Re: [HACKERS] [PATCH] COPY vs \copy HINT

On 7 September 2016 at 04:19, Christoph Berg wrote: > I like your new version, it's crisp and transports the right message. OK, updated with Tom's tweaked version of Christoph's wording per discussion. Thanks all. -- Craig Ringer http://www.2ndQuadrant.com/

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2016/08/26 15:13, Ashutosh Bapat wrote: On Fri, Aug 26, 2016 at 11:37 AM, Masahiko Sawada > wrote: On Fri, Aug 26, 2016 at 3:03 PM, Ashutosh Bapat >

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On 2016/09/07 3:12, Corey Huinker wrote: > On Fri, Sep 2, 2016 at 5:07 AM, Amit Langote wrote: >> I am not familiar with win32 stuff too, so I don't have much to say about >> that. Maybe someone else can chime in to help with that. > > The regressions basically *can't* test this because we'd

Re: [HACKERS] ICU integration

On Tue, Sep 6, 2016 at 10:40 AM, Doug Doole wrote: > - Suppose in ICU X.X, AA = Å but in ICU Y.Y AA != Å. Further suppose there > was an RI constraint where the primary key used AA and the foreign key used > Å. If ICU was updated, the RI constraint between the rows would

Re: [HACKERS] [PATCH] add option to pg_dumpall to exclude tables from the dump

On Fri, Aug 19, 2016 at 12:38 PM, Jim Nasby wrote: > On 8/18/16 5:01 PM, Tom Lane wrote: > >> I agree, but I think mandating a database name (which I suppose could be >>> > *) with the specifiers would solve that issue. >>> >> Hmm, something like "-T dbname1:pattern1 -T

Re: [HACKERS] ICU integration

On Tue, Sep 6, 2016 at 10:40 AM, Doug Doole wrote: >> The ICU ABI (not API) is also versioned. The way that this is done is >> that all functions are actually macros to a versioned symbol. So >> ucol_open() is actually a macro that expands to, say, ucol_open_57() in >>

Re: [HACKERS] Logical Replication WIP

Review of 0002-Add-SUBSCRIPTION-catalog-and-DDL.patch: (As you had already mentioned, some of the review items in 0001 apply analogously here.) Changes needed to compile: --- a/src/backend/commands/subscriptioncmds.c +++ b/src/backend/commands/subscriptioncmds.c @@ -218,7 +218,7 @@

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 5:50 PM, Claudio Freire wrote: > However, I agree that it's not worth the risk conflating the two > optimizations. That one can be done later as a separate patch. I'm rather fond of the assertions about tape number that exist within root_displace

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 9:19 PM, Peter Geoghegan wrote: > On Tue, Sep 6, 2016 at 4:55 PM, Claudio Freire wrote: >> I noticed, but here n = state->memtupcount >> >> + Assert(memtuples[0].tupindex == newtup->tupindex); >> + >> +

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 4:55 PM, Claudio Freire wrote: > I noticed, but here n = state->memtupcount > > + Assert(memtuples[0].tupindex == newtup->tupindex); > + > + CHECK_FOR_INTERRUPTS(); > + > + n = state->memtupcount; /* n is heap's

Re: [HACKERS] Speedup twophase transactions

>> On 06 Sep 2016, at 12:03, Michael Paquier wrote: >> >> On Tue, Sep 6, 2016 at 5:58 PM, Stas Kelvich >> wrote: >>> Oh, I was preparing new version of patch, after fresh look on it. Probably, >>> I should >>> said that in this topic. I’ve

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 8:28 PM, Peter Geoghegan wrote: > On Tue, Sep 6, 2016 at 12:57 PM, Claudio Freire > wrote: >> Patch lacks any new tests, but the changed code paths seem covered >> sufficiently by existing tests. A little bit of fuzzing on the

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On Wed, Sep 7, 2016 at 7:53 AM, Craig Ringer wrote: > On 7 Sep. 2016 02:14, "Corey Huinker" wrote: >> > >> Having regression tests for this is extremely problematic, because the >> program invoked would need to be an invokable command on any

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 12:57 PM, Claudio Freire wrote: > Patch lacks any new tests, but the changed code paths seem covered > sufficiently by existing tests. A little bit of fuzzing on the patch > itself, like reverting some key changes, or flipping some key > comparisons,

Re: [HACKERS] improved DefElem list processing

Peter Eisentraut writes: > Here are two WIP patches to improve the DefElem list processing that is > used by many utility commands. > One factors out the duplicate checks, which are currently taking up a > lot of space with duplicate code. I haven't applied

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On 7 Sep. 2016 02:14, "Corey Huinker" wrote: > > Having regression tests for this is extremely problematic, because the program invoked would need to be an invokable command on any architecture supported by postgres. I'm pretty sure no such command exists. Your best bet

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 2:46 PM, Peter Geoghegan wrote: > Feel free to make a counter-proposal for a cap. I'm not attached to > 500. I'm mostly worried about blatant waste with very large workMem > sizings. Tens of thousands of tapes is just crazy. The amount of data > that you

Re: [HACKERS] kqueue

On 2016-06-03 01:45, Thomas Munro wrote: On Fri, Jun 3, 2016 at 4:02 AM, Alvaro Herrera wrote: Tom Lane wrote: Andres Freund writes: pg_strtoi? I think that's what Thomas did upthread. Are you taking this one then? I'd go with just

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 12:34 AM, Heikki Linnakangas wrote: > I'm reviewing patches 1-3 in this series, i.e. those patches that are not > directly related to parallelism, but are independent improvements to > merging. That's fantastic! Thanks! I'm really glad you're picking

Re: [HACKERS] patch: function xmltable

> libxml2 and our XPATH function doesn't support default namespace ( > http://plasmasturm.org/log/259/ ). This is pretty useful feature - so I > implemented. This is the mayor issue of libxml2 library. Another difference > between XPATH function and XMLTABLE function is using two phase searching >

Re: [HACKERS] [PATCH] COPY vs \copy HINT

Re: Tom Lane 2016-09-06 <17637.1473192...@sss.pgh.pa.us> > Christoph's idea isn't bad. We could tweak it to: > > COPY TO instructs the PostgreSQL server process to write a file. > > COPY FROM instructs the PostgreSQL server process to read a file. > > This seems to cover both the

Re: [HACKERS] [PATCH] Alter or rename enum value

Andrew Dunstan writes: > Are we also going to have an exists test for the original thing being > renamed? Exists tests on renames do strike me as somewhat cumbersome, to > say the least. I'm less opposed to that part, because it's at least got *some* precedent in existing

Re: [HACKERS] patch: function xmltable

Hi 2016-09-06 6:54 GMT+02:00 Craig Ringer : > On 4 September 2016 at 16:06, Pavel Stehule > wrote: > > Hi > > > > minor update - using DefElem instead own private parser type > > I'm really glad that you're doing this and I'll take a look at it

Re: [HACKERS] [PATCH] COPY vs \copy HINT

Craig Ringer writes: > Tom, any preference here? > I'm probably inclined to go for your original wording and accept that > it's just too hard to hint at the client/server process split in a > single short message. I think my original wording is pretty hopeless for the

Re: [HACKERS] [PATCH] Alter or rename enum value

On 09/06/2016 02:30 PM, Tom Lane wrote: Robert Haas writes: On Mon, Sep 5, 2016 at 11:40 PM, Tom Lane wrote: ... And again, it's hard to get excited about having these options for RENAME VALUE when no one has felt a need for them yet in RENAME

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Mon, Aug 15, 2016 at 9:33 PM, Peter Geoghegan wrote: > The patch is intended to be applied on top of parallel B-Tree patches > 0001-* and 0002-* [1]. I happened to test it with parallelism, but > these are all independently useful, and will be entered as a separate > CF entry

Re: [HACKERS] Bug in 9.6 tuplesort batch memory growth logic

On Tue, Sep 6, 2016 at 12:51 PM, Tom Lane wrote: > I rewrote the comment and pushed it. Thank you. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Bug in 9.6 tuplesort batch memory growth logic

I wrote: > I see. The comment could do with a bit of rewriting, perhaps. I rewrote the comment and pushed it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 12:39 PM, Peter Geoghegan wrote: > On Tue, Sep 6, 2016 at 12:08 AM, Heikki Linnakangas wrote: >>> I attach a patch that changes how we maintain the heap invariant >>> during tuplesort merging. > >> Nice! > > Thanks! BTW, the way that

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Sep 6, 2016 at 12:08 AM, Heikki Linnakangas wrote: >> I attach a patch that changes how we maintain the heap invariant >> during tuplesort merging. > Nice! Thanks! >> This new approach is more or less the *conventional* way to maintain >> the heap invariant when

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 09/06/2016 04:49 AM, Amit Kapila wrote: > On Mon, Sep 5, 2016 at 11:34 PM, Tomas Vondra > wrote: >> >> >> On 09/05/2016 06:03 AM, Amit Kapila wrote: >>> So, in short we have to compare three >>> approaches here. >>> >>> 1) Group mode to reduce CLOGControlLock

Re: [HACKERS] Tuplesort merge pre-reading

On Tue, Sep 6, 2016 at 12:08 PM, Peter Geoghegan wrote: > Offhand, I would think that taken together this is very important. I'd > certainly want to see cases in the hundreds of megabytes or gigabytes > of work_mem alongside your 4MB case, even just to be able to talk >

Re: [HACKERS] [GENERAL] C++ port of Postgres

On Tue, Sep 6, 2016 at 3:12 PM, Tom Lane wrote: >> (2) It seems like there are still a few big questions about this commit: >>- Is it wanted at the moment? It didn't seem like there's a >> consensus about whether or not this enhancement should be >> merged, even

Re: [HACKERS] [GENERAL] C++ port of Postgres

Christian Convey writes: > Could someone help me with a few procedural questions? > (1) This page: https://wiki.postgresql.org/wiki/Reviewing_a_Patch > lists the current commitfest's manager as "(vacant)". But this page: > https://commitfest.postgresql.org/ seems to

Re: [HACKERS] Tuplesort merge pre-reading

On Tue, Sep 6, 2016 at 5:20 AM, Heikki Linnakangas wrote: > I wrote a quick patch to test that, attached. It seems to improve > performance, at least in this small test case: > > create table lotsofints(i integer); > insert into lotsofints select random() * 10.0 from >

Re: [HACKERS] Logical Replication WIP

On 06/09/16 20:14, Peter Eisentraut wrote: On 9/3/16 5:14 AM, Petr Jelinek wrote: What are the BKI_ROWTYPE_OID assignments for? Are they necessary here? (Maybe this was just copied from pg_subscription?) Yes they are. Please explain/document why. It does not match other catalogs, which

Re: [HACKERS] Quorum commit for multiple synchronous replication.

On 08/29/2016 06:52 AM, Fujii Masao wrote: > Also I like the following Simon's idea. > > https://www.postgresql.org/message-id/canp8+jlhfbvv_pw6grasnupw+bdk5dctu7gwpnap-+-zwvk...@mail.gmail.com > --- > * first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now > * any k

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

Simon Riggs writes: > On 6 September 2016 at 19:23, Robert Haas wrote: >> On Tue, Sep 6, 2016 at 2:16 PM, Simon Riggs wrote: >>> What occurs to me is that we can exactly predict how many tuples we >>> are going to get when we

Re: [HACKERS] [GENERAL] C++ port of Postgres

On 08/31/2016 04:41 PM, Peter Eisentraut wrote: I developed a minimally invasive patch for C++ support a few years ago shortly after I wrote that blog post. Since there appears to have been some interest here now, I have updated that and split it up into logical chunks. So here you go.

Re: [HACKERS] [GENERAL] C++ port of Postgres

On Wed, Aug 31, 2016 at 9:41 AM, Peter Eisentraut wrote: >> Joy, do you have an idea what a *minimally invasive* patch for C++ >> support would look like? That's certainly the first step here. > > I developed a minimally invasive patch for C++ support a few years

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 2:51 PM, Simon Riggs wrote: > On 6 September 2016 at 19:23, Robert Haas wrote: >> On Tue, Sep 6, 2016 at 2:16 PM, Simon Riggs wrote: >>> What occurs to me is that we can exactly predict how many tuples

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On 6 September 2016 at 19:23, Robert Haas wrote: > On Tue, Sep 6, 2016 at 2:16 PM, Simon Riggs wrote: >> What occurs to me is that we can exactly predict how many tuples we >> are going to get when we autovacuum, since we measure that and we know >>

Re: [HACKERS] \timing interval

Corey Huinker writes: > On Sun, Sep 4, 2016 at 7:05 PM, Jim Nasby wrote: >> I'd find this useful in the final output of EXPLAIN ANALYZE as well; any >> objections to adding it? > It's sorta out of my hands now, but what Tom said earlier is that

Re: [HACKERS] Showing parallel status in \df+

Hi 2016-09-06 0:05 GMT+02:00 Tom Lane : > I wrote: > > Pavel Stehule writes: > >> Using footer for this purpose is little bit strange. What about > following > >> design? > >> 1. move out source code of PL functions from \df+ > >> 2. allow not unique

Re: [HACKERS] \timing interval

On Sun, Sep 4, 2016 at 7:05 PM, Jim Nasby wrote: > On 9/3/16 2:35 PM, Tom Lane wrote: > >> I pushed the patch using this: >> >> Time: 176460001.200 ms (2 d 01:01:00.001) >> >> and all else as before. >> > > I'd find this useful in the final output of EXPLAIN ANALYZE as

Re: [HACKERS] [PATCH] Alter or rename enum value

Robert Haas writes: > On Mon, Sep 5, 2016 at 11:40 PM, Tom Lane wrote: >> ... And again, it's >> hard to get excited about having these options for RENAME VALUE when no >> one has felt a need for them yet in RENAME COLUMN. I'm especially dubious >>

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 3:45 AM, Simon Riggs wrote: > On 5 September 2016 at 21:58, Claudio Freire wrote: > > How long does that part ever take? Is there any substantial gain from > this? > >> Btw, without a further patch to prefetch pages

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 2:16 PM, Simon Riggs wrote: > What occurs to me is that we can exactly predict how many tuples we > are going to get when we autovacuum, since we measure that and we know > what the number is when we trigger it. > > So there doesn't need to be any

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On 6 September 2016 at 19:09, Robert Haas wrote: > On Tue, Sep 6, 2016 at 2:06 PM, Simon Riggs wrote: >> On 6 September 2016 at 19:00, Tom Lane wrote: >>> Robert Haas writes: Yeah, but I've seen

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 3:11 PM, Tom Lane wrote: > We could get around (1) by something like Robert's idea of segmented > allocation, but TBH I've seen nothing on this thread to make me think > it's necessary or would even result in any performance improvement > at all. The

Re: [HACKERS] Logical Replication WIP

On 9/3/16 5:14 AM, Petr Jelinek wrote: >> What are the BKI_ROWTYPE_OID assignments for? Are they necessary >> > here? (Maybe this was just copied from pg_subscription?) >> > > Yes they are. Please explain/document why. It does not match other catalogs, which either use it for relcache

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On Fri, Sep 2, 2016 at 5:07 AM, Amit Langote wrote: > > I am not familiar with win32 stuff too, so I don't have much to say about > that. Maybe someone else can chime in to help with that. > The regressions basically *can't* test this because we'd need a shell

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

Simon Riggs writes: > Is there a reason we can't use repalloc here? (1) repalloc will probably copy the data. (2) that answer doesn't excuse you from choosing a limit. We could get around (1) by something like Robert's idea of segmented allocation, but TBH I've seen

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 2:09 PM, Robert Haas wrote: > There are two possible problems, either of which is necessarily fatal: I meant to write "neither of which" not "either of which". -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 2:06 PM, Simon Riggs wrote: > On 6 September 2016 at 19:00, Tom Lane wrote: >> Robert Haas writes: >>> Yeah, but I've seen actual breakage from exactly this issue on >>> customer systems even with the 1GB

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 2:00 PM, Tom Lane wrote: > Robert Haas writes: >> Yeah, but I've seen actual breakage from exactly this issue on >> customer systems even with the 1GB limit, and when we start allowing >> 100GB it's going to get a whole lot worse.

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On 6 September 2016 at 19:00, Tom Lane wrote: > Robert Haas writes: >> Yeah, but I've seen actual breakage from exactly this issue on >> customer systems even with the 1GB limit, and when we start allowing >> 100GB it's going to get a whole lot worse. >

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

Robert Haas writes: > Yeah, but I've seen actual breakage from exactly this issue on > customer systems even with the 1GB limit, and when we start allowing > 100GB it's going to get a whole lot worse. While it's not necessarily a bad idea to consider these things, I think

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 11:22 PM, Claudio Freire wrote: > CREATE INDEX could also allocate 218GB, you just need to index enough > columns and you'll get that. > > Aside from the fact that CREATE INDEX will only allocate what is going > to be used and VACUUM will

Re: [HACKERS] [PATCH] Alter or rename enum value

On Mon, Sep 5, 2016 at 11:40 PM, Tom Lane wrote: > The opportunity cost here is potential user confusion. The only > closely parallel rename operation we have is ALTER TABLE RENAME COLUMN, > and that doesn't have a column-level IF EXISTS option; it has a > table-level IF

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 2:39 PM, Robert Haas wrote: >> I could attempt that, but I don't see the difference between >> vacuum and create index in this case. Both could allocate a huge chunk >> of the virtual address space if maintainance work mem says so, both >>

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

On Tue, Sep 6, 2016 at 9:43 PM, Peter Eisentraut wrote: > On 8/11/16 9:12 PM, Michael Paquier wrote: >> Note that pg_dump[all] and pg_upgrade already have safeguards against >> those things per the same routines putting quotes for execution as >> commands into

Re: [HACKERS] Logical Replication WIP

On 01/09/16 08:29, Erik Rijkers wrote: On 2016-09-01 01:04, Erik Rijkers wrote: On 2016-08-31 22:51, Petr Jelinek wrote: Here are some small changes to logical-replication.sgml ... and other .sgml files. Thanks I'll integrate these into next iteration of the patch, -- Petr Jelinek

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make initdb's suggested "pg_ctl start" command line more reliabl

Claudio Freire writes: > On Tue, Sep 6, 2016 at 2:08 PM, Tom Lane wrote: >> Dash is considered a character that needs quoting. It might be possible >> to avoid that if we could be certain that appendShellString's output would >> never be placed in a

Re: [HACKERS] ICU integration

> The ICU ABI (not API) is also versioned. The way that this is done is > that all functions are actually macros to a versioned symbol. So > ucol_open() is actually a macro that expands to, say, ucol_open_57() in > ICU version 57. (They also got rid of a dot in their versions a while > ago.)

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 6, 2016 at 10:28 PM, Claudio Freire wrote: >> The problem with this is that we allocate the entire amount of >> maintenance_work_mem even when the number of actual dead tuples turns >> out to be very small. That's not so bad if the amount of memory we're >>

Re: [HACKERS] Suggestions for first contribution?

Thanks everyone for the suggestions. It sounds like the most useful thing I can do at the moment is perform code reviews. I assumed I'd need more experience with the PG code base, but I keep on reading that newcomers' reviews are welcome. Unless someone has a better idea, I'll start with that.

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

Peter Eisentraut writes: > On 8/11/16 9:12 PM, Michael Paquier wrote: >> Note that pg_dump[all] and pg_upgrade already have safeguards against >> those things per the same routines putting quotes for execution as >> commands into psql and shell. So attached is a

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make initdb's suggested "pg_ctl start" command line more reliabl

On 2016-09-06 13:08:51 -0400, Tom Lane wrote: > Dash is considered a character that needs quoting. It might be possible > to avoid that if we could be certain that appendShellString's output would > never be placed in a spot where it could be taken for a switch, but that > seems like a large

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make initdb's suggested "pg_ctl start" command line more reliabl

On Tue, Sep 6, 2016 at 2:08 PM, Tom Lane wrote: >> The not-quoting-if-not-needed doesn't appear to do anything useful for me: >> 'pg-install/bin/pg_ctl' -D 'pg-install/var/data' -l logfile start > > Dash is considered a character that needs quoting. It might be possible >

Re: [HACKERS] SELECT FOR UPDATE regression in 9.5

Marko Tiikkaja wrote: > On 2016-09-06 6:02 PM, Marti Raudsepp wrote: > >This issue is also reproducible on the current master branch. In an > >assertions-enabled build, it traps an assertion in HeapTupleHeaderGetCmax > >called by heap_lock_tuple. The following test case demonstrates the issue... >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make initdb's suggested "pg_ctl start" command line more reliabl

Peter Eisentraut writes: > On 8/20/16 3:05 PM, Tom Lane wrote: >> Make initdb's suggested "pg_ctl start" command line more reliable. > A couple of problems with this: > The not-quoting-if-not-needed doesn't appear to do anything useful for me: >

Re: [HACKERS] SELECT FOR UPDATE regression in 9.5

On 2016-09-06 6:02 PM, Marti Raudsepp wrote: This issue is also reproducible on the current master branch. In an assertions-enabled build, it traps an assertion in HeapTupleHeaderGetCmax called by heap_lock_tuple. The following test case demonstrates the issue... I think you found a

Re: [HACKERS] (Comment)Bug in CteScanNext

Jim Nasby writes: > On 9/3/16 1:30 PM, Tom Lane wrote: >> Or we could add something like "But first, we must deal with the special >> case of reversing direction after reaching EOF." > I'm working on that, but one thing isn't clear to me... why do we only > skip past

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Sun, Sep 4, 2016 at 8:10 PM, Jim Nasby wrote: > On 9/4/16 1:46 AM, Simon Riggs wrote: >>> >>> > The patch also makes vacuum free the dead_tuples before starting >>> > truncation. It didn't seem necessary to hold onto it beyond that >>> > point, and it might help give

Re: [HACKERS] GiST penalty functions [PoC]

On 08/29/2016 09:04 AM, Andrew Borodin wrote: In this message I address only first problem. I want to construct a penalty function that will choose: 1.Entry with a zero volume and smallest margin, that can accommodate insertion tuple without extension, if one exists; 2.Entry with

Re: [HACKERS] WIP: Covering + unique indexes.

28.08.2016 09:13, Amit Kapila: On Mon, Aug 15, 2016 at 8:15 PM, Anastasia Lubennikova wrote: @@ -590,7 +622,14 @@ _bt_buildadd(BTWriteState *wstate, BTPageState *state, IndexTuple itup) if (last_off == P_HIKEY) { Assert(state->btps_minkey == NULL); -

Re: [HACKERS] Optimization for lazy_scan_heap

On Mon, Sep 5, 2016 at 8:57 PM, Masahiko Sawada wrote: >> What performance difference does this make, in a realistic use case? > > I have yet to measure performance effect but it would be effect for > very large frozen table. I think if you are proposing this patch because

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Sat, Sep 3, 2016 at 8:55 AM, Claudio Freire wrote: > The attached patch allows setting maintainance_work_mem or > autovacuum_work_mem higher than 1GB (and be effective), by turning the > allocation of the dead_tuples into a huge allocation. > > This results in fewer

  1   2   >