Re: [HACKERS] Is tuplesort_heap_siftup() a misnomer?

On Thu, Sep 8, 2016 at 1:14 PM, Andres Freund wrote: > Is this issue really worth keeping several hackers busy? I don't think it's fair to put it to me specifically that I'm doing that. That said, I would like to see this resolved without further bikeshedding. -- Peter Geoghegan -- Sent via

Re: [HACKERS] Preventing deadlock on parallel backup

I agree. It is an ugly hack. But to me, the reduced window for failure is important. And that way an failure will happen right away to be submitted to my operators as soon as possible. The queue jumping logic can not use the distributed transaction id? On my logic, if a connection requests a sha

[HACKERS] Why does PageIndexTupleDelete insist tuple size be maxaligned?

While perusing the proposed PageIndexTupleOverwrite patch, I noticed that PageIndexTupleDelete throws an error if size != MAXALIGN(size), where "size" is the ItemIdGetLength value. This seems wrong now that I look at it, because PageAddItem does not insist on tuple sizes being maxaligned. It does

[HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

Hi, On Debian unstable I just got a failure when running the regression tests: andres@alap4:~/build/postgres/dev-assert/vpath/src/bin/pg_rewind$ make check rm -rf '/home/andres/build/postgres/dev-assert/vpath'/tmp_install /bin/mkdir -p '/home/andres/build/postgres/dev-assert/vpath'/tmp_install/log

Re: [HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

Andres Freund wrote: > ISTM that the easiest fix is to just tack -I '$(srcdir)' into the prove > flags like: > PROVE = @PROVE@ > PG_PROVE_FLAGS = -I $(top_srcdir)/src/test/perl/ -I '$(srcdir)' > PROVE_FLAGS = --verbose > > I don't think there's any security concerns for us here. Maybe not, but

Re: [HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

On 2016-09-08 17:58:03 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > ISTM that the easiest fix is to just tack -I '$(srcdir)' into the prove > > flags like: > > PROVE = @PROVE@ > > PG_PROVE_FLAGS = -I $(top_srcdir)/src/test/perl/ -I '$(srcdir)' > > PROVE_FLAGS = --verbose > > > > I d

Re: [HACKERS] Is tuplesort_heap_siftup() a misnomer?

Peter Geoghegan writes: > On Thu, Sep 8, 2016 at 1:14 PM, Andres Freund wrote: >> Is this issue really worth keeping several hackers busy? > I don't think it's fair to put it to me specifically that I'm doing > that. That said, I would like to see this resolved without further > bikeshedding. W

Re: [HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

Andres Freund wrote: > On 2016-09-08 17:58:03 -0300, Alvaro Herrera wrote: > > Andres Freund wrote: > > > > > ISTM that the easiest fix is to just tack -I '$(srcdir)' into the prove > > > flags like: > > > PROVE = @PROVE@ > > > PG_PROVE_FLAGS = -I $(top_srcdir)/src/test/perl/ -I '$(srcdir)' > > >

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Hey Alvaro, can you comment on this bit in the proposed patch? + for (i = FirstOffsetNumber; i <= itemcount; i++) + { + ItemId ii = PageGetItemId(phdr, i); + + /* Normally here was following assertion +

Re: [HACKERS] Add support for restrictive RLS policies

Stephen Frost writes: > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: >> Can't you keep those words as Sconst or something (DefElems?) until the >> execution phase, so that they don't need to be keywords at all? > Seems like we could do that, though I'm not convinced that it really > gains u

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Tom Lane wrote: > Hey Alvaro, can you comment on this bit in the proposed patch? > > + for (i = FirstOffsetNumber; i <= itemcount; i++) > + { > + ItemId ii = PageGetItemId(phdr, i); > + > + /* Normally here was following asse

Re: [HACKERS] Preventing deadlock on parallel backup

Lucas writes: > The queue jumping logic can not use the distributed transaction id? If we had such a thing as a distributed transaction id, maybe the answer could be yes. We don't. I did wonder whether using a shared snapshot might be a workable proxy for that, but haven't pursued it.

Re: [HACKERS] Is tuplesort_heap_siftup() a misnomer?

On Thu, Sep 8, 2016 at 2:09 PM, Tom Lane wrote: > Well, my vote is that it ain't broke and we shouldn't fix it. To take a step back, what prompted this whole discussion is the patch that I wrote that shifts down, replacing calls to tuplesort_heap_siftup() and tuplesort_heap_insert with one new ca

Re: [HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

On 2016-09-08 18:13:06 -0300, Alvaro Herrera wrote: > I suppose -I$(srcdir) should be fine. (Why the quotes?) Because quoting correctly seems like a good thing to do? Most people won't have whitespace in there, but it doesn't seem impossible? > > check-world appears to mostly run (still doing s

Re: [HACKERS] Is tuplesort_heap_siftup() a misnomer?

On Thu, Sep 8, 2016 at 2:19 PM, Peter Geoghegan wrote: >> Peter, looking at your "displace" patch in this light, I think >> tuplesort_heap_root_displace() and tuplesort_heap_delete_top() (as I'm >> calling it now), should share a common subroutine. Displace replaces the top >> node with the new no

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Alvaro Herrera writes: > Tom Lane wrote: >> That comment seems utterly wrong to me, because both PageIndexTupleDelete >> and PageIndexMultiDelete certainly contain assertions that every item on >> the page has storage. Are you expecting that any BRIN index items >> wouldn't? If they wouldn't, is

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> That comment seems utterly wrong to me, because both PageIndexTupleDelete > >> and PageIndexMultiDelete certainly contain assertions that every item on > >> the page has storage. Are you expecting that any BRIN index items > >> wou

Re: [HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

Andres Freund wrote: > On 2016-09-08 18:13:06 -0300, Alvaro Herrera wrote: > > I suppose -I$(srcdir) should be fine. (Why the quotes?) > > Because quoting correctly seems like a good thing to do? Most people > won't have whitespace in there, but it doesn't seem impossible? Well, I think they are

Re: [HACKERS] Preventing deadlock on parallel backup

Tom, Yes, it is what I mean. Is what pg_dump uses to get things synchronized. It seems to me a clear marker that the same task is using more than one connection to accomplish the one job. Em 08/09/2016 6:34 PM, "Tom Lane" escreveu: > Lucas writes: > > The queue jumping logic can not use the di

[HACKERS] mdtruncate leaking fd.c handles?

Hi, Am I missing something or is md.c:mdtruncate() leaking open files? The relevant piece of code is: if (priorblocks > nblocks) { /* * This segment is no longer active (and has already been unlinked

Re: [HACKERS] mdtruncate leaking fd.c handles?

Andres Freund writes: > Am I missing something or is md.c:mdtruncate() leaking open files? Yeah, I think you're right. > This only really matters for VACUUM truncate and ON COMMIT TRUNCATE temp > table truncation afaics. Also, you'd need a table > 1GB to leak anything at all, which probably hel

Re: [HACKERS] mdtruncate leaking fd.c handles?

On 2016-09-08 18:39:56 -0400, Tom Lane wrote: > Andres Freund writes: > > Am I missing something or is md.c:mdtruncate() leaking open files? > > Yeah, I think you're right. > > > This only really matters for VACUUM truncate and ON COMMIT TRUNCATE temp > > table truncation afaics. > > Also, you'

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On 9 Sep. 2016 03:45, "Corey Huinker" wrote: > > > Stylistically, would a separate .pl file for the emitter be preferable to something inline like > >> perl -e 'print "a\tb\tcc\t4\n"; print "b\tc\tdd\t5\n"' I'd be fine with that and a suitable comment. Just be careful with different platforms' s

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

On Thu, Sep 8, 2016 at 6:59 PM, Craig Ringer wrote: > On 9 Sep. 2016 03:45, "Corey Huinker" wrote: > > > > > > > Stylistically, would a separate .pl file for the emitter be preferable > to something inline like > > > >> perl -e 'print "a\tb\tcc\t4\n"; print "b\tc\tdd\t5\n"' > > I'd be fine with

Re: [HACKERS] High-CPU consumption on information_schema (only) query

On Wed, Sep 7, 2016 at 4:37 PM, Robins Tharakan wrote: > > Hi, > > An SQL (with only information_schema related JOINS) when triggered, runs with max CPU (and never ends - killed after 2 days). > - It runs similarly (very slow) on a replicated server that acts as a read-only slave. > - Top shows

Re: [HACKERS] High-CPU consumption on information_schema (only) query

On 2016-09-07 23:37:31 +, Robins Tharakan wrote: > If someone asks for I could provide SQL + EXPLAIN, but it feels irrelevant > here. Why is that? information_schema are normal sql queries, and some of them far from trivial. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] ICU integration

On Thu, Sep 8, 2016 at 8:16 AM, Tom Lane wrote: >> I understand that in principle, but I don't see operating system >> providers shipping a bunch of ICU versions to facilitate that. They >> will usually ship one. > > I agree with that estimate, and I would further venture that even if we > wanted

Re: [HACKERS] ICU integration

On 9 September 2016 at 00:19, Peter Eisentraut wrote: > On 9/8/16 11:16 AM, Tom Lane wrote: >> This is a problem, if ICU won't guarantee cross-version compatibility, >> because it destroys the argument that moving to ICU would offer us >> collation behavior stability. > > It would offer a signific

Re: [HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

On 9/8/16 6:04 PM, Alvaro Herrera wrote: > Andres Freund wrote: >> On 2016-09-08 18:13:06 -0300, Alvaro Herrera wrote: >>> I suppose -I$(srcdir) should be fine. (Why the quotes?) >> >> Because quoting correctly seems like a good thing to do? Most people >> won't have whitespace in there, but it do

Re: [HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

On 2016-09-08 20:15:46 -0400, Peter Eisentraut wrote: > On 9/8/16 6:04 PM, Alvaro Herrera wrote: > > Andres Freund wrote: > >> On 2016-09-08 18:13:06 -0300, Alvaro Herrera wrote: > >>> I suppose -I$(srcdir) should be fine. (Why the quotes?) > >> > >> Because quoting correctly seems like a good thi

Re: [HACKERS] mdtruncate leaking fd.c handles?

On 2016-09-08 15:50:42 -0700, Andres Freund wrote: > > Andres Freund writes: > > > Am I missing something or is md.c:mdtruncate() leaking open files? > Will fix. And done. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.po

Re: [HACKERS] _mdfd_getseg can be expensive

On 2016-08-31 15:15:16 -0700, Peter Geoghegan wrote: > On Wed, Aug 31, 2016 at 3:08 PM, Andres Freund wrote: > > On August 31, 2016 3:06:23 PM PDT, Peter Geoghegan wrote: > > > >>In other painfully pedantic news, I should point out that > >>sizeof(size_t) isn't necessarily word size (the most gen

Re: [HACKERS] ICU integration

> On 9 September 2016 at 00:19, Peter Eisentraut > wrote: >> On 9/8/16 11:16 AM, Tom Lane wrote: >>> This is a problem, if ICU won't guarantee cross-version compatibility, >>> because it destroys the argument that moving to ICU would offer us >>> collation behavior stability. >> >> It would offer

Re: Install extensions using update scripts (was Re: [HACKERS] Remove superuser() checks from pgstattuple)

Hi, On 2016-09-07 09:46:32 -0400, Tom Lane wrote: > At this point it's awfully tempting to make ALTER EXTENSION UPDATE grow > a CASCADE option to allow automatic installation of new requirements > of the new version, but I didn't do that here. Sounds like a plan. After the refactoring that shoul

Re: Install extensions using update scripts (was Re: [HACKERS] Remove superuser() checks from pgstattuple)

On 2016-09-07 13:44:28 -0400, Tom Lane wrote: > + > +Installing Extensions using Update Scripts > + > + > + An extension that has been around for awhile will probably exist in Wanted to cry typo for 'awhile' here, but apparently that's actually a word. The German in me is pleased.

Re: [HACKERS] Quorum commit for multiple synchronous replication.

On Thu, Sep 8, 2016 at 6:26 PM, Masahiko Sawada wrote: > "k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward > compatibility but most users would think "k(n1, n2, n3)" as quorum > after introduced quorum. > I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2, > n3)" s

Re: [HACKERS] Declarative partitioning - another take

On 2016/09/08 21:38, Rajkumar Raghuwanshi wrote: > On Wed, Sep 7, 2016 at 3:58 PM, Amit Langote wrote: >> On 2016/09/07 17:56, Rajkumar Raghuwanshi wrote: >>> >>> In this case not sure how to create partition table. Do we have something >>> like we have UNBOUNDED for range partition or oracle have

Re: [HACKERS] feature request: explain "with details" option

On 9 September 2016 at 01:40, Roger Pack wrote: > My apologies if this was already requested before... > > I think it would be fantastic if postgres had an "explain the explain" option: > Today's explain tells us what loops and scans were used, and relative > costs, etc. It doesn't seem to tell *

Re: [HACKERS] An extra error for client disconnection on Windows

On Thu, Jun 2, 2016 at 6:51 PM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello. > > After a process termination without PQfinish() of a client, > server emits the following log message not seen on Linux boxes. > > > LOG: could not receive data from client: An existing connect

Re: [HACKERS] ICU integration

On 9 September 2016 at 08:51, Tatsuo Ishii wrote: >> On 9 September 2016 at 00:19, Peter Eisentraut >> wrote: >>> On 9/8/16 11:16 AM, Tom Lane wrote: This is a problem, if ICU won't guarantee cross-version compatibility, because it destroys the argument that moving to ICU would offer us

Re: [HACKERS] Stopping logical replication protocol

On 9 September 2016 at 03:56, Vladimir Gordiychuk wrote: >>It'd helpful if you summarize the changes made when posting revisions. > > Can we use as summary about changes first message? I meant "what did you change between the last patch I posted and the one you just posted" ? I'm looking at the

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On 09/09/16 07:09, Jeff Janes wrote: On Wed, Sep 7, 2016 at 3:29 AM, Ashutosh Sharma > wrote: > Thanks to Ashutosh Sharma for doing the testing of the patch and > helping me in analyzing some of the above issues. Hi All, I would like to summarize

Re: [HACKERS] Stopping logical replication protocol

On Fri, Sep 9, 2016 at 11:37 AM, Craig Ringer wrote: > When writing TAP tests for Perl you must ensure you use only Perl > 5.8.8-compatible code and only the core modules plus IPC::Run . You'll > usually be fine if you just avoid importing things you don't see other > modules already using. Don't

Re: [HACKERS] ICU integration

On Thu, Sep 8, 2016 at 6:48 PM, Craig Ringer wrote: > Pity ICU doesn't offer versioned collations within a single install. > Though I can understand why they don't. There are two separate issues with collator versioning. ICU can probably be used in a way that clearly decouples these two issues, w

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

That storage assertion fired during usual update table set x=random() without conditions. Also Make check fails without it (for brin usage, gist is ok with it). Cannot type quotation properly, I'm on a phone for a long time. Regards, Andrey Borodin. -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Fri, Sep 9, 2016 at 12:39 AM, Jeff Janes wrote: > > I plan to do testing using my own testing harness after changing it to > insert a lot of dummy tuples (ones with negative values in the pseudo-pk > column, which are never queried by the core part of the harness) and > deleting them at random

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Thu, Sep 8, 2016 at 11:40 PM, Masahiko Sawada wrote: > > > Making the vacuum possible to choose between two data representations > sounds good. > I implemented the patch that changes dead tuple representation to bitmap > before. > I will measure the performance of bitmap representation again a

Re: [HACKERS] GiST penalty functions [PoC]

Thank you for your attention to details, Mikhail. pack_float_good() looks good. But I'm not sure inline strict init is allowed under ansi C. Converting to regular ancient form b.fp = v; won't change compile result, would it? Regards, Andrey Borodin. -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] Stopping logical replication protocol

On 9 September 2016 at 10:37, Craig Ringer wrote: > I'm looking at the revised patch now. Considerably improved. I fixed a typo from decondig to decoding that's throughout all the callback names. This test is wrong: +while ((change = ReorderBufferIterTXNNext(rb, iterstate)) != NULL

Re: [HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

Andres Freund writes: > On 2016-09-08 20:15:46 -0400, Peter Eisentraut wrote: >> We don't support build directories with spaces in them, but we support >> installation directories with spaces in them. So I guess that means >> your point is valid. > Even if not necessary in this specific case, I

Re: [HACKERS] Logical Replication WIP

Review of 0003-Define-logical-replication-protocol-and-output-plugi.patch: (This is still based on the Aug 31 patch set, but at quick glance I didn't see any significant changes in the Sep 8 set.) Generally, this all seems mostly fine. Everything is encapsulated well enough that problems are loc

Re: [HACKERS] feature request: explain "with details" option

Craig Ringer writes: > On 9 September 2016 at 01:40, Roger Pack wrote: >> Today's explain tells us what loops and scans were used, and relative >> costs, etc. It doesn't seem to tell *why* the planner elected to use >> what it did. > One thing that's been discussed here is to have a way to see

Re: [HACKERS] [sqlsmith] Failed assertion in joinrels.c

On Thu, Sep 8, 2016 at 7:21 PM, Tom Lane wrote: > We could make it work like that without breaking the ABI if we were > to add a NOERROR bit to the allowed "flags". However, after looking > around a bit I'm no longer convinced what I said above is a good idea. > In particular, if we put the respo

Re: [HACKERS] Patch: Implement failover on libpq connect level.

On Wed, Sep 7, 2016 at 7:26 PM, Victor Wagner wrote: > No, algorithm here is more complicated. It must ensure that there would > not be second attempt to connect to host, for which unsuccessful > connection attempt was done. So, there is list rearrangement. >Algorithm for pick random list element

Re: [HACKERS] Quorum commit for multiple synchronous replication.

On 09/09/2016 03:28 AM, Michael Paquier wrote: > On Thu, Sep 8, 2016 at 6:26 PM, Masahiko Sawada wrote: >> "k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward >> compatibility but most users would think "k(n1, n2, n3)" as quorum >> after introduced quorum. >> I wish we can change the

<    1   2