Re: Strange coding in _mdfd_openseg()

2019-04-04 Thread Thomas Munro
On Thu, Apr 4, 2019 at 4:16 PM Kyotaro HORIGUCHI wrote: > At Wed, 3 Apr 2019 13:47:46 -0700, Andres Freund wrote > in <20190403204746.2yumq7c2mirmo...@alap3.anarazel.de> > > Yea, I totally agree it's weird. I'm not sure if I'd go for an assertion > > of equality, or just invert the >= (which I

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Andres Freund
Hi, On 2019-04-05 08:38:34 +0300, Darafei "Komяpa" Praliaskouski wrote: > On Fri, Apr 5, 2019 at 6:58 AM Tom Lane wrote: > > > Andres Freund writes: > > > I think the right approach would be to do all of this in heap_insert and > > > heap_multi_insert. Whenever starting to work on a page, if

Re: GiST VACUUM

2019-04-04 Thread Andrey Borodin
Hi! > 4 апр. 2019 г., в 20:15, Heikki Linnakangas написал(а): > > On 25/03/2019 15:20, Heikki Linnakangas wrote: >> On 24/03/2019 18:50, Andrey Borodin wrote: >>> I was working on new version of gist check in amcheck and understand one >>> more thing: >>> >>> /* Can this page be recycled yet?

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Komяpa
On Fri, Apr 5, 2019 at 6:58 AM Tom Lane wrote: > Andres Freund writes: > > I think the right approach would be to do all of this in heap_insert and > > heap_multi_insert. Whenever starting to work on a page, if INSERT_FROZEN > > is specified, remember whether it is either currently empty, or is

Re: Why does "toast_tuple_target" allow values below TOAST_TUPLE_TARGET?

2019-04-04 Thread David Rowley
On Fri, 5 Apr 2019 at 18:25, Amit Langote wrote: > > On 2019/04/05 14:09, David Rowley wrote: > > Or rather, why does the reloption allow values below the compile-time > > constant? > > Maybe there is already a discussion in progress on the topic? > > * Caveats from reloption toast_tuple_target

Re: Why does "toast_tuple_target" allow values below TOAST_TUPLE_TARGET?

2019-04-04 Thread Amit Langote
On 2019/04/05 14:09, David Rowley wrote: > Or rather, why does the reloption allow values below the compile-time > constant? Maybe there is already a discussion in progress on the topic? * Caveats from reloption toast_tuple_target *

Re: [GSoC] application ideas

2019-04-04 Thread Andrey Borodin
Hi! We are discussing GSoC details offlist, but I'll put some recommendations on your proposal to the list. > 3 апр. 2019 г., в 2:53, pantilimonov misha написал(а): > > Andrey, thank you for your reply. > >> 24.03.2019, 12:12, "Andrey Borodin" : >> >> I like the idea of more efficient

Why does "toast_tuple_target" allow values below TOAST_TUPLE_TARGET?

2019-04-04 Thread David Rowley
Or rather, why does the reloption allow values below the compile-time constant? It looks like we currently allow values as low as 128. The problem there is that heap_update() and heap_prepare_insert() both only bother calling toast_insert_or_update() when the tuple's length is above

Re: [HACKERS] Block level parallel vacuum

2019-04-04 Thread Masahiko Sawada
On Fri, Apr 5, 2019 at 4:51 AM Robert Haas wrote: > > On Thu, Apr 4, 2019 at 6:28 AM Masahiko Sawada wrote: > > These patches conflict with the current HEAD. Attached the updated patches. > > They'll need another rebase. > Thank you for the notice. Rebased. Regards, -- Masahiko Sawada NIPPON

RE: Timeout parameters

2019-04-04 Thread Jamison, Kirk
On Thursday, April 4, 2019 5:20PM (GMT+9), Ryohei Nagaura wrote: > > From: Fabien COELHO > > * About socket_timeout v12 patch, I'm not sure there is a consensus. > I think so too. I just made the patch being able to be committed anytime. > > Finally, I rebased all the patches because they have

Re: Caveats from reloption toast_tuple_target

2019-04-04 Thread Pavan Deolasee
On Thu, Apr 4, 2019 at 11:36 AM Michael Paquier wrote: > > > I mean that toast_tuple_target is broken as-is, because it should be > used on the new tuples of a relation as a threshold to decide if this > tuple should be toasted or not, but we don't actually use the > reloption value for that

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Andres Freund
Hi, On 2019-04-04 23:57:58 -0400, Tom Lane wrote: > Andres Freund writes: > > I think the right approach would be to do all of this in heap_insert and > > heap_multi_insert. Whenever starting to work on a page, if INSERT_FROZEN > > is specified, remember whether it is either currently empty, or

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Andres Freund
Hi, On 2019-04-05 09:20:36 +0530, Pavan Deolasee wrote: > On Fri, Apr 5, 2019 at 9:05 AM Andres Freund wrote: > > I think the right approach would be to do all of this in heap_insert and > > heap_multi_insert. Whenever starting to work on a page, if INSERT_FROZEN > > is specified, remember

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Tom Lane
Andres Freund writes: > I think the right approach would be to do all of this in heap_insert and > heap_multi_insert. Whenever starting to work on a page, if INSERT_FROZEN > is specified, remember whether it is either currently empty, or is > already marked as all-visible. If previously empty,

Re: fix the spelling mistakes of comments

2019-04-04 Thread Tom Lane
Michael Paquier writes: > On Thu, Apr 04, 2019 at 07:57:02AM -0400, Robert Haas wrote: >> On Wed, Apr 3, 2019 at 6:55 AM Liu, Huailing >> wrote: * This module contains the code for waiting and release★ of backends. >>> I think the word marked★ should be 'releasing'. >> It could be

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Pavan Deolasee
On Fri, Apr 5, 2019 at 8:37 AM Andres Freund wrote: > Hi, > > On 2019-04-05 00:06:04 -0300, Alvaro Herrera wrote: > > > > > Hmm, isn't there already a critical section in visibilitymap_set itself? > > There is, but the proposed code sets all visible on the page, and marks > the buffer dirty,

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Pavan Deolasee
Hi, On Fri, Apr 5, 2019 at 9:05 AM Andres Freund wrote: > > > I think the right approach would be to do all of this in heap_insert and > heap_multi_insert. Whenever starting to work on a page, if INSERT_FROZEN > is specified, remember whether it is either currently empty, or is > already marked

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Andres Freund
Hi, On 2019-04-05 00:06:04 -0300, Alvaro Herrera wrote: > On 2019-Apr-04, Andres Freund wrote: > > I still think this is the wrong architecture. > > Hmm. I think the right approach would be to do all of this in heap_insert and heap_multi_insert. Whenever starting to work on a page, if

Re: speeding up planning with partitions

2019-04-04 Thread Amit Langote
On 2019/04/05 12:18, David Rowley wrote: > On Fri, 5 Apr 2019 at 16:09, Amit Langote > wrote: >> Although, we still have ways >> to go in terms of scaling generic plan execution to larger partition >> counts, solution(s) for which have been proposed by David but haven't made >> it into master

Re: Refactoring the checkpointer's fsync request queue

2019-04-04 Thread Thomas Munro
On Thu, Apr 4, 2019 at 11:47 PM Thomas Munro wrote: > Pushed. Thanks for all the work on this! I managed to break this today while testing with RELSEG_SIZE set to 1 block (= squillions of 8kb files). The problem is incorrect arguments to _mdfd_getseg(), in code added recently by me. Without

Re: speeding up planning with partitions

2019-04-04 Thread David Rowley
On Fri, 5 Apr 2019 at 16:09, Amit Langote wrote: > Although, we still have ways > to go in terms of scaling generic plan execution to larger partition > counts, solution(s) for which have been proposed by David but haven't made > it into master yet. Is that a reference to the last paragraph in

RE: Problem during Windows service start

2019-04-04 Thread Higuchi, Daisuke
Hi, Thank you for picking up this and I'm sorry for delay reply. >> If wait_for_postmaster returns POSTMASTER_STILL_STARTING will it >> be correct to set the status of windows service to SERVICE_START_PENDING ? Yes, I think this is the best. Currently, I do not have good solution to change

Re: speeding up planning with partitions

2019-04-04 Thread Amit Langote
On 2019/04/05 6:59, David Rowley wrote: > On Fri, 5 Apr 2019 at 07:33, Floris Van Nee wrote: >> I had a question about the performance of pruning of functions like now() >> and current_date. I know these are handled differently, as they cannot be >> excluded during the first phases of planning.

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Andres Freund
Hi, On 2019-04-05 00:06:04 -0300, Alvaro Herrera wrote: > On 2019-Apr-04, Andres Freund wrote: > > > On 2019-04-04 12:23:08 -0700, Andres Freund wrote: > > > Also, how is this code even close to correct? > > > CheckAndSetPageAllVisible() modifies the buffer in a crucial way, and > > > there's no

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > Kerberos tests are now failing for me (macOS). I'm seeing > > psql: error: could not connect to server: Over-size error packet sent by > the server. > not ok 3 - GSS encryption without auth > > # Failed test 'GSS

RE: libpq debug log

2019-04-04 Thread Iwata, Aya
Hi, > The basic idea being: > > - Each line is a whole message. > - The line begins with <<< for a message received and >>> for a message sent. > - Strings in single quotes are those sent/received as a fixed number of bytes. > - Strings in double quotes are those sent/received as a string. > -

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-04 Thread Imai, Yoshikazu
On Fri, Apr 5, 2019 at 0:05 AM, Tsunakawa, Takayuki wrote: > From: Peter Eisentraut [mailto:peter.eisentr...@2ndquadrant.com] > > I can't detect any performance improvement with the patch applied to > > current master, using the test case from Yoshikazu Imai (2019-03-19). > > That's strange...

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-04 Thread Tsunakawa, Takayuki
Hi Amit-san, Imai-snan, From: Amit Langote [mailto:langote_amit...@lab.ntt.co.jp] > I was able to detect it as follows. > plan_cache_mode = auto > >HEAD: 1915 tps > Patched: 2394 tps > > plan_cache_mode = custom (non-problematic: generic plan is never created) > >HEAD: 2402 tps >

Re: fix the spelling mistakes of comments

2019-04-04 Thread Michael Paquier
On Thu, Apr 04, 2019 at 07:57:02AM -0400, Robert Haas wrote: > On Wed, Apr 3, 2019 at 6:55 AM Liu, Huailing > wrote: >> * This module contains the code for waiting and release★ of backends. >> * All code in this module executes on the primary. The core streaming >> * replication transport

Re: pg_rewind vs superuser

2019-04-04 Thread Michael Paquier
On Thu, Apr 04, 2019 at 01:19:44PM +0200, Magnus Hagander wrote: > All of it, or just the checkpoint part? I assume just the checkpoint part? > AFAIK it does require superuser in those earlier versions? I meant of course the checkpoint part down to 9.5, and the rest down to 11, so done this way.

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-04 Thread Imai, Yoshikazu
On Fri, Apr 5, 2019 at 1:31 AM, Amit Langote wrote: > On 2019/04/05 5:42, Peter Eisentraut wrote: > > On 2019-04-04 06:58, Amit Langote wrote: > >> Also, since the "speed up partition planning" patch went in > >> (428b260f8), it might be possible to see the performance boost even > >> with the

Re: Server Crash due to assertion failure in _bt_check_unique()

2019-04-04 Thread Ashutosh Sharma
On Thu, Apr 4, 2019 at 10:12 PM Peter Geoghegan wrote: > On Thu, Apr 4, 2019 at 4:06 AM Ashutosh Sharma > wrote: > > Attached is the patch with above changes. Please let me know if my > understanding is wrong. Thanks. > > You have it right. This bug slipped in towards the end of development, >

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-04 Thread Amit Langote
On 2019/04/05 5:42, Peter Eisentraut wrote: > On 2019-04-04 06:58, Amit Langote wrote: >> Also, since the "speed up partition planning" patch went in (428b260f8), >> it might be possible to see the performance boost even with the >> partitioning example you cited upthread. > > I can't detect any

Re: New vacuum option to do only freezing

2019-04-04 Thread Masahiko Sawada
On Fri, Apr 5, 2019 at 4:06 AM Robert Haas wrote: > > On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada wrote: > > Attached the updated version patch. > > Committed with a little bit of documentation tweaking. > Thank you for committing them! Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-04 Thread Masahiko Sawada
On Thu, Apr 4, 2019 at 10:07 PM Julien Rouhaud wrote: > > On Thu, Apr 4, 2019 at 1:23 PM Masahiko Sawada wrote: > > > > On Thu, Apr 4, 2019 at 1:26 PM Tsunakawa, Takayuki > > wrote: > > > > > > From: Fujii Masao [mailto:masao.fu...@gmail.com] > > > > reloption for TOAST is also required? > > >

RE: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-04 Thread Tsunakawa, Takayuki
From: Masahiko Sawada [mailto:sawada.m...@gmail.com] > "VACUUM" needs or "vacuum" is more appropriate here? Looking at the same file and some other files, "vacuum" looks appropriate because it represents the vacuum action, not the specific VACUUM command. > The format of the documentation of

Re: Protect syscache from bloating with negative cache entries

2019-04-04 Thread Kyotaro HORIGUCHI
Thank you for the comment. At Thu, 4 Apr 2019 15:44:35 -0400, Robert Haas wrote in > On Thu, Apr 4, 2019 at 8:53 AM Kyotaro HORIGUCHI > wrote: > > So it seems to me that the simplest "Full" version wins. The > > attached is rebsaed version. dlist_move_head(entry) is removed as > > mentioned

Re: COPY FROM WHEN condition

2019-04-04 Thread Andres Freund
Hi, On 2019-04-05 12:59:06 +1300, David Rowley wrote: > I read through the final commit to see what had changed and I noticed > that I had forgotten to remove nbuffers from CopyMultiInsertInfo when > changing from a hash table to a List. > > Patch to fix is attached. Pushed, thanks.

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-04 Thread Tsunakawa, Takayuki
Hi Peter, Imai-san, From: Peter Eisentraut [mailto:peter.eisentr...@2ndquadrant.com] > I can't detect any performance improvement with the patch applied to > current master, using the test case from Yoshikazu Imai (2019-03-19). That's strange... Peter, Imai-san, can you compare your test

Re: COPY FROM WHEN condition

2019-04-04 Thread David Rowley
On Fri, 5 Apr 2019 at 12:32, Andres Freund wrote: > I've pushed this now. Besides those naming changes, I'd to re-add the > zero initialization (I did end up seing compiler warnings when compiling > with optimizations). Also some comment fixes. Thanks for pushing. > I added one more flush

Re: Changes to pg_dump/psql following collation "C" in the catalog

2019-04-04 Thread Tom Lane
"Daniel Verite" writes: > I think psql and pg_dump need to adjust, just like the 3rd party tools > will, at least those that support collation-aware search in the catalog. > PFA a patch that implements the slight changes needed. > I'll add an entry for it in the next CF. Hm, if that's as much as

Re: COPY FROM WHEN condition

2019-04-04 Thread Andres Freund
Hi, On 2019-04-04 12:04:28 -0700, Andres Freund wrote: > On 2019-04-03 22:20:00 -0700, Andres Freund wrote: > > On 2019-04-03 20:00:09 +1300, David Rowley wrote: > > > Oops, I forgot about that one. v4 attached. > > > > I'm pretty happy with this. I'm doing some minor changes (e.g. don't > >

Re: Changes to pg_dump/psql following collation "C" in the catalog

2019-04-04 Thread Chapman Flack
>> "Daniel Verite" writes: >>> One consequence of using the "C" collation in the catalog versus >>> the db collation As an intrigued Person Following At Home, I was happy when I found this little three-message thread had more context in [1] and [2]. :) -Chap [1]

Re: PostgreSQL Buildfarm Client Release 10

2019-04-04 Thread Andrew Dunstan
On Thu, Apr 4, 2019 at 6:18 PM Tom Lane wrote: > > Andrew Dunstan writes: > > The release can be downloaded from > > https://github.com/PGBuildFarm/client-code/archive/REL_10.tar.gz or > > https://buildfarm.postgresql.org/downloads/latest-client.tgz > > I don't actually see it on the

Re: PostgreSQL Buildfarm Client Release 10

2019-04-04 Thread Tom Lane
Andrew Dunstan writes: > The release can be downloaded from > https://github.com/PGBuildFarm/client-code/archive/REL_10.tar.gz or > https://buildfarm.postgresql.org/downloads/latest-client.tgz I don't actually see it on the buildfarm.postgresql.org server? regards, tom

Re: Inadequate executor locking of indexes

2019-04-04 Thread David Rowley
On Fri, 5 Apr 2019 at 08:26, Tom Lane wrote: > Pushed with some minor tweaking, mostly comments. Thanks for tweaking and pushing this. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: speeding up planning with partitions

2019-04-04 Thread David Rowley
On Fri, 5 Apr 2019 at 07:33, Floris Van Nee wrote: > I had a question about the performance of pruning of functions like now() and > current_date. I know these are handled differently, as they cannot be > excluded during the first phases of planning. However, curerntly, this new > patch makes

Re: Refactoring the checkpointer's fsync request queue

2019-04-04 Thread Thomas Munro
On Fri, Apr 5, 2019 at 10:53 AM Thomas Munro wrote: > Ok, here is a patch that adds a one-typedef header and uses > SegmentIndex to replace all cases of BlockNumber and int holding a > segment number (where as an "index" or a "count"). (sorry, I meant "SegmentNumber", not "SegmentIndex") --

PostgreSQL Buildfarm Client Release 10

2019-04-04 Thread Andrew Dunstan
Announcing Release 10 of the PostgreSQL Buildfarm client Principal feature: support for non-standard repositories: . support multi-element branch names, such as “dev/featurename” or “bug/ticket_number/branchname” . provide a get_branches() method in SCM module . support regular expression

Re: Refactoring the checkpointer's fsync request queue

2019-04-04 Thread Thomas Munro
On Fri, Apr 5, 2019 at 2:03 AM Alvaro Herrera wrote: > On 2019-Apr-04, Thomas Munro wrote: > > I don't think it's project policy to put a single typedef into its own > > header like that, and I'm not sure where else to put it. > > shrug. Looks fine to me. I suppose if we don't have it anywhere,

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Andres Freund
Hi, On 2019-04-04 12:23:08 -0700, Andres Freund wrote: > Also, how is this code even close to correct? > CheckAndSetPageAllVisible() modifies the buffer in a crucial way, and > there's no WAL logging? Without even a comment arguing why that's OK (I > don't think it is)? Peter Geoghegan just

Re: New vacuum option to do only freezing

2019-04-04 Thread Bossart, Nathan
On 4/4/19, 12:06 PM, "Robert Haas" wrote: > Committed with a little bit of documentation tweaking. Thanks! I noticed a very small typo in the new documentation. diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index fdd8151220..c652f8b0bc 100644 ---

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2019-04-04 Thread Tom Lane
didier writes: > [ 0001-Add-sqlstate-output-mode-to-VERBOSITY_v1.patch ] Pushed with some mostly-cosmetic adjustments. The main non-cosmetic thing I did was to adjust the logic so that if no SQLSTATE is available, it acts like TERSE mode. Otherwise, we'd print nothing at all except "ERROR:",

Re: query logging of prepared statements

2019-04-04 Thread Justin Pryzby
On Thu, Apr 04, 2019 at 03:07:04PM -0300, Alvaro Herrera wrote: > which does not look good -- the statement is nowhere to be seen. The commit > message quotes this as desirable output: Good catch. Unnamed statements sent behind the scenes by pqExecParams weren't being logged. I specifically

Re: propagating replica identity to partitions

2019-04-04 Thread Alvaro Herrera
On 2019-Mar-29, Peter Eisentraut wrote: > On 2019-03-28 18:16, Simon Riggs wrote: > > SET TABLESPACE should not recurse because it copies the data, while > > holding long locks. If that was ever fixed so it happened concurrently, > > I would agree this could recurse by default. > > Since a

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-04 Thread Peter Eisentraut
On 2019-04-04 06:58, Amit Langote wrote: > Also, since the "speed up partition planning" patch went in (428b260f8), > it might be possible to see the performance boost even with the > partitioning example you cited upthread. I can't detect any performance improvement with the patch applied to

Re: proposal: pg_restore --convert-to-text

2019-04-04 Thread Alvaro Herrera
I just pushed it with trivial changes: * rebased for cc8d41511721 * changed wording in the error message * added a new test for the condition in t/001_basic.pl * Added the "-" in the --help line of -f. Andrew G. never confirmed that this change is sufficient to appease users being confused

Re: [HACKERS] Block level parallel vacuum

2019-04-04 Thread Robert Haas
On Thu, Apr 4, 2019 at 6:28 AM Masahiko Sawada wrote: > These patches conflict with the current HEAD. Attached the updated patches. They'll need another rebase. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3

2019-04-04 Thread Robert Haas
On Thu, Apr 4, 2019 at 11:52 AM Antonin Houska wrote: > Robert Haas wrote: > > I'm willing to put some effort into trying to get this into v13 if > > you're willing to keep hacking on it, but there's probably a fair > > amount to do and a year can go by in a hurry. > > That'd be appreciated,

Re: Protect syscache from bloating with negative cache entries

2019-04-04 Thread Robert Haas
On Thu, Apr 4, 2019 at 8:53 AM Kyotaro HORIGUCHI wrote: > So it seems to me that the simplest "Full" version wins. The > attached is rebsaed version. dlist_move_head(entry) is removed as > mentioned above in that patch. 1. I really don't think this patch has any business changing the existing

Re: Inadequate executor locking of indexes

2019-04-04 Thread Tom Lane
David Rowley writes: > Wrong patch. Here's what I meant to send. Pushed with some minor tweaking, mostly comments. Some notes: * We now have a general convention that queries always take the same lock type on indexes as on their parent tables, but that convention is not respected by index

Re: Changes to pg_dump/psql following collation "C" in the catalog

2019-04-04 Thread Daniel Verite
Tom Lane wrote: > "Daniel Verite" writes: > > One consequence of using the "C" collation in the catalog versus > > the db collation is that pg_dump -t with a regexp may not find > > the same tables as before. It happens when these conditions are > > all met: > > - the collation of the

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Andres Freund
Hi, On 2019-04-04 16:15:54 -0300, Alvaro Herrera wrote: > On 2019-Apr-04, Andres Freund wrote: > > > I'm totally not OK with this from a layering > > POV. CheckAndSetAllVisibleBulkInsertState is entirely heap specific > > (without being named such), whereas all the heap specific bits are > >

Re: Row Level Security − leakproof-ness and performance implications

2019-04-04 Thread Peter Eisentraut
On 2019-03-18 20:08, Peter Eisentraut wrote: > I agree that it would be useful to document and discuss better which > built-in operators are leak-proof and which are not. But I don't think > the CREATE POLICY reference page is the place to do it. Note that the > leak-proofness mechanism was

Re: New vacuum option to do only freezing

2019-04-04 Thread Robert Haas
On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada wrote: > Attached the updated version patch. Committed with a little bit of documentation tweaking. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: query logging of prepared statements

2019-04-04 Thread Alvaro Herrera
On 2019-Apr-04, Alvaro Herrera wrote: > I think we could improve on this by setting a "logged" flag in the > portal; if the Parse logs the statement, then don't include the > statement in further lines, otherwise do. Also: I think such a flag could help the case of a query that takes long enough

Re: speeding up planning with partitions

2019-04-04 Thread Floris Van Nee
Hi all, First of all I would like to thank everyone involved in this patch for their hard work on this. This is a big step forward. I've done some performance and functionality testing with the patch that was committed to master and it looks very good. I had a question about the performance of

Re: query logging of prepared statements

2019-04-04 Thread Alvaro Herrera
On 2019-Apr-04, Alvaro Herrera wrote: > However, turning duration logging off and using log_statement=all, this is > what > I get: > > 2019-04-04 14:58:42.564 -03 [31685] LOG: statement: SET search_path = > testlibpq3 > 2019-04-04 14:58:42.565 -03 [31685] LOG: execute > 2019-04-04

Re: query logging of prepared statements

2019-04-04 Thread Alvaro Herrera
On 2019-Mar-04, Justin Pryzby wrote: > Thanks for reviewing. I'm also interested in discussion about whether this > change is undesirable for someone else for some reason ? For me, the existing > output seems duplicative and "denormalized". :) Some digging turned up that the function you're

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Robbie Harwood
Tom Lane writes: > I wrote: >> Stephen Frost writes: >>> So I'm a bit surprised that it's taking 4 minutes for you. I wonder if >>> there might be an issue related to the KDC wanting to get some amount of >>> random data and the system you're on isn't producing random bytes very >>> fast..? >

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Robbie Harwood
Tom Lane writes: > Stephen Frost writes: >> * Tom Lane (t...@sss.pgh.pa.us) wrote: >>> Well, if the caller thinks what is being passed back is an int, >>> it will do a 32-to-64-bit widening, which is almost certainly >>> going to result in a corrupted pointer. > >> Oh, good point. Interesting

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-04 Thread Tomas Vondra
Hi, I've been looking at this patch for a while, and it seems pretty much RFC, so barring objections I'll take care of that once I do a bit more testing and review. Unless someone else wants to take care of that. FWIW I wonder if we should add the code for partitioned tables to CopyFrom,

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Watching the test logs, I see that essentially all the time on the RHEL6 >> machine is consumed by the two >> # Running: /usr/sbin/kdb5_util create -s -P secret0 >> steps. Is there a case for merging the two scripts so we only

Re: [GSoC 2019] Proposal: Develop Performance Farm Database and Website

2019-04-04 Thread Mark Wong
Hi Ilaria, Edited for bottom posting. :) On Fri, Mar 29, 2019 at 03:01:05PM +0100, Ilaria wrote: > > Am 29.03.2019 um 13:52 schrieb Peter Eisentraut > > : > > > >> On 2019-03-29 13:04, Robert Haas wrote: > >>> On Tue, Mar 26, 2019 at 9:10 AM Ila B. wrote: > >>> I am Ilaria Battiston, an

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Tom Lane
I wrote: > Stephen Frost writes: >> So I'm a bit surprised that it's taking 4 minutes for you. I wonder if >> there might be an issue related to the KDC wanting to get some amount of >> random data and the system you're on isn't producing random bytes very >> fast..? > Not sure. This is my

Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3

2019-04-04 Thread Antonin Houska
Robert Haas wrote: > I'm willing to put some effort into trying to get this into v13 if > you're willing to keep hacking on it, but there's probably a fair > amount to do and a year can go by in a hurry. That'd be appreciated, especially by my boss. It doesn't seem likely that in this situation

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > Kerberos tests are now failing for me (macOS). I'm seeing > > psql: error: could not connect to server: Over-size error packet sent by > the server. > not ok 3 - GSS encryption without auth > > # Failed test 'GSS

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> There must be something about the x86_64 ABI that allows this to >> accidentally work -- maybe integers are presumed to be sign-extended >> to 64 bits by callee not caller? I added some logging and verified >> that pgstat.c is

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> Well, if the caller thinks what is being passed back is an int, > >> it will do a 32-to-64-bit widening, which is almost certainly > >> going to result in a corrupted

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Peter Eisentraut
On 2019-04-04 17:16, Tom Lane wrote: > BTW, the kerberos test suite takes nearly 4 minutes for me, is > it supposed to be so slow? I've seen this on some virtualized machines that didn't have a lot of entropy. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Well, if the caller thinks what is being passed back is an int, >> it will do a 32-to-64-bit widening, which is almost certainly >> going to result in a corrupted pointer. > Oh, good point. Interesting that it still works then.

Re: GiST VACUUM

2019-04-04 Thread Heikki Linnakangas
On 25/03/2019 15:20, Heikki Linnakangas wrote: On 24/03/2019 18:50, Andrey Borodin wrote: I was working on new version of gist check in amcheck and understand one more thing: /* Can this page be recycled yet? */ bool gistPageRecyclable(Page page) { return PageIsNew(page) ||

Re: POC: GROUP BY optimization

2019-04-04 Thread Dmitry Dolgov
> On Thu, Jan 31, 2019 at 12:24 PM Andres Freund wrote: > > As nothing has happened since, I'm marking this as returned with > feedback. This patch was on my radar for some time in the past and we've seen use cases where it could be pretty useful (probably even without the incremental sort

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> I'm not very sure why the integer/pointer confusion in pgstat_bestart > >> doesn't cause hard crashes when using gss auth --- or does > >> this suite not actually test that?

Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3

2019-04-04 Thread Robert Haas
On Thu, Apr 4, 2019 at 9:57 AM Antonin Houska wrote: > I think I finally understand. Originally I thought the question is how to > compute correct page checksum while the hint bits can be changed w/o exclusive > lock on the buffer. Now I realize that it's more about *recovery*: if the hint > bit

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I'm not very sure why the integer/pointer confusion in pgstat_bestart >> doesn't cause hard crashes when using gss auth --- or does >> this suite not actually test that? > Isn't it just saying that because of the implicit

Re: [HACKERS] Weaker shmem interlock w/o postmaster.pid

2019-04-04 Thread Noah Misch
On Wed, Apr 03, 2019 at 07:05:43PM -0700, Noah Misch wrote: > On Mon, Apr 01, 2019 at 08:19:56AM +, Daniel Gustafsson wrote: > > On Monday, April 1, 2019 12:42 AM, Noah Misch wrote: > > > On Fri, Mar 29, 2019 at 09:53:51AM +, Daniel Gustafsson wrote: > > > > > > This seems like a case

Re: pg_upgrade: Pass -j down to vacuumdb

2019-04-04 Thread Tom Lane
Peter Eisentraut writes: > I think the real fix is to have pg_upgrade copy over the statistics. > They are right there after all, just take them. Well, it's not "just take them", we need to develop some infrastructure that will permit coping with cross-version differences in the stats storage.

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > On Thu, Apr 4, 2019 at 05:20 Peter Eisentraut < > > peter.eisentr...@2ndquadrant.com> wrote: > >> Kerberos tests are now failing for me (macOS). > > > Interesting, they work locally for me on Ubuntu. Unfortunately, I

Re: [PATCH v20] GSSAPI encryption support

2019-04-04 Thread Tom Lane
Stephen Frost writes: > On Thu, Apr 4, 2019 at 05:20 Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> wrote: >> Kerberos tests are now failing for me (macOS). > Interesting, they work locally for me on Ubuntu. Unfortunately, I don’t > have macOS. This only happens when encryption is

Re: [HACKERS] generated columns

2019-04-04 Thread Amit Langote
On Thu, Apr 4, 2019 at 8:01 PM Peter Eisentraut wrote: > > On 2019-04-04 11:42, Amit Langote wrote: > > Hmm, I'm afraid we might get bug reports if we go with this. Why is it OK > > to get null in this case when a user explicitly asked for 'foo'? > > The way stored generated columns work on

Re: pg_upgrade: Pass -j down to vacuumdb

2019-04-04 Thread Peter Eisentraut
On 2019-04-03 23:24, Justin Pryzby wrote: > I just did a test on one of our large-but-not-huge customers. With > stats_target=1, analyzing a 145GB partitioned table looks like it'll take > perhaps an hour; they have ~1TB data, so delaying services during ANALYZE > would > nullify the utility of

Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3

2019-04-04 Thread Antonin Houska
Robert Haas wrote: > On Thu, Mar 14, 2019 at 9:26 AM Antonin Houska wrote: > > > Hint bits also rely on this principle. I thought there might be some > > > interaction between this work and wal_log_hints for this reason, but I see > > > nothing of that sort in the patch. > > > > I'm not sure

Re: COPY FROM WHEN condition

2019-04-04 Thread David Rowley
On Thu, 4 Apr 2019 at 18:20, Andres Freund wrote: > I'm pretty happy with this. I'm doing some minor changes (e.g. don't > like the function comment formatting that much, the tableam callback > needs docs, stuff like that), and then I'm going to push it tomorrow. > > I'm planning to attribute it

Re: Retronym: s/magnetic disk/main data/

2019-04-04 Thread David Fetter
On Thu, Apr 04, 2019 at 05:49:36PM +1300, Thomas Munro wrote: > Hello, > > I propose that in v13 we redefine "MD" (from md.c) to mean "main data" > (instead of "magnetic disk"). That's the standard storage layout for > typical table and index AMs. As opposed to the proposed undo and SLRU >

Re: pg_upgrade version checking questions

2019-04-04 Thread Daniel Gustafsson
On Wednesday, March 27, 2019 1:43 PM, Christoph Berg wrote: > Re: Daniel Gustafsson 2019-03-26 > pC-NMmh4vQLQP76YTwY4AuoD4OdNw9egikekyQpXFpgqmTlGjIZXOTd2W5RDZPpRski5N3ADRrLYgLk6QUuvmuT5fWC9acPAYyDU1AVxJcU=@yesql.se > > > 0003 - Make -B default to CWD and remove the exec_path check > >

Re: Inadequate executor locking of indexes

2019-04-04 Thread David Rowley
On Fri, 5 Apr 2019 at 02:22, David Rowley wrote: > v2 attached. Wrong patch. Here's what I meant to send. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services use_rellockmode_for_indexes_too_v3.patch Description: Binary

Re: Inadequate executor locking of indexes

2019-04-04 Thread David Rowley
On Thu, 4 Apr 2019 at 15:35, Amit Langote wrote: > BTW, get_actual_variable_range is static to selfuncs.c and other public > functions that are entry point to get_actual_variable_range's > functionality appear to require having built a RelOptInfo first, which > means (even for third party code)

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-04 Thread Julien Rouhaud
On Thu, Apr 4, 2019 at 1:23 PM Masahiko Sawada wrote: > > On Thu, Apr 4, 2019 at 1:26 PM Tsunakawa, Takayuki > wrote: > > > > From: Fujii Masao [mailto:masao.fu...@gmail.com] > > > reloption for TOAST is also required? > > > > # I've come back to the office earlier than planned... > > > > Hm,

Re: Refactoring the checkpointer's fsync request queue

2019-04-04 Thread Alvaro Herrera
On 2019-Apr-04, Thomas Munro wrote: > I don't think it's project policy to put a single typedef into its own > header like that, and I'm not sure where else to put it. shrug. Looks fine to me. I suppose if we don't have it anywhere, it's just because we haven't needed that particular trick

  1   2   >