Re: Extending USE_MODULE_DB to more test suite types

2019-04-03 Thread Noah Misch
On Mon, Apr 01, 2019 at 01:01:11PM -0400, Andrew Dunstan wrote: > On Mon, Apr 1, 2019 at 9:52 AM Noah Misch wrote: > > I plan to back-patch the PostgreSQL patch, to combat buildfarm noise. > > Perhaps > > someone has test automation that sets USE_MODULE_DB and nonetheless probes > > the > >

Re: Google Summer of Code: question about GiST API advancement project

2019-04-03 Thread GUO Rui
Dear Andrey Borodin, I discussed the above topic with the professors at my school, and got the following points: 1. The case that the volume of an MBB is 0 should be very rare, and if the data is skewed (e.g. only a few nodes have non-NULL value on a dimension) then the data can be pre-proceeded

Re: Pluggable Storage - Andres's take

2019-04-03 Thread Justin Pryzby
I reviewed new docs for $SUBJECT. Find attached proposed changes. There's one XXX item I'm unsure what it's intended to say. Justin >From a3d290bf67af2a34e44cd6c160daf552b56a13b5 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 4 Apr 2019 00:48:09 -0500 Subject: [PATCH v1] Fine tune

Re: COPY FROM WHEN condition

2019-04-03 Thread Andres Freund
Hi, On 2019-04-03 20:00:09 +1300, David Rowley wrote: > Oops, I forgot about that one. v4 attached. I'm pretty happy with this. I'm doing some minor changes (e.g. don't like the function comment formatting that much, the tableam callback needs docs, stuff like that), and then I'm going to push

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-03 Thread Amit Langote
Hi, On 2019/04/04 13:37, Tsunakawa, Takayuki wrote: > Hi Peter, > > From: Peter Eisentraut [mailto:peter.eisentr...@2ndquadrant.com] >> I did a bit of performance testing, both a plain pgbench and the >> suggested test case with 4096 partitions. I can't detect any >> performance improvements.

Retronym: s/magnetic disk/main data/

2019-04-03 Thread Thomas Munro
Hello, I propose that in v13 we redefine "MD" (from md.c) to mean "main data" (instead of "magnetic disk"). That's the standard storage layout for typical table and index AMs. As opposed to the proposed undo and SLRU SMGRs that provide layouts specialised for different life cycles. -- Thomas

Re: Refactoring the checkpointer's fsync request queue

2019-04-03 Thread Thomas Munro
On Thu, Apr 4, 2019 at 5:36 PM Andres Freund wrote: > On 2019-04-03 21:19:45 -0700, Shawn Debnath wrote: > > +typedef struct FileTag > > +{ > > + int16 handler;/* SyncRequstHandler value, > > saving space */ > > + int16 forknum;/*

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-03 Thread Tsunakawa, Takayuki
Hi Peter, From: Peter Eisentraut [mailto:peter.eisentr...@2ndquadrant.com] > I did a bit of performance testing, both a plain pgbench and the > suggested test case with 4096 partitions. I can't detect any > performance improvements. In fact, within the noise, it tends to be > just a bit on the

Re: Refactoring the checkpointer's fsync request queue

2019-04-03 Thread Andres Freund
On 2019-04-03 21:19:45 -0700, Shawn Debnath wrote: > On Thu, Apr 04, 2019 at 02:01:14PM +1300, Thomas Munro wrote: > > On Thu, Apr 4, 2019 at 11:39 AM Thomas Munro wrote: > > > ... Perhaps > > > that is an argument for putting the sync handler number *inside* the > > > FileTag, since we currently

Re: Transaction commits VS Transaction commits (with parallel) VS query mean time

2019-04-03 Thread Amit Kapila
On Wed, Apr 3, 2019 at 10:45 AM Haribabu Kommi wrote: > > Thanks for the review. > > While changing the approach to use the is_parallel_worker_flag, I thought > that the rest of the stats are also not required to be updated and also those > are any way write operations and those values are zero

RE: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-03 Thread Tsunakawa, Takayuki
From: Fujii Masao [mailto:masao.fu...@gmail.com] > reloption for TOAST is also required? # I've come back to the office earlier than planned... Hm, there's no reason to not provide toast.vacuum_shrink_enabled. Done with the attached patch. Regards Takayuki Tsunakawa

Re: Checksum errors in pg_stat_database

2019-04-03 Thread Michael Paquier
On Wed, Apr 03, 2019 at 11:56:14AM +0200, Julien Rouhaud wrote: > But there's still the problem of reporting errors on shared relation, > so pg_stat_database doesn't really fit for that. If we go with a > checksum centric view, it'd be strange to have some of the counters in > another view.

Re: Concurrency bug with vacuum full (cluster) and toast

2019-04-03 Thread Michael Paquier
On Wed, Apr 03, 2019 at 11:26:20AM -0400, Stephen Frost wrote: > If you're referring to the v12 open items list, then, no, I wouldn't > think it would be as it's not a new issue (unless I've misunderstood). > Only regressions from prior versions are appropriate for the v12 open > items list,

Re: pg_rewind vs superuser

2019-04-03 Thread Michael Paquier
On Wed, Apr 03, 2019 at 11:28:50AM +0200, Magnus Hagander wrote: > As pointed out by Michael Banck as a comment on my blogpost, the pg_rewind > documentation says it requires superuser permissions on the remote server. > > Is that really so, though? I haven't tested it, but from a quick look at >

RE: Problem with default partition pruning

2019-04-03 Thread Yuzuko Hosoya
Amit-san, Thanks for the comments. > > Thanks for dividing patches that way. > > Would it be a good idea to add some new test cases to these patches, just so > it's easily apparent what > we're changing? Yes, I agree with you. > > So, we could add the test case presented by Thibaut at the

message when starting standby without setting restore_command

2019-04-03 Thread Kyotaro HORIGUCHI
Hello. When I tried to start server just putting recovery.signal, I got the following message. > FATAL: must specify restore_command when standby mode is not enabled I got a bit confused to see the message. Formerly this message was shown when recovery.conf that is setting standby_mode to yes

Re: [PATCH v20] GSSAPI encryption support

2019-04-03 Thread Michael Paquier
On Wed, Apr 03, 2019 at 10:09:54PM -0400, Stephen Frost wrote: > Yes, that’s a fair point. I’ll work on adding documentation to > protocol.sgml for the GSSAPI encrypted setup and message passing. Thanks. I have added an open item to track. -- Michael signature.asc Description: PGP signature

Re: Strange coding in _mdfd_openseg()

2019-04-03 Thread Kyotaro HORIGUCHI
Hello. At Wed, 3 Apr 2019 13:47:46 -0700, Andres Freund wrote in <20190403204746.2yumq7c2mirmo...@alap3.anarazel.de> > Hi, > > On 2019-04-04 09:24:49 +1300, Thomas Munro wrote: > > On Wed, Apr 3, 2019 at 5:34 PM Kyotaro HORIGUCHI > > wrote: > > > I may be missing something, but it seems

Re: Inadequate executor locking of indexes

2019-04-03 Thread Amit Langote
On 2019/04/04 11:13, David Rowley wrote: > On Thu, 4 Apr 2019 at 15:01, Amit Langote > wrote: >> Sorry, I didn't understand why it wouldn't be OK to pass NoLock to >> index_open, for example, here: >> >> @@ -5191,7 +5191,14 @@ get_actual_variable_range(PlannerInfo *root, >> VariableStatData

Re: New vacuum option to do only freezing

2019-04-03 Thread Masahiko Sawada
On Thu, Apr 4, 2019 at 9:18 AM Kyotaro HORIGUCHI wrote: > > Hello. > > At Wed, 3 Apr 2019 11:55:00 -0400, Robert Haas wrote > in > > On Wed, Apr 3, 2019 at 1:32 AM Masahiko Sawada > > wrote: > > > Attached the updated version patches including the > > > DISABLE_PAGE_SKIPPING part (0003). > >

Re: allow online change primary_conninfo

2019-04-03 Thread Andres Freund
On 2019-04-04 11:06:05 +0900, Michael Paquier wrote: > On Thu, Apr 04, 2019 at 12:27:55AM +0300, Sergei Kornilov wrote: > > Not agree. Latest patch version perform walreceiver restart without > > switch to a different method as discussed. Here is no race condition > > between startup process and

Re: Inadequate executor locking of indexes

2019-04-03 Thread David Rowley
On Thu, 4 Apr 2019 at 15:01, Amit Langote wrote: > Sorry, I didn't understand why it wouldn't be OK to pass NoLock to > index_open, for example, here: > > @@ -5191,7 +5191,14 @@ get_actual_variable_range(PlannerInfo *root, > VariableStatData *vardata, > * necessarily on

Re: [PATCH v20] GSSAPI encryption support

2019-04-03 Thread Stephen Frost
Greetings, On Wed, Apr 3, 2019 at 22:02 Michael Paquier wrote: > On Wed, Apr 03, 2019 at 05:51:06PM -0400, Stephen Frost wrote: > > Thanks so much for pushing on it for so long, it’s a great feature to > have! > > Glad to see that the final result is using an API layer in > be-secure.c and that

Re: allow online change primary_conninfo

2019-04-03 Thread Michael Paquier
On Thu, Apr 04, 2019 at 12:27:55AM +0300, Sergei Kornilov wrote: > Not agree. Latest patch version perform walreceiver restart without > switch to a different method as discussed. Here is no race condition > between startup process and walreceiver because conninfo passed via > WalRcvData struct as

Re: [HACKERS] Weaker shmem interlock w/o postmaster.pid

2019-04-03 Thread Noah Misch
On Mon, Apr 01, 2019 at 08:19:56AM +, Daniel Gustafsson wrote: > On Monday, April 1, 2019 12:42 AM, Noah Misch wrote: > > On Fri, Mar 29, 2019 at 09:53:51AM +, Daniel Gustafsson wrote: > > > > This seems like a case where it would be useful to log a shmdt() error or > > > do > > > an

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-04-03 Thread Kyotaro HORIGUCHI
Thank you for looking this. At Wed, 3 Apr 2019 10:16:02 -0400, Robert Haas wrote in > On Tue, Apr 2, 2019 at 6:54 AM Kyotaro HORIGUCHI > wrote: > > > By using DELETE and INSERT records to implement an UPDATE, you lose the > > > ctid > > > chain and infomask bits that were present before

Re: [PATCH v20] GSSAPI encryption support

2019-04-03 Thread Michael Paquier
On Wed, Apr 03, 2019 at 05:51:06PM -0400, Stephen Frost wrote: > Thanks so much for pushing on it for so long, it’s a great feature to have! Glad to see that the final result is using an API layer in be-secure.c and that we have tests. Now, shouldn't there be some documentation in protocol.sgml

Re: Inadequate executor locking of indexes

2019-04-03 Thread Amit Langote
Hi David, Thanks for updating the patch. On 2019/04/04 9:14, David Rowley wrote: > I also looked over other index_open() calls in the planner and found a > bunch of places in selfuncs.c that we open an index to grab some > information then close it again releasing the lock. At this stage >

Re: Simplify redability of some tests for toast_tuple_target in strings.sql

2019-04-03 Thread Michael Paquier
On Wed, Apr 03, 2019 at 09:38:57AM +, Daniel Gustafsson wrote: > +1, that's much more readable. Thanks! Thanks for the lookup, done. -- Michael signature.asc Description: PGP signature

Re: Refactoring the checkpointer's fsync request queue

2019-04-03 Thread Thomas Munro
On Thu, Apr 4, 2019 at 11:39 AM Thomas Munro wrote: > ... Perhaps > that is an argument for putting the sync handler number *inside* the > FileTag, since we currently intend to do that with smgr IDs in > BufferTag (stealing space from ForkNumber). Here is a version like that. I like it better

Re: partitioned tables referenced by FKs

2019-04-03 Thread Amit Langote
On 2019/04/04 4:45, Jesper Pedersen wrote: > On 4/3/19 1:52 PM, Alvaro Herrera wrote: >> Pushed, many thanks Amit and Jesper for reviewing. >> > > Thank you for working on this feature. +1, thank you. Regards, Amit

Re: shared-memory based stats collector

2019-04-03 Thread Kyotaro HORIGUCHI
Hello. At Wed, 3 Apr 2019 12:56:59 -0700, Andres Freund wrote in <20190403195659.fcmk2i7ruxhty...@alap3.anarazel.de> > Unfortunately I don't think it's realistic to target this to v12. I > think it was unlikely to make at the beginning of the CF, but since then > development just wasn't quick

RE: minimizing pg_stat_statements performance overhead

2019-04-03 Thread Raymond Martin
From: Robert Haas >> >> One thing that needs some thought here is what happens if the value of >> pgss_enabled() changes. For example we don't want a situation where >> if the value changes from off to on between one stage of processing >> and another, the server crashes. >> >> I don't know

Re: New vacuum option to do only freezing

2019-04-03 Thread Kyotaro HORIGUCHI
Hello. At Wed, 3 Apr 2019 11:55:00 -0400, Robert Haas wrote in > On Wed, Apr 3, 2019 at 1:32 AM Masahiko Sawada wrote: > > Attached the updated version patches including the > > DISABLE_PAGE_SKIPPING part (0003). > > I am confused about nleft_dead_tuples. It looks like it gets > incremented

Re: Inadequate executor locking of indexes

2019-04-03 Thread David Rowley
On Wed, 3 Apr 2019 at 06:03, Tom Lane wrote: > > David Rowley writes: > > On Thu, 14 Mar 2019 at 21:52, Amit Langote > > Maybe you're right about being able to use rellockmode for indexes, > > but I assume that we lowered the lock level for indexes for some > > reason, and this would reverse

RE: minimizing pg_stat_statements performance overhead

2019-04-03 Thread Raymond Martin
Hi Christoph, > you sent the patch as UTF-16, could you re-send it as plain ascii? Apologies. I re-attached the plain ascii version. -- Raymond Martin rama...@microsoft.com Azure Database for PostgreSQL check_pgss_enabled.patch Description: check_pgss_enabled.patch

Re: Refactoring the checkpointer's fsync request queue

2019-04-03 Thread Thomas Munro
On Thu, Apr 4, 2019 at 10:44 AM Shawn Debnath wrote: > On Thu, Apr 04, 2019 at 01:08:31AM +1300, Thomas Munro wrote: > > As a sanity check on the programming interface this thing gives you, I > > tried teaching the SLRUs to use the fsync queue. I finished up making > > a few small improvements,

Re: explain plans with information about (modified) gucs

2019-04-03 Thread Tomas Vondra
Hi, I've committed this, with some minor documentation tweaks. I've also fixed a minor bug in the last patch, where the group with settings was not properly labeled in some formats (e.g. json). Thanks to all the reviewers! regards -- Tomas Vondra http://www.2ndQuadrant.com

Re: clean up docs for log_statement_sample_rate

2019-04-03 Thread Alvaro Herrera
On 2019-Mar-28, Justin Pryzby wrote: > Following the dicussion here, > https://www.postgresql.org/message-id/flat/CAD21AoB9%2By8N4%2BFan-ne-_7J5yTybPttxeVKfwUocKp4zT1vNQ%40mail.gmail.com#90a8316b1e643532e1cdb352c91c22a7 > > I'm proposing these changes to clean up docs for previous (more or less

Re: [PATCH v20] GSSAPI encryption support

2019-04-03 Thread Stephen Frost
Greetings Robbie, On Wed, Apr 3, 2019 at 17:47 Robbie Harwood wrote: > Stephen Frost writes: > > > On Wed, Apr 3, 2019 at 16:01 Andres Freund wrote: > >> On 2019-04-03 10:43:33 -0400, Stephen Frost wrote: > >> > >>> I'll push this in a few hours unless there's anything else. > >> > >> The CF

Re: [PATCH v20] GSSAPI encryption support

2019-04-03 Thread Robbie Harwood
Stephen Frost writes: > On Wed, Apr 3, 2019 at 16:01 Andres Freund wrote: >> On 2019-04-03 10:43:33 -0400, Stephen Frost wrote: >> >>> I'll push this in a few hours unless there's anything else. >> >> The CF entry for this is still open - is there any work missing? Just >> trying to do some

Re: Log a sample of transactions

2019-04-03 Thread Alvaro Herrera
On 2019-Mar-29, Adrien NAYRAT wrote: > On 3/29/19 3:06 AM, Kuroda, Hayato wrote: > > Dear Adrien, > > Hello Kuroda-san, > > > > > I understood the cost of randomizing is very low. Maybe it's OK.. > > I'll change the status to "Ready For Committer." > > Thanks, I hope it will be commited in

Re: Implementing Incremental View Maintenance

2019-04-03 Thread Greg Stark
On Sun, 31 Mar 2019 at 23:22, Yugo Nagata wrote: > > Firstly, this will handle simple definition views which includes only > selection, projection, and join. Standard aggregations (count, sum, avg, > min, max) are not planned to be implemented in the first patch, but these > are commonly used in

Re: allow online change primary_conninfo

2019-04-03 Thread Sergei Kornilov
Hi >>  > I think we unfortunately got to mark this as returned with >>  > feedback. I've not done so, but just switched the entry to waiting on >>  > author. >> >>  Why returned with feedback? Why waiting on author? I didn't receive a >>  feedback for latest published patch version. What can I

Re: pg_upgrade: Pass -j down to vacuumdb

2019-04-03 Thread Justin Pryzby
On Wed, Apr 03, 2019 at 04:42:14PM -0400, Jeff Janes wrote: > So maybe the first stage could > be run by pg_upgrade itself, while the new server is still running on a > linux socket in a private directory. I think that would take too long. It would be less of an issue if there was

Re: CPU costs of random_zipfian in pgbench

2019-04-03 Thread Tom Lane
Fabien COELHO writes: >> Ah, so we now we can get rid of the TState * being passed around >> separately for expression execution, too? > Indeed. Indeed. Pushed with some additional cleanup. regards, tom lane

Re: allow online change primary_conninfo

2019-04-03 Thread Andres Freund
Hi, On 2019-04-03 23:49:54 +0300, Sergei Kornilov wrote: > > I think we unfortunately got to mark this as returned with > > feedback. I've not done so, but just switched the entry to waiting on > > author. > > Why returned with feedback? Why waiting on author? I didn't receive a > feedback for

Re: allow online change primary_conninfo

2019-04-03 Thread Sergei Kornilov
Hi > I think we unfortunately got to mark this as returned with > feedback. I've not done so, but just switched the entry to waiting on > author. Why returned with feedback? Why waiting on author? I didn't receive a feedback for latest published patch version. What can I do as author? Patch

Re: Strange coding in _mdfd_openseg()

2019-04-03 Thread Andres Freund
Hi, On 2019-04-04 09:24:49 +1300, Thomas Munro wrote: > On Wed, Apr 3, 2019 at 5:34 PM Kyotaro HORIGUCHI > wrote: > > I may be missing something, but it seems possible that > > _mdfd_getseg calls it with segno > opensegs. > > > > | for (nextsegno = reln->md_num_open_segs[forknum]; > > Here

Re: pg_upgrade: Pass -j down to vacuumdb

2019-04-03 Thread Jeff Janes
On Fri, Mar 29, 2019 at 5:58 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-03-28 02:43, Jeff Janes wrote: > > At first blush I thought it was obvious that you would not want to run > > analyze-in-stages in parallel. But after thinking about it some more > > and

Re: Strange coding in _mdfd_openseg()

2019-04-03 Thread Thomas Munro
On Wed, Apr 3, 2019 at 5:34 PM Kyotaro HORIGUCHI wrote: > I may be missing something, but it seems possible that > _mdfd_getseg calls it with segno > opensegs. > > | for (nextsegno = reln->md_num_open_segs[forknum]; Here nextsegno starts out equal to opensegs. > | nextsegno <=

Re: minimizing pg_stat_statements performance overhead

2019-04-03 Thread legrand legrand
Robert Haas wrote > On Tue, Apr 2, 2019 at 5:37 AM Christoph Berg > myon@ > wrote: >> Re: Raymond Martin 2019-04-01 > BN8PR21MB121708579A3782866DF1F745B1550@.outlook > >> > Thanks again Fabien. I am attaching the patch to this email in the hope >> of getting it approved during the next

Re: jsonpath

2019-04-03 Thread Alexander Korotkov
On Wed, Apr 3, 2019 at 11:03 PM Andres Freund wrote: > > On 2019-03-30 18:25:12 +0300, Alexander Korotkov wrote: > > I'm going to push there 3 attached patches for jsonpath. > > I noticed that > https://commitfest.postgresql.org/22/1472/ > https://commitfest.postgresql.org/22/1473/ > are still

Re: [PATCH v20] GSSAPI encryption support

2019-04-03 Thread Stephen Frost
Greetings, On Wed, Apr 3, 2019 at 16:01 Andres Freund wrote: > Hi, > > On 2019-04-03 10:43:33 -0400, Stephen Frost wrote: > > I'll push this in a few hours unless there's anything else. > > The CF entry for this is still open - is there any work missing? Just > trying to do some triage... > >

Re: Conflict handling for COPY FROM

2019-04-03 Thread Andres Freund
Hi, On 2019-03-25 12:50:13 +0400, David Steele wrote: > On 2/20/19 8:03 PM, Andres Freund wrote: > > On February 20, 2019 6:05:53 AM PST, Andrew Dunstan > > wrote: > > > > > > Why log to a file at all? We do have, you know, a database handy, where > > > we might more usefully log errors. You

Re: jsonpath

2019-04-03 Thread Andres Freund
On 2019-03-30 18:25:12 +0300, Alexander Korotkov wrote: > I'm going to push there 3 attached patches for jsonpath. I noticed that https://commitfest.postgresql.org/22/1472/ https://commitfest.postgresql.org/22/1473/ are still open and marked as needs-review. Is there a reason for that? - Andres

Re: [PATCH v20] GSSAPI encryption support

2019-04-03 Thread Andres Freund
Hi, On 2019-04-03 10:43:33 -0400, Stephen Frost wrote: > I'll push this in a few hours unless there's anything else. The CF entry for this is still open - is there any work missing? Just trying to do some triage... https://commitfest.postgresql.org/22/1647/ - Andres

Re: allow online change primary_conninfo

2019-04-03 Thread Andres Freund
Hi, On 2019-03-25 12:32:21 +0400, David Steele wrote: > On 3/4/19 7:19 AM, Michael Paquier wrote: > > On Sat, Mar 02, 2019 at 01:49:51PM +0300, Sergei Kornilov wrote: > > > This might be not the right way, but I can't think of a better way > > > to not switch to a different method than split of

Re: shared-memory based stats collector

2019-04-03 Thread Andres Freund
Hi, Unfortunately I don't think it's realistic to target this to v12. I think it was unlikely to make at the beginning of the CF, but since then development just wasn't quick enough to warrant aiming for it. It's a large and somewhat complex patch, and has some significant risks associated.

Re: CPU costs of random_zipfian in pgbench

2019-04-03 Thread Fabien COELHO
Ah, so we now we can get rid of the TState * being passed around separately for expression execution, too? Indeed. I would have thought that the compiler would have warned if it is unused, but because of the recursion it is uselessly used. Ok, maybe the sentence above is not very clear.

Re: partitioned tables referenced by FKs

2019-04-03 Thread Jesper Pedersen
On 4/3/19 1:52 PM, Alvaro Herrera wrote: Pushed, many thanks Amit and Jesper for reviewing. Thank you for working on this feature. Best regards, Jesper

Re: FETCH FIRST clause WITH TIES option

2019-04-03 Thread Tomas Vondra
On Wed, Apr 03, 2019 at 03:08:05PM -0400, Tom Lane wrote: Tomas Vondra writes: I've tried to fix the merge conflict (essentially by moving some of the code to adjust_limit_rows_costs(), but I'm wondering if the code added to create_limit_path is actually correct ... Firstly, this seriously

Re: FETCH FIRST clause WITH TIES option

2019-04-03 Thread Tom Lane
Tomas Vondra writes: > I've tried to fix the merge conflict (essentially by moving some of the > code to adjust_limit_rows_costs(), but I'm wondering if the code added to > create_limit_path is actually correct > ... > Firstly, this seriously needs some comment explaining why we do this. I've

Re: FETCH FIRST clause WITH TIES option

2019-04-03 Thread Tomas Vondra
Hi, Unfortunately this got broken again, this time by aef65db676 :-( I've tried to fix the merge conflict (essentially by moving some of the code to adjust_limit_rows_costs(), but I'm wondering if the code added to create_limit_path is actually correct if (count_est != 0) { double

Re: [PATCH] src/test/modules/dummy_index -- way to test reloptions from inside of access method

2019-04-03 Thread Nikolay Shaplov
В письме от вторник, 19 марта 2019 г. 16:09:13 MSK пользователь Michael Paquier написал: > > So I created src/test/modules/dummy_index, it does no real indexing, but > > it > > has all types of reloptions that can be set (reloption_int, > > reloption_real, > > reloption_bool, reloption_string

Re: partitioned tables referenced by FKs

2019-04-03 Thread Alvaro Herrera
Pushed, many thanks Amit and Jesper for reviewing. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Caveats from reloption toast_tuple_target

2019-04-03 Thread Robert Haas
On Wed, Apr 3, 2019 at 2:38 AM Michael Paquier wrote: > Hi all, > (Adding Simon as the author of toast_tuple_target, as well Andrew and > Pavan in CC.) > > toast_tuple_target has been introduced in 2017 by c251336 as of v11. > And while reviewing Pavan's patch to have more complex control over >

Re: pg_basebackup ignores the existing data directory permissions

2019-04-03 Thread Robert Haas
On Fri, Mar 29, 2019 at 6:05 AM Peter Eisentraut wrote: > On 2019-03-26 03:26, Michael Paquier wrote: > > Do we really want to extend the replication protocol to control that? > > Perhaps we are losing sight of the original problem, which is that if > you create the target directory with the

Re: New vacuum option to do only freezing

2019-04-03 Thread Robert Haas
On Wed, Apr 3, 2019 at 1:32 AM Masahiko Sawada wrote: > Attached the updated version patches including the > DISABLE_PAGE_SKIPPING part (0003). I am confused about nleft_dead_tuples. It looks like it gets incremented whenever we set tupgone = true, regardless of whether we are doing index

Re: Concurrency bug with vacuum full (cluster) and toast

2019-04-03 Thread Stephen Frost
Greetings, * Justin Pryzby (pry...@telsasoft.com) wrote: > On Fri, Mar 22, 2019 at 02:27:07PM -0400, Robert Haas wrote: > > On Tue, Mar 19, 2019 at 1:37 PM Alexander Korotkov > > wrote: > > > Thank you for pointing, but none of the threads you pointed describe > > > this exact problem. Now I

Re: Concurrency bug with vacuum full (cluster) and toast

2019-04-03 Thread Justin Pryzby
Hi, On Fri, Mar 22, 2019 at 02:27:07PM -0400, Robert Haas wrote: > On Tue, Mar 19, 2019 at 1:37 PM Alexander Korotkov > wrote: > > Thank you for pointing, but none of the threads you pointed describe > > this exact problem. Now I see this bug have a set of cute siblings :) > > Yeah. I really

Re: [PATCH v20] GSSAPI encryption support

2019-04-03 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Wed, Apr 3, 2019 at 12:22 AM Joe Conway wrote: > > On 4/2/19 6:18 PM, Stephen Frost wrote: > > > On Tue, Apr 2, 2019 at 18:10 Peter Eisentraut > > > > > > wrote: > > > > > > On 2019-02-23

Re: Minimal logical decoding on standbys

2019-04-03 Thread Amit Khandekar
On Tue, 2 Apr 2019 at 21:34, Andres Freund wrote: > > Hi, > > On 2019-04-02 15:26:52 +0530, Amit Khandekar wrote: > > On Thu, 14 Mar 2019 at 15:00, Amit Khandekar wrote: > > > I managed to get a recovery conflict by : > > > 1. Setting hot_standby_feedback to off > > > 2. Creating a logical

Re: minimizing pg_stat_statements performance overhead

2019-04-03 Thread Robert Haas
On Tue, Apr 2, 2019 at 5:37 AM Christoph Berg wrote: > Re: Raymond Martin 2019-04-01 > > > Thanks again Fabien. I am attaching the patch to this email in the hope of > > getting it approved during the next commit fest. > > you sent the patch as UTF-16, could you re-send it as plain ascii? One

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-04-03 Thread Robert Haas
On Tue, Apr 2, 2019 at 6:54 AM Kyotaro HORIGUCHI wrote: > > By using DELETE and INSERT records to implement an UPDATE, you lose the ctid > > chain and infomask bits that were present before crash recovery. If that's > > okay in these circumstances, please write a comment explaining why. > >

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-04-03 Thread Heikki Linnakangas
On 02/04/2019 08:58, Andrey Lepikhov wrote: On 25/03/2019 15:21, Heikki Linnakangas wrote: I had another quick look. I still think using the "generic xlog AM" for this is a wrong level of abstraction, and we should use the XLOG_FPI records for this directly. We can extend XLOG_FPI so that it

Re: [Patch] Mingw: Fix import library extension, build actual static libraries

2019-04-03 Thread Robert Haas
On Tue, Apr 2, 2019 at 5:32 AM Sandro Mani wrote: > Any chance these patches could be considered? You should add them to the Open CommitFest, currently https://commitfest.postgresql.org/23/ That CommitFest is scheduled to start in July; we are about to go to feature freeze for v12, and your

Re: ToDo: show size of partitioned table

2019-04-03 Thread Pavel Stehule
st 3. 4. 2019 v 14:01 odesílatel Alvaro Herrera napsal: > On 2019-Apr-03, Kyotaro HORIGUCHI wrote: > > > Hello. > > > > At Wed, 3 Apr 2019 12:55:06 +0900, Amit Langote < > langote_amit...@lab.ntt.co.jp> wrote in < > ee892049-0fe4-afe6-cbbf-31cf44fa8...@lab.ntt.co.jp> > > > On 2019/04/03 12:02,

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-03 Thread Fujii Masao
On Thu, Mar 28, 2019 at 11:45 AM Tsunakawa, Takayuki wrote: > > From: Robert Haas [mailto:robertmh...@gmail.com] > > You're both right and I'm wrong. > > > > However, I think it would be better to stick with the term 'truncate' > > which is widely-used already, rather than introducing a new term.

Re: partitioned tables referenced by FKs

2019-04-03 Thread Robert Haas
On Tue, Apr 2, 2019 at 11:07 AM Tom Lane wrote: > The problem this patch is running into is that we'd like to make the > validity of dropping a table partition depend on whether there are > individual rows in some other table that FK-reference rows in the target > partition. That's just

Re: Refactoring the checkpointer's fsync request queue

2019-04-03 Thread Thomas Munro
On Tue, Apr 2, 2019 at 11:09 PM Thomas Munro wrote: > I'm going to do some more testing and tidying tomorrow (for example I > think the segment.h header is silly and I'd like to remove that), and > commit this. As a sanity check on the programming interface this thing gives you, I tried teaching

Re: ToDo: show size of partitioned table

2019-04-03 Thread Alvaro Herrera
On 2019-Apr-03, Kyotaro HORIGUCHI wrote: > Hello. > > At Wed, 3 Apr 2019 12:55:06 +0900, Amit Langote > wrote in > > > On 2019/04/03 12:02, Kyotaro HORIGUCHI wrote: > > > \dPn doesn't show children because the are of 'r' relkind. And > > > \dPn * doesn't show type for children. > ... > > I

Re: Column lookup in a row performance

2019-04-03 Thread Павлухин Иван
Tom, thanks for your answer. It definitely makes a picture in my mind more clear. вт, 2 апр. 2019 г. в 18:41, Tom Lane : > > =?UTF-8?B?0J/QsNCy0LvRg9GF0LjQvSDQmNCy0LDQvQ==?= writes: > >> (1) Backwards compatibility, and (2) it's not clear that a different > >> layout would be a win for all

Re: [HACKERS] Cached plans and statement generalization

2019-04-03 Thread Konstantin Knizhnik
New version of the patch with fixed error of autopreparing CTE queries. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/doc/src/sgml/autoprepare.sgml b/doc/src/sgml/autoprepare.sgml new file mode 100644 index 000..b3309bd

fix the spelling mistakes of comments

2019-04-03 Thread Liu, Huailing
Hi, everyone When reading the codes in file src/backend/replication/syncrep.c, I have found two spelling mistakes of comments. First -

Re: [GSoC] application ideas

2019-04-03 Thread pantilimonov misha
Andrey, thank you for your reply. > 24.03.2019, 12:12, "Andrey Borodin" : > > I like the idea of more efficient BufferTag->Page data structure. I'm not > sure cache locality is a real problem there, but I believe this idea deserves > giving it a shot. > I'd happily review your proposal and

Re: Checksum errors in pg_stat_database

2019-04-03 Thread Julien Rouhaud
On Wed, Apr 3, 2019 at 11:31 AM Magnus Hagander wrote: > > On Wed, Apr 3, 2019 at 10:44 AM Julien Rouhaud wrote: >> >> On Wed, Apr 3, 2019 at 3:43 AM Michael Paquier wrote: >> > >> > > I can somewhat agree that splitting it on a per database level might >> > > even >> > > at that be overdoing

fix memory overflow in ecpg preproc module

2019-04-03 Thread Liu, Huailing
Hi, everyone I have found a potential memory overflow in ecpg preproc module. Here is: https://github.com/postgres/postgres/blob/REL9_5_16/src/interfaces/ecpg/preproc/pgc.l In parse_include() function --- for (ip = include_paths;

Re: Simplify redability of some tests for toast_tuple_target in strings.sql

2019-04-03 Thread Daniel Gustafsson
On Wednesday, April 3, 2019 8:59 AM, Michael Paquier wrote: > Hi all, > > Some tests for toast_tuple_target introduced by c251336 check if a > toast relation is empty or not using that: > +select 0 = pg_relation_size('pg_toast.pg_toast_'||(select oid from > pg_class where relname = >

Re: Checksum errors in pg_stat_database

2019-04-03 Thread Magnus Hagander
On Wed, Apr 3, 2019 at 10:44 AM Julien Rouhaud wrote: > On Wed, Apr 3, 2019 at 3:43 AM Michael Paquier > wrote: > > > > > I can somewhat agree that splitting it on a per database level might > even > > > at that be overdoing it. What might actually be more interesting from a > > >

pg_rewind vs superuser

2019-04-03 Thread Magnus Hagander
As pointed out by Michael Banck as a comment on my blogpost, the pg_rewind documentation says it requires superuser permissions on the remote server. Is that really so, though? I haven't tested it, but from a quick look at the code it looks like it needs pg_ls_dir(), pg_stat_file() and

Re: Ordered Partitioned Table Scans

2019-04-03 Thread Julien Rouhaud
Hi, On Wed, Apr 3, 2019 at 7:47 AM David Rowley wrote: > > On Wed, 3 Apr 2019 at 14:01, Amit Langote > wrote: > > I don't have any more comments. > > Great. Many thanks for having a look at this. Going by [1], Julien > also seems pretty happy, so I'm going to change this over to ready for >

Re: Checksum errors in pg_stat_database

2019-04-03 Thread Julien Rouhaud
On Wed, Apr 3, 2019 at 3:43 AM Michael Paquier wrote: > > > I can somewhat agree that splitting it on a per database level might even > > at that be overdoing it. What might actually be more interesting from a > > failure-location perspective would be tablespace, rather than any of the > >

Simplify redability of some tests for toast_tuple_target in strings.sql

2019-04-03 Thread Michael Paquier
Hi all, Some tests for toast_tuple_target introduced by c251336 check if a toast relation is empty or not using that: +select 0 = pg_relation_size('pg_toast.pg_toast_'||(select oid from pg_class where relname = 'toasttest'))/current_setting('block_size')::integer as blocks; This is

Re: COPY FROM WHEN condition

2019-04-03 Thread David Rowley
On Wed, 3 Apr 2019 at 18:56, Andres Freund wrote: > > +/* Class the buffer full if there are >= this many bytes of tuples stored > > */ > > +#define MAX_BUFFERED_BYTES 65535 > > Random aside: This seems pretty small (but should be changed separately. Yeah, my fingers were hovering

Re: [PATCH v20] GSSAPI encryption support

2019-04-03 Thread Magnus Hagander
On Wed, Apr 3, 2019 at 12:22 AM Joe Conway wrote: > On 4/2/19 6:18 PM, Stephen Frost wrote: > > Greetings, > > > > On Tue, Apr 2, 2019 at 18:10 Peter Eisentraut > > > > wrote: > > > > On 2019-02-23 17:27, Stephen Frost wrote: > > >> About

Re: [HACKERS] generated columns

2019-04-03 Thread Amit Langote
On 2019/04/03 3:54, Erik Rijkers wrote: > create schema if not exists "tmp"; > CREATE SCHEMA > create server if not exists "tmpserver" foreign data wrapper file_fdw; > CREATE SERVER > drop   foreign table if exists tmp.pg_head cascade; > DROP FOREIGN TABLE > create foreign table  

Caveats from reloption toast_tuple_target

2019-04-03 Thread Michael Paquier
Hi all, (Adding Simon as the author of toast_tuple_target, as well Andrew and Pavan in CC.) toast_tuple_target has been introduced in 2017 by c251336 as of v11. And while reviewing Pavan's patch to have more complex control over the compression threshold of a tuple, I have bumped into some

Re: Re: A separate table level option to control compression

2019-04-03 Thread Michael Paquier
On Wed, Apr 03, 2019 at 11:40:57AM +0530, Pavan Deolasee wrote: > On Wed, Apr 3, 2019 at 10:19 AM Michael Paquier wrote: >> - The comments in tuptoaster.h need to be updated to outline the >> difference between the compression invocation and the toast invocation >> thresholds. The wording could

Re: Re: A separate table level option to control compression

2019-04-03 Thread Pavan Deolasee
On Wed, Apr 3, 2019 at 10:19 AM Michael Paquier wrote: > > > I have been looking at this patch more, and here are some notes: > - The tests can be really simplified using directly reltoastrelid, so > I changed the queries this way. I am aware that the surroundings > hardcode directly the

  1   2   >