Re: pgbench - implement strict TPC-B benchmark

2019-08-05 Thread Fabien COELHO
Hello Robert, Ok, one thread cannot feed an N core server if enough client are executed per thread and the server has few things to do. Right ... where N is, uh, TWO. Yes, two indeed… For low-work cpu-bound load, given libpq & system overheads, you cannot really hope for a better deal. I

Re: The unused_oids script should have a reminder to use the 8000-8999 OID range

2019-08-05 Thread Peter Geoghegan
On Mon, Aug 5, 2019 at 11:13 PM Tom Lane wrote: > > Right. Besides, adding something along the lines Michael described > > necessitates fixing the problems that it creates. We'll run out of > > blocks of 5 contiguous OIDs (or whatever) far sooner than we'll run > > out of single OIDs. > > Well, if

Re: Recent failures in IsolationCheck deadlock-hard

2019-08-05 Thread Tom Lane
Thomas Munro writes: > On Sat, Aug 3, 2019 at 2:11 AM Tom Lane wrote: >> Also worth noting is that anole failed its first try at the new >> deadlock-parallel isolation test: >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=anole&dt=2019-08-01%2015%3A48%3A16 > And friarbird (also CLOBBE

Re: The unused_oids script should have a reminder to use the 8000-8999 OID range

2019-08-05 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Aug 5, 2019 at 10:41 PM Tom Lane wrote: >> But as long as the script >> tells you how many OIDs are available, what's the problem? Just run >> it again if you want a different suggestion, or make your own choice. > Right. Besides, adding something along the lin

Re: Recent failures in IsolationCheck deadlock-hard

2019-08-05 Thread Thomas Munro
On Sat, Aug 3, 2019 at 2:11 AM Tom Lane wrote: > Thomas Munro writes: > > There have been five failures on three animals like this, over the > > past couple of months: > > Also worth noting is that anole failed its first try at the new > deadlock-parallel isolation test: > > https://buildfarm.pos

Re: The unused_oids script should have a reminder to use the 8000-8999 OID range

2019-08-05 Thread Peter Geoghegan
On Mon, Aug 5, 2019 at 10:41 PM Tom Lane wrote: > There was some discussion of that upthread, and Peter argued that many > patches only need one OID anyway so why try harder. I'm not totally > sure I buy that --- my sense is that even simple patches tend to add > several related functions not jus

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Ian Barwick
On 8/6/19 11:16 AM, Stephen Frost wrote: > Greetings, > > * Ian Barwick (ian.barw...@2ndquadrant.com) wrote: >> On 8/6/19 9:52 AM, Stephen Frost wrote:> Greetings, >>> * Tom Lane (t...@sss.pgh.pa.us) wrote: + + External tools might also + modify postgresql.auto.conf

Re: The unused_oids script should have a reminder to use the 8000-8999 OID range

2019-08-05 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Aug 5, 2019 at 8:47 PM Michael Paquier wrote: >> Wouldn't it be better to keep some room at the end of the allowed >> array? Or at least avoid suggesting ranges where there is less than >> 3-5 OIDs available consecutively. > Not in my view. There is value in ha

Re: pg can create duplicated index without any errors even warnning

2019-08-05 Thread Tom Lane
Alex writes: > postgres=# create table t (a int, b int); > CREATE TABLE > postgres=# create index m on t(a); > CREATE INDEX > postgres=# create index m2 on t(a); > CREATE INDEX > is this by design? Yes. regards, tom lane

Re: Putting kerberos/ldap logs somewhere useful

2019-08-05 Thread Michael Paquier
On Mon, Aug 05, 2019 at 07:26:45PM -0400, Tom Lane wrote: > Hence, I propose the attached. This just moves the actual log > files ... +1 for this. The patch looks good. > we could possibly move the daemons' .conf files as well, > but I think they're probably not variable enough to be interestin

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Michael Paquier
On Mon, Aug 05, 2019 at 10:16:16PM -0400, Stephen Frost wrote: > * Ian Barwick (ian.barw...@2ndquadrant.com) wrote: >> On 8/6/19 9:52 AM, Stephen Frost wrote:> Greetings, >>> * Tom Lane (t...@sss.pgh.pa.us) wrote: Concretely, how about the attached? (Digging around in config.sgml, I

Re: POC: Cleaning up orphaned files using undo logs

2019-08-05 Thread Amit Kapila
On Mon, Aug 5, 2019 at 6:29 PM Robert Haas wrote: > > On Mon, Aug 5, 2019 at 6:16 AM Amit Kapila wrote: > > For zheap, we collect all the records of a page, apply them together > > and then write the entire page in WAL. The progress of transaction is > > updated at either transaction end (rollba

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-05 Thread Thomas Munro
On Tue, Aug 6, 2019 at 4:35 AM Andres Freund wrote: > On 2019-08-05 20:58:05 +1200, Thomas Munro wrote: > > 1. Commit dafaa3efb75 "Implement genuine serializable isolation > > level." (2011) locked the root tuple, because it set t_self to *tid. > > Possibly due to confusion about the effect of th

Re: The unused_oids script should have a reminder to use the 8000-8999 OID range

2019-08-05 Thread Peter Geoghegan
On Mon, Aug 5, 2019 at 8:47 PM Michael Paquier wrote: > So it would be possible to get 9998- as suggestion. In which > case, one can basically finish with this message: > Suggested random unused OID: (1 consecutive OID(s) available > starting here) I strongly doubt that this will ever b

Re: pg can create duplicated index without any errors even warnning

2019-08-05 Thread Michael Paquier
On Mon, Aug 05, 2019 at 08:16:11PM -0700, Peter Geoghegan wrote: > Yes. Being able to do this is useful for several reasons. For example, > it's useful to be able to create a new, equivalent index before > dropping the original when the original is bloated. (You could use > REINDEX instead, but tha

Re: The unused_oids script should have a reminder to use the 8000-8999 OID range

2019-08-05 Thread Michael Paquier
On Mon, Aug 05, 2019 at 09:00:26PM +0200, Julien Rouhaud wrote: > Thanks! What you have committed does that: +do +{ + $suggestion = int(8000 + rand(2000)); +} while (grep(/^$suggestion$/, @{$oids})); So it would be possible to get 9998- as suggestion. In which case, one can basically finish

Re: Assertion for logically decoding multi inserts into the catalog

2019-08-05 Thread Michael Paquier
On Tue, Aug 06, 2019 at 12:52:09AM +0200, Daniel Gustafsson wrote: > Yeah, this is clearly fat-fingered, the intent is to only run the Assert in > case XLH_INSERT_CONTAINS_NEW_TUPLE is set in xlrec->flags, as it only applies > under that condition. The attached is tested in both in the multi-inser

Re: pg can create duplicated index without any errors even warnning

2019-08-05 Thread Peter Geoghegan
On Mon, Aug 5, 2019 at 7:34 PM Alex wrote: > is this by design? Yes. Being able to do this is useful for several reasons. For example, it's useful to be able to create a new, equivalent index before dropping the original when the original is bloated. (You could use REINDEX instead, but that has s

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-05 Thread Masahiko Sawada
On Tue, Aug 6, 2019 at 9:42 AM Bruce Momjian wrote: > > On Wed, Jul 31, 2019 at 04:58:49PM +0900, Masahiko Sawada wrote: > > On Wed, Jul 31, 2019 at 3:29 PM Masahiko Sawada > > wrote: > > > > > > > > > For WAL encryption, before flushing WAL we encrypt whole 8k WAL page > > > and then write onl

Re: Problem with default partition pruning

2019-08-05 Thread Amit Langote
Hi Alvaro, On Mon, Aug 5, 2019 at 11:39 PM Alvaro Herrera wrote: > On 2019-Aug-05, yuzuko wrote: > > > So I proposed moving the if() block to the current place. > > The latest patch can solve both queries but I found the latter > > problem can be solved by setting constraint_exclusion = on. > > S

pg can create duplicated index without any errors even warnning

2019-08-05 Thread Alex
postgres=# create table t (a int, b int); CREATE TABLE postgres=# create index m on t(a); CREATE INDEX postgres=# create index m2 on t(a); CREATE INDEX postgres=# \d t Table "demo.t" Column | Type | Collation | Nullable | Default +-+---+--+--

Re: Ought to use heap_multi_insert() for pg_attribute/depend insertions?

2019-08-05 Thread Michael Paquier
On Mon, Aug 05, 2019 at 04:45:59PM +0900, Michael Paquier wrote: > + if (referenced->numrefs == 1) > + recordDependencyOn(depender, &referenced->refs[0], behavior); > + else > + recordMultipleDependencies(depender, > + referenced->refs, referenced->n

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, * Ian Barwick (ian.barw...@2ndquadrant.com) wrote: > On 8/6/19 9:52 AM, Stephen Frost wrote:> Greetings, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> Robert Haas writes: > >>> On Mon, Aug 5, 2019 at 2:42 PM Tom Lane wrote: > I don't think we need to go on about it at great len

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Ian Barwick
On 8/6/19 9:52 AM, Stephen Frost wrote:> Greetings, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Robert Haas writes: >>> On Mon, Aug 5, 2019 at 2:42 PM Tom Lane wrote: I don't think we need to go on about it at great length, but it seems to me that it'd be reasonable to point out that

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2019-08-05 Thread Peter Geoghegan
On Wed, Jul 31, 2019 at 9:23 AM Anastasia Lubennikova wrote: > > * Included my own pageinspect hack to visualize the minimum TIDs in > > posting lists. It's broken out into a separate patch file. The code is > > very rough, but it might help someone else, so I thought I'd include > > it. > Cool, I

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-05 Thread Bruce Momjian
On Wed, Jul 31, 2019 at 09:25:01AM -0400, Sehrope Sarkuni wrote: > On Tue, Jul 30, 2019 at 4:48 PM Bruce Momjian wrote: > > I had more time to think about the complexity of adding relfilenode to > the IV.  Since relfilenode is only unique within a database/tablespace, > we would need

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > On Mon, Aug 5, 2019 at 2:42 PM Tom Lane wrote: > >> I don't think we need to go on about it at great length, but it seems > >> to me that it'd be reasonable to point out that (a) you'd be well > >> advised not to touch t

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-05 Thread Bruce Momjian
On Wed, Jul 31, 2019 at 04:11:03PM +0900, Masahiko Sawada wrote: > On Wed, Jul 31, 2019 at 5:48 AM Bruce Momjian wrote: > > I am thinking for the heap/index IV, it would be: > > > > uint64 lsn; > > unint32 page number; > > /* only uses 11 bits for a zero-based CTR counter f

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-05 Thread Bruce Momjian
On Wed, Jul 31, 2019 at 09:43:00AM -0400, Sehrope Sarkuni wrote: > On Wed, Jul 31, 2019 at 2:32 AM Masahiko Sawada wrote: > > Just to confirm, we have 21 bits left for nonce in CTR? We have LSN (8 > bytes), page-number (4 bytes) and counter (11 bits) in 16 bytes nonce > space. Even th

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-05 Thread Bruce Momjian
On Wed, Jul 31, 2019 at 04:58:49PM +0900, Masahiko Sawada wrote: > On Wed, Jul 31, 2019 at 3:29 PM Masahiko Sawada wrote: > > > > > > For WAL encryption, before flushing WAL we encrypt whole 8k WAL page > > and then write only the encrypted data of the new WAL record using > > pg_pwrite() rather

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-05 Thread Bruce Momjian
On Wed, Jul 31, 2019 at 03:29:59PM +0900, Masahiko Sawada wrote: > Just to confirm, we have 21 bits left for nonce in CTR? We have LSN (8 > bytes), page-number (4 bytes) and counter (11 bits) in 16 bytes nonce > space. Even though we have 21 bits left we cannot store relfilenode to > the IV. No.

Re: Implementing Incremental View Maintenance

2019-08-05 Thread Tatsuo Ishii
It's not mentioned below but some bugs including seg fault when --enable-casser is enabled was also fixed in this patch. BTW, I found a bug with min/max support in this patch and I believe Yugo is working on it. Details: https://github.com/sraoss/pgsql-ivm/issues/20 Best regards, -- Tatsuo Ishii

Re: Psql patch to show access methods info

2019-08-05 Thread Alexander Korotkov
On Wed, Jul 24, 2019 at 4:59 PM Alexander Korotkov wrote: > On Wed, Jul 24, 2019 at 9:01 AM Andres Freund wrote: > > Based on a quick skim of the thread - which means I most definitely > > missed things - there's not been discussion of why we actually want to > > add this. Who's the prospective

Re: POC: Cleaning up orphaned files using undo logs

2019-08-05 Thread Robert Haas
On Mon, Aug 5, 2019 at 12:42 PM Andres Freund wrote: > A good move in the right direction, imo. I spent some more time thinking about this and talking to Thomas about it and I'd like to propose a somewhat more aggressive restructuring proposal, with the aim of getting a cleaner separation between

Re: An out-of-date comment in nodeIndexonlyscan.c

2019-08-05 Thread Thomas Munro
On Fri, Jun 28, 2019 at 12:55 PM Ashwin Agrawal wrote: > On Thu, Jun 27, 2019 at 4:33 PM Thomas Munro wrote: >> I wonder if it might be possible to avoid page locks on both the heap >> *and* index in some limited cases, as I mentioned over here (just an >> idea, could be way off base): >> >> http

Putting kerberos/ldap logs somewhere useful

2019-08-05 Thread Tom Lane
I got frustrated just now because this: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2019-08-05%2021%3A18%3A23 is essentially undebuggable, thanks to the buildfarm's failure to capture any error output from slapd. That's not the buildfarm script's fault: it's willing to captu

Re: Redacting information from logs

2019-08-05 Thread Jeff Davis
On Mon, 2019-08-05 at 14:44 -0700, Andres Freund wrote: > at the cost of sometimes more complicated % syntax (i.e. %1$d to > refer > to the first argument). > > I think the probable loss of gcc format warnings would be the biggest > issue with this whole proposal, and potential translator trouble

Re: Assertion for logically decoding multi inserts into the catalog

2019-08-05 Thread Daniel Gustafsson
> On 31 Jul 2019, at 19:20, Heikki Linnakangas wrote: > This patch makes the assertion more strict than it was before. I don't see > how it could possibly make a regression failure go away. On the contrary. So, > huh? Yeah, this is clearly fat-fingered, the intent is to only run the Assert in

Re: Redacting information from logs

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 14:32:36 -0700, Jeff Davis wrote: > On Mon, 2019-08-05 at 14:10 -0700, Andres Freund wrote: > > I was thinking that it'd just store a struct ErrParam, which'd > > reference > > the passed value and metadata like the name (for structured log > > output) and redaction category. Th

Re: Redacting information from logs

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 14:26:44 -0700, Jeff Davis wrote: > On Sun, 2019-08-04 at 11:17 -0700, Andres Freund wrote: > > I'm imagining something like > > > > #define pg_printf(fmt, ...) \ > > do { \ > > if ( __builtin_constant_p(fmt)) \ > > { \ > > static processed_fmt pr

Re: Redacting information from logs

2019-08-05 Thread Jeff Davis
On Mon, 2019-08-05 at 14:10 -0700, Andres Freund wrote: > I was thinking that it'd just store a struct ErrParam, which'd > reference > the passed value and metadata like the name (for structured log > output) and redaction category. The bigger problem I see is handling > the different types of arg

Re: Redacting information from logs

2019-08-05 Thread Jeff Davis
On Sun, 2019-08-04 at 11:17 -0700, Andres Freund wrote: > I'm imagining something like > > #define pg_printf(fmt, ...) \ > do { \ > if ( __builtin_constant_p(fmt)) \ > { \ > static processed_fmt processed_fmt_ = {.format = fmt}; \ > if (unlikely(!process

Re: Redacting information from logs

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 13:37:50 -0700, Jeff Davis wrote: > On Sat, 2019-08-03 at 19:14 -0400, Tom Lane wrote: > > It seems to me that it'd be sufficient to do the annotation by > > inserting wrapper functions, like the errparam() you suggest above. > > If we just had errparam() choosing whether to retu

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 22:46:47 +0200, Daniel Migowski wrote: > > Arguably the proposed owning_object field would be a bit redundant with > > the already existing ident/MemoryContextSetIdentifier field, which > > e.g. already associates the query string with the contexts used for a > > prepared stateme

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-08-05 Thread Daniel Migowski
Am 05.08.2019 um 19:16 schrieb Andres Freund: On 2019-07-28 06:20:40 +, Daniel Migowski wrote: how do you want to generalize it? Are you thinking about a view solely for the display of the memory usage of different objects? I'm not quite sure. I'm just not sure that adding separate infrastr

Re: pgbench - implement strict TPC-B benchmark

2019-08-05 Thread Fabien COELHO
Hello Andres, Which is a (somehow disappointing) * 3.3 speedup. The impact on the 3 complex expressions tests is not measurable, though. I don't know why that could be disappointing. We put in much more work for much smaller gains in other places. Probably, but I thought I would have a bett

Re: Index Skip Scan

2019-08-05 Thread Dmitry Dolgov
> On Mon, Aug 5, 2019 at 12:05 PM Floris Van Nee > wrote: > > The root->distinct_pathkeys is already filtered for redundant keys, so column > 'a' is not in there anymore. Still, it'd be much faster to use the skip scan > here, because a regular scan will scan all entries with a=1, even though > we

Re: Redacting information from logs

2019-08-05 Thread Jeff Davis
On Sat, 2019-08-03 at 19:14 -0400, Tom Lane wrote: > It seems to me that it'd be sufficient to do the annotation by > inserting wrapper functions, like the errparam() you suggest above. > If we just had errparam() choosing whether to return "..." instead of > its argument string, we'd have what we

Re: Cleanup of intro.sgml

2019-08-05 Thread Chapman Flack
On 8/5/19 3:20 PM, Joshua D. Drake wrote: > intro.sgml today. Patch attached. Things I noticed quickly: broken up in to categoriess/in to/into/ Unstructured data via JSON(or XML ?) s/Partioniing/Partitioning/ Regards, -Chap

Re: pgbench - implement strict TPC-B benchmark

2019-08-05 Thread Robert Haas
On Fri, Aug 2, 2019 at 2:38 AM Fabien COELHO wrote: > Ok, one thread cannot feed an N core server if enough client are executed > per thread and the server has few things to do. Right ... where N is, uh, TWO. > The point I'm clumsily trying to make is that pgbench-specific overheads > are quite

Re: tableam vs. TOAST

2019-08-05 Thread Tom Lane
Robert Haas writes: > I don't feel entirely convinced that there's any rush to do all of > this right now, and the more I change the harder it is to make sure > that I haven't broken anything. How strongly do you feel about this > stuff? FWIW, I agree with your comment further up that this patch

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Robert Haas
On Mon, Aug 5, 2019 at 3:07 PM Tom Lane wrote: > Concretely, how about the attached? Works for me, for whatever that's worth. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: tableam vs. TOAST

2019-08-05 Thread Robert Haas
On Fri, Aug 2, 2019 at 6:42 PM Andres Freund wrote: > Hm, those all include writing, right? And for read-only we don't expect > any additional overhead, correct? The write overhead is probably too > large show a bit of function call overhead - but if so, it'd probably be > on unlogged tables? And

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-08-05 Thread Daniel Migowski
Am 05.08.2019 um 18:30 schrieb Konstantin Knizhnik: On 31.07.2019 1:38, Daniel Migowski wrote: Am 31.07.2019 um 00:29 schrieb Tom Lane: Daniel Migowski writes: Ok, just have read about the commitfest thing. Is the patch OK for that? Or is there generally no love for a mem_usage column here? I

Cleanup of intro.sgml

2019-08-05 Thread Joshua D. Drake
-hackers, I went through and made some readability and modernization of the intro.sgml today. Patch attached. JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc Postgres centered full stack support, consulting and development. Advocate: @amplifypostgres || Get help: h

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Tom Lane
Robert Haas writes: > On Mon, Aug 5, 2019 at 2:42 PM Tom Lane wrote: >> I don't think we need to go on about it at great length, but it seems >> to me that it'd be reasonable to point out that (a) you'd be well >> advised not to touch the file while the postmaster is up, and (b) >> last setting w

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, On Mon, Aug 5, 2019 at 14:43 Tom Lane wrote: > Robert Haas writes: > > On Mon, Aug 5, 2019 at 2:29 PM Tom Lane wrote: > >> I think Stephen is not being unreasonable to suggest that we need some > >> documentation about what external tools may safely do to pg.auto.conf. > >> So someb

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Tom Lane
Robert Haas writes: > But that's not the problem. The problem is that ALTER SYSTEM modifies > the first match instead of the last one, when it's a well-established > principle that when reading from a PostgreSQL configuration file, the > system adopts the value from the last match, not the first

Re: The unused_oids script should have a reminder to use the 8000-8999 OID range

2019-08-05 Thread Julien Rouhaud
On Mon, Aug 5, 2019 at 8:51 PM Peter Geoghegan wrote: > > Pushed a version with similar wording just now. Thanks!

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, On Mon, Aug 5, 2019 at 14:38 Robert Haas wrote: > On Mon, Aug 5, 2019 at 2:29 PM Tom Lane wrote: > > Robert Haas writes: > > > All we need to do to resolve this issue is have Tom commit his patch. > > > > I think Stephen is not being unreasonable to suggest that we need some > > doc

Re: The unused_oids script should have a reminder to use the 8000-8999 OID range

2019-08-05 Thread Peter Geoghegan
On Fri, Aug 2, 2019 at 1:28 PM Julien Rouhaud wrote: > I'm fine with it! Pushed a version with similar wording just now. Thanks! -- Peter Geoghegan

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Robert Haas
On Mon, Aug 5, 2019 at 2:35 PM Stephen Frost wrote: > I dare say that if we had some documentation around what to expect and how to > deal with it, for our own tools as well as external ones, then maybe we > wouldn’t be in this situation in the first place. Clearly ALTER SYSTEM and > the pg_ba

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Robert Haas
On Mon, Aug 5, 2019 at 2:42 PM Tom Lane wrote: > I don't think we need to go on about it at great length, but it seems > to me that it'd be reasonable to point out that (a) you'd be well > advised not to touch the file while the postmaster is up, and (b) > last setting wins. Those things are equa

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Tom Lane
Robert Haas writes: > On Mon, Aug 5, 2019 at 2:29 PM Tom Lane wrote: >> I think Stephen is not being unreasonable to suggest that we need some >> documentation about what external tools may safely do to pg.auto.conf. >> So somebody's got to write that. > I mean, really? We're going to document

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Robert Haas
On Mon, Aug 5, 2019 at 2:29 PM Tom Lane wrote: > Robert Haas writes: > > All we need to do to resolve this issue is have Tom commit his patch. > > I think Stephen is not being unreasonable to suggest that we need some > documentation about what external tools may safely do to pg.auto.conf. > So s

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, On Mon, Aug 5, 2019 at 14:29 Tom Lane wrote: > Robert Haas writes: > > All we need to do to resolve this issue is have Tom commit his patch. > > I think Stephen is not being unreasonable to suggest that we need some > documentation about what external tools may safely do to pg.auto.c

Re: POC: Cleaning up orphaned files using undo logs

2019-08-05 Thread Andres Freund
Hi, (as I was out of context due to dealing with bugs, I've switched to lookign at the current zheap/undoprocessing branch. On 2019-07-30 01:02:20 -0700, Andres Freund wrote: > +/* > + * Insert a previously-prepared undo records. > + * > + * This function will write the actual undo record into th

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Tom Lane
Robert Haas writes: > All we need to do to resolve this issue is have Tom commit his patch. I think Stephen is not being unreasonable to suggest that we need some documentation about what external tools may safely do to pg.auto.conf. So somebody's got to write that. But I agree that we could go

Re: Unused header file inclusion

2019-08-05 Thread Tom Lane
Alvaro Herrera writes: > Then there's the removal, which is in tuplesort.c because of > INT_MAX as added by commit d26559dbf356 and still present ... One has to be especially wary of removing system-header inclusions; the fact that they don't seem to be needed on your own machine doesn't prove t

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, On Mon, Aug 5, 2019 at 14:11 Andres Freund wrote: > On 2019-08-05 13:34:39 -0400, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > On 2019-08-05 12:43:24 -0400, Tom Lane wrote: > > > > On the other hand, people have also opined that they want full error > > >

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Robert Haas
On Mon, Aug 5, 2019 at 1:29 PM Stephen Frost wrote: > No, that isn't the point of the controversy nor does it relate, at all, > to what you quoted above, so I don't think there's much value in > responding to the discussion about WARNING or not that you put together > below. Well, if that's not w

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 13:34:39 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On 2019-08-05 12:43:24 -0400, Tom Lane wrote: > > > On the other hand, people have also opined that they want full error > > > checking on the inserted values, and that seems out of reach with

Re: Unused header file inclusion

2019-08-05 Thread Alvaro Herrera
On 2019-Aug-04, vignesh C wrote: > Made the fixes based on your comments, updated patch has the changes > for the same. Well, you fixed the two things that seem to me quoted as examples of problems, but you didn't fix other occurrences of the same issues elsewhere. For example, you remove lwlock

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-08-05 12:43:24 -0400, Tom Lane wrote: > > On the other hand, people have also opined that they want full error > > checking on the inserted values, and that seems out of reach with less > > than a full running system (mumble extensio

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-05 Thread legrand legrand
Kyotaro Horiguchi-4 wrote > At Sun, 4 Aug 2019 00:04:01 -0700 (MST), legrand legrand < > legrand_legrand@ > > wrote in < > 1564902241482-0.post@.nabble >> >> > However having the nested queryid in >> > pg_stat_activity would be convenient to track >> > what is a long stored functions currentl

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Aug 5, 2019 at 10:21 AM Stephen Frost wrote: > > Just to be clear, I brought up this exact concern back in *November*: > > > > https://www.postgresql.org/message-id/20181127153405.GX3415%40tamriel.snowman.net > > > > And now becaus

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > On Mon, Aug 5, 2019 at 12:25 PM Tom Lane wrote: > >> Perhaps as a future improvement, but it's not happening for v12, > >> at least not unless you accept "use ALTER SYSTEM in a standalone > >> backend" as a usable answer

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-08-05 Thread Andres Freund
Hi, On 2019-07-28 06:20:40 +, Daniel Migowski wrote: > how do you want to generalize it? Are you thinking about a view solely > for the display of the memory usage of different objects? I'm not quite sure. I'm just not sure that adding separate infrastructure for various objects is a sutainab

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Robert Haas
On Mon, Aug 5, 2019 at 10:21 AM Stephen Frost wrote: > Just to be clear, I brought up this exact concern back in *November*: > > https://www.postgresql.org/message-id/20181127153405.GX3415%40tamriel.snowman.net > > And now because this was pushed forward and the concerns that I raised > ignored, w

Re: [HACKERS] Cached plans and statement generalization

2019-08-05 Thread Konstantin Knizhnik
On 01.08.2019 19:56, Konstantin Knizhnik wrote: On 31.07.2019 19:56, Heikki Linnakangas wrote: On 09/07/2019 23:59, Konstantin Knizhnik wrote: Fixed patch version of the path is attached. Much of the patch and the discussion has been around the raw parsing, and guessing which literals ar

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 12:43:24 -0400, Tom Lane wrote: > Yeah, good point. There are a lot of other cases where you really > don't want system startup to happen, too. Agreed. > On the other hand, people have also opined that they want full error > checking on the inserted values, and that seems out

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Tom Lane
Robert Haas writes: > On Mon, Aug 5, 2019 at 12:25 PM Tom Lane wrote: >> Perhaps as a future improvement, but it's not happening for v12, >> at least not unless you accept "use ALTER SYSTEM in a standalone >> backend" as a usable answer. > I'm not sure that would even work for the cases at issue

Re: POC: Cleaning up orphaned files using undo logs

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 11:25:10 -0400, Robert Haas wrote: > The obvious thing to do seems to be to have UndoLogControl objects own > SmgrRelations. That would be something of a novelty, since it looks > like currently only a Relation ever owns an SMgrRelation, but the smgr > infrastructure seems to hav

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 20:58:05 +1200, Thomas Munro wrote: > On Sat, Aug 3, 2019 at 11:56 AM Ashwin Agrawal wrote: > > On Wed, Jul 31, 2019 at 2:06 PM Andres Freund wrote: > >> I'm also a bit confused why we don't need to pass in the offset of the > >> current tuple, rather than the HOT root tuple

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Robert Haas
On Mon, Aug 5, 2019 at 12:25 PM Tom Lane wrote: > Perhaps as a future improvement, but it's not happening for v12, > at least not unless you accept "use ALTER SYSTEM in a standalone > backend" as a usable answer. I'm not sure that would even work for the cases at issue ... because we're talking a

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-08-05 Thread Konstantin Knizhnik
On 31.07.2019 1:38, Daniel Migowski wrote: Am 31.07.2019 um 00:29 schrieb Tom Lane: Daniel Migowski writes: Ok, just have read about the commitfest thing. Is the patch OK for that? Or is there generally no love for a mem_usage column here? If it was, I really would add some memory monitor

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Tom Lane
David Fetter writes: > On Mon, Aug 05, 2019 at 03:52:07PM +0900, Ian Barwick wrote: >> On 8/4/19 1:59 AM, Tom Lane wrote: >>> I think we should just accept the facts on the ground, which are that >>> some tools modify pg.auto.conf by appending to it >> +1. It's just a text file... > So are cront

Re: pgbench - implement strict TPC-B benchmark

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 17:38:23 +0200, Fabien COELHO wrote: > Which is a (somehow disappointing) * 3.3 speedup. The impact on the 3 > complex expressions tests is not measurable, though. I don't know why that could be disappointing. We put in much more work for much smaller gains in other places. >

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread David Fetter
On Mon, Aug 05, 2019 at 03:52:07PM +0900, Ian Barwick wrote: > On 8/4/19 1:59 AM, Tom Lane wrote:> Tomas Vondra > writes: > >> On Fri, Aug 02, 2019 at 06:08:02PM -0700, Andres Freund wrote: > >>> We're WAY WAY past feature freeze. This isn't the time to rewrite guc.c, > >>> guc-file.l to be suita

Re: pgbench - implement strict TPC-B benchmark

2019-08-05 Thread Fabien COELHO
Hello Andres, If not, do you think advisable to spend time improving the evaluator & variable stuff and possibly other places for an overall 15% gain? Also, what would be the likelyhood of such optimization patch to pass? I could do a limited variable management improvement patch, eventually

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Christoph Berg
Re: Tomas Vondra 2019-08-03 <20190803124111.2aaddumd7url5wmq@development> > If we really want to give external tools a sensible (and optional) API > to access the file, a simple command-line tool seems much better. Say we > have something like > > pg_config_file -f PATH --set KEY VALUE > pg_co

Re: POC: Cleaning up orphaned files using undo logs

2019-08-05 Thread Robert Haas
On Tue, Jul 30, 2019 at 4:02 AM Andres Freund wrote: > I'm a bit worried about expanding the use of > ReadBufferWithoutRelcache(). Not so much because of the relcache itself, > but because it requires doing separate smgropen() calls. While not > crazily expensive, it's also not free. Especially co

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Tom Lane
Isaac Morland writes: > Here's a radical suggestion: replace postgresql.auto.conf with a directory > containing multiple files. Each file is named after a configuration > parameter, and its content is the value of the parameter. Hmm ... that seems like a lot of work and code churn --- in particul

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Tom Lane
Tomas Vondra writes: > On Mon, Aug 05, 2019 at 10:21:39AM -0400, Stephen Frost wrote: >> I'd be happier with one set of code at least being the recommended >> approach to modifying the file and only one set of code in our codebase >> that's messing with .auto.conf, so that, hopefully, it's done >>

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Tomas Vondra
On Mon, Aug 05, 2019 at 10:21:39AM -0400, Stephen Frost wrote: Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: On Fri, Aug 02, 2019 at 06:38:46PM -0400, Stephen Frost wrote: >On Fri, Aug 2, 2019 at 18:27 Tom Lane wrote: >>Tomas Vondra writes: >>> There seems to be a consensus

Re: Problem with default partition pruning

2019-08-05 Thread Alvaro Herrera
On 2019-Aug-05, yuzuko wrote: > So I proposed moving the if() block to the current place. > The latest patch can solve both queries but I found the latter > problem can be solved by setting constraint_exclusion = on. So we have three locations for that test; one is where it currently is, which ha

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Isaac Morland
Here's a radical suggestion: replace postgresql.auto.conf with a directory containing multiple files. Each file is named after a configuration parameter, and its content is the value of the parameter. So to remove a special configuration parameter, delete its file. To set it, write the file, repla

Re: block-level incremental backup

2019-08-05 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Aug 2, 2019 at 9:13 AM vignesh C wrote: > > + rc = system(copycmd); > > I don't think this patch should be calling system() in the first place. +1. Thanks, Stephen signature.asc Description: PGP signature

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-05 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Fri, Aug 02, 2019 at 06:38:46PM -0400, Stephen Frost wrote: > >On Fri, Aug 2, 2019 at 18:27 Tom Lane wrote: > >>Tomas Vondra writes: > >>> There seems to be a consensus that this this not a pg_basebackup issue > >>> (i.e. dupli

Re: SSL Connection still showing TLSv1.3 even it is disabled in ssl_ciphers

2019-08-05 Thread Tom Lane
tushar writes: > when  i connect to psql terminal - > psql.bin (10.9) > SSL connection (protocol: TLSv1.3, cipher: *TLS_AES_256_GCM_SHA384*, > bits: 256, compression: off) > Type "help" for help. > postgres=# show ssl_ciphers ; > ssl_ciphers > -

  1   2   >