Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Kuntal Ghosh
Hello Andres, On Tue, Aug 6, 2019 at 1:26 PM Andres Freund wrote: > > +/* Each worker queue is a binary heap. */ > > +typedef struct > > +{ > > + binaryheap *bh; > > + union > > + { > > + UndoXidQueue *xid_elems; > > + UndoSizeQueue *size_elems; > > +

Re: Handling RestrictInfo in expression_tree_walker

2019-08-07 Thread Amit Langote
Hi, On Wed, Aug 7, 2019 at 5:07 PM Konstantin Knizhnik wrote: > On 07.08.2019 10:42, Amit Langote wrote: > > You may also want to read this discussion: > > > > https://www.postgresql.org/message-id/553FC9BC.5060402%402ndquadrant.com > > > Thank you very much for response and pointing me to this

s/rewinded/rewound/?

2019-08-07 Thread Thomas Munro
Hello, The word "rewinded" appears in our manual and in a comment. That sounds strange to my ears. Isn't it a mistake? Oxford lists the form as "poetic" and "rare", and then says it was used by one specific Victorian poet. Perhaps I'll send them a pull request: it's now G. M. Hopkins and

Re: s/rewinded/rewound/?

2019-08-07 Thread Michael Paquier
On Wed, Aug 07, 2019 at 10:53:45AM +0200, Magnus Hagander wrote: > To me this sounds like a classic non-English-native-speaker-mistake. But > it seems at least the one in the docs come from Bruce, who definitely is... > So perhaps it's intentional to refer to "what pg_rewind does", and not >

Re: partition routing layering in nodeModifyTable.c

2019-08-07 Thread Etsuro Fujita
Amit-san, On Wed, Aug 7, 2019 at 4:28 PM Amit Langote wrote: > On Wed, Aug 7, 2019 at 12:00 PM Etsuro Fujita wrote: > > IIUC, I think we reached a consensus at least on the 0001 patch. > > Andres, would you mind if I commit that patch? > > I just noticed obsolete references to

Re: SegFault on 9.6.14

2019-08-07 Thread vignesh C
On Wed, Jul 31, 2019 at 9:37 AM Amit Kapila wrote: > > On Wed, Jul 31, 2019 at 12:05 AM Robert Haas wrote: > > > > On Thu, Jul 18, 2019 at 9:45 AM Tom Lane wrote: > > > I think this is going in the wrong direction. Nodes should *always* > > > assume that a rescan is possible until ExecEndNode

Re: Problem with default partition pruning

2019-08-07 Thread yuzuko
Hello, > > Well, if this is really all that duplicative, one thing we could do is > > run this check in get_partprune_steps_internal only if > > constraint_exclusion is a value other than on; we should achieve the > > same effect with no repetition. Patch for that is attached. However, > > if I

Re: Built-in connection pooler

2019-08-07 Thread Konstantin Knizhnik
Hi, Li Thank you very much for reporting the problem. On 07.08.2019 7:21, Li Japin wrote: I inspect the code, and find the following code in DefineRelation function: if (stmt->relation->relpersistence != RELPERSISTENCE_TEMP     && stmt->oncommit != ONCOMMIT_DROP)    

Re: Handling RestrictInfo in expression_tree_walker

2019-08-07 Thread Konstantin Knizhnik
On 07.08.2019 10:42, Amit Langote wrote: Hi Konstantin, On Wed, Aug 7, 2019 at 4:24 PM Konstantin Knizhnik wrote: Hi hackers, I wonder if there is some particular reason for not handling T_RestrictInfo node tag in expression_tree_walker? There are many data structure in Postgres which

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Masahiko Sawada
On Wed, Aug 7, 2019 at 2:55 AM Bruce Momjian wrote: > > On Wed, Aug 7, 2019 at 12:31:58AM +0900, Masahiko Sawada wrote: > > Well, so you mean that for example we encrypt only 100 bytes WAL > > record when append 100 bytes WAL records? > > > > For WAL encryption, if we encrypt the entire 8k WAL

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-07 Thread Evgeny Efimkin
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed HI! patch is look good for me. The new status of this patch

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Thomas Munro
Hi, I'll be responding to a bunch of long review emails in this thread point by point separately, but just picking out a couple of points here that jumped out at me: On Wed, Aug 7, 2019 at 9:18 AM Andres Freund wrote: > > + { > > + /* > > +

Re: proposal: type info support functions for functions that use "any" type

2019-08-07 Thread Pavel Stehule
čt 1. 8. 2019 v 11:01 odesílatel Thomas Munro napsal: > On Sat, Jul 27, 2019 at 5:45 PM Pavel Stehule > wrote: > > pá 26. 7. 2019 v 22:53 odesílatel Tom Lane napsal: > >> I wrote: > >> > TBH, I don't like this proposal one bit. As far as I can see, the > idea > >> > is to let a function's

Re: [PATCH] Absolute passwordfile path

2019-08-07 Thread Amit Langote
Hello, On Wed, Aug 7, 2019 at 3:05 PM Danylo Hlynskyi wrote: > > The pool_passwd option [1] is specified relative to config file. But for > greater flexibility absolute path should be accepted as well. > > If pool_passwd option starts with /, let's treat it as absolute path. > Otherwise, it is

Re: [PATCH] Absolute passwordfile path

2019-08-07 Thread Danylo Hlynskyi
Yes, I've resent it to pgpool-hack...@pgpool.net Sorry for the noise ср, 7 серп. 2019 о 09:18 Amit Langote пише: > Hello, > > On Wed, Aug 7, 2019 at 3:05 PM Danylo Hlynskyi > wrote: > > > > The pool_passwd option [1] is specified relative to config file. But for > greater flexibility absolute

Re: FETCH FIRST clause PERCENT option

2019-08-07 Thread Surafel Temesgen
Hi On Wed, Aug 7, 2019 at 6:11 AM Kyotaro Horiguchi wrote: > > I have some comments. > > This patch uses distinct parameters for exact number and > percentage. On the othe hand planner has a notion of > tuple_fraction covering the both. The same handling is also used > for tuple number

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Dilip Kumar
On Tue, Aug 6, 2019 at 1:26 PM Andres Freund wrote: > > Hi, > > On 2019-08-05 11:29:34 -0700, Andres Freund wrote: > > Need to do something else for a bit. More later. > > Here we go. > Thanks for the review. I will work on them. Currently, I need suggestions on some of the review comments. > >

Re: Small patch to fix build on Windows

2019-08-07 Thread Kyotaro Horiguchi
Hi, At Tue, 6 Aug 2019 22:50:14 +0300, Dmitry Igrishin wrote in > The attached self-documented patch fixes build on Windows in case when > path to Python has embedded spaces. - $solution->{options}->{python} . "\\python -c \"$pythonprog\""; +

Re: partition routing layering in nodeModifyTable.c

2019-08-07 Thread Amit Langote
On Wed, Aug 7, 2019 at 12:00 PM Etsuro Fujita wrote: > IIUC, I think we reached a consensus at least on the 0001 patch. > Andres, would you mind if I commit that patch? I just noticed obsolete references to es_result_relation_info that 0002 failed to remove. One of them is in fdwhandler.sgml:

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Dilip Kumar
On Tue, Aug 6, 2019 at 1:26 PM Andres Freund wrote: > > Hi, > > On 2019-08-05 11:29:34 -0700, Andres Freund wrote: > > Need to do something else for a bit. More later. > > > + * false, otherwise. > > + */ > > +static bool > > +UndoAlreadyApplied(FullTransactionId full_xid, UndoRecPtr

Re: Small patch to fix build on Windows

2019-08-07 Thread Dmitry Igrishin
ср, 7 авг. 2019 г. в 11:29, Kyotaro Horiguchi : > > Hi, > > At Tue, 6 Aug 2019 22:50:14 +0300, Dmitry Igrishin wrote > in > > The attached self-documented patch fixes build on Windows in case when > > path to Python has embedded spaces. > > - $solution->{options}->{python} . "\\python

Handling RestrictInfo in expression_tree_walker

2019-08-07 Thread Konstantin Knizhnik
Hi hackers, I wonder if there is some particular reason for not handling T_RestrictInfo node tag in expression_tree_walker? There are many data structure in Postgres which contains lists of RestrictInfo or expression with RestrictInfo as parameter (for example orclause in RestrictInfo). To

Re: no default hash partition

2019-08-07 Thread Magnus Hagander
On Wed, Aug 7, 2019 at 5:26 AM Robert Haas wrote: > On Tue, Aug 6, 2019 at 6:58 PM Tom Lane wrote: > > Hmm. So given the point about it being hard to predict which hash > > partitions would receive what values ... under what circumstances > > would it be sensible to not create a full set of

Fwd: [PATCH] Absolute passwordfile path

2019-08-07 Thread Danylo Hlynskyi
The pool_passwd option [1] is specified relative to config file. But for greater flexibility absolute path should be accepted as well. If pool_passwd option starts with /, let's treat it as absolute path. Otherwise, it is treated as relative path. Patch attached. Original author - Derek Kulinski

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Amit Kapila
On Tue, Aug 6, 2019 at 1:26 PM Andres Freund wrote: > On 2019-08-05 11:29:34 -0700, Andres Freund wrote: > I am responding to some of the points where I need some more inputs or some discussion is required. Some of the things need more thoughts which I will respond later and some are quite

Re: Handling RestrictInfo in expression_tree_walker

2019-08-07 Thread Amit Langote
Hi Konstantin, On Wed, Aug 7, 2019 at 4:24 PM Konstantin Knizhnik wrote: > > Hi hackers, > > I wonder if there is some particular reason for not handling > T_RestrictInfo node tag in expression_tree_walker? > There are many data structure in Postgres which contains lists of > RestrictInfo or

Re: s/rewinded/rewound/?

2019-08-07 Thread Magnus Hagander
On Wed, Aug 7, 2019 at 10:49 AM Thomas Munro wrote: > Hello, > > The word "rewinded" appears in our manual and in a comment. That > sounds strange to my ears. Isn't it a mistake? Oxford lists the form > as "poetic" and "rare", and then says it was used by one specific > Victorian poet.

Re: proposal: type info support functions for functions that use "any" type

2019-08-07 Thread Pavel Stehule
Hi pá 26. 7. 2019 v 22:53 odesílatel Tom Lane napsal: > I wrote: > > TBH, I don't like this proposal one bit. As far as I can see, the idea > > is to let a function's support function redefine the function's declared > > argument and result types on-the-fly according to no predetermined rules,

Re: Problem with default partition pruning

2019-08-07 Thread Amit Langote
On Wed, Aug 7, 2019 at 3:30 PM yuzuko wrote: > > In short, I propose to get this done as the patch I posted in > > https://postgr.es/m/20190806133053.GA23706@alvherre.pgsql > > > I agree with your proposal. Also, I confirmed a default partition was pruned > as expected with your patch. +1.

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Dilip Kumar
On Tue, Jul 30, 2019 at 1:54 PM Dilip Kumar wrote: > > On Tue, Jul 30, 2019 at 12:21 PM Thomas Munro wrote: > > > > One data structure that could perhaps hold this would be > > UndoLogTableEntry (the per-backend cache, indexed by undo log number, > > with pretty fast lookups; used for things

SQL/JSON path: collation for comparisons, minor typos in docs

2019-08-07 Thread Markus Winand
Hi! I was playing around with JSON path quite a bit and might have found one case where the current implementation doesn’t follow the standard. The functionality in question are the comparison operators except ==. They use the database default collation rather then the standard-mandated

Re: Built-in connection pooler

2019-08-07 Thread Ryan Lambert
> First of all default value of this parameter is 1000, not 10. Oops, my bad! Sorry about that, I'm not sure how I got that in my head last night but I see how that would make it act strange now. I'll adjust my notes before re-testing. :) Thanks, *Ryan Lambert* On Wed, Aug 7, 2019 at 4:57

Re: s/rewinded/rewound/?

2019-08-07 Thread Liudmila Mantrova
On 8/7/19 12:00 PM, Michael Paquier wrote: On Wed, Aug 07, 2019 at 10:53:45AM +0200, Magnus Hagander wrote: To me this sounds like a classic non-English-native-speaker-mistake. But it seems at least the one in the docs come from Bruce, who definitely is... So perhaps it's intentional to refer

Re: block-level incremental backup

2019-08-07 Thread Jeevan Chalke
On Mon, Aug 5, 2019 at 7:13 PM Robert Haas wrote: > On Fri, Aug 2, 2019 at 9:13 AM vignesh C wrote: > > + rc = system(copycmd); > > I don't think this patch should be calling system() in the first place. > So, do you mean we should just do fread() and fwrite() for the whole file? I thought it

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Bruce Momjian
On Wed, Aug 7, 2019 at 05:13:31PM +0900, Masahiko Sawada wrote: > I understood. IIUC in your approach postgres processes encrypt WAL > records when inserting to the WAL buffer. So WAL data is encrypted > even on the WAL buffer. > > It works but I think the implementation might be complex; For

Re: stress test for parallel workers

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 02:57, Thomas Munro wrote: On Wed, Jul 24, 2019 at 5:15 PM Tom Lane wrote: So I think I've got to take back the assertion that we've got some lurking generic problem. This pattern looks way more like a platform-specific issue. Overaggressive OOM killer would fit the facts on

Re: stress test for parallel workers

2019-08-07 Thread Tom Lane
Heikki Linnakangas writes: > On 07/08/2019 02:57, Thomas Munro wrote: >> On Wed, Jul 24, 2019 at 5:15 PM Tom Lane wrote: >>> So I think I've got to take back the assertion that we've got >>> some lurking generic problem. This pattern looks way more >>> like a platform-specific issue.

initdb: Use varargs macro for PG_CMD_PRINTF

2019-08-07 Thread Peter Eisentraut
Small patch to simplify some no longer necessary complicated code, using varargs macros. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From 5004af6cc9174e9cdc20a6bf4f959d22e209a8e8 Mon Sep 17 00:00:00 2001 From:

Re: Regression test failure in regression test temp.sql

2019-08-07 Thread Tom Lane
Michael Paquier writes: > While browsing the buildfarm failures, I have found this problem on > anole for the test temp: > ... > This could be solved just with an ORDER BY as per the attached. Any > objections? There's no reason to expect stability of row order in pg_class, so in principle this

Re: SQL/JSON path: collation for comparisons, minor typos in docs

2019-08-07 Thread Alexander Korotkov
Hi! On Wed, Aug 7, 2019 at 2:25 PM Markus Winand wrote: > I was playing around with JSON path quite a bit and might have found one case > where the current implementation doesn’t follow the standard. > > The functionality in question are the comparison operators except ==. They > use the

Unix-domain socket support on Windows

2019-08-07 Thread Peter Eisentraut
It works! (apparently as of Windows 10 version 1803) Here are some patches to get a discussion rolling. Basically, it just works, but you need to define your own struct sockaddr_un. (This is what configure currently uses as a proxy for HAVE_UNIX_SOCKETS, so (a) that needs a bit of tweaking,

Re: Unix-domain socket support on Windows

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 16:56, Peter Eisentraut wrote: It works! Cool! Am I reading the patches correctly, that getpeereid() still doesn't work on Windows? That means that peer authentication doesn't work, right? That's a bit sad. One of the big advantages of unix domain sockets over TCP is peer

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Thomas Munro
On Thu, Aug 1, 2019 at 1:22 AM Amit Kapila wrote: > On Wed, Jul 31, 2019 at 10:13 AM Amit Kapila wrote: > > On Tue, Jul 30, 2019 at 5:26 PM Thomas Munro wrote: > > > but > > > here's a small thing: I managed to reach an LWLock self-deadlock in > > > the undo worker launcher: > > > > > > > I

Re: initdb: Use varargs macro for PG_CMD_PRINTF

2019-08-07 Thread Tom Lane
Peter Eisentraut writes: > Small patch to simplify some no longer necessary complicated code, using > varargs macros. +1 regards, tom lane

Re: no default hash partition

2019-08-07 Thread Stephen Frost
Greetings, * Amit Langote (amitlangot...@gmail.com) wrote: > On Wed, Aug 7, 2019 at 1:59 PM Kyotaro Horiguchi > wrote: > > At Tue, 6 Aug 2019 23:26:19 -0400, Robert Haas > > wrote: > > > On Tue, Aug 6, 2019 at 6:58 PM Tom Lane wrote: > > > I think, as Amit says, that having an automatic

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-07 Thread Heikki Linnakangas
On 06/08/2019 13:35, Thomas Munro wrote: On Tue, Aug 6, 2019 at 9:26 PM Heikki Linnakangas wrote: Attached is a patch that contains your fix.txt with the changes for clarity mentioned above, and an isolationtester test case. LGTM. Pushed, thanks! - Heikki

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 13:52, Dilip Kumar wrote: I have one more problem related to compression of the command id field. Basically, the problem is that we don't set the command id in the WAL and we will always store FirstCommandId in the undo[1]. So suppose there were 2 operations under a different

Re: s/rewinded/rewound/?

2019-08-07 Thread Chapman Flack
On 08/07/19 04:48, Thomas Munro wrote: > as "poetic" and "rare", and then says it was used by one specific > Victorian poet. Perhaps I'll send them a pull request: it's now G. M. > Hopkins and PostgreSQL? It does seem counter, original, spare, strange. Regards, -Chap

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 14:42, Thomas Munro wrote: On Wed, Aug 7, 2019 at 11:28 PM Heikki Linnakangas wrote: The list of tests in src/test/isolation/isolation_schedule has grown over the years. Originally, they were all related to Serializable Snapshot Isolation, but there are different kinds of

Re: block-level incremental backup

2019-08-07 Thread Ibrar Ahmed
On Wed, Aug 7, 2019 at 2:47 PM Jeevan Chalke wrote: > > > On Mon, Aug 5, 2019 at 7:13 PM Robert Haas wrote: > >> On Fri, Aug 2, 2019 at 9:13 AM vignesh C wrote: >> > + rc = system(copycmd); >> >> I don't think this patch should be calling system() in the first place. >> > > So, do you mean we

Grouping isolationtester tests in the schedule

2019-08-07 Thread Heikki Linnakangas
The list of tests in src/test/isolation/isolation_schedule has grown over the years. Originally, they were all related to Serializable Snapshot Isolation, but there are different kinds of concurrency tests there now. More tests is good, but the schedule file has grown into a big inscrutable

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Sehrope Sarkuni
On Mon, Aug 5, 2019 at 9:02 PM Bruce Momjian wrote: > On Wed, Jul 31, 2019 at 09:25:01AM -0400, Sehrope Sarkuni wrote: > > Even if we do not include a separate per-relation salt or things like > > relfilenode when generating a derived key, we can still include other > types of > > immutable

Regression test failure in regression test temp.sql

2019-08-07 Thread Michael Paquier
Hi all, While browsing the buildfarm failures, I have found this problem on anole for the test temp: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=anole=2019-08-07%2006%3A39%3A35 select relname from pg_class where relname like 'temp_parted_oncommit_test%'; relname

Re: stress test for parallel workers

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 16:57, Tom Lane wrote: Heikki Linnakangas writes: On 07/08/2019 02:57, Thomas Munro wrote: On Wed, Jul 24, 2019 at 5:15 PM Tom Lane wrote: So I think I've got to take back the assertion that we've got some lurking generic problem. This pattern looks way more like a

Re: Store FullTransactionId in TwoPhaseFileHeader/GlobalTransactionData

2019-08-07 Thread vignesh C
On Mon, Aug 5, 2019 at 8:31 AM Andres Freund wrote: > > Hi, > > On 2019-08-05 14:44:37 +1200, Thomas Munro wrote: > > Yeah. I think we're agreed for now that we don't want to change > > procarray (though we still need to figure out how to compute the 64 > > bit horizons correctly and efficiently)

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Heikki Linnakangas
On 05/08/2019 16:24, Robert Haas wrote: On Sun, Aug 4, 2019 at 5:16 AM Heikki Linnakangas wrote: I feel that the level of abstraction is not quite right. There are a bunch of fields, like uur_block, uur_offset, uur_tuple, that are probably useful for some UNDO resource managers (zheap I

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Thomas Munro
On Wed, Aug 7, 2019 at 11:28 PM Heikki Linnakangas wrote: > The list of tests in src/test/isolation/isolation_schedule has grown > over the years. Originally, they were all related to Serializable > Snapshot Isolation, but there are different kinds of concurrency tests > there now. More tests is

Re: Small patch to fix build on Windows

2019-08-07 Thread Juan José Santamaría Flecha
On Wed, Aug 7, 2019 at 11:11 AM Dmitry Igrishin wrote: > > ср, 7 авг. 2019 г. в 11:29, Kyotaro Horiguchi : > > > > Solution.pm has the following line: > > > > > my $opensslcmd = > > > $self->{options}->{openssl} . "\\bin\\openssl.exe version 2>&1"; > > > > AFAICS that's all. > Thank

Re: Small patch to fix build on Windows

2019-08-07 Thread Dmitry Igrishin
ср, 7 авг. 2019 г. в 15:33, Juan José Santamaría Flecha : > > On Wed, Aug 7, 2019 at 11:11 AM Dmitry Igrishin wrote: > > > > ср, 7 авг. 2019 г. в 11:29, Kyotaro Horiguchi : > > > > > > Solution.pm has the following line: > > > > > > > my $opensslcmd = > > > >

Re: Handling RestrictInfo in expression_tree_walker

2019-08-07 Thread Tom Lane
Konstantin Knizhnik writes: > Frankly speaking I do not see some good reasons for not handling > RestrictInfo in expression_tree_worker. It can really simplify writing > of mutators/walkers. I don't buy this; what seems more likely is that you're trying to apply an expression tree mutator to

Rethinking opclass member checks and dependency strength

2019-08-07 Thread Tom Lane
Over in [1] we realized that it would be a good idea to remove the <@ operator from contrib/intarray's GiST opclasses. Unfortunately, doing that isn't a simple matter of generating an extension update script containing ALTER OPERATOR FAMILY DROP OPERATOR, because that operator is marked as

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Heikki Linnakangas wrote: > The elephant in the room is the 'timeouts' test, which takes about 40 > seconds, out of a total runtime of 90 seconds. So we'd really want to run > that in parallel with everything else. Or split 'timeouts' into multiple > tests that could run in

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Bruce Momjian
On Wed, Aug 7, 2019 at 11:41:51AM -0400, Sehrope Sarkuni wrote: > On Wed, Aug 7, 2019 at 7:19 AM Bruce Momjian wrote: > > On Wed, Aug  7, 2019 at 05:13:31PM +0900, Masahiko Sawada wrote: > > I understood. IIUC in your approach postgres processes encrypt WAL > > records when

Re: s/rewinded/rewound/?

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Tom Lane wrote: > Magnus Hagander writes: > > To me this sounds like a classic non-English-native-speaker-mistake. But > > it seems at least the one in the docs come from Bruce, who definitely is... > > He might've just been committing somebody else's words without having >

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Chapman Flack
On 08/07/19 11:39, Tom Lane wrote: > Pavel Stehule writes: >> I should to use a cache accessed via fn_extra. There will be stored data >> about function parameters (types). If I understand correctly, these data >> should be stable in query, and then recheck is not necessary. Is it true? > > I

Re: s/rewinded/rewound/?

2019-08-07 Thread Geoff Winkless
On Wed, 7 Aug 2019 at 16:59, Alvaro Herrera wrote: > He didn't > add a mailing list reference, but this is easy to find at > https://postgr.es/m/20160720180706.gf24...@momjian.us > I lean towards the view that he was using the literal program name as a > verb, rather than trying to decline a verb

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Pavel Stehule
st 7. 8. 2019 v 17:39 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > I should to use a cache accessed via fn_extra. There will be stored data > > about function parameters (types). If I understand correctly, these data > > should be stable in query, and then recheck is not necessary.

Re: no default hash partition

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Amit Langote wrote: > That hash-partitioned tables can't have default partition is mentioned > in the CREATE TABLE page: > > "If DEFAULT is specified, the table will be created as a default > partition of the parent table. The parent can either be a list or > range partitioned

Re: stress test for parallel workers

2019-08-07 Thread Tom Lane
Heikki Linnakangas writes: > On 07/08/2019 16:57, Tom Lane wrote: >> Also, if you're using systemd or something else that thinks it >> ought to interfere with where cores get dropped, that could be >> a problem. > I think they should just go to a file called "core", I don't think I've > changed

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Sehrope Sarkuni
On Wed, Aug 7, 2019 at 7:19 AM Bruce Momjian wrote: > On Wed, Aug 7, 2019 at 05:13:31PM +0900, Masahiko Sawada wrote: > > I understood. IIUC in your approach postgres processes encrypt WAL > > records when inserting to the WAL buffer. So WAL data is encrypted > > even on the WAL buffer. > I

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Tom Lane wrote: > Something related I've been wondering about is whether we could > parallelize the isolation tests. A difficulty here is that the > slowest ones tend to also be timing-sensitive, such that running > them in parallel would increase the risk of failure. But we >

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Tom Lane
Pavel Stehule writes: > I should to use a cache accessed via fn_extra. There will be stored data > about function parameters (types). If I understand correctly, these data > should be stable in query, and then recheck is not necessary. Is it true? I wouldn't trust that. You don't really know

Re: Removing unneeded self joins

2019-08-07 Thread Konstantin Knizhnik
On 05.08.2019 14:24, Andrey Lepikhov wrote: On 02/08/2019 04:54, Thomas Munro wrote: On Thu, Jun 27, 2019 at 6:42 PM Andrey Lepikhov wrote: Version v.17 of the patch that fix the bug see in attachment. While moving this to the September CF, I noticed that it needs to be updated for the

Re: no default hash partition

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Tom Lane wrote: > Alvaro Herrera writes: > > Actually, it also says this (in the blurb for the PARTITION OF clause): > > > Creates the table as a partition of the > > specified > > parent table. The table can be created either as a partition for > > specific > >

crash 11.5~

2019-08-07 Thread Justin Pryzby
A daily report crashed repeatedly this morning running pg11.4. I compiled 11.5 and reproduced it there, too, so I'm including backtrace with -O0. I'm trying to dig further into it, but it seems to be crashing under load, but not when I try to narrow down to a single report, which seem to run to

Re: Unix-domain socket support on Windows

2019-08-07 Thread Peter Eisentraut
On 2019-08-07 16:06, Heikki Linnakangas wrote: > Am I reading the patches correctly, that getpeereid() still doesn't work > on Windows? That means that peer authentication doesn't work, right? > That's a bit sad. One of the big advantages of unix domain sockets over > TCP is peer

Re: no default hash partition

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Amit Langote wrote: > That hash-partitioned tables can't have default partition is mentioned > in the CREATE TABLE page: > > "If DEFAULT is specified, the table will be created as a default > partition of the parent table. The parent can either be a list or > range partitioned

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Tom Lane
Heikki Linnakangas writes: > The list of tests in src/test/isolation/isolation_schedule has grown > over the years. Originally, they were all related to Serializable > Snapshot Isolation, but there are different kinds of concurrency tests > there now. More tests is good, but the schedule file

Re: no default hash partition

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-06, Stephen Frost wrote: > Yeah, that's a fair argument, but giving the user a way to say that > would address it. As in, "create me a list-partitioned table for these > values, plus a default." Anyhow, I'm sure that I'm taking this beyond > what we need to do right now, just

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Tom Lane wrote: > The problem in "timeouts" is that it has to use drearily long timeouts > to be sure that the behavior will be stable even on really slow machines > (think CLOBBER_CACHE_ALWAYS or valgrind --- it can take seconds for them > to reach a waiting state that other

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Pavel Stehule
st 7. 8. 2019 v 18:39 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > st 7. 8. 2019 v 17:39 odesílatel Tom Lane napsal: > >> I wouldn't trust that. You don't really know what the lifespan of > >> a fn_extra cache is. > > > fn_extra cache cannot be longer than query. > > There are

Re: Unix-domain socket support on Windows

2019-08-07 Thread Magnus Hagander
On Wed, Aug 7, 2019 at 4:59 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-08-07 16:06, Heikki Linnakangas wrote: > > Am I reading the patches correctly, that getpeereid() still doesn't work > > on Windows? That means that peer authentication doesn't work, right? > >

Re: s/rewinded/rewound/?

2019-08-07 Thread Tom Lane
Magnus Hagander writes: > On Wed, Aug 7, 2019 at 10:49 AM Thomas Munro wrote: >> The word "rewinded" appears in our manual and in a comment. That >> sounds strange to my ears. Isn't it a mistake? Certainly. > To me this sounds like a classic non-English-native-speaker-mistake. But > it

Re: stress test for parallel workers

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 17:45, Tom Lane wrote: Heikki Linnakangas writes: On 07/08/2019 16:57, Tom Lane wrote: Also, if you're using systemd or something else that thinks it ought to interfere with where cores get dropped, that could be a problem. I think they should just go to a file called "core",

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 18:52, Alvaro Herrera wrote: On 2019-Aug-07, Tom Lane wrote: Something related I've been wondering about is whether we could parallelize the isolation tests. A difficulty here is that the slowest ones tend to also be timing-sensitive, such that running them in parallel would

Re: crash 11.5~ (and 11.4)

2019-08-07 Thread Justin Pryzby
Just found this, although I'm not sure what to do about it. If it's corrupt table data, I can restore from backup. ts=# VACUUM FREEZE VERBOSE child.huawei_umts_ucell_201908; INFO: 0: aggressively vacuuming "child.huawei_umts_ucell_201908" LOCATION: lazy_scan_heap, vacuumlazy.c:502 ERROR:

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Tom Lane
Pavel Stehule writes: > st 7. 8. 2019 v 17:39 odesílatel Tom Lane napsal: >> I wouldn't trust that. You don't really know what the lifespan of >> a fn_extra cache is. > fn_extra cache cannot be longer than query. There are fn_extra caches that are not tied to queries. Admittedly they're for

Re: no default hash partition

2019-08-07 Thread Tom Lane
Alvaro Herrera writes: > Actually, it also says this (in the blurb for the PARTITION OF clause): > Creates the table as a partition of the specified > parent table. The table can be created either as a partition for > specific > values using FOR VALUES or as a default

Re: Avoid full GIN index scan when possible

2019-08-07 Thread Tom Lane
Nikita Glukhov writes: > Attached 6th version of the patches. I spent a bit of time looking at these. Attached is a proposed revision of the 0001 patch, with some minor changes: * I didn't adopt your move of the "Non-default modes require the index to have placeholders" test to after the

Re: Problem with default partition pruning

2019-08-07 Thread Simon Riggs
On Tue, 6 Aug 2019 at 23:18, Alvaro Herrera wrote: > On 2019-Aug-06, Alvaro Herrera wrote: > > > Well, if this is really all that duplicative, one thing we could do is > > run this check in get_partprune_steps_internal only if > > constraint_exclusion is a value other than on; we should achieve

Re: crash 11.5~ (and 11.4)

2019-08-07 Thread Justin Pryzby
I checked this still happens with max_parallel_workers_per_gather=0. Now, I just reproduced using SELECT * FROM that table: (gdb) p thisatt->attrelid $4 = 2015128626 ts=# SELECT 2015128626::regclass; regclass | child.huawei_umts_ucell_201908 (gdb) p thisatt->attnum $1 = 2 (gdb) p attnum # For

Re: Duplicated LSN in ReorderBuffer

2019-08-07 Thread Andres Freund
Hi, On 2019-08-07 16:19:13 -0400, Alvaro Herrera wrote: > On 2019-Jul-26, Andres Freund wrote: > > > 2) We could simply assign the subtransaction to the parent using > >ReorderBufferAssignChild() in SnapBuildProcessNewCid() or it's > >caller. That ought to also fix the bug > > > >I

Re: Unused header file inclusion

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-05, Tom Lane wrote: > Alvaro Herrera writes: > > Then there's the removal, which is in tuplesort.c because of > > INT_MAX as added by commit d26559dbf356 and still present ... > > One has to be especially wary of removing system-header inclusions; > the fact that they don't seem to

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-07 Thread Ashwin Agrawal
On Fri, Aug 2, 2019 at 4:56 PM Ashwin Agrawal wrote: > > On Wed, Jul 31, 2019 at 2:06 PM Andres Freund wrote: > >> Looking at the code as of master, we currently have: >> > > Super awesome feedback and insights, thank you! > > - PredicateLockTuple() calls SubTransGetTopmostTransaction() to

Re: initdb: Use varargs macro for PG_CMD_PRINTF

2019-08-07 Thread Ibrar Ahmed
Hi, Patch does not apply, rebased patch on ( 68343b4ad75305391b38f4b42734dc07f2fe7ee2) attached On Wed, Aug 7, 2019 at 7:04 PM Tom Lane wrote: > Peter Eisentraut writes: > > Small patch to simplify some no longer necessary complicated code, using > > varargs macros. > > +1 > >

Re: Adding a test for speculative insert abort case

2019-08-07 Thread Melanie Plageman
On Wed, Jul 24, 2019 at 11:48 AM Andres Freund wrote: > > diff --git a/src/test/isolation/specs/insert-conflict-specconflict.spec > b/src/test/isolation/specs/insert-conflict-specconflict.spec > > index 3a70484fc2..7f29fb9d02 100644 > > ---

Re: no default hash partition

2019-08-07 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Aug-07, Tom Lane wrote: >> Hm, that's rather confusingly worded IMO. Is the antecedent of "this >> option" just DEFAULT, or does it mean that you can't use FOR VALUES, >> or perchance it means that you can't use a PARTITION OF clause >> at all? > Uh, you're

Re: Problem with default partition pruning

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Simon Riggs wrote: > I saw your recent commit and it scares me in various places, noted below. > > "Commit: Apply constraint exclusion more generally in partitioning" > > "This applies particularly to the default partition..." > > My understanding of the thread was the

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Aug-07, Tom Lane wrote: >> The problem in "timeouts" is that it has to use drearily long timeouts >> to be sure that the behavior will be stable even on really slow machines >> (think CLOBBER_CACHE_ALWAYS or valgrind --- it can take seconds for them >> to reach a

Documentation clarification re: ANALYZE

2019-08-07 Thread Isaac Morland
I'm looking at https://www.postgresql.org/docs/current/sql-analyze.html, where it says “Without a table_and_columns list, ANALYZE processes every table and materialized view in the current database that the current user has permission to analyze.”. I don’t believe there is a separate “analyze”

  1   2   >