Re: Index Skip Scan

2019-07-10 Thread Jesper Pedersen
Hi, On 7/9/19 10:14 PM, Thomas Munro wrote: Thomas, do you have any ideas for this ? I can see that MySQL did the functionality in two change sets (base and function support), but like you said we shouldn't paint ourselves into a corner. I think amskip() could be augmented by later patches to

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-10 Thread Melanie Plageman
On Wed, Jul 10, 2019 at 12:40 AM Michael Paquier wrote: > On Wed, Jul 10, 2019 at 12:51:28PM +0530, Amit Kapila wrote: > > It would be good if we can come up with something like that. It will > > be helpful for zheap, where in some cases we get different row > > ordering due to in-place updates.

Re: pg_checksums (or checksums in general) vs tableam

2019-07-10 Thread Michael Paquier
On Wed, Jul 10, 2019 at 11:42:34AM +0200, Magnus Hagander wrote: > pg_checksums enumerate the files. What if there are files there from a > different tableam? Isn't pg_checksums just going to badly fail then, since > it assumes everything is heap? > > Also, do we allow AMs that don't support check

Re: pg_receivewal documentation

2019-07-10 Thread Jesper Pedersen
Hi, On 7/10/19 4:04 AM, Michael Paquier wrote: On Wed, Jul 10, 2019 at 12:22:02AM +0200, Laurenz Albe wrote: Works for me. Marked as "ready for committer". Hmm. synchronous_commit is user-settable, which means that it is possible to enforce a value in the connection string doing the connect

Re: extension patch of CREATE OR REPLACE TRIGGER

2019-07-10 Thread Surafel Temesgen
Hi Takamichi Osumi, On Tue, Jul 9, 2019 > I've rebased the previous patch to be applied > I don't test your patch fully yet but here are same comment. There are same white space issue like here - bool is_internal) + bool is_internal, + Oid existing_constraint_oid) in a few place + // trigoid

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Joe Conway
On 7/10/19 8:34 AM, Stephen Frost wrote: > Greetings, > > * Joe Conway (m...@joeconway.com) wrote: >> On 7/9/19 7:28 PM, Stephen Frost wrote: >> > * Joe Conway (m...@joeconway.com) wrote: >> >> On 7/9/19 5:42 PM, Tomas Vondra wrote: >> >> > There are two basic ways to construct nonces - CSPRNG and

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Joe Conway
On 7/10/19 2:45 AM, Masahiko Sawada wrote: > On Wed, Jul 10, 2019 at 11:06 AM Stephen Frost wrote: >> >> Greetings, >> >> * Ryan Lambert (r...@rustprooflabs.com) wrote: >> > > What I think Tomas is getting at here is that we don't write a page only >> > > once. >> > >> > > A nonce of tableoid+page

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Stephen Frost
Greetings, * Joe Conway (m...@joeconway.com) wrote: > On 7/9/19 7:28 PM, Stephen Frost wrote: > > * Joe Conway (m...@joeconway.com) wrote: > >> On 7/9/19 5:42 PM, Tomas Vondra wrote: > >> > There are two basic ways to construct nonces - CSPRNG and sequences, and > >> > then a combination of both,

Re: pg_receivewal documentation

2019-07-10 Thread Jesper Pedersen
Hi, On 7/9/19 6:22 PM, Laurenz Albe wrote: Works for me. Marked as "ready for committer". Thank you ! Best regards, Jesper

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Joe Conway
On 7/10/19 4:24 AM, Antonin Houska wrote: > Joe Conway wrote: > >> On 7/8/19 6:04 PM, Stephen Frost wrote: >> > * Bruce Momjian (br...@momjian.us) wrote: >> >> Uh, well, renaming the user was a big problem, but that is the only case >> >> I can think of. I don't see that as an issue for block or

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Joe Conway
On 7/10/19 4:47 AM, Antonin Houska wrote: > Tomas Vondra wrote: >> I don't think that works, because that'd mean we're encrypting the same >> page with the same nonce over and over, which means reusing the reuse >> (even if you hash/encrypt it). Or did I miss something? > > I found out that it's

Re: Comment typo in tableam.h

2019-07-10 Thread Amit Kapila
On Mon, Jul 8, 2019 at 10:21 PM Ashwin Agrawal wrote: > > > On Sat, Jul 6, 2019 at 12:05 AM Amit Kapila wrote: >> >> On Tue, Jul 2, 2019 at 1:00 AM Ashwin Agrawal wrote: >> > Please find attached v2 of patch 1 without objectionable comment change. >> > v1 of patch 2 attaching here just for conv

Re: Minimal logical decoding on standbys

2019-07-10 Thread Amit Khandekar
On Wed, 10 Jul 2019 at 08:44, Andres Freund wrote: > > Hi, > > Thanks for the new version! Looks like we're making progress towards > something committable here. > > I think it'd be good to split the patch into a few pieces. I'd maybe do > that like: > 1) WAL format changes (plus required other ch

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Joe Conway
On 7/10/19 2:38 AM, Masahiko Sawada wrote: > On Tue, Jul 9, 2019 at 9:01 PM Joe Conway wrote: >> >> On 7/9/19 6:07 AM, Peter Eisentraut wrote: >> > On 2019-07-08 18:09, Joe Conway wrote: >> >> In my mind, and in practice to a >> >> large extent, a postgres tablespace == a unique mount point. >> >

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Joe Conway
On 7/10/19 2:40 AM, Masahiko Sawada wrote: > On Tue, Jul 9, 2019 at 10:16 PM Joe Conway wrote: >> >> On 7/9/19 8:39 AM, Ryan Lambert wrote: >> > Hi Thomas, >> > >> >> CBC mode does require >> >> random nonces, other modes may be fine with even sequences as long as >> >> the values are not reused.

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Joe Conway
On 7/9/19 10:06 PM, Stephen Frost wrote: > Greetings, > > * Ryan Lambert (r...@rustprooflabs.com) wrote: >> > What I think Tomas is getting at here is that we don't write a page only >> > once. >> >> > A nonce of tableoid+pagenum will only be unique the first time we write >> > out that page. Se

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Joe Conway
On 7/9/19 7:28 PM, Stephen Frost wrote: > Greetings, > > * Joe Conway (m...@joeconway.com) wrote: >> On 7/9/19 5:42 PM, Tomas Vondra wrote: >> > There are two basic ways to construct nonces - CSPRNG and sequences, and >> > then a combination of both, i.e. one part is generated from a sequence >> >

pg_checksums (or checksums in general) vs tableam

2019-07-10 Thread Magnus Hagander
How is this intended to work? pg_checksums enumerate the files. What if there are files there from a different tableam? Isn't pg_checksums just going to badly fail then, since it assumes everything is heap? Also, do we allow AMs that don't support checksumming data? Do we have any checks for tabl

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-07-10 Thread David Rowley
On Wed, 3 Jul 2019 at 19:35, Michael Paquier wrote: > This has been reverted as of f5db56f, still it seems to me that this > was moving in the right direction. I've pushed this again, this time with the cleanup code done in the right order. -- David Rowley http://www.2ndQuadr

Re: pg_log_fatal vs pg_log_error

2019-07-10 Thread Antonin Houska
Peter Eisentraut wrote: > On 2019-06-17 14:19, Antonin Houska wrote: > > Can anyone please give me a hint (and possibly add some comments to the > > code) > > when pg_log_fatal() should be used in frontend code and when it's > > appropriate > > to call pg_log_error()? The current use does not s

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Antonin Houska
Tomas Vondra wrote: > On Tue, Jul 09, 2019 at 03:50:39PM -0400, Bruce Momjian wrote: > >On Tue, Jul 9, 2019 at 02:09:38PM -0400, Joe Conway wrote: > >> On 7/9/19 11:11 AM, Bruce Momjian wrote: > >> > Good point about nonce and IV. I wonder if running the nonce > >> > through the cipher with the

Re: Excessive memory usage in multi-statement queries w/ partitioning

2019-07-10 Thread Kyotaro Horiguchi
Hi, At Wed, 10 Jul 2019 16:35:18 +0900, Amit Langote wrote in > On Tue, Jul 9, 2019 at 6:21 AM Tom Lane wrote: > > > > Amit Langote writes: > > > [ parse-plan-memcxt_v2.patch ] > > > > I got around to looking at this finally. > > Thanks for the review. > > > I'm not at all happy with > > th

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-10 Thread Antonin Houska
Joe Conway wrote: > On 7/8/19 6:04 PM, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > >> Uh, well, renaming the user was a big problem, but that is the only case > >> I can think of. I don't see that as an issue for block or WAL sequence > >> numbers. If we want to use a d

Re: pg_receivewal documentation

2019-07-10 Thread Michael Paquier
On Wed, Jul 10, 2019 at 12:22:02AM +0200, Laurenz Albe wrote: > Works for me. > > Marked as "ready for committer". Hmm. synchronous_commit is user-settable, which means that it is possible to enforce a value in the connection string doing the connection. Isn't that something we had better enfor

Re: FETCH FIRST clause PERCENT option

2019-07-10 Thread Kyotaro Horiguchi
Hello. At Wed, 10 Jul 2019 15:02:57 +0900 (Tokyo Standard Time), Kyotaro Horiguchi wrote in <20190710.150257.260806103.horikyota@gmail.com> > It is seen by a simpler test. > > create table t as select a from generate_series(0, 9) a; > analyze t; > explain analyze select * from t order

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-10 Thread Michael Paquier
On Wed, Jul 10, 2019 at 12:51:28PM +0530, Amit Kapila wrote: > It would be good if we can come up with something like that. It will > be helpful for zheap, where in some cases we get different row > ordering due to in-place updates. As of now, we try to add Order By > or do some extra magic to ge

Re: Excessive memory usage in multi-statement queries w/ partitioning

2019-07-10 Thread Amit Langote
On Tue, Jul 9, 2019 at 6:21 AM Tom Lane wrote: > > Amit Langote writes: > > [ parse-plan-memcxt_v2.patch ] > > I got around to looking at this finally. Thanks for the review. > I'm not at all happy with > the fact that it's added a plantree copy step to the only execution > path through exec_si

Re: Contribution to Perldoc for TestLib module in Postgres

2019-07-10 Thread Michael Paquier
On Tue, Jul 09, 2019 at 03:16:01PM +0200, Daniel Gustafsson wrote: > The v2 patch is somewhat confused as it has Windows carriage returns rather > than newlines, so it replaces the entire file making the diff hard to read. > It > also includes a copy of TestLib and the v1 patch and has a lot of w

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-07-10 Thread Michael Paquier
On Tue, Jul 09, 2019 at 10:48:49PM +0800, Paul Guo wrote: > Yes, the patches changed Makefile so that pg_rewind and pg_basebackup could > use some common code, but for Windows build, I'm not sure where are those > window build files. Does anyone know about that? Thanks. The VS scripts are located

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-10 Thread Amit Kapila
On Wed, Jul 10, 2019 at 10:12 AM Michael Paquier wrote: > > On Tue, Jul 09, 2019 at 11:54:29AM -0700, Melanie Plageman wrote: > > It might be worth post-processing results files to ignore row ordering > > in some cases to allow for easier comparison. Has this been proposed > > in the past? > > Not

make -jn fails by requiring not-yet-generated include files.

2019-07-10 Thread Kyotaro Horiguchi
Sorry, the subject of the previous mail was wrong. I resend it with the correct subject. I found the following make's behavior is annoying (at dab81b9953). make distclean ./configure make all make -j4 clean all relpath.c:21:10: fatal error: catalog/pg_tablespace_d.h: No such file or

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-10 Thread Michael Paquier
On Tue, Jul 09, 2019 at 06:59:59PM -0400, Tom Lane wrote: > Alvaro Herrera writes: >> >> >>Unique constraints on partitioned tables (as well as primary keys) >>must constrain all the partition key columns. This limitation exists >>because PostgreSQL can only e

<    1   2