Re: warning to publication created and wal_level is not set to logical

2019-07-08 Thread Lucas Viecelli
Follow the correct file, I added the wrong patch in the previous email > Attached is the rebased > thanks a lot *Lucas Viecelli* diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c index 1ac1a71bd9..902180cedc 100644 ---

Re: range_agg

2019-07-08 Thread Pavel Stehule
Hi po 8. 7. 2019 v 18:47 odesílatel Paul A Jungwirth < p...@illuminatedcomputing.com> napsal: > On Sat, Jul 6, 2019 at 12:13 PM Jeff Davis wrote: > > > > On Fri, 2019-07-05 at 09:58 -0700, Paul A Jungwirth wrote: > > > user-defined range types. So how about I start on it and see how it > > >

Re: tableam vs. TOAST

2019-07-08 Thread Prabhat Sahu
On Mon, Jul 8, 2019 at 9:06 PM Robert Haas wrote: > On Tue, Jun 25, 2019 at 2:19 AM Prabhat Sahu > wrote: > > I have tested the TOAST patches(v3) with different storage options > like(MAIN, EXTERNAL, EXTENDED, etc.), and > > combinations of compression and out-of-line storage options. > > I

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread David G. Johnston
On Mon, Jul 8, 2019 at 8:35 PM Bruce Momjian wrote: > On Mon, Jul 8, 2019 at 11:29:00PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: > > >>> I fixed that, but I'm wondering if we should back-patch that fix > > >>> or leave

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Ryan Lambert
Hey everyone, Here is my input regarding nonces and randomness. > As I understand it, the NIST recommendation is a 96-bit *random* nonce, I could not find that exact requirement in the NIST documents, though given the volume of that library it would be possible to miss. The recommendation I

Re: warning to publication created and wal_level is not set to logical

2019-07-08 Thread Lucas Viecelli
Hi Thomas. Attached is the rebased > The July Commitfest has started. This patch is in "Needs review" > status, but it doesn't apply. If I read the above discussion > correctly, it seems there is agreement that a warning here is a good > idea to commit this patch. Could you please post a

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 11:29:00PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: > >>> I fixed that, but I'm wondering if we should back-patch that fix > >>> or leave the back branches alone. > > >> +0.5 for back-patching. > > >

Re: [RFC] [PATCH] Flexible "partition pruning" hook

2019-07-08 Thread Mike Palmiotto
On Mon, Jul 8, 2019 at 10:59 AM Mike Palmiotto wrote: > > On Sun, Jul 7, 2019 at 9:46 PM Thomas Munro wrote: >> >> On Sat, Apr 6, 2019 at 3:06 PM Andres Freund wrote: >> > I've moved this to the next CF, and marked it as targeting v13. >> >> Hi Mike, >> >> Commifest 1 for PostgreSQL 13 is here.

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread Tom Lane
Bruce Momjian writes: > On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: >>> I fixed that, but I'm wondering if we should back-patch that fix >>> or leave the back branches alone. >> +0.5 for back-patching. > Uh, if this was done in a major release I am thinking we have to mention

Re: pgbench - add \aset to store results of a combined query

2019-07-08 Thread Ibrar Ahmed
> SELECT 1 AS one \; > SELECT 2 AS two UNION SELECT 2 \; > SELECT 3 AS three \aset > > will set both "one" and "three", while "two" is not set because there were > two rows. It is a kind of more permissive \gset. Are you sure two is not set :)? SELECT 2 AS two UNION SELECT 2; -- only

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Tue, Jul 09, 2019 at 03:04:27PM +1200, Thomas Munro wrote: > It's my mistake. I'll fix it later today. Thanks! -- Michael signature.asc Description: PGP signature

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread David G. Johnston
On Mon, Jul 8, 2019 at 7:59 PM Michael Paquier wrote: > Attached is an idea of patch for the documentation, using this > wording: > + > + > + When defining a primary key on a partitioned table, the primary > + key column must be included in the partition key. > + > +

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Tom Lane
Michael Paquier writes: > Attached is an idea of patch for the documentation, using this > wording: > + > + > + When defining a primary key on a partitioned table, the primary > + key column must be included in the partition key. > + > + Isn't it the other way

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 2:45 PM Michael Paquier wrote: > On Tue, Jul 09, 2019 at 02:30:51PM +1200, Thomas Munro wrote: > > Yeah. I had obviously never noticed that test script. +1 for just > > enabling hash joins the top of join_hash.sql in 12+, and the > > equivalent section in 11's join.sql

Re: doc: improve PG 12 to_timestamp()/to_date() wording

2019-07-08 Thread Bruce Momjian
On Sat, Jul 6, 2019 at 03:24:25PM -0500, Justin Pryzby wrote: > On Tue, Apr 30, 2019 at 07:14:04PM -0500, Justin Pryzby wrote: > > On Tue, Apr 30, 2019 at 09:48:14PM +0300, Alexander Korotkov wrote: > > > I'd like to add couple of comments from my side. > > > > > > - returns an error

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 10:37:37PM -0400, Bruce Momjian wrote: > On Fri, Jul 5, 2019 at 09:20:07PM +, PG Doc comments form wrote: >> In the documentation for Postgres 11 table partitioning, there is no mention >> of the requirement that the Primary Key of a partitioned table must contain >>

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Michael Paquier writes: > On Tue, Jul 09, 2019 at 02:30:51PM +1200, Thomas Munro wrote: >> Yeah. I had obviously never noticed that test script. +1 for just >> enabling hash joins the top of join_hash.sql in 12+, and the >> equivalent section in 11's join.sql (which is luckily at the end of >>

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Tue, Jul 09, 2019 at 02:30:51PM +1200, Thomas Munro wrote: > Yeah. I had obviously never noticed that test script. +1 for just > enabling hash joins the top of join_hash.sql in 12+, and the > equivalent section in 11's join.sql (which is luckily at the end of > the file). Right, I did not

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Bruce Momjian
On Fri, Jul 5, 2019 at 09:20:07PM +, PG Doc comments form wrote: > The following documentation comment has been logged on the website: > > Page: https://www.postgresql.org/docs/11/ddl-partitioning.html > Description: > > In the documentation for Postgres 11 table partitioning, there is no

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 2:22 PM Tom Lane wrote: > Thomas Munro writes: > > Based on a suggestion from Andres (if I recall correctly), I wrapped > > each individual test in savepoint/rollback, and then set just the GUCs > > needed to get the plan shape and execution code path I wanted to > >

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Thomas Munro writes: > On Tue, Jul 9, 2019 at 2:19 AM Tom Lane wrote: >> Given the purposes of this test, I think it'd be reasonable to force >> both enable_hashjoin = on and enable_mergejoin = off at the very top >> of the join_hash script, or the corresponding place in join.sql in >> v11.

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Michael Paquier writes: > On Mon, Jul 08, 2019 at 03:21:41PM -0400, Tom Lane wrote: >> Having said that, join_hash.sql in particular seems to have zero >> value if it's not testing hash joins, so I think it'd be reasonable >> for it to override a global enable_hashjoin = off setting. None of >>

RE: [PATCH] Speedup truncates of relation forks

2019-07-08 Thread Jamison, Kirk
Hi Thomas, Thanks for checking. > On Fri, Jul 5, 2019 at 3:03 PM Jamison, Kirk wrote: > > I updated the patch which is similar to V3 of the patch, but > > addressing my problem in (5) in the previous email regarding > FreeSpaceMapVacuumRange. > > It seems to pass the regression test now. Kindly

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 2:19 AM Tom Lane wrote: > Given the purposes of this test, I think it'd be reasonable to force > both enable_hashjoin = on and enable_mergejoin = off at the very top > of the join_hash script, or the corresponding place in join.sql in > v11. Thomas, was there a specific

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread James Coleman
Note: As I was writing this, I saw a new email come in from Tomas with a new patch series, and some similar observations. I'll look at that patch series more, but I think it's likely far more complete, so will end up going with that. I wanted to send this email anyway to at least capture the

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread Tomas Vondra
On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote: On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra wrote: On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote: >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra > wrote: >> >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 03:21:41PM -0400, Tom Lane wrote: > Having said that, join_hash.sql in particular seems to have zero > value if it's not testing hash joins, so I think it'd be reasonable > for it to override a global enable_hashjoin = off setting. None of > the other regression test

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread Bruce Momjian
On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: > > While I was fooling with it I noticed that the existing code for -n > > is buggy. The documentation says clearly that only the first > > argument is a candidate to be -n: > > > > If the first argument is an unquoted -n the

Re: Improve search for missing parent downlinks in amcheck

2019-07-08 Thread Peter Geoghegan
On Sun, Jul 7, 2019 at 7:53 PM Thomas Munro wrote: > On Wed, May 1, 2019 at 12:58 PM Peter Geoghegan wrote: > > I will think about a simple fix, but after the upcoming point release. > > There is no hurry. > > A bureaucratic question: What should the status be for this CF entry? I have plans

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-08 Thread Bruce Momjian
On Mon, Jul 1, 2019 at 09:51:12AM -0400, Alvaro Herrera wrote: > Now that we have REINDEX CONCURRENTLY, I think reindexdb is going to > gain more popularity. > > Please don't reuse a file name as generic as "parallel.c" -- it's > annoying when navigating source. Maybe conn_parallel.c

Re: Increasing default value for effective_io_concurrency?

2019-07-08 Thread Bruce Momjian
On Wed, Jul 3, 2019 at 11:42:49AM -0400, Robert Haas wrote: > On Wed, Jul 3, 2019 at 11:24 AM Tomas Vondra > wrote: > > Maybe. And it would probably work for the systems I used for benchmarks. > > > > It however assumes two things: (a) the storage system actually has > > spindles and (b) you

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 07:27:12PM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > Operationally, how would that work? We unlock them all on boot but > > somehow make them inaccessible to some backends after that? > > That could work and doesn't seem like an

Re: OpenSSL specific value under USE_SSL instead of USE_OPENSSL

2019-07-08 Thread Bruce Momjian
On Thu, Jun 27, 2019 at 04:02:45PM +0200, Daniel Gustafsson wrote: > The ssl_ciphers GUC for configuring cipher suites sets the default value based > on USE_SSL but it’s actually an OpenSSL specific value. As with other such > changes, it works fine now but will become an issue should we support

Re: Extra quote_all_identifiers in _dumpOptions

2019-07-08 Thread Bruce Momjian
On Thu, Jun 27, 2019 at 12:12:15PM +0300, Arthur Zakirov wrote: > Hello hackers, > > While working on pg_dump I noticed the extra quote_all_identifiers in > _dumpOptions struct. I attached the patch. > > It came from refactoring by 0eea8047bf. There is also a discussion: >

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 06:43:31PM -0400, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > > On

Re: Avoiding deadlock errors in CREATE INDEX CONCURRENTLY

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 10:33 AM Goel, Dhruv wrote: > I have attached the revised patch. I ran check-world multiple times on my > machine and it seems to succeed now. Do you mind kicking-off the CI build > with the latest patch? Thanks. It's triggered automatically when you post patches to the

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-07-08 Thread Thomas Munro
On Tue, Jul 2, 2019 at 5:46 PM Paul Guo wrote: > Rebased, aligned with recent changes in pg_rewind/pg_basebackup and then > retested. Thanks. Hi Paul, A minor build problem on Windows: src/bin/pg_rewind/pg_rewind.c(32): fatal error C1083: Cannot open include file: 'backup_common.h': No such

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 06:43:31PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > On Mon, Jul 8, 2019 at 05:41:51PM -0400, Stephen

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 06:23:13PM -0400, Bruce Momjian wrote: > Yes, 'postgres' can be used to create a nice md5 rainbow table that > works on many servers --- good point. Are rainbow tables possible with > something like AES? > > > I appreciate that *some* of this might not be completely

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Mon, Jul 8, 2019 at 05:41:51PM -0400, Stephen Frost wrote: > > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > >

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread Alexander Korotkov
On Thu, Jul 4, 2019 at 4:25 PM James Coleman wrote: > Process questions: > - Do I need to explicitly move the patch somehow to the next CF? We didn't manage to register it on current (July) commitfest. So, please, register it on next (September) commitfest. > - Since I've basically taken over

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread Alexander Korotkov
On Mon, Jun 3, 2019 at 12:18 AM Tomas Vondra wrote: > For a moment I thought we could/should look at the histogram, becase that > could tell us if there are groups "before" the first MCV one, but I don't > think we should do that, for two reasons. Firstly, rare values may not get > to the

Re: FETCH FIRST clause PERCENT option

2019-07-08 Thread Ryan Lambert
Hi Surafel, The v5 patch [1] applies cleanly and passes make installcheck-world. My concern with this is its performance. As a user I would expect using this feature to enable queries to run faster than they would simply pulling the full table. I tested on tables ranging from 10k rows up to 10

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Joe Conway
On 7/8/19 6:04 PM, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: >> Uh, well, renaming the user was a big problem, but that is the only case >> I can think of. I don't see that as an issue for block or WAL sequence >> numbers. If we want to use a different nonce, we have to

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jul 8, 2019 at 05:41:51PM -0400, Stephen Frost wrote: > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > Well, if it was a necessary features, I assume TLS

Detailed questions about pg_xact_commit_timestamp

2019-07-08 Thread Morris de Oryx
I have some specific questions about pg_xact_commit_timestamp, and am hoping that this is the right place to ask. I read a lot of the commentary about the original patch, and the contributors seem to be here. If I'm asking in the wrong place, just let me know. I'm working on a design for a

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 05:41:51PM -0400, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > Well, if it was a necessary features, I assume TLS 1.3 would have found > > > a way to make it secure, no? Certainly they are

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 05:41:51PM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > Well, if it was a necessary features, I assume TLS 1.3 would have found > > a way to make it secure, no? Certainly they are not shipping TLS 1.3 > > with a known weakness. > > As

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 02:39:44PM -0400, Stephen Frost wrote: > > > > Of course, we can discuss if what websites do with over-the-wire > > > > encryption is sensible to compare to what we want to do in PG for > > > > data-at-rest, but then

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 09:30:03PM +0200, Tomas Vondra wrote: > I think Bruce's proposal was to minimize the time the key is "unlocked" > in memory by only unlocking them when the user connects and supplies > some sort of secret (passphrase), and remove them from memory when the > user

Re: Excessive memory usage in multi-statement queries w/ partitioning

2019-07-08 Thread Tom Lane
Amit Langote writes: > [ parse-plan-memcxt_v2.patch ] I got around to looking at this finally. I'm not at all happy with the fact that it's added a plantree copy step to the only execution path through exec_simple_query. That's a very significant overhead, in many use-cases, to solve something

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 02:39:44PM -0400, Stephen Frost wrote: > > > Of course, we can discuss if what websites do with over-the-wire > > > encryption is sensible to compare to what we want to do in PG for > > > data-at-rest, but then we shouldn't be talking about what websites do, > > > it'd make

Re: Add missing operator <->(box, point)

2019-07-08 Thread Alexander Korotkov
On Mon, Jul 8, 2019 at 11:39 PM Nikita Glukhov wrote: > On 08.07.2019 18:22, Alexander Korotkov wrote: > For me it doesn't look worth having two distinct functions > gist_box_distance_helper() and gist_bbox_distance(). What about > having just one and leave responsibility for recheck flag to the

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-08 Thread Julien Rouhaud
On Mon, Jul 8, 2019 at 9:08 PM Julien Rouhaud wrote: > > I'll resubmit the parallel patch using per-table only > approach Attached. 0001-Export-vacuumdb-s-parallel-infrastructure.patch Description: Binary data 0002-Add-parallel-processing-to-reindexdb.patch Description: Binary data

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 12:09:58PM -0400, Joe Conway wrote: > On 7/8/19 11:56 AM, Peter Eisentraut wrote: > > On 2019-07-08 17:47, Stephen Frost wrote: > >> Of course, we can discuss if what websites do with over-the-wire > >> encryption is sensible to compare to what we want to do in PG for > >>

Re: Add missing operator <->(box, point)

2019-07-08 Thread Nikita Glukhov
Attached 3rd version of the patches. On 02.07.2019 21:55, Alexander Korotkov wrote: On Tue, Jul 2, 2019 at 9:19 PM Nikita Glukhov wrote: We could use commuted "const <-> var" operators for kNN searches, but the current implementation requires the existence of "var <-> const" operators, and

Re: Ltree syntax improvement

2019-07-08 Thread Alvaro Herrera
On 2019-Jul-08, Dmitry Belyavsky wrote: > I did not introduce any functions. I've just changed the parser. I mean the C-level functions -- count_parts_ors() and so on. > I'm not sure that it makes sense to remove any tests as most of them were > written to catch really happened bugs during the

Re: Built-in connection pooler

2019-07-08 Thread Konstantin Knizhnik
On 08.07.2019 3:37, Thomas Munro wrote: On Tue, Jul 2, 2019 at 3:11 AM Konstantin Knizhnik wrote: On 01.07.2019 12:57, Thomas Munro wrote: Interesting work. No longer applies -- please rebase. Rebased version of the patch is attached. Also all this version of built-ni proxy can be found

Re: Ltree syntax improvement

2019-07-08 Thread Dmitry Belyavsky
Dear Alvaro, On Mon, Jul 8, 2019 at 11:16 PM Alvaro Herrera wrote: > On 2019-Jul-08, Dmitry Belyavsky wrote: > > > Dear Thomas, > > > > Thank you for your proofreading! > > > > Please find the updated patch attached. It also contains the missing > > escaping. > > I think all these functions

Re: Ltree syntax improvement

2019-07-08 Thread Alvaro Herrera
On 2019-Jul-08, Dmitry Belyavsky wrote: > Dear Thomas, > > Thank you for your proofreading! > > Please find the updated patch attached. It also contains the missing > escaping. I think all these functions you're adding should have a short sentence explaining what it does. I'm not really

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Tomas Vondra
On Mon, Jul 08, 2019 at 12:09:58PM -0400, Joe Conway wrote: On 7/8/19 11:56 AM, Peter Eisentraut wrote: On 2019-07-08 17:47, Stephen Frost wrote: Of course, we can discuss if what websites do with over-the-wire encryption is sensible to compare to what we want to do in PG for data-at-rest, but

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Tomas Vondra
On Mon, Jul 08, 2019 at 12:16:04PM -0400, Bruce Momjian wrote: ... Anyway, I will to research the reasonable data size that can be secured with a single key via AES. I will look at how PGP encrypts large files too. IMO there are various recommendations about this, for example from NIST.

Assertion for logically decoding multi inserts into the catalog

2019-07-08 Thread Daniel Gustafsson
My patch for using heap_multi_insert in the catalog [1] failed the logical decoding part of test/recovery [2]. The assertion it failed on seems to not take multi inserts into the catalog into consideration, while the main logic does. This assertion hasn't tripped since there are no multi inserts

Re: Declared but no defined functions

2019-07-08 Thread Tom Lane
Ashwin Agrawal writes: > Do we wish to make this a tool and have it in src/tools, either as part of > find_static tool after renaming that one to more generic name or > independent script. Well, the scripts described so far are little more than jury-rigged hacks, with lots of room for false

Re: [HACKERS] Cached plans and statement generalization

2019-07-08 Thread Konstantin Knizhnik
On 08.07.2019 2:23, Thomas Munro wrote: On Tue, Jul 2, 2019 at 3:29 AM Konstantin Knizhnik wrote: Attached please find rebased version of the patch. Also this version can be found in autoprepare branch of this repository https://github.com/postgrespro/postgresql.builtin_pool.git on github.

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Tomas Vondra
On Mon, Jul 08, 2019 at 02:39:44PM -0400, Stephen Frost wrote: Greetings, * Bruce Momjian (br...@momjian.us) wrote: On Mon, Jul 8, 2019 at 11:47:33AM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jul 8, 2019 at 11:18:01AM -0400, Joe Conway wrote: > > >

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Michael Paquier writes: > Well, another thing I'd like to think about is if there is any point > to keep regressplans.sh and the relevant options in postgres at this > stage. From the top of the file one can read that: The point of regressplans.sh is to see if anything goes seriously wrong when

Re: tableam: abstracting relation sizing code

2019-07-08 Thread Robert Haas
On Wed, Jun 12, 2019 at 9:14 AM Robert Haas wrote: > On Tue, Jun 11, 2019 at 7:17 PM Andres Freund wrote: > > Just to understand: What version are you targeting? It seems pretty > > clearly v13 material to me? > > My current plan is to commit this to v13 as soon as the tree opens. Committed.

Re: Ltree syntax improvement

2019-07-08 Thread Dmitry Belyavsky
Dear Thomas, Thank you for your proofreading! Please find the updated patch attached. It also contains the missing escaping. On Mon, Jul 8, 2019 at 10:39 AM Thomas Munro wrote: > On Wed, Apr 17, 2019 at 5:29 AM Dmitry Belyavsky > wrote: > > I've applied your patch. > > From my point of view,

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-08 Thread Julien Rouhaud
On Mon, Jul 8, 2019 at 9:57 AM Michael Paquier wrote: > > On Fri, Jul 05, 2019 at 07:25:41PM +0200, Julien Rouhaud wrote: > > On Fri, Jul 5, 2019 at 6:16 PM Peter Eisentraut > > wrote: > >> Isn't that also the case for your proposal? We are not going to release > >> a new reindexdb before a

Re: Declared but no defined functions

2019-07-08 Thread Ashwin Agrawal
On Sat, Jul 6, 2019 at 4:32 PM Masahiko Sawada wrote: > Indeed. I've tried to search again with the following script and got > more such functions. > > for func in `git ls-files | egrep "\w+\.h$" | xargs cat | egrep -v > "(^typedef)|(DECLARE)|(BKI)" | egrep "^(extern )*[\_0-9A-Za-z]+ >

Re: progress report for ANALYZE

2019-07-08 Thread Julien Rouhaud
On Mon, Jul 8, 2019 at 8:44 PM Robert Haas wrote: > > On Mon, Jul 8, 2019 at 2:18 PM Alvaro Herrera > wrote: > > Yeah, I got the impression that that was determined to be the desirable > > behavior, so I made it do that, but I'm not really happy about it > > either. We're not too late to

Re: progress report for ANALYZE

2019-07-08 Thread Robert Haas
On Mon, Jul 8, 2019 at 2:18 PM Alvaro Herrera wrote: > Yeah, I got the impression that that was determined to be the desirable > behavior, so I made it do that, but I'm not really happy about it > either. We're not too late to change the CREATE INDEX behavior, but > let's discuss what is it that

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 11:47:33AM -0400, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Mon, Jul 8, 2019 at 11:18:01AM -0400, Joe Conway wrote: > > > > On 7/8/19 10:19 AM, Bruce Momjian wrote: > > > > > When

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Tomas Vondra
On Mon, Jul 08, 2019 at 11:25:10AM -0400, Bruce Momjian wrote: On Mon, Jul 8, 2019 at 11:18:01AM -0400, Joe Conway wrote: On 7/8/19 10:19 AM, Bruce Momjian wrote: > When people are asking for multiple keys (not just for key rotation), > they are asking to have multiple keys that can be

Re: Multivariate MCV list vs. statistics target

2019-07-08 Thread Tomas Vondra
On Sun, Jul 07, 2019 at 12:02:38AM +0200, Tomas Vondra wrote: Hi, apparently v1 of the ALTER STATISTICS patch was a bit confused about length of the VacAttrStats array passed to statext_compute_stattarget, causing segfaults. Attached v2 patch fixes that, and it also makes sure we print warnings

Re: progress report for ANALYZE

2019-07-08 Thread Alvaro Herrera
On 2019-Jul-08, Robert Haas wrote: > On Mon, Jul 8, 2019 at 5:29 AM Tatsuro Yamada > wrote: > > 17520|13599|postgres|16387|f|16387|scanning table|4425|4425 > > 17520|13599|postgres|16387|f|16387|analyzing sample|0|0 > > 17520|13599|postgres|16387|f|16387||0|0 <-- Is it Okay?? > > Why do we

Re: progress report for ANALYZE

2019-07-08 Thread Robert Haas
On Mon, Jul 8, 2019 at 5:29 AM Tatsuro Yamada wrote: > 17520|13599|postgres|16387|f|16387|scanning table|4425|4425 > 17520|13599|postgres|16387|f|16387|analyzing sample|0|0 > 17520|13599|postgres|16387|f|16387||0|0 <-- Is it Okay?? Why do we zero out the block numbers when we switch phases?

Re: [PATCH] kNN for btree

2019-07-08 Thread Alexander Korotkov
On Mon, Jul 1, 2019 at 8:47 PM Nikita Glukhov wrote: > On 01.07.2019 13:41, Thomas Munro wrote: > > On Tue, Mar 26, 2019 at 4:30 AM Nikita Glukhov > > wrote: > Fixed two bugs in patches 3 and 10 (see below). > Patch 3 was extracted from the main patch 5 (patch 4 in v9). > >>> This

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Mon, Jul 8, 2019 at 5:53 PM Michael Paquier wrote: > I have begun playing with regressplans.sh which enforces various > combinations of "-f s|i|n|m|h" when running the regression tests, and > I have noticed that -fh can cause the server to become stuck in the > test join_hash.sql with this

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 06:49:44PM +1200, Thomas Munro wrote: > "simple" has 20k rows and "extremely_skewed" has 20k rows but the > planner thinks it only has 2. So this going to take O(n^2) time and n > is 20k. Not sure what to do about that. Maybe "join_hash" should be > skipped for the -h (=

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Michael Paquier writes: > I have begun playing with regressplans.sh which enforces various > combinations of "-f s|i|n|m|h" when running the regression tests, and > I have noticed that -fh can cause the server to become stuck in the > test join_hash.sql with this query (not sure which portion of

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Robert Haas
On Mon, Jul 8, 2019 at 11:08 AM Tom Lane wrote: > FWIW, I was imagining the action as being (1) detach all the child > partitions, (2) make parent into a non-partitioned table, (3) > drop the target column in each of these now-independent tables. > No data movement. Other than the need to

Re: Comment typo in tableam.h

2019-07-08 Thread Ashwin Agrawal
On Sat, Jul 6, 2019 at 12:05 AM Amit Kapila wrote: > On Tue, Jul 2, 2019 at 1:00 AM Ashwin Agrawal wrote: > > Please find attached v2 of patch 1 without objectionable comment change. > v1 of patch 2 attaching here just for convenience, no modifications made to > it. > > > > 0001* > * See

Re: range_agg

2019-07-08 Thread Paul A Jungwirth
On Sat, Jul 6, 2019 at 12:13 PM Jeff Davis wrote: > > On Fri, 2019-07-05 at 09:58 -0700, Paul A Jungwirth wrote: > > user-defined range types. So how about I start on it and see how it > > goes? I expect I can follow the existing code for range types pretty > > closely, so maybe it won't be too

Re: Switching PL/Python to Python 3 by default in PostgreSQL 12

2019-07-08 Thread Tom Lane
I wrote: > But I could support having a way for individual installations to change > what the synonym means locally. Perhaps we could think about how to do > that in conjunction with the project of getting rid of pg_pltemplate > that's been kicked around before [1][2][3]. ... actually, if we had

Re: errbacktrace

2019-07-08 Thread Alvaro Herrera
On 2019-Jul-08, Dmitry Dolgov wrote: > > On Sat, Jun 29, 2019 at 7:41 AM Jaime Casanova > > wrote: > > > > This is certainly a very useful thing. Sadly, it doesn't seem to compile > > when > > trying to use libunwind. > > Yeah, the same for me. To make it works I've restricted libunwind to

Re: Switching PL/Python to Python 3 by default in PostgreSQL 12

2019-07-08 Thread Tom Lane
Peter Eisentraut writes: > I'm not trying to dismiss the importance of managing the Python > transition. But this issue has been known for many years, and the > current setup is more or less in line with the wider world. For > example, the Debian release that came out over the weekend still

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 11:47:33AM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jul 8, 2019 at 11:18:01AM -0400, Joe Conway wrote: > > > On 7/8/19 10:19 AM, Bruce Momjian wrote: > > > > When people are asking for multiple keys (not just for

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Joe Conway
On 7/8/19 11:56 AM, Peter Eisentraut wrote: > On 2019-07-08 17:47, Stephen Frost wrote: >> Of course, we can discuss if what websites do with over-the-wire >> encryption is sensible to compare to what we want to do in PG for >> data-at-rest, but then we shouldn't be talking about what websites do,

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread James Coleman
On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra wrote: > > On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote: > >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra > > wrote: > >> > >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote: > >> >On Sun, Jul 7, 2019 at 5:02 PM Tomas

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Peter Eisentraut
On 2019-07-08 17:47, Stephen Frost wrote: > Of course, we can discuss if what websites do with over-the-wire > encryption is sensible to compare to what we want to do in PG for > data-at-rest, but then we shouldn't be talking about what websites do, > it'd make more sense to look at other

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 11:18:01AM -0400, Joe Conway wrote: > > On 7/8/19 10:19 AM, Bruce Momjian wrote: > > > When people are asking for multiple keys (not just for key rotation), > > > they are asking to have multiple keys that can be

Re: tableam vs. TOAST

2019-07-08 Thread Robert Haas
On Tue, Jun 25, 2019 at 2:19 AM Prabhat Sahu wrote: > I have tested the TOAST patches(v3) with different storage options like(MAIN, > EXTERNAL, EXTENDED, etc.), and > combinations of compression and out-of-line storage options. > I have used a few dummy tables with various tuple count say 10k,

Re: errbacktrace

2019-07-08 Thread Dmitry Dolgov
> On Sat, Jun 29, 2019 at 7:41 AM Jaime Casanova > wrote: > > This is certainly a very useful thing. Sadly, it doesn't seem to compile when > trying to use libunwind. Yeah, the same for me. To make it works I've restricted libunwind to local unwinding only: #ifdef USE_LIBUNWIND #define

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 11:18:01AM -0400, Joe Conway wrote: > On 7/8/19 10:19 AM, Bruce Momjian wrote: > > When people are asking for multiple keys (not just for key rotation), > > they are asking to have multiple keys that can be supplied by users only > > when they need to access the data. Yes,

Re: Add missing operator <->(box, point)

2019-07-08 Thread Alexander Korotkov
On Mon, Mar 11, 2019 at 2:49 AM Nikita Glukhov wrote: > 2. Add <-> to GiST box_ops. >Extracted gist_box_distance_helper() common for gist_box_distance() and >gist_bbox_distance(). For me it doesn't look worth having two distinct functions gist_box_distance_helper() and

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Tom Lane
Alvaro Herrera writes: > That said, I'm not sure I see the use case for an ALTER TABLE .. DROP > COLUMN command that turns a partitioned table (with existing partitions > containing data) into one non-partitioned table with all data minus the > partitioning column(s). Yeah, it'd be a lot of work

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Tom Lane
Robert Haas writes: > On Mon, Jul 8, 2019 at 11:02 AM Robert Haas wrote: >> On Mon, Jul 8, 2019 at 10:32 AM Alvaro Herrera >> wrote: >>> That said, I'm not sure I see the use case for an ALTER TABLE .. DROP >>> COLUMN command that turns a partitioned table (with existing partitions >>>

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Joe Conway
On 7/8/19 10:19 AM, Bruce Momjian wrote: > When people are asking for multiple keys (not just for key rotation), > they are asking to have multiple keys that can be supplied by users only > when they need to access the data. Yes, the keys are always in the > datbase, but the feature request is

  1   2   >