Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread Alexander Korotkov
On Mon, Jun 3, 2019 at 12:18 AM Tomas Vondra wrote: > For a moment I thought we could/should look at the histogram, becase that > could tell us if there are groups "before" the first MCV one, but I don't > think we should do that, for two reasons. Firstly, rare values may not get > to the

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 06:43:31PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > On Mon, Jul 8, 2019 at 05:41:51PM -0400, Stephen

Re: Increasing default value for effective_io_concurrency?

2019-07-08 Thread Bruce Momjian
On Wed, Jul 3, 2019 at 11:42:49AM -0400, Robert Haas wrote: > On Wed, Jul 3, 2019 at 11:24 AM Tomas Vondra > wrote: > > Maybe. And it would probably work for the systems I used for benchmarks. > > > > It however assumes two things: (a) the storage system actually has > > spindles and (b) you

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Tue, Jul 09, 2019 at 03:04:27PM +1200, Thomas Munro wrote: > It's my mistake. I'll fix it later today. Thanks! -- Michael signature.asc Description: PGP signature

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Joe Conway
On 7/8/19 6:04 PM, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: >> Uh, well, renaming the user was a big problem, but that is the only case >> I can think of. I don't see that as an issue for block or WAL sequence >> numbers. If we want to use a different nonce, we have to

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jul 8, 2019 at 05:41:51PM -0400, Stephen Frost wrote: > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > Well, if it was a necessary features, I assume TLS

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread Alexander Korotkov
On Thu, Jul 4, 2019 at 4:25 PM James Coleman wrote: > Process questions: > - Do I need to explicitly move the patch somehow to the next CF? We didn't manage to register it on current (July) commitfest. So, please, register it on next (September) commitfest. > - Since I've basically taken over

Re: Avoiding deadlock errors in CREATE INDEX CONCURRENTLY

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 10:33 AM Goel, Dhruv wrote: > I have attached the revised patch. I ran check-world multiple times on my > machine and it seems to succeed now. Do you mind kicking-off the CI build > with the latest patch? Thanks. It's triggered automatically when you post patches to the

Re: Improve search for missing parent downlinks in amcheck

2019-07-08 Thread Peter Geoghegan
On Sun, Jul 7, 2019 at 7:53 PM Thomas Munro wrote: > On Wed, May 1, 2019 at 12:58 PM Peter Geoghegan wrote: > > I will think about a simple fix, but after the upcoming point release. > > There is no hurry. > > A bureaucratic question: What should the status be for this CF entry? I have plans

RE: [PATCH] Speedup truncates of relation forks

2019-07-08 Thread Jamison, Kirk
Hi Thomas, Thanks for checking. > On Fri, Jul 5, 2019 at 3:03 PM Jamison, Kirk wrote: > > I updated the patch which is similar to V3 of the patch, but > > addressing my problem in (5) in the previous email regarding > FreeSpaceMapVacuumRange. > > It seems to pass the regression test now. Kindly

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 2:22 PM Tom Lane wrote: > Thomas Munro writes: > > Based on a suggestion from Andres (if I recall correctly), I wrapped > > each individual test in savepoint/rollback, and then set just the GUCs > > needed to get the plan shape and execution code path I wanted to > >

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 2:45 PM Michael Paquier wrote: > On Tue, Jul 09, 2019 at 02:30:51PM +1200, Thomas Munro wrote: > > Yeah. I had obviously never noticed that test script. +1 for just > > enabling hash joins the top of join_hash.sql in 12+, and the > > equivalent section in 11's join.sql

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 11:29:00PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: > >>> I fixed that, but I'm wondering if we should back-patch that fix > >>> or leave the back branches alone. > > >> +0.5 for back-patching. > > >

Re: OpenSSL specific value under USE_SSL instead of USE_OPENSSL

2019-07-08 Thread Bruce Momjian
On Thu, Jun 27, 2019 at 04:02:45PM +0200, Daniel Gustafsson wrote: > The ssl_ciphers GUC for configuring cipher suites sets the default value based > on USE_SSL but it’s actually an OpenSSL specific value. As with other such > changes, it works fine now but will become an issue should we support

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread Tomas Vondra
On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote: On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra wrote: On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote: >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra > wrote: >> >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Michael Paquier writes: > On Mon, Jul 08, 2019 at 03:21:41PM -0400, Tom Lane wrote: >> Having said that, join_hash.sql in particular seems to have zero >> value if it's not testing hash joins, so I think it'd be reasonable >> for it to override a global enable_hashjoin = off setting. None of >>

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Tue, Jul 09, 2019 at 02:30:51PM +1200, Thomas Munro wrote: > Yeah. I had obviously never noticed that test script. +1 for just > enabling hash joins the top of join_hash.sql in 12+, and the > equivalent section in 11's join.sql (which is luckily at the end of > the file). Right, I did not

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 10:37:37PM -0400, Bruce Momjian wrote: > On Fri, Jul 5, 2019 at 09:20:07PM +, PG Doc comments form wrote: >> In the documentation for Postgres 11 table partitioning, there is no mention >> of the requirement that the Primary Key of a partitioned table must contain >>

Re: tableam vs. TOAST

2019-07-08 Thread Prabhat Sahu
On Mon, Jul 8, 2019 at 9:06 PM Robert Haas wrote: > On Tue, Jun 25, 2019 at 2:19 AM Prabhat Sahu > wrote: > > I have tested the TOAST patches(v3) with different storage options > like(MAIN, EXTERNAL, EXTENDED, etc.), and > > combinations of compression and out-of-line storage options. > > I

Detailed questions about pg_xact_commit_timestamp

2019-07-08 Thread Morris de Oryx
I have some specific questions about pg_xact_commit_timestamp, and am hoping that this is the right place to ask. I read a lot of the commentary about the original patch, and the contributors seem to be here. If I'm asking in the wrong place, just let me know. I'm working on a design for a

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread Bruce Momjian
On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: > > While I was fooling with it I noticed that the existing code for -n > > is buggy. The documentation says clearly that only the first > > argument is a candidate to be -n: > > > > If the first argument is an unquoted -n the

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Michael Paquier writes: > On Tue, Jul 09, 2019 at 02:30:51PM +1200, Thomas Munro wrote: >> Yeah. I had obviously never noticed that test script. +1 for just >> enabling hash joins the top of join_hash.sql in 12+, and the >> equivalent section in 11's join.sql (which is luckily at the end of >>

Re: doc: improve PG 12 to_timestamp()/to_date() wording

2019-07-08 Thread Bruce Momjian
On Sat, Jul 6, 2019 at 03:24:25PM -0500, Justin Pryzby wrote: > On Tue, Apr 30, 2019 at 07:14:04PM -0500, Justin Pryzby wrote: > > On Tue, Apr 30, 2019 at 09:48:14PM +0300, Alexander Korotkov wrote: > > > I'd like to add couple of comments from my side. > > > > > > - returns an error

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread David G. Johnston
On Mon, Jul 8, 2019 at 7:59 PM Michael Paquier wrote: > Attached is an idea of patch for the documentation, using this > wording: > + > + > + When defining a primary key on a partitioned table, the primary > + key column must be included in the partition key. > + > +

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Mon, Jul 8, 2019 at 05:41:51PM -0400, Stephen Frost wrote: > > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > >

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-07-08 Thread Thomas Munro
On Tue, Jul 2, 2019 at 5:46 PM Paul Guo wrote: > Rebased, aligned with recent changes in pg_rewind/pg_basebackup and then > retested. Thanks. Hi Paul, A minor build problem on Windows: src/bin/pg_rewind/pg_rewind.c(32): fatal error C1083: Cannot open include file: 'backup_common.h': No such

Re: Extra quote_all_identifiers in _dumpOptions

2019-07-08 Thread Bruce Momjian
On Thu, Jun 27, 2019 at 12:12:15PM +0300, Arthur Zakirov wrote: > Hello hackers, > > While working on pg_dump I noticed the extra quote_all_identifiers in > _dumpOptions struct. I attached the patch. > > It came from refactoring by 0eea8047bf. There is also a discussion: >

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 2:19 AM Tom Lane wrote: > Given the purposes of this test, I think it'd be reasonable to force > both enable_hashjoin = on and enable_mergejoin = off at the very top > of the join_hash script, or the corresponding place in join.sql in > v11. Thomas, was there a specific

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread David G. Johnston
On Mon, Jul 8, 2019 at 8:35 PM Bruce Momjian wrote: > On Mon, Jul 8, 2019 at 11:29:00PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: > > >>> I fixed that, but I'm wondering if we should back-patch that fix > > >>> or leave

Re: FETCH FIRST clause PERCENT option

2019-07-08 Thread Ryan Lambert
Hi Surafel, The v5 patch [1] applies cleanly and passes make installcheck-world. My concern with this is its performance. As a user I would expect using this feature to enable queries to run faster than they would simply pulling the full table. I tested on tables ranging from 10k rows up to 10

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 06:23:13PM -0400, Bruce Momjian wrote: > Yes, 'postgres' can be used to create a nice md5 rainbow table that > works on many servers --- good point. Are rainbow tables possible with > something like AES? > > > I appreciate that *some* of this might not be completely

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 03:21:41PM -0400, Tom Lane wrote: > Having said that, join_hash.sql in particular seems to have zero > value if it's not testing hash joins, so I think it'd be reasonable > for it to override a global enable_hashjoin = off setting. None of > the other regression test

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Thomas Munro writes: > On Tue, Jul 9, 2019 at 2:19 AM Tom Lane wrote: >> Given the purposes of this test, I think it'd be reasonable to force >> both enable_hashjoin = on and enable_mergejoin = off at the very top >> of the join_hash script, or the corresponding place in join.sql in >> v11.

Re: range_agg

2019-07-08 Thread Pavel Stehule
Hi po 8. 7. 2019 v 18:47 odesílatel Paul A Jungwirth < p...@illuminatedcomputing.com> napsal: > On Sat, Jul 6, 2019 at 12:13 PM Jeff Davis wrote: > > > > On Fri, 2019-07-05 at 09:58 -0700, Paul A Jungwirth wrote: > > > user-defined range types. So how about I start on it and see how it > > >

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 06:43:31PM -0400, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > > On

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 07:27:12PM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > Operationally, how would that work? We unlock them all on boot but > > somehow make them inaccessible to some backends after that? > > That could work and doesn't seem like an

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-08 Thread Bruce Momjian
On Mon, Jul 1, 2019 at 09:51:12AM -0400, Alvaro Herrera wrote: > Now that we have REINDEX CONCURRENTLY, I think reindexdb is going to > gain more popularity. > > Please don't reuse a file name as generic as "parallel.c" -- it's > annoying when navigating source. Maybe conn_parallel.c

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread James Coleman
Note: As I was writing this, I saw a new email come in from Tomas with a new patch series, and some similar observations. I'll look at that patch series more, but I think it's likely far more complete, so will end up going with that. I wanted to send this email anyway to at least capture the

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Bruce Momjian
On Fri, Jul 5, 2019 at 09:20:07PM +, PG Doc comments form wrote: > The following documentation comment has been logged on the website: > > Page: https://www.postgresql.org/docs/11/ddl-partitioning.html > Description: > > In the documentation for Postgres 11 table partitioning, there is no

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Tom Lane
Michael Paquier writes: > Attached is an idea of patch for the documentation, using this > wording: > + > + > + When defining a primary key on a partitioned table, the primary > + key column must be included in the partition key. > + > + Isn't it the other way

Re: [RFC] [PATCH] Flexible "partition pruning" hook

2019-07-08 Thread Mike Palmiotto
On Mon, Jul 8, 2019 at 10:59 AM Mike Palmiotto wrote: > > On Sun, Jul 7, 2019 at 9:46 PM Thomas Munro wrote: >> >> On Sat, Apr 6, 2019 at 3:06 PM Andres Freund wrote: >> > I've moved this to the next CF, and marked it as targeting v13. >> >> Hi Mike, >> >> Commifest 1 for PostgreSQL 13 is here.

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread Tom Lane
Bruce Momjian writes: > On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: >>> I fixed that, but I'm wondering if we should back-patch that fix >>> or leave the back branches alone. >> +0.5 for back-patching. > Uh, if this was done in a major release I am thinking we have to mention

Re: pgbench - add \aset to store results of a combined query

2019-07-08 Thread Ibrar Ahmed
> SELECT 1 AS one \; > SELECT 2 AS two UNION SELECT 2 \; > SELECT 3 AS three \aset > > will set both "one" and "three", while "two" is not set because there were > two rows. It is a kind of more permissive \gset. Are you sure two is not set :)? SELECT 2 AS two UNION SELECT 2; -- only

Re: warning to publication created and wal_level is not set to logical

2019-07-08 Thread Lucas Viecelli
Hi Thomas. Attached is the rebased > The July Commitfest has started. This patch is in "Needs review" > status, but it doesn't apply. If I read the above discussion > correctly, it seems there is agreement that a warning here is a good > idea to commit this patch. Could you please post a

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Ryan Lambert
Hey everyone, Here is my input regarding nonces and randomness. > As I understand it, the NIST recommendation is a 96-bit *random* nonce, I could not find that exact requirement in the NIST documents, though given the volume of that library it would be possible to miss. The recommendation I

Re: warning to publication created and wal_level is not set to logical

2019-07-08 Thread Lucas Viecelli
Follow the correct file, I added the wrong patch in the previous email > Attached is the rebased > thanks a lot *Lucas Viecelli* diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c index 1ac1a71bd9..902180cedc 100644 ---

Re: POC: Cleaning up orphaned files using undo logs

2019-07-08 Thread Robert Haas
On Mon, Jul 8, 2019 at 6:57 AM Amit Kapila wrote: > I didn't have other use cases in mind and I think to some extent this > argument holds true for existing binaryheap_allocate allocate. If we > want to make it more generic, then shouldn't we try to even change the > existing binaryheap_allocate

Re: [RFC] [PATCH] Flexible "partition pruning" hook

2019-07-08 Thread Mike Palmiotto
On Sun, Jul 7, 2019 at 9:46 PM Thomas Munro wrote: > On Sat, Apr 6, 2019 at 3:06 PM Andres Freund wrote: > > I've moved this to the next CF, and marked it as targeting v13. > > Hi Mike, > > Commifest 1 for PostgreSQL 13 is here. I was just going through the > automated build results for the

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 11:18:01AM -0400, Joe Conway wrote: > > On 7/8/19 10:19 AM, Bruce Momjian wrote: > > > When people are asking for multiple keys (not just for key rotation), > > > they are asking to have multiple keys that can be

Re: Switching PL/Python to Python 3 by default in PostgreSQL 12

2019-07-08 Thread Tom Lane
Peter Eisentraut writes: > I'm not trying to dismiss the importance of managing the Python > transition. But this issue has been known for many years, and the > current setup is more or less in line with the wider world. For > example, the Debian release that came out over the weekend still

Re: Switching PL/Python to Python 3 by default in PostgreSQL 12

2019-07-08 Thread Tom Lane
I wrote: > But I could support having a way for individual installations to change > what the synonym means locally. Perhaps we could think about how to do > that in conjunction with the project of getting rid of pg_pltemplate > that's been kicked around before [1][2][3]. ... actually, if we had

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Peter Eisentraut
On 2019-07-08 17:47, Stephen Frost wrote: > Of course, we can discuss if what websites do with over-the-wire > encryption is sensible to compare to what we want to do in PG for > data-at-rest, but then we shouldn't be talking about what websites do, > it'd make more sense to look at other

Re: Comment typo in tableam.h

2019-07-08 Thread Ashwin Agrawal
On Sat, Jul 6, 2019 at 12:05 AM Amit Kapila wrote: > On Tue, Jul 2, 2019 at 1:00 AM Ashwin Agrawal wrote: > > Please find attached v2 of patch 1 without objectionable comment change. > v1 of patch 2 attaching here just for convenience, no modifications made to > it. > > > > 0001* > * See

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Robert Haas
On Mon, Jul 8, 2019 at 11:08 AM Tom Lane wrote: > FWIW, I was imagining the action as being (1) detach all the child > partitions, (2) make parent into a non-partitioned table, (3) > drop the target column in each of these now-independent tables. > No data movement. Other than the need to

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread James Coleman
On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra wrote: > > On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote: > >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra > > wrote: > >> > >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote: > >> >On Sun, Jul 7, 2019 at 5:02 PM Tomas

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Joe Conway
On 7/8/19 11:56 AM, Peter Eisentraut wrote: > On 2019-07-08 17:47, Stephen Frost wrote: >> Of course, we can discuss if what websites do with over-the-wire >> encryption is sensible to compare to what we want to do in PG for >> data-at-rest, but then we shouldn't be talking about what websites do,

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 11:47:33AM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jul 8, 2019 at 11:18:01AM -0400, Joe Conway wrote: > > > On 7/8/19 10:19 AM, Bruce Momjian wrote: > > > > When people are asking for multiple keys (not just for

Re: errbacktrace

2019-07-08 Thread Alvaro Herrera
On 2019-Jul-08, Dmitry Dolgov wrote: > > On Sat, Jun 29, 2019 at 7:41 AM Jaime Casanova > > wrote: > > > > This is certainly a very useful thing. Sadly, it doesn't seem to compile > > when > > trying to use libunwind. > > Yeah, the same for me. To make it works I've restricted libunwind to

Re: range_agg

2019-07-08 Thread Paul A Jungwirth
On Sat, Jul 6, 2019 at 12:13 PM Jeff Davis wrote: > > On Fri, 2019-07-05 at 09:58 -0700, Paul A Jungwirth wrote: > > user-defined range types. So how about I start on it and see how it > > goes? I expect I can follow the existing code for range types pretty > > closely, so maybe it won't be too

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Mon, Jul 8, 2019 at 5:53 PM Michael Paquier wrote: > I have begun playing with regressplans.sh which enforces various > combinations of "-f s|i|n|m|h" when running the regression tests, and > I have noticed that -fh can cause the server to become stuck in the > test join_hash.sql with this

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Michael Paquier writes: > I have begun playing with regressplans.sh which enforces various > combinations of "-f s|i|n|m|h" when running the regression tests, and > I have noticed that -fh can cause the server to become stuck in the > test join_hash.sql with this query (not sure which portion of

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 06:49:44PM +1200, Thomas Munro wrote: > "simple" has 20k rows and "extremely_skewed" has 20k rows but the > planner thinks it only has 2. So this going to take O(n^2) time and n > is 20k. Not sure what to do about that. Maybe "join_hash" should be > skipped for the -h (=

Re: (select query)/relation as first class citizen

2019-07-08 Thread Roman Pekar
Hi, what do you think about this idea in general? If you don't have to think about implementation for now? From my point of view writing Sql queries is very close to how functional language work if you treat "select" queries as functions without side-effects, and having query being

Re: Ltree syntax improvement

2019-07-08 Thread Thomas Munro
On Wed, Apr 17, 2019 at 5:29 AM Dmitry Belyavsky wrote: > I've applied your patch. > From my point of view, there is no major difference between case and chain if > here. > Neither case nor ifs allow extracting the common code to separate function - > just because there seem to be no identical

Re: Replication & recovery_min_apply_delay

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 07:56:25PM +1200, Thomas Munro wrote: > On Thu, Jan 31, 2019 at 3:34 AM Alvaro Herrera > wrote: > > On 2019-Jan-30, Konstantin Knizhnik wrote: > > > I wonder if it can be considered as acceptable solution of the problem or > > > there can be some better approach? > > > >

Re: Implementing Incremental View Maintenance

2019-07-08 Thread Tatsuo Ishii
> As for how to make internal columns invisible to SELECT *, previously > there have been discussions about doing that using a new flag in > pg_attribute: > > https://www.postgresql.org/message-id/flat/CAEepm%3D3ZHh%3Dp0nEEnVbs1Dig_UShPzHUcMNAqvDQUgYgcDo-pA%40mail.gmail.com Now that I realized

Re: Add test case for sslinfo

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 02:11:34PM +0800, Hao Wu wrote: > Thank you for your quick response! I work on greenplum, and I didn't see > this folder(src/test/ssl/ssl) before. > I will add more certificates to test and resend again. Not having duplicates would be nice. > Do you have any suggestion

Re: Add test case for sslinfo

2019-07-08 Thread Hao Wu
Hi Thomas, Thank you for your quick response! I work on greenplum, and I didn't see this folder(src/test/ssl/ssl) before. I will add more certificates to test and resend again. Do you have any suggestion about the missing PGDATA? Since the test needs to configure postgresql.conf, maybe there are

Re: Implementing Incremental View Maintenance

2019-07-08 Thread Thomas Munro
On Fri, Jun 28, 2019 at 10:56 PM Yugo Nagata wrote: > Attached is a WIP patch of IVM which supports some aggregate functions. Hi Nagata-san and Hoshiai-san, Thank you for working on this. I enjoyed your talk at PGCon. I've added Kevin Grittner just in case he missed this thread; he has talked

Re: Data-only pg_rewind, take 2

2019-07-08 Thread Thomas Munro
On Mon, Mar 18, 2019 at 8:46 PM Chris Travers wrote: > On Mon, Mar 18, 2019 at 4:09 AM Michael Paquier wrote: >> On Sun, Mar 17, 2019 at 09:00:57PM +0800, Chris Travers wrote: >> > I also added test cases and some docs. I don't know if the docs are >> > sufficient. Feedback is appreciated. >>

Re: FETCH FIRST clause PERCENT option

2019-07-08 Thread Surafel Temesgen
Hi Thomas, Thank you for informing me Hi Surafel, > > There's a call to adjust_limit_rows_costs() hiding under > contrib/postgres_fdw, so this fails check-world. > Fixed . I also include the review given by Ryan in attached patch regards Surafel diff --git a/contrib/postgres_fdw/postgres_fdw.c

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-08 Thread Michael Paquier
On Fri, Jul 05, 2019 at 07:25:41PM +0200, Julien Rouhaud wrote: > On Fri, Jul 5, 2019 at 6:16 PM Peter Eisentraut > wrote: >> Isn't that also the case for your proposal? We are not going to release >> a new reindexdb before a new REINDEX. > > Sure, but my point was that once the new reindexdb

Re: [PATCH] kNN for btree

2019-07-08 Thread Thomas Munro
On Tue, Jul 2, 2019 at 5:47 AM Nikita Glukhov wrote: > Attached 12th version of the patches rebased onto the current master. Hi Nikita, make check-world fails for me, and in tmp_install/log/install.log I see: btree_int2.c:97:9: error: implicit declaration of function 'int2dist' is invalid in

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Amit Langote
On Mon, Jul 8, 2019 at 4:11 AM Tom Lane wrote: > (Moved from pgsql-bugs thread at [1]) Thanks. > Consider > > regression=# create domain d1 as int; > CREATE DOMAIN > regression=# create table t1 (f1 d1) partition by range(f1); > CREATE TABLE > regression=# alter table t1 drop column f1; >

Re: Replication & recovery_min_apply_delay

2019-07-08 Thread Thomas Munro
On Thu, Jan 31, 2019 at 3:34 AM Alvaro Herrera wrote: > On 2019-Jan-30, Konstantin Knizhnik wrote: > > I wonder if it can be considered as acceptable solution of the problem or > > there can be some better approach? > > I didn't find one. It sounds like you are in agreement that there is a

Re: Duplicated LSN in ReorderBuffer

2019-07-08 Thread Masahiko Sawada
On Mon, Jul 8, 2019 at 9:00 AM Thomas Munro wrote: > > On Wed, Jun 26, 2019 at 2:46 AM Ildar Musin wrote: > > Attached is a simple patch that uses subxid instead of top-level xid > > in ReorderBufferAddNewTupleCids() call. It seems to fix the bug, but > > i'm not sure that this is a valid

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread Tomas Vondra
On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote: On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra wrote: On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote: >On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra > wrote: >> We're running query like this: >> >> SELECT a, sum(b),

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 06:04:28PM +0900, Masahiko Sawada wrote: > On Sun, Jul 7, 2019 at 1:05 AM Bruce Momjian wrote: > > What about referential integrity constraints that need to check primary > > keys in the encrypted tables? I also don't see a way of delaying that, > > and if you can't do

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Robert Haas
On Mon, Jul 8, 2019 at 11:02 AM Robert Haas wrote: > On Mon, Jul 8, 2019 at 10:32 AM Alvaro Herrera > wrote: > > That said, I'm not sure I see the use case for an ALTER TABLE .. DROP > > COLUMN command that turns a partitioned table (with existing partitions > > containing data) into one

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Tom Lane
Robert Haas writes: > On Mon, Jul 8, 2019 at 11:02 AM Robert Haas wrote: >> On Mon, Jul 8, 2019 at 10:32 AM Alvaro Herrera >> wrote: >>> That said, I'm not sure I see the use case for an ALTER TABLE .. DROP >>> COLUMN command that turns a partitioned table (with existing partitions >>>

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Joe Conway
On 7/8/19 10:19 AM, Bruce Momjian wrote: > When people are asking for multiple keys (not just for key rotation), > they are asking to have multiple keys that can be supplied by users only > when they need to access the data. Yes, the keys are always in the > datbase, but the feature request is

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread James Coleman
On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra wrote: > We're running query like this: > > SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 > ORDER BY 1, 2, 3 > > but we're trying to add the incremental sort *before* the aggregation, > because the optimizer also considers

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread Tomas Vondra
On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote: On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra wrote: We're running query like this: SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3 but we're trying to add the incremental sort *before*

Re: allow_system_table_mods stuff

2019-07-08 Thread Bruce Momjian
On Sun, Jul 7, 2019 at 11:45:49PM -0400, Bruce Momjian wrote: > On Mon, Jun 24, 2019 at 11:20:51AM -0400, Tom Lane wrote: > > I do see value in two switches not one, but it's what I said above, > > to not need to give people *more* chance-to-break-things than they > > had before when doing manual

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread James Coleman
On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra wrote: > > On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote: > >On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra > > wrote: > >> We're running query like this: > >> > >> SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Alvaro Herrera
On 2019-Jul-07, Tom Lane wrote: > Ideally, perhaps, a DROP CASCADE like this would not cascade to > the whole table but only to the table's partitioned-ness property, > leaving you with a non-partitioned table with most of its data > intact. It would take a lot of work to make that happen

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Robert Haas
On Mon, Jul 8, 2019 at 10:32 AM Alvaro Herrera wrote: > That said, I'm not sure I see the use case for an ALTER TABLE .. DROP > COLUMN command that turns a partitioned table (with existing partitions > containing data) into one non-partitioned table with all data minus the > partitioning

Re: Add missing operator <->(box, point)

2019-07-08 Thread Alexander Korotkov
On Mon, Mar 11, 2019 at 2:49 AM Nikita Glukhov wrote: > 2. Add <-> to GiST box_ops. >Extracted gist_box_distance_helper() common for gist_box_distance() and >gist_bbox_distance(). For me it doesn't look worth having two distinct functions gist_box_distance_helper() and

Re: errbacktrace

2019-07-08 Thread Dmitry Dolgov
> On Sat, Jun 29, 2019 at 7:41 AM Jaime Casanova > wrote: > > This is certainly a very useful thing. Sadly, it doesn't seem to compile when > trying to use libunwind. Yeah, the same for me. To make it works I've restricted libunwind to local unwinding only: #ifdef USE_LIBUNWIND #define

Re: Excessive memory usage in multi-statement queries w/ partitioning

2019-07-08 Thread Julien Rouhaud
Hi Amit, On Mon, Jul 8, 2019 at 10:52 AM Amit Langote wrote: > > On Thu, Jul 4, 2019 at 6:52 PM Julien Rouhaud wrote: > > On Tue, May 28, 2019 at 6:57 AM Amit Langote wrote: > > > >> Maybe like the attached? I'm not sure if we need to likewise be > > > >> concerned > > > >> about

Re: Broken defenses against dropping a partitioning column

2019-07-08 Thread Tom Lane
Alvaro Herrera writes: > That said, I'm not sure I see the use case for an ALTER TABLE .. DROP > COLUMN command that turns a partitioned table (with existing partitions > containing data) into one non-partitioned table with all data minus the > partitioning column(s). Yeah, it'd be a lot of work

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 11:18:01AM -0400, Joe Conway wrote: > On 7/8/19 10:19 AM, Bruce Momjian wrote: > > When people are asking for multiple keys (not just for key rotation), > > they are asking to have multiple keys that can be supplied by users only > > when they need to access the data. Yes,

Re: tableam vs. TOAST

2019-07-08 Thread Robert Haas
On Tue, Jun 25, 2019 at 2:19 AM Prabhat Sahu wrote: > I have tested the TOAST patches(v3) with different storage options like(MAIN, > EXTERNAL, EXTENDED, etc.), and > combinations of compression and out-of-line storage options. > I have used a few dummy tables with various tuple count say 10k,

RE: [PATCH] memory leak in ecpglib

2019-07-08 Thread kuroda.hay...@fujitsu.com
Dear Meskes, Zhang, I think this modification is not enough, and I have an another idea. >> If your patch is committed, in your example, any operation for cur1 will not >> be accepted. > Although the return value after calling ecpg_get_con_name_by_cursor_name(cur1) > is NULL, in

Re: (select query)/relation as first class citizen

2019-07-08 Thread Pavel Stehule
Hi po 8. 7. 2019 v 9:33 odesílatel Roman Pekar napsal: > Hi, > > what do you think about this idea in general? If you don't have to think > about implementation for now? From my point of view writing Sql queries is > very close to how functional language work if you treat "select" queries as >

Re: progress report for ANALYZE

2019-07-08 Thread Tatsuro Yamada
Hi Alvaro, I'll review your patch in this week. :) I tested your patch on 6b854896. Here is a result. See below: - [Session #1] create table hoge as select * from generate_series(1, 100) a; analyze verbose hoge; [Session #2] \a \t

Re: Re: Reviving the "Stopping logical replication protocol" patch from Vladimir Gordichuk

2019-07-08 Thread Thomas Munro
On Thu, Mar 7, 2019 at 8:19 PM David Steele wrote: > On 2/16/19 10:38 PM, Dave Cramer wrote: > > Thanks for looking at this. FYI, I did not originally write this, rather > > the original author has not replied to requests. > > JDBC could use this, I assume others could as well. > > > > That said

Re: dropdb --force

2019-07-08 Thread Pavel Stehule
po 8. 7. 2019 v 0:07 odesílatel Thomas Munro napsal: > On Thu, Jun 27, 2019 at 7:15 AM Pavel Stehule > wrote: > > fixed > > Hi Pavel, > > FYI t/050_dropdb.pl fails consistently with this patch applied: > > https://travis-ci.org/postgresql-cfbot/postgresql/builds/555234838 with attached patch

Re: Excessive memory usage in multi-statement queries w/ partitioning

2019-07-08 Thread Amit Langote
Hi Julien, Thanks for taking a look at this. On Thu, Jul 4, 2019 at 6:52 PM Julien Rouhaud wrote: > On Tue, May 28, 2019 at 6:57 AM Amit Langote wrote: > > >> Maybe like the attached? I'm not sure if we need to likewise be > > >> concerned > > >> about exec_sql_string() being handed

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Masahiko Sawada
On Sun, Jul 7, 2019 at 1:05 AM Bruce Momjian wrote: > > On Fri, Jul 5, 2019 at 10:24:39PM +0200, Tomas Vondra wrote: > > I agree this is a pretty crucial challenge, and those requirements seem > > in direct conflict. Users use encryption to protect privacy of the data, > > but we need access to

  1   2   >