Re: dropping column prevented due to inherited index

2019-10-14 Thread Michael Paquier
On Sat, Oct 12, 2019 at 03:08:41PM +0900, Michael Paquier wrote: > On Fri, Oct 11, 2019 at 06:39:47PM -0300, Alvaro Herrera wrote: >> Typo "resursing". This comment seems a bit too long to me. Maybe >> "Recursion having ended, drop everything that was collected." suffices. >> (Fits in one line.)

Re: Columns correlation and adaptive query optimization

2019-10-14 Thread legrand legrand
Hello Konstantin, What you have proposed regarding join_selectivity and multicolumn statistics is a very good new ! Regarding your auto_explain modification, maybe an "advisor" mode would also be helpfull (with auto_explain_add_statistics_threshold=-1 for exemple). This would allow to track

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-14 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 03:10:21PM -0300, Alvaro Herrera wrote: > On 2019-Oct-13, Justin Pryzby wrote: > > > Looks like it's a race condition and dereferencing *holder=NULL. The first > > crash was probably the same bug, due to report query running during "reindex > > CONCURRENTLY", and probably

Re: "pg_ctl: the PID file ... is empty" at end of make check

2019-10-14 Thread Tom Lane
[ blast from the past dept. ] Thomas Munro writes: > On Thu, Nov 29, 2018 at 3:30 AM Tom Lane wrote: >> Thomas Munro writes: >>> https://github.com/macdice/unlinktest >> Bleah. But you can do better than ask whether it's a bug: you can >> quote POSIX: >> ... >> Not a lot of wiggle room

Re: v12.0: ERROR: could not find pathkey item to sort

2019-10-14 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 01:30:29PM -0500, Justin Pryzby wrote: > BTW it probably should've been documented as an "Open Item" for v12. https://commitfest.postgresql.org/25/2278/ I realized possibly people were thinking of that as a "feature" and not a bugfix for backpatch (?) But, my issue is a

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-14 Thread Alvaro Herrera
On 2019-Oct-13, Justin Pryzby wrote: > Looks like it's a race condition and dereferencing *holder=NULL. The first > crash was probably the same bug, due to report query running during "reindex > CONCURRENTLY", and probably finished at nearly the same time as another > locker. Ooh, right, makes

Re: Fix most -Wundef warnings

2019-10-14 Thread Mark Dilger
On 10/13/19 12:25 PM, Peter Eisentraut wrote: diff --git a/contrib/hstore/hstore_compat.c b/contrib/hstore/hstore_compat.c index 1d4e7484e4..d75e9cb23f 100644 --- a/contrib/hstore/hstore_compat.c +++ b/contrib/hstore/hstore_compat.c @@ -299,7 +299,7 @@ hstoreUpgrade(Datum orig) if

tuplesort test coverage

2019-10-14 Thread Andres Freund
Hi, [1] made me look at tuplesorts test coverage at https://coverage.postgresql.org/src/backend/utils/sort/tuplesort.c.gcov.html We don't have coverage for a quite a number of things: - cluster for expression indexes (line 935) - sorts exceeding INT_MAX / 2 memory (line 1337), but that seems hard

Re: BRIN index which is much faster never chosen by planner

2019-10-14 Thread David Rowley
On Tue, 15 Oct 2019 at 08:43, Jeremy Finzel wrote: > I wanted to follow up on this specific issue. Isn't this the heart of the > matter and a fundamental problem? If I want to rely on BRIN indexes as in a > straightforward case as explained in OP, but I don't know if the planner will > be

Re: stress test for parallel workers

2019-10-14 Thread Tom Lane
I wrote: > Filed at > https://bugzilla.kernel.org/show_bug.cgi?id=205183 > We'll see what happens ... Further to this --- I went back and looked at the outlier events where we saw an infinite_recurse failure on a non-Linux-PPC64 platform. There were only three: mereswine| ARMv7

Re: v12.0: ERROR: could not find pathkey item to sort

2019-10-14 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 02:06:02PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Fri, Oct 11, 2019 at 10:48:37AM -0400, Tom Lane wrote: > >> Could you provide a self-contained test case please? > > > [ test case ] > > Oh, this is the same issue Amit described in > >

Re: Collation versioning

2019-10-14 Thread Thomas Munro
On Fri, Oct 11, 2019 at 11:41 PM Thomas Munro wrote: > On Thu, Oct 10, 2019 at 8:38 AM Peter Eisentraut > wrote: > > Actually, I had to revert that because pg_dump and pg_upgrade tests need > > to be updated, but that seems doable. > > [Returning from a couple of weeks mostly away from

Re: "pg_ctl: the PID file ... is empty" at end of make check

2019-10-14 Thread Thomas Munro
On Tue, Oct 15, 2019 at 1:55 PM Tom Lane wrote: > Thomas Munro writes: > > Agreed. Secret non-shareable bug report filed. Fingers crossed. > > Since that conversation, longfin has shown the same symptom > just once more: > > longfin | REL_11_STABLE | 2019-07-28 22:29:03 | recoveryCheck |

Re: CREATE TEXT SEARCH DICTIONARY segfaulting on 9.6+

2019-10-14 Thread Tomas Vondra
I spent a bit of time investigating this, and it seems the new code is somewhat too trusting when it comes to data from the affix/dict files. In this particular case, it boils down to this code in NISortDictionary: if (Conf->useFlagAliases) { for (i = 0; i < Conf->nspell; i++)

Re: [HACKERS] Block level parallel vacuum

2019-10-14 Thread Masahiko Sawada
On Mon, Oct 14, 2019 at 6:37 PM Amit Kapila wrote: > > On Sat, Oct 12, 2019 at 4:50 PM Amit Kapila wrote: > > On Sat, Oct 12, 2019 at 11:29 AM Masahiko Sawada > > wrote: > > > > > > > I see a much bigger problem with the way this patch collects the index > > stats in shared memory. IIUC, it

ProcArrayGroupClearXid() compare-exchange style

2019-10-14 Thread Noah Misch
ProcArrayGroupClearXid() has this: while (true) { nextidx = pg_atomic_read_u32(>procArrayGroupFirst); ... if (pg_atomic_compare_exchange_u32(>procArrayGroupFirst,

Re: Transparent Data Encryption (TDE) and encrypted files

2019-10-14 Thread Masahiko Sawada
On Mon, Oct 14, 2019 at 3:42 PM Antonin Houska wrote: > > Masahiko Sawada wrote: > > > On Wed, Oct 9, 2019 at 3:57 PM Antonin Houska wrote: > > > > > > Moon, Insung wrote: > > > > > > v04-0011-Make-buffile.c-aware-of-encryption.patch in [1] changes > > > buffile.c so > > > that data is read

Re: Fix most -Wundef warnings

2019-10-14 Thread Tom Lane
Peter Eisentraut writes: > During the cleanup of the _MSC_VER versions (commit > 38d8dce61fff09daae0edb6bcdd42b0c7f10ebcd), I found it useful to use > -Wundef, but that resulted in a bunch of gratuitous warnings. Here is a > patch to fix those. Most of these are just stylistic cleanups, but the

Re: Tighten error control for OpenTransientFile/CloseTransientFile

2019-10-14 Thread Michael Paquier
On Fri, Oct 04, 2019 at 01:39:38PM -0700, Andres Freund wrote: > but that reasoning seems bogus to me. For one, on just about any > platform close always closes the fd, even when returning an error > (unless you pass in a bad fd, in which case it obviously doesn't). So > the reasoning that this

Re: [HACKERS] Block level parallel vacuum

2019-10-14 Thread Amit Kapila
On Sat, Oct 12, 2019 at 4:50 PM Amit Kapila wrote: > On Sat, Oct 12, 2019 at 11:29 AM Masahiko Sawada > wrote: > > > > I see a much bigger problem with the way this patch collects the index > stats in shared memory. IIUC, it allocates the shared memory (DSM) > for all the index stats, in the

Re: Minimal logical decoding on standbys

2019-10-14 Thread nil socket
Sorry to intervene in between, But what about timeline change? Thank you.

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-10-14 Thread Dilip Kumar
On Thu, Oct 3, 2019 at 4:03 AM Tomas Vondra wrote: > > On Wed, Oct 02, 2019 at 04:27:30AM +0530, Amit Kapila wrote: > >On Tue, Oct 1, 2019 at 7:21 PM Tomas Vondra > >wrote: > > > >> On Tue, Oct 01, 2019 at 06:55:52PM +0530, Amit Kapila wrote: > >> > > >> >On further testing, I found that the

Re: Transparent Data Encryption (TDE) and encrypted files

2019-10-14 Thread Antonin Houska
Masahiko Sawada wrote: > On Wed, Oct 9, 2019 at 3:57 PM Antonin Houska wrote: > > > > Moon, Insung wrote: > > > > v04-0011-Make-buffile.c-aware-of-encryption.patch in [1] changes buffile.c > > so > > that data is read and written in 8kB blocks if encryption is enabled. In > > order > > to

Remove obsolete information schema tables

2019-10-14 Thread Peter Eisentraut
I propose this patch to remove the information schema tables SQL_LANGUAGES, which was eliminated in SQL:2008, and SQL_PACKAGES, which was eliminated in SQL:2011. Since they were dropped by the SQL standard, the information in them was no longer updated and therefore no longer useful. This also

Re: v12.0: ERROR: could not find pathkey item to sort

2019-10-14 Thread Tom Lane
Justin Pryzby writes: > On Sun, Oct 13, 2019 at 01:30:29PM -0500, Justin Pryzby wrote: >> BTW it probably should've been documented as an "Open Item" for v12. > https://commitfest.postgresql.org/25/2278/ > I realized possibly people were thinking of that as a "feature" and not a > bugfix for

Re: Non-Active links being referred in our source code

2019-10-14 Thread vignesh C
On Tue, Oct 8, 2019 at 10:35 AM Michael Paquier wrote: > > On Mon, Oct 07, 2019 at 05:11:40PM +0200, Juan José Santamaría Flecha wrote: > > About the broken links in win32_port.h, they are all referring to > > ntstatus. As for first case that shows the code groups, there is an up > > to date

Re: d25ea01275 and partitionwise join

2019-10-14 Thread Justin Pryzby
On Thu, Sep 19, 2019 at 05:15:37PM +0900, Amit Langote wrote: > Please find attached updated patches. Tom pointed me to this thread, since we hit it in 12.0 https://www.postgresql.org/message-id/flat/16802.1570989962%40sss.pgh.pa.us#070f6675a11dff17760b1cfccf1c038d I can't say much about the

Re: BRIN index which is much faster never chosen by planner

2019-10-14 Thread Jeremy Finzel
> > The other issue is that the estimation of pages fetched using bitmap > heap scan is rather crude - but that's simply hard, and I don't think we > can fundamentally improve this. > I wanted to follow up on this specific issue. Isn't this the heart of the matter and a fundamental problem? If

Add A Glossary

2019-10-14 Thread Corey Huinker
Attached is a v1 patch to add a Glossary to the appendix of our current documentation. I believe that our documentation needs a glossary for a few reasons: 1. It's hard to ask for help if you don't know the proper terminology of the problem you're having. 2. Readers who are new to databases may

Re: [HACKERS] Block level parallel vacuum

2019-10-14 Thread Dilip Kumar
On Mon, Oct 14, 2019 at 3:10 PM Amit Kapila wrote: > > On Sat, Oct 12, 2019 at 4:50 PM Amit Kapila wrote: > > On Sat, Oct 12, 2019 at 11:29 AM Masahiko Sawada > > wrote: > > > > > > > I see a much bigger problem with the way this patch collects the index > > stats in shared memory. IIUC, it

Re: stress test for parallel workers

2019-10-14 Thread Tom Lane
Andres Freund writes: > Probably requires reproducing on a pretty recent kernel first, to have a > decent chance of being investigated... How recent do you think it needs to be? The machine I was testing on yesterday is under a year old: uname -m = ppc64le uname -r = 4.18.19-100.fc27.ppc64le

Columns correlation and adaptive query optimization

2019-10-14 Thread Konstantin Knizhnik
Hi hackers, Errors in selectivity estimations is one of the main reason of bad plans generation by Postgres optimizer. Postgres estimates selectivity based on the collected statistic (histograms). While it is able to more or less precisely estimated selectivity of simple predicate for

Re: v12.0: ERROR: could not find pathkey item to sort

2019-10-14 Thread Tom Lane
Justin Pryzby writes: > On Fri, Oct 11, 2019 at 10:48:37AM -0400, Tom Lane wrote: >> Could you provide a self-contained test case please? > [ test case ] Oh, this is the same issue Amit described in

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-14 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 06:06:43PM +0900, Michael Paquier wrote: > On Fri, Oct 11, 2019 at 07:44:46PM -0500, Justin Pryzby wrote: > > Unfortunately, there was no core file, and I'm still trying to reproduce it. > > Forgot to set ulimit -c? Having a backtrace would surely help. Fortunately (?)

Re: stress test for parallel workers

2019-10-14 Thread Andres Freund
Hi, On 2019-10-13 13:44:59 +1300, Thomas Munro wrote: > On Sun, Oct 13, 2019 at 1:06 PM Tom Lane wrote: > > I don't think any further proof is required that this is > > a kernel bug. Where would be a good place to file it? > > linuxppc-...@lists.ozlabs.org might be the right place. > >