missing estimation for coalesce function

2019-11-26 Thread Pavel Stehule
Hi I have a report from my customer about migration his application from Oracle to Postgres. The most significant issue was missing correct estimation for coalesce function. He had to rewrite coalesce(var, X) = X to "var IS NULL or var = X". Then the result was very satisfactory. Example:

Re: Problem while updating a foreign table pointing to a partitioned table on foreign server

2019-11-26 Thread Etsuro Fujita
On Tue, Nov 26, 2019 at 12:37 PM Etsuro Fujita wrote: > I was planning to work on this in this commitfest, but sorry, I didn't > have time due to other priorities. Probably, I won't have time for > this in the development cycle for v13. So I'll mark this as RWF, > unless anyone wants to work on

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-11-26 Thread Justin Pryzby
This same crash occured on a 2nd server. Also qemu/KVM, but this time on a 2ndary ZFS tablespaces which (fails to) include the missing relfilenode. Linux database7 3.10.0-957.10.1.el7.x86_64 #1 SMP Mon Mar 18 15:06:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux This is

Re: psql - improve test coverage from 41% to 88%

2019-11-26 Thread Michael Paquier
Hi Fabien, On Tue, Sep 17, 2019 at 04:48:06PM +0530, vignesh C wrote: > Few comments: > + [ 'START TRANSACTION', [ qr{ISOLATION LEVEL}, qr{(?!BEGIN)} ] ], > + [ 'TABLE', [ qr{ONLY} ] ], # hmmm... > + [ 'TRUNCATE', [ qr{CONTINUE IDENTITY} ] ], > + [ 'UNLISTEN', [ ] ], > > We can remove # hmmm...

Re: dropdb --force

2019-11-26 Thread vignesh C
On Tue, Nov 26, 2019 at 11:37 AM Amit Kapila wrote: > > On Mon, Nov 25, 2019 at 11:22 PM vignesh C wrote: > > > > On Sun, Nov 24, 2019 at 5:06 PM Pavel Stehule > > wrote: > > > > > > > > > > > > ne 24. 11. 2019 v 11:25 odesílatel vignesh C napsal: > > >> > > >> On Sat, Nov 23, 2019 at 4:42 PM

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-11-26 Thread Michael Paquier
On Wed, Nov 27, 2019 at 12:54:16PM +0900, Michael Paquier wrote: > It would be nice to add tab completion for this new clause in psql. > This is not ready for committer yet in my opinion, and more work is > done, so I am marking it as returned with feedback for now. And I have somewhat missed to

Re: progress report for ANALYZE

2019-11-26 Thread Michael Paquier
On Wed, Nov 27, 2019 at 12:45:41PM +0900, Tatsuro Yamada wrote: > Fixed. Patch was waiting on input from author, so I have switched it back to "Needs review", and moved it to next CF while on it as you are working on it. -- Michael signature.asc Description: PGP signature

Re: pg_upgrade fails with non-standard ACL

2019-11-26 Thread Michael Paquier
On Wed, Nov 27, 2019 at 11:35:14AM +0900, Artur Zakirov wrote: > I've started to implement new backend function similar to > pg_describe_object() and tried to make new version of the patch. But I'm > wondering now if it is possible to backpatch new functions to older > Postgres releases?

Re: Remove page-read callback from XLogReaderState.

2019-11-26 Thread Alvaro Herrera
On 2019-Nov-27, Kyotaro Horiguchi wrote: > At Thu, 24 Oct 2019 14:51:01 +0900 (JST), Kyotaro Horiguchi > wrote in > > Rebased. > > 0dc8ead463 hit this. Rebased. Please review the pg_waldump.c hunks in 0001; they revert recent changes. -- Álvaro Herrera

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-11-26 Thread Michael Paquier
On Wed, Nov 27, 2019 at 12:54:16PM +0900, Michael Paquier wrote: > + /* Skip all mapped relations if TABLESPACE is specified */ > + if (OidIsValid(tableSpaceOid) && > + classtuple->relfilenode == 0) > + { > + if (!system_warning) > +

Re: Remove page-read callback from XLogReaderState.

2019-11-26 Thread Michael Paquier
On Wed, Nov 27, 2019 at 12:09:23PM +0900, Kyotaro Horiguchi wrote: > 0dc8ead463 hit this. Rebased. Note: Moved to next CF. -- Michael signature.asc Description: PGP signature

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-11-26 Thread Michael Paquier
On Tue, Nov 26, 2019 at 11:09:55PM +0100, Masahiko Sawada wrote: > Thank you for working on this. I have been looking at the latest patch as well. > I looked at v4 patch. Here are some comments: > > + /* Skip all mapped relations if TABLESPACE is specified */ > + if

Re: progress report for ANALYZE

2019-11-26 Thread Tatsuro Yamada
Hi Amit-san! I think include_children and current_relid are not enough to understand the progress of analyzing inheritance trees, because even with current_relid being updated, I can't tell how many more there will be. I think it'd be better to show the total number of children and the number

Re: progress report for ANALYZE

2019-11-26 Thread Tatsuro Yamada
Hi Amit-san, On Wed, Nov 27, 2019 at 11:04 AM Tatsuro Yamada wrote: Regarding to other total number columns, I'll create another patch to add these columns "index_vacuum_total" and "index_rebuild_count" on the other views. :) Maybe you meant "index_rebuild_total"? Yeah, you are right! :)

Re: Remove page-read callback from XLogReaderState.

2019-11-26 Thread Kyotaro Horiguchi
At Thu, 24 Oct 2019 14:51:01 +0900 (JST), Kyotaro Horiguchi wrote in > Rebased. 0dc8ead463 hit this. Rebased. regards. -- Kyotaro Horiguchi NTT Open Source Software Center >From d9007aee88f7400b0f03ced1b80584964a1b0b79 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Thu, 5 Sep 2019

Re: ERROR: attribute number 6 exceeds number of columns 5

2019-11-26 Thread Kyotaro Horiguchi
At Tue, 26 Nov 2019 10:49:11 -0500, Tom Lane wrote in > Andreas Joseph Krogh writes: > > Run the attached script and you'll get: > > > psql -f error.sql -d test > > psql:error.sql:37: ERROR: attribute number 6 exceeds number of columns 5 > > Hmm, interesting. IMO, that *should* have thrown

Re: progress report for ANALYZE

2019-11-26 Thread Alvaro Herrera
On 2019-Nov-27, Amit Langote wrote: > On Tue, Nov 26, 2019 at 9:22 PM Alvaro Herrera > wrote: > > > > On 2019-Nov-26, Tatsuro Yamada wrote: > > > > > > I wonder whether we need the total number of ext stats on > > > > pg_stat_progress_analyze or not. As you might know, there is the same > > > >

Re: [HACKERS] Block level parallel vacuum

2019-11-26 Thread Amit Kapila
On Wed, Nov 27, 2019 at 12:52 AM Masahiko Sawada wrote: > > > I've incorporated the comments I got so far including the above and > the memory alignment issue. > Thanks, I will look into the new version. BTW, why haven't you posted 0001 patch (IndexAM API's patch)? I think without that we need

Re: progress report for ANALYZE

2019-11-26 Thread Amit Langote
Yamada-san, On Wed, Nov 27, 2019 at 11:04 AM Tatsuro Yamada wrote: > Regarding to other total number columns, > I'll create another patch to add these columns "index_vacuum_total" and > "index_rebuild_count" on the other views. :) Maybe you meant "index_rebuild_total"? Thanks, Amit

Re: pg_upgrade fails with non-standard ACL

2019-11-26 Thread Artur Zakirov
Thank you for reviews! On 2019/11/21 17:53, Michael Paquier wrote: On Fri, Nov 15, 2019 at 11:30:02AM +0300, Grigory Smolkin wrote: On 11/9/19 5:26 AM, Michael Paquier wrote: Another question I have: do we need to care more about other extra ACLs applied to other object types? For example a

Re: progress report for ANALYZE

2019-11-26 Thread Amit Langote
On Tue, Nov 26, 2019 at 9:22 PM Alvaro Herrera wrote: > > On 2019-Nov-26, Tatsuro Yamada wrote: > > > > I wonder whether we need the total number of ext stats on > > > pg_stat_progress_analyze or not. As you might know, there is the same > > > counter on pg_stat_progress_vacuum and

Re: SegFault on 9.6.14

2019-11-26 Thread Amit Kapila
On Wed, Nov 20, 2019 at 5:12 PM Amit Kapila wrote: > > On Mon, Nov 18, 2019 at 2:22 PM Amit Kapila wrote: > > > > I have modified the commit message as proposed above and additionally > > added comments in nodeLimit.c. I think we should move ahead with this > > bug-fix patch. If we don't like

Re: progress report for ANALYZE

2019-11-26 Thread Tatsuro Yamada
Hi Alvaro! On 2019/11/26 21:22, Alvaro Herrera wrote: On 2019-Nov-26, Tatsuro Yamada wrote: I wonder whether we need the total number of ext stats on pg_stat_progress_analyze or not. As you might know, there is the same counter on pg_stat_progress_vacuum and pg_stat_progress_cluster. For

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

2019-11-26 Thread Amit Langote
On Wed, Nov 27, 2019 at 3:25 AM Tom Lane wrote: > Amit Langote writes: > > If inh_root_relid meant that, it would no longer be useful to > > examine_variable. In examine_variable, we need to map a child table's > > relid to the relid of its root parent table. If the root parent > > itself is

Re: [HACKERS] Regression tests vs existing users in an installation

2019-11-26 Thread Tom Lane
Peter Eisentraut writes: > I found this directory, and it seems like a good place to add some more > tests (other thread), but I'm puzzled why it's hidden under modules/, > since it's not, well, a module. Why is it not in src/test/? Doing it like this allowed it to be picked up automatically

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-11-26 Thread Masahiko Sawada
On Wed, 20 Nov 2019 at 19:16, Alexey Kondratov wrote: > > Hi Steve, > > Thank you for review. > > On 17.11.2019 3:53, Steve Singer wrote: > > The following review has been posted through the commitfest application: > > make installcheck-world: tested, passed > > Implements feature: tested,

Re: Collation versions on Windows (help wanted, apply within)

2019-11-26 Thread Peter Eisentraut
On 2019-11-26 21:39, Thomas Munro wrote: On Fri, Nov 8, 2019 at 12:44 PM Thomas Munro wrote: The reason for returning an empty string for "C" and "POSIX" is the following comment for get_collation_actual_version(): * A particular provider must always either return a non-NULL string or

Re: [HACKERS] Regression tests vs existing users in an installation

2019-11-26 Thread Peter Eisentraut
On 2019-06-29 19:21, Tom Lane wrote: Stephen Frost writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: We could make the new subdirectory be something specific like "src/test/modules/test_rolenames", but I think very likely we'll be wanting some additional test scripts that we likewise deem unsafe

Re: Collation versioning

2019-11-26 Thread Thomas Munro
On Fri, Nov 8, 2019 at 5:40 PM Laurenz Albe wrote: > On Fri, 2019-11-08 at 15:04 +1300, Thomas Munro wrote: > > 3. We don't know if pre-13 indexes are corrupted or not, and we'll > > record that with a special value just as in proposal #1, except that > > we could show a different hint for that

Re: Collation versions on Windows (help wanted, apply within)

2019-11-26 Thread Thomas Munro
On Fri, Nov 8, 2019 at 12:44 PM Thomas Munro wrote: > The reason for returning an empty string for "C" and "POSIX" is the > following comment for get_collation_actual_version(): > > * A particular provider must always either return a non-NULL string or return > * NULL (if it doesn't support

Re: Remove configure --disable-float4-byval and --disable-float8-byval

2019-11-26 Thread Tom Lane
Peter Eisentraut writes: > My revised proposal is to remove --disable-float8-byval as a configure > option but keep it as an option in pg_config_manual.h. It is no longer > useful as a user-facing option, but as was pointed out, it is somewhat > useful for developers, so pg_config_manual.h

Re: Remove configure --disable-float4-byval and --disable-float8-byval

2019-11-26 Thread Peter Eisentraut
My revised proposal is to remove --disable-float8-byval as a configure option but keep it as an option in pg_config_manual.h. It is no longer useful as a user-facing option, but as was pointed out, it is somewhat useful for developers, so pg_config_manual.h seems like the right place. --

Re: pglz performance

2019-11-26 Thread Tomas Vondra
On Tue, Nov 26, 2019 at 08:17:13PM +0100, Peter Eisentraut wrote: On 2019-11-26 10:43, Tomas Vondra wrote: In general, I think the results for both patches seem clearly a win, but maybe patch 1 is bit better, especially on the newer (xeon) CPU. So I'd probably go with that one. Patch 1 is

Re: [HACKERS] Block level parallel vacuum

2019-11-26 Thread Masahiko Sawada
On Tue, 26 Nov 2019 at 13:34, Amit Kapila wrote: > > On Mon, Nov 25, 2019 at 5:36 PM Amit Kapila wrote: > > > > 2. > > lazy_parallel_vacuum_or_cleanup_indexes() > > { > > .. > > .. > > } > > > > Here, it seems that we can increment/decrement the > > VacuumActiveNWorkers even when there is no

Re: pglz performance

2019-11-26 Thread Peter Eisentraut
On 2019-11-26 10:43, Tomas Vondra wrote: In general, I think the results for both patches seem clearly a win, but maybe patch 1 is bit better, especially on the newer (xeon) CPU. So I'd probably go with that one. Patch 1 is also the simpler patch, so it seems clearly preferable. -- Peter

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

2019-11-26 Thread Tom Lane
Amit Langote writes: > On Thu, Nov 21, 2019 at 6:34 AM Tom Lane wrote: >> The comment for inh_root_relid seems rather inadequate, since it >> fails to mention the special case for UNION ALL subqueries. >> But do we even need that special case? It looks to me like the >> walk-up-to-parent code

Re: Copyright information in source files

2019-11-26 Thread vignesh C
On Sun, Nov 24, 2019 at 8:44 PM Tom Lane wrote: > > John Naylor writes: > > On Sat, Nov 23, 2019 at 11:39 PM vignesh C wrote: > >> * Copyright (c) 2016-2019, PostgreSQL Global Development Group > > > While we're talking about copyrights, I noticed while researching > > something else that the

Re: ERROR: attribute number 6 exceeds number of columns 5

2019-11-26 Thread Tom Lane
Andreas Joseph Krogh writes: > Run the attached script and you'll get: > psql -f error.sql -d test > psql:error.sql:37: ERROR: attribute number 6 exceeds number of columns 5 Hmm, interesting. IMO, that *should* have thrown an error, but of course not that one. The ADD COLUMN operations are

Re: benchmarking Flex practices

2019-11-26 Thread Tom Lane
John Naylor writes: > It seems something is not quite right in v9 with the error position reporting: > SELECT U&'wrong: +0061' UESCAPE '+'; > ERROR: invalid Unicode escape character at or near "'+'" > LINE 1: SELECT U&'wrong: +0061' UESCAPE '+'; > -^ >

Re: FETCH FIRST clause WITH TIES option

2019-11-26 Thread Alvaro Herrera
I rebased this patch, and my proposed changes are in 0002. Looking at the changes in ExecLimit, I'm wondering if it would be better to add a new state to the state machine there -- instead of doing all the work in duplicative code in the LIMIT_INWINDOW case, have that one only save the current

[PATCH] Increase the maximum value track_activity_query_size

2019-11-26 Thread v . makarov
Hi Hackers, Some ORMs may generate queries larger than the maximum possible value of track_activity_query_size (100 kB). Is there any reasons to limit the maximum value of track_activity_query_size to such small value? Increasing the maximum value to 1 MB will help partially solve this

ERROR: attribute number 6 exceeds number of columns 5

2019-11-26 Thread Andreas Joseph Krogh
Run the attached script and you'll get: psql -f error.sql -d test psql:error.sql:37: ERROR: attribute number 6 exceeds number of columns 5 Splitting up the alter-table like this makes it work: alter table access add column start_timestamp timestamp not null DEFAULT CURRENT_TIMESTAMP, add

RE: [PATCH] Remove twice assignment with var pageop (nbtree.c).

2019-11-26 Thread Ranier Vilela
Same case on nbtpage.c at line 1637, with var opaque. make check, passed all 195 tests here with all commits. Ranier Vileladiff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 268f869a36..144fefccad 100644 --- a/src/backend/access/nbtree/nbtpage.c +++

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-26 Thread Kyotaro Horiguchi
At Sun, 24 Nov 2019 22:08:39 -0500, Noah Misch wrote in > On Mon, Nov 25, 2019 at 11:08:54AM +0900, Kyotaro Horiguchi wrote: > > At Sat, 23 Nov 2019 16:21:36 -0500, Noah Misch wrote in > > > I noticed an additional defect: > > > > > > BEGIN; > > > CREATE TABLE t (c) AS SELECT 1; > > >

Re: [HACKERS] Block level parallel vacuum

2019-11-26 Thread Amit Kapila
On Mon, Nov 25, 2019 at 5:36 PM Amit Kapila wrote: > > 2. > lazy_parallel_vacuum_or_cleanup_indexes() > { > .. > .. > } > > Here, it seems that we can increment/decrement the > VacuumActiveNWorkers even when there is no work performed by the > leader backend. How about moving increment/decrement

Re: progress report for ANALYZE

2019-11-26 Thread Alvaro Herrera
On 2019-Nov-26, Tatsuro Yamada wrote: > > I wonder whether we need the total number of ext stats on > > pg_stat_progress_analyze or not. As you might know, there is the same > > counter on pg_stat_progress_vacuum and pg_stat_progress_cluster. > > For example, index_vacuum_count and

[PATCH] Remove twice assignment with var pageop (nbtree.c).

2019-11-26 Thread Ranier Vilela
Hi, The var pageop has twice assigment, maybe is a mistake? The assigned in the line 593, has no effect? Ranier Vileladiff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 44f6283950..29c7b41c8c 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++

Re: benchmarking Flex practices

2019-11-26 Thread John Naylor
On Tue, Nov 26, 2019 at 5:51 AM Tom Lane wrote: > > [ My apologies for being so slow to get back to this ] No worries -- it's a nice-to-have, not something our users are excited about. > It struck me though that there's another solution we haven't discussed, > and that's to make the token

Re: A problem about partitionwise join

2019-11-26 Thread Etsuro Fujita
Hi Richard, On Fri, Aug 30, 2019 at 3:08 AM Etsuro Fujita wrote: > On Thu, Aug 29, 2019 at 6:45 PM Richard Guo wrote: > > Attached is a patch as an attempt to address this issue. The idea is > > quite straightforward. When building partition info for joinrel, we > > generate any possible

Re: Attempt to consolidate reading of XLOG page

2019-11-26 Thread Antonin Houska
Alvaro Herrera wrote: > On 2019-Nov-25, Antonin Houska wrote: > > > Alvaro Herrera wrote: > > > > I see no reason to leave ws_off. We can move that to XLogReaderState; I > > > did that here. We also need the offset in WALReadError, though, so I > > > added it there too. Conceptually it

Re: pglz performance

2019-11-26 Thread Tomas Vondra
On Mon, Nov 25, 2019 at 05:29:40PM +0900, Michael Paquier wrote: On Mon, Nov 25, 2019 at 01:21:27PM +0500, Andrey Borodin wrote: I think status Needs Review describes what is going on better. It's not like something is awaited from my side. Indeed. You are right so I have moved the patch

RE: GROUPING SETS and SQL standard

2019-11-26 Thread Phil Florent
Thank you, it's noticed. Seems Oracle does not like too much "grouping sets". We discovered we had more serious "wrong results" bugs with this clause in our migration process. Anyway we don't have to maintain a double compatibility and soon it won't be a problem anymore. Regards Phil