Re: Have we tried to treat CTE as SubQuery in planner?

2020-11-13 Thread Andy Fan
On Sat, Nov 14, 2020 at 2:44 PM Jesse Zhang wrote: > Hi, > > On Fri, Nov 13, 2020 at 10:04 PM Andy Fan wrote: > > > > Hi: > > > > Take the following example: > > > > insert into cte1 select i, i from generate_series(1, 100)i; > > create index on cte1(a); > > > > explain > > with cte1 as

Re: Have we tried to treat CTE as SubQuery in planner?

2020-11-13 Thread Andy Fan
On Sat, Nov 14, 2020 at 2:14 PM Tom Lane wrote: > Andy Fan writes: > > Take the following example: > > > insert into cte1 select i, i from generate_series(1, 100)i; > > create index on cte1(a); > > > explain > > with cte1 as (select * from cte1) > > select * from c where a = 1; > > > It

Re: Have we tried to treat CTE as SubQuery in planner?

2020-11-13 Thread Jesse Zhang
Hi, On Fri, Nov 13, 2020 at 10:04 PM Andy Fan wrote: > > Hi: > > Take the following example: > > insert into cte1 select i, i from generate_series(1, 100)i; > create index on cte1(a); > > explain > with cte1 as (select * from cte1) > select * from c where a = 1; > ITYM: EXPLAIN WITH c AS

Re: Have we tried to treat CTE as SubQuery in planner?

2020-11-13 Thread Julien Rouhaud
On Sat, Nov 14, 2020 at 2:04 PM Andy Fan wrote: > > Hi: > > Take the following example: > > insert into cte1 select i, i from generate_series(1, 100)i; > create index on cte1(a); > > explain > with cte1 as (select * from cte1) > select * from c where a = 1; > > It needs to do seq scan on

Re: Have we tried to treat CTE as SubQuery in planner?

2020-11-13 Thread Tom Lane
Andy Fan writes: > Take the following example: > insert into cte1 select i, i from generate_series(1, 100)i; > create index on cte1(a); > explain > with cte1 as (select * from cte1) > select * from c where a = 1; > It needs to do seq scan on the above format, however it is pretty > quick

Have we tried to treat CTE as SubQuery in planner?

2020-11-13 Thread Andy Fan
Hi: Take the following example: insert into cte1 select i, i from generate_series(1, 100)i; create index on cte1(a); explain with cte1 as (select * from cte1) select * from c where a = 1; It needs to do seq scan on the above format, however it is pretty quick if we change the query to

Re: Supporting = operator in gin/gist_trgm_ops

2020-11-13 Thread Alexander Korotkov
Hi! On Fri, Nov 13, 2020 at 1:47 PM Georgios wrote: > In short, I think v3 of the patch looks good to change to 'RFC' status. > Given the possible costing concerns, I will refrain from changing the > status just yet, to give the opportunity to more reviewers to chime in. > If in the next few

Re: "pg_ctl: the PID file ... is empty" at end of make check

2020-11-13 Thread Tom Lane
Thomas Munro writes: > My bug report got an automated-looking message telling me to retest in > Big Sur beta 6 back in September, and I've just now upgraded an old > x86 Mac to Big Sur 11.01 and I can no longer reproduce the problem, so > it looks like it was fixed! Thanks, Apple. Hah! I've

Re: "pg_ctl: the PID file ... is empty" at end of make check

2020-11-13 Thread Thomas Munro
On Fri, Oct 18, 2019 at 2:26 PM Thomas Munro wrote: > On Fri, Oct 18, 2019 at 1:21 AM Tom Lane wrote: > > Thomas Munro writes: > > > On Tue, Oct 15, 2019 at 1:55 PM Tom Lane wrote: > > >> and now prairiedog has shown it too: > > >>

Re: Misc typos

2020-11-13 Thread Michael Paquier
On Fri, Nov 13, 2020 at 12:43:41PM +0100, Daniel Gustafsson wrote: > When spellchecking one of my patches for submission, a few other typos fell > out > as per the attached. The one change which isn't in a comment is the object > address class description used for error messages: > > -

Re: PATCH: Batch/pipelining support for libpq

2020-11-13 Thread Alvaro Herrera
Here's a v25. I made a few more changes to the docs per David's suggestions; I also reordered the sections quite a bit. It's now like this: * Batch Mode * Using Batch Mode * Issuing Queries * Processing Results * Error Handling * Interleaving Result Processing and Query

Re: Zedstore - compressed in-core columnar storage

2020-11-13 Thread Tomas Vondra
On 11/13/20 8:07 PM, Jacob Champion wrote: > On Nov 12, 2020, at 2:40 PM, Tomas Vondra > wrote: >> >> Hi, >> >> Thanks for the updated patch. It's a quite massive amount of code - I I >> don't think we had many 2MB patches in the past, so this is by no means >> a full review. > > Thanks for

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-11-13 Thread Tom Lane
Alexander Lakhin writes: > 13.11.2020 23:16, Tom Lane wrote: >> That would soon lead us to changing every stat() caller in the system >> to have Windows-specific looping logic. No thanks. If we need to do >> this, let's put in a Windows wrapper layer comparable to pgwin32_open() >> for open().

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-11-13 Thread Alexander Lakhin
13.11.2020 23:16, Tom Lane wrote: > Alexander Lakhin writes: >> Shouldn't pg_ls_dir_files() retry stat() on ERROR_ACCESS_DENIED just >> like the pgwin32_open() does to ignore files in "delete pending" state? > That would soon lead us to changing every stat() caller in the system > to have

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-11-13 Thread Tom Lane
Alexander Lakhin writes: > Shouldn't pg_ls_dir_files() retry stat() on ERROR_ACCESS_DENIED just > like the pgwin32_open() does to ignore files in "delete pending" state? That would soon lead us to changing every stat() caller in the system to have Windows-specific looping logic. No thanks. If

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-11-13 Thread Alexander Lakhin
Hello hackers, 31.03.2020 19:41, Tom Lane wrote: > Justin Pryzby writes: >> I suggest to leave stat() alone in your patch for stable releases. I think >> it's okay if we change behavior so that a broken symlink is skipped instead >> of >> erroring (as a side effect of skipping ENOENT with

Re: Hash support for row types

2020-11-13 Thread Tom Lane
Peter Eisentraut writes: > Here is an updated patch with the type cache integration added. > To your point, this now checks each fields hashability before > considering a row type as hashable. It can still have run-time errors > for untyped record datums, but that's not something we can do

Re: Zedstore - compressed in-core columnar storage

2020-11-13 Thread Jacob Champion
On Nov 12, 2020, at 2:40 PM, Tomas Vondra wrote: > > Hi, > > Thanks for the updated patch. It's a quite massive amount of code - I I > don't think we had many 2MB patches in the past, so this is by no means > a full review. Thanks for taking a look! You're not kidding about the patch size.

Re: [PATCH] Support negative indexes in split_part

2020-11-13 Thread Tom Lane
Jacob Champion writes: > Patch looks good to me. Seems like a useful feature, and I agree that the > two-pass implementation makes the change very easy to review. LGTM too. I made a couple of cosmetic improvements and pushed it. > Quick note on test coverage: gcov marks the "needle not found"

Re: error_severity of brin work item

2020-11-13 Thread Justin Pryzby
On Fri, Nov 13, 2020 at 01:39:31PM -0300, Alvaro Herrera wrote: > On 2020-Nov-13, Justin Pryzby wrote: > > > I saw a bunch of these in my logs: > > > > log_time | 2020-10-25 22:59:45.619-07 > > database | > > left | could not open relation with OID 292103095 > > left | processing work

Table AM modifications to accept column projection lists

2020-11-13 Thread Soumyadeep Chakraborty
Hello, This patch introduces a set of changes to the table AM APIs, making them accept a column projection list. That helps columnar table AMs, so that they don't need to fetch all columns from disk, but only the ones actually needed. The set of changes in this patch is not exhaustive - there

Re: Add docs stub for recovery.conf

2020-11-13 Thread Bruce Momjian
On Fri, Nov 13, 2020 at 01:27:34PM +0800, Craig Ringer wrote: > On Fri, Nov 13, 2020 at 11:50 AM Bruce Momjian wrote: >   > > > So you are saying you don't think you are getting sufficient thought > > into your proposal, and getting just a reflex?  Just because we don't > > agree

Re: error_severity of brin work item

2020-11-13 Thread Alvaro Herrera
On 2020-Nov-13, Justin Pryzby wrote: > I saw a bunch of these in my logs: > > log_time | 2020-10-25 22:59:45.619-07 > database | > left | could not open relation with OID 292103095 > left | processing work entry for relation > "ts.child.alarms_202010_alarm_clear_time_idx" > > Those

Re: [PATCH] Combine same ternary types in GIN and TSearch

2020-11-13 Thread Tom Lane
Heikki Linnakangas writes: > On 13/11/2020 11:04, Pavel Borisov wrote: >> For historical reasons, now we have two differently named but similar >> ternary data types in TSearch and Gin text-related types. Before v13 >> there was also Gin's private TS_execute() version, from which we >>

Re: Strange behavior with polygon and NaN

2020-11-13 Thread Tom Lane
Kyotaro Horiguchi writes: > At Tue, 10 Nov 2020 14:30:08 -0500, Tom Lane wrote in >> For instance, {1,-1,0} is the line "x = y". We could argue about >> whether it'd be sensible to return zero for the distance between that >> and the point (inf,inf), but surely any point with one inf and one

don't allocate HashAgg hash tables when running explain only

2020-11-13 Thread Alexey Bashtanov
Hi, I got somewhat scared when my explain took a few seconds to complete and used a few gigs of RAM. To reproduce try the following: discard temp; create temp table a as select to_timestamp(generate_series(1, 7000)) i; analyze a; set work_mem to '3GB'; explain select distinct a1.i - a2.i from

error_severity of brin work item

2020-11-13 Thread Justin Pryzby
I saw a bunch of these in my logs: log_time | 2020-10-25 22:59:45.619-07 database | left | could not open relation with OID 292103095 left | processing work entry for relation "ts.child.alarms_202010_alarm_clear_time_idx" Those happen following a REINDEX job on that index. I think

Re: [PATCH] remove deprecated v8.2 containment operators

2020-11-13 Thread Tom Lane
Peter Eisentraut writes: > On 2020-11-12 23:28, Tom Lane wrote: >> I'm on board with pulling these now --- 8.2 to v14 is plenty of >> deprecation notice. However, the patch seems incomplete in that >> the code support for these is still there -- look for >> RTOldContainedByStrategyNumber and

Re: WIP: WAL prefetch (another approach)

2020-11-13 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 11/13/20 3:20 AM, Thomas Munro wrote: > > I'm not really sure what to do about achive restore scripts that > > block. That seems to be fundamentally incompatible with what I'm > > doing here. > > IMHO we can't do much about

Re: [PATCH] remove deprecated v8.2 containment operators

2020-11-13 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Thu, Nov 12, 2020 at 11:28 PM Tom Lane wrote: > > > The changes to the contrib modules appear to be incomplete in some ways. > > > In cube, hstore, and seg, there are no changes to the extension > > > scripts to remove the

Re: recovery_target immediate timestamp

2020-11-13 Thread Fujii Masao
On 2020/11/13 8:39, Euler Taveira wrote: On Wed, 11 Nov 2020 at 22:40, Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: On 2020/11/12 6:00, Euler Taveira wrote: > The first patch adds a new message that prints the latest completed checkpoint > when the consistent

Re: POC: Cleaning up orphaned files using undo logs

2020-11-13 Thread Antonin Houska
Amit Kapila wrote: > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote: > > > > > > No background undo > > -- > > > > Reduced complexity of the patch seems to be the priority at the moment. Amit > > suggested that cleanup of an orphaned relation file is simple enough to be >

Re: [HACKERS] logical decoding of two-phase transactions

2020-11-13 Thread Amit Kapila
On Wed, Nov 11, 2020 at 4:30 PM Ajin Cherian wrote: > > Did some further tests on the problem I saw and I see that it does not > have anything to do with this patch. I picked code from top of head. > If I have enough changes in a transaction to initiate streaming, then > it will also stream the

Re: [HACKERS] logical decoding of two-phase transactions

2020-11-13 Thread Ajin Cherian
On Fri, Nov 13, 2020 at 9:44 PM Amit Kapila wrote: > > On Thu, Nov 12, 2020 at 2:28 PM Ajin Cherian wrote: > Hmm, introducing an additional boolean variable for this doesn't seem > like a good idea neither the other alternative suggested by you. How > about if we change the comment to make it

Re: [HACKERS] logical decoding of two-phase transactions

2020-11-13 Thread Amit Kapila
On Tue, Nov 10, 2020 at 4:19 PM Ajin Cherian wrote: > > I was doing some testing, and I found some issues. Two issues. The > first one, seems to be a behaviour that might be acceptable, the > second one not so much. > I was using test_decoding, not sure how this might behave with the > pg_output

Re: MultiXact\SLRU buffers configuration

2020-11-13 Thread Andrey Borodin
> 10 нояб. 2020 г., в 23:07, Tomas Vondra > написал(а): > > On 11/10/20 7:16 AM, Andrey Borodin wrote: >> >> >> but this picture was not stable. >> > > Seems we haven't made much progress in reproducing the issue :-( I guess > we'll need to know more about the machine where this happens.

Re: [PATCH] Combine same ternary types in GIN and TSearch

2020-11-13 Thread Pavel Borisov
> > GIN is not just for full-text search, so using TSTernaryValue in > GinScanKeyData is wrong. And it would break existing extensions. > > I didn't look much further than that, but I've got a feeling that > combining those is a bad idea. TSTernaryValue is used in text-search > code, even when

Misc typos

2020-11-13 Thread Daniel Gustafsson
When spellchecking one of my patches for submission, a few other typos fell out as per the attached. The one change which isn't in a comment is the object address class description used for error messages: - "extented statistics", + "extended statistics", It's used

Re: POC: Cleaning up orphaned files using undo logs

2020-11-13 Thread Antonin Houska
Thomas Munro wrote: > On Thu, Nov 12, 2020 at 10:15 PM Antonin Houska wrote: > I saw that -- great news! -- and have been meaning to write for a > while. I think I am nearly ready to talk about it again. I'm looking forward to it :-) > 100% that it's worth trying to do something much

Re: Detecting File Damage & Inconsistencies

2020-11-13 Thread Simon Riggs
On Fri, 13 Nov 2020 at 00:50, tsunakawa.ta...@fujitsu.com wrote: > > From: Simon Riggs > > If a rogue user/process is suspected, this would allow you to identify > > more easily the changes made by specific sessions/users. > > Isn't that kind of auditing a job of pgAudit or log_statement = mod?

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-13 Thread Bharath Rupireddy
On Thu, Nov 12, 2020 at 10:06 AM Fujii Masao wrote: > > Thanks for the analysis! I pushed the patch. > Thanks! Since we are replacing custom SIGHUP and SIGTERM handlers with standard ones, how about doing the same thing in worker_spi.c? I posted a patch previously [1] in this mail thread. If it

Re: Online checksums patch - once again

2020-11-13 Thread Heikki Linnakangas
On 12/11/2020 15:17, Daniel Gustafsson wrote: On 5 Oct 2020, at 14:14, Heikki Linnakangas wrote: I would expect the checksums worker to be automatically started at postmaster startup. Can we make that happen? A dynamic background worker has to be registered from a regular backend, so it's

Re: Multi Inserts in CREATE TABLE AS - revived patch

2020-11-13 Thread Bharath Rupireddy
On Tue, Nov 10, 2020 at 3:47 PM Paul Guo wrote: > > Thanks for doing this. There might be another solution - use raw insert > interfaces (i.e. raw_heap_insert()). > Attached is the test (not formal) patch that verifies this idea. > raw_heap_insert() writes the page into the > table files

Re: [PATCH] Combine same ternary types in GIN and TSearch

2020-11-13 Thread Heikki Linnakangas
On 13/11/2020 11:04, Pavel Borisov wrote: Hi, hackers! For historical reasons, now we have two differently named but similar ternary data types in TSearch and Gin text-related types. Before v13 there was also Gin's private TS_execute() version, from which we eventually shifted to Tsearch's

Re: Supporting = operator in gin/gist_trgm_ops

2020-11-13 Thread Georgios
‐‐‐ Original Message ‐‐‐ On Friday, November 13, 2020 10:50 AM, Julien Rouhaud wrote: > On Wed, Nov 11, 2020 at 8:34 PM Georgios Kokolatos > gkokola...@protonmail.com wrote: > > > The following review has been posted through the commitfest application: > > make installcheck-world:

Re: WIP: WAL prefetch (another approach)

2020-11-13 Thread Tomas Vondra
On 11/13/20 3:20 AM, Thomas Munro wrote: > > ... > > I'm not really sure what to do about achive restore scripts that > block. That seems to be fundamentally incompatible with what I'm > doing here. > IMHO we can't do much about that, except for documenting it - if the prefetch can't work

Re: [HACKERS] logical decoding of two-phase transactions

2020-11-13 Thread Amit Kapila
On Thu, Nov 12, 2020 at 2:28 PM Ajin Cherian wrote: > > On Wed, Nov 11, 2020 at 12:35 AM Amit Kapila wrote: > I have rearranged the code in DecodeCommit/Abort/Prepare so > > that it does only the required things (like in DecodeCommit was still > > processing subtxns even when it has to just

RE: Terminate the idle sessions

2020-11-13 Thread kuroda.hay...@fujitsu.com
Dear Li, I read your patch, and I think the documentation is too simple to avoid all problems. (I think if some connection pooling is used, the same problem will occur.) Could you add some explanations in the doc file? I made an example: ``` Note that this values should be set to zero if you

Re: Supporting = operator in gin/gist_trgm_ops

2020-11-13 Thread Julien Rouhaud
On Wed, Nov 11, 2020 at 8:34 PM Georgios Kokolatos wrote: > > The following review has been posted through the commitfest application: > make installcheck-world: tested, passed > Implements feature: tested, passed > Spec compliant: not tested > Documentation:not

Re: logical streaming of xacts via test_decoding is broken

2020-11-13 Thread Amit Kapila
On Thu, Nov 12, 2020 at 3:10 PM Dilip Kumar wrote: > > On Tue, Nov 10, 2020 at 7:20 PM Amit Kapila wrote: > > > > On Tue, Nov 10, 2020 at 2:25 PM Dilip Kumar wrote: > > > 3. Can you please prepare a separate patch for test case changes so > > that it would be easier to verify that it fails

pg_rewind copies

2020-11-13 Thread Heikki Linnakangas
If a file is modified and becomes larger in the source system while pg_rewind is running, pg_rewind can leave behind a partial copy of file. That's by design, and it's OK for relation files because they're replayed from WAL. But it can cause trouble for configuration files. I ran into this

Re: [PATCH] remove deprecated v8.2 containment operators

2020-11-13 Thread Magnus Hagander
On Thu, Nov 12, 2020 at 11:28 PM Tom Lane wrote: > > > The changes to the contrib modules appear to be incomplete in some ways. > > In cube, hstore, and seg, there are no changes to the extension > > scripts to remove the operators. All you're doing is changing the C > > code to no longer

Re: Parallel INSERT (INTO ... SELECT ...)

2020-11-13 Thread Greg Nancarrow
On Wed, Nov 4, 2020 at 2:18 PM Amit Kapila wrote: > > Or you might want to consider moving the check related to > IsModifySupportedInParallelMode() inside > PrepareParallelModeForModify(). That way the code might look a bit > cleaner. > Posting an updated Parallel SELECT for "INSERT INTO ...

[PATCH] Combine same ternary types in GIN and TSearch

2020-11-13 Thread Pavel Borisov
Hi, hackers! For historical reasons, now we have two differently named but similar ternary data types in TSearch and Gin text-related types. Before v13 there was also Gin's private TS_execute() version, from which we eventually shifted to Tsearch's TS_execute(). To make things more even and

Re: Parallel copy

2020-11-13 Thread Amit Kapila
On Wed, Nov 11, 2020 at 10:42 PM vignesh C wrote: > > On Tue, Nov 10, 2020 at 7:27 PM Amit Kapila wrote: > > > > On Tue, Nov 10, 2020 at 7:12 PM vignesh C wrote: > > > > > > On Tue, Nov 3, 2020 at 2:28 PM Amit Kapila > > > wrote: > > > > > > > > > > I have worked to provide a patch for the

Re: Bogus documentation for bogus geometric operators

2020-11-13 Thread Pavel Borisov
> > 1. The patch removes <^ and >^ from func.sgml, which is fine, but shouldn't there be an addition for the new operators? (I think > I fully agree and added "point" as a possible input type for <<| and |>> in manual. PFA v5 > undocumented. Maybe instead of removing, change the text to be >

Re: In-placre persistance change of a relation

2020-11-13 Thread Kyotaro Horiguchi
At Fri, 13 Nov 2020 07:15:41 +, "osumi.takami...@fujitsu.com" wrote in > Hello, Tsunakawa-San > Thanks for sharing it! > > Do you know the reason why data copy was done before? And, it may be > > odd for me to ask this, but I think I saw someone referred to the past > > discussion that

Re: Support for NSS as a libpq TLS backend

2020-11-13 Thread Jacob Champion
On Nov 11, 2020, at 10:57 AM, Jacob Champion wrote: > > False alarm -- the stderr debugging I'd added in to track down the > assertion tripped up the "no stderr" tests. Zero failing tests now. I took a look at the OpenSSL interop problems you mentioned upthread. I don't see a hang like you did,