Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2022-02-22 Thread Tomas Vondra
On 2/22/22 01:36, Fujii Masao wrote: > > > On 2022/02/18 22:28, Tomas Vondra wrote: >> Hi, >> >> here's a slightly updated version of the patch series. > > Thanks for updating the patches! > > >> The 0001 part >> adds tracking of serv

Re: logical decoding and replication of sequences

2022-02-22 Thread Tomas Vondra
On 2/19/22 06:33, Amit Kapila wrote: > On Sat, Feb 19, 2022 at 7:48 AM Tomas Vondra > wrote: >> >> Do we need to handle both FOR ALL TABLES and FOR ALL SEQUENCES at the >> same time? That seems like a reasonable thing people might want. >> > > +1. That see

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2022-02-22 Thread Tomas Vondra
On 2/22/22 21:12, Tomas Vondra wrote: > On 2/22/22 01:36, Fujii Masao wrote: >> >> >> On 2022/02/18 22:28, Tomas Vondra wrote: >>> Hi, >>> >>> here's a slightly updated version of the patch series. >> >> Thanks for u

Re: logical decoding and replication of sequences

2022-02-23 Thread Tomas Vondra
On 2/23/22 12:10, Amit Kapila wrote: > On Wed, Feb 23, 2022 at 4:54 AM Tomas Vondra > wrote: >> >> 7) This updates psql to do roughly the same thing as for tables, so \dRp >> now list sequences added either directly or through schema, so you might >> get

Re: logical decoding and replication of sequences

2022-02-23 Thread Tomas Vondra
On 2/23/22 18:33, Euler Taveira wrote: > On Wed, Feb 23, 2022, at 1:07 PM, Tomas Vondra wrote: >> Maybe, but I don't think it's very common to have that many >> schemas added to the same publication. And it probably does not >> make much difference whether you

Re: Use generation context to speed up tuplesorts

2022-02-23 Thread Tomas Vondra
the situation, and recompact. Create a new context, copy > all the tuples over, delete the old context. That could be a win even in less > adversarial situations than "a single tuple per generation block". > What about pointers to the chunks in the old memory context? regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Tomas Vondra
bit smaller, thanks to saving 20B. regards [1] https://www.postgresql.org/message-id/17406-71e02820ae79bb40%40postgresql.org -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL CompanyFrom c6c5471aebf49da96e47deb5286a6f7065068bd7 Mon Sep 17 00:00:00 2001 Fro

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Tomas Vondra
On 2/24/22 23:06, Andres Freund wrote: > Hi, > > On 2022-02-24 22:15:20 +0100, Tomas Vondra wrote: >> After thinking about this I only see two options: >> >> 1) Don't apply the patch and tell everyone using ltree_gist they need to >> rebuild the inde

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Tomas Vondra
On 2/24/22 23:13, Tomas Vondra wrote: > > > On 2/24/22 23:06, Andres Freund wrote: >> Hi, >> >> On 2022-02-24 22:15:20 +0100, Tomas Vondra wrote: >>> After thinking about this I only see two options: >>> >>> 1) Don't apply the patch

Re: row filtering for logical replication

2022-03-02 Thread Tomas Vondra
Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL CompanyFrom e781f840e38701c63d8b57ff36bd520f2cced6ad Mon Sep 17 00:00:00 2001 From: Tomas Vondra Date: Sat, 26 Feb 2022 17:33:09 +0100 Subject: [PATCH 1/3] Verify changing WHERE condition for a publication Commit

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-03-04 Thread Tomas Vondra
On 3/4/22 20:29, Nikita Glukhov wrote: > On 25.02.2022 00:15, Tomas Vondra wrote: > >> Hi, >> >> Victor Yegorov reported a crash related to a GiST index on ltree [1], >> following a pg_upgrade from 12.x to 14.x, with a data set reproducing >> this. I spent

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-03-04 Thread Tomas Vondra
On 3/4/22 23:09, Nikita Glukhov wrote: > On 04.03.2022 23:28, Tom Lane wrote: > >> Tomas Vondra writes: >>> On 3/4/22 20:29, Nikita Glukhov wrote: >>>> So, we probably have corrupted indexes that were updated since such >>>> "incomplete" u

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-03-06 Thread Tomas Vondra
On 3/6/22 08:09, Alexander Korotkov wrote: > Hi! > > Sorry for this terrible oversight by me. > > On Sat, Mar 5, 2022 at 10:13 AM Tomas Vondra > wrote: >> On 3/4/22 23:09, Nikita Glukhov wrote: >>> On 04.03.2022 23:28, Tom Lane wrote: >>> >>>&g

Re: row filtering for logical replication

2022-03-06 Thread Tomas Vondra
On 3/3/22 21:07, Euler Taveira wrote: > On Thu, Mar 3, 2022, at 7:47 AM, Amit Kapila wrote: >> LGTM. I'll push this tomorrow unless Tomas or Euler feels otherwise. > Sounds good to me. > +1 -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: logical decoding and replication of sequences

2022-03-07 Thread Tomas Vondra
On 3/1/22 12:53, Amit Kapila wrote: > On Mon, Feb 28, 2022 at 5:16 PM Amit Kapila wrote: >> >> On Sat, Feb 12, 2022 at 6:04 AM Tomas Vondra >> wrote: >>> >>> On 2/10/22 19:17, Tomas Vondra wrote: >>>> I've polished & pushed the

Re: logical decoding and replication of sequences

2022-03-07 Thread Tomas Vondra
On 2/28/22 12:46, Amit Kapila wrote: > On Sat, Feb 12, 2022 at 6:04 AM Tomas Vondra > wrote: >> >> On 2/10/22 19:17, Tomas Vondra wrote: >>> I've polished & pushed the first part adding sequence decoding >>> infrastructure etc. Attached are the two rem

Re: logical decoding and replication of sequences

2022-03-07 Thread Tomas Vondra
On 3/7/22 17:39, Tomas Vondra wrote: > > > On 3/1/22 12:53, Amit Kapila wrote: >> On Mon, Feb 28, 2022 at 5:16 PM Amit Kapila wrote: >>> >>> On Sat, Feb 12, 2022 at 6:04 AM Tomas Vondra >>> wrote: >>>> >>>> On 2/10/22 19:1

Re: logical decoding and replication of sequences

2022-03-07 Thread Tomas Vondra
On 3/7/22 17:53, Tomas Vondra wrote: > On 2/28/22 12:46, Amit Kapila wrote: >> On Sat, Feb 12, 2022 at 6:04 AM Tomas Vondra >> wrote: >>> >>> On 2/10/22 19:17, Tomas Vondra wrote: >>>> I've polished & pushed the first part adding sequence

Re: WIP: WAL prefetch (another approach)

2022-03-08 Thread Tomas Vondra
ink about committing this soon. +1. I don't have the capacity/hardware to do more testing at the moment, but all of this looks reasonable. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: logical decoding and replication of sequences

2022-03-08 Thread Tomas Vondra
On 3/7/22 22:25, Tomas Vondra wrote: > > > On 3/7/22 17:53, Tomas Vondra wrote: >> On 2/28/22 12:46, Amit Kapila wrote: >>> On Sat, Feb 12, 2022 at 6:04 AM Tomas Vondra >>> wrote: >>>> >>>> On 2/10/22 19:17, Tomas Vondra wrote: &g

Re: logical decoding and replication of sequences

2022-03-08 Thread Tomas Vondra
On 3/7/22 22:11, Tomas Vondra wrote: > > > On 3/7/22 17:39, Tomas Vondra wrote: >> >> >> On 3/1/22 12:53, Amit Kapila wrote: >>> On Mon, Feb 28, 2022 at 5:16 PM Amit Kapila wrote: >>>> >>>> On Sat, Feb 12, 2022 at 6:04 AM Tomas Vo

Re: logical decoding and replication of sequences

2022-03-09 Thread Tomas Vondra
On 3/9/22 12:41, Amit Kapila wrote: > On Wed, Mar 9, 2022 at 4:14 AM Tomas Vondra > wrote: >> >> On 3/7/22 22:11, Tomas Vondra wrote: >>> >>> I've pushed this simple fix. Not sure it'll fix the assert failures on >>> skink/locust, though.

Re: Column Filtering in Logical Replication

2022-03-09 Thread Tomas Vondra
havior of row filter is okay > whereas for column filter it uses the wrong column list. We will share > the tests and results with you in a later email. We are trying to > unify the column filter queries with row filter to make their behavior > the same and will share the findings once it is done. I hope if we are > able to achieve this that we will reduce the chances of bugs in this > area. > OK, I'll take a look at that email. > Note: I think the first two patches for tests are not required after > commit ceb57afd3c. > Right. Will remove. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-03-10 Thread Tomas Vondra
LTREE_GET_ASIGLEN() macros. > > No feedback yet. I'm going to push this if no objections. > WFM. I certainly don't have a better idea how to fix this. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Column Filtering in Logical Replication

2022-03-10 Thread Tomas Vondra
On 3/9/22 10:20, Peter Eisentraut wrote: > > On 07.03.22 16:18, Tomas Vondra wrote: >> AFAICS these issues should be resolved by the adoption of the row-filter >> approach (i.e. it should fail the same way as for row filter). > > The first two patches (additional t

Re: Column Filtering in Logical Replication

2022-03-10 Thread Tomas Vondra
On 3/10/22 19:17, Tomas Vondra wrote: > On 3/9/22 11:12, houzj.f...@fujitsu.com wrote: >> Hi, >> >> Here are some tests and results about the table sync query of >> column filter patch and row filter. >> >> 1) multiple publications which publish schema of

Re: Column Filtering in Logical Replication

2022-03-10 Thread Tomas Vondra
On 3/10/22 04:09, Amit Kapila wrote: > On Wed, Mar 9, 2022 at 3:33 PM Amit Kapila wrote: >> >> On Mon, Mar 7, 2022 at 8:48 PM Tomas Vondra >> wrote: >> >>> OK, I reworked this to do the same thing as the row filtering patch. >>> >> >>

Re: Column Filtering in Logical Replication

2022-03-10 Thread Tomas Vondra
On 3/11/22 03:46, Amit Kapila wrote: > On Fri, Mar 11, 2022 at 12:44 AM Tomas Vondra > wrote: >> >> On 3/10/22 04:09, Amit Kapila wrote: >>> On Wed, Mar 9, 2022 at 3:33 PM Amit Kapila wrote: >>>> >>>> On Mon, Mar 7, 2022 at 8:48 PM Tomas Vondr

Re: Column Filtering in Logical Replication

2022-03-11 Thread Tomas Vondra
On 3/11/22 08:05, wangw.f...@fujitsu.com wrote: > On Fri, Mar 11, 2022 at 9:57 AM Tomas Vondra > wrote: >> > Hi Tomas, > Thanks for your patches. > > On Mon, Mar 9, 2022 at 9:53 PM Tomas Vondra > wrote: >> On Wed, Mar 9, 2022 at 6:04 PM Amit Kapila wrote:

Re: Column Filtering in Logical Replication

2022-03-11 Thread Tomas Vondra
On 3/11/22 10:52, Amit Kapila wrote: > On Fri, Mar 11, 2022 at 7:26 AM Tomas Vondra > wrote: >> >> On 3/10/22 20:10, Tomas Vondra wrote: >>> >>> >>> FWIW I think the reason is pretty simple - pgoutput_row_filter_init is >>> b

Re: logical decoding and replication of sequences

2022-03-11 Thread Tomas Vondra
On 3/11/22 12:34, Amit Kapila wrote: > On Tue, Mar 8, 2022 at 11:59 PM Tomas Vondra > wrote: >> >> On 3/7/22 22:25, Tomas Vondra wrote: >>>> >>>> Interesting. I can think of one reason that might cause this - we log >>>> the first seq

Re: Column Filtering in Logical Replication

2022-03-14 Thread Tomas Vondra
On 3/14/22 10:53, Amit Kapila wrote: > On Mon, Mar 14, 2022 at 2:37 AM Tomas Vondra > wrote: >> >> On 3/12/22 05:30, Amit Kapila wrote: >>>> ... >>> >>> Okay, please find attached. I have done basic testing of this, if we >>> agree wi

Re: Column Filtering in Logical Replication

2022-03-14 Thread Tomas Vondra
On 3/14/22 12:12, houzj.f...@fujitsu.com wrote: > On Monday, March 14, 2022 5:08 AM Tomas Vondra > wrote: >> >> On 3/12/22 05:30, Amit Kapila wrote: >>>> ... >>> >>> Okay, please find attached. I have done basic testing of this, if we >>

Re: Column Filtering in Logical Replication

2022-03-14 Thread Tomas Vondra
On 3/14/22 13:47, Amit Kapila wrote: > On Mon, Mar 14, 2022 at 5:42 PM Tomas Vondra > wrote: >> >> On 3/14/22 12:12, houzj.f...@fujitsu.com wrote: >>> On Monday, March 14, 2022 5:08 AM Tomas Vondra >>> wrote: >> >> Anyway, the fix does not

Re: Column Filtering in Logical Replication

2022-03-15 Thread Tomas Vondra
On 3/15/22 05:43, Amit Kapila wrote: > On Mon, Mar 14, 2022 at 4:42 PM houzj.f...@fujitsu.com > wrote: >> >> On Monday, March 14, 2022 5:08 AM Tomas Vondra >> wrote: >>> >>> On 3/12/22 05:30, Amit Kapila wrote: >>>>> ... >>>&g

Re: Column Filtering in Logical Replication

2022-03-16 Thread Tomas Vondra
On 3/15/22 09:30, Tomas Vondra wrote: > > > On 3/15/22 05:43, Amit Kapila wrote: >> On Mon, Mar 14, 2022 at 4:42 PM houzj.f...@fujitsu.com >> wrote: >>> >>> On Monday, March 14, 2022 5:08 AM Tomas Vondra >>> wrote: >>>> >>&

Re: Column Filtering in Logical Replication

2022-03-17 Thread Tomas Vondra
arly next week. Unless someone thinks it's definitely not ready for that ... [1] https://www.postgresql.org/message-id/E1nUsch-0008rQ-FW%40gemulon.postgresql.org -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Column Filtering in Logical Replication

2022-03-18 Thread Tomas Vondra
On 3/18/22 06:52, Amit Kapila wrote: > On Fri, Mar 18, 2022 at 12:47 AM Tomas Vondra > wrote: >> >> I pushed the second fix. Interestingly enough, wrasse failed in the >> 013_partition test. I don't see how that could be caused by this >> particular commit

Re: Column Filtering in Logical Replication

2022-03-18 Thread Tomas Vondra
On 3/18/22 15:43, Tomas Vondra wrote: > > > On 3/18/22 06:52, Amit Kapila wrote: >> On Fri, Mar 18, 2022 at 12:47 AM Tomas Vondra >> wrote: >>> >>> I pushed the second fix. Interestingly enough, wrasse failed in the >>> 013_partition te

Re: Column Filtering in Logical Replication

2022-03-20 Thread Tomas Vondra
On 3/20/22 07:23, Amit Kapila wrote: > On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila wrote: >> >> On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra >> wrote: >> >>> So the question is why those two sync workers never complete - I guess >>> there's s

Re: logical decoding and replication of sequences

2022-03-21 Thread Tomas Vondra
On 3/21/22 14:05, Peter Eisentraut wrote: > On 20.03.22 23:55, Tomas Vondra wrote: >> Attached is a rebased patch, addressing most of the remaining issues. > > This looks okay to me, if the two FIXMEs are addressed.  Remember to > also update protocol.sgml if you change LOGICA

Re: PoC: using sampling to estimate joins / complex conditions

2022-03-21 Thread Tomas Vondra
On 3/22/22 00:35, Andres Freund wrote: > Hi, > > On 2022-01-21 01:06:37 +0100, Tomas Vondra wrote: >> Yeah, I haven't updated some of the test output because some of those >> changes are a bit wrong (and I think that's fine for a PoC patch). I >> shou

Re: PATCH: generate fractional cheapest paths in generate_orderedappend_path

2022-03-21 Thread Tomas Vondra
On 3/22/22 01:18, Andres Freund wrote: > Hi, > > On 2022-01-14 01:39:30 +0100, Tomas Vondra wrote: >> Are you interested / willing to do some of this work? > > This patch hasn't moved in the last two months. I think it may be time to > mark it as returned with fee

Re: Partial aggregates pushdown

2022-03-22 Thread Tomas Vondra
ctions is the right term, considering those functions are not "serializing" anything. If anything, it's the remote node that is serializing the agg state and the local not is deserializing it. Or maybe I just misunderstand where are the converter functions executed? regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Column Filtering in Logical Replication

2022-03-23 Thread Tomas Vondra
On 3/21/22 12:55, Amit Kapila wrote: > On Sat, Mar 19, 2022 at 3:56 AM Tomas Vondra > wrote: >> >> ... >> >> However, while looking at how pgoutput, I realized one thing - for row >> filters we track them "per operation", depending on which ope

Re: Column Filtering in Logical Replication

2022-03-23 Thread Tomas Vondra
On 3/21/22 12:28, Amit Kapila wrote: > On Fri, Mar 18, 2022 at 8:13 PM Tomas Vondra > wrote: >> >> Ah, thanks for reminding me - it's hard to keep track of all the issues >> in threads as long as this one. >> >> BTW do you have any opinion on the SET C

Re: Additional improvements to extended statistics

2020-12-01 Thread Tomas Vondra
On 12/1/20 9:15 AM, Dean Rasheed wrote: > On Sun, 29 Nov 2020 at 21:02, Tomas Vondra > wrote: >> >> Those are fairly minor issues. I don't have any deeper objections, and >> it seems committable. Do you plan to do that sometime soon? >> > > OK, I've

Re: Additional improvements to extended statistics

2020-12-02 Thread Tomas Vondra
On 12/2/20 4:51 PM, Dean Rasheed wrote: > On Sun, 29 Nov 2020 at 21:02, Tomas Vondra > wrote: >> >> I wonder how much of the comment before clauselist_selectivity should >> move to clauselist_selectivity_ext - it does talk about range clauses >> and so on, but c

Re: PoC/WIP: Extended statistics on expressions

2020-12-07 Thread Tomas Vondra
On 12/7/20 10:56 AM, Dean Rasheed wrote: > On Thu, 3 Dec 2020 at 15:23, Tomas Vondra > wrote: >> >> Attached is a patch series rebased on top of 25a9e54d2d. > > After reading this thread and [1], I think I prefer the name > "standard" rather than

Re: PoC/WIP: Extended statistics on expressions

2020-12-08 Thread Tomas Vondra
On 12/7/20 5:02 PM, Dean Rasheed wrote: > On Mon, 7 Dec 2020 at 14:15, Tomas Vondra > wrote: >> >> On 12/7/20 10:56 AM, Dean Rasheed wrote: >>> it might actually be >>> neater to have separate documented syntaxes for single- and >>> multi-column

Re: Additional improvements to extended statistics

2020-12-08 Thread Tomas Vondra
hes seem fine to me. > These are both quite small patches, that hopefully won't interfere > with any of the other extended stats patches. > I haven't tried, but it should not interfere with it too much. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

extended statistics - functional dependencies vs. MCV lists

2020-12-10 Thread Tomas Vondra
pendencies). regards [1] https://www.postgresql.org/message-id/CAMcsB%3Dy%3D3G_%2Bs_zFYPu2-O6OMWOvOkb3t1MU%3D907yk5RC_RaYw%40mail.gmail.com -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Fix generate_useful_gather_paths for parallel unsafe pathkeys

2020-12-15 Thread Tomas Vondra
we can get around that. We could assume require_parallel_safe=true, but we still need to pass the PlannerInfo pointer. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company >From 857e653607656d10586e2a0f1b936576513d4277 Mon Sep 17 00:00:00 2001 From: j

Re: enable_incremental_sort changes query behavior

2020-12-15 Thread Tomas Vondra
On 12/16/20 1:51 AM, Jaime Casanova wrote: > On Tue, Dec 1, 2020 at 4:08 AM Anastasia Lubennikova > wrote: >> >> On 01.12.2020 03:08, James Coleman wrote: >>> On Tue, Nov 3, 2020 at 4:39 PM Tomas Vondra >>> wrote: >>>> I've pushed the 0

Re: Fix generate_useful_gather_paths for parallel unsafe pathkeys

2020-12-21 Thread Tomas Vondra
Hi, I've pushed (and backpatched) all the fixes posted to this thread. I believe that covers all the incremental sort fixes, so I've marked [1] as committed. [1] https://commitfest.postgresql.org/31/2754/ regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The

Re: libpq compression

2020-12-22 Thread Tomas Vondra
, or so. Why would zstd need so much memory? It seems like a pretty serious disadvantage, so how could it become so popular? regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Fix typo about generate_gather_paths

2020-12-22 Thread Tomas Vondra
er Merge paths, but generate_useful_gather_paths also generates paths with sorts (which are clearly not order-preserving). So I think this comment will need a bit more work to update ... regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: libpq compression

2020-12-22 Thread Tomas Vondra
On 12/22/20 7:31 PM, Andrey Borodin wrote: 22 дек. 2020 г., в 23:15, Tomas Vondra написал(а): On 12/22/20 6:56 PM, Robert Haas wrote: On Tue, Dec 22, 2020 at 6:24 AM Daniil Zakhlystov wrote: When using bidirectional compression, Postgres resource usage correlates with the selected

Re: libpq compression

2020-12-22 Thread Tomas Vondra
On 12/22/20 8:03 PM, Tom Lane wrote: Tomas Vondra writes: I don't see aby benchmark results in this thread, allowing me to make that conclusion, and I find it hard to believe that 200MB/client is a sensible trade-off. It assumes you have that much memory, and it may allow easy DoS a

Re: Parallel bitmap index scan

2020-12-22 Thread Tomas Vondra
On 11/11/20 8:52 PM, Tomas Vondra wrote: > Hi, > > I took a look at this today, doing a bit of stress-testing, and I can > get it to crash because of segfaults in pagetable_create (not sure if > the issue is there, it might be just a symptom of an issue elsewhere). > > Attac

Re: How is this possible "publication does not exist"

2020-12-24 Thread Tomas Vondra
_entry ended up calling LoadPublications - did we just create the entry? regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pglz compression performance, take two

2020-12-26 Thread Tomas Vondra
dback. I don't see why it couldn't land in PG14. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: WIP: BRIN multi-range indexes

2021-01-02 Thread Tomas Vondra
Hi, On 1/2/21 7:42 AM, Thomas Munro wrote: > On Sun, Dec 20, 2020 at 1:16 PM Tomas Vondra > wrote: >> Attached is an updated version of the patch series, rebased on current >> master, and results for benchmark comparing the various bloom variants. > > Perhaps src/inclu

Re: Handing off SLRU fsyncs to the checkpointer

2021-01-03 Thread Tomas Vondra
of header files. In particular, it removed these functions - ShutdownCLOG(); - ShutdownCommitTs(); - ShutdownSUBTRANS(); - ShutdownMultiXact(); but we still have $ git grep ShutdownCLOG src/include/access/clog.h:extern void ShutdownCLOG(void); regards -- Tomas Vondra EnterpriseDB: http://www.

Re: [PATCH] Simple progress reporting for COPY command

2021-01-04 Thread Tomas Vondra
t; and "processed". I'd just say "number of bytes processed so far" instead. Other than that, it seems fine. I'm sure we could add more features, but it seems like a good start - I plan to get this committed once I get a patch fixing the docs issues. regards --

Re: PoC/WIP: Extended statistics on expressions

2021-01-04 Thread Tomas Vondra
quot;()" for the kinds list. Will fix. Again, this suggests there are TAP tests missing. That's it for now. I'll look at the optimiser changes next, and try to post more comments later this week. Thanks! regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: PoC/WIP: Extended statistics on expressions

2021-01-05 Thread Tomas Vondra
On 1/5/21 3:10 PM, Dean Rasheed wrote: On Tue, 5 Jan 2021 at 00:45, Tomas Vondra wrote: On 1/4/21 4:34 PM, Dean Rasheed wrote: * In src/bin/psql/describe.c, I think the \d output should also exclude the "expressions" stats kind and just list the other kinds (or have no kinds l

Re: [PATCH] Simple progress reporting for COPY command

2021-01-06 Thread Tomas Vondra
right thing to do? Imagine you know the number of lines in a file. You can't really use (lines_processed / total_lines) to measure progress, because that may ignore many "skipped" rows. So maybe this should be changed to count all rows. OTOH we still have bytes_pr

Re: list of extended statistics on psql

2021-01-06 Thread Tomas Vondra
t won't work on 10 and 11, which does not support the "mcv" stats. I don't recall what exactly is our policy regarding new psql with older server versions, but it seems strange to check for 10.0 and then fail anyway for "supported" versions. regards -- Tomas Von

Re: list of extended statistics on psql

2021-01-06 Thread Tomas Vondra
quot;Dependencies_size". It looks like long names, but acceptable? Seems acceptable - I don't have a better idea. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH] Simple progress reporting for COPY command

2021-01-07 Thread Tomas Vondra
On 1/7/21 12:06 PM, Josef Šimánek wrote: st 6. 1. 2021 v 22:44 odesílatel Tomas Vondra napsal: On 1/5/21 11:02 AM, Josef Šimánek wrote: I'm attaching the whole patch since commitfest failed to ingest the last incremental on CI. Yeah, the whole patch needs to be attached fo

Re: list of extended statistics on psql

2021-01-07 Thread Tomas Vondra
On 1/7/21 3:47 PM, Alvaro Herrera wrote: On 2021-Jan-07, Tomas Vondra wrote: On 1/7/21 1:46 AM, Tatsuro Yamada wrote: I overlooked the check for MCV in the logic building query because I created the patch as a new feature on PG14. I'm not sure whether we should do back patch o

Re: [PATCH] Simple progress reporting for COPY command

2021-01-07 Thread Tomas Vondra
On 1/7/21 2:17 AM, Justin Pryzby wrote: On Wed, Jan 06, 2021 at 10:44:49PM +0100, Tomas Vondra wrote: On 1/5/21 11:02 AM, Josef Šimánek wrote: I'm attaching the whole patch since commitfest failed to ingest the last incremental on CI. Yeah, the whole patch needs to be attached fo

Re: [PATCH] Simple progress reporting for COPY command

2021-01-07 Thread Tomas Vondra
On 1/7/21 7:56 PM, Josef Šimánek wrote: čt 7. 1. 2021 v 19:51 odesílatel Matthias van de Meent napsal: On Wed, 6 Jan 2021 at 22:45, Tomas Vondra wrote: On 1/5/21 11:02 AM, Josef Šimánek wrote: I'm attaching the whole patch since commitfest failed to ingest the last incremental

Re: list of extended statistics on psql

2021-01-07 Thread Tomas Vondra
On 1/8/21 12:52 AM, Tatsuro Yamada wrote: Hi, On 2021/01/08 0:56, Tomas Vondra wrote: On 1/7/21 3:47 PM, Alvaro Herrera wrote: On 2021-Jan-07, Tomas Vondra wrote: On 1/7/21 1:46 AM, Tatsuro Yamada wrote: I overlooked the check for MCV in the logic building query because I created the

Re: list of extended statistics on psql

2021-01-08 Thread Tomas Vondra
On 1/8/21 1:14 AM, Tomas Vondra wrote: > > > On 1/8/21 12:52 AM, Tatsuro Yamada wrote: >> Hi, >> >> On 2021/01/08 0:56, Tomas Vondra wrote: >>> On 1/7/21 3:47 PM, Alvaro Herrera wrote: >>>> On 2021-Jan-07, Tomas Vondra wrote: >>>>

Re: PoC/WIP: Extended statistics on expressions

2021-01-08 Thread Tomas Vondra
On 1/8/21 3:35 AM, Justin Pryzby wrote: > On Fri, Jan 08, 2021 at 01:57:29AM +0100, Tomas Vondra wrote: >> Attached is a patch fixing most of the issues. There are a couple >> exceptions: > > In the docs: > > ... > Thanks! Checking the docs and comments is a tediou

Re: support for MERGE

2021-01-09 Thread Tomas Vondra
er branches depending on this. The 0002 patch fixes that by simply setting it to OK before the call, which makes the valgrind error go away. But maybe it should be fixed in a different way (e.g. by setting it at the beginning of table_tuple_delete). regards -- Tomas Vondra EnterpriseDB: http://

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2021-01-10 Thread Tomas Vondra
needs fixing to make ths patch RFC. Attached is the test script I'm using, and a v5 of the patch - rebased on current master, with some minor tweaks to comments etc. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company >From 6ccaa4f526b

Re: pg13.2: invalid memory alloc request size NNNN

2021-02-12 Thread Tomas Vondra
? The question is whether this is due to the VM getting killed in some strange way (what VM system is this, how is the storage mounted?) or whether the recovery is borked and failed to do the right thing. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: WIP: WAL prefetch (another approach)

2021-02-12 Thread Tomas Vondra
On 2/12/21 5:46 AM, Andres Freund wrote: Hi, On 2021-02-12 00:42:04 +0100, Tomas Vondra wrote: Yeah, that's a good point. I think it'd make sense to keep track of recent FPIs and skip prefetching such blocks. But how exactly should we implement that, how many blocks do we need to

Re: WIP: WAL prefetch (another approach)

2021-02-14 Thread Tomas Vondra
On 2/13/21 10:39 PM, Stephen Frost wrote: Greetings, * Andres Freund (and...@anarazel.de) wrote: On 2021-02-12 00:42:04 +0100, Tomas Vondra wrote: Yeah, that's a good point. I think it'd make sense to keep track of recent FPIs and skip prefetching such blocks. But how exactly

Re: Improvements and additions to COPY progress reporting

2021-02-15 Thread Tomas Vondra
the COPY progress patch. It's a bit buried, maybe it should be posted separately. OTOH it's pretty small. 3) 0003 - Some whitespace noise, triggering "git am" warnings. - Might be good to briefly explain what the regression test does with the triggers, e

Re: POC: postgres_fdw insert batching

2021-02-15 Thread Tomas Vondra
t to mention the relevant doc link: > >     > https://www.postgresql.org/docs/devel/fdw-callbacks.html#FDW-CALLBACKS-UPDATE > <https://www.postgresql.org/docs/devel/fdw-callbacks.html#FDW-CALLBACKS-UPDATE> > Thanks, I'll get this fixed. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: POC: postgres_fdw insert batching

2021-02-15 Thread Tomas Vondra
be doing is simply initializing the info only for CMD_INSERT operations. Which does the trick, but it affects everything, i.e. all updates, no? Not just cross-partition updates. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: POC: postgres_fdw insert batching

2021-02-16 Thread Tomas Vondra
On 2/16/21 10:25 AM, Amit Langote wrote: On Tue, Feb 16, 2021 at 1:36 AM Tomas Vondra wrote: On 2/5/21 3:52 AM, Amit Langote wrote: Tsunakwa-san, On Mon, Jan 25, 2021 at 1:21 PM tsunakawa.ta...@fujitsu.com wrote: From: Amit Langote Yes, it can be simplified by using a local join to

Re: PoC/WIP: Extended statistics on expressions

2021-02-16 Thread Tomas Vondra
On 1/27/21 12:02 PM, Dean Rasheed wrote: > On Fri, 22 Jan 2021 at 03:49, Tomas Vondra > wrote: >> >> Whooops. A fixed version attached. >> > > The change to pg_stats_ext_exprs isn't quite right, because now it > cross joins expressions and their stats, wh

Re: POC: postgres_fdw insert batching

2021-02-17 Thread Tomas Vondra
On 2/17/21 9:51 AM, Amit Langote wrote: > On Wed, Feb 17, 2021 at 5:46 PM Amit Langote wrote: >> On Wed, Feb 17, 2021 at 12:04 AM Tomas Vondra >> wrote: >>> On 2/16/21 10:25 AM, Amit Langote wrote: >>>> On Tue, Feb 16, 2021 at 1:36 AM Tomas Vondra >>>

Re: POC: postgres_fdw insert batching

2021-02-17 Thread Tomas Vondra
On 2/17/21 8:36 PM, Tomas Vondra wrote: > > ... > > Thanks. The patch seems reasonable, but it's a bit too large for a fix, > so I'll go ahead and push one of the previous fixes restricting batching > to plain INSERT commands. But this seems useful, so I suggest adding

Re: Improvements and additions to COPY progress reporting

2021-02-18 Thread Tomas Vondra
oint of writing the initial patchset. Would > "io_operations", "io_port", "operates_through" or "through" maybe be > better? > No idea. Let's see if someone has a better proposal ... regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PoC] Non-volatile WAL buffer

2021-02-18 Thread Tomas Vondra
even if we can live > without WAL, it is still not clear whether we really want to live > without it. > > One more idea: using multiword CAS approach  requires us to make changes > as editing sequences. > Such editing sequence is actually ready WAL records. So implementors of >

Re: [PATCH] Present all committed transaction to the output plugin

2021-02-21 Thread Tomas Vondra
plugin might indicate whether it's interested in empty transactions or not. If not, we can do what we do now. Otherwise the empty transactions would be passed to the output plugin. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Use extended statistics to estimate (Var op Var) clauses

2021-03-01 Thread Tomas Vondra
On 3/1/21 8:58 PM, Mark Dilger wrote: On Nov 12, 2020, at 5:14 PM, Tomas Vondra wrote: <0001-Support-estimation-of-clauses-of-the-form-V-20201113.patch> Your patch no longer applies. Can we get a new version please? I do not plan to work on this patch in the 2021-03 comm

Re: enable_incremental_sort changes query behavior

2021-03-01 Thread Tomas Vondra
supported by Google Cloud, maybe they haven't upgraded their PostgreSQL version yet. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: WIP: BRIN multi-range indexes

2021-03-02 Thread Tomas Vondra
On 3/3/21 12:44 AM, Alvaro Herrera wrote: > On 2021-Mar-03, Tomas Vondra wrote: > >> 1) The 0001 patch allows passing of all scan keys to BRIN opclasses, >> which is needed for the minmax-multi to work. But it also modifies the >> existing opclasses (minmax and inclusi

Re: WIP: BRIN multi-range indexes

2021-03-02 Thread Tomas Vondra
On 3/3/21 1:17 AM, Alvaro Herrera wrote: > On 2021-Mar-03, Tomas Vondra wrote: > >> That's kinda my point - I agree the size of the patch is not the >> primary concern, but it makes the minmax/inclusion code a bit more >> complicated (because they now have to loop

Re: Increase value of OUTER_VAR

2021-03-04 Thread Tomas Vondra
everyone. Switching to small negative values is a much better idea, but it's going to be more invasive - we'll have to offset the values in the bitmapsets, or we'll have to invent a new bitmapset variant that can store negative values directly (e.g. by keeping two separate bitmaps

Re: Increase value of OUTER_VAR

2021-03-04 Thread Tomas Vondra
On 3/4/21 4:16 PM, Tom Lane wrote: > Tomas Vondra writes: >> IMO just bumping up the constants from ~65k to 1M is a net loss, for >> most users. We add this to bitmapsets, which means we're using ~8kB with >> the current values, but this jumps to 128kB with this higher

Re: WIP: BRIN multi-range indexes

2021-03-04 Thread Tomas Vondra
On 3/4/21 5:30 PM, John Naylor wrote: > > On Tue, Mar 2, 2021 at 7:32 PM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: >> Ummm, in brin_minmax_multi_distance_timetz()? I don't think timetz can >> be infinite, no? I think brin_minmax_mul

Re: PoC/WIP: Extended statistics on expressions

2021-03-04 Thread Tomas Vondra
antage of using a pglist is that we can free it by calling > list_free_deep(Typ), rather than looping to free each of its elements. > But maybe for bootstrap.c it doesn't matter, and we can just write: > | Typ = NULL; /* Leak the old Typ array */ > Thanks. I'll switch this in the next version of the patch series. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

  1   2   3   4   5   6   7   8   9   10   >