Re: adding wait_start column to pg_locks

2021-02-15 Thread Fujii Masao
On 2021/02/15 15:17, Fujii Masao wrote: On 2021/02/10 10:43, Fujii Masao wrote: On 2021/02/09 23:31, torikoshia wrote: On 2021-02-09 22:54, Fujii Masao wrote: On 2021/02/09 19:11, Fujii Masao wrote: On 2021/02/09 18:13, Fujii Masao wrote: On 2021/02/09 17:48, torikoshia wrote: On

Re: A reloption for partitioned tables - parallel_workers

2021-02-15 Thread Amit Langote
Hi, On Tue, Feb 16, 2021 at 1:06 AM Laurenz Albe wrote: > On Mon, 2021-02-15 at 17:53 +0900, Amit Langote wrote: > > On Mon, Feb 15, 2021 at 5:28 PM Seamus Abshere wrote: > > > It turns out parallel_workers may be a useful reloption for certain uses > > > of partitioned tables, > > > at least

Re: [PoC] Non-volatile WAL buffer

2021-02-15 Thread Takashi Menjo
Hi, I made a new page at PostgreSQL Wiki to gather and summarize information and discussion about PMEM-backed WAL designs and implementations. Some parts of the page are TBD. I will continue to maintain the page. Requests are welcome. Persistent Memory for WAL

Re: 64-bit XIDs in deleted nbtree pages

2021-02-15 Thread Peter Geoghegan
On Mon, Feb 15, 2021 at 7:26 PM Peter Geoghegan wrote: > Actually, there is one more reason why I bring up idea 1 now: I want > to hear your thoughts on the index AM API questions now, which idea 1 > touches on. Ideally all of the details around the index AM VACUUM APIs > (i.e. when and where the

Re: ERROR: invalid spinlock number: 0

2021-02-15 Thread Michael Paquier
On Tue, Feb 16, 2021 at 12:43:42PM +0900, Fujii Masao wrote: > On 2021/02/16 6:28, Andres Freund wrote: >> So what? It's just about free to initialize a spinlock, whether it's >> using the fallback implementation or not. Initializing upon walsender >> startup adds a lot of complications, because

progress reporting for partitioned REINDEX

2021-02-15 Thread Justin Pryzby
It looks like we missed this in a6642b3ae. I think it's an odd behavior of pg_stat_progress_create_index to simultaneously show the global progress as well as the progress for the current partition ... It seems like for partitioned reindex, reindex_index() should set the AM, which is used in the

libpq PQresultErrorMessage vs PQerrorMessage API issue

2021-02-15 Thread Craig Ringer
Hi all This morning for the the umpteenth time I saw: some error message: [blank here] output from a libpq program. That's because passing a NULL PGresult to PQgetResultErrorMessage() returns "". But a NULL PGresult is a normal result from PQexec when it fails to submit a query due to an

Re: A reloption for partitioned tables - parallel_workers

2021-02-15 Thread Amit Langote
Hi Seamus, Please see my reply below. On Tue, Feb 16, 2021 at 1:35 AM Seamus Abshere wrote: > On Mon, Feb 15, 2021, at 3:53 AM, Amit Langote wrote: > > On Mon, Feb 15, 2021 at 5:28 PM Seamus Abshere wrote: > > > It turns out parallel_workers may be a useful reloption for certain uses > > > of

Re: Libpq support to connect to standby server as priority

2021-02-15 Thread Greg Nancarrow
On Fri, Feb 12, 2021 at 2:42 PM vignesh C wrote: > > Thanks, just one minor thing I missed in doc/src/sgml/libpq.sgml. > > > > +The support of read-write transactions is determined by the > > value of the > > +default_transaction_read_only and > > +in_hot_standby

RE: [POC] Fast COPY FROM command for the table with foreign partitions

2021-02-15 Thread tsunakawa.ta...@fujitsu.com
From: Amit Langote > Think I have mentioned upthread that this looks better as: > > if (rootResultRelInfo->ri_usesMultiInsert) > leaf_part_rri->ri_usesMultiInsert = ExecMultiInsertAllowed(leaf_part_rri); > > This keeps the logic confined to ExecInitPartitionInfo() where it > belongs. No

Re: repeated decoding of prepared transactions

2021-02-15 Thread Amit Kapila
On Thu, Feb 11, 2021 at 4:06 PM Amit Kapila wrote: > > On Mon, Feb 8, 2021 at 2:01 PM Markus Wanner > wrote: > > > Now, coming back to the restart case where the prepared transaction > can be sent again by the publisher. I understand yours and others > point that we should not send prepared

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-15 Thread David Rowley
On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley wrote: >> The reason I don't really like this is that it really depends where >> you want to use RelOptInfo.notnullattrs. If someone wants to use it >> to optimise something before the base quals are

Re: ERROR: invalid spinlock number: 0

2021-02-15 Thread Fujii Masao
On 2021/02/16 6:28, Andres Freund wrote: Hi, On 2021-02-15 19:45:21 +0900, Michael Paquier wrote: On Mon, Feb 15, 2021 at 10:47:05PM +1300, Thomas Munro wrote: Why not initialise it in WalRcvShmemInit()? I was thinking about doing that as well, but we have no real need to initialize this

Re: logical replication seems broken

2021-02-15 Thread Amit Kapila
On Mon, Feb 15, 2021 at 7:50 PM vignesh C wrote: > > On Mon, Feb 15, 2021 at 6:14 PM wrote: > > > > > > > On 2021.02.15. 12:31 Amit Kapila wrote: > > > On Mon, Feb 15, 2021 at 11:53 AM vignesh C wrote: > > > > On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > > > I compiled just now a

Re: 64-bit XIDs in deleted nbtree pages

2021-02-15 Thread Peter Geoghegan
On Mon, Feb 15, 2021 at 3:15 AM Masahiko Sawada wrote: > Yes. I think this would simplify the problem by resolving almost all > problems related to indefinite deferring page recycle. > > We will be able to recycle almost all just-deleted pages in practice > especially when btvacuumscan() took a

Re: [DOC] add missing "[ NO ]" to various "DEPENDS ON" synopses

2021-02-15 Thread Ian Lawrence Barwick
2021年2月16日(火) 10:20 Michael Paquier : > On Mon, Feb 15, 2021 at 03:57:04PM +0900, Ian Lawrence Barwick wrote: > > Indeed it does. Not the most exciting of use cases, though I imagine it > > might come in handy for anyone developing an extension, and the > > existing implementation is inconsistent

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-02-15 Thread Bharath Rupireddy
On Mon, Feb 15, 2021 at 8:13 AM Bharath Rupireddy wrote: > > On Sat, Feb 13, 2021 at 11:41 AM japin wrote: > > > IIUC, with the current patch, the new ALTER SUBSCRIPTION ... ADD/DROP > > > errors out on the first publication that already exists/that doesn't > > > exist right? What if there are

Re: proposal - psql - use pager for \watch command

2021-02-15 Thread Thomas Munro
On Fri, Jan 8, 2021 at 10:36 PM Pavel Stehule wrote: > ne 19. 4. 2020 v 19:27 odesílatel Pavel Stehule > napsal: >> last week I finished pspg 3.0 https://github.com/okbob/pspg . pspg now >> supports pipes, named pipes very well. Today the pspg can be used as pager >> for output of \watch

Re: PATCH: Batch/pipelining support for libpq

2021-02-15 Thread Craig Ringer
On Tue, 16 Feb 2021 at 09:19, Craig Ringer wrote: > > So long as there is a way to "send A", "send B", "send C", "read results > from A", "send D", and there's a way for the application to associate some > kind of state (an application specific id or index, a pointer to an > application

Re: Implementing Incremental View Maintenance

2021-02-15 Thread Yugo NAGATA
Hi, Attached is a rebased patch (v22a). Ragards, Yugo Nagata -- Yugo NAGATA IVM_patches_v22a.tar.gz Description: application/gzip

Re: [POC] verifying UTF-8 using SIMD instructions

2021-02-15 Thread John Naylor
On Mon, Feb 15, 2021 at 9:18 AM Heikki Linnakangas wrote: > Attached is the first attempt at using SSE4 to do the validation, but first I'll answer your questions about the fallback. I should mention that v2 had a correctness bug for 4-byte characters that I found when I was writing regression

Re: Snapshot scalability patch issue

2021-02-15 Thread Peter Geoghegan
On Mon, Feb 15, 2021 at 5:30 PM Andres Freund wrote: > Done. Thanks for noticing/reporting! Great, thanks! -- Peter Geoghegan

Re: Snapshot scalability patch issue

2021-02-15 Thread Andres Freund
Hi, On 2021-02-15 15:08:40 -0800, Andres Freund wrote: > On 2021-02-14 18:42:18 -0800, Peter Geoghegan wrote: > > The call to heap_page_prune() within lazy_scan_heap() passes a bool > > literal ('false') as its fourth argument. But the fourth argument is > > of type TransactionId, not bool. This

Fwd: Row description Metadata information

2021-02-15 Thread Aleksei Ivanov
Not sure that previous email was sent correctly. If it was duplicated, sorry for the inconvenience. Hi, hackers, I have one question related to returned information in the row description for prepared statement. For example Select $1 * 2 and then Bind 1.6 to it. The returned result is correct

Re: [DOC] add missing "[ NO ]" to various "DEPENDS ON" synopses

2021-02-15 Thread Michael Paquier
On Mon, Feb 15, 2021 at 03:57:04PM +0900, Ian Lawrence Barwick wrote: > Indeed it does. Not the most exciting of use cases, though I imagine it > might come in handy for anyone developing an extension, and the > existing implementation is inconsistent (in place for ALTER INDEX, > and partially for

Re: PATCH: Batch/pipelining support for libpq

2021-02-15 Thread Craig Ringer
On Thu, 11 Feb 2021 at 07:51, Alvaro Herrera wrote: > On 2021-Jan-21, Alvaro Herrera wrote: > > > As you can see in an XXX comment in the libpq test program, the current > > implementation has the behavior that PQgetResult() returns NULL after a > > batch is finished and has reported

Re: Snapbuild woes followup

2021-02-15 Thread Andres Freund
Hi, On 2021-01-29 14:04:47 +0530, Amit Kapila wrote: > On Tue, Jan 26, 2021 at 2:18 AM Andres Freund wrote: > > > > Hi, > > > > On 2021-01-25 12:00:08 -0800, Andres Freund wrote: > > > > > /* > > > > >* For backward compatibility reasons this has to be stored in the > > > > > wrongly > >

Re: pg_replication_origin_session_setup and superuser

2021-02-15 Thread Michael Paquier
On Mon, Feb 15, 2021 at 09:37:53AM +, Zohar Gofer wrote: > In my mind the requirement for superuser is too strong. I think that > requiring privileges of a replication user is more suitable. This > way we can require that only a user with replication privileges will > actually do replication,

RE: [HACKERS] logical decoding of two-phase transactions

2021-02-15 Thread osumi.takami...@fujitsu.com
Hi On Tuesday, February 16, 2021 8:33 AM Peter Smith > On Fri, Feb 12, 2021 at 5:59 PM osumi.takami...@fujitsu.com > wrote: > > (2) > > > > File : v39-0006-Support-2PC-txn-Subscription-option.patch > > > > @@ -213,6 +219,15 @@ parse_subscription_options(List *options, > >

Re: [HACKERS] logical decoding of two-phase transactions

2021-02-15 Thread Peter Smith
On Fri, Feb 12, 2021 at 5:59 PM osumi.takami...@fujitsu.com wrote: > (2) > > File : v39-0006-Support-2PC-txn-Subscription-option.patch > > @@ -213,6 +219,15 @@ parse_subscription_options(List *options, > *streaming_given = true; > *streaming =

Re: Snapshot scalability patch issue

2021-02-15 Thread Andres Freund
Hi, On 2021-02-14 18:42:18 -0800, Peter Geoghegan wrote: > The call to heap_page_prune() within lazy_scan_heap() passes a bool > literal ('false') as its fourth argument. But the fourth argument is > of type TransactionId, not bool. This has been the case since the > snapshot scalability work

Re: CREATE INDEX CONCURRENTLY on partitioned index

2021-02-15 Thread Zhihong Yu
Hi, For v13-0006-More-refactoring.patch : + /* It's not a shared catalog, so refuse to move it to shared tablespace */ + if (params->tablespaceOid == GLOBALTABLESPACE_OID && false) + ereport(ERROR, Do you intend to remove the ineffective check ? + else + heapRelation =

Re: ERROR: invalid spinlock number: 0

2021-02-15 Thread Andres Freund
Hi, On 2021-02-15 19:45:21 +0900, Michael Paquier wrote: > On Mon, Feb 15, 2021 at 10:47:05PM +1300, Thomas Munro wrote: > > Why not initialise it in WalRcvShmemInit()? > > I was thinking about doing that as well, but we have no real need to > initialize this stuff in most cases, say standalone

Re: partial heap only tuples

2021-02-15 Thread Bossart, Nathan
On 2/13/21, 8:26 AM, "Andres Freund" wrote: > On 2021-02-09 18:48:21 +, Bossart, Nathan wrote: >> In order to be eligible for cleanup, the final tuple in the >> corresponding PHOT/HOT chain must also be eligible for cleanup, or all >> indexes must have been updated later in the chain before

Re: PG vs LLVM 12 on seawasp, next round

2021-02-15 Thread Andrew Dunstan
On 1/24/21 11:06 PM, Tom Lane wrote: > Noah Misch writes: >> On Mon, Jan 18, 2021 at 09:29:53PM +0100, Fabien COELHO wrote: >>> ... Last two months were a >>> little overworked for me so I let slip quite a few things. If you want to >>> disable the animal as Tom suggests, do as you want. >>

Re: PG vs LLVM 12 on seawasp, next round

2021-02-15 Thread Fabien COELHO
I've added an explicit LD_LIBRARY_PATH, which will be triggered at some point later. This seems to have fixed the issue. I'm sorry for the noise and quite baffled anyway, because according to my change logs it does not seem that I modified anything from my side about the dynamic library

Re: CREATE INDEX CONCURRENTLY on partitioned index

2021-02-15 Thread Justin Pryzby
On Mon, Feb 15, 2021 at 10:06:47PM +0300, Anastasia Lubennikova wrote: > On 28.01.2021 17:30, Justin Pryzby wrote: > > On Thu, Jan 28, 2021 at 09:51:51PM +0900, Masahiko Sawada wrote: > > > On Mon, Nov 30, 2020 at 5:22 AM Justin Pryzby > > > wrote: > > > > On Sat, Oct 31, 2020 at 01:31:17AM

Re: SSL SNI

2021-02-15 Thread Peter Eisentraut
On 2021-02-15 18:40, Jesse Zhang wrote: I imagine this also (finally) opens up the possibility for the server to present a different certificate for each hostname based on SNI. This eliminates the requirement for wildcard certs where the cluster is running on a host with multiple (typically two

Re: CREATE INDEX CONCURRENTLY on partitioned index

2021-02-15 Thread Anastasia Lubennikova
On 28.01.2021 17:30, Justin Pryzby wrote: On Thu, Jan 28, 2021 at 09:51:51PM +0900, Masahiko Sawada wrote: On Mon, Nov 30, 2020 at 5:22 AM Justin Pryzby wrote: On Sat, Oct 31, 2020 at 01:31:17AM -0500, Justin Pryzby wrote: Forking this thread, since the existing CFs have been closed.

Re: CREATE INDEX CONCURRENTLY on partitioned index

2021-02-15 Thread Anastasia Lubennikova
On 28.01.2021 17:30, Justin Pryzby wrote: On Thu, Jan 28, 2021 at 09:51:51PM +0900, Masahiko Sawada wrote: On Mon, Nov 30, 2020 at 5:22 AM Justin Pryzby wrote: On Sat, Oct 31, 2020 at 01:31:17AM -0500, Justin Pryzby wrote: Forking this thread, since the existing CFs have been closed.

Re: PG vs LLVM 12 on seawasp, next round

2021-02-15 Thread Andres Freund
Hi, On 2021-02-15 10:05:32 +0100, Fabien COELHO wrote: > > show? What do you have in seawap's LD_LIBRARY_PATH, and is the > > problem that you need to add /home/fabien/clgtk/lib to it > > Argh. Would it be so stupid? :-( I thought the configuration stuff would > manage the link path

Re: SSL SNI

2021-02-15 Thread Jesse Zhang
Hi Peter, I imagine this also (finally) opens up the possibility for the server to present a different certificate for each hostname based on SNI. This eliminates the requirement for wildcard certs where the cluster is running on a host with multiple (typically two to three) hostnames and the

Re: MultiXact\SLRU buffers configuration

2021-02-15 Thread Andrey Borodin
> 23 дек. 2020 г., в 21:31, Gilles Darold написал(а): > > Sorry for the response delay, we have run several others tests trying to > figure out the performances gain per patch but unfortunately we have very > heratic results. With the same parameters and patches the test doesn't > returns

Re: Improve new hash partition bound check error messages

2021-02-15 Thread Peter Eisentraut
On 2021-02-03 15:52, Peter Eisentraut wrote: On 2021-02-02 13:26, Heikki Linnakangas wrote: How about this? CREATE TABLE fail_part PARTITION OF hash_parted FOR VALUES WITH (MODULUS 25, REMAINDER 3); ERROR: every hash partition modulus must be a factor of the next larger modulus DETAIL: 25 is

Re: POC: postgres_fdw insert batching

2021-02-15 Thread Tomas Vondra
On 2/5/21 3:52 AM, Amit Langote wrote: > Tsunakwa-san, > > On Mon, Jan 25, 2021 at 1:21 PM tsunakawa.ta...@fujitsu.com > wrote: >> From: Amit Langote >>> Yes, it can be simplified by using a local join to prevent the update of >>> the foreign >>> partition from being pushed to the remote

Re: A reloption for partitioned tables - parallel_workers

2021-02-15 Thread Seamus Abshere
hi, Here we go, my first patch... solves https://www.postgresql.org/message-id/7d6fdc20-857c-4cbe-ae2e-c0ff9520e...@www.fastmail.com Best, Seamus On Mon, Feb 15, 2021, at 3:53 AM, Amit Langote wrote: > Hi Seamus, > > On Mon, Feb 15, 2021 at 5:28 PM Seamus Abshere wrote: > > It turns out

Re: POC: postgres_fdw insert batching

2021-02-15 Thread Tomas Vondra
On 2/5/21 2:55 AM, Ian Lawrence Barwick wrote: > ... > > There's a minor typo in the doc's version of the > ExecForeignBatchInsert() declaration; > is: > >     TupleTableSlot ** >     ExecForeignBatchInsert(EState *estate, >                       ResultRelInfo *rinfo, >

Re: Improvements and additions to COPY progress reporting

2021-02-15 Thread Tomas Vondra
Hi, I agree with these changes in general - I have a couple minor comment: 1) 0001 - the SGML docs are missing a couple tags - The blocks in copyfrom.cc/copyto.c should be reworked - I don't think we do this in our codebase. Move the variable declarations to the beginning, get rid of the out

Re: A reloption for partitioned tables - parallel_workers

2021-02-15 Thread Laurenz Albe
On Mon, 2021-02-15 at 17:53 +0900, Amit Langote wrote: > On Mon, Feb 15, 2021 at 5:28 PM Seamus Abshere wrote: > > It turns out parallel_workers may be a useful reloption for certain uses of > > partitioned tables, > > at least if they're made up of fancy column store partitions (see > > > >

Re: PostgreSQL <-> Babelfish integration

2021-02-15 Thread Finnerty, Jim
We are applying the Babelfish commits to the REL_12_STABLE branch now, and the plan is to merge them into the REL_13_STABLE and master branch ASAP after that. There should be a publicly downloadable git repository before very long. On 2/12/21, 2:35 PM, "Peter Geoghegan" wrote: CAUTION:

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-02-15 Thread Joel Jacobson
Hi all, I've reviewed Mark's anyarray_anyelement_operators-v2.patch and the only remaining issue I've identified is the opr_sanity problem. Mark seems to be in need of some input here from more experienced hackers, see below. Hopefully someone can guide him in the right direction. /Joel On

Re: logical replication seems broken

2021-02-15 Thread vignesh C
On Mon, Feb 15, 2021 at 6:14 PM wrote: > > > > On 2021.02.15. 12:31 Amit Kapila wrote: > > On Mon, Feb 15, 2021 at 11:53 AM vignesh C wrote: > > > On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > > I compiled just now a binary from HEAD, and a binary from HEAD+patch > > > > HEAD is

SSL SNI

2021-02-15 Thread Peter Eisentraut
A customer asked about including Server Name Indication (SNI) into the SSL connection from the client, so they can use an SSL-aware proxy to route connections. There was a thread a few years ago where this was briefly discussed but no patch appeared.[0] I whipped up a quick patch and it did

Re: Possible dereference after null check (src/backend/executor/ExecUtils.c)

2021-02-15 Thread Ranier Vilela
Em sex., 12 de fev. de 2021 às 13:11, Ranier Vilela escreveu: > Em sex., 12 de fev. de 2021 às 03:28, Kyotaro Horiguchi < > horikyota@gmail.com> escreveu: > >> At Wed, 10 Feb 2021 19:54:46 -0300, Ranier Vilela >> wrote in >> > Hi, >> > >> > Per Coverity. >> > >> > The functions

Re: pg_cryptohash_final possible out-of-bounds access (per Coverity)

2021-02-15 Thread Ranier Vilela
Em dom., 14 de fev. de 2021 às 22:28, Michael Paquier escreveu: > On Sun, Feb 14, 2021 at 11:39:47AM -0300, Ranier Vilela wrote: > > What do you think? > > That's not a good idea for two reasons: > 1) There is CRC32 to worry about, which relies on a different logic. > 2) It would become easier

Re: a misbehavior of partition row movement (?)

2021-02-15 Thread Amit Langote
Thank you for looking at this. On Mon, Feb 15, 2021 at 4:12 PM Masahiko Sawada wrote: > On Wed, Jan 20, 2021 at 7:04 PM Amit Langote wrote: > > This shows that the way we've made these triggers behave in general > > can cause some unintended behaviors for foreign keys during > > cross-partition

Re: snowball update

2021-02-15 Thread Peter Eisentraut
On 2021-02-12 22:00, Oleg Bartunov wrote: We don't have (and really it's impossible) regression test for stemmers, so maybe we should warn users about possible inconsistencies of old tsvectors and new stemmers ? Yeah, it's analogous to collation and Unicode updates. We could invent a

Re: [POC] verifying UTF-8 using SIMD instructions

2021-02-15 Thread Heikki Linnakangas
On 13/02/2021 03:31, John Naylor wrote: On Mon, Feb 8, 2021 at 6:17 AM Heikki Linnakangas > wrote: > > I also tested the fallback implementation from the simdjson library > (included in the patch, if you uncomment it in simdjson-glue.c): > >   mixed | ascii >

Re: Online checksums patch - once again

2021-02-15 Thread Daniel Gustafsson
> On 11 Feb 2021, at 14:10, Bruce Momjian wrote: > > On Wed, Feb 10, 2021 at 01:26:18PM -0500, Bruce Momjian wrote: >> On Wed, Feb 10, 2021 at 03:25:58PM +0100, Magnus Hagander wrote: >>> A fairly large amount of this complexity comes out of the fact that it >>> now supports restarting and

Re: logical replication seems broken

2021-02-15 Thread er
> On 2021.02.15. 12:31 Amit Kapila wrote: > On Mon, Feb 15, 2021 at 11:53 AM vignesh C wrote: > > On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > I compiled just now a binary from HEAD, and a binary from HEAD+patch > > > HEAD is still broken; your patch rescues it, so yes, fixed. >

Re: logical replication seems broken

2021-02-15 Thread vignesh C
On Mon, Feb 15, 2021 at 5:02 PM Amit Kapila wrote: > > On Mon, Feb 15, 2021 at 11:53 AM vignesh C wrote: > > > > On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > > > > > > > I compiled just now a binary from HEAD, and a binary from HEAD+patch > > > > > > HEAD is still broken; your

pg_replication_origin_session_setup and superuser

2021-02-15 Thread Zohar Gofer
Hi, Problem description: While working on a homegrown limited solution to replace (a very limited set of) golden gate capabilities we have created a CDC solution using the WAL capabilities. The data flows like this: PG1 --> Debezium(wal2json) --> Kafka1 --> MM2 --> Kafka2 --> Kafka Connect

Re: ERROR: invalid spinlock number: 0

2021-02-15 Thread Fujii Masao
On 2021/02/15 19:45, Michael Paquier wrote: On Mon, Feb 15, 2021 at 10:47:05PM +1300, Thomas Munro wrote: Why not initialise it in WalRcvShmemInit()? I was thinking about doing that as well, but we have no real need to initialize this stuff in most cases, say standalone deployments. In

Re: logical replication seems broken

2021-02-15 Thread Amit Kapila
On Mon, Feb 15, 2021 at 11:53 AM vignesh C wrote: > > On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > > > > I compiled just now a binary from HEAD, and a binary from HEAD+patch > > > > HEAD is still broken; your patch rescues it, so yes, fixed. > > > > Maybe a test (check or

Re: Refactoring HMAC in the core code

2021-02-15 Thread Michael Paquier
On Sat, Jan 23, 2021 at 01:43:20PM +0900, Michael Paquier wrote: > Rebased patch is attached wiht SHA1 added as of a8ed6bb. Now that > SHA1 is part of the set of options for cryptohashes, a lot of code of > pgcrypto can be cleaned up thanks to the refactoring done here, but > I am leaving that as

Re: 64-bit XIDs in deleted nbtree pages

2021-02-15 Thread Masahiko Sawada
On Sun, Feb 14, 2021 at 3:47 PM Peter Geoghegan wrote: > > On Fri, Feb 12, 2021 at 9:04 PM Peter Geoghegan wrote: > > On Fri, Feb 12, 2021 at 8:38 PM Masahiko Sawada > > wrote: > > > I agree that there already are huge problems in that case. But I think > > > we need to consider an append-only

Re: ERROR: invalid spinlock number: 0

2021-02-15 Thread Michael Paquier
On Mon, Feb 15, 2021 at 10:47:05PM +1300, Thomas Munro wrote: > Why not initialise it in WalRcvShmemInit()? I was thinking about doing that as well, but we have no real need to initialize this stuff in most cases, say standalone deployments. In particular for the fallback implementation of

Re: increase size of pg_commit_ts buffers

2021-02-15 Thread Noah Misch
On Fri, Jan 15, 2021 at 07:07:44PM -0300, Alvaro Herrera wrote: > I wrote this patch last year in response to a customer issue and I > thought I had submitted it here, but evidently I didn't. So here it is. > > The short story is: in commit 5364b357fb11 we increased the size of > pg_commit (née

Re: ERROR: invalid spinlock number: 0

2021-02-15 Thread Thomas Munro
On Mon, Feb 15, 2021 at 9:27 PM Michael Paquier wrote: > On Thu, Feb 11, 2021 at 11:30:13PM +0900, Fujii Masao wrote: > > Yes, so what about the attached patch? > > I see. So the first error triggering the spinlock error would cause > a transaction failure because the fallback implementation of

Re: PG vs LLVM 12 on seawasp, next round

2021-02-15 Thread Fabien COELHO
Hello Thomas, Thanks for looking at this, I'm currently far behind on many things and not very responsive:-( Here is the creation of llvmjit.so: g++ ... -o llvmjit.so ... -L/home/fabien/clgtk/lib ... -lLLVMOrcJIT ... That'd be from llvm-config --ldflags or similar, from this binary:

Re: A reloption for partitioned tables - parallel_workers

2021-02-15 Thread Amit Langote
Hi Seamus, On Mon, Feb 15, 2021 at 5:28 PM Seamus Abshere wrote: > It turns out parallel_workers may be a useful reloption for certain uses of > partitioned tables, at least if they're made up of fancy column store > partitions (see >

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2021-02-15 Thread Amit Langote
Tsunakawa-san, Andrey, On Mon, Feb 15, 2021 at 1:54 PM tsunakawa.ta...@fujitsu.com wrote: > From: Justin Pryzby > > This is crashing during fdw check. > > http://cfbot.cputube.org/andrey-lepikhov.html > > > > Maybe it's related to this patch: > > |commit 6214e2b2280462cbc3aa1986e350e167651b3905

A reloption for partitioned tables - parallel_workers

2021-02-15 Thread Seamus Abshere
hi, It turns out parallel_workers may be a useful reloption for certain uses of partitioned tables, at least if they're made up of fancy column store partitions (see https://www.postgresql.org/message-id/7d6fdc20-857c-4cbe-ae2e-c0ff9520ed55%40www.fastmail.com). Would somebody tell me what I'm

Re: ERROR: invalid spinlock number: 0

2021-02-15 Thread Michael Paquier
On Thu, Feb 11, 2021 at 11:30:13PM +0900, Fujii Masao wrote: > Yes, so what about the attached patch? I see. So the first error triggering the spinlock error would cause a transaction failure because the fallback implementation of atomics uses a spinlock for this variable, and it may not

Re: Some regular-expression performance hacking

2021-02-15 Thread Joel Jacobson
On Mon, Feb 15, 2021, at 04:11, Tom Lane wrote: >I got these runtimes (non-cassert builds): > >HEAD 313661.149 ms (05:13.661) >+0001 297397.293 ms (04:57.397) 5% better than HEAD >+0002 151995.803 ms (02:31.996) 51% better than HEAD >+0003 139843.934 ms (02:19.844) 55% better than HEAD >+0004