Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-08 Thread Amit Kapila
On Fri, Jan 8, 2021 at 12:21 PM Antonin Houska wrote: > > Amit Kapila wrote: > > > > As an alternative, have you considered allocation of the XID even in > > > parallel > > > mode? I imagine that the first parallel worker that needs the XID for > > > insertions allocates it and shares it with

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-08 Thread Laurenz Albe
On Thu, 2021-01-07 at 16:14 -0500, Stephen Frost wrote: > I expected there'd be some disagreement on this, but I do continue to > feel that it's sensible to enable checksums by default. +1 I think the problem here (apart from the original line of argumentation) is that there are two kinds of

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-08 Thread Andres Freund
Hi, On Fri, Jan 8, 2021, at 01:53, Laurenz Albe wrote: > On Thu, 2021-01-07 at 16:14 -0500, Stephen Frost wrote: > > I expected there'd be some disagreement on this, but I do continue to > > feel that it's sensible to enable checksums by default. > > +1 I don't disagree with this in principle,

Re: Global snapshots

2021-01-08 Thread Fujii Masao
On 2021/01/01 12:14, tsunakawa.ta...@fujitsu.com wrote: Hello, Fujii-san and I discussed how to move the scale-out development forward. We are both worried that Clock-SI is (highly?) likely to infringe the said Microsoft's patent. So we agreed we are going to investigate the Clock-SI

Re: Disable WAL logging to speed up data loading

2021-01-08 Thread Masahiko Sawada
On Fri, Jan 8, 2021 at 2:12 PM tsunakawa.ta...@fujitsu.com wrote: > > From: Robert Haas > > Were the issues that I mentioned regarding GIST (and maybe other AMs) > > in the last paragraph of > > http://postgr.es/m/CA+TgmoZEZ5RONS49C7mEpjhjndqMQtVrz_LCQUkpRW > > dmrev...@mail.gmail.com > >

Re: Added schema level support for publication.

2021-01-08 Thread Amit Kapila
On Thu, Jan 7, 2021 at 10:03 PM vignesh C wrote: > > This feature adds schema option while creating publication. Users will > be able to specify one or more schemas while creating publication, > when the user specifies schema option, then the data changes for the > tables present in the schema

Re: Add session statistics to pg_stat_database

2021-01-08 Thread Masahiro Ikeda
On 2021-01-08 18:34, Laurenz Albe wrote: On Fri, 2021-01-08 at 12:00 +0900, Masahiro Ikeda wrote: 2. monitoring.sgml > > IIUC, "active_time" includes the time executes a fast-path function > > and > > "idle in transaction" includes "idle in transaction(aborted)" time. > > Why don't you

Re: [PATCH] Simple progress reporting for COPY command

2021-01-08 Thread Matthias van de Meent
On Thu, 7 Jan 2021 at 23:00, Josef Šimánek wrote: > > čt 7. 1. 2021 v 22:37 odesílatel Tomas Vondra > napsal: > > > > I'm not particularly attached to the "lines" naming, it just seemed OK > > to me. So if there's consensus to rename this somehow, I'm OK with it. > > The problem I do see here is

Re: WIP: System Versioned Temporal Table

2021-01-08 Thread Andrew Dunstan
On 1/8/21 7:33 AM, Simon Riggs wrote: > > * What happens if you ask for a future time? > It will give an inconsistent result as it scans, so we should refuse a > query for time > current_timestamp. That seems like a significant limitation. Can we fix it instead of refusing the query? cheers

Re: plpgsql variable assignment with union is broken

2021-01-08 Thread Merlin Moncure
On Thu, Jan 7, 2021 at 10:55 AM Pavel Stehule wrote: >> Again, if this is narrowly confined to assignment into set query >> operations, maybe this is not so bad. But is it? > > PLpgSQL_Expr: opt_target_list > <--><--><-->from_clause where_clause > <--><--><-->group_clause having_clause

Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW

2021-01-08 Thread japin
On Thu, 07 Jan 2021 at 17:53, Bharath Rupireddy wrote: > On Mon, Dec 28, 2020 at 5:56 PM Bharath Rupireddy > wrote: >> >> On Tue, Dec 22, 2020 at 7:01 PM Bharath Rupireddy >> wrote: >> > Currently, $subject is not allowed. We do plan the mat view query >> > before every refresh. I propose to

Re: Add session statistics to pg_stat_database

2021-01-08 Thread Laurenz Albe
On Fri, 2021-01-08 at 12:00 +0900, Masahiro Ikeda wrote: > 2. monitoring.sgml > > > > IIUC, "active_time" includes the time executes a fast-path function > > > and > > > "idle in transaction" includes "idle in transaction(aborted)" time. > > > Why don't you reference pg_stat_activity's "state"

Re: a misbehavior of partition row movement (?)

2021-01-08 Thread Amit Langote
Hi, On Mon, Dec 28, 2020 at 10:01 PM Arne Roland wrote: > While I'd agree that simply deleting with "on delete cascade" on tuple > rerouting is a strong enough violation of the spirit of partitioning changing > that behavior, I am surprised by the idea to make a differentiation between > fks

Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW

2021-01-08 Thread Bharath Rupireddy
On Fri, Jan 8, 2021 at 1:50 PM japin wrote: > Thanks for updating the patch! > > + /* Get the data generating query. */ > + dataQuery = get_matview_query(stmt, , ); > > - /* > -* Check for active uses of the relation in the current transaction, > such > -* as

Re: Proposal: Global Index

2021-01-08 Thread Julien Rouhaud
On Fri, Jan 8, 2021 at 4:02 PM 曾文旌 wrote: > > > 2021年1月7日 22:16,Bruce Momjian 写道: > > > > On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: > >> I've been following this topic for a long time. It's been a year since the > >> last response. > >> It was clear that our customers wanted this

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-08 Thread vignesh C
On Fri, Dec 11, 2020 at 4:30 PM Greg Nancarrow wrote: > > Posting an updated set of patches to address recent feedback: > > - Removed conditional-locking code used in parallel-safety checking > code (Tsunakawa-san feedback). It turns out that for the problem test > case, no parallel-safety

Re: Single transaction in the tablesync worker?

2021-01-08 Thread Peter Smith
On Fri, Jan 8, 2021 at 1:02 PM Hou, Zhijie wrote: > > > PSA the v12 patch for the Tablesync Solution1. > > > > Differences from v11: > > + Added PG docs to mention the tablesync slot > > + Refactored tablesync slot drop (done by > > DropSubscription/process_syncing_tables_for_sync) > > +

Re: Proposal: Global Index

2021-01-08 Thread 曾文旌
> 2021年1月7日 22:16,Bruce Momjian 写道: > > On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: >> I've been following this topic for a long time. It's been a year since the >> last response. >> It was clear that our customers wanted this feature as well, and a large >> number of them mentioned

Re: In-placre persistance change of a relation

2021-01-08 Thread Kyotaro Horiguchi
At Fri, 08 Jan 2021 14:47:05 +0900 (JST), Kyotaro Horiguchi wrote in > This version RelationChangePersistence() is changed not to choose > in-place method for indexes other than btree. It seems to be usable > with all kind of indexes other than Gist, but at the mement it applies > only to

Re: Improper use about DatumGetInt32

2021-01-08 Thread Peter Eisentraut
On 2020-12-25 08:45, Michael Paquier wrote: If we really think that we ought to differentiate, then we could do what pg_stat_statement does, and have a separate C function that's called with the obsolete signature (pg_stat_statements_1_8 et al). With the 1.8 flavor, it is possible to pass down

Re: Why latestRemovedXid|cuteoff_xid are always sent?

2021-01-08 Thread Michail Nikolaev
Hello, Peter. Thanks for your explanation. One of the reasons I was asking - is an idea to use the same technique in the "LP_DEAD index hint bits on standby" WIP patch to reduce the amount of additional WAL. Now I am sure such optimization should work correctly. Thanks, Michail.

Re: proposal: enhancing plpgsql debug API - returns text value of variable content

2021-01-08 Thread Pavel Stehule
Hi rebase Regards Pavel diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c index 3a9349b724..208ba181fc 100644 --- a/src/pl/plpgsql/src/pl_exec.c +++ b/src/pl/plpgsql/src/pl_exec.c @@ -4123,6 +4123,8 @@ plpgsql_estate_setup(PLpgSQL_execstate *estate, {

Re: Single transaction in the tablesync worker?

2021-01-08 Thread Peter Smith
Hi Amit. PSA the v13 patch for the Tablesync Solution1. Differences from v12: + Fixed whitespace errors of v12-0001 + Modify TCOPYDONE state comment (houzj feedback) + WIP fix for AlterSubscripion_refresh (Amit feedback) Features: * The tablesync slot is now permanent instead of

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-08 Thread Amit Kapila
On Wed, Jan 6, 2021 at 2:09 PM Antonin Houska wrote: > > Greg Nancarrow wrote: > > > Posting an updated set of patches to address recent feedback: > > Following is my review. > .. > > v11-0003-Enable-parallel-INSERT-and-or-SELECT-for-INSERT-INTO.patch >

Re: proposal - psql - use pager for \watch command

2021-01-08 Thread Pavel Stehule
ne 19. 4. 2020 v 19:27 odesílatel Pavel Stehule napsal: > Hi, > > last week I finished pspg 3.0 https://github.com/okbob/pspg . pspg now > supports pipes, named pipes very well. Today the pspg can be used as pager > for output of \watch command. Sure, psql needs attached patch. > > I propose new

Re: proposal: schema variables

2021-01-08 Thread Erik Rijkers
On 2021-01-08 07:20, Pavel Stehule wrote: Hi just rebase [schema-variables-20200108.patch] Hey Pavel, My gcc 8.3.0 compile says: (on debian 10/Buster) utility.c: In function ‘CreateCommandTag’: utility.c:2332:8: warning: this statement may fall through [-Wimplicit-fallthrough=] tag =

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-01-08 Thread Alvaro Herrera
On 2020-Dec-01, Alvaro Herrera wrote: > On 2020-Nov-30, Justin Pryzby wrote: > Thanks for all the comments. I'll incorporate everything and submit an > updated version later. Here's a rebased version 5, with the typos fixed. More comments below. > > The attname "detached" is a stretch of

Re: pgbench: option delaying queries till connections establishment?

2021-01-08 Thread Thomas Munro
On Sun, Jan 3, 2021 at 9:49 AM Fabien COELHO wrote: > > Just for fun, the attached 0002 patch is a quick prototype of a > > replacement for that stuff that seems to work OK on a Mac here. (I'm > > not sure if the Windows part makes sense or works.) > > Thanks! That will definitely help because I

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-08 Thread David Steele
On 1/8/21 5:03 AM, Andres Freund wrote: On Fri, Jan 8, 2021, at 01:53, Laurenz Albe wrote: The serious crowd are more likely to choose a non-default setting to avoid paying the price for a feature that they don't need. I don't really buy this argument. That way we're going to have an ever

Re: Proposal: Global Index

2021-01-08 Thread Bruce Momjian
On Fri, Jan 8, 2021 at 11:26:48AM +0800, 曾文旌 wrote: > > On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: > >> I've been following this topic for a long time. It's been a year since the > >> last response. > >> It was clear that our customers wanted this feature as well, and a large > >>

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-08 Thread Bruce Momjian
On Fri, Jan 8, 2021 at 10:53:35AM +0100, Laurenz Albe wrote: > On Thu, 2021-01-07 at 16:14 -0500, Stephen Frost wrote: > > I expected there'd be some disagreement on this, but I do continue to > > feel that it's sensible to enable checksums by default. > > +1 > > I think the problem here (apart

Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW

2021-01-08 Thread japin
On Fri, 08 Jan 2021 at 17:24, Bharath Rupireddy wrote: > On Fri, Jan 8, 2021 at 1:50 PM japin wrote: >> Thanks for updating the patch! >> >> + /* Get the data generating query. */ >> + dataQuery = get_matview_query(stmt, , ); >> >> - /* >> -* Check for active uses of

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-08 Thread Bruce Momjian
On Fri, Jan 8, 2021 at 09:46:58AM -0500, David Steele wrote: > On 1/8/21 5:03 AM, Andres Freund wrote: > > On Fri, Jan 8, 2021, at 01:53, Laurenz Albe wrote: > > > > > > The serious crowd are more likely to choose a non-default setting > > > to avoid paying the price for a feature that they

Re: WIP: System Versioned Temporal Table

2021-01-08 Thread Ryan Lambert
On Fri, Jan 8, 2021 at 5:34 AM Simon Riggs wrote: > On Fri, Jan 8, 2021 at 7:34 AM Simon Riggs > wrote: > > > > On Fri, Jan 8, 2021 at 7:13 AM Simon Riggs > wrote: > > > > > I've minimally rebased the patch to current head so that it compiles > > > and passes current make check. > > > > Full

Re: Improper use about DatumGetInt32

2021-01-08 Thread Peter Eisentraut
On 2021-01-08 10:21, Peter Eisentraut wrote: I think on 64-bit systems it's actually safe, but on 32-bit systems (with USE_FLOAT8_BYVAL), if you use the new binaries with the old SQL-level definitions, you'd get the int4 that is passed in interpreted as a pointer, which would lead to very bad

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-08 Thread Bruce Momjian
On Fri, Jan 8, 2021 at 05:06:24PM +0100, Magnus Hagander wrote: > On Fri, Jan 8, 2021 at 4:57 PM Bruce Momjian wrote: > > I think once we have better online enabling of checksums people can more > > easily test the overhead on their workloads. > > Yeah, definitely. > > If they have equivalent

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-08 Thread Magnus Hagander
On Fri, Jan 8, 2021 at 4:57 PM Bruce Momjian wrote: > > On Fri, Jan 8, 2021 at 09:46:58AM -0500, David Steele wrote: > > On 1/8/21 5:03 AM, Andres Freund wrote: > > > On Fri, Jan 8, 2021, at 01:53, Laurenz Albe wrote: > > > > > > > > The serious crowd are more likely to choose a non-default

Re: Key management with tests

2021-01-08 Thread Bruce Momjian
On Fri, Jan 8, 2021 at 03:33:44PM -0500, Stephen Frost wrote: > > No, I don't think so. Stephen imported the entire NIST test suite. It > > was so comperhensive, it detected several OpenSSL bugs for zero-length > > strings, which I already reported, but we would never be encrypting > >

Re: Key management with tests

2021-01-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Fri, Jan 8, 2021 at 03:33:44PM -0500, Stephen Frost wrote: > > > Anyway, I think we need to figure out how to trim. The first part would > > > be to figure out whether we need 128 _and_ 256-bit tests, and then see > > > what items are

Re: Enhance traceability of wal_level changes for backup management

2021-01-08 Thread Stephen Frost
Greetings, * osumi.takami...@fujitsu.com (osumi.takami...@fujitsu.com) wrote: > On Thursday, January 7, 2021 2:40 AM Stephen Frost wrote: > > * osumi.takami...@fujitsu.com (osumi.takami...@fujitsu.com) wrote: > > > You said > > > > The use case I imagined is that the user temporarily changes > >

Re: Key management with tests

2021-01-08 Thread Bruce Momjian
On Fri, Jan 8, 2021 at 03:34:23PM -0500, Stephen Frost wrote: > > All the tests pass now. The current src/test directory is 19MB, and > > adding these tests takes it to 23MB, or a 20% increase. That seems like > > a lot. It is testing 128-bit and 256-bit keys --- should we do fewer > > tests,

Re: WIP: System Versioned Temporal Table

2021-01-08 Thread Ryan Lambert
On Fri, Jan 8, 2021 at 11:38 AM Simon Riggs wrote: > On Fri, Jan 8, 2021 at 4:50 PM Ryan Lambert > wrote: > > > > On Fri, Jan 8, 2021 at 5:34 AM Simon Riggs > wrote: > >> > >> On Fri, Jan 8, 2021 at 7:34 AM Simon Riggs < > simon.ri...@enterprisedb.com> wrote: > >> > > >> > On Fri, Jan 8, 2021

Re: Key management with tests

2021-01-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Jan 7, 2021 at 04:08:49PM -0300, Álvaro Herrera wrote: > > On 2021-Jan-07, Bruce Momjian wrote: > > > > > All the tests pass now. The current src/test directory is 19MB, and > > > adding these tests takes it to 23MB, or a 20%

Re: A failure of standby to follow timeline switch

2021-01-08 Thread Alvaro Herrera
Masao-san: Are you intending to act as committer for these? Since the bug is mine I can look into it, but since you already did all the reviewing work, I'm good with you giving it the final push. 0001 looks good to me; let's get that one committed quickly so that we can focus on the interesting

Re: Key management with tests

2021-01-08 Thread Stephen Frost
Greetings Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Fri, Jan 1, 2021 at 01:07:50AM -0500, Bruce Momjian wrote: > > On Thu, Dec 31, 2020 at 11:50:47PM -0500, Bruce Momjian wrote: > > > I have completed the key management patch with tests created by Stephen > > > Frost. Original patch

Re: PoC/WIP: Extended statistics on expressions

2021-01-08 Thread Tomas Vondra
On 1/8/21 3:35 AM, Justin Pryzby wrote: > On Fri, Jan 08, 2021 at 01:57:29AM +0100, Tomas Vondra wrote: >> Attached is a patch fixing most of the issues. There are a couple >> exceptions: > > In the docs: > > ... > Thanks! Checking the docs and comments is a tedious work, I appreciate you going

Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW

2021-01-08 Thread japin
On Sat, 09 Jan 2021 at 09:38, Bharath Rupireddy wrote: > On Fri, Jan 8, 2021 at 9:50 PM japin wrote: >> > Attaching v3 patches, please consider these for further review. >> > >> >> I find that both the declaration and definition of >> match_matview_with_new_data() >> have a tab between type

Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW

2021-01-08 Thread Bharath Rupireddy
On Fri, Jan 8, 2021 at 9:50 PM japin wrote: > > Attaching v3 patches, please consider these for further review. > > > > I find that both the declaration and definition of > match_matview_with_new_data() > have a tab between type and variable. We can use pgindent to fix it. > What do you think?

Re: Improper use about DatumGetInt32

2021-01-08 Thread Michael Paquier
On Fri, Jan 08, 2021 at 04:54:47PM +0100, Peter Eisentraut wrote: > Updated patch that does that. Thanks. Looks sane seen from here. +/* LCOV_EXCL_START */ Does it really make sense to add those markers here? It seems to me that we would ignore any new coverage if regression tests based on

Re: list of extended statistics on psql

2021-01-08 Thread Tomas Vondra
On 1/8/21 1:14 AM, Tomas Vondra wrote: > > > On 1/8/21 12:52 AM, Tatsuro Yamada wrote: >> Hi, >> >> On 2021/01/08 0:56, Tomas Vondra wrote: >>> On 1/7/21 3:47 PM, Alvaro Herrera wrote: On 2021-Jan-07, Tomas Vondra wrote: > On 1/7/21 1:46 AM, Tatsuro Yamada wrote: >> I

Re: Add table access method as an option to pgbench

2021-01-08 Thread David Zhang
tablespace is an extraneous word ? Thanks a lot for pointing this out. I will fix it in next patch once get all issues clarified. On Sun, Dec 27, 2020 at 09:14:53AM -0400, Fabien COELHO wrote: src/test/regress/sql/create_am.sql:CREATE ACCESS METHOD heap2 TYPE TABLE HANDLER

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-08 Thread Bharath Rupireddy
On Fri, Jan 8, 2021 at 9:55 AM Bharath Rupireddy wrote: > I will make the changes and post a new patch set soon. Attaching v9 patch set that has addressed the review comments on the disconnect function returning setof records, documentation changes, and postgres_fdw--1.0-1.1.sql changes. Please

Re: [PATCH] Simple progress reporting for COPY command

2021-01-08 Thread Bharath Rupireddy
On Fri, Jan 8, 2021 at 7:00 PM Matthias van de Meent wrote: > On Thu, 7 Jan 2021 at 23:00, Josef Šimánek wrote: > > > > čt 7. 1. 2021 v 22:37 odesílatel Tomas Vondra > > napsal: > > > > > > I'm not particularly attached to the "lines" naming, it just seemed OK > > > to me. So if there's

Re: Single transaction in the tablesync worker?

2021-01-08 Thread Amit Kapila
On Fri, Jan 8, 2021 at 2:55 PM Peter Smith wrote: > > On Fri, Jan 8, 2021 at 1:02 PM Hou, Zhijie wrote: > > > > > 3. > > + /* > > +* To build a slot name for the sync work, we are limited to > > NAMEDATALEN - > > +* 1 characters. > > +* > > +* The name is

Re: logical replication worker accesses catalogs in error context callback

2021-01-08 Thread Bharath Rupireddy
On Thu, Jan 7, 2021 at 5:47 AM Masahiko Sawada wrote: > > On Wed, Jan 6, 2021 at 11:42 AM Masahiko Sawada wrote: > > > > On Wed, Jan 6, 2021 at 11:02 AM Andres Freund wrote: > > > > > > Hi, > > > > > > Due to a debug ereport I just noticed that worker.c's > > > slot_store_error_callback is

Re: [HACKERS] logical decoding of two-phase transactions

2021-01-08 Thread Amit Kapila
On Tue, Jan 5, 2021 at 4:26 PM Amit Kapila wrote: > > On Tue, Jan 5, 2021 at 2:11 PM Ajin Cherian wrote: > > > > > > I've addressed the above comments and the patch is attached. I've > > called it v36-0007. > > > > Thanks, I have pushed this after minor wordsmithing. > The test case is failing