Re: Commit fest manager for 2021-01

2020-12-24 Thread Michael Paquier
On Fri, Dec 25, 2020 at 04:35:30PM +0900, Masahiko Sawada wrote: > Hmm, on the left of the logout button, I can see only the 'edit > profile' button and 'Activity log' button. Maybe that's a cache issue with your browser? Magnus, any ideas? I cannot control the permissions of the app, but if

Re: Improper use about DatumGetInt32

2020-12-24 Thread Michael Paquier
On Fri, Dec 04, 2020 at 03:58:22PM -0300, Alvaro Herrera wrote: > I don't know if it's possible to determine (at function execution time) > that we're running with the old extension version; if so it might > suffice to throw a warning but still have the SQL function run the same > C function.

Re: Commit fest manager for 2021-01

2020-12-24 Thread Masahiko Sawada
On Fri, Dec 25, 2020 at 2:57 PM Michael Paquier wrote: > > On Thu, Dec 24, 2020 at 07:29:37PM +0900, Masahiko Sawada wrote: > > Thank you. After re-logging in it looks the same as before but > > something will change on the CommitFest page? > > There should be a link to a new menu called

Commitfest 2021-01

2020-12-24 Thread Masahiko Sawada
Hello, hackers! 2021-01 commitfest will start from the new year, in a week. I'm happy to volunteer to be the CFM for this one. It's time to register your patch in the commitfest, if not yet. If you already have a patch in the commitfest, update its status and make sure it still applies and that

Re: [bug fix] ALTER TABLE SET LOGGED/UNLOGGED on a partitioned table does nothing silently

2020-12-24 Thread Michael Paquier
On Wed, Dec 09, 2020 at 10:56:45AM -0300, Alvaro Herrera wrote: > Sounds good. I think this thread is a good place to collect those > patches, but if you would prefer to have a new thread, feel free to > start one (I'd suggest CC'ing me and Tsunakawa-san). There is an entry listed in the CF for

Re: [bug fix] ALTER TABLE SET LOGGED/UNLOGGED on a partitioned table does nothing silently

2020-12-24 Thread Michael Paquier
On Wed, Dec 09, 2020 at 09:52:17AM -0300, Alvaro Herrera wrote: > On 2020-Dec-09, tsunakawa.ta...@fujitsu.com wrote: >> The new partition will have a property specified when the user creates >> it. That is, while the storage property of each storage unit >> (=partition) is basically independent,

Re: Disable WAL logging to speed up data loading

2020-12-24 Thread Michael Paquier
On Thu, Dec 03, 2020 at 03:52:47AM +, tsunakawa.ta...@fujitsu.com wrote: > The code looks good, and the performance seems to be nice, so I > marked this ready for committer. FWIW, I am extremely afraid of this proposal because this is basically a footgun able to corrupt customer instances,

Re: create table like: ACCESS METHOD

2020-12-24 Thread Michael Paquier
On Wed, Dec 09, 2020 at 02:13:29PM -0600, Justin Pryzby wrote: > I thought this was a good idea, but didn't hear back when I raised it before. > > Failing to preserve access method is arguably a bug, reminiscent of CREATE > STATISTICS and 5564c1181. But maybe it's not important to backpatch a

Re: Commit fest manager for 2021-01

2020-12-24 Thread Michael Paquier
On Thu, Dec 24, 2020 at 07:29:37PM +0900, Masahiko Sawada wrote: > Thank you. After re-logging in it looks the same as before but > something will change on the CommitFest page? There should be a link to a new menu called "administration" on the left of the existing logout button at the top.

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-12-24 Thread Tang, Haiying
Hi Amit, >I think one table with a varying amount of data is sufficient for the vacuum >test. >I think with more number of tables there is a greater chance of variation. >We have previously used multiple tables in one of the tests because of the >Truncate operation (which uses

Re: Parallel Inserts in CREATE TABLE AS

2020-12-24 Thread Dilip Kumar
On Fri, Dec 25, 2020 at 10:04 AM Amit Kapila wrote: > > On Fri, Dec 25, 2020 at 9:54 AM Bharath Rupireddy > wrote: > > > > On Fri, Dec 25, 2020 at 7:12 AM vignesh C wrote: > > > On Thu, Dec 24, 2020 at 11:29 AM Amit Kapila > > > wrote: > > > > > > > > On Thu, Dec 24, 2020 at 10:25 AM vignesh

Re: Parallel Inserts in CREATE TABLE AS

2020-12-24 Thread Amit Kapila
On Fri, Dec 25, 2020 at 9:54 AM Bharath Rupireddy wrote: > > On Fri, Dec 25, 2020 at 7:12 AM vignesh C wrote: > > On Thu, Dec 24, 2020 at 11:29 AM Amit Kapila > > wrote: > > > > > > On Thu, Dec 24, 2020 at 10:25 AM vignesh C wrote: > > > > > > > > On Tue, Dec 22, 2020 at 2:16 PM Bharath

Re: Parallel Inserts in CREATE TABLE AS

2020-12-24 Thread Dilip Kumar
On Fri, Dec 25, 2020 at 9:54 AM Bharath Rupireddy wrote: > > On Fri, Dec 25, 2020 at 7:12 AM vignesh C wrote: > > On Thu, Dec 24, 2020 at 11:29 AM Amit Kapila > > wrote: > > > > > > On Thu, Dec 24, 2020 at 10:25 AM vignesh C wrote: > > > > > > > > On Tue, Dec 22, 2020 at 2:16 PM Bharath

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-12-24 Thread Amit Kapila
On Fri, Dec 25, 2020 at 9:28 AM Tang, Haiying wrote: > > Hi Amit, > > >But how can we conclude NBuffers/128 is the maximum relation size? > >Because the maximum size would be where the performance is worse than > >the master, no? I guess we need to try by NBuffers/64, NBuffers/32, > > till we

Re: Parallel Inserts in CREATE TABLE AS

2020-12-24 Thread Bharath Rupireddy
On Fri, Dec 25, 2020 at 7:12 AM vignesh C wrote: > On Thu, Dec 24, 2020 at 11:29 AM Amit Kapila wrote: > > > > On Thu, Dec 24, 2020 at 10:25 AM vignesh C wrote: > > > > > > On Tue, Dec 22, 2020 at 2:16 PM Bharath Rupireddy > > > wrote: > > > > > > > > On Tue, Dec 22, 2020 at 12:32 PM Bharath

Re: Deadlock between backend and recovery may not be detected

2020-12-24 Thread Kyotaro Horiguchi
At Wed, 23 Dec 2020 21:42:47 +0900, Fujii Masao wrote in > you. Attached > is the updated of the patch. What about this version? The patch contains a hunk in the following structure. + if (got_standby_lock_timeout) + goto cleanup; + + if

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-12-24 Thread Tang, Haiying
Hi Amit, >But how can we conclude NBuffers/128 is the maximum relation size? >Because the maximum size would be where the performance is worse than >the master, no? I guess we need to try by NBuffers/64, NBuffers/32, > till we get the threshold where master performs better. You are right,

Re: Deadlock between backend and recovery may not be detected

2020-12-24 Thread Masahiko Sawada
On Wed, Dec 23, 2020 at 9:42 PM Fujii Masao wrote: > > > > On 2020/12/23 19:28, Masahiko Sawada wrote: > > On Tue, Dec 22, 2020 at 11:58 PM Fujii Masao > > wrote: > >> > >> > >> > >> On 2020/12/22 20:42, Fujii Masao wrote: > >>> > >>> > >>> On 2020/12/22 10:25, Masahiko Sawada wrote: > On

Re: Add table access method as an option to pgbench

2020-12-24 Thread Justin Pryzby
> --- a/doc/src/sgml/ref/pgbench.sgml > +++ b/doc/src/sgml/ref/pgbench.sgml > @@ -359,6 +359,16 @@ pgbench options > d > > > > + > + > --table-access-method=TABLEAM > + > + > +Create tables using the specified table access method, rather than >

Re: A failure of standby to follow timeline switch

2020-12-24 Thread Kyotaro Horiguchi
Thank you for looking this. At Thu, 24 Dec 2020 15:33:04 +0900, Fujii Masao wrote in > When I applied two patches in the master branch and > ran "make check-world", I got the following error. > > == creating database "contrib_regression" == > # Looks like you planned

Re: New Table Access Methods for Multi and Single Inserts

2020-12-24 Thread Justin Pryzby
On Thu, Dec 24, 2020 at 05:48:42AM +0530, Bharath Rupireddy wrote: > I'm not posting the updated 0002 to 0004 patches, I plan to do so > after a couple of reviews happen on the design of the APIs in 0001. > > Thoughts? Are you familiar with this work ? https://commitfest.postgresql.org/31/2717/

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-12-24 Thread Tang, Haiying
Hi Kirk, >Perhaps there is a confusing part in the presented table where you indicated >master(512), master(256), master(128). >Because the master is not supposed to use the BUF_DROP_FULL_SCAN_THRESHOLD and >just execute the existing default full scan of NBuffers. >Or I may have misunderstood

Re: Parallel Inserts in CREATE TABLE AS

2020-12-24 Thread vignesh C
On Thu, Dec 24, 2020 at 11:29 AM Amit Kapila wrote: > > On Thu, Dec 24, 2020 at 10:25 AM vignesh C wrote: > > > > On Tue, Dec 22, 2020 at 2:16 PM Bharath Rupireddy > > wrote: > > > > > > On Tue, Dec 22, 2020 at 12:32 PM Bharath Rupireddy > > > Attaching v14 patch set that has above changes.

Re: Feature request: Connection string parsing for postgres_fdw

2020-12-24 Thread Michael Paquier
On Fri, Dec 25, 2020 at 09:59:09AM +0900, Ian Lawrence Barwick wrote: > Basically a wrapper around PQconninfoParse(), I've had the code knocking > around > for a while now and finally got round to packaging it into an > extension [1]. It's > on my todo list to submit a patch based on this to

Re: Feature request: Connection string parsing for postgres_fdw

2020-12-24 Thread Ian Lawrence Barwick
2020年12月23日(水) 22:05 Eric Hanson : > > I'm trying to store connection to postgres_fdw in the database I want to be > able to store the full breadth of connection styles and all the different > types of connections that libpq supports. But having some troubles. > > Postgres_fdw wants options

Re: Incorrect allocation handling for cryptohash functions with OpenSSL

2020-12-24 Thread Michael Paquier
On Mon, Dec 21, 2020 at 04:28:26PM -0500, Robert Haas wrote: > TBH, I think there's no point in return an error here at all, because > it's totally non-specific. You have no idea what failed, just that > something failed. Blech. If we want to check that ctx is non-NULL, we > should do that with an

Re: In-placre persistance change of a relation

2020-12-24 Thread Kyotaro Horiguchi
Hello. At Thu, 24 Dec 2020 17:02:20 +0900 (JST), Kyotaro Horiguchi wrote in > The patch is attached to the next message. The reason for separating this message is that I modified this so that it could solve another issue. There's a complain about orphan files after crash. [1] 1:

Re: Preventing hangups in bgworker start/stop during DB shutdown

2020-12-24 Thread Tom Lane
I wrote: > Bharath Rupireddy writes: >> 4) IIUC, in the patch we mark slot->terminate = true only for >> BGW_NEVER_RESTART kind bg workers, what happens if a bg worker has >> bgw_restart_time seconds and don't we hit the hanging issue(that we >> are trying to solve here) for those bg workers? >

Re: Better client reporting for "immediate stop" shutdowns

2020-12-24 Thread Tom Lane
Bharath Rupireddy writes: > On Tue, Dec 22, 2020 at 11:02 PM Tom Lane wrote: >> There might be an argument for emitting the "unexpected SIGQUIT" >> text if we find garbage in sigquit_reason. Any thoughts about that? > Although I can't think of any case now, IMHO we can still have a > default

Re: Preventing hangups in bgworker start/stop during DB shutdown

2020-12-24 Thread Tom Lane
Bharath Rupireddy writes: > On Wed, Dec 23, 2020 at 3:10 AM Tom Lane wrote: >> Here's an attempt at closing the race condition discussed in [1] >> (and in some earlier threads, though I'm too lazy to find them). > 2) What if postmaster enters pmState >= PM_STOP_BACKENDS state after > it calls

Re: How is this possible "publication does not exist"

2020-12-24 Thread Marco Slot
I've been running into a similar issue and am a little puzzled by it, especially since it survives restarts. On Fri, Dec 20, 2019 at 2:39 AM Tomas Vondra wrote: > Yeah, I don't see this error message anywhere in our sources on 11 or > 12, so perhaps debezium does something funny? It's not clear

Re: postgres_fdw - cached connection leaks if the associated user mapping/foreign server is dropped

2020-12-24 Thread Fujii Masao
On 2020/12/24 23:30, Bharath Rupireddy wrote: On Thu, Dec 24, 2020 at 7:43 PM Fujii Masao wrote: Even when we're in the midst of transaction, if that transaction has not used the cached connections yet, we close them immediately. So, to make the comment more precise, what about updating the

Re: postgres_fdw - cached connection leaks if the associated user mapping/foreign server is dropped

2020-12-24 Thread Bharath Rupireddy
On Thu, Dec 24, 2020 at 7:43 PM Fujii Masao wrote: > Even when we're in the midst of transaction, if that transaction has not used > the cached connections yet, we close them immediately. So, to make the > comment more precise, what about updating the comment as follows? > > -

Re: postgres_fdw - cached connection leaks if the associated user mapping/foreign server is dropped

2020-12-24 Thread Fujii Masao
On 2020/12/24 15:42, Bharath Rupireddy wrote: On Thu, Dec 24, 2020 at 7:21 AM Fujii Masao wrote: On 2020/12/23 23:40, Bharath Rupireddy wrote: On Wed, Dec 23, 2020 at 7:31 PM Fujii Masao wrote: I agree to make pgfdw_xact_callback() close the connection when entry->invalidated == true.

Re: Movement of restart_lsn position movement of logical replication slots is very slow

2020-12-24 Thread Amit Kapila
On Thu, Dec 24, 2020 at 7:30 PM Jammie wrote: > > Sorry dont have the debug setup handy. However the sql commands now works > though to move the restart_lsn of the slots in standlone code from psql. > > A few followup questions. > > What is catalog_xmin in the pg_replication_slots ? and how is

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-12-24 Thread Amit Kapila
On Wed, Dec 23, 2020 at 6:27 PM k.jami...@fujitsu.com wrote: > > > It compiles. Passes the regression tests too. > Your feedbacks are definitely welcome. > Thanks, the patches look good to me now. I have slightly edited the patches for comments, commit messages, and removed the duplicate

Re: Movement of restart_lsn position movement of logical replication slots is very slow

2020-12-24 Thread Jammie
Sorry dont have the debug setup handy. However the sql commands now works though to move the restart_lsn of the slots in standlone code from psql. A few followup questions. What is catalog_xmin in the pg_replication_slots ? and how is it playing role in moving the restart_lsn of the slot. I am

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-12-24 Thread k.jami...@fujitsu.com
On Thu, December 24, 2020 6:02 PM JST, Tang, Haiying wrote: > Hi Amit, Kirk > > >One idea could be to remove "nBlocksToInvalidate < > >BUF_DROP_FULL_SCAN_THRESHOLD" part of check "if (cached && > >nBlocksToInvalidate < BUF_DROP_FULL_SCAN_THRESHOLD)" so that it > always > >use optimized path for

Re: Preventing hangups in bgworker start/stop during DB shutdown

2020-12-24 Thread Bharath Rupireddy
On Wed, Dec 23, 2020 at 3:10 AM Tom Lane wrote: > Here's an attempt at closing the race condition discussed in [1] > (and in some earlier threads, though I'm too lazy to find them). > > The core problem is that the bgworker management APIs were designed > without any thought for exception

Re: How is this possible "publication does not exist"

2020-12-24 Thread Tomas Vondra
On 12/24/20 12:50 PM, Marco Slot wrote: I've been running into a similar issue and am a little puzzled by it, especially since it survives restarts. Interesting. Which PostgreSQL version are you using? Any idea how to reproduce it? Were there any failures right before the issue appeared? I

Re: Feature request: Connection string parsing for postgres_fdw

2020-12-24 Thread Ashutosh Bapat
On Wed, Dec 23, 2020 at 7:42 PM Eric Hanson wrote: > > > > On Wed, Dec 23, 2020 at 5:39 AM Ashutosh Bapat > wrote: >> >> https://www.postgresql.org/docs/13/libpq-connect.html#LIBPQ-PARAMKEYWORDS >> lists the parameters that postgres_fdw accepts. "dbname" can be more >> than just dbname. See >>

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-12-24 Thread Amit Kapila
On Thu, Dec 24, 2020 at 2:31 PM Tang, Haiying wrote: > > Hi Amit, Kirk > > >One idea could be to remove "nBlocksToInvalidate < > >BUF_DROP_FULL_SCAN_THRESHOLD" part of check "if (cached && > >nBlocksToInvalidate < BUF_DROP_FULL_SCAN_THRESHOLD)" so that it always > >use optimized path for the

Re: Postgres Spark connector

2020-12-24 Thread Giuseppe Broccolo
Hi Zhihong, On Wed, 23 Dec 2020, 17:55 Zhihong Yu, wrote: > Hi, > I searched for Postgres support in Apache Spark. > I found Spark doc related to JDBC. > > I wonder if the community is aware of Spark connector for Postgres > (hopefully open source) where predicate involving jsonb columns can be

Re: Commit fest manager for 2021-01

2020-12-24 Thread Masahiko Sawada
On Mon, Dec 21, 2020 at 5:21 PM Magnus Hagander wrote: > > On Sun, Dec 20, 2020 at 10:57 PM Masahiko Sawada > wrote: > > > > On Sun, Dec 20, 2020 at 10:27 PM Magnus Hagander > > wrote: > > > > > > On Sat, Dec 19, 2020 at 6:00 AM Michael Paquier > > > wrote: > > > > > > > > On Sat, Dec 19,

Re: On login trigger: take three

2020-12-24 Thread Konstantin Knizhnik
On 22.12.2020 21:19, Pavel Stehule wrote: út 22. 12. 2020 v 12:42 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> napsal: On 22.12.2020 12:25, Pavel Stehule wrote: regress tests fails      sysviews                     ... FAILED    112 ms test

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-12-24 Thread Tang, Haiying
Hi Amit, Kirk >One idea could be to remove "nBlocksToInvalidate < >BUF_DROP_FULL_SCAN_THRESHOLD" part of check "if (cached && >nBlocksToInvalidate < BUF_DROP_FULL_SCAN_THRESHOLD)" so that it always >use optimized path for the tests. Then use the relation size as >NBuffers/128, NBuffers/256,

Re: Consider Parallelism While Planning For REFRESH MATERIALIZED VIEW

2020-12-24 Thread Bharath Rupireddy
On Wed, Dec 23, 2020 at 9:14 AM Hou, Zhijie wrote: > > Thanks for taking a look at the patch. > > > > The intention of the patch is to just enable the parallel mode while > > planning > > the select part of the materialized view, but the insertions do happen in > > the leader backend itself.

Re: doc review for v14

2020-12-24 Thread Michael Paquier
On Mon, Dec 21, 2020 at 10:11:53PM -0600, Justin Pryzby wrote: > As I did last 2 years, I reviewed docs for v14... Thanks for gathering all that! > This year I've started early, since it takes more than a little effort and > it's > not much fun to argue the change in each individual hunk.

Re: In-placre persistance change of a relation

2020-12-24 Thread Kyotaro Horiguchi
Thanks for the comment! Sorry for the late reply. At Fri, 4 Dec 2020 07:49:22 +, "tsunakawa.ta...@fujitsu.com" wrote in > From: Kyotaro Horiguchi > > > No, not really. The issue is more around what happens if we crash > > > part way through. At crash recovery time, the system catalogs