Re: adding partitioned tables to publications

2020-04-03 Thread Tom Lane
Petr Jelinek writes: > On 03/04/2020 17:51, Tom Lane wrote: >> But the forked-off children have to write the gcov files independently, >> don't they? > Hmm that's very good point. I did see these missing coverage issue when > running tests that explicitly start more instances of postgres before

Re: WAL usage calculation patch

2020-04-03 Thread Amit Kapila
On Fri, Apr 3, 2020 at 7:36 PM Amit Kapila wrote: > > On Fri, Apr 3, 2020 at 9:40 AM Amit Kapila wrote: > > > > On Fri, Apr 3, 2020 at 9:35 AM Dilip Kumar wrote: > > > > > > I have analyzed the WAL and there could be multiple reasons for the > > > same. With small data, I have noticed that

Re: adding partitioned tables to publications

2020-04-03 Thread Petr Jelinek
On 03/04/2020 17:51, Tom Lane wrote: Petr Jelinek writes: On 03/04/2020 16:59, Tom Lane wrote: Petr Jelinek writes: AFAIK gcov can't handle multiple instances of same process being started as it just overwrites the coverage files. So for TAP test it will report bogus info (as in some code

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Petr Jelinek
On 04/04/2020 05:06, Andres Freund wrote: Hi, Peter, Petr, CCed you because it's probably a bug somewhere around the initial copy code for logical replication. On 2020-04-03 20:48:09 -0400, Robert Haas wrote: 'serinus' is also failing. This is less obviously related: Hm. Tests passed once

Re: Proposal: is_castable

2020-04-03 Thread Pavel Stehule
Hi > So it's a two step process for me currently because of this, I would love > if there was a better way to handle > this type of work though, because my plpgsql functions using exception > blocks are not exactly great > for performance. > Probably we can for some important buildin types

Re: proposal \gcsv

2020-04-03 Thread Pavel Stehule
so 4. 4. 2020 v 0:24 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > We can have a new commands for cloning print environments and switch to > one > > shot environment. It can be based just on enhancing of \pset command > > > \pset save anyidentifier .. serialize settings > > \pset load

Re: Add A Glossary

2020-04-03 Thread Laurenz Albe
On Fri, 2020-04-03 at 16:01 -0500, Justin Pryzby wrote: > BTW it's now visible at: > https://www.postgresql.org/docs/devel/glossary.html Great! Some comments: - SQL object: There are more kinds of objects, like roles or full text dictionaries. Perhaps better: Anything that is created

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Andres Freund
Hi, Peter, Petr, CCed you because it's probably a bug somewhere around the initial copy code for logical replication. On 2020-04-03 20:48:09 -0400, Robert Haas wrote: > 'serinus' is also failing. This is less obviously related: Hm. Tests passed once since then. > 2020-04-04 02:08:57.299 CEST

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 9:52 PM Tom Lane wrote: > Robert Haas writes: > > 'prairiedog' is also unhappy, and it looks related: > > Yeah, gaur also failed in the same place. Both of those are > alignment-picky 32-bit hardware, so I'm thinking the problem is > pg_gmtime() trying to fetch a 64-bit

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Tom Lane
Robert Haas writes: > Interestingly, on my machine, rmtree coped with a mode 0 directory > just fine, but mode 0400 was more than its tiny brain could handle, so > the originally committed fix had code to revert 0400 back to 0700, but > I didn't add similar code to revert from 0 back to 0700

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Tom Lane
Robert Haas writes: > 'prairiedog' is also unhappy, and it looks related: Yeah, gaur also failed in the same place. Both of those are alignment-picky 32-bit hardware, so I'm thinking the problem is pg_gmtime() trying to fetch a 64-bit pg_time_t from an insufficiently aligned address. I'm

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-03 Thread David Rowley
On Fri, 3 Apr 2020 at 21:56, Andy Fan wrote: > > > > On Fri, Apr 3, 2020 at 12:08 PM David Rowley wrote: >> >> On Fri, 3 Apr 2020 at 16:40, Tom Lane wrote: >> > (It occurs to me BTW that we've been overly conservative about using >> > NOT NULL constraints in planning, because of failing to

Re: Proposal: is_castable

2020-04-03 Thread Adam Brusselback
> What would you actually do with it? I am one of the users of these do-it-yourself functions, and I use them in my ETL pipelines heavily. For me, data gets loaded into a staging table, all columns text, and I run a whole bunch of validation queries on the data prior to it moving to the next

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 8:12 PM Tom Lane wrote: > Yeah, so it would seem. The buildfarm script uses rmtree to clean out > the old build tree. The man page for File::Path suggests (but can't > quite bring itself to say in so many words) that by default, rmtree > will adjust the permissions on

Re: EINTR while resizing dsm segment.

2020-04-03 Thread Thomas Munro
On Thu, Apr 2, 2020 at 9:25 PM Kyotaro Horiguchi wrote: > I provided the subject, and added -hackers. > > > Hello, > > I am running postgres 11.5 and we were having issues with shared segments. > > So I increased the max_connection as suggested by you guys and reduced my > > work_mem to 600M. > >

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 6:13 PM Tom Lane wrote: > Locally, I observe that "make clean" in src/bin/pg_validatebackup fails > to clean up the tmp_check directory left behind by "make check". Fixed. I also tried to fix 'lapwing', which was complaining about about a call to pg_gmtime, saying that it

Re: [PATCH] Fix for slow GIN index queries when "gin_fuzzy_search_limit" setting is relatively small for large tables

2020-04-03 Thread Adé
Great. Thanks for refactoring it further and fixing other bugs in there (and making it more clean too)! Regards, Ade On Fri, Apr 3, 2020 at 1:18 PM Tom Lane wrote: > I wrote: > > I'm inclined to back-patch this. Given how fuzzy the definition > > of gin_fuzzy_search_limit is, it seems

Re: backup manifests

2020-04-03 Thread Tom Lane
BTW, some of the buildfarm is showing a simpler portability problem: they think you were too cavalier about the difference between time_t and pg_time_t. (On a platform with 32-bit time_t, that's an actual bug, probably.) lapwing is actually failing:

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-03 Thread Alexander Korotkov
Hi! On Fri, Apr 3, 2020 at 9:51 PM Anna Akenteva wrote: > In the patch that I reviewed, you could do things like: > WAIT FOR > LSN lsn0, > LSN lsn1 TIMEOUT time1, > LSN lsn2 TIMEOUT time2; > and such a statement was in practice equivalent to > WAIT FOR

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Tom Lane
Robert Haas writes: > On Fri, Apr 3, 2020 at 6:48 PM Tom Lane wrote: >> I'm guessing that we're looking at a platform-specific difference in >> whether "rm -rf" fails outright on an unreadable subdirectory, or >> just tries to carry on by unlinking it anyway. > My intention was that it would be

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 5:58 PM Fabien COELHO wrote: > seawasp just failed the same way. Good news, I can see "configure" under > "HEAD/pgsql". Ah, good. > The only strange thing under buildroot I found is: > >

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 6:48 PM Tom Lane wrote: > I'm guessing that we're looking at a platform-specific difference in > whether "rm -rf" fails outright on an unreadable subdirectory, or > just tries to carry on by unlinking it anyway. My intention was that it would be cleaned by the TAP

Re: Ltree syntax improvement

2020-04-03 Thread Nikita Glukhov
On 02.04.2020 19:16, Tom Lane wrote: Nikita Glukhov writes: Rebased patch attached. Thanks for rebasing! The cfbot's not very happy though: 4842ltxtquery_io.c: In function ‘makepol’: 4843ltxtquery_io.c:188:13: error: ‘escaped’ may be used uninitialized in this function

Re: vacuum_defer_cleanup_age inconsistently applied on replicas

2020-04-03 Thread Peter Geoghegan
On Fri, Apr 3, 2020 at 4:18 PM Peter Geoghegan wrote: > OTOH, I wonder if it's possible that vacuum_defer_cleanup_age was > deliberately intended to affect the behavior of > XLogWalRcvSendHSFeedback(), which is probably one of the most common > reasons why GetOldestXmin() is called on standbys.

Re: vacuum_defer_cleanup_age inconsistently applied on replicas

2020-04-03 Thread Peter Geoghegan
On Fri, Apr 3, 2020 at 3:53 PM Andres Freund wrote: > GetOldestXmin() applies vacuum_defer_cleanup_age only when > !RecoveryInProgress(). In contrast to that GetSnapshotData() applies it > unconditionally. > > I'm not actually clear whether including vacuum_defer_cleanup_age on a > replica is

Re: [HACKERS] Restricting maximum keep segments by repslots

2020-04-03 Thread Alvaro Herrera
So, the more I look at this patch, the less I like the way the slots are handled. * I think it's a mistake to try to do anything in KeepLogSeg itself; that function is merely in charge of some arithmetic. I propose to make that function aware of the new size limitation (so that it doesn't

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Tom Lane
I wrote: > I'm guessing that we're looking at a platform-specific difference in > whether "rm -rf" fails outright on an unreadable subdirectory, or > just tries to carry on by unlinking it anyway. Yeah... on my RHEL6 box, "make check" cleans up the working directories under tmp_check, but on a

vacuum_defer_cleanup_age inconsistently applied on replicas

2020-04-03 Thread Andres Freund
Hi, GetOldestXmin() applies vacuum_defer_cleanup_age only when !RecoveryInProgress(). In contrast to that GetSnapshotData() applies it unconditionally. I'm not actually clear whether including vacuum_defer_cleanup_age on a replica is meaningful. But it strikes me as odd to have that behavioural

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Tom Lane
Thomas Munro writes: > Same here, on elver. I see pg_subtrans has been chmod(0)'d, > presumably by the perl subroutine mutilate_open_directory_fails. I > see this in my inbox (the build farm wrote it to stderr or stdout > rather than the log file): > cannot chdir to child for >

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Stephen Frost
Greetings, * Thomas Munro (thomas.mu...@gmail.com) wrote: > On Sat, Apr 4, 2020 at 11:13 AM Tom Lane wrote: > > Fabien COELHO writes: > > > The only strange thing under buildroot I found is: > > > > >

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Thomas Munro
On Sat, Apr 4, 2020 at 11:13 AM Tom Lane wrote: > Fabien COELHO writes: > > The only strange thing under buildroot I found is: > > > HEAD/pgsql.build/src/bin/pg_validatebackup/tmp_check/t_003_corruption_master_data/backup/open_directory_fails/pg_subtrans/ > > > this last directory perms are

Re: proposal \gcsv

2020-04-03 Thread Tom Lane
Pavel Stehule writes: > We can have a new commands for cloning print environments and switch to one > shot environment. It can be based just on enhancing of \pset command > \pset save anyidentifier .. serialize settings > \pset load anyidentifier .. load setting > \pset oneshot [anyidentifer] ..

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Alvaro Herrera
On 2020-Apr-03, Tom Lane wrote: > I wonder if VPATH versus not-VPATH might be a relevant factor ... Oh, absolutely. The ones that failed show, in the last successful run, the configure line invoked as "./configure", while the animals that are still running are invoking configure from some other

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Tom Lane
Fabien COELHO writes: > The only strange thing under buildroot I found is: > HEAD/pgsql.build/src/bin/pg_validatebackup/tmp_check/t_003_corruption_master_data/backup/open_directory_fails/pg_subtrans/ > this last directory perms are d- which seems to break cleanup. Locally, I observe

Re: backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Fabien COELHO
Hello Robert, Done now. Meanwhile, two more machines have reported the mysterious message: sh: ./configure: not found ...that first appeared on spurfowl a few hours ago. The other two machines are eelpout and elver, both of which list Thomas Munro as a maintainer. spurfowl lists Stephen

backup manifests and contemporaneous buildfarm failures

2020-04-03 Thread Robert Haas
[ splitting this off into a separate thread ] On Fri, Apr 3, 2020 at 5:07 PM Robert Haas wrote: > I'lll go see about adding that. Done now. Meanwhile, two more machines have reported the mysterious message: sh: ./configure: not found ...that first appeared on spurfowl a few hours ago. The

Re: backup manifests

2020-04-03 Thread Justin Pryzby
On Fri, Apr 03, 2020 at 03:22:23PM -0400, Robert Haas wrote: > I've pushed all the patches. I didn't manage to look at this in advance but have some doc fixes. word-diff: diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index 536de9a698..d84afb7b18 100644 ---

Re: d25ea01275 and partitionwise join

2020-04-03 Thread Tom Lane
Amit Langote writes: > Updated patches attached. I looked through these and committed 0001+0002, with some further comment-polishing. However, I have no faith at all in 0003. It is blithely digging through COALESCE expressions with no concern for whether they came from full joins or not, or

Re: backup manifests

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 4:54 PM Alvaro Herrera wrote: > Maybe it needs perl2host? *jaw drops* Wow, OK, yeah, that looks like the thing. Thanks for the suggestion; I didn't know that existed (and I kinda wish I still didn't). I'lll go see about adding that. -- Robert Haas EnterpriseDB:

Re: Add A Glossary

2020-04-03 Thread Erik Rijkers
On 2020-04-03 22:51, Alvaro Herrera wrote: On 2020-Apr-03, Erik Rijkers wrote: On 2020-04-03 18:45, Alvaro Herrera wrote: > Pushed now. Many thanks to Corey who put the main thrust, and to Jürgen > and Roger for the great help, and to Justin for the extensive review and > Fabien for the

Re: Add A Glossary

2020-04-03 Thread Justin Pryzby
On Fri, Apr 03, 2020 at 05:51:43PM -0300, Alvaro Herrera wrote: > - The internal representation of one value of a SQL > + The internal representation of one value of an SQL I'm not sure about this one. The new glossary says "a SQL" seven times, and doesn't say "an sql" at all. "An SQL"

Re: backup manifests

2020-04-03 Thread Alvaro Herrera
On 2020-Apr-03, Robert Haas wrote: > sub tempdir_short > { > > return File::Temp::tempdir(CLEANUP => 1); > } > > And File::Temp's documentation says that the temporary directory is > picked using File::Spec's tmpdir(), which says that it knows about > different operating systems and

Re: Add A Glossary

2020-04-03 Thread Alvaro Herrera
On 2020-Apr-03, Erik Rijkers wrote: > On 2020-04-03 18:45, Alvaro Herrera wrote: > > Pushed now. Many thanks to Corey who put the main thrust, and to Jürgen > > and Roger for the great help, and to Justin for the extensive review and > > Fabien for the initial discussion. > > A few

Re: backup manifests

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 3:53 PM Robert Haas wrote: > 2. Also, a bunch of machines were super-unhappy with > 003_corruption.pl, failing with this sort of thing: > > pg_basebackup: error: could not get COPY data stream: ERROR: symbolic > link target too long for tar format: file name

Re: Binary support for pgoutput plugin

2020-04-03 Thread Dave Cramer
On Fri, 3 Apr 2020 at 03:43, Petr Jelinek wrote: > Hi, > > On 08/03/2020 00:18, Dave Cramer wrote: > > On Fri, 6 Mar 2020 at 08:54, Petr Jelinek > > wrote: > > > > Hi Dave, > > > > On 29/02/2020 18:44, Dave Cramer wrote: > > > > > > > > >

Re: proposal \gcsv

2020-04-03 Thread Pavel Stehule
st 1. 4. 2020 v 18:29 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > It can work, but it is not user friendly - my proposal was motivated by > > using some quick csv exports to gplot's pipe. > > I kind of liked the stack idea, myself. It's simpler than what I was > suggesting and it

Re: backup manifests

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 3:22 PM Robert Haas wrote: > It looks like the buildfarm is unhappy though, so I guess I'd better > go look at that. I fixed two things so far, and there seems to be at least one more possible issue that I don't understand. 1. Apparently, we have an automated perlcritic

Re: backup manifests

2020-04-03 Thread Robert Haas
On Thu, Apr 2, 2020 at 4:34 PM David Steele wrote: > +1. These would be great tests to have and a win for pg_basebackup > overall but I don't think they should be a prerequisite for this commit. Not to mention the server. I can't say that I have a lot of confidence that all of the server

Re: snapshot too old issues, first around wraparound and then more.

2020-04-03 Thread Andres Freund
Hi, On 2020-04-03 14:32:09 +0530, Amit Kapila wrote: > On Fri, Apr 3, 2020 at 6:52 AM Peter Geoghegan wrote: > > > > On Thu, Apr 2, 2020 at 5:17 PM Andres Freund wrote: > > > Since this is a feature that can result in wrong query results (and > > > quite possibly crashes / data corruption), I

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-03 Thread Anna Akenteva
I did some code cleanup and added tests - both for the standalone WAIT FOR statement and for WAIT FOR as a part of BEGIN. The new patch is attached. On 2020-04-03 17:29, Alexey Kondratov wrote: On 2020-04-01 02:26, Anna Akenteva wrote: - WAIT FOR [ANY | ALL] event [, ...] - BEGIN [ WORK |

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-03 Thread Justin Pryzby
On Wed, Apr 01, 2020 at 08:08:36AM -0500, Justin Pryzby wrote: > Or maybe you'd want me to squish my changes into yours and resend after any > review comments ? I didn't hear any feedback, so I've now squished all "parenthesized" and "fix" commits. 0004 reduces duplicative error handling, as a

Re: Syntax rules for a text value inside the literal for a user-defined type—doc section “8.16.2. Constructing Composite Values”

2020-04-03 Thread Bryn Llewellyn
On 03-Apr-2020, at 00:05, David G. Johnston wrote: On Thu, Apr 2, 2020 at 8:46 PM Bryn Llewellyn mailto:b...@yugabyte.com>> wrote: On 02-Apr-2020, at 19:25, Tom Lane mailto:t...@sss.pgh.pa.us>> wrote: Bryn Llewellyn mailto:b...@yugabyte.com>> writes: > The documentation in section “8.16.2.

Re: pgsql: Improve handling of parameter differences in physical replicatio

2020-04-03 Thread Robert Haas
On Mon, Mar 30, 2020 at 2:10 PM Andres Freund wrote: > One important issue seems to me to be the size of the array that > TransactionIdIsInProgress() allocates: > /* > * If first time through, get workspace to remember main XIDs in. We > * malloc it permanently to avoid

Re: Add A Glossary

2020-04-03 Thread Erik Rijkers
On 2020-04-03 18:45, Alvaro Herrera wrote: Pushed now. Many thanks to Corey who put the main thrust, and to Jürgen and Roger for the great help, and to Justin for the extensive review and Fabien for the initial discussion. A few improvements: 'its value that cannot' should be 'its value

Re: Add A Glossary

2020-04-03 Thread Roger Harkavy
On Fri, Apr 3, 2020 at 1:34 PM Corey Huinker wrote: > Thanks for all your work on this! > And to add on to Corey's message of thanks, I also want to thank everyone for their input and assistance on that. I am very grateful for the opportunity to contribute to this project!

Re: Datum values consistency within one query

2020-04-03 Thread Tom Lane
Paul Ramsey writes: > So, if I tested for VARATT_IS_EXTENDED(), and then for > VARATT_IS_EXTERNAL_ONDISK(attr) and then did > VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr), I could use va_valueid + > va_toastrelid as keys in the cache for things that passed that filter? I'm pretty sure

Re: Add A Glossary

2020-04-03 Thread Corey Huinker
> > we have it, we can start thinking of patching the main part of the docs > to make reference to it by using in key spots. Right now > the glossary links to itself, but it makes lots of sense to have other > places point to it. > I have some ideas about how to patch the main docs, but will

Re: Datum values consistency within one query

2020-04-03 Thread Paul Ramsey
> On Apr 2, 2020, at 4:30 PM, Tom Lane wrote: > > Paul Ramsey writes: >> Getting the datum value is really fast, so I can have a cache that >> keeps the latest detoasted object around, and update it when the datum >> changes, and store the cache information in the parent context. Like >> so:

Re: [PATCH] Fix for slow GIN index queries when "gin_fuzzy_search_limit" setting is relatively small for large tables

2020-04-03 Thread Tom Lane
I wrote: > I'm inclined to back-patch this. Given how fuzzy the definition > of gin_fuzzy_search_limit is, it seems unlikely that anyone would > be depending on the current buggy behavior. And done. Thanks for the bug report and patch! regards, tom lane

Re: Add A Glossary

2020-04-03 Thread Alvaro Herrera
Pushed now. Many thanks to Corey who put the main thrust, and to Jürgen and Roger for the great help, and to Justin for the extensive review and Fabien for the initial discussion. This is just a starting point. Let's keep improving it. And how that we have it, we can start thinking of patching

Re: zombie connections

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 11:50 AM Tom Lane wrote: > (2) It only wins if a statement timeout is active, otherwise it makes > things worse, because then you need setitimer() at statement start > and end just to enable/disable the socket check timeout. Whereas > if you just let a once-a-minute

Re: VACUUM memory management

2020-04-03 Thread Ibrar Ahmed
On Mon, Mar 16, 2020 at 6:35 PM David Steele wrote: > On 1/28/20 1:36 PM, Ibrar Ahmed wrote: > > On Wed, Jan 22, 2020 at 11:17 AM k.jami...@fujitsu.com > > I might have missed something more, > > > > but I'll continue reviewing after the rebased patch. > > > > Yes, I am working

Re: adding partitioned tables to publications

2020-04-03 Thread Tom Lane
Petr Jelinek writes: > On 03/04/2020 16:59, Tom Lane wrote: >> Petr Jelinek writes: >>> AFAIK gcov can't handle multiple instances of same process being started >>> as it just overwrites the coverage files. So for TAP test it will report >>> bogus info (as in some code that's executed will look

Re: zombie connections

2020-04-03 Thread Tom Lane
Robert Haas writes: > On Fri, Apr 3, 2020 at 10:34 AM Tom Lane wrote: >> In general I think the threshold problem for a patch like this will be >> "how do you keep the added overhead down". As Robert noted upthread, >> timeout.c is quite a bit shy of being able to handle timeouts that >>

Re: Should we add xid_current() or a int8->xid cast?

2020-04-03 Thread Mark Dilger
> On Apr 2, 2020, at 7:39 PM, Thomas Munro wrote: > > These apply cleanly, build and pass check-world on mac, and the documentation and regression test changes surrounding txid look good to me. FYI, (not the responsibility of this patch), we never quite define what the abbreviation "xip"

Re: adding partitioned tables to publications

2020-04-03 Thread Petr Jelinek
On 03/04/2020 16:59, Tom Lane wrote: Petr Jelinek writes: AFAIK gcov can't handle multiple instances of same process being started as it just overwrites the coverage files. So for TAP test it will report bogus info (as in some code that's executed will look as not executed). Hm, really? I

Re: adding partitioned tables to publications

2020-04-03 Thread Tom Lane
Petr Jelinek writes: > AFAIK gcov can't handle multiple instances of same process being started > as it just overwrites the coverage files. So for TAP test it will report > bogus info (as in some code that's executed will look as not executed). Hm, really? I routinely run "make check" (ie,

Re: adding partitioned tables to publications

2020-04-03 Thread Petr Jelinek
Hi, On 03/04/2020 16:25, Amit Langote wrote: On Fri, Apr 3, 2020 at 6:34 PM Amit Langote wrote: I am checking test coverage at the moment and should have the patches ready by sometime later today. Attached updated patches. I confirmed using a coverage build that all the new code in

Re: zombie connections

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 10:34 AM Tom Lane wrote: > In general I think the threshold problem for a patch like this will be > "how do you keep the added overhead down". As Robert noted upthread, > timeout.c is quite a bit shy of being able to handle timeouts that > persist across statements. I

Re: zombie connections

2020-04-03 Thread Tom Lane
Pavel Stehule writes: > I prefer simple solution without any "intelligence". It is much safer to > close connect and rollback. Then it is clean protocol - when server didn't > reported successful end of operation, then operation was reverted - always. It would be a fatal mistake to imagine that

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-03 Thread Alexey Kondratov
On 2020-04-01 02:26, Anna Akenteva wrote: On 2020-03-27 04:15, Kartyshov Ivan wrote: Anna, feel free to work on this patch. Ivan and I worked on this patch a bit more. We fixed the bugs that we could find and cleaned up the code. For now, we've kept both options: WAIT as a standalone

Re: adding partitioned tables to publications

2020-04-03 Thread Amit Langote
On Fri, Apr 3, 2020 at 6:34 PM Amit Langote wrote: > I am checking test coverage at the moment and should have the patches > ready by sometime later today. Attached updated patches. I confirmed using a coverage build that all the new code in logical/worker.c due to 0002 is now covered. For some

Re: Proposal: is_castable

2020-04-03 Thread Tom Lane
=?UTF-8?Q?Micha=C5=82_Wadas?= writes: > Currently there is no way to check if CAST will succeed. > Therefore I propose adding new function: is_castable > SELECT is_castable('foo' as time) // false What would you actually do with it? > Similar features are implemented in: > - SQL Server (as

where should I stick that backup?

2020-04-03 Thread Robert Haas
There are a couple of things that pg_basebackup can't do that might be an issue for some users. One of them is that you might want to do something like encrypt your backup. Another is that you might want to store someplace other than in the filesystem, like maybe S3. We could certainly teach

Re: WAL usage calculation patch

2020-04-03 Thread Amit Kapila
On Fri, Apr 3, 2020 at 9:40 AM Amit Kapila wrote: > > On Fri, Apr 3, 2020 at 9:35 AM Dilip Kumar wrote: > > > > I have analyzed the WAL and there could be multiple reasons for the > > same. With small data, I have noticed that while inserting in the > > system index there was a Page Split and

Re: zombie connections

2020-04-03 Thread Pavel Stehule
pá 3. 4. 2020 v 15:52 odesílatel Isaac Morland napsal: > On Fri, 3 Apr 2020 at 08:30, Robert Haas wrote: > >> >> I don't have a terribly specific idea about how to improve this, but >> is there some way that we could, at least periodically, check the >> socket to see whether it's dead? Noticing

Re: zombie connections

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 9:52 AM Isaac Morland wrote: > Does it make any difference if the query is making changes? If the query is > just computing a result and returning it to the client, there is no point in > continuing once the socket is closed. But if it is updating data or making > DDL

Re: zombie connections

2020-04-03 Thread Isaac Morland
On Fri, 3 Apr 2020 at 08:30, Robert Haas wrote: > > I don't have a terribly specific idea about how to improve this, but > is there some way that we could, at least periodically, check the > socket to see whether it's dead? Noticing the demise of the client > after a configurable interval (maybe

Re: Nicer error when connecting to standby with hot_standby=off

2020-04-03 Thread James Coleman
On Thu, Apr 2, 2020 at 5:53 PM David Zhang wrote: > > The following review has been posted through the commitfest application: > make installcheck-world: not tested > Implements feature: tested, passed > Spec compliant: not tested > Documentation:not tested > > I

Re: zombie connections

2020-04-03 Thread Robert Haas
On Fri, Apr 3, 2020 at 9:34 AM Konstantin Knizhnik wrote: > There was a patch on commitfest addressing this problem: > https://commitfest.postgresql.org/21/1882/ > It it currently included in PostgrePro EE, but the author of the patch > is not working in our company any more. > Should we

Re: User Interface for WAL usage data

2020-04-03 Thread Julien Rouhaud
On Fri, Apr 3, 2020 at 11:58 AM Dilip Kumar wrote: > > On Fri, Apr 3, 2020 at 1:57 PM Amit Kapila wrote: > > > > On Fri, Apr 3, 2020 at 11:29 AM Amit Kapila wrote: > > > > > > On Fri, Apr 3, 2020 at 11:14 AM Justin Pryzby > > > wrote: > > > > > > > > On Fri, Apr 03, 2020 at 10:52:02AM +0530,

Re: zombie connections

2020-04-03 Thread Konstantin Knizhnik
On 03.04.2020 15:29, Robert Haas wrote: Hi, Suppose that the server is executing a lengthy query, and the client breaks the connection. The operating system will be aware that the connection is no more, but PostgreSQL doesn't notice, because it's not try to read from or write to the socket.

Re: zombie connections

2020-04-03 Thread Julien Rouhaud
On Fri, Apr 03, 2020 at 02:40:30PM +0200, Pavel Stehule wrote: > pá 3. 4. 2020 v 14:30 odesílatel Robert Haas napsal: > > > > Suppose that the server is executing a lengthy query, and the client > > breaks the connection. The operating system will be aware that the > > connection is no more, but

Re: [Proposal] Global temporary tables

2020-04-03 Thread Prabhat Sahu
Hi Wenjing, Please check the allowed values for boolean parameter "on_commit_delete_rows". postgres=# create global temp table gtt1(c1 int) with(on_commit_delete_rows='true'); CREATE TABLE Similarly we can successfully create GTT by using the values as: 'true','false', true, false, 'ON', 'OFF',

Re: zombie connections

2020-04-03 Thread Pavel Stehule
pá 3. 4. 2020 v 14:30 odesílatel Robert Haas napsal: > Hi, > > Suppose that the server is executing a lengthy query, and the client > breaks the connection. The operating system will be aware that the > connection is no more, but PostgreSQL doesn't notice, because it's not > try to read from or

zombie connections

2020-04-03 Thread Robert Haas
Hi, Suppose that the server is executing a lengthy query, and the client breaks the connection. The operating system will be aware that the connection is no more, but PostgreSQL doesn't notice, because it's not try to read from or write to the socket. It's not paying attention to the socket at

Re: Proposal: is_castable

2020-04-03 Thread Pavel Stehule
Hi pá 3. 4. 2020 v 13:45 odesílatel Michał Wadas napsal: > Currently there is no way to check if CAST will succeed. > > Therefore I propose adding new function: is_castable > > SELECT is_castable('foo' as time) // false > SELECT is_castable('123' as numeric) // true > SELECT is_castable(1.5 as

Proposal: is_castable

2020-04-03 Thread Michał Wadas
Currently there is no way to check if CAST will succeed. Therefore I propose adding new function: is_castable SELECT is_castable('foo' as time) // false SELECT is_castable('123' as numeric) // true SELECT is_castable(1.5 as int) // true SELECT is_castable('1.5' as int) // false Many users write

Re: WIP/PoC for parallel backup

2020-04-03 Thread Kashif Zeeshan
Hi Asif When a non-existent slot is used with tablespace then correct error is displayed but then the backup folder is not cleaned and leaves a corrupt backup. Steps === edb@localhost bin]$ [edb@localhost bin]$ mkdir /home/edb/tbl1 [edb@localhost bin]$ mkdir /home/edb/tbl_res [edb@localhost

Re: User Interface for WAL usage data

2020-04-03 Thread Dilip Kumar
On Fri, Apr 3, 2020 at 1:57 PM Amit Kapila wrote: > > On Fri, Apr 3, 2020 at 11:29 AM Amit Kapila wrote: > > > > On Fri, Apr 3, 2020 at 11:14 AM Justin Pryzby wrote: > > > > > > On Fri, Apr 03, 2020 at 10:52:02AM +0530, Amit Kapila wrote: > > > > On Fri, Apr 3, 2020 at 10:41 AM Justin Pryzby

Re: Online checksums verification in the backend

2020-04-03 Thread Julien Rouhaud
On Fri, Apr 03, 2020 at 12:24:50PM +0900, Masahiko Sawada wrote: > On Sat, 28 Mar 2020 at 21:19, Julien Rouhaud wrote: > > > The current patch still checks SearchSysCacheExists1() before > relation_open. Why do we need to call SearchSysCacheExists1() here? I > think if the given relation doesn't

Re: adding partitioned tables to publications

2020-04-03 Thread Amit Langote
On Fri, Apr 3, 2020 at 4:52 PM Petr Jelinek wrote: > On 02/04/2020 14:23, Peter Eisentraut wrote: > > On 2020-03-30 17:42, Amit Langote wrote: > >> I have updated the comments in apply_handle_tuple_routing() (see 0002) > >> to better explain what's going on with UPDATE handling. I also > >>

Re: snapshot too old issues, first around wraparound and then more.

2020-04-03 Thread Amit Kapila
On Fri, Apr 3, 2020 at 6:52 AM Peter Geoghegan wrote: > > On Thu, Apr 2, 2020 at 5:17 PM Andres Freund wrote: > > Since this is a feature that can result in wrong query results (and > > quite possibly crashes / data corruption), I don't think we can just > > leave this unfixed. But given the

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-03 Thread Andy Fan
On Fri, Apr 3, 2020 at 12:08 PM David Rowley wrote: > On Fri, 3 Apr 2020 at 16:40, Tom Lane wrote: > > (It occurs to me BTW that we've been overly conservative about using > > NOT NULL constraints in planning, because of failing to consider that. > > Addition or drop of NOT NULL has to cause a

Re: A bug when use get_bit() function for a long bytea string

2020-04-03 Thread Daniel Verite
movead...@highgo.ca wrote: > >I believe the formula for the upper limit in the 32-bit case is: > > (len <= PG_INT32_MAX / 8) ? (len*8 - 1) : PG_INT32_MAX; > > >These functions could benefit from a comment mentioning that > >they cannot reach the full extent of a bytea, because of the

Re: WIP/PoC for parallel backup

2020-04-03 Thread Asif Rehman
On Thu, Apr 2, 2020 at 8:45 PM Robert Haas wrote: > On Thu, Apr 2, 2020 at 11:17 AM Asif Rehman > wrote: > >> Why would you need to do that? As long as the process where > >> STOP_BACKUP can do the check, that seems good enough. > > > > Yes, but the user will get the error only after the

Re: [Proposal] Global temporary tables

2020-04-03 Thread Pavel Stehule
pá 3. 4. 2020 v 9:52 odesílatel 曾文旌 napsal: > In my opinion > 1 We are developing GTT according to the SQL standard, not Oracle. > > 2 The implementation differences you listed come from pg and oracle > storage modules and DDL implementations. > > 2.1 issue 1 and issue 2 > The creation of Normal

Re: User Interface for WAL usage data

2020-04-03 Thread Amit Kapila
On Fri, Apr 3, 2020 at 11:29 AM Amit Kapila wrote: > > On Fri, Apr 3, 2020 at 11:14 AM Justin Pryzby wrote: > > > > On Fri, Apr 03, 2020 at 10:52:02AM +0530, Amit Kapila wrote: > > > On Fri, Apr 3, 2020 at 10:41 AM Justin Pryzby > > > wrote: > > > > > > > > > > > > > >WAL: records=2359

Re: Yet another fast GiST build

2020-04-03 Thread Yuri Astrakhan
Awesome addition! Would it make sense to use x86's BMI2's PDEP instruction, or is the interleave computation too small of a percentage to introduce not-so-easy-to-port code? Also, I think it needs a bit more documentation to explain the logic, i.e. a link to

Re: [Proposal] Global temporary tables

2020-04-03 Thread 曾文旌
In my opinion 1 We are developing GTT according to the SQL standard, not Oracle. 2 The implementation differences you listed come from pg and oracle storage modules and DDL implementations. 2.1 issue 1 and issue 2 The creation of Normal table/GTT defines the catalog and initializes the data

  1   2   >