Re: [HACKERS] WAL logging problem in 9.4.3?

2019-03-26 Thread Kyotaro HORIGUCHI
Hello. I revised the patch I think addressing all your comments. Differences from v7 patch are: v9-0001: - Renamed the script from 016_ to 017_. - Added some additional tests. v9-0002: - Fixed _bt_blwritepage(). It is re-modified by v9-0007. v9-0003: New patch. - Refactors out xlog s

Re: Re: FETCH FIRST clause WITH TIES option

2019-03-26 Thread Surafel Temesgen
On Mon, Mar 25, 2019 at 11:56 AM David Steele wrote: > This patch no longer passes testing so marked Waiting on Author. > > Thank you for informing. Fixed diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml index 06d611b64c..b3b045ea87 100644 --- a/doc/src/sgml/ref/select.sg

Re: [HACKERS] Block level parallel vacuum

2019-03-26 Thread Kyotaro HORIGUCHI
Hello. At Thu, 21 Mar 2019 15:51:40 -0400, Robert Haas wrote in > On Tue, Mar 19, 2019 at 3:59 AM Kyotaro HORIGUCHI > wrote: > > The leader doesn't continue heap-scan while index vacuuming is > > running. And the index-page-scan seems eat up CPU easily. If > > index vacuum can run simultaneous

RE: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-26 Thread Tsunakawa, Takayuki
From: David Steele [mailto:da...@pgmasters.net] > This patch appears to have been stalled for a while. > > Takayuki -- the ball appears to be in your court. Perhaps it would be > helpful to summarize what you think are next steps? disable_index_cleanup is handled by Sawada-san in another thread.

Re: libpq compression

2019-03-26 Thread Konstantin Knizhnik
On 25.03.2019 13:38, David Steele wrote: On 3/25/19 1:04 PM, Konstantin Knizhnik wrote: On 25.03.2019 11:06, David Steele wrote: Konstantin, This patch appears to be failing tests so I have marked it Waiting on Author. I have also removed the reviewer since no review had been done. Ma

Re: pg_malloc0() instead of pg_malloc()+memset()

2019-03-26 Thread Michael Paquier
On Mon, Mar 25, 2019 at 01:18:05PM +, Daniel Gustafsson wrote: > When reading another codepath, I happened to notice a few codepaths where we > do > pg_malloc() immediately followed by a memset( .. 0, ..), without there being > a > justification (that I can see) for not using pg_malloc0() in

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-03-26 Thread Michael Paquier
On Wed, Mar 20, 2019 at 01:55:51PM +0800, Andrey Borodin wrote: > I'm a bit confused by by console output routines. E.g. in > pg_rewind's main() you call pg_fatal()s, and printf(), and pg_log() > with various levels. Shouldn't we use all the pg_* functions? pg_fatal() would exit immediately, and s

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-03-26 Thread Tsunakawa, Takayuki
From: David Rowley [mailto:david.row...@2ndquadrant.com] > On Mon, 25 Mar 2019 at 23:44, Peter Eisentraut > wrote: > > Perhaps "speeding up planning with partitions" needs to be accepted first? > > Yeah, I think it likely will require that patch to be able to measure > the gains from this patch.

RE: Timeout parameters

2019-03-26 Thread Jamison, Kirk
On Tuesday, March 26, 2019 2:35 PM (GMT+9), Ryohei Nagaura wrote: >> Your patch applies, however in TCP_backend_v10 patch, your >> documentation is missing a closing tag so it could not be >> tested. >> When that's fixed, it passes the make check. >Oops! Fixed. Ok. Confirmed the fix. Minor nit

Re: [HACKERS] Block level parallel vacuum

2019-03-26 Thread Kyotaro HORIGUCHI
Hello. I forgot to mention a point. At Fri, 22 Mar 2019 14:02:36 +0900, Masahiko Sawada wrote in > Attached the updated version patch. 0001 patch allows all existing > vacuum options an boolean argument. 0002 patch introduces parallel > lazy vacuum. 0003 patch adds -P (--parallel) option to vac

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-03-26 Thread Amit Langote
Tsunakawa-san, On 2019/03/26 17:21, Tsunakawa, Takayuki wrote: > From: David Rowley [mailto:david.row...@2ndquadrant.com] >> On Mon, 25 Mar 2019 at 23:44, Peter Eisentraut >> wrote: >>> Perhaps "speeding up planning with partitions" needs to be accepted first? >> >> Yeah, I think it likely will r

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-03-26 Thread David Rowley
On Tue, 26 Mar 2019 at 21:23, Tsunakawa, Takayuki wrote: > Thank you David for explaining. Although I may not understand the effect of > "speeding up planning with partitions" patch, this patch takes effect even > without it. That is, perform the following in the same session: > > 1. SELECT c

Re: Offline enabling/disabling of data checksums

2019-03-26 Thread Michael Paquier
On Sat, Mar 23, 2019 at 02:14:02PM +0100, Fabien COELHO wrote: > Here is an attempt at improving the Notes. > > Mostly it is a reordering from more important (cluster corruption) to less > important (if interrupted a restart is needed), some reordering from problem > to solutions instead of soluti

Re: pg_malloc0() instead of pg_malloc()+memset()

2019-03-26 Thread Daniel Gustafsson
On Tuesday, March 26, 2019 9:00 AM, Michael Paquier wrote: > On Mon, Mar 25, 2019 at 01:18:05PM +, Daniel Gustafsson wrote: > > > When reading another codepath, I happened to notice a few codepaths where > > we do > > pg_malloc() immediately followed by a memset( .. 0, ..), without there >

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-03-26 Thread Pavan Deolasee
On Fri, Mar 22, 2019 at 12:19 PM Masahiko Sawada wrote: > I've looked at the patch and have comments and questions. > > +/* > + * While we are holding the lock on the page, check if all tuples > + * in the page are marked frozen at insertion. We can safely mark > + * such page all

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-03-26 Thread Andrey Lepikhov
On 25/03/2019 15:21, Heikki Linnakangas wrote: On 25/03/2019 09:57, David Steele wrote: On 2/6/19 2:08 PM, Andrey Lepikhov wrote: The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So  lsn field

Re: psql display of foreign keys

2019-03-26 Thread Peter Eisentraut
On 2019-03-26 03:42, Alvaro Herrera wrote: > Patch tester didn't like that one bit. Here's v10 with the fixup > applied. Looks good to me. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: pg_upgrade: Pass -j down to vacuumdb

2019-03-26 Thread Daniel Gustafsson
On Tuesday, March 26, 2019 3:20 AM, Michael Paquier wrote: > On Mon, Mar 25, 2019 at 05:57:50PM -0400, Tom Lane wrote: > > > In short, I'm not convinced that most of this patch is an improvement > > on the status quo. I think we'd be best off to just take the idea > > of explicitly mentioning add

Re: partitioned tables referenced by FKs

2019-03-26 Thread Amit Langote
Hi Alvaro, On 2019/03/22 6:54, Alvaro Herrera wrote: > Here's v7; Needs rebasing on top of 940311e4bb3. 0001: + Oid objectClass = getObjectClass(thisobj); I guess you meant to use ObjectClass, not Oid here. Tested 0002 a bit more and found some problems. cre

Re: jsonpath

2019-03-26 Thread Alexander Korotkov
On Sun, Mar 24, 2019 at 9:09 PM Alexander Korotkov wrote: > On Sun, Mar 24, 2019 at 7:45 PM Andres Freund wrote: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=snapper&dt=2019-03-23%2013%3A01%3A28 > > > > 2019-03-23 14:28:31.147 CET [18056:45] pg_regress/jsonpath LOG: statement: >

Re: Log a sample of transactions

2019-03-26 Thread Adrien NAYRAT
On 3/26/19 1:54 AM, Kuroda, Hayato wrote: Dear David, I have a will and already read the patch, but I thought it's not my turn. Sorry. Hello, Adrien, I did not find any test for log_min_duration that could help me. LCOV indicate there is no tests on this part (look check_log_duration()

RE: speeding up planning with partitions

2019-03-26 Thread Imai, Yoshikazu
Amit-san, On Tue, Mar 26, 2019 at 7:17 AM, Amit Langote wrote: > Rebased patches attached. I could only do the code review of v36-0001. On Mon, Mar 25, 2019 at 11:35 AM, Amit Langote wrote: > On 2019/03/23 6:07, Tom Lane wrote: > > Amit Langote writes: > >> [ v34 patch set ] > > > > I had a bi

Re: Transaction commits VS Transaction commits (with parallel) VS query mean time

2019-03-26 Thread Amit Kapila
On Mon, Mar 25, 2019 at 6:55 PM Haribabu Kommi wrote: > > On Sat, Mar 23, 2019 at 11:10 PM Amit Kapila wrote: >> >> On Sat, Mar 23, 2019 at 9:50 AM Alvaro Herrera >> wrote: >> > >> > On 2019-Mar-23, Amit Kapila wrote: >> > >> > > I think some users might also be interested in the write transact

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-03-26 Thread David Rowley
On Tue, 26 Mar 2019 at 21:55, David Rowley wrote: > > On Tue, 26 Mar 2019 at 21:23, Tsunakawa, Takayuki > wrote: > > Thank you David for explaining. Although I may not understand the effect > > of "speeding up planning with partitions" patch, this patch takes effect > > even without it. That

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-03-26 Thread Heikki Linnakangas
On 26/03/2019 11:29, Andrey Lepikhov wrote: On 25/03/2019 15:21, Heikki Linnakangas wrote: Hmm. When do we create all-zero pages during index build? That seems pretty surprising. GIST uses buffered pages. During GIST build it is possible (very rarely) what no one index tuple was written to the

Re: Psql patch to show access methods info

2019-03-26 Thread Kyotaro HORIGUCHI
Thank you for the new version. At Fri, 22 Mar 2019 21:29:09 +0300, Sergey Cherkashin wrote in > Taking into account the wishes of all the reviewers, the current > position of the patch is as follows: > > The \dA command displays a list of access methods. > > # \dA > List of access me

Re: Ordered Partitioned Table Scans

2019-03-26 Thread Julien Rouhaud
On Tue, Mar 26, 2019 at 3:13 AM David Rowley wrote: > > On Tue, 26 Mar 2019 at 09:02, Julien Rouhaud wrote: > > FTR this patch doesn't apply since single child [Merge]Append > > suppression (8edd0e7946) has been pushed. > > Thanks for letting me know. I've attached v14 based on current master.

Re: pg_upgrade: Pass -j down to vacuumdb

2019-03-26 Thread Peter Eisentraut
On 2019-03-25 22:57, Tom Lane wrote: > + fprintf(script, "echo %sYou may wish to add --jobs=N for parallel > analyzing.%s\n", > + ECHO_QUOTE, ECHO_QUOTE); But then you get that information after you have already started the script. I don't find any information about this

Improvement of installation document

2019-03-26 Thread Yugo Nagata
Hi, One of our clients suggested that the installation document[1] lacks description about requriements of installing *-devel packages. For example, postgresqlxx-devel is required for using --with-pgsql, and openssl-devel for --with-openssl, and so on, but these are not documented. [1] http://w

Re: Improvement of installation document

2019-03-26 Thread Yugo Nagata
Hi, I apologize that I accidentally sent the following email to this list. Please disregard this. I am sorry for making a lot of noise. Regard, On Tue, 26 Mar 2019 20:38:31 +0900 Yugo Nagata wrote: > Hi, > > One of our clients suggested that the installation document[1] lacks > description

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Dean Rasheed
On Mon, 25 Mar 2019 at 23:36, Tomas Vondra wrote: > > Attached is an updated patch, fixing all the issues pointed out so far. > Unless there are some objections, I plan to commit the 0001 part by the > end of this CF. Part 0002 is a matter for PG13, as previously agreed. > Yes, I think that's rea

Re: FETCH FIRST clause PERCENT option

2019-03-26 Thread Surafel Temesgen
On Thu, Feb 28, 2019 at 11:16 PM Tomas Vondra wrote: > > To give you a (admittedly, somewhat contrived and artificial example): > > SELECT * FROM t1 WHERE id IN ( > SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY > ); > > Maybe this example is bogus and/or does not rea

Re: warning to publication created and wal_level is not set to logical

2019-03-26 Thread Lucas Viecelli
>> One idea that might be useful is to have walsenders refuse to transmit > >> any logical-replication data if they see wal_level is too low. That > >> would get users' attention pretty quickly. > > > They do: > I checked this before creating the patch > > Oh, OK, then this seems like it's basi

Re: Problems with plan estimates in postgres_fdw

2019-03-26 Thread Etsuro Fujita
(2019/03/20 20:47), Etsuro Fujita wrote: Attached is an updated version of the patch set. One thing I noticed while self-reviewing the patch for UPPERREL_FINAL is: the previous versions of the patch don't show EPQ plans in EXPLAIN, as shown in the below example, so we can't check if those pla

Should the docs have a warning about pg_stat_reset()?

2019-03-26 Thread David Rowley
As I mentioned in [1], I've had a few cases recently about auto-vacuum not working. On the other thread, it was all about auto-vacuum being configured to run too slowly. The other culprit for auto-vacuum not working is when people periodically use pg_stat_reset(). The problem with pg_stat_reset()

Re: pg_upgrade: Pass -j down to vacuumdb

2019-03-26 Thread Tom Lane
Peter Eisentraut writes: > On 2019-03-25 22:57, Tom Lane wrote: >> +fprintf(script, "echo %sYou may wish to add --jobs=N for parallel >> analyzing.%s\n", >> +ECHO_QUOTE, ECHO_QUOTE); > But then you get that information after you have already started the script. Yes, but

Re: partitioned tables referenced by FKs

2019-03-26 Thread Jesper Pedersen
Hi Amit, On 3/26/19 2:06 AM, Amit Langote wrote: Wouldn't you get the same numbers on HEAD too? IOW, I'm not sure how the patch here, which seems mostly about getting DDL in order to support foreign keys on partitioned tables, would have affected the result of this benchmark. Can you clarify y

Re: Improvement of installation document

2019-03-26 Thread Tatsuo Ishii
> One of our clients suggested that the installation document[1] lacks > description > about requriements of installing *-devel packages. For example, > postgresqlxx-devel > is required for using --with-pgsql, and openssl-devel for --with-openssl, and > so on, > but these are not documented. >

Re: Refactoring the checkpointer's fsync request queue

2019-03-26 Thread Robert Haas
On Tue, Mar 26, 2019 at 12:20 AM Thomas Munro wrote: > I've been trying to decide if that is a problem. Maybe there is a > performance angle, and I'm also wondering if it might increase the > risk of missing a write-back error. Of course we'll find a proper > solution to that problem (perhaps fd

Re: Offline enabling/disabling of data checksums

2019-03-26 Thread Fabien COELHO
Bonjour Michaël, Here is an attempt at improving the Notes. [...] So, the ordering of the notes for each paragraph is as follows: 1) Replication issues when mixing different checksum setups across nodes. 2) Consistency of the operations if killed. 3) Don't start Postgres while the operatio

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-26 Thread Robert Haas
On Tue, Mar 26, 2019 at 3:57 AM Tsunakawa, Takayuki wrote: > From: David Steele [mailto:da...@pgmasters.net] > > This patch appears to have been stalled for a while. > > > > Takayuki -- the ball appears to be in your court. Perhaps it would be > > helpful to summarize what you think are next step

Re: Special role for subscriptions

2019-03-26 Thread Robert Haas
On Thu, Mar 21, 2019 at 9:28 PM Michael Paquier wrote: > By the way, as the commit fest is coming to its end in a couple of > days, and that we are still discussing how the thing should be shaped, > I would recommend to mark the patch as returned with feedback. Any > objections with that? +1. I

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Dean Rasheed
On Tue, 26 Mar 2019 at 11:59, Dean Rasheed wrote: > > On Mon, 25 Mar 2019 at 23:36, Tomas Vondra > wrote: > > > > Attached is an updated patch... > > I just looked through the latest set of changes and I have a couple of > additional review comments: > I just spotted another issue while reading

Re: Misleading errors with column references in default expressions and partition bounds

2019-03-26 Thread Tom Lane
Michael Paquier writes: > One idea which came from Amit, and it seems to me that it is a good > idea, would be to have more context-related error messages directly in > transformColumnRef(), so as we can discard at an early stage column > references which are part of contexts where there is no mea

Re: Shouldn't current_schema() be at least PARALLEL RESTRICTED?

2019-03-26 Thread Daniel Gustafsson
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Reviewing the codepath in question (as well as the commit that change

Re: jsonpath

2019-03-26 Thread Tom Lane
Alexander Korotkov writes: > Got access to that buildfarm animal thanks to Tom Turelinckx. Now > running check-world in a loop on the same commit hash with same build > options. Error wasn't triggered yet. I notice that snapper is using force_parallel_mode = regress ... have you got that enable

Re: [HACKERS] Block level parallel vacuum

2019-03-26 Thread Masahiko Sawada
On Tue, Mar 26, 2019 at 10:19 AM Haribabu Kommi wrote: > > > On Fri, Mar 22, 2019 at 4:06 PM Masahiko Sawada wrote: >> >> >> Attached the updated version patch. 0001 patch allows all existing >> vacuum options an boolean argument. 0002 patch introduces parallel >> lazy vacuum. 0003 patch adds -P

Re: psql display of foreign keys

2019-03-26 Thread Alvaro Herrera
On 2019-Mar-26, Peter Eisentraut wrote: > On 2019-03-26 03:42, Alvaro Herrera wrote: > > Patch tester didn't like that one bit. Here's v10 with the fixup > > applied. > > Looks good to me. Thanks! I ran "make installcheck-parallel" using this psql version on all supported branches plus 9.2. T

Re: FETCH FIRST clause PERCENT option

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 03:06:52PM +0300, Surafel Temesgen wrote: On Thu, Feb 28, 2019 at 11:16 PM Tomas Vondra wrote: To give you a (admittedly, somewhat contrived and artificial example): SELECT * FROM t1 WHERE id IN ( SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY

Re: libpq compression

2019-03-26 Thread Konstantin Knizhnik
Version of the patch correctly working when no compression algorithm are avaiable. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/configure b/configure index 8068108..8ebd961 100755 --- a/configure +++ b/configure @@ -701,6 +

Re: jsonpath

2019-03-26 Thread Alexander Korotkov
On Tue, Mar 26, 2019 at 5:32 PM Tom Lane wrote: > Alexander Korotkov writes: > > Got access to that buildfarm animal thanks to Tom Turelinckx. Now > > running check-world in a loop on the same commit hash with same build > > options. Error wasn't triggered yet. > > I notice that snapper is usin

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 11:59:56AM +, Dean Rasheed wrote: On Mon, 25 Mar 2019 at 23:36, Tomas Vondra wrote: Attached is an updated patch, fixing all the issues pointed out so far. Unless there are some objections, I plan to commit the 0001 part by the end of this CF. Part 0002 is a matter

Re: New vacuum option to do only freezing

2019-03-26 Thread Masahiko Sawada
On Sat, Mar 23, 2019 at 3:25 AM Robert Haas wrote: > > On Fri, Mar 8, 2019 at 12:14 AM Masahiko Sawada wrote: > > IIUC we've discussed the field-and-value style vacuum option. I > > suggested that since we have already the disable_page_skipping option > > the disable_page_skipping option would be

Re: Enable data checksums by default

2019-03-26 Thread Christoph Berg
Re: Tom Lane 2019-03-22 <4368.1553270...@sss.pgh.pa.us> > Christoph Berg writes: > > I think, the next step in that direction would be to enable data > > checksums by default. They make sense in most setups, > > Well, that is exactly the point that needs some proof, not just > an unfounded assert

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 01:37:33PM +, Dean Rasheed wrote: On Tue, 26 Mar 2019 at 11:59, Dean Rasheed wrote: On Mon, 25 Mar 2019 at 23:36, Tomas Vondra wrote: > > Attached is an updated patch... I just looked through the latest set of changes and I have a couple of additional review comme

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-26 Thread Masahiko Sawada
On Tue, Mar 26, 2019 at 10:30 PM Robert Haas wrote: > > On Tue, Mar 26, 2019 at 3:57 AM Tsunakawa, Takayuki > wrote: > > From: David Steele [mailto:da...@pgmasters.net] > > > This patch appears to have been stalled for a while. > > > > > > Takayuki -- the ball appears to be in your court. Perhap

Re: Should the docs have a warning about pg_stat_reset()?

2019-03-26 Thread Euler Taveira
Em ter, 26 de mar de 2019 às 09:54, David Rowley escreveu: > > As I mentioned in [1], I've had a few cases recently about auto-vacuum > not working. On the other thread, it was all about auto-vacuum being > configured to run too slowly. The other culprit for auto-vacuum not > working is when peop

Re: MSVC Build support with visual studio 2019

2019-03-26 Thread Andrew Dunstan
On 3/20/19 8:36 PM, Haribabu Kommi wrote: > Hi Hackers, > > Here I attached a patch that supports building of PostgreSQL with VS 2019. > VS 2019 is going to release on Apr 2nd 2019, it will be good if version 12 > supports compiling. The attached for is for review, it may needs some > updates > o

Re: speeding up planning with partitions

2019-03-26 Thread Tom Lane
Amit Langote writes: > 0002 is a new patch to get rid of the duplicate RTE and PlanRowMark that's > created for partitioned parent table, as it's pointless. I was planning > to propose it later, but decided to post it now if we're modifying the > nearby code anyway. Good idea, but it needs a bit

[PATCH][HOTFIX] vacuum_cost_delay type change from int to real have not been done everywhere

2019-03-26 Thread Nikolay Shaplov
Hi! In caf626b2 type of vacuum_cost_delay have been switched from int to real, everywhere, but not in *RelOpts[] arrays. For some reason it continued to build and work. But I think it is better to move vacuum_cost_delay from int to real there too... Patch attached. PS. As you can see current

Re: Tid scan improvements

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 19:11:13 +1300, Edmund Horner wrote: > The changes in heapam.c were required for backward scan support, as > used by ORDER BY ctid DESC and MAX(ctid); and also for FETCH LAST and > FETCH PRIOR. I have removed the backward scans functionality from the > current set of patches, bu

Re: PostgreSQL Participates in GSoC 2019!

2019-03-26 Thread Ritom Sonowal
Hi, I have applied and submitted an initial draft of my proposal for GSoC 2019 through the Summer of Code site. The project is titled 'pgAdmin4 Graphing Query Tool'. I would like to get some feedback for the same so that I can improve on making the final proposal better. The link to the draft is

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Alvaro Herrera
On 2019-Mar-26, Alvaro Herrera wrote: > > 2019-03-26 00:49:02.208 EDT [5c99ae9e.20cc:6] LOG: server process (PID > > 8368) was terminated by exception 0xC028 > > 0xC028 is STATUS_BAD_STACK, per > https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9b

basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, As detailed in https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de the way the backend's basebackup checksum verification works makes its error detection capabilities very dubious. I think we need to fix this before the next set of backbranch releases, or at the very least

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Andrew Dunstan
On 3/26/19 12:53 PM, Alvaro Herrera wrote: > On 2019-Mar-26, Alvaro Herrera wrote: > >>> 2019-03-26 00:49:02.208 EDT [5c99ae9e.20cc:6] LOG: server process (PID >>> 8368) was terminated by exception 0xC028 >> 0xC028 is STATUS_BAD_STACK, per >> https://docs.microsoft.com/en-us/openspecs/w

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Tom Lane
Alvaro Herrera writes: >> 0xC028 is STATUS_BAD_STACK, per >> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55 >> Not sure how credible/useful a stack trace is going to be. > BTW I think we should update our message to use this URL inst

[Patch] Base backups and random or zero pageheaders (was: Online verification of checksums)

2019-03-26 Thread Michael Banck
Hi, Am Dienstag, den 19.03.2019, 13:00 -0700 schrieb Andres Freund: > CREATE TABLE corruptme AS SELECT g.i::text AS data FROM generate_series(1, > 100) g(i); > SELECT pg_relation_size('corruptme'); > postgres[22890][1]=# SELECT current_setting('data_directory') || '/' || > pg_relation_filepa

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-26 Thread Robert Haas
On Tue, Mar 26, 2019 at 11:23 AM Masahiko Sawada wrote: > > I don't see a patch with the naming updated, here or there, and I'm > > going to be really unhappy if we end up with inconsistent naming > > between two patches that do such fundamentally similar things. -1 > > from me to committing eith

Re: [Patch] Base backups and random or zero pageheaders (was: Online verification of checksums)

2019-03-26 Thread Andres Freund
On 2019-03-26 18:22:55 +0100, Michael Banck wrote: > Hi, > > Am Dienstag, den 19.03.2019, 13:00 -0700 schrieb Andres Freund: > > CREATE TABLE corruptme AS SELECT g.i::text AS data FROM generate_series(1, > > 100) g(i); > > SELECT pg_relation_size('corruptme'); > > postgres[22890][1]=# SELECT

Re: [PATCH][HOTFIX] vacuum_cost_delay type change from int to real have not been done everywhere

2019-03-26 Thread Tom Lane
Nikolay Shaplov writes: > In caf626b2 type of vacuum_cost_delay have been switched from int to real, > everywhere, but not in *RelOpts[] arrays. Ugh. > For some reason it continued to build and work. I'm not quite sure why it worked either; apparently, the type of that array entry doesn't have

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Alvaro Herrera
On 2019-Mar-26, Tom Lane wrote: > Alvaro Herrera writes: > >> 0xC028 is STATUS_BAD_STACK, per > >> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55 > >> Not sure how credible/useful a stack trace is going to be. > > > BTW I think we s

Re: [Patch] Base backups and random or zero pageheaders (was: Online verification of checksums)

2019-03-26 Thread Michael Banck
Hi, Am Dienstag, den 26.03.2019, 10:30 -0700 schrieb Andres Freund: > On 2019-03-26 18:22:55 +0100, Michael Banck wrote: > > Am Dienstag, den 19.03.2019, 13:00 -0700 schrieb Andres Freund: > > > CREATE TABLE corruptme AS SELECT g.i::text AS data FROM > > > generate_series(1, 100) g(i); > > >

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Mar-26, Tom Lane wrote: >> Alvaro Herrera writes: >>> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55 >> I've never cared for the ntstatus.h reference, but how stable is >> the URL you suggest going to be

Re: [HACKERS] generated columns

2019-03-26 Thread Pavel Stehule
Hi út 26. 3. 2019 v 14:33 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2019-03-20 03:51, Michael Paquier wrote: > > On Mon, Mar 18, 2019 at 03:14:09PM +0100, Pavel Stehule wrote: > >> postgres=# update foo set name = 'bbbxx' where id = 1; -- error > >> ERROR: no g

Re: Enable data checksums by default

2019-03-26 Thread Peter Geoghegan
On Fri, Mar 22, 2019 at 9:07 AM Tom Lane wrote: > IMO, the main value of checksums is that they allow the Postgres > project to deflect blame. That's nice for us but I'm not sure > that it's a benefit for users. I've seen little if any data to > suggest that checksums actually catch enough probl

Re: patch to allow disable of WAL recycling

2019-03-26 Thread Jerry Jelinek
On Thu, Mar 7, 2019 at 6:26 PM Thomas Munro wrote: > On Fri, Mar 8, 2019 at 12:35 PM Jerry Jelinek > wrote: > > On Thu, Mar 7, 2019 at 3:09 PM Thomas Munro > wrote: > >> My understanding is that it's not really the COW-ness that makes it > >> not necessary, it's the fact that fdatasync() doesn'

Re: [HACKERS] generated columns

2019-03-26 Thread Pavel Stehule
út 26. 3. 2019 v 19:52 odesílatel Pavel Stehule napsal: > Hi > > út 26. 3. 2019 v 14:33 odesílatel Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> napsal: > >> On 2019-03-20 03:51, Michael Paquier wrote: >> > On Mon, Mar 18, 2019 at 03:14:09PM +0100, Pavel Stehule wrote: >> >> postgres=# u

Re: [HACKERS] generated columns

2019-03-26 Thread Pavel Stehule
Hi út 26. 3. 2019 v 14:33 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2019-03-20 03:51, Michael Paquier wrote: > > On Mon, Mar 18, 2019 at 03:14:09PM +0100, Pavel Stehule wrote: > >> postgres=# update foo set name = 'bbbxx' where id = 1; -- error > >> ERROR: no g

Re: explain plans with information about (modified) gucs

2019-03-26 Thread Tomas Vondra
On Mon, Mar 18, 2019 at 11:31:48AM +0100, Rafia Sabih wrote: On Sun, 24 Feb 2019 at 00:06, Tomas Vondra wrote: Hi, attached is an updated patch, fixing and slightly tweaking the docs. Barring objections, I'll get this committed later next week. I was having a look at this patch, and this

Re: Usage of epoch in txid_current

2019-03-26 Thread Thomas Munro
On Tue, Mar 26, 2019 at 12:58 PM Thomas Munro wrote: > ... I think you could probably reclaim that space by > using a more compact representation of vacuumFlags, overflowed, > delayChkpt, nxids (it's funny, the comment says "as tightly as > possible", which clearly isn't the case). Woops, I take

Re: partitioned tables referenced by FKs

2019-03-26 Thread Alvaro Herrera
Hello Amit On 2019-Mar-26, Amit Langote wrote: > + Oid objectClass = getObjectClass(thisobj); > > I guess you meant to use ObjectClass, not Oid here. Absolutely. > Tested 0002 a bit more and found some problems. Thanks for the thorough testing and bug analysis!

Re: Fix XML handling with DOCTYPE

2019-03-26 Thread Ryan Lambert
Ok, I'll give it a go. > If you happened to feel moved to look over a documentation patch, that > would be what this CF entry most needs in the waning days of the > commitfest. Is the xml-functions-type-docfix-4.patch [1] the one needing review? I'll test applying it and review the changes in

Re: speeding up planning with partitions

2019-03-26 Thread Tom Lane
Amit Langote writes: > On 2019/03/23 6:07, Tom Lane wrote: >> I also feel like you used a dartboard while deciding where to insert the >> call in query_planner(); dropping it into the middle of a sequence of >> equivalence-class-related operations seems quite random. Maybe we could >> delay that

Re: Fix XML handling with DOCTYPE

2019-03-26 Thread Tom Lane
Ryan Lambert writes: > Is the xml-functions-type-docfix-4.patch [1] the one needing review? I'll > test applying it and review the changes in better detail. Is there a > section in the docs that shows how to verify if the updated pages render > properly? I would assume the pages are build when

Fwd: Gsoc proposal perffarn

2019-03-26 Thread Victor Kukshiev
Hello, my name is Victor Kuvshiev. Currently I'm third-year student of Petrozavodsk State University, studying information systems and technologies. I have relatively good knowledge of HTML, CSS and Python also have some skills in javascript language. example of my works: ruletka, console game i

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > As detailed in > https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > the way the backend's basebackup checksum verification works makes its > error detection capabilities very dubious. I disagree that it's 'very dubiou

RE: Planning counters in pg_stat_statements (using pgss_store)

2019-03-26 Thread legrand legrand
here is a new version: - "track_planning" GUC added to permit to keep previous behavior unchanged - columns names have been changed / added: total_plan_time, total_exec_time, total_time - trailing whitespaces and comments wider than 80 characters not fixed

Re: partitioned tables referenced by FKs

2019-03-26 Thread Alvaro Herrera
On 2019-Mar-26, Alvaro Herrera wrote: > Thanks for the thorough testing and bug analysis! It was spot-on. I've > applied your two proposed fixes, as well as added a new test setup that > covers both these bugs. The attached set is rebased on 7c366ac969ce. Attached is rebased on 126d63122232.

Re: basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 19:22:03 -0400, Stephen Frost wrote: > Greetings, > > * Andres Freund (and...@anarazel.de) wrote: > > As detailed in > > https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > > the way the backend's basebackup checksum verification works makes its > > error d

Re: basebackup checksum verification

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 04:49:21PM -0700, Andres Freund wrote: Hi, On 2019-03-26 19:22:03 -0400, Stephen Frost wrote: Greetings, * Andres Freund (and...@anarazel.de) wrote: > As detailed in > https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > the way the backend's baseba

Re: Pluggable Storage - Andres's take

2019-03-26 Thread Andres Freund
Hi, On 2019-02-22 14:52:08 -0500, Robert Haas wrote: > On Fri, Feb 22, 2019 at 11:19 AM Amit Khandekar > wrote: > > Thanks for the review. Attached v2. > > Thanks. I took this, combined it with Andres's > v12-0040-WIP-Move-xid-horizon-computation-for-page-level-.patch, did > some polishing of

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Tue, Mar 26, 2019 at 04:49:21PM -0700, Andres Freund wrote: > >On 2019-03-26 19:22:03 -0400, Stephen Frost wrote: > >>* Andres Freund (and...@anarazel.de) wrote: > >>> As detailed in > >>> https://postgr.es/m/20190319200050.ncuxe

Re: Ordered Partitioned Table Scans

2019-03-26 Thread David Rowley
Thanks for having another look. On Wed, 27 Mar 2019 at 00:22, Julien Rouhaud wrote: > A few, mostly nitpicking, comments: > > + if (rel->part_scheme != NULL && IS_SIMPLE_REL(rel) && > + partitions_are_ordered(root, rel)) > > shouldn't the test be IS_PARTITIONED_REL(rel) instead of testing

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-03-26 Thread Tsunakawa, Takayuki
From: Amit Langote [mailto:langote_amit...@lab.ntt.co.jp] > My understanding of what David wrote is that the slowness of bloated hash > table is hard to notice, because planning itself is pretty slow. With the > "speeding up planning with partitions" patch, planning becomes quite fast, > so the bl

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 1:06, Tom Lane wrote: > Amit Langote writes: >> 0002 is a new patch to get rid of the duplicate RTE and PlanRowMark that's >> created for partitioned parent table, as it's pointless. I was planning >> to propose it later, but decided to post it now if we're modifying the >> nearby c

Re: basebackup checksum verification

2019-03-26 Thread Peter Geoghegan
On Tue, Mar 26, 2019 at 5:10 PM Tomas Vondra wrote: > Bogus might be a bit too harsh, but yeah - failure to reliably detect > obviously > invalid checksums when the LSN just happens to be high due to randomness is > not > a good thing. We'll still detect pages corrupted in other places, but this

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-03-26 Thread Tsunakawa, Takayuki
From: David Rowley [mailto:david.row...@2ndquadrant.com] > Here a benchmark doing that using pgbench's script weight feature. Wow, I didn't know that pgbench has evolved to have such a convenient feature. Thanks for telling me how to utilize it in testing. PostgreSQL is cool! Regards Takayuki

Re: basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 20:18:31 -0400, Stephen Frost wrote: > > >>I thought Robert's response was generally good, pointing out that > > >>we're talking about this being an issue if the corruption happens in a > > >>certain set of bytes. That said, I'm happy to see improvements in > > >>this area but I

minimizing pg_stat_statements performance overhead

2019-03-26 Thread Raymond Martin
Hello hackers, This email is regarding the Postgres pg_stat_statements extension. I noticed that enabling pg_stat_statements can effect performance. I thought that changing the pg_stat_statements.track parameter to 'none' could reduce this overhead without requiring a restart to remove it from

Re: basebackup checksum verification

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 08:18:31PM -0400, Stephen Frost wrote: Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: On Tue, Mar 26, 2019 at 04:49:21PM -0700, Andres Freund wrote: >On 2019-03-26 19:22:03 -0400, Stephen Frost wrote: >>* Andres Freund (and...@anarazel.de) wrote: >>> As

  1   2   >