Re: pgbench - implement strict TPC-B benchmark

2019-08-27 Thread Fabien COELHO
Hello Dmitry, Well, it could be added? While doing benchmarking using different tools, including pgbench, I found it useful as a temporary hack to add copy freeze and maintenance_work_mem options (the last one not as an env variable, just as a set before, although not sure if it's a best

Re: REINDEX filtering in the backend

2019-08-27 Thread Michael Paquier
On Wed, Aug 28, 2019 at 02:02:08PM +0900, Michael Paquier wrote: > + index = index_open(indexOid, AccessShareLock); > + numAtts = index->rd_index->indnatts; > + index_close(index, AccessShareLock); > Wouldn't it be better to close that after doing the scan? > > Nit: I am pretty

Re: REINDEX filtering in the backend

2019-08-27 Thread Michael Paquier
On Thu, Jul 11, 2019 at 11:14:20PM +0200, Julien Rouhaud wrote: > I didn't want to spend too much time enjoying bison and adding new > unreserved keywords, so for now I just implemented this syntax to > start a discussion for this feature in the next commitfest: > > REINDEX ( FILTER = COLLATION )

Re: Re: Email to hackers for test coverage

2019-08-27 Thread Michael Paquier
On Tue, Aug 27, 2019 at 03:57:20PM +0800, movead...@highgo.ca wrote: > I think your way is much better, so I change the patch and it is > 'regression_20190827.patch' now. Thanks for the new patch, I have committed the part for float4. > There are code lines related to NULL values in >

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2019-08-27 Thread Peter Geoghegan
On Fri, Aug 16, 2019 at 8:56 AM Anastasia Lubennikova wrote: > Now the algorithm is the following: > - In case page split is needed, pass both tuples to _bt_split(). > _bt_findsplitloc() is now aware of upcoming replacement of origtup with > neworigtup, so it uses correct item size where

Re: Cleanup isolation specs from unused steps

2019-08-27 Thread Michael Paquier
On Tue, Aug 27, 2019 at 07:05:50PM +0530, Asim R P wrote: > Thank you for the feedback. I've changed patch 0002 accordingly, please > take another look: > https://www.postgresql.org/message-id/CANXE4TdvSi7Yia_5sV82%2BMHf0WcUSN9u6_X8VEUBv-YStphd%3DQ%40mail.gmail.com Thanks! Let's move the

Re: Statement timeout in pg_rewind

2019-08-27 Thread Michael Paquier
On Tue, Aug 27, 2019 at 10:45:27AM +0200, Alexander Kukushkin wrote: > Done, please see the next version attached. I have made the new error message consistent with run_simple_query to avoid more work to translators and because it is possible to know immediately the code path involved thanks to

Re: pg11.5: ExecHashJoinNewBatch: glibc detected...double free or corruption (!prev)

2019-08-27 Thread Merlin Moncure
On Mon, Aug 26, 2019 at 12:01 PM Tom Lane wrote: > > Justin Pryzby writes: > > On Mon, Aug 26, 2019 at 12:45:24PM -0400, Tom Lane wrote: > >> However ... there is some pretty interesting info at > >> https://bugzilla.redhat.com/show_bug.cgi?id=1338673 > >> suggesting that compiling with a

Re: [PATCH] Make configuration file "include" directive handling more robust

2019-08-27 Thread Ian Barwick
On 8/25/19 4:39 AM, Tom Lane wrote: Ian Barwick writes: On 7/17/19 5:34 PM, Kyotaro Horiguchi wrote:> Hello. I don't think this is new to 12. No, though I'm not sure how much this would be seen as a bugfix and how far back it would be sensible to patch. I think this is worth considering

Re: doc: clarify "pg_signal_backend" default role

2019-08-27 Thread Ian Barwick
On 8/28/19 7:04 AM, Tom Lane wrote: Ian Barwick writes: Currently the documentation for the default role "pg_signal_backend" states, somewhat ambiguously, "Send signals to other backends (eg: cancel query, terminate)", giving the impression other signals (e.g. SIGHUP) can be sent too, which

Re: PostgreSQL and Real Application Testing (RAT)

2019-08-27 Thread Jaime Casanova
On Tue, 27 Aug 2019 at 19:33, Nikolay Samokhvalov wrote: > On Tue, Aug 27, 2019 at 3:47 AM ROS Didier wrote: > >> Hi >> >> >> >> In my business, one of the things blocking the migration from Oracle to >> PostgreSQL is not having the equivalent of Oracle Real Application Testing . >> >> This

Re: PostgreSQL and Real Application Testing (RAT)

2019-08-27 Thread Nikolay Samokhvalov
On Tue, Aug 27, 2019 at 3:47 AM ROS Didier wrote: > Hi > > > > In my business, one of the things blocking the migration from Oracle to > PostgreSQL is not having the equivalent of Oracle Real Application Testing . > > This product captures a charge in production and replay it in a test >

Re: Zedstore - compressed in-core columnar storage

2019-08-27 Thread Alexandra Wang
On Tue, Aug 27, 2019 at 12:03 AM Ashutosh Sharma wrote: > My point is, once we find the leaf page containing the given tid, we go > through each item in the page until we find the data corresponding to the > given tid which means we kind of perform a sequential scan at the page > level. I'm

Re: pg11.5: ExecHashJoinNewBatch: glibc detected...double free or corruption (!prev)

2019-08-27 Thread Merlin Moncure
On Sun, Aug 25, 2019 at 9:35 PM Thomas Munro wrote: > > On Mon, Aug 26, 2019 at 1:44 PM Justin Pryzby wrote: > > On Mon, Aug 26, 2019 at 01:09:19PM +1200, Thomas Munro wrote: > > > On Sun, Aug 25, 2019 at 3:15 PM Peter Geoghegan wrote: > > > > I was reminded of this issue from last year, which

Re: doc: clarify "pg_signal_backend" default role

2019-08-27 Thread Tom Lane
Ian Barwick writes: > Currently the documentation for the default role "pg_signal_backend" states, > somewhat ambiguously, "Send signals to other backends (eg: cancel query, > terminate)", > giving the impression other signals (e.g. SIGHUP) can be sent too, which is > currently not the case. >

Re: row filtering for logical replication

2019-08-27 Thread a . kondratov
Hi Euler, On 2019-02-03 13:14, Andres Freund wrote: On 2018-11-23 13:15:08 -0300, Euler Taveira wrote: Besides the problem presented by Hironobu-san, I'm doing some cleanup and improving docs. I also forget to declare pg_publication_rel TOAST table. Thanks for your review. As far as I can

Re: Missing newline in pg_upgrade usage()

2019-08-27 Thread Tom Lane
Daniel Gustafsson writes: > In 959f6d6a1821 it seems that I fat-fingered and missed a newline in the usage > output, which makes --check become appended to -B rather than on its own line. > The attached diff fixes that. Roger, pushed. regards, tom lane

Missing newline in pg_upgrade usage()

2019-08-27 Thread Daniel Gustafsson
In 959f6d6a1821 it seems that I fat-fingered and missed a newline in the usage output, which makes --check become appended to -B rather than on its own line. The attached diff fixes that. cheers ./daniel pg_upgrade_help_check.diff Description: Binary data

Re: PostgreSQL and Real Application Testing (RAT)

2019-08-27 Thread Jaime Casanova
On Tue, 27 Aug 2019 at 05:47, ROS Didier wrote: > Hi > > > > In my business, one of the things blocking the migration from Oracle to > PostgreSQL is not having the equivalent of Oracle Real Application Testing . > > This product captures a charge in production and replay it in a test >

Re: block-level incremental backup

2019-08-27 Thread Robert Haas
On Fri, Aug 16, 2019 at 6:23 AM Jeevan Chalke wrote: > [ patches ] Reviewing 0002 and 0003: - Commit message for 0003 claims magic number and checksum are 0, but that (fortunately) doesn't seem to be the case. - looks_like_rel_name actually checks whether it looks like a *non-temporary*

basebackup.c's sendFile() ignores read errors

2019-08-27 Thread Robert Haas
While reviewing a proposed patch to basebackup.c this morning, I found myself a bit underwhelmed by the quality of the code and comments in basebackup.c's sendFile(). I believe it's already been pointed out that the the retry logic here is not particularly correct, and the comments demonstrate a

Re: pgbench - implement strict TPC-B benchmark

2019-08-27 Thread Dmitry Dolgov
> On Mon, Aug 5, 2019 at 10:46 PM Fabien COELHO wrote: > > > The index builds are done serially. The vacuum could be replaced by COPY > > FREEZE. > > Well, it could be added? While doing benchmarking using different tools, including pgbench, I found it useful as a temporary hack to add copy

Re: range bug in resolve_generic_type?

2019-08-27 Thread Paul A Jungwirth
On Tue, Aug 27, 2019 at 8:23 AM Tom Lane wrote: > > resolve_generic_type(ANYARRAYOID, x, ANYRANGEOID) - this will return > > an array of the *range type*, but that contracts the normal > > relationship between anyelement and anyrange. It should return an > > array of the range's element type. > >

Re: range bug in resolve_generic_type?

2019-08-27 Thread Tom Lane
Paul A Jungwirth writes: > I was looking at resolve_generic_type to add anymultirange support, > and the range logic doesn't seem correct to me. Hmmm... > resolve_generic_type(ANYARRAYOID, x, ANYRANGEOID) - this will return > an array of the *range type*, but that contracts the normal >

Re: old_snapshot_threshold vs indexes

2019-08-27 Thread Tom Lane
Thomas Munro writes: > On Tue, Aug 27, 2019 at 1:54 PM Tom Lane wrote: >> +1. That fix is also back-patchable, which adding fields to relcache >> entries would not be. > There is a fly in the ointment: REL9_6_STABLE's copy of > RelationHasUnloggedIndex() is hardcoded to return true for hash >

range bug in resolve_generic_type?

2019-08-27 Thread Paul A Jungwirth
Hello, I was looking at resolve_generic_type to add anymultirange support, and the range logic doesn't seem correct to me. This function takes 3 type Oids: - declared_type is the declared type of a function parameter whose actual type it would like to deduce. - context_{declared,actual}_type are

Re: "ago" times on buildfarm status page

2019-08-27 Thread Andrew Dunstan
On 8/27/19 10:15 AM, Tom Lane wrote: > "Tom Turelinckx" writes: >> On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote: >>> I think this is the problem: >>> 'scmrepo' => '/home/pgbf/pgmirror.git', >>> Probably this isn't updated often enough. It probably has little to do with >>> the clock

Re: "ago" times on buildfarm status page

2019-08-27 Thread Tom Lane
"Tom Turelinckx" writes: > On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote: >> I think this is the problem: >> 'scmrepo' => '/home/pgbf/pgmirror.git', >> Probably this isn't updated often enough. It probably has little to do with >> the clock settings. > The configuration is

Re: "ago" times on buildfarm status page

2019-08-27 Thread Andrew Dunstan
On 8/27/19 8:45 AM, Andrew Dunstan wrote: > On 8/27/19 4:33 AM, Tom Turelinckx wrote: >> On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote: >>> I think this is the problem: >>> >>> 'scmrepo' => '/home/pgbf/pgmirror.git', >>> >>> Probably this isn't updated often enough. It probably has

Re: PostgreSQL and Real Application Testing (RAT)

2019-08-27 Thread Thomas Kellerer
ROS Didier schrieb am 27.08.2019 um 12:47: > In my business, one of the things blocking the migration from Oracle > to PostgreSQL is not having the equivalent of Oracle Real Application > Testing . > > This product captures a charge in production and replay it in a test > environment. > > this

Re: Why overhead of SPI is so large?

2019-08-27 Thread Robert Haas
On Sat, Aug 24, 2019 at 12:01 PM David Fetter wrote: > No, it's lying to the RDBMS, so it's pilot error. The problem of > determining from the function itself whether it is in fact immutable > is, in general, equivalent to the Halting Problem, so no, we can't > figure it out. We do need to trust

Re: Cleanup isolation specs from unused steps

2019-08-27 Thread Asim R P
On Fri, Aug 23, 2019 at 9:08 PM Alvaro Herrera wrote: > > On 2019-Aug-23, Asim R P wrote: > > > As part of the fault injector patch set [1], I added a new "blocking" > > keyword to isolation grammar so that a step can be declared as blocking. > > See patch

Re: Fault injection framework

2019-08-27 Thread Asim R P
On Tue, Aug 27, 2019 at 12:35 PM Michael Paquier wrote: > > You may want to double-check whitespaces in your patch set, and 0002 > does not apply because of conflicts in isolationtester.h (my fault!). > I've rebased the patch set against the latest master and tried to resolve whitespace issues.

Re: "ago" times on buildfarm status page

2019-08-27 Thread Andrew Dunstan
On 8/27/19 4:33 AM, Tom Turelinckx wrote: > On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote: >> I think this is the problem: >> >> 'scmrepo' => '/home/pgbf/pgmirror.git', >> >> Probably this isn't updated often enough. It probably has little to do with >> the clock settings. >> >> This

Re: block-level incremental backup

2019-08-27 Thread vignesh C
On Fri, Aug 16, 2019 at 8:07 PM Ibrar Ahmed wrote: > > What do you mean by bigger file, a file greater than 1GB? In which case you > get file > 1GB? > > > Few comments: Comment: + buf = (char *) malloc(statbuf->st_size); + if (buf == NULL) + ereport(ERROR, + (errcode(ERRCODE_OUT_OF_MEMORY), +

PostgreSQL and Real Application Testing (RAT)

2019-08-27 Thread ROS Didier
Hi In my business, one of the things blocking the migration from Oracle to PostgreSQL is not having the equivalent of Oracle Real Application Testing . This product captures a charge in production and replay it in a test environment. this allows to know the impacts of a migration to a newer

Re: Statement timeout in pg_rewind

2019-08-27 Thread Alexander Kukushkin
On Tue, 27 Aug 2019 at 08:36, Michael Paquier wrote: > I'd rather be on the safe side and as we are looking at this at this > area.. Who knows if this logic is going to change in the future and > how it will change. Agree. > Oops, I misread this part. What about a simple wrapper >

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-08-27 Thread Kyotaro Horiguchi
Hello. At Sun, 25 Aug 2019 22:08:43 -0700, Noah Misch wrote in <20190826050843.gb3153...@rfd.leadboat.com> noah> On Thu, Aug 22, 2019 at 09:06:06PM +0900, Kyotaro Horiguchi wrote: noah> > At Mon, 19 Aug 2019 23:03:14 -0700, Noah Misch wrote in <20190820060314.ga3086...@rfd.leadboat.com> > > >

Re: Creating partitions automatically at least on HASH?

2019-08-27 Thread Rafia Sabih
On Mon, 26 Aug 2019 at 19:46, Fabien COELHO wrote: > > Hello Rafia, > > >>CREATE TABLE Stuff (...) > >> PARTITION BY [HASH | RANGE | LIST] (…) > >>DO NONE -- this is the default > >>DO [IMMEDIATE|DEFERRED] USING (…) > >> > >> Where the USING part would be generic keword

Re: "ago" times on buildfarm status page

2019-08-27 Thread Tom Turelinckx
On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote: > I think this is the problem: > > 'scmrepo' => '/home/pgbf/pgmirror.git', > > Probably this isn't updated often enough. It probably has little to do with > the clock settings. > > This is the kind of old-fashioned way of doing things.

Re: pgbench - allow to create partitioned tables

2019-08-27 Thread Asif Rehman
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Thanks. All looks good, making it ready for committer. Regards,

Re: A problem about partitionwise join

2019-08-27 Thread Richard Guo
On Tue, Aug 27, 2019 at 8:51 AM Amit Langote wrote: > Hi Richard, > > On Mon, Aug 26, 2019 at 6:33 PM Richard Guo wrote: > > > > Hi All, > > > > To generate partitionwise join, we need to make sure there exists an > > equi-join condition for each pair of partition keys, which is performed > >

Re: Re: Email to hackers for test coverage

2019-08-27 Thread movead...@highgo.ca
On Tue, 27 Aug 2019 14:07:48 +0800 mich...@paquier.xyz wrote: > There is a section in float4.sql which deals with overflow and > underflow, so wouldn't it be better to move the tests there? You > could just trigger the failures with that: > =# insert into float4_tbl values ('-10e-70'::float8);

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-08-27 Thread Kyotaro Horiguchi
At Tue, 27 Aug 2019 15:49:32 +0900 (Tokyo Standard Time), Kyotaro Horiguchi wrote in <20190827.154932.250364935.horikyota@gmail.com> > 128GB shared buffers contain 16M buffers. On my > perhaps-Windows-Vista-era box, such loop takes 15ms. (Since it > has only 6GB, the test is ignoring the

Re: Optimize single tuple fetch from nbtree index

2019-08-27 Thread Floris Van Nee
>> It seems that it contradicts the very idea of your patch, so probably we >> should look for other ways to optimize this use-case. >> Maybe this restriction can be relaxed for write only tables, that never >> have to reread the page because of visibility, or something like that. >> Also we

Re: Fault injection framework

2019-08-27 Thread Michael Paquier
On Thu, Aug 22, 2019 at 07:45:09PM +0530, Asim R P wrote: > Fault injection was discussed a few months ago at PGCon in Ottawa. At > least a few folks showed interest and so I would like to present what we > have been using in Greenplum. > > The attached patch set contains the fault injector

Re: Zedstore - compressed in-core columnar storage

2019-08-27 Thread Ashutosh Sharma
On Tue, Aug 27, 2019 at 6:03 AM Ashwin Agrawal wrote: > Hope that helps to clarify the confusion. > Thanks for the explanation. Yes, it does clarify my doubt to some extent. My point is, once we find the leaf page containing the given tid, we go through each item in the page until we find the

Re: refactoring - share str2*int64 functions

2019-08-27 Thread Fabien COELHO
Bonjour Michaël, The error returning stuff is simple enough, but I'm unclear about what to do with pg_uint64, which has a totally different signature. Should it be aligned? I am not sure what you mean with aligned here. I meant same signature. If you mean consistent, getting into a state

Re: Statement timeout in pg_rewind

2019-08-27 Thread Michael Paquier
On Mon, Aug 26, 2019 at 03:42:46PM +0200, Alexander Kukushkin wrote: > Well, I was thinking about it and came to the conclusion that we are > neither taking heavy locks nor explicitly opening a transaction and > therefore we can avoid changing them. > But maybe you are right, having them set to

Re: fix "Success" error messages

2019-08-27 Thread Michael Paquier
On Mon, Aug 26, 2019 at 09:40:23PM +0200, Peter Eisentraut wrote: > Here is an updated patch set that rearranges this a bit according to > your suggestions, and also fixes some similar issues in pg_checksums. Thanks for the new patch, and you are right that pg_checksums has been slacking here.

Re: Re: Email to hackers for test coverage

2019-08-27 Thread Michael Paquier
On Mon, Aug 26, 2019 at 05:10:59PM +0800, movead...@highgo.ca wrote: > Thanks for your remind, I have modified the patch and now it is > 'regression_20190826.patch' in attachment, and I have done a successful > test on Cygwin. There is a section in float4.sql which deals with overflow and