Re: [HACKERS] Cache Hash Index meta page.

On Tue, Sep 13, 2016 at 12:55 PM, Mithun Cy wrote: > On Thu, Sep 8, 2016 at 11:21 PM, Jesper Pedersen < > jesper.peder...@redhat.com> wrote: >> >> > For the archives, this patch conflicts with the WAL patch [1]. >> >> > [1] https://www.postgresql.org/message-id/CAA4eK1JS%2BSiRSQBzEFp >> nsSmxZKin

Re: [HACKERS] pageinspect: Hash index support

On Tue, Sep 20, 2016 at 12:19 AM, Michael Paquier wrote: > > Note: the patch checks if a superuser is calling the new functions, > which is a good thing. > If we only have the bytea functions and the user needs to supply the raw pages themselves, rather than having the function go get the raw pa

Re: [HACKERS] pageinspect: Hash index support

On Mon, Sep 26, 2016 at 10:39 AM, Jesper Pedersen < jesper.peder...@redhat.com> wrote: > Hi, > > On 09/23/2016 12:10 AM, Peter Eisentraut wrote: > >> >> > - As of very recently, we don't need to move pageinspect--1.5.sql to >> pageinspect--1.6.sql anymore. Just add pageinspect--1.5--1.6.sql. >>

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Tue, Sep 20, 2016 at 10:27 PM, Amit Kapila wrote: > On Tue, Sep 20, 2016 at 10:24 PM, Jeff Janes wrote: > > On Thu, Sep 15, 2016 at 11:42 PM, Amit Kapila > > wrote: > >> > >> > >> Okay, Thanks for pointing out the same. I have fixed

Re: [HACKERS] Hash Indexes

On Wed, Sep 21, 2016 at 12:44 PM, Geoff Winkless wrote: > On 21 September 2016 at 13:29, Robert Haas wrote: > > I'd be curious what benefits people expect to get. > > An edge case I came across the other day was a unique index on a large > string: postgresql popped up and told me that I couldn't

Re: [HACKERS] Hash Indexes

On Thu, Sep 15, 2016 at 7:13 AM, Robert Haas wrote: > On Thu, Sep 15, 2016 at 1:41 AM, Amit Kapila > wrote: > > I think it is possible without breaking pg_upgrade, if we match all > > items of a page at once (and save them as local copy), rather than > > matching item-by-item as we do now. We a

Re: [HACKERS] pageinspect: Hash index support

On Tue, Sep 20, 2016 at 11:14 PM, Michael Paquier wrote: > > + > + The type information will be 'm' for a metadata > page, > + 'v' for an overflow page, > 'b' for a bucket page, > + 'i' for a bitmap page, and > 'u' for an unused page. > + > > Other functions don't go in

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Thu, Sep 15, 2016 at 11:42 PM, Amit Kapila wrote: > > Okay, Thanks for pointing out the same. I have fixed it. Apart from > that, I have changed _hash_alloc_buckets() to initialize the page > instead of making it completely Zero because of problems discussed in > another related thread [1].

Re: [HACKERS] pageinspect: Hash index support

On Tue, Sep 20, 2016 at 5:40 AM, Jesper Pedersen wrote: > On 09/20/2016 03:19 AM, Michael Paquier wrote: > >> You did not get right the comments from Alvaro upthread. The following >> functions are added with this patch: >> function hash_metap(text) >> function hash_metap_bytea(bytea) >> funct

Re: [HACKERS] Hash Indexes

On Sun, Sep 18, 2016 at 11:44 PM, Amit Kapila wrote: > On Mon, Sep 19, 2016 at 11:20 AM, Mark Kirkwood > wrote: > > On 17/09/16 06:38, Andres Freund wrote: > > > > While I see the point of what you are saying here, I recall all previous > > discussions about has indexes tended to go a bit like t

Re: [HACKERS] README of hash index

On Fri, Sep 16, 2016 at 4:20 AM, Amit Kapila wrote: > Currently README of hash module contain algorithms written in below form. > > The insertion algorithm is rather similar: > > pin meta page and take buffer content lock in shared mode > loop: > compute bucket number for target hash key > releas

Re: [HACKERS] Hash Indexes

On Thu, Sep 15, 2016 at 7:23 AM, Andres Freund wrote: > Hi, > > On 2016-05-10 17:39:22 +0530, Amit Kapila wrote: > > For making hash indexes usable in production systems, we need to improve > > its concurrency and make them crash-safe by WAL logging them. > > One earlier question about this is wh

Re: [HACKERS] Hash Indexes

On Tue, May 10, 2016 at 5:09 AM, Amit Kapila wrote: > > > Although, I don't think it is a very good idea to take any performance > data with WIP patch, still I couldn't resist myself from doing so and below > are the performance numbers. To get the performance data, I have dropped > the primary

Re: [HACKERS] Hash Indexes

On Tue, Sep 13, 2016 at 9:31 AM, Jeff Janes wrote: > === > > +Vacuum acquires cleanup lock on bucket to remove the dead tuples and or > tuples > +that are moved due to split. The need for cleanup lock to remove dead > tuples > +is to ensure that scans' returns c

Re: [HACKERS] pageinspect: Hash index support

On Tue, Aug 30, 2016 at 10:06 AM, Alvaro Herrera wrote: > Jesper Pedersen wrote: > > Hi, > > > > Attached is a patch that adds support for hash indexes in pageinspect. > > > > The functionality (hash_metap, hash_page_stats and hash_page_items) > follows > > the B-tree functions. > > I suggest tha

Re: [HACKERS] Hash Indexes

On Wed, Sep 7, 2016 at 9:32 PM, Amit Kapila wrote: > On Wed, Sep 7, 2016 at 11:49 PM, Jeff Janes wrote: > > On Thu, Sep 1, 2016 at 8:55 PM, Amit Kapila > wrote: > >> > >> > >> I have fixed all other issues you have raised. Updated patch is > &g

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Sun, Sep 11, 2016 at 7:40 PM, Amit Kapila wrote: > On Mon, Sep 12, 2016 at 7:00 AM, Jeff Janes wrote: > > On Thu, Sep 8, 2016 at 12:09 PM, Jeff Janes > wrote: > > > >> > >> I plan to do testing using my own testing harness after changing it to > >

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Thu, Sep 8, 2016 at 12:09 PM, Jeff Janes wrote: > I plan to do testing using my own testing harness after changing it to > insert a lot of dummy tuples (ones with negative values in the pseudo-pk > column, which are never queried by the core part of the harness) and > deleting th

Re: [HACKERS] feature request: explain "with details" option

On Thu, Sep 8, 2016 at 10:40 AM, Roger Pack wrote: > My apologies if this was already requested before... > > I think it would be fantastic if postgres had an "explain the explain" > option: > Today's explain tells us what loops and scans were used, and relative > costs, etc. It doesn't seem to

Re: [HACKERS] High-CPU consumption on information_schema (only) query

On Wed, Sep 7, 2016 at 4:37 PM, Robins Tharakan wrote: > > Hi, > > An SQL (with only information_schema related JOINS) when triggered, runs with max CPU (and never ends - killed after 2 days). > - It runs similarly (very slow) on a replicated server that acts as a read-only slave. > - Top shows

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Wed, Sep 7, 2016 at 3:29 AM, Ashutosh Sharma wrote: > > Thanks to Ashutosh Sharma for doing the testing of the patch and > > helping me in analyzing some of the above issues. > > Hi All, > > I would like to summarize the test-cases that i have executed for > validating WAL logging in hash inde

Re: [HACKERS] Hash Indexes

On Thu, Sep 1, 2016 at 8:55 PM, Amit Kapila wrote: > > I have fixed all other issues you have raised. Updated patch is > attached with this mail. > I am finding the comments (particularly README) quite hard to follow. There are many references to an "overflow bucket", or similar phrases. I thi

Re: [HACKERS] restoration after crash slowness, any way to improve?

On Wed, Aug 31, 2016 at 6:26 PM, Joshua D. Drake wrote: > -hackers, > > So this is more of a spit balling thread than anything. As I understand > it, if we have a long running transaction or a large number of wal logs and > we crash, we then have to restore those logs on restart to the last known

Re: [HACKERS] Notice lock waits

On Tue, Aug 9, 2016 at 5:17 PM, Jim Nasby wrote: > On 8/5/16 12:00 PM, Jeff Janes wrote: > >> So I created a new guc, notice_lock_waits, which acts like >> log_lock_waits but sends the message as NOTICE so it will show up on >> interactive connections like psql. >&g

Re: [HACKERS] distinct estimate of a hard-coded VALUES list

On Mon, Aug 22, 2016 at 10:19 AM, Robert Haas wrote: > On Sat, Aug 20, 2016 at 4:58 PM, Tom Lane wrote: > > Jeff Janes writes: > >> On Thu, Aug 18, 2016 at 2:25 PM, Tom Lane wrote: > >>> It does know it, what it doesn't know is how many duplicates there are

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Tue, Aug 23, 2016 at 10:05 PM, Amit Kapila wrote: > On Wed, Aug 24, 2016 at 2:37 AM, Jeff Janes wrote: > > > > > After an intentionally created crash, I get an Assert triggering: > > > > TRAP: FailedAssertion("!(((freep)[(bitmapbit)/32] & > >

[HACKERS] recent compiler warnings

Sorry for starting a new thread, I can't find the correct one to reply to. Using: gcc version 4.8.3 20140627 [gcc-4_8-branch revision 212064] (SUSE Linux) commit ed0097e4f9e6b1 has introduced two compiler warnings: gistutil.c: In function 'gistproperty': gistutil.c:855:16: warning: variable 'rd_

Re: [HACKERS] Write Ahead Logging for Hash Indexes

Hi Amit, Thanks for working on this. When building with --enable-cassert, I get compiler warning: hash.c: In function 'hashbucketcleanup': hash.c:722: warning: 'new_bucket' may be used uninitialized in this function After an intentionally created crash, I get an Assert triggering: TRAP: Faile

Re: [HACKERS] distinct estimate of a hard-coded VALUES list

On Thu, Aug 18, 2016 at 2:25 PM, Tom Lane wrote: > Jeff Janes writes: > > So even though it knows that 6952 values have been shoved in the bottom, > it > > thinks only 200 are going to come out of the aggregation. This seems > like > > a really lousy estimate. In mo

Re: [HACKERS] sslmode=require fallback

On Sat, Jul 30, 2016 at 11:18 AM, Bruce Momjian wrote: > On Fri, Jul 29, 2016 at 11:27:06AM -0400, Peter Eisentraut wrote: > > On 7/29/16 11:13 AM, Bruce Momjian wrote: > > > Yes, I am thinking of a case where Postgres is down but a malevolent > > > user starts a Postgres server on 5432 to gather

[HACKERS] distinct estimate of a hard-coded VALUES list

I have a query which contains a where clause like: aid =ANY(VALUES (1),(45),(87), <6948 more>, (447)) for example: perl -le 'print "explain (analyze) select sum(abalance) from pgbench_accounts where aid=ANY(VALUES "; print join ",", map "($_)", sort {$a<=>$b} map int(rand(500)), 1..6952

[HACKERS] EXLCUDE constraints and Hash indexes

From: https://www.postgresql.org/docs/9.4/static/sql-createtable.html "The access method must support amgettuple (see Chapter 55); at present this means GIN cannot be used. Although it's allowed, there is little point in using B-tree or hash indexes with an exclusion constraint, because this does

Re: [HACKERS] Undiagnosed bug in Bloom index

On Sat, Aug 13, 2016 at 6:22 PM, Tom Lane wrote: > Jeff Janes writes: >> I am getting corrupted Bloom indexes in which a tuple in the table >> heap is not in the index. > > Will push a fix in a bit. After 36 hours of successful running on two different machines (one

Re: [HACKERS] COPY FREEZE and PD_ALL_VISIBLE

On Tue, Nov 3, 2015 at 6:37 AM, Simon Riggs wrote: > On 3 November 2015 at 15:23, Amit Kapila wrote: >> >> On Fri, Oct 23, 2015 at 6:29 AM, Simon Riggs >> wrote: >>> >>> On 21 October 2015 at 13:31, Jeff Janes wrote: >>> >>>> Index-o

[HACKERS] Undiagnosed bug in Bloom index

I am getting corrupted Bloom indexes in which a tuple in the table heap is not in the index. I see it as early as commit a9284849b48b, with commit e13ac5586c49c cherry picked onto it. I don't see it before a9284849b48b because the test-case seg faults before anything interesting can happen. I th

Re: [HACKERS] Btree Index on PostgreSQL and Wiredtiger (MongoDB3.2)

On Fri, Aug 12, 2016 at 1:40 AM, Mark Kirkwood wrote: > After examining the benchmark design - I see we are probably not being > helped by the repeated insertion of keys all of form 'userxxx' leading > to some page splitting. But shouldn't that still leave us with a 75% full index, rather tha

Re: [HACKERS] new autovacuum criterion for visible pages

On Thu, Aug 11, 2016 at 8:32 AM, Amit Kapila wrote: > On Thu, Aug 11, 2016 at 2:09 AM, Jeff Janes wrote: >> I wanted to create a new relopt named something like >> autovacuum_vacuum_pagevisible_factor which would cause autovacuum to >> vacuum a table once less than a ce

[HACKERS] new autovacuum criterion for visible pages

I wanted to create a new relopt named something like autovacuum_vacuum_pagevisible_factor which would cause autovacuum to vacuum a table once less than a certain fraction of the relation's pages are marked allvisible. I wanted some feedback on some things. 1) One issue is that pg_class.relpages a

Re: [HACKERS] Wait events monitoring future development

On Mon, Aug 8, 2016 at 10:03 AM, Bruce Momjian wrote: > On Mon, Aug 8, 2016 at 04:43:40PM +0530, Amit Kapila wrote: >> >According to developers, overhead is small, but many people have doubts >> > that it can be much more significant for intensive workloads. Obviously, it >> > is not an easy

Re: [HACKERS] Notice lock waits

On Fri, Aug 5, 2016 at 12:17 PM, Tom Lane wrote: > Jeff Janes writes: >> I have it PGC_SUSET because it does send some tiny amount of >> information about the blocking process (the PID) to the blocked >> process. That is probably too paranoid, because the PID can be see

[HACKERS] Notice lock waits

One time too many, I ran some minor change using psql on a production server and was wondering why it was taking so much longer than it did on the test server. Only to discover, after messing around with opening new windows and running queries against pg_stat_activity and pg_locks and so on, that

Re: [HACKERS] [RFC] Change the default of update_process_title to off

On Fri, Aug 5, 2016 at 3:25 AM, Tsunakawa, Takayuki wrote: >> From: Tom Lane [mailto:t...@sss.pgh.pa.us] >> Yeah, I think I agree. It would be bad to disable it by default on Unix, >> because ps(1) is a very standard tool there, but the same argument doesn't >> hold for Windows. > > It seems that

Re: [HACKERS] Pgbench performance tuning?

On Thu, Aug 4, 2016 at 10:48 AM, Greg Stark wrote: > I'm trying to run pgbench on a moderately beefy machine (4-core 3.4GHz > with 32G of ram and md mirrored spinning rust drives) at scale 300 > with 32 clients with duration of 15min. I'm getting TPS numbers > between 60-150 which seems surprising

Re: [HACKERS] Cache Hash Index meta page.

On Fri, Jul 22, 2016 at 3:02 AM, Mithun Cy wrote: > I have created a patch to cache the meta page of Hash index in > backend-private memory. This is to save reading the meta page buffer every > time when we want to find the bucket page. In “_hash_first” call, we try to > read meta page buffer twic

Re: [HACKERS] pg_basebackup wish list

On Thu, Jul 28, 2016 at 4:44 AM, Amit Kapila wrote: > On Tue, Jul 26, 2016 at 11:58 AM, Fujii Masao wrote: >> On Wed, Jul 13, 2016 at 3:06 AM, Jeff Janes wrote: >>> On Tue, Jul 12, 2016 at 10:48 AM, Peter Eisentraut >>> wrote: >>>> On 7/12/16 12:53

[HACKERS] 9.6beta subplan target list bug

Found a bug: create table foo as select x, floor(random()*100)::int as y, floor(random()*100)::int as z from generate_series(1,100) f(x); explain select count(y) filter (where y<0.01) from foo where x < 1; ERROR: variable not found in subplan target list STATEMENT: select count(y) filt

Re: [HACKERS] fixes for the Danish locale

On Thu, Jul 21, 2016 at 11:49 AM, Jeff Janes wrote: > On Thu, Jul 21, 2016 at 9:44 AM, Tom Lane wrote: >> Jeff Janes writes: >>> In Danish, the sequence 'aa' is sometimes treated as a single letter >>> which collates after 'z'. >>> Some

Re: [HACKERS] fixes for the Danish locale

On Thu, Jul 21, 2016 at 2:11 PM, Tom Lane wrote: > Jeff Janes writes: >> In Danish, the sequence 'aa' is sometimes treated as a single letter >> which collates after 'z'. >> Some regression tests got into 9.5, and are still in 9.6beta3, which >>

Re: [HACKERS] fixes for the Danish locale

On Thu, Jul 21, 2016 at 9:44 AM, Tom Lane wrote: > Jeff Janes writes: >> In Danish, the sequence 'aa' is sometimes treated as a single letter >> which collates after 'z'. >> Some regression tests got into 9.5, and are still in 9.6beta3, which >>

[HACKERS] fixes for the Danish locale

In Danish, the sequence 'aa' is sometimes treated as a single letter which collates after 'z'. Some regression tests got into 9.5, and are still in 9.6beta3, which fail due to assuming they know how things will sort or compare. I thought the easiest way to deal with it was just to change the test

[HACKERS] plperl loading files

After putting "plperl" into shared_preload_libraries so that things get loaded upon server start, I was surprised to see that each backend was still accessing a handful of perl files the first time it used a plperl function. I see that the cause is src/pl/plperl/plc_trusted.pl, which loads some mo

Re: [HACKERS] pg_basebackup wish list

On Tue, Jul 12, 2016 at 10:48 AM, Peter Eisentraut wrote: > On 7/12/16 12:53 PM, Jeff Janes wrote: >> The --help message for pg_basebackup says: >> >> -Z, --compress=0-9 compress tar output with given compression level >> >> But -Z0 is then rejected as 

[HACKERS] pg_basebackup wish list

I've been having some adventures with pg_basebackup lately, and had some suggestions based on those. The --help message for pg_basebackup says: -Z, --compress=0-9 compress tar output with given compression level But -Z0 is then rejected as 'invalid compression level "0"'. The real docs do s

Re: [HACKERS] Disable WAL completely - Performance and Persistency research

On Thu, Jul 7, 2016 at 1:01 AM, Netanel Katzburg wrote: > Hi All, > > As part of my masters at TAU, I'm currently conducting some research > regarding new persistent memory technology. > I'm using PG for this research and would like to better understand some of > the performance bottlenecks. > For

Re: [HACKERS] pgbench unable to scale beyond 100 concurrent connections

On Wed, Jun 29, 2016 at 5:36 AM, Sachin Kotwal wrote: > Hi Fabien, > > Sorry for very short report. > I feel pgbench is not so complex tool. > > Please see below answers to your questions. > > > On Wed, Jun 29, 2016 at 5:07 PM, Fabien COELHO wrote: >> >> >> Hello Sachin, >> >> Your report is very

Re: [HACKERS] [BUGS] BUG #14155: bloom index error with unlogged table

On Thu, Jun 2, 2016 at 9:03 PM, Tom Lane wrote: > I wrote: >> Jeff Janes writes: >>> My biggest gripe with it at the moment is that the signature size should be >>> expressed in bits, and then internally rounded up to a multiple of 16, >>> rather than having

Re: [HACKERS] Reviewing freeze map code

On Wed, May 18, 2016 at 3:57 PM, Alvaro Herrera wrote: > Andres Freund wrote: > >> >> If we had a checking module for all this it'd possibly be sufficient, >> but we don't. > > Here's an idea. We need core-blessed extensions (src/extensions/, you > know I've proposed this before), so why not take

Re: [HACKERS] [BUGS] BUG #14155: bloom index error with unlogged table

On May 24, 2016 5:27 PM, "David G. Johnston" wrote: > > Moving my griping to -hackers only > > On Tue, May 24, 2016 at 8:08 PM, Tom Lane wrote: >> >> dig...@126.com writes: >> > postgres=# create unlogged table u_tbl (id int); >> > CREATE TABLE >> > postgres=# create index idx_u_tbl on u_tbl usin

Re: [HACKERS] effective_io_concurrency in 9.6beta

On Tue, May 24, 2016 at 11:34 AM, Alvaro Herrera wrote: > Tom Lane wrote: >> Jeff Janes writes: >> > commit 1aba62ec made zero be an illegal value for effective_io_concurrency. >> > i think this was an accident. If not, then the sample postgresql.conf >>

[HACKERS] effective_io_concurrency in 9.6beta

commit 1aba62ec made zero be an illegal value for effective_io_concurrency. i think this was an accident. If not, then the sample postgresql.conf (at least) needs to be updated. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscri

[HACKERS] To-Do item: skip table scan for adding column with provable check constraints

I recently had to run something like: alter table pgbench_accounts add locked text check (locked != 'unlocked'); And was surprised that it took several minutes to complete as it scanned the whole table. The new column is going to start out as NULL in every row, so there is no need to validate th

Re: [HACKERS] Reviewing freeze map code

On Wed, May 18, 2016 at 7:09 AM, Joe Conway wrote: > On 05/18/2016 09:55 AM, Victor Yegorov wrote: >> 2016-05-18 16:45 GMT+03:00 Robert Haas > >: >> >> No, that's what the existing FREEZE option does. This new option is >> about unnecessarily vacuuming pages

Re: [HACKERS] seg fault in contrib/bloom

On Tue, May 17, 2016 at 2:02 PM, Tom Lane wrote: > Jeff Janes writes: >> I'm getting seg faults on contrib/bloom when updating a tuple which >> was found via a bloom index. > > I pushed a patch that should fix this, but without having tried to > reproduce the p

[HACKERS] seg fault in contrib/bloom

I'm getting seg faults on contrib/bloom when updating a tuple which was found via a bloom index. It does not happen on every update, but it does happen within a few seconds of run time, so it is readily reproducible. The test harness is a bit of a mess, I'll try to clean it up and post it if no o

Re: [HACKERS] 10.0

On Sat, May 14, 2016 at 8:37 PM, Tom Lane wrote: > Jeff Janes writes: >> There are lots of improvement which get done to in-memory data >> structures that wouldn't require a pg_dump/pg_upgrade, which could in >> principle be ported into prior major versions if we had t

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Mon, May 16, 2016 at 8:28 AM, Andres Freund wrote: > On 2016-05-10 17:58:33 -0700, Andres Freund wrote: >> FWIW, I've commented out the relevant sections from xlog_redo and since >> then I've not been able to reproduce the issue. > > A couple days of running later, and it hasn't yet re-triggere

Re: [HACKERS] 10.0

On Sat, May 14, 2016 at 7:51 PM, Greg Sabino Mullane wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: RIPEMD160 > > >> Wasn't there some controversy about switching to major.minor versioning >> this in -advocacy? >> >> http://www.postgresql.org/message-id/ee13fd2bb44cb086b457be34e81d5...@bigl

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Tue, May 10, 2016 at 4:02 PM, Andres Freund wrote: > On 2016-05-10 15:53:38 -0700, Jeff Janes wrote: >> >> But isn't CreateCheckPoint called at the end of the checkpoint, not >> the start of it? > > No, CreateCheckPoint() does it all. > > > CreateC

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Tue, May 10, 2016 at 2:00 PM, Andres Freund wrote: > On 2016-05-10 09:19:16 -0700, Andres Freund wrote: >> On 2016-05-10 08:09:02 -0400, Robert Haas wrote: >> > On Tue, May 10, 2016 at 3:05 AM, Andres Freund wrote: >> > > The easy way to trigger this problem would be to have an oid wraparound

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Tue, May 10, 2016 at 9:19 AM, Andres Freund wrote: > On 2016-05-10 08:09:02 -0400, Robert Haas wrote: >> On Tue, May 10, 2016 at 3:05 AM, Andres Freund wrote: >> > The easy way to trigger this problem would be to have an oid wraparound >> > - but the WAL shows that that's not the case here. I

Re: [HACKERS] atomic pin/unpin causing errors

On Fri, May 6, 2016 at 11:24 AM, Andres Freund wrote: > >> I have been trying (and failing) to reproduce the problem in more >> recent releases, with and without cassert. Here is pg_config output >> of one of my current attempts: > > If you say "recent releases" you mean that you've not been able

Re: [HACKERS] atomic pin/unpin causing errors

On Thu, May 5, 2016 at 11:52 AM, Andres Freund wrote: > Hi Jeff, > > On 2016-04-29 10:38:55 -0700, Jeff Janes wrote: >> I don't see the problem with an cassert-enabled, probably because it >> is just too slow to ever reach the point where the problem occurs. > &g

Re: [HACKERS] Initial release notes created for 9.6

On Thu, May 5, 2016 at 10:32 AM, Tom Lane wrote: > I've pushed a first cut at release notes for 9.6. There's a good deal > of work to do yet: Thanks for assembling the notes. This item: "Avoid some spurious waits for AccessExclusiveLocks in hot-standby queries" Should be something like Avoid

Re: [HACKERS] atomic pin/unpin causing errors

On Tue, May 3, 2016 at 4:05 PM, Andres Freund wrote: > Hi Jeff, > > On 2016-04-29 10:38:55 -0700, Jeff Janes wrote: >> I've bisected the errors I was seeing, discussed in >> http://www.postgresql.org/message-id/CAMkU=1xqehc0ok4d+tkjfq1nvuho37pyrkhjp6q8oxifmx7...@mail.gm

[HACKERS] atomic pin/unpin causing errors

I've bisected the errors I was seeing, discussed in http://www.postgresql.org/message-id/CAMkU=1xqehc0ok4d+tkjfq1nvuho37pyrkhjp6q8oxifmx7...@mail.gmail.com It look like they first appear in: commit 48354581a49c30f5757c203415aa8412d85b0f70 Author: Andres Freund Date: Sun Apr 10 20:12:32 2016 -0

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

On Thu, Apr 21, 2016 at 11:00 PM, Noah Misch wrote: > On Mon, Apr 18, 2016 at 05:48:17PM +0300, Teodor Sigaev wrote: >> >>Added, see attached patch (based on v3.1) >> > >> >With this applied, I am getting a couple errors I have not seen before >> >after extensive crash recovery testing: >> >ERROR:

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

On Mon, Apr 18, 2016 at 7:48 AM, Teodor Sigaev wrote: >>> Added, see attached patch (based on v3.1) >> >> >> With this applied, I am getting a couple errors I have not seen before >> after extensive crash recovery testing: >> ERROR: attempted to delete invisible tuple >> ERROR: unexpected chunk

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

On Tue, Apr 12, 2016 at 9:53 AM, Teodor Sigaev wrote: > > With pending cleanup patch backend will try to get lock on metapage with > ConditionalLockPage. Will it interrupt autovacum worker? Correct, ConditionalLockPage should not interrupt the autovacuum worker. >> >> Alvaro's recommendation, t

[HACKERS] parallel query vs extensions

I think there are a lot of extensions which create functions which could benefit from being declared parallel safe. But how does one go about doing that? create extension xml2; select xml_valid(filler),count(*) from pgbench_accounts group by 1; Time: 3205.830 ms alter function xml_valid (text)

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

On Tue, Apr 12, 2016 at 11:38 AM, Andres Freund wrote: > >> The bottom line >> here, IMHO, is not that there's anything wrong with our ring buffer >> implementation, but that if you run PostgreSQL on a system where the >> I/O is hitting a 5.25" floppy (not to say 8") the performance may be >> le

[HACKERS] tab completion for alter extension

tab completion for "alter extension foobar update" yields a list of tables. That is actively misleading. (This is not new in 9.6.) It should complete to nothing, or "TO" followed by a list of available versions. The attached patch takes approach 2. I thought of adding a ";" to the list complet

[HACKERS] Sort push down through a nested loop, for 9.7

I was testing to see if the newer changes in 9.6 fixed some planning issues I've seen in prior versions. It does not, for the ones of interest to me, but while looking into it I see what seems to be a missing optimization (in all versions). If the first child of a nested loop produces naturally o

Re: [HACKERS] Support for N synchronous standby servers - take 2

On Wed, Apr 6, 2016 at 1:23 AM, Fujii Masao wrote: > Okay, I pushed the patch! > Many thanks to all involved in the development of this feature! Thanks, a nice feature. When I compile now without cassert, I get the compiler warning: syncrep.c: In function 'SyncRepUpdateConfig': syncrep.c:878:6

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

On Thu, Apr 7, 2016 at 4:33 PM, Tom Lane wrote: > Jeff Janes writes: >> To summarize the behavior change: > >> In the released code, an inserting backend that violates the pending >> list limit will try to clean the list, even if it is already being >> cleaned.

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

On Wed, Apr 6, 2016 at 9:52 AM, Teodor Sigaev wrote: > I'm inclining to push v3.1 as one of two winners by size/performance and, > unlike to pending lock patch, it doesn't change an internal logic of lock > machinery. This restricts the memory used by ordinary backends when doing the cleanup to

Re: [HACKERS] snapshot too old, configured by time

On Mon, Apr 4, 2016 at 8:38 PM, Peter Geoghegan wrote: > On Sun, Apr 3, 2016 at 2:09 PM, Jeff Janes wrote: >> Also, HOT-cleanup should stop the bloat increase once the snapshot >> crosses the old_snapshot_threshold without even needing to wait until >> the next autovac runs

Re: [HACKERS] snapshot too old, configured by time

On Wed, Mar 30, 2016 at 12:34 PM, Kevin Grittner wrote: > On Sat, Mar 19, 2016 at 1:27 AM, Jeff Janes wrote: > >> I'm not sure if this is operating as expected. >> >> I set the value to 1min. >> >> I set up a test like this: >> >> pgbench -

Re: [HACKERS] Draft release notes for next week's releases

ate, but: + + + Correctly handle wraparound cases in the pg_subtrans + startup logic for hot standby (Jeff Janes) + + This applies to all recovery scenarios, whether they are hot standby or just plain-old automatic crash recovery. (However, it does only matter when prepare

[HACKERS] trivial typo in vacuum progress doc

"skipped blocks are included this total" The word "in" is missing Cheers, Jeff diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml new file mode 100644 index cb22afb..105d541 *** a/doc/src/sgml/monitoring.sgml --- b/doc/src/sgml/monitoring.sgml *** SELECT pg_stat

Re: [HACKERS] multivariate statistics v14

On Sun, Mar 20, 2016 at 4:34 PM, Tomas Vondra wrote: > > > D'oh. Thanks for reporting. Attached is v16, hopefully fixing the few > remaining whitespace issues. Hi Tomas, I'm trying out v16 against a common problem, where postgresql thinks it is likely top stop early during a "order by (index exp

Re: [HACKERS] pgbench stats per script & other stuff

On Fri, Jul 17, 2015 at 6:50 AM, Fabien wrote: > > This patch adds per-script statistics & other improvements to pgbench > > Rationale: Josh asked for the per-script stats:-) > > Some restructuring is done so that all stats (-l --aggregate-interval > --progress --per-script-stats, latency & lag...

Re: [HACKERS] pgbench stats per script & other stuff

On Sat, Mar 19, 2016 at 11:34 AM, Alvaro Herrera wrote: > Jeff Janes wrote: >> On Sat, Mar 19, 2016 at 8:41 AM, Alvaro Herrera >> wrote: >> > I pushed your 25, with some additional minor tweaks. I hope I didn't >> > break anything; please test. >> >

Re: [HACKERS] Weighted Stats

On Fri, Mar 18, 2016 at 11:34 PM, David Fetter wrote: > On Fri, Mar 18, 2016 at 06:12:12PM -0700, Jeff Janes wrote: >> Also, I think it might not give the correct answer even without >> negative weights: >> >> create table foo as select floor(random()*1)::int

Re: [HACKERS] logger process infinite loop

On Fri, Mar 18, 2016 at 10:39 PM, Andres Freund wrote: > Hi, > > On 2016-03-18 21:59:01 -0700, Jeff Janes wrote: >> While testing some patches on my laptop, I noticed my knee getting >> uncomfortably warm. It turns out I has accumulating deranged logging >> processes,

Re: [HACKERS] snapshot too old, configured by time

On Thu, Mar 17, 2016 at 2:15 PM, Kevin Grittner wrote: > New patch just to merge in recent commits -- it was starting to > show some bit-rot. Tests folded in with main patch. I'm not sure if this is operating as expected. I set the value to 1min. I set up a test like this: pgbench -i pgbench

[HACKERS] logger process infinite loop

While testing some patches on my laptop, I noticed my knee getting uncomfortably warm. It turns out I has accumulating deranged logging processes, needing kill -9 to get rid of them. The culprit is: commit c4901a1e03a7730e4471fd1143f1caf79695493d Author: Andres Freund Date: Fri Mar 18 11:43:5

Re: [HACKERS] Idle In Transaction Session Timeout, revived

On Wed, Mar 16, 2016 at 8:32 AM, Robert Haas wrote: > > Committed with slight changes to the docs, and I added a flag variable > instead of relying on IdleInTransactionSessionTimeout not changing at > an inopportune time. Thanks for committing, this should be a useful feature. I get a pretty str

Re: [HACKERS] Fuzzy substring searching with the pg_trgm extension

On Mon, Mar 14, 2016 at 9:27 AM, Artur Zakirov wrote: > On 14.03.2016 18:48, David Steele wrote: >> >> Hi Jeff, >> >> On 2/25/16 5:00 PM, Jeff Janes wrote: >> >>> But, It doesn't sound like I am going to win that debate. Given that, >>>

Re: [HACKERS] Weighted Stats

On Tue, Mar 15, 2016 at 8:36 AM, David Fetter wrote: > > Please find attached a patch that uses the float8 version to cover the > numeric types. Is there a well-defined meaning for having a negative weight? If no, should it be disallowed? I don't know what I was expecting, but not this: selec

Re: [HACKERS] memory leak in GIN

On Fri, Mar 11, 2016 at 11:40 PM, Jaime Casanova wrote: > Hi, > > On the spanish list, Felipe de Jesús Molina Bravo, reported a few days > back that a query that worked well in 9.4 consume all memory in 9.5. > With the self contained test he provided us i reproduced the problem > in 9.5 and 9.6dev

<    1   2   3   4   5   6   7   8   9   10   >