Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-13 Thread Peter Geoghegan
improved since 2005 - their performance was improved quite a bit in 9.0. Here's a more recent analysis: http://www.depesz.com/index.php/2010/06/28/should-you-use-hash-index/ -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent

Re: [PERFORM] What about implementing a bitmap index? Any use cases?

2011-09-18 Thread Peter Geoghegan
to introduce on-disk bitmap indexes to Postgres. -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [PERFORM] random_page_cost = 2.0 on Heroku Postgres

2012-02-12 Thread Peter Geoghegan
doesn't pay the considerable overhead of that instrumentation across the board). -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your

Re: [PERFORM] Update blocking a select count(*)?

2012-06-15 Thread Peter Geoghegan
that will block a select statement's AccessShare lock. -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http

Re: [PERFORM] optimizing queries using IN and EXISTS

2012-07-18 Thread Peter Geoghegan
equivalent. http://www.postgresql.org/docs/current/static/functions-subquery.html See the notes about NULL under IN. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance

[PERFORM] [repost] Help me develop new commit_delay advice

2012-08-02 Thread Peter Geoghegan
This has been reposted to this list from the pgsql-hackers list, at the request of Josh Berkus. Hopefully there will be more interest here. -- Forwarded message -- From: Peter Geoghegan pe...@2ndquadrant.com Date: 29 July 2012 16:39 Subject: Help me develop new commit_delay advice

Re: [PERFORM] average query performance measuring

2012-08-21 Thread Peter Geoghegan
way to monitor query execution costs on earlier versions, I think that I'll probably have new information about that for my talk at Postgres Open. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance

Re: [PERFORM] [repost] Help me develop new commit_delay advice

2012-10-08 Thread Peter Geoghegan
On 6 September 2012 04:20, Greg Smith g...@2ndquadrant.com wrote: On 08/02/2012 02:02 PM, Peter Geoghegan wrote: I dug up what I wrote when trying to provide better advice for this circa V8.3. That never really gelled into something worth publishing at the time. But I see some similar

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Peter Geoghegan
into a column, analyzed and grouped by that. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Peter Geoghegan
costs essentially nothing to maintain, and simply represent an ongoing obligation for ANALYZE to provide statistics for an expression? -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list

Re: [PERFORM] PostreSQL v9.2 uses a lot of memory in Windows XP

2012-11-14 Thread Peter Geoghegan
* Postgres is doing with memory, but need to summarise it usefully. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription

Re: SOLVED - RE: [PERFORM] Poor performance using CTE

2012-11-14 Thread Peter Geoghegan
that for years, so it's already something that we've effectively committed to. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your

Re: [PERFORM] Poor performance using CTE

2012-11-21 Thread Peter Geoghegan
-- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Poor performance using CTE

2012-11-21 Thread Peter Geoghegan
On 21 November 2012 15:21, Andrew Dunstan and...@dunslane.net wrote: And I continue to think that spelling it OFFSET 0 is horribly obscure. I'm not sure that it's any more obscure than the very idea of an optimisation fence. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL

Re: [PERFORM] serious under-estimation of n_distinct for clustered distributions

2012-12-29 Thread Peter Geoghegan
/commands/analyze.c (IBM Research Report RJ 10025 is referenced there). The general advice here is: 1) Increase default_statistics_target for the column. 2) If that doesn't help, consider using the following DDL: alter table foo alter column bar set ( n_distinct = 5.0); -- Peter Geoghegan

Re: [PERFORM] Poor performance on an aggregate query

2013-04-15 Thread Peter Geoghegan
a functional index on length(discussion_post.id)? -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] pg_stat_statements query normalization

2013-06-17 Thread Peter Geoghegan
(essentially, the internal representation that the rewriter stage processes). -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] pg_stat_statements behavior in crash recovery

2013-06-19 Thread Peter Geoghegan
to disk when there's a clean shutdown. pg_stat_statements is similar to the statistics collector here. Why are you posting this to the -performance list? -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your

Re: [PERFORM] Why is n_distinct always -1 for range types?

2013-09-25 Thread Peter Geoghegan
histograms of lower and upper bounds for range types, and the fraction of empty ranges. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Postgres Query Plan Live Lock

2014-02-04 Thread Peter Geoghegan
/wiki/Hint_Bits -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Performance Benchmarking for data-warehousing instance?

2014-02-07 Thread Peter Geoghegan
, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Lack of index usage when doing array casts

2014-02-21 Thread Peter Geoghegan
standard packages, I'm afraid. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Turn off Hyperthreading! WAS: 60 core performance with 9.3

2014-08-20 Thread Peter Geoghegan
, it is crucially important to consider power saving features in most benchmarks these days, where that might not have been true a few years ago. The CPU scaling governor can alter the outcome of many benchmarks quite significantly. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-02 Thread Peter Geoghegan
free. Hashing is really cheap when the CPU is bottlenecked on memory bandwidth. If you're interested, download the patch, and enable the debug traces. You'll see HyperLogLog accurately indicate the cardinality of text datums as they're copied into local memory before sorting. -- Regards, Peter

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-02 Thread Peter Geoghegan
be crazy). Streaming algorithms like HyperLogLog are very recent ideas, as these things go. I wouldn't be all that discouraged by the fact that it might not have been put to use in this way (for database statistics) by somebody before now. -- Regards, Peter Geoghegan -- Sent via pgsql-performance

Re: [PERFORM] working around JSONB's lack of stats?

2015-01-28 Thread Peter Geoghegan
, and things like that. I tend to think that that's a common enough use-case. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] working around JSONB's lack of stats?

2015-01-28 Thread Peter Geoghegan
within arrays are keys for our purposes, and these are often used for tags and so on. But Strings that are the key of an object/pair are much less useful to index, in my estimation. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] Index creation running now for 14 hours

2015-08-26 Thread Peter Geoghegan
to complete it. That much is very clear. If you're really worried about these costs, I suggest enabling trace_sort locally, and monitoring the progress of this sort in the logs. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] Slow Index Creation, why is it not consuming more memory.

2015-12-07 Thread Peter Geoghegan
settings are unlikely to help unless they result in a fully internal sort. There is evidence that the heap that tuple sorting uses benefits from *lower* settings. Sometimes as low as 64MB. We're working to make this better in 9.6. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mai

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-24 Thread Peter Geoghegan
f you're arguing for the idea that B-Trees should reliably keep tuples in order by a tie-break condition, that seems difficult to implement, and likely not worth it in practice. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to yo

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-24 Thread Peter Geoghegan
ch, but it's probably better to focus on duplicate storage or even leaf page compression, as Stephen mentioned. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Adding a ROLLUP switches to GroupAggregate unexpectedly

2016-04-09 Thread Peter Geoghegan
cardinality inputs, too. I bet that was a factor here. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Odd sudden performance degradation related to temp object churn

2017-08-14 Thread Peter Geoghegan
lly confuses us is > that this job has been running for years with no issue remotely approaching > this one. We are also using pgpool. Did you happen to notice that this occurred when you upgrading point release? If so, what version did you move from/to? -- Peter Geoghegan -- Sent via pgsq

Re: estimate correlation of index separately from table (Re: [PERFORM] index fragmentation on insert-only table with non-unique column)

2017-07-07 Thread Peter Geoghegan
mizations are closely intertwined things, and the lack of clarity on how they all fit together is probably holding back an implementation of any one of them. [1] https://www.postgresql.org/message-id/flat/20160524173914.GA11880%40telsasoft.com#20160524173914.ga11...@telsasoft.com -- Peter Ge

Re: [PERFORM] Slow query after 9.3 to 9.6 migration

2017-07-26 Thread Peter Geoghegan
PDATEs? I ask these questions because I think it's possible that this is explained by a regression in 9.5's handling of index bloat, described here: http://postgr.es/m/CAH2-Wz=sfakvmv1x9jh19ej8am8tzn9f-yecips9hrrrqss...@mail.gmail.com I'm trying to track down cases where this could be an issu

Re: [PERFORM] Efficiently merging and sorting collections of sorted rows

2017-06-23 Thread Peter Geoghegan
et Postgres to not do a > quick sort here? I would like that too. There is a patch that does what I think you're describing, but it seems to be in limbo: https://commitfest.postgresql.org/11/409/ -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.or

Re: [PERFORM] Pageinspect bt_metap help

2017-09-17 Thread Peter Geoghegan
want to learn more about the B-Tree code, I suggest that you start by looking at the code for contrib/amcheck. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Pageinspect bt_metap help

2017-09-18 Thread Peter Geoghegan
allowing it. We can instead just pretend that it's shorter, knowing that upper levels don't contain useful information. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] 10x faster sort performance on Skylake CPU vs Ivy Bridge

2017-08-25 Thread Peter Geoghegan
faster machine but > some non-C locale on the slower. strcoll() is pretty darn expensive > compared to strcmp() :-( strcoll() is very noticeably slower on macOS, too. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to yo