[HACKERS] nested loop semijoin estimates

2015-05-29 Thread Tomas Vondra
... regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training Services testcase.sql Description: application/sql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] nested loop semijoin estimates

2015-05-29 Thread Tomas Vondra
On 05/30/15 01:20, Tomas Vondra wrote: Notice the cost - it's way lover than the previous plan (9.2 vs ~111k), yet this plan was not chosen. So either the change broke something (e.g. by violating some optimizer assumption), or maybe there's a bug somewhere else ... After a bit more

Re: [HACKERS] PATCH: use foreign keys to improve join estimates v1

2015-05-18 Thread Tomas Vondra
. Then again - we can't just discard that, we need to estimate that (and use it to compute the actual selectivity). I agree that if we want to do anything fancy with this, we will have to teach clauselist_selectivity() about foreign keys. -- Tomas Vondra http://www

Re: [HACKERS] multivariate statistics / patch v6

2015-05-15 Thread Tomas Vondra
patch first, and then add features in follow-up patches. I see. Let's work on this for now. Thanks! -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] multivariate statistics / patch v6

2015-05-15 Thread Tomas Vondra
Hello, On 05/15/15 08:29, Kyotaro HORIGUCHI wrote: Hello, At Thu, 14 May 2015 12:35:50 +0200, Tomas Vondra tomas.von...@2ndquadrant.com wrote in 55547a86.8020...@2ndquadrant.com ... Regarding the functional dependencies - you're right there's room for improvement. For example it only works

Re: [HACKERS] multivariate statistics / patch v6

2015-05-14 Thread Tomas Vondra
On 05/13/15 10:31, Kyotaro HORIGUCHI wrote: Hello, this might be somewhat out of place but strongly related to this patch so I'll propose this here. This is a proposal of new feature for this patch or asking for your approval for my moving on this as a different (but very close) project. ===

Re: [HACKERS] PATCH: pgbench - merging transaction logs

2015-05-02 Thread Tomas Vondra
required for fprintf, it may still be an improvement. Yep. fprintf does a lot of processing, so it is the main issue. The question is whether the processing happens while holding the lock, though. I don't think that's the case. -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Tomas Vondra
Hi, On 04/30/15 22:57, Robert Haas wrote: On Tue, Mar 31, 2015 at 3:02 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: attached is v4 of the patch implementing adaptive ndistinct estimator. So, I took a look at this today. It's interesting work, but it looks more like a research project

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Tomas Vondra
. -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] FIX : teach expression walker about RestrictInfo

2015-04-29 Thread Tomas Vondra
Hi, On 04/29/15 05:55, Tom Lane wrote: Tomas Vondra tomas.von...@2ndquadrant.com writes: On 04/28/15 21:50, Tom Lane wrote: RestrictInfo is not a general expression node and support for it has been deliberately omitted from expression_tree_walker(). So I think what you are proposing is a bad

Re: [HACKERS] FIX : teach expression walker about RestrictInfo

2015-04-29 Thread Tomas Vondra
On 04/29/15 18:26, Tom Lane wrote: Tomas Vondra tomas.von...@2ndquadrant.com writes: ... OK, let me explain the context a bit more. When working on the multivariate statistics patch, I need to choose which stats to use for estimating the clauses. I do that in clauselist_selectivity(), although

Re: [HACKERS] alternative compression algorithms?

2015-04-29 Thread Tomas Vondra
Hi, On 04/29/15 23:54, Robert Haas wrote: On Mon, Apr 20, 2015 at 9:03 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Sure, it's not an ultimate solution, but it might help a bit. I do have other ideas how to optimize this, but in the planner every milisecond counts. Looking at 'perf top

Re: [HACKERS] alternative compression algorithms?

2015-04-29 Thread Tomas Vondra
On 04/30/15 02:42, Robert Haas wrote: On Wed, Apr 29, 2015 at 6:55 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: I'm not convinced not compressing the data is a good idea - it suspect it would only move the time to TOAST, increase memory pressure (in general and in shared buffers

Re: [HACKERS] FIX : teach expression walker about RestrictInfo

2015-04-28 Thread Tomas Vondra
Hi, On 04/28/15 21:50, Tom Lane wrote: Tomas Vondra tomas.von...@2ndquadrant.com writes: the attached trivial patch adds handling of RestrictInfo nodes into expression_tree_walker(). RestrictInfo is not a general expression node and support for it has been deliberately omitted from

[HACKERS] FIX : teach expression walker about RestrictInfo

2015-04-28 Thread Tomas Vondra
to know about RestrictInfo, but I don't really see why would that be the case. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training Services diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c index

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2015-04-28 Thread Tomas Vondra
that in a new version, hopefully in a few days. Some of the fdw's need a patch as well in order to compile, see attached. Thanks, I forgot to tweak the clauselist_selectivity() calls contrib :-( -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-24 Thread Tomas Vondra
? That should also work. OK, updated patch attached with just that one change. Patch looks good to me, will mark it as Ready for Committer if Tomas or anyone else doesn't have more to add in terms of review. FWIW, I'm OK with the patch as is. Your reviews were spot on. regards -- Tomas

Re: [HACKERS] alternative compression algorithms?

2015-04-20 Thread Tomas Vondra
On 04/20/15 05:07, Andres Freund wrote: Hi, On 2015-04-19 22:51:53 +0200, Tomas Vondra wrote: The reason why I'm asking about this is the multivariate statistics patch - while optimizing the planning overhead, I realized that considerable amount of time is spent decompressing the statistics

Re: [HACKERS] TABLESAMPLE patch

2015-04-10 Thread Tomas Vondra
from BERNOULLI. -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

[HACKERS] PATCH: use foreign keys to improve join estimates v1

2015-04-06 Thread Tomas Vondra
option is reconstructing clauses somehow. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training Services From 5450be4c3af455ddae37b8949b293f8c01fc5bdd Mon Sep 17 00:00:00 2001 From: Tomas Vondra t...@fuzzy.cz Date: Wed, 1 Apr

Re: [HACKERS] PATCH: nbklno in _hash_splitbucket unused (after ed9cc2b)

2015-04-04 Thread Tomas Vondra
On 04/05/15 01:26, Petr Jelinek wrote: On 05/04/15 01:03, Tomas Vondra wrote: Hi there, the changes in _hash_splitbucket() made nblkno unused when compiled without asserts. Attached is a simple fix to get rid of the warnings. This has been already fixed, see b7e1652d5 . D'oh! Sorry

Re: [HACKERS] File count restriction of directory limits number of relations inside a database.

2015-04-04 Thread Tomas Vondra
this. In other words, you should probably fix the filesystem configuration, not PostgreSQL code. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers

[HACKERS] PATCH: nbklno in _hash_splitbucket unused (after ed9cc2b)

2015-04-04 Thread Tomas Vondra
Hi there, the changes in _hash_splitbucket() made nblkno unused when compiled without asserts. Attached is a simple fix to get rid of the warnings. regards Tomas diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c index 9a77945..e32c4fc 100644 ---

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-04 Thread Tomas Vondra
Hi, On 04/03/15 15:46, Greg Stark wrote: The simple workaround for this was adding a fallback to GEE when f[1] or f[2] is 0. GEE is another estimator described in the paper, behaving much better in those cases. For completeness, what's the downside in just always using GEE? That's a good

Re: [HACKERS] TABLESAMPLE patch

2015-04-01 Thread Tomas Vondra
. Otherwise I think the patch is OK. I'll wait for Petr to fix these issues, and then mark it as ready for committer. What do you think, Amit? (BTW you should probably add yourself as reviewer in the CF app, as you've provided a lot of feedback here.) -- Tomas Vondra http://www

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-03-31 Thread Tomas Vondra
327 11120 The results do change a bit based on the sample, but these values are a good example of the values I'm getting. The other examples (with skewed but smooth distributions) work as good as before. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] PATCH: pgbench - merging transaction logs

2015-03-28 Thread Tomas Vondra
thing we can do right now is buffering the data into larger chunks and writing those chunks. That amortizes the costs of locking. Using O_APPEND, as suggested by Andres, seems like a promising idea. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2015-03-24 Thread Tomas Vondra
in my working copy, but I will double-check that. Admittedly, the management of the stats (e.g. removing stats when the table is dropped) is one of the incomplete parts. You have to delete the rows manually from pg_mv_statistic. -- -- Tomas Vondra http://www.2ndQuadrant.com

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
On 23.3.2015 19:52, Peter Geoghegan wrote: On Mon, Mar 23, 2015 at 11:50 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Not sure how that's supposed to improve the situation? Also, when you change the status to 'stalled', it only makes it more difficult to identify why it was stalled

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-03-23 Thread Tomas Vondra
On 23.3.2015 23:02, Jim Nasby wrote: On 3/22/15 2:59 PM, Tomas Vondra wrote: On 22.3.2015 20:25, Fabien COELHO wrote: I guess Tomas put 2 formats because there was 2 time formats to begin with, but truncating/rouding if someone really wants seconds is quite easy. Yes, that's why I added

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
, and I'd continue going that way. I do work in that direction. OTOH I think it's useful to provide some sort of minimum usable API so that people can actually use it without messing with catalogs directly. It certainly won't have all the bells and whistles, though. -- Tomas Vondrahttp

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
On 23.3.2015 22:53, Jeff Janes wrote: On Mon, Mar 23, 2015 at 11:52 AM, Tomas Vondra Sorry, the 23/2 one is the one I meant. I got confused over which of the emails listed as having an attachment but no patch was the one that actually had a patch. (If the commitfest app can't correctly

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
for the last commitfest. But the feedback is welcome, of course. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
it was stalled (was it waiting for author or a review?). What might be done is tracking time since last patch/review, but I really don't know how we're going to identify that considering the problems with identifying which messages are patches. -- Tomas Vondrahttp://www.2ndQuadrant.com

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
://www.postgresql.org/message-id/attachment/37076/randomize.sql) at 2015-02-27 19:10:21 http://www.postgresql.org/message-id/54f0c11d.7000...@2ndquadrant.com/ from Tomas Vondra tomas.vondra at 2ndquadrant.com http://2ndquadrant.com But that patch failed the majority of make check checks in my hands

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-03-22 Thread Tomas Vondra
, having just one would open the door to calling it something like %u (for Unix timestamp), I guess that is okay as well. Whatever, I don't really care. It's slightly confusing because unix timestams are usually integers, but IMHO that's minor difference. -- Tomas Vondrahttp

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-03-22 Thread Tomas Vondra
you need to make sure 1 millisecond is formatted as .001 and not .1. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-03-22 Thread Tomas Vondra
that the 03 does that on its own: sh printf %03d\n 0 1 2 000 001 002 Oh, right - one dot too many. Thanks, will fix that. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list

[HACKERS] barnacle (running CLOBBER_CACHE_RECURSIVELY) seems stuck since November

2015-03-22 Thread Tomas Vondra
if it's relevant, but this is the machine with Intel C compiler (2013). Attached is a memory context info for the 7953 backend (with more than 1GB VIRT) - not sure if it's relevant, so just in case. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support

[HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-03-21 Thread Tomas Vondra
thing as %t and %m, but formatting the value as a number. Patch attached, I'll add it to CF 2015-06. regards -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-03-21 Thread Tomas Vondra
On 22.3.2015 02:35, Bruce Momjian wrote: On Sun, Mar 22, 2015 at 12:47:12AM +0100, Tomas Vondra wrote: Hi, from time to time I need to correlate PostgreSQL logs to other logs, containing numeric timestamps - a prime example of that is pgbench. With %t and %m that's not quite trivial, because

Re: [HACKERS] PATCH: pgbench - merging transaction logs

2015-03-20 Thread Tomas Vondra
, in which case a merge script is needed, but I do not think that would be a bad thing. I disagree with this - I use transaction logs (either complete or sampled) quite often. I also explained why I think a separate merge script is awkward to use. -- Tomas Vondrahttp://www

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2015-03-20 Thread Tomas Vondra
version of the patch, so either it's unrelated, or I've fixed it since posting the last version. regards -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] GSoC - Idea Discussion

2015-03-19 Thread Tomas Vondra
for. Also, we badly need more details on this - current status, examples, and especially project plan explaining the scope. It's impossible to say whether the sort can be implemented within the GSoC time frame. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] PATCH: pgbench - merging transaction logs

2015-03-15 Thread Tomas Vondra
be provided then. That however is not the goal of this patch. The thread emulation is there for a reason, and I certainly am not going to work on eliminating it (not sure that's even possible). -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA

Re: [HACKERS] PATCH: pgbench - merging transaction logs

2015-03-15 Thread Tomas Vondra
little benefit. I really dislike the features supported only in some cases (although most deployments probably support proper threading these days). -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql

[HACKERS] xloginsert.c hole_length warning on gcc 4.8.3

2015-03-13 Thread Tomas Vondra
the variables at the very beginning (and simplifying the first block a bit) seems like a good fix - patch attached. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services diff --git a/src/backend/access/transam/xloginsert.c b/src

Re: [HACKERS] logical column ordering

2015-03-12 Thread Tomas Vondra
On 12.3.2015 14:17, Alvaro Herrera wrote: Tomas Vondra wrote: On 12.3.2015 03:16, Tom Lane wrote: I agree though that it's worth considering defining pg_attribute.attnum as the logical column position so as to minimize the effects on client-side code. I doubt there is much stuff client

Re: [HACKERS] logical column ordering

2015-03-12 Thread Tomas Vondra
-- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] using CustomScan to inject nodes into the plan

2015-03-11 Thread Tomas Vondra
I'm not trying to replace the join itself - I'd like to inject a node below / above it. regards -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] using CustomScan to inject nodes into the plan

2015-03-11 Thread Tomas Vondra
On 11.3.2015 21:53, Tom Lane wrote: Tomas Vondra tomas.von...@2ndquadrant.com writes: I've been experimenting with using CustomScan to inject nodes into the plan - I'm experimenting a bit, and this seemed like a nice way to do that in an extension, outside the tree. Sadly

[HACKERS] PATCH: pgbench - merging transaction logs

2015-03-08 Thread Tomas Vondra
the log format (--rate, --aggregate-interval and --latency-limit). I'll add this to CF 2016-06. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services diff --git a/contrib/pgbench/pgbench.c b/contrib/pgbench/pgbench.c index

[HACKERS] PATCH: pgbench - logging aggregated info and transactions at the same time

2015-03-08 Thread Tomas Vondra
merge) should be trivial. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services diff --git a/contrib/pgbench/pgbench.c b/contrib/pgbench/pgbench.c index 706fdf5..fb0cf00 100644 --- a/contrib/pgbench/pgbench.c +++ b/contrib

Re: [HACKERS] Weirdly pesimistic estimates in optimizer

2015-03-02 Thread Tomas Vondra
: (fk = random_fk_uniq.fk) Which limits the amount of random I/O by only scanning the index, and is even faster than the first query. regard -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-02-27 Thread Tomas Vondra
On 27.2.2015 17:59, Stephen Frost wrote: All, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: The other feature that'd be cool to have is a debugging function on top of the view, i.e. a function pg_hba_check(host, ip, db, user, pwd) showing which hba rule matched. But that's certainly

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-02-27 Thread Tomas Vondra
, ip, db, user, pwd) showing which hba rule matched. But that's certainly nontrivial. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
is more important. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 19:23, Alvaro Herrera wrote: Tomas Vondra wrote: Physical ordering is still determined by the CREATE TABLE command, In theory, you should be able to UPDATE attphysnum in pg_attribute too when the table is empty, and new tuples inserted would follow the specified ordering

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 26.2.2015 23:42, Kevin Grittner wrote: Tomas Vondra tomas.von...@2ndquadrant.com wrote: Over the time I've heard various use cases for this patch, but in most cases it was quite speculative. If you have an idea where this might be useful, can you explain it here, or maybe point me

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 20:34, Alvaro Herrera wrote: Tomas Vondra wrote: I think we could calls to the randomization functions into some of the regression tests (say 'create_tables.sql'), but that makes regression tests ... well, random, and I'm not convinced that's a good thing. Also, this makes

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
by reordering columns in CREATE TABLE - should be done automatically (no user interface required) - seems useful both for on-disk physical tuples, and virtual tuples Anything else? -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 19:50, Alvaro Herrera wrote: Tomas Vondra wrote: On 26.2.2015 23:42, Kevin Grittner wrote: One use case is to be able to suppress default display of columns that are used for internal purposes. For example, incremental maintenance of materialized views will require storing

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
... well, random, and I'm not convinced that's a good thing. Also, this makes regression tests harder to think, because SELECT * does different things depending on the attlognum order. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 21:42, Josh Berkus wrote: On 02/27/2015 12:25 PM, Tomas Vondra wrote: On 27.2.2015 21:09, Josh Berkus wrote: Tomas, So for an API, 100% of the use cases I have for this feature would be satisfied by: ALTER TABLE __ ALTER COLUMN _ SET ORDER # and: ALTER TABLE

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 23:48, Josh Berkus wrote: On 02/27/2015 12:48 PM, Tomas Vondra wrote: On 27.2.2015 21:42, Josh Berkus wrote: On 02/27/2015 12:25 PM, Tomas Vondra wrote: On 27.2.2015 21:09, Josh Berkus wrote: Tomas, So for an API, 100% of the use cases I have for this feature would

Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-02-25 Thread Tomas Vondra
you're doing in this patch, except that it only affects the optimizer and not the executor. Do you have any plans in this direction? If not, I'll pick this up as I do have that on my TODO. regards -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-25 Thread Tomas Vondra
On 24.2.2015 19:08, Jim Nasby wrote: On 2/22/15 8:32 PM, Tomas Vondra wrote: On 23.2.2015 03:20, Jim Nasby wrote: On 2/22/15 5:41 PM, Tomas Vondra wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production use (to minimize the impact): 1

Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-02-25 Thread Tomas Vondra
: ... clause is (Node*) ... if (IsA(clause, Var)) { Var *var = (Var*)clause; ... } or Var * var = NULL; if (! IsA(clause, Var)) // error / continue var = (Var*)clause; -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Fillfactor for GIN indexes

2015-02-25 Thread Tomas Vondra
On 25.2.2015 10:20, Alexander Korotkov wrote: On Tue, Feb 24, 2015 at 5:15 PM, Tomas Vondra tomas.von...@2ndquadrant.com mailto:tomas.von...@2ndquadrant.com wrote: I hoped lowering the fillfactor will improve this, but fillfactor=75 had pretty much no effect in this case. Is that expected

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-24 Thread Tomas Vondra
On 24.2.2015 05:09, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Tomas I believe the small regressions (1-10%) for small data sets, Tomas might be caused by this 'random padding' effect, because that's Tomas probably where L1/L2 cache is most important

Re: [HACKERS] Fillfactor for GIN indexes

2015-02-24 Thread Tomas Vondra
the previous discussion talked about random updates, not inserts, so maybe that's the culprit? -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] KNN-GiST with recheck

2015-02-24 Thread Tomas Vondra
to do a bit of testing on this (assuming it's still needed), but the patches need rebasing - Heikki fixed a few issues, so they don't apply cleanly. regards -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via

Re: [HACKERS] Abbreviated keys for text cost model fix

2015-02-23 Thread Tomas Vondra
Hi, On 22.2.2015 22:30, Peter Geoghegan wrote: On Sun, Feb 22, 2015 at 1:19 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: In short, this fixes all the cases except for the ASC sorted data. I haven't done any code review, but I think we want this. I'll use data from the i5-2500k

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-23 Thread Tomas Vondra
Hi, On 23.2.2015 11:59, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Tomas Interesting, but I think Gavin was asking about how much Tomas variation was there for each tested case (e.g. query executed on Tomas the same code / dataset). And in those cases

Re: [HACKERS] Abbreviated keys for text cost model fix

2015-02-23 Thread Tomas Vondra
On 23.2.2015 19:22, Peter Geoghegan wrote: On Mon, Feb 23, 2015 at 8:40 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: The durations are much higher than without the single unsorted row added at the end. Queries often take 20x longer to finish (on the same code), depending on the scale

Re: [HACKERS] Abbreviated keys for text cost model fix

2015-02-23 Thread Tomas Vondra
On 24.2.2015 01:44, Peter Geoghegan wrote: On Mon, Feb 23, 2015 at 4:41 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Are you going to add this into the CF? Would be nice to get this into 9.5. Strictly speaking it should go to 2015-06 I guess, but I'd consider it part of one

Re: [HACKERS] Abbreviated keys for text cost model fix

2015-02-23 Thread Tomas Vondra
On 23.2.2015 21:52, Peter Geoghegan wrote: On Mon, Feb 23, 2015 at 11:17 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: So while it's true that for the 3rd query we get much worse results compared to the other queries (i.e. we don't get 400% speedup but ~3% slowdown compared to master

Re: [HACKERS] logical column ordering

2015-02-23 Thread Tomas Vondra
CF, we're aiming for 9.6 if things go well. regards -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c index c5892d3..7528724 100644 --- a/contrib/dblink

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2015-02-22 Thread Tomas Vondra
On 22.2.2015 09:14, Jeff Davis wrote: On Wed, 2015-01-07 at 20:07 +0100, Tomas Vondra wrote: So I started digging in the code and I noticed this: hash_mem = MemoryContextMemAllocated(aggstate-hashcontext, true); which is IMHO obviously wrong, because that accounts only

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-22 Thread Tomas Vondra
influence the results, no? -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Abbreviated keys for text cost model fix

2015-02-22 Thread Tomas Vondra
of collations could actually matter here. That's a good point, but all the tests were executed with en_US.UTF-8. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-22 Thread Tomas Vondra
) would be appropriate. regards -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] TABLESAMPLE patch

2015-02-22 Thread Tomas Vondra
to be done in this CF (or that it makes the patch uncommitable). For example, this seems like a very nice project for the GSoC (clear scope, not too large, ...). -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-22 Thread Tomas Vondra
On 23.2.2015 03:20, Jim Nasby wrote: On 2/22/15 5:41 PM, Tomas Vondra wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production use (to minimize the impact): 1) no index support:-( I'd like to see support for more relation types (at least

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 23:09, Peter Geoghegan wrote: On Sat, Feb 21, 2015 at 12:15 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Although I cannot easily explain the disparity in performance between 1M and 5M sized sets for this query: select count(distinct randtxt) from stuff_text You did

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-02-21 Thread Tomas Vondra
On 22.2.2015 02:38, Jeff Davis wrote: On Wed, 2015-01-28 at 23:25 +0100, Tomas Vondra wrote: 3) moves the assert into the 'if (release)' branch You missed one, but I got it. Oh, thanks! 4) includes the comments proposed by Ali Akbar in his reviews Warnings at makeArrayResult

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
Hi Gavin, On 21.2.2015 06:35, Gavin Flower wrote: On 21/02/15 18:18, Tomas Vondra wrote: OK, so I've repeated the benchmarks with both patches applied, and I think the results are interesting. I extended the benchmark a bit - see the SQL script attached. 1) multiple queries

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 19:57, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: The gains for text are also very nice, although in this case that only happens for the smallest scale (1M rows), and for larger scales it's actually slower than current

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Idea: GSoC - Query Rewrite with Materialized Views

2015-02-20 Thread Tomas Vondra
. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Combining Aggregates

2015-02-20 Thread Tomas Vondra
On 20.2.2015 21:23, Peter Eisentraut wrote: On 2/20/15 3:09 PM, Tomas Vondra wrote: On 20.2.2015 21:01, Peter Eisentraut wrote: Is there a case where the combining function is different from the transition function, other than for count? It's different in all the cases when the aggregate

Re: [HACKERS] Re: Abbreviated keys for Datum tuplesort

2015-02-20 Thread Tomas Vondra
15.3 0.95 so for a small dataset the speedup is very nice, but for larger sets there's ~5% slowdown. Is this expected? -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-02-20 Thread Tomas Vondra
On 21.2.2015 00:14, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 1:33 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: For example with the same percentile_disc() test as in the other thread: create table stuff as select random()::numeric as randnum from generate_series(1,100

Re: [HACKERS] Idea: GSoC - Query Rewrite with Materialized Views

2015-02-20 Thread Tomas Vondra
On 21.2.2015 00:20, Kevin Grittner wrote: Tomas Vondra tomas.von...@2ndquadrant.com wrote: I share the view that this would be very valuable, but the scope far exceeds what can be done within a single GSoC project. But maybe we could split that into multiple pieces, and Eric would implement

Re: [HACKERS] Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-02-20 Thread Tomas Vondra
On 21.2.2015 01:45, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 4:42 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Isn't this patch about adding abbreviated keys for Numeric data type? That's how I understood it, and looking into numeric_sortsup.patch seems to confirm

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-20 Thread Tomas Vondra
On 21.2.2015 02:00, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Right...so don't test a datum sort case, since that isn't supported at all in the master branch. Your test case is invalid for that reason. Tomas What do you mean by 'Datum sort case

Re: [HACKERS] Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-02-20 Thread Tomas Vondra
On 21.2.2015 01:17, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 4:11 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: So you're testing both the patches (numeric + datum tuplesort) at the same time? No, I was just testing two similar patches separately. I.e. master vs. each patch

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-20 Thread Tomas Vondra
Hi, On 21.2.2015 02:06, Tomas Vondra wrote: On 21.2.2015 02:00, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Right...so don't test a datum sort case, since that isn't supported at all in the master branch. Your test case is invalid for that reason

Re: [HACKERS] Combining Aggregates

2015-02-20 Thread Tomas Vondra
and the 'serialize/deserialize'). regards -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

[HACKERS] failures with tuplesort and ordered set aggregates (due to 5cefbf5a6c44)

2015-02-20 Thread Tomas Vondra
tuplesort_begin_datum() when (use_tuples == true), which only sets 'onlyKey' and leaves (sortKeys == NULL). So 'mergeruns' fails because it does not expect that. -- Tomas Vondrahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

<    3   4   5   6   7   8   9   10   11   12   >