Re: [HACKERS] less expensive pg_buffercache on big shmem

2016-09-02 Thread Tomas Vondra
ries? How can be semiconsistent 5x slower than nonconsistent? Wouldn't that make it significantly slower than the consistent method? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via p

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2016-09-02 Thread Tomas Vondra
On 09/03/2016 02:21 AM, Tomas Vondra wrote: A few minor comments regarding the patch: 1) CopyStartSend seems pretty pointless - It only has one function call in it, and is called on exactly one place (and all other places simply call allowLongStringInfo directly). I'd get rid of this function

Re: [HACKERS] Issue with bgworker, SPI and pgstat_report_stat

2016-09-02 Thread Tomas Vondra
enance tasks? Sure, this may only help for bgworkers that do stuff fairly close to regular backends, but maybe that's enough. In any case, I think adding the pgstat_report_stat() into worker_spi seems like a reasonable (and backpatchable) fix. regards -- Tomas Vondra http://ww

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2016-09-02 Thread Tomas Vondra
lients are unlikely to handle even the 2GB we might send them without breaking the protocol, so I guess this is fine. And 4GB seems like the best we can do. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -

Re: [HACKERS] incomplete removal of not referenced CTEs

2016-09-01 Thread Tomas Vondra
On 09/01/2016 09:58 PM, Andres Freund wrote: > On 2016-09-01 15:46:45 -0400, Tom Lane wrote: >> Tomas Vondra <tomas.von...@2ndquadrant.com> writes: >>> While investigating a CTE-related query, I've noticed that we don't >>> really remove all unreachable CT

[HACKERS] incomplete removal of not referenced CTEs

2016-09-01 Thread Tomas Vondra
remove the CTEs recursively (and decrease the refcount). regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your su

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-31 Thread Tomas Vondra
On 08/31/2016 11:43 PM, Andres Freund wrote: > On 2016-08-31 23:40:46 +0200, Tomas Vondra wrote: >> It's an improvement (and it's pretty much exactly what I proposed >> upthread). But it does not solve the problems with pg_statistic for >> example (each backend needs it's ow

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-31 Thread Tomas Vondra
On 08/31/2016 09:20 PM, Vik Fearing wrote: > On 08/24/2016 06:16 PM, Robert Haas wrote: >> On Tue, Aug 23, 2016 at 6:11 PM, Tomas Vondra >> <tomas.von...@2ndquadrant.com> wrote: >>> Could someone please explain how the unlogged tables are supposed to fix the >&g

Re: [HACKERS] Bogus sizing parameters in some AllocSetContextCreate calls

2016-08-28 Thread Tomas Vondra
On 08/28/2016 04:59 PM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: I'm however pretty sure this will have absolutely no impact on profiles. Yeah, I do not believe that either. Also, I think that the intent of the existing code is to defend against bad parameter

Re: [HACKERS] Bogus sizing parameters in some AllocSetContextCreate calls

2016-08-28 Thread Tomas Vondra
y no impact on profiles. It might save a few cycles, but this only runs when creating the memory context and all profiles I've seen are about palloc/pfree. It might matter when creating many memory contexts, but then you probably have other things to worry about. regards -- Tomas Vondra

Re: [HACKERS] Bogus sizing parameters in some AllocSetContextCreate calls

2016-08-28 Thread Tomas Vondra
les, but this only runs when creating the memory context and all profiles I've seen are about palloc/pfree. It might matter when creating many memory contexts, but then you probably have other things to worry about. regards -- Tomas Vondra http://www.2ndQuadrant.com Postg

[HACKERS] src/include/catalog/pg_foreign_table.h still refers genbki.sh

2016-08-27 Thread Tomas Vondra
Hi, I've noticed the comment in src/include/catalog/pg_foreign_table.h still talks about genbki.sh, but AFAIK it was replaced with genbki.pl. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services --

Re: [HACKERS] pg_dump with tables created in schemas created by extensions

2016-08-26 Thread Tomas Vondra
t want the extension/index dependencies? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] Renaming some binaries

2016-08-26 Thread Tomas Vondra
e calling initdb directly (and there's quite a few of them, I'm sure, exactly because it's simpler to pass the parameters directly), but also for all those already using pg_ctl. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Tra

Re: [HACKERS] Better tracking of free space during SP-GiST index build

2016-08-24 Thread Tomas Vondra
would be worth/interesting to test? ISTM it'd be interesting to test both index creation, maintenance and querying, while varying the fillfactor. What other parameters would be interesting to tweak, and what datasets might be useful? regards -- Tomas Vondra http://www

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-24 Thread Tomas Vondra
On 08/24/2016 12:20 AM, Andres Freund wrote: On 2016-08-23 19:18:04 -0300, Claudio Freire wrote: On Tue, Aug 23, 2016 at 7:11 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: Could someone please explain how the unlogged tables are supposed to fix the catalog bloat problem, as

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-23 Thread Tomas Vondra
On 08/24/2016 12:38 AM, Claudio Freire wrote: On Tue, Aug 23, 2016 at 7:25 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: Could someone please explain how the unlogged tables are supposed to fix the catalog bloat problem, as stated in the initial patch submission? We'd stil

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-23 Thread Tomas Vondra
On 08/24/2016 12:18 AM, Claudio Freire wrote: On Tue, Aug 23, 2016 at 7:11 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: On 08/22/2016 10:32 PM, Robert Haas wrote: ... 1. The number of tables for which we would need to add a duplicate, unlogged table is formidable. Yo

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-23 Thread Tomas Vondra
in the initial patch submission? We'd still need to insert/delete the catalog rows when creating/dropping the temporary tables, causing the bloat. Or is there something I'm missing? -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: [HACKERS] distinct estimate of a hard-coded VALUES list

2016-08-22 Thread Tomas Vondra
values similarly to what ANALYZE does. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: htt

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-18 Thread Tomas Vondra
ge is that it requires changes to applications. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http:/

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-18 Thread Tomas Vondra
is curing the symptoms and not the cause. I think a proper design needs to prevent the DROP by using dependencies. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers

Re: [HACKERS] Index Onlys Scan for expressions

2016-08-15 Thread Tomas Vondra
it might be possible to use explicit parentheses to make it work, no? That is, will this work? select 100 + (a * 1000 + b) from ... Or will it still break the IOS? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: [HACKERS] multivariate statistics (v19)

2016-08-15 Thread Tomas Vondra
On 08/10/2016 06:41 AM, Michael Paquier wrote: On Wed, Aug 3, 2016 at 10:58 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: 1) enriching the query tree with multivariate statistics info Right now all the stuff related to multivariate statistics estimation happens in claus

Re: [HACKERS] multivariate statistics (v19)

2016-08-10 Thread Tomas Vondra
On 08/10/2016 02:23 PM, Michael Paquier wrote: On Wed, Aug 10, 2016 at 8:33 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: On 08/10/2016 06:41 AM, Michael Paquier wrote: Patch 0001: there have been comments about that before, and you have put the checks on RestrictInfo in a

Re: [HACKERS] multivariate statistics (v19)

2016-08-10 Thread Tomas Vondra
On 08/10/2016 02:24 PM, Michael Paquier wrote: On Wed, Aug 10, 2016 at 8:50 PM, Petr Jelinek <p...@2ndquadrant.com> wrote: On 10/08/16 13:33, Tomas Vondra wrote: On 08/10/2016 06:41 AM, Michael Paquier wrote: On Wed, Aug 3, 2016 at 10:58 AM, Tomas Vondra 2) combining multiple stat

Re: [HACKERS] multivariate statistics (v19)

2016-08-10 Thread Tomas Vondra
On 08/10/2016 03:29 PM, Ants Aasma wrote: On Wed, Aug 3, 2016 at 4:58 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: 2) combining multiple statistics I think the ability to combine multivariate statistics (covering different subsets of conditions) is important and useful, b

Re: [HACKERS] multivariate statistics (v19)

2016-08-10 Thread Tomas Vondra
On 08/10/2016 06:41 AM, Michael Paquier wrote: On Wed, Aug 3, 2016 at 10:58 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: ... But more importantly, I think we'll need to show some of the data in EXPLAIN output. With per-column statistics it's fairly straightforward to determine

Re: [HACKERS] multivariate statistics (v19)

2016-08-05 Thread Tomas Vondra
On 08/05/2016 06:24 AM, Michael Paquier wrote: On Wed, Aug 3, 2016 at 10:58 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: Attached is v19 of the "multivariate stats" patch series - essentially v18 rebased on top of current master. Aside from a few bug fixes, the

[HACKERS] PATCH: two slab-like memory allocators

2016-08-02 Thread Tomas Vondra
, and probably things I haven't thought about. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 0001-simple-slab-allocator-fixed-size-allocations.patch Description: binary/octet-stream 0002-generati

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-01 Thread Tomas Vondra
installcheck I get the failures every time - regression diff attached. The first failure in "rolenames" is expected, because of clash with existing user name. The remaining two failures are not. I only get the failure for "installcheck" but not "check" fo

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-07-31 Thread Tomas Vondra
e storage filename. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-07-30 Thread Tomas Vondra
support for memory store for column's statistics. Some separate questions is about production statistics - pg_stat_user_table, .. That seems to work (both analyze and pg_stat_user_tables). Not sure where it's in the code, and I'm not willing to reverse engineer it. regards -- Tomas Vondra

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-07-29 Thread Tomas Vondra
. It seems the patch delays building of visibility map, or something like that. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) T

[HACKERS] (re)start in our init scripts seems broken

2016-07-19 Thread Tomas Vondra
(in shutdown), so the postmaster does not overwrite the file. And of course the PID still matches a running process. Is there a reason why it's coded like this? I think we should use the pg_ctl instead or (at the very least) check the postmaster return code. Also, perhaps we should add

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Tomas Vondra
On 07/13/2016 07:37 PM, Tom Lane wrote: Peter Geoghegan <p...@heroku.com> writes: On Wed, Jul 13, 2016 at 7:53 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: In the thread [1] dealing with hashjoin bug introduced in 9.5, Tom voiced his dislike of dense_alloc. I kinda ag

[HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Tomas Vondra
rc/include/utils/memutils.h#L49 regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.

Re: [HACKERS] Whether to back-patch fix for aggregate transtype width estimates

2016-06-19 Thread Tomas Vondra
using per-group memory contexts. So also unrelated to this. So while I'm a fan of improving our planning, I'd lean towards not back-patching this particular bit. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: [HACKERS] New design for FK-based join selectivity estimation

2016-06-19 Thread Tomas Vondra
Hi On 06/18/2016 06:52 PM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: A few more comments, about re-reading the patch more thoroughly. I wouldn't say any of those qualify as bugs, but rather as discussion about some of the design choices: 1) NULL handling

Re: [HACKERS] New design for FK-based join selectivity estimation

2016-06-17 Thread Tomas Vondra
t one, but don't risk making it worse." 3) ForeignKeyOptInfo->rinfos as a List Can we actually get a list of matching RestrictInfos for a single foreign key? I've been unable to construct such query. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Deve

Re: [HACKERS] New design for FK-based join selectivity estimation

2016-06-16 Thread Tomas Vondra
Hi, On 06/16/2016 01:00 AM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: Attached is a reworked patch, mostly following the new design proposal from this thread. I've whacked this around quite a bit and am now reasonably happy with it. The issue of planning spe

Re: [HACKERS] New design for FK-based join selectivity estimation

2016-06-13 Thread Tomas Vondra
happens in costsize.c, of course, but was modified to use the pre-matched info. I have my doubts about this part, and I'm sure Tom had something less complex / more efficient in mind (using the pre-matched info). regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-06-06 Thread Tomas Vondra
On 06/06/2016 07:40 PM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: On 06/06/2016 06:15 PM, Tom Lane wrote: This checks that you found a joinclause mentioning foreignrel. But foreignrel need have nothing to do with the foreign key; it could be any table in the

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-06-06 Thread Tomas Vondra
it might seem. What if the foreign keys overlap, for example? Or what if the keys go in opposite directions (cycle). And so on ... regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hac

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-06-06 Thread Tomas Vondra
On 06/06/2016 06:15 PM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: On 06/04/2016 08:15 PM, Tom Lane wrote: * Make RelationGetFKeyList cache a list of ForeignKeyOptInfo structs, not just constraint OIDs. It's insane that the relcache scans pg_constraint to c

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-06-05 Thread Tomas Vondra
gt;relid == rightvar->varno) && (fkrel->relid == leftvar->varno) && (fkinfo->confkeys[i] == rightvar->varattno) && (fkinfo->conkeys[i] == leftvar->varattno)) {

Re: [HACKERS] New design for FK-based join selectivity estimation

2016-06-05 Thread Tomas Vondra
s. ISTM the proposed design has the same issue, no? What happens if the relations are partitioned? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hac

[HACKERS] Re: pg9.6 segfault using simple query (related to use fk for join estimates)

2016-06-02 Thread Tomas Vondra
if there's a consensus it's not doomed anyway (for 9.6). regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

2016-05-31 Thread Tomas Vondra
On 05/31/2016 07:24 PM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: On 05/31/2016 06:59 PM, Tom Lane wrote: I'm confused here --- are you speaking of having removed if (msg->cutoff_time > req->request_time) req->request_time

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

2016-05-31 Thread Tomas Vondra
On 05/31/2016 06:59 PM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: On 05/26/2016 10:10 PM, Tom Lane wrote: I posted a patch at https://www.postgresql.org/message-id/13023.1464213...@sss.pgh.pa.us which I think is functionally equivalent to what you hav

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

2016-05-31 Thread Tomas Vondra
Hi, On 05/26/2016 10:10 PM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: Attached is a patch that should fix the coalescing, including the clock skew detection. In the end I reorganized the code a bit, moving the check at the end, after the clock skew det

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

2016-05-26 Thread Tomas Vondra
Hi, On 05/26/2016 10:10 PM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: Attached is a patch that should fix the coalescing, including the clock skew detection. In the end I reorganized the code a bit, moving the check at the end, after the clock skew det

Re: [HACKERS] what to revert

2016-05-10 Thread Tomas Vondra
Hi, On 05/10/2016 07:36 PM, Robert Haas wrote: On Tue, May 10, 2016 at 12:31 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: The following table shows the differences between the disabled and reverted cases like this: sum('reverted' results with N c

Re: [HACKERS] what to revert

2016-05-10 Thread Tomas Vondra
that you consider that to be a "clearly measurable difference". I mean, it was measured and it is a difference, but it seems to be well within the noise. Even though it is based on 150 samples, I'm not sure we should consider it statistically significant. Well, luckily we're i

Re: [HACKERS] what to revert

2016-05-10 Thread Tomas Vondra
Hi, On 05/10/2016 10:29 AM, Kevin Grittner wrote: On Mon, May 9, 2016 at 9:01 PM, Tomas Vondra <tomas.von...@2ndquadrant.com <mailto:tomas.von...@2ndquadrant.com>> wrote: Over the past few days I've been running benchmarks on a fairly large NUMA box (4 sockets, 32 cores / 64 wi

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-08 Thread Tomas Vondra
-- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services fkest_fixes_v2.patch Description: binary/octet-stream -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] what to revert

2016-05-06 Thread Tomas Vondra
Hi, On 05/04/2016 12:42 AM, Andres Freund wrote: On 2016-05-03 20:57:13 +0200, Tomas Vondra wrote: On 05/03/2016 07:41 PM, Andres Freund wrote: ... I'm pretty sure that I said that somewhere else at least once: But to be absolutely clear, I'm *not* really concerned with the performance

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-05 Thread Tomas Vondra
s he made a lot of effort to salvage it over the last few days, so I'd like to set the record straight. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-04 Thread Tomas Vondra
ing only with FKs and not with bare unique indexes. FWIW it's a real-world problem with multi-column FKs. As David pointed out upthread, a nice example of this issue is Q9 in the TPC-H bench, where the underestimate leads to HashAggregate and then OOM failure. regards -- Tomas Vondra

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-04 Thread Tomas Vondra
Hi, On 05/04/2016 08:54 PM, Tom Lane wrote: David Rowley <david.row...@2ndquadrant.com> writes: On 4 May 2016 at 09:18, David Rowley <david.row...@2ndquadrant.com> wrote: On 4 May 2016 at 02:10, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: There are probably a few

Re: [HACKERS] what to revert

2016-05-03 Thread Tomas Vondra
can get us some numbers, but I haven't been following the snapshot_too_old so I'll need some guidance on what to test. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers maili

Re: [HACKERS] what to revert

2016-05-03 Thread Tomas Vondra
k it's a positive development that we can take this attitude to reverting patches. It should not be seen as a big personal failure, because it isn't. Stigmatizing reverts incentivizes behavior that leads to bad outcomes. Absolutely +1 -- Tomas Vondra http://www.2ndQuadrant.com Post

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-03 Thread Tomas Vondra
e doing that. If this gets reverted, then probably 015e88942aa50f0d419ddac00e63bb06d6e62e86 should also be reverted. Probably. I think the code is fine / correct, but as the FK estimation patch was the only user, there's not much point in keeping it if that gets reverted. regards -- Tomas Vondra

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-03 Thread Tomas Vondra
atch is improving estimates with multi-column foreign keys. I believe that covers a vast majority of foreign keys in the wild. If that's deemed insufficient, we'll have to resort to a more complex improvement, perhaps something akin to the cache proposed in in the unijoin patch. But if that's requir

Re: [HACKERS] multivariate statistics v14

2016-04-10 Thread Tomas Vondra
ason why I said before below, but it was never seriously considered. I would likely have said this myself but didn't even get that far. Your contribution was useful and went further than anybody else's review, so thank you. 100% agreed. Thanks for the useful feedback. -- Tomas Vond

Re: [HACKERS] multivariate statistics v14

2016-04-10 Thread Tomas Vondra
tion and implementation, including a bit of math and figures (which is what we don't have in READMEs or SGML). I haven't updated it recently, and it probably got buried in the thread, but perhaps this would be a better way to provide the context? regards -- Tomas Vondra htt

Re: [HACKERS] multivariate statistics v14

2016-04-09 Thread Tomas Vondra
Hi, On 04/09/2016 01:21 AM, Tatsuo Ishii wrote: From: Tomas Vondra <tomas.von...@2ndquadrant.com> ... My feedback regarding docs were: - There's no docs for pg_mv_statistic (should be added to "49. System Catalogs") - The word "multivariate statistics" or

Re: [HACKERS] multivariate statistics v14

2016-04-08 Thread Tomas Vondra
On 04/08/2016 05:55 PM, Robert Haas wrote: On Tue, Mar 29, 2016 at 11:18 AM, David Steele <da...@pgmasters.net> wrote: On 3/28/16 4:42 AM, Tomas Vondra wrote: Yes, those are valid omissions. I plan to address them, and I'd also considering adding a section to 65.1 (How the Planne

Re: [HACKERS] PATCH: use foreign keys to improve join estimates v1

2016-04-07 Thread Tomas Vondra
Hi, On 04/07/2016 01:23 PM, Simon Riggs wrote: On 7 April 2016 at 00:15, Tomas Vondra <tomas.von...@2ndquadrant.com <mailto:tomas.von...@2ndquadrant.com>> wrote: Right. Fixed. 0001 committed. I added comments and a fastpath when no triggers are present. For 0002, I w

Re: [HACKERS] PATCH: use foreign keys to improve join estimates v1

2016-04-06 Thread Tomas Vondra
r the index list is valid or not, so the extra field is justified. I think you're right. I've copied the logic for indexes, but clearly for foreign keys we don't need this flag. I've removed it. Also, it seems the patch forgot to update RelationDestroyRelation(). Right. Fixed. regards

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-05 Thread Tomas Vondra
Hi, On 04/04/2016 02:25 PM, Tomas Vondra wrote: On 04/04/2016 02:06 PM, Teodor Sigaev wrote: The above-described topic is currently a PostgreSQL 9.6 open item. Teodor, since you committed the patch believed to have created it, you own this open item. If that responsibility lies elsewhere

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-04 Thread Tomas Vondra
waiting of Tomas testing. Suppose, it isn't possible now? so, will do myself, after that I will publish results. Ah, damn. This completely slipped from my TODO list. I'll rerun the tests either today or tomorrow, and report the results here. regards -- Tomas Vondra http://www

Re: [HACKERS] PATCH: use foreign keys to improve join estimates v1

2016-04-03 Thread Tomas Vondra
On 04/03/2016 10:06 PM, Simon Riggs wrote: On 14 March 2016 at 19:42, Tomas Vondra <tomas.von...@2ndquadrant.com <mailto:tomas.von...@2ndquadrant.com>> wrote: ... I'd like to split this into 2 patches 1) Add FK info to relcache 2) use FK info in planner Would the credit f

Re: [HACKERS] Using quicksort for every external sort run

2016-04-03 Thread Tomas Vondra
ually observed data ... regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Using quicksort for every external sort run

2016-04-03 Thread Tomas Vondra
least some of the regressions? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: index-only scans with partial indexes

2016-03-31 Thread Tomas Vondra
ss those issues ;-) regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresq

Re: [HACKERS] PATCH: index-only scans with partial indexes

2016-03-30 Thread Tomas Vondra
t could be a net win. FWIW, as mentioned upthread, I added the following condition to decline ripping index predicates from base restrictinfo without understanding the reason to do so. U, I'm a bit confused. Which condition? regards -- Tomas Vondra http://www.2ndQuadran

Re: [HACKERS] improving GROUP BY estimation

2016-03-30 Thread Tomas Vondra
the comment as proposed by Dean. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services estimate-num-groups-v4.patch Description: binary/octet-stream -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] Using quicksort for every external sort run

2016-03-29 Thread Tomas Vondra
board* yet, but for int4 cases I could make 1MB work_mem queries faster with gigabytes of data on my laptop. I believe I tested various datum sort cases there, like "select count(distinct(foo)) from bar"; those are a very pure test of the patch. Well, I'd guess those conclusions ma

Re: [HACKERS] multivariate statistics v14

2016-03-28 Thread Tomas Vondra
On 03/26/2016 08:09 PM, Alvaro Herrera wrote: Tomas Vondra wrote: There are a few places where I reverted the pgindent formatting, because it seemed a bit too weird - the first one are the lists of function prototypes in common.h/mvstat.h, the second one are function calls to _greedy

Re: [HACKERS] multivariate statistics v14

2016-03-28 Thread Tomas Vondra
, those are valid omissions. I plan to address them, and I'd also considering adding a section to 65.1 (How the Planner Uses Statistics), explaining more thoroughly how the planner uses multivariate stats. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development,

Re: [HACKERS] multivariate statistics v14

2016-03-25 Thread Tomas Vondra
On 03/24/2016 06:45 PM, Alvaro Herrera wrote: Tomas Vondra wrote: +values[Anum_pg_mv_statistic_stamcv - 1] = PointerGetDatum(data); Why the double space (that's actually in several places in several of the patches). To align the whole block like this: nulls

Re: [HACKERS] Using quicksort for every external sort run

2016-03-23 Thread Tomas Vondra
Hi, On 03/24/2016 03:00 AM, Peter Geoghegan wrote: On Tue, Mar 22, 2016 at 3:35 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: I've tested the patch you've sent on 2016/3/11, which I believe is the last version. I haven't tuned the replacement_sort_mem at all. because my underst

Re: [HACKERS] multivariate statistics v14

2016-03-23 Thread Tomas Vondra
n the manual. Am I missing something? No, thanks for spotting those mistakes. I'll fix them and submit a new version of the patch - either later today or perhaps tomorrow. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: [HACKERS] multivariate statistics v14

2016-03-22 Thread Tomas Vondra
tistics.sgml:86:26:X: reference to non-existent ID "SQL-ALTERSTATISTICS" I believe this is because reference.sgml is missing a call to (per report by Alvaro Herrera). thanks -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Tra

Re: [HACKERS] [PATCH] fix DROP OPERATOR to reset links to itself on commutator and negator

2016-03-22 Thread Tomas Vondra
he empty line at the end of the function (before simple_heap_update). I think the comments will need rewording, but I'll leave that to a native speaker. regards Tomas -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Using quicksort for every external sort run

2016-03-22 Thread Tomas Vondra
Hi, On 03/22/2016 11:07 PM, Peter Geoghegan wrote: On Tue, Mar 22, 2016 at 2:27 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: Each query was executed 5x for each work_mem value (between 8MB and 1GB), and then a median of the runs was computed and that's what's on the "

Re: [HACKERS] multivariate statistics v14

2016-03-22 Thread Tomas Vondra
parts, who mentioned he might look at the patch this week. So I'm not sure splitting 0006 into smaller pieces makes sense at this point. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-h

Re: [HACKERS] multivariate statistics v14

2016-03-22 Thread Tomas Vondra
do expect them to be considerably more expensive. Is this a sufficient explanation? Of course, there's a fair amount of additional complexity that I have not mentioned here (e.g. selecting the right combination of stats). regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreS

Re: [HACKERS] checkpointer continuous flushing

2016-03-22 Thread Tomas Vondra
transactions where processed. Huh, that seems weird. That's not how percentiles or CDFs work, and I don't quite understand what would that tell us. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via p

Re: [HACKERS] multivariate statistics v14

2016-03-22 Thread Tomas Vondra
on whether combining the statistics by using the overlapping clauses as "conditions" is the right thing to do (or whether a more expensive approach is needed). None of that however invalidates the preceding patches. regards -- Tomas Vondra http://www.2ndQuadrant.com Po

Re: [HACKERS] multivariate statistics v14

2016-03-22 Thread Tomas Vondra
Hi, On 03/22/2016 06:53 AM, Jeff Janes wrote: On Sun, Mar 20, 2016 at 4:34 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: D'oh. Thanks for reporting. Attached is v16, hopefully fixing the few remaining whitespace issues. Hi Tomas, I'm trying out v16 against a common problem,

Re: [HACKERS] checkpointer continuous flushing

2016-03-22 Thread Tomas Vondra
. No brainer again. I infer from this figure that with the initial version 60% of transactions have trouble being processed on time, while this is maybe about 35% with the new version. Yep. -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA

Re: [HACKERS] checkpointer continuous flushing

2016-03-21 Thread Tomas Vondra
ter the transactions were scheduled. Again, the new code is winning. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] multivariate statistics v14

2016-03-21 Thread Tomas Vondra
tern". Many files need a pgindent pass. OK. -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] multivariate statistics v14

2016-03-21 Thread Tomas Vondra
ISTICS? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Aggregate

2016-03-20 Thread Tomas Vondra
Hi, On 03/21/2016 12:30 AM, David Rowley wrote: On 21 March 2016 at 09:47, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: ... I'm not sure changing the meaning of enable_hashagg like this is a good idea. It worked as a hard switch before, while with this change that would not be th

Re: [HACKERS] Combining Aggregates

2016-03-20 Thread Tomas Vondra
Hi, On 03/20/2016 04:48 AM, David Rowley wrote: On 17 March 2016 at 14:25, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: On 03/16/2016 12:03 PM, David Rowley wrote: Patch 2: This adds the serial/deserial aggregate infrastructure, pg_dump support, CREATE AGGREGATE changes, and nod

Re: [HACKERS] Parallel Aggregate

2016-03-20 Thread Tomas Vondra
old one was way much informative despite being a single line: /* Consider reasons to disable hashing, but only if we can sort instead */ BTW create_grouping_paths probably grew to a size when splitting it into smaller pieces would be helpful? regards -- Tomas Vondra http

Re: [HACKERS] PATCH: index-only scans with partial indexes

2016-03-20 Thread Tomas Vondra
one. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services partial-index-only-scan-v9.patch Description: binary/octet-stream -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

<    1   2   3   4   5   6   7   8   9   10   >