Re: [HACKERS] PoC: Partial sort

Hi! Tom committed upper planner pathification patch. Partial sort patch rebased to master is attached. It was quite huge rebase in planner part of the patch. But I think now patch becomes better, much more logical. It's probably, something was missed after rebase. I'm going to examine this

[HACKERS] [PATCH] Obsolete wording in PL/Perl comment

The comment in hv_store_string() says that negative key length to hv_store() for UTF-8 is not documented, and mentions that 5.6 doesn't track UTF-8-ness of keys. However, the negative length convention has been documented since 5.16¹, and 5.6 is no longer supported. The attached patch updates

Re: [HACKERS] memory leak in GIN

On Fri, Mar 11, 2016 at 11:40 PM, Jaime Casanova wrote: > Hi, > > On the spanish list, Felipe de Jesús Molina Bravo, reported a few days > back that a query that worked well in 9.4 consume all memory in 9.5. > With the self contained test he provided us i

Re: [HACKERS] Sanity checking for ./configure options?

On 2/26/16 9:29 PM, Peter Eisentraut wrote: Your code and comments suggest that you can specify the port to configure by setting PGPORT, but that is not the case. test == is not portable (bashism). Error messages should have consistent capitalization. Indentation in configure is two spaces.

Re: [HACKERS] Improve error handling in pltcl

On 3/3/16 8:51 AM, Pavel Stehule wrote: Hi I am testing behave, and some results looks strange Thanks for the review! postgres=# \sf foo CREATE OR REPLACE FUNCTION public.foo() RETURNS void LANGUAGE plpgsql AS $function$ begin raise exception sqlstate 'ZZ666' using message='hello,

Re: [HACKERS] improving GROUP BY estimation

On 4 March 2016 at 13:10, Tomas Vondra wrote: > The risk associated with over-estimation is that we may switch from > HashAggregate to GroupAggregate, and generally select paths better > suited for large number of rows. Not great, because the plan may not be >

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

On Thu, Mar 03, 2016 at 06:08:07AM +0100, Tomas Vondra wrote: > On 02/03/2016 06:46 AM, Noah Misch wrote: > >On Mon, Feb 01, 2016 at 07:03:45PM +0100, Tomas Vondra wrote: > >>On 12/22/2015 03:49 PM, Noah Misch wrote: > >>>On Mon, Feb 18, 2013 at 06:19:12PM -0300, Alvaro Herrera wrote: > I have

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

On 3/11/16 6:45 PM, Peter Geoghegan wrote: I'll add that if people like the interface you propose. (Overloading the VACUUM cost delay functions to cause a delay for amcheck functions, too). I thought that had already been overloaded by CIC, but I'm not finding reference to it... ANALYZE does

Re: [HACKERS] WIP: Upper planner pathification

Hello, I'm now checking the new planner implementation to find out the way to integrate CustomPath to the upper planner also. CustomPath node is originally expected to generate various kind of plan node, not only scan/join, and its interface is designed to support them. For example, we can expect

Re: [HACKERS] memory leak in GIN

Jeff Janes writes: > I bisected it down to: > d88976cfa1302e8dccdcbfe55e9e29faee8c0cdf is the first bad commit > commit d88976cfa1302e8dccdcbfe55e9e29faee8c0cdf > Author: Heikki Linnakangas > Date: Wed Feb 4 17:40:25 2015 +0200 > Use a

Re: [HACKERS] [patch] Proposal for \crosstabview in psql

On 3/13/16 12:48 AM, Pavel Stehule wrote: crosstabview is really visualization tool. **But now, there are not any other tool available from terminal.** So this can be significant help to all people who would to use this functionality. Not just the terminal either. Offhand I'm not aware of

Re: [HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

On Sat, Mar 12, 2016 at 1:25 AM, Kevin Grittner wrote: > On Thu, Mar 10, 2016 at 11:31 PM, Thomas Munro > wrote: > >> Here's a much simpler version with more comments > >> It handles the same set of isolation test specs. > > I'm impressed that

Re: [HACKERS] Parallel Aggregate

On Mon, Mar 14, 2016 at 8:44 AM, David Rowley wrote: > On 12 March 2016 at 16:31, David Rowley wrote: >> I've attached an updated patch which is based on commit 7087166, >> things are really changing fast in the grouping path area at

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

On Sun, 2016-03-13 at 18:46 -0400, Noah Misch wrote: > On Thu, Mar 03, 2016 at 06:08:07AM +0100, Tomas Vondra wrote: > > > > On 02/03/2016 06:46 AM, Noah Misch wrote: > > > > > > On Mon, Feb 01, 2016 at 07:03:45PM +0100, Tomas Vondra wrote: > > > > > > > > On 12/22/2015 03:49 PM, Noah Misch

Re: [HACKERS] Parallel Aggregate

Hi, I've done some testing with one of my data sets in an 8VPU virtual environment and this is looking really, really good. My test query is: SELECT pageview, sum(pageview_count) FROM fact_agg_2015_12 GROUP BY date_trunc('DAY'::text, pageview); The query returns 15 rows. The fact_agg table is

Re: [HACKERS] Parallel Aggregate

On 14 March 2016 at 14:16, James Sewell wrote: > I've done some testing with one of my data sets in an 8VPU virtual > environment and this is looking really, really good. > > My test query is: > > SELECT pageview, sum(pageview_count) > FROM fact_agg_2015_12 > GROUP BY

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

On Fri, Mar 11, 2016 at 7:08 AM, Dilip Kumar wrote: > > On Thu, Mar 10, 2016 at 8:26 PM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >> I don't think we can rely on median that much if we have only 3 runs. >> For 3 runs we can only apply Kornfeld method

Re: [HACKERS] Relation extension scalability

On 3/11/16 9:57 PM, Petr Jelinek wrote: I also think some kind of clamp is a good idea. It's not that uncommon to run max_connections significantly higher than 100, so the extension could be way larger than 16MB. In those cases this patch could actually make things far worse as

Re: [HACKERS] psql metaqueries with \gexec

On 2/22/16 1:01 PM, Corey Huinker wrote: In the mean time, update patch attached. Really attached this time. I'm getting a warning from this patch: common.c:947:8: warning: variable 'success' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]

Re: [HACKERS] psql metaqueries with \gexec

The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Still needs documentation. The new status of this patch is: Waiting

Re: [HACKERS] Reworks of CustomScan serialization/deserialization

> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Petr Jelinek > Sent: Friday, March 11, 2016 12:27 AM > To: Kaigai Kouhei(海外 浩平); pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Reworks of CustomScan

[HACKERS] pg_stat_get_progress_info(NULL) blows up

Hi I guess pg_stat_get_progress_info should either be strict (see attached) or check for NULL. -- Thomas Munro http://www.enterprisedb.com pg_stat_get_progress_info-strict.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

On Sat, Mar 12, 2016 at 7:46 PM, Matt Kelly wrote: > You can actually pretty easily produce a test case by setting up streaming > replication between servers running two different version of glibc. > > I actually wrote a tool that spins up a pair of VMs using vagrant and then

Re: [HACKERS] Recovery test failure for recovery_min_apply_delay on hamster

On Wed, Mar 9, 2016 at 9:05 PM, Michael Paquier wrote: > On Wed, Mar 9, 2016 at 12:29 PM, Alvaro Herrera > wrote: >> Michael Paquier wrote: >>> After sleeping (best debugger ever) on that, actually a way popped up >>> in my mind, and I propose

Re: [HACKERS] Parallel Aggregate

On 12 March 2016 at 16:31, David Rowley wrote: > I've attached an updated patch which is based on commit 7087166, > things are really changing fast in the grouping path area at the > moment, but hopefully the dust is starting to settle now. The attached patch fixes

Re: [HACKERS] improving GROUP BY estimation

Hi, On Sun, 2016-03-13 at 15:24 +, Dean Rasheed wrote: > On 4 March 2016 at 13:10, Tomas Vondra > wrote: > > > > The risk associated with over-estimation is that we may switch from > > HashAggregate to GroupAggregate, and generally select paths better > >

Re: [HACKERS] checkpointer continuous flushing - V18

On Sat, Mar 12, 2016 at 5:21 PM, Jeff Janes wrote: > Would the wiki be a good place for such tips? Not as formal as the > documentation, and more centralized (and editable) than a collection > of blog posts. That general direction makes sense, but I'm not sure if the Wiki

Re: [HACKERS] checkpointer continuous flushing - V18

On 3/13/16 6:30 PM, Peter Geoghegan wrote: On Sat, Mar 12, 2016 at 5:21 PM, Jeff Janes wrote: Would the wiki be a good place for such tips? Not as formal as the documentation, and more centralized (and editable) than a collection of blog posts. That general direction

Re: [HACKERS] Parallel Aggregate

Hi, Happy to test, really looking forward to seeing this stuff in core. The explain analyze is below: Finalize HashAggregate (cost=810142.42..810882.62 rows=59216 width=16) (actual time=2282.092..2282.202 rows=15 loops=1) Group Key: (date_trunc('DAY'::text, pageview_start_tstamp)) ->

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

On 2016/03/14 11:51, Tom Lane wrote: > Etsuro Fujita writes: >> On 2016/03/13 4:46, Andres Freund wrote: >>> ... The difference apears to be the >>> check that's now in build_simple_rel() - there was nothing hitting the >>> user mapping code before for file_fdw. >>

Re: [HACKERS] Parallel Aggregate

Hi again, I've been playing around with inheritance combined with this patch. Currently it looks like you are taking max(parallel_degree) from all the child tables and using that for the number of workers. For large machines it makes much more sense to use sum(parallel_degree) - but I've just

Re: [HACKERS] WIP: Upper planner pathification

> -Original Message- > From: Petr Jelinek [mailto:p...@2ndquadrant.com] > Sent: Monday, March 14, 2016 12:18 PM > To: Kaigai Kouhei(海外 浩平); Tom Lane; David Rowley > Cc: Robert Haas; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] WIP: Upper planner pathification > > On 14/03/16

Re: [HACKERS] Parallel Aggregate

On 14 March 2016 at 16:39, James Sewell wrote: > > I've been testing how this works with partitioning (which seems to be > strange, but I'll post separately about that) and something odd seems to be > going on now with the parallel triggering: > > postgres=# create

Re: [HACKERS] pg_stat_get_progress_info(NULL) blows up

On 2016/03/14 10:54, Thomas Munro wrote: > Hi > > I guess pg_stat_get_progress_info should either be strict (see > attached) or check for NULL. Thanks a lot for reporting and the patch. I think that's an oversight. Thanks, Amit -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Parallel Aggregate

On 14 March 2016 at 17:05, James Sewell wrote: > > Hi again, > > I've been playing around with inheritance combined with this patch. Currently > it looks like you are taking max(parallel_degree) from all the child tables > and using that for the number of workers. > >

Re: [HACKERS] Reworks of CustomScan serialization/deserialization

> On 14/03/16 02:53, Kouhei Kaigai wrote: > >> -Original Message- > >> From: pgsql-hackers-ow...@postgresql.org > >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Petr Jelinek > >> Sent: Friday, March 11, 2016 12:27 AM > >> To: Kaigai Kouhei(海外 浩平); pgsql-hackers@postgresql.org

Re: [HACKERS] WIP: Upper planner pathification

On 14/03/16 02:43, Kouhei Kaigai wrote: > > CustomPath node is originally expected to generate various kind of plan > node, not only scan/join, and its interface is designed to support them. > For example, we can expect a CustomPath that generates "CustomSort". > > On the other hands, upper path

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

On Fri, Mar 11, 2016 at 5:21 PM, Haribabu Kommi wrote: > > On Fri, Mar 11, 2016 at 12:00 AM, Amit Kapila wrote: > > > >> I am not able to find the reason for this error. This error is occurring > >> only > >> when the PostgreSQL is started as a

Re: [HACKERS] Parallel Aggregate

On 14 March 2016 at 14:52, James Sewell wrote: > One question - how is the upper limit of workers chosen? See create_parallel_paths() in allpaths.c. Basically the bigger the relation (in pages) the more workers will be allocated, up until max_parallel_degree. There is

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

Hi, On 2016/03/13 4:46, Andres Freund wrote: On 2016-03-12 11:56:24 -0500, Robert Haas wrote: On Fri, Mar 11, 2016 at 10:15 PM, Andres Freund wrote: On 2016-01-28 19:09:01 +, Robert Haas wrote: Only try to push down foreign joins if the user mapping OIDs match.

Re: [HACKERS] Reworks of CustomScan serialization/deserialization

On 14/03/16 02:53, Kouhei Kaigai wrote: >> -Original Message- >> From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Petr Jelinek >> Sent: Friday, March 11, 2016 12:27 AM >> To: Kaigai Kouhei(海外 浩平); pgsql-hackers@postgresql.org >> Subject:

[HACKERS] Obsolete comment in postgres_fdw.c

Hi, Here is the comments for foreign_join_ok in postgres_fdw.c: /* * Assess whether the join between inner and outer relations can be pushed down * to the foreign server. As a side effect, save information we obtain in this * function to PgFdwRelationInfo passed in. * * Joins that satisfy

Re: [HACKERS] Parallel Aggregate

Cool, I've been testing how this works with partitioning (which seems to be strange, but I'll post separately about that) and something odd seems to be going on now with the parallel triggering: postgres=# create table a as select * from base_p2015_11; SELECT 2000 postgres=# select * from a

Re: [HACKERS] Parallel Aggregate

On Mon, Mar 14, 2016 at 3:05 PM, David Rowley wrote: > > Things to try: > 1. alter table a add column ts_date date; update a set ts_date = > date_trunc('DAY',ts); vacuum full analyze ts; > 2. or, create index on a (date_trunc('DAY',ts)); analyze a; > 3. or for

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

Etsuro Fujita writes: > On 2016/03/13 4:46, Andres Freund wrote: >> ... The difference apears to be the >> check that's now in build_simple_rel() - there was nothing hitting the >> user mapping code before for file_fdw. > Exactly. > I'm not sure it's worth

Re: [HACKERS] Relation extension scalability

On Mon, Mar 14, 2016 at 5:02 AM, Jim Nasby wrote: > Well, 16MB is 2K pages, which is what you'd get if 100 connections were > all blocked and we're doing 20 pages per waiter. That seems like a really > extreme scenario, so maybe 4MB is a good compromise. That's unlikely

Re: [HACKERS] Relation extension scalability

On 14/03/16 03:29, Dilip Kumar wrote: On Mon, Mar 14, 2016 at 5:02 AM, Jim Nasby > wrote: Well, 16MB is 2K pages, which is what you'd get if 100 connections were all blocked and we're doing 20 pages per waiter. That seems

Re: [HACKERS] [GENERAL] OS X 10.11.3, psql, bus error 10, 9.5.1

On 12 Mar 2016 10:58 pm, "Tom Lane" wrote: > > I wrote: > > That's confusing because it implies that -fno-common is the default, > > which it evidently is not. But anyway, my diagnosis is that you're > > breaking something about the linker's behavior with that switch. > I

Re: [HACKERS] Performance improvement for joins where outer side is unique

On 12 March 2016 at 11:43, Tom Lane wrote: > I wrote: >> I wondered why, instead of inventing an extra semantics-modifying flag, >> we couldn't just change the jointype to *be* JOIN_SEMI when we've >> discovered that the inner side is unique. > > BTW, to clarify: I'm not

Re: [HACKERS] auto_explain sample rate

On Sat, Mar 12, 2016 at 3:20 PM, Julien Rouhaud wrote: > On 11/03/2016 17:55, Robert Haas wrote: > > On Fri, Mar 11, 2016 at 11:33 AM, Tomas Vondra > > wrote: > >> A bit late, but I think we should rename the GUC variable to > >>

Re: [HACKERS] Refectoring of receivelog.c

On Sun, Mar 13, 2016 at 12:15 AM, Tomas Vondra wrote: > On 03/11/2016 11:15 AM, Magnus Hagander wrote: > >> >> ... > >> >> >> Pushed with updated comments and a named stsruct. >> > > Pretty sure this memset call in pg_basebackup.c is incorrect, as it passes >