Re: [HACKERS] Declarative partitioning

2016-03-07 Thread Amit Langote
Hi Corey, Sorry for replying so late. On 2016/02/25 3:31, Corey Huinker wrote: > [ ... ] > So I would assume that we'd use a syntax that presumed the columns were in > a composite range type. > > Which means your creates would look like (following Robert Haas's implied > suggestion that we

Re: [HACKERS] extend pgbench expressions with functions

2016-03-07 Thread Fabien COELHO
- 32-b: add double functions, including double variables - 32-c: remove \setrandom support (advice to use \set + random instead) Here is a rebased version after Tom's updates, 33-b & 33-c. I also extended the floating point syntax to signed accept signed exponents, and changed the regexpr

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread José Luis Tallón
On 03/07/2016 07:30 AM, Tsunakawa, Takayuki wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Mark Kirkwood For cloud - in particular Openstack (which I am working with ATM), the biggest thing would be: - multi-master replication or

Re: [HACKERS] The plan for FDW-based sharding

2016-03-07 Thread Craig Ringer
On 5 March 2016 at 23:41, Kevin Grittner wrote: > > > I'd be really interested in some ideas on how that information might be > > usefully accessed. If we could write info on when to apply commits to the > > xlog in serializable mode that'd be very handy, especially when

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-03-07 Thread Amit Langote
Horiguchi-san, Thanks for a quick reply, :) On 2016/03/07 18:18, Kyotaro HORIGUCHI wrote: > At Mon, 7 Mar 2016 16:16:30 +0900, Amit Langote wrote: >> On 2016/03/07 13:02, Kyotaro HORIGUCHI wrote: >>> The 0001-P.. adds the following interface functions. >>> >>> I don't like to treat the target

Re: [HACKERS] ExecGather() + nworkers

2016-03-07 Thread Amit Kapila
On Mon, Jan 11, 2016 at 3:14 AM, Peter Geoghegan wrote: > > On Sun, Jan 10, 2016 at 9:13 AM, Robert Haas wrote: > > Now, you might wonder why it is that the leader cannot also sort runs, > just as a worker would. It's possible, but it isn't exactly >

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-03-07 Thread Shulgin, Oleksandr
On Fri, Mar 4, 2016 at 7:27 PM, Robert Haas wrote: > On Thu, Mar 3, 2016 at 2:48 AM, Shulgin, Oleksandr > wrote: > > On Wed, Mar 2, 2016 at 7:33 PM, Alvaro Herrera > > > wrote: > >> Shulgin, Oleksandr wrote: > >> >

Re: [HACKERS] pg_basebackup compression TODO item

2016-03-07 Thread Benedikt Grundmann
On Sun, Mar 6, 2016 at 7:36 PM, Euler Taveira wrote: > On 03-03-2016 14:44, Magnus Hagander wrote: > > On Thu, Mar 3, 2016 at 6:34 PM, Andres Freund > > wrote: > > > > On 2016-03-03 18:31:03 +0100, Magnus Hagander wrote:

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-03-07 Thread Tomas Vondra
Hi, On Mon, 2016-03-07 at 12:17 +0100, Shulgin, Oleksandr wrote: > On Fri, Mar 4, 2016 at 7:27 PM, Robert Haas > wrote: > On Thu, Mar 3, 2016 at 2:48 AM, Shulgin, Oleksandr > wrote: > > On Wed, Mar 2, 2016 at 7:33 PM,

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread MauMau
From: Craig Ringer -- We could help ORMs solve the N+1 SELECTs problem and help them avoid transferring vast join projections unnecessarily. That'd make PostgreSQL pretty compelling for exactly the users we're mostly too busy dismissing to

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Craig Ringer
On 7 March 2016 at 17:06, Tatsuo Ishii wrote: > > * Only 24 open source projects are listed as interoperable. > > Open Source Projects Using PostgreSQL > > https://wiki.postgresql.org/wiki/OpenSource_Projects_Using_PostgreSQL > > It's pity that some very popular OSS

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Craig Ringer
On 6 March 2016 at 13:46, Mark Kirkwood wrote: > > For cloud - in particular Openstack (which I am working with ATM), the > biggest thing would be: > > - multi-master replication > Working on it ;) > or failing that: > > - self managing single master failover

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-03-07 Thread Amit Langote
On 2016/03/07 19:11, Amit Langote wrote: > we should re-introduce[1] a fixed-size char st_progress_message[] field. Sorry, that [1] does not refer to anything, just a leftover from my draft. I thought I had a link handy for an email where some sort of justification was given as to why

Re: [HACKERS] Greeting for coming back, and where is PostgreSQL going

2016-03-07 Thread Michael Paquier
On Sun, Mar 6, 2016 at 2:28 PM, MauMau wrote: > I'm relieved to know that community people use Emacs for editing SGML/XML. > My main editor on Linux is Emacs. Yes, I'm using emacs too for sgml editing. This proves to be quick handy at the end. -- Michael -- Sent via

Re: [HACKERS] WIP: Upper planner pathification

2016-03-07 Thread Amit Kapila
On Mon, Mar 7, 2016 at 11:52 AM, Peter Geoghegan wrote: > > On Sun, Mar 6, 2016 at 9:59 PM, Tom Lane wrote: > > Perhaps it was intentional when written, but if Robert's advice is correct > > that the new upper-planner path nodes should copy up parallel_degree

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-07 Thread Etsuro Fujita
On 2016/03/05 5:45, Robert Haas wrote: Some comments on the latest version. I haven't reviewed the postgres_fdw changes in detail here, so this is just about the core changes. Thank you for taking the time to review the patch! I see that show_plan_tlist checks whether the operation is any

Re: [HACKERS] Proposal: RETURNING primary_key()

2016-03-07 Thread Tom Lane
Craig Ringer writes: > On 8 March 2016 at 08:56, Igal @ Lucee.org wrote: >> I'm not sure why it was not accepted at the end? > The biggest issue, though it might not be clear from that thread, is that > what exactly it means to "return generated keys" is

Re: [HACKERS] silent data loss with ext4 / all current versions

2016-03-07 Thread Michael Paquier
On Mon, Mar 7, 2016 at 3:38 PM, Andres Freund wrote: > On 2016-03-05 19:54:05 -0800, Andres Freund wrote: >> I started working on this; delayed by taking longer than planned on the >> logical decoding stuff (quite a bit complicated by >>

Re: [HACKERS] The plan for FDW-based sharding

2016-03-07 Thread Craig Ringer
On 7 March 2016 at 23:02, Robert Haas wrote: > On Fri, Mar 4, 2016 at 11:17 PM, Craig Ringer > wrote: > > If FDW-based sharding works, I'm happy enough, I have no horse in this > race. > > If it doesn't work I don't much care either. What I'm

Re: [HACKERS] Badly designed error reporting code in controldata_utils.c

2016-03-07 Thread Andres Freund
On 2016-03-08 13:45:25 +0900, Michael Paquier wrote: > On Mon, Mar 7, 2016 at 10:26 AM, Andres Freund wrote: > > FWIW I'm considering implementing elog/ereport properly for the > > frontend. We've grown several hacks around that not being present, and > > I think those by now

Re: [HACKERS] Badly designed error reporting code in controldata_utils.c

2016-03-07 Thread Tom Lane
Joe Conway writes: > Committed/pushed with exit(EXIT_FAILURE) Thanks! I lit off a new run on gaur/pademelon to confirm. Should have results in six hours or so... regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [PATH] Jsonb, insert a new value into an array at arbitrary position

2016-03-07 Thread Vitaly Burovoy
On 2016-02-29 17:19+00, Dmitry Dolgov <9erthali...@gmail.com> wrote: On 2016-02-24 19:37+00, Petr Jelinek wrote: >> Also this patch needs documentation. > I've added new version in attachments, thanks. Hello! The first pass of a review is below. 1. Rename the "flag"

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Michael Paquier
On Mon, Mar 7, 2016 at 9:25 PM, Craig Ringer wrote: > pg_rewind will help a lot there if it proves robust enough - FWIW, some of my colleagues are doing a lot of QE/QA on a HA solution based on pg_rewind, and it is proving to be quite stable for the moment, they are having a hard time breaking it

Re: [HACKERS] silent data loss with ext4 / all current versions

2016-03-07 Thread Andres Freund
Hi, On 2016-03-08 12:01:18 +0900, Michael Paquier wrote: > I have spent a couple of hours looking at that in details, and the > patch is neat. Cool. Doing some more polishing right now. Will be back with an updated version soonish. Did you do some testing? > + * This routine ensures that,

Re: [HACKERS] Freeze avoidance of very large table.

2016-03-07 Thread Kyotaro HORIGUCHI
Hello, thank you for updating this tool. At Mon, 7 Mar 2016 14:03:08 -0500, Robert Haas wrote in > On Mon, Mar 7, 2016 at 12:41 PM, Masahiko Sawada > wrote: > > Attached latest

[HACKERS] Allowing to run a buildfarm animal under valgrind

2016-03-07 Thread Andres Freund
Hi, I'm setting up a buildfarm animal that runs under valgrind. Unfortunately there's not really any good solution to force make check et al. to start postgres wrapped in valgrind. For now I've resorted to adding something like sub replace_postgres { my $srcdir=$use_vpath ? "../pgsql/" :

Re: [HACKERS] silent data loss with ext4 / all current versions

2016-03-07 Thread Michael Paquier
On Tue, Mar 8, 2016 at 12:18 PM, Andres Freund wrote: > On 2016-03-08 12:01:18 +0900, Michael Paquier wrote: >> I have spent a couple of hours looking at that in details, and the >> patch is neat. > > Cool. Doing some more polishing right now. Will be back with an updated >

Re: [HACKERS] pam auth - add rhost item

2016-03-07 Thread Haribabu Kommi
On Tue, Dec 29, 2015 at 10:46 AM, Grzegorz Sampolski wrote: > Hi. > I thought link on commitfest to github url was sufficient. > Sorry. Attached new patch. I reviewed and tested the patch. With the addition of new RHOST member to the passed items in the PAM authentication

Re: [HACKERS] Badly designed error reporting code in controldata_utils.c

2016-03-07 Thread Michael Paquier
On Mon, Mar 7, 2016 at 10:26 AM, Andres Freund wrote: > FWIW I'm considering implementing elog/ereport properly for the > frontend. We've grown several hacks around that not being present, and > I think those by now have a higher aggregate complexity than a proper >

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Tsunakawa, Takayuki
Hello, Josh, > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Josh berkus> > Crossing this over to pgsql-advocacy list where it really belongs. > That's what that list is *for*. > > Especially since the discussion on -hackers has focused on

Re: [HACKERS] Allowing to run a buildfarm animal under valgrind

2016-03-07 Thread Andres Freund
Hi, On 2016-03-07 17:39:30 -0800, Andres Freund wrote: > I'm setting up a buildfarm animal that runs under valgrind. Which is now running as 'skink'. The first failed due to a missing trick in the wrapper script, but the second one looks like it had a legit issue:

[HACKERS] empty array case in plperl_ref_from_pg_array not handled correctly

2016-03-07 Thread Andres Freund
Hi, Per the new valgrind animal we get: http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=skink=2016-03-08%2004%3A22%3A00 2016-03-08 05:56:05.566 UTC [56de6971.723:5] LOG: statement: select plperl_sum_array('{}'); ==1827== Invalid write of size 4 ==1827==at 0x14E35DD1: plperl_ref_from_pg_array

Re: [HACKERS] Proposal: RETURNING primary_key()

2016-03-07 Thread Craig Ringer
On 8 March 2016 at 08:56, Igal @ Lucee.org wrote: > > I'm not sure why it was not accepted at the end? The biggest issue, though it might not be clear from that thread, is that what exactly it means to "return generated keys" is poorly defined by JDBC, and not necessarily the

Re: [HACKERS] Optimizer questions

2016-03-07 Thread Tom Lane
Konstantin Knizhnik writes: > Attached please find improved version of the optimizer patch for LIMIT clause. This patch isn't anywhere close to working after 3fc6e2d7f5b652b4. (TBH, the reason I was negative about this upthread is that I had that one in the oven and

Re: [HACKERS] Badly designed error reporting code in controldata_utils.c

2016-03-07 Thread Michael Paquier
On Tue, Mar 8, 2016 at 1:51 PM, Andres Freund wrote: > On 2016-03-08 13:45:25 +0900, Michael Paquier wrote: >> On Mon, Mar 7, 2016 at 10:26 AM, Andres Freund wrote: >> > FWIW I'm considering implementing elog/ereport properly for the >> > frontend. We've

Re: [HACKERS] silent data loss with ext4 / all current versions

2016-03-07 Thread Andres Freund
On 2016-03-08 12:26:34 +0900, Michael Paquier wrote: > On Tue, Mar 8, 2016 at 12:18 PM, Andres Freund wrote: > > On 2016-03-08 12:01:18 +0900, Michael Paquier wrote: > >> I have spent a couple of hours looking at that in details, and the > >> patch is neat. > > > > Cool. Doing

Re: [HACKERS] Recovery test failure for recovery_min_apply_delay on hamster

2016-03-07 Thread Michael Paquier
On Wed, Mar 2, 2016 at 2:04 PM, Michael Paquier wrote: > Here are a couple of ways to address this problem: > 1) Remove the check before applying the delay > 2) Increase recovery_min_apply_delay to a time that will allow even > slow machines to see a difference. By

Re: [HACKERS] Typo in logicaldecoding docs

2016-03-07 Thread Tom Lane
David Rowley writes: > The attached fixes a small error in the logicaldecoding docs. Pushed, thanks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] silent data loss with ext4 / all current versions

2016-03-07 Thread Andres Freund
Hi, On 2016-03-08 16:21:45 +0900, Michael Paquier wrote: > + durable_link_or_rename(tmppath, path, ERROR); > + durable_rename(path, xlogfpath, ERROR); > You may want to add a (void) cast in front of those calls for correctness. "correctness"? This is neatnikism, not correctness. I've

Re: [HACKERS] Fix misspelling of "parallel"

2016-03-07 Thread Tom Lane
David Rowley writes: > The attached fixes a small spelling error in a comment. "grep" found another one. Pushed, thanks! regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Craig Ringer
On 7 March 2016 at 20:55, MauMau wrote: > From: Craig Ringer > -- > We could help ORMs solve the N+1 SELECTs problem and help them avoid > transferring vast join projections unnecessarily. That'd make PostgreSQL > pretty

Re: [HACKERS] silent data loss with ext4 / all current versions

2016-03-07 Thread Michael Paquier
On Tue, Mar 8, 2016 at 2:55 PM, Andres Freund wrote: > On 2016-03-08 12:26:34 +0900, Michael Paquier wrote: >> On Tue, Mar 8, 2016 at 12:18 PM, Andres Freund wrote: >> > On 2016-03-08 12:01:18 +0900, Michael Paquier wrote: >> >> I have spent a couple of

Re: [HACKERS] Using quicksort for every external sort run

2016-03-07 Thread Peter Geoghegan
On Mon, Feb 15, 2016 at 3:45 PM, Greg Stark wrote: > I was thinking about this over the past couple weeks. I'm starting to > think the quicksort runs gives at least the beginnings of a way > forward on this front. As I've already pointed out several times, I wrote a tool that

Re: [HACKERS] checkpointer continuous flushing - V18

2016-03-07 Thread Fabien COELHO
Hello Andres, Now I cannot see how having one context per table space would have a significant negative performance impact. The 'dirty data' etc. limits are global, not per block device. By having several contexts with unflushed dirty data the total amount of dirty data in the kernel

Re: [HACKERS] The plan for FDW-based sharding

2016-03-07 Thread Konstantin Knizhnik
On 03/07/2016 04:28 AM, Robert Haas wrote: On Fri, Mar 4, 2016 at 10:54 PM, Craig Ringer wrote: I've got to say that this is somewhat reminicient of the discussions around in-core pooling, where argument 1 is applied to justify excluding pooling from core/contrib. I

Re: [HACKERS] gzclose don't set properly the errno in pg_restore

2016-03-07 Thread Kyotaro HORIGUCHI
Hello, At Sun, 6 Mar 2016 22:09:20 -0300, Fabrízio de Royes Mello wrote in > Hi all, > > I'm facing with a strange error message during a failed pg_restore: > > pg_restore: processing data for table

[HACKERS] Splitting lengthy sgml files

2016-03-07 Thread Tatsuo Ishii
There are very lengthy (over 10k lines, for example) SGML files in docs. While working on translating docs using GitHub, I noticed that sometimes diffs are not showed in pull requests due to the limitation of GitHub, which makes me pretty difficult to review PR. Any chance to split those lengthy

Re: [HACKERS] Relation extension scalability

2016-03-07 Thread Robert Haas
On Fri, Mar 4, 2016 at 11:49 PM, Amit Kapila wrote: > I think one thing which needs more thoughts about this approach is that we > need to maintain some number of slots so that group extend for different > relations can happen in parallel. Do we want to provide

Re: [HACKERS] Splitting lengthy sgml files

2016-03-07 Thread Tom Lane
Tatsuo Ishii writes: > There are very lengthy (over 10k lines, for example) SGML files in > docs. While working on translating docs using GitHub, I noticed that > sometimes diffs are not showed in pull requests due to the limitation > of GitHub, which makes me pretty

Re: [HACKERS] The plan for FDW-based sharding

2016-03-07 Thread Robert Haas
On Fri, Mar 4, 2016 at 11:17 PM, Craig Ringer wrote: > If FDW-based sharding works, I'm happy enough, I have no horse in this race. > If it doesn't work I don't much care either. What I'm worried about is it if > works like partitioning using inheritance works - horribly

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-03-07 Thread Robert Haas
On Sun, Mar 6, 2016 at 11:02 PM, Kyotaro HORIGUCHI wrote: > At Sat, 5 Mar 2016 16:41:29 +0900, Amit Langote > wrote in >> On Sat, Mar 5, 2016 at 4:24 PM, Amit Langote

Re: [HACKERS] WIP: Upper planner pathification

2016-03-07 Thread Robert Haas
On Sun, Mar 6, 2016 at 10:32 AM, Tom Lane wrote: > Amit Kapila writes: >> On Sat, Mar 5, 2016 at 10:11 PM, Tom Lane wrote: >>> Is there some reason why hash and nestloop are safe but merge isn't? > >> I think it is because we

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-03-07 Thread Michael Paquier
On Sun, Mar 6, 2016 at 5:55 AM, Andrew Dunstan wrote: > On 03/05/2016 01:31 PM, Michael Paquier wrote: >> On Sat, Mar 5, 2016 at 11:34 PM, Andrew Dunstan >> wrote: >>> >>> Here is a translation into perl of the sed script, courtesy of the s2p >>>

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

2016-03-07 Thread Robert Haas
On Fri, Mar 4, 2016 at 2:37 PM, Andres Freund wrote: > On 2016-02-12 15:56:45 +0100, Andres Freund wrote: >> Hi, >> >> >> On 2016-02-10 23:26:20 -0500, Robert Haas wrote: >> > I think the part about whacking around the FDW API is a little more >> > potentially objectionable to

Re: [HACKERS] The plan for FDW-based sharding

2016-03-07 Thread Kevin Grittner
On Mon, Mar 7, 2016 at 6:13 AM, Craig Ringer wrote: > On 5 March 2016 at 23:41, Kevin Grittner wrote: >> The only place you *need* to vary from commit order for correctness >> is when there are overlapping SERIALIZABLE transactions, one >> modifies data

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-03-07 Thread Robert Haas
On Sat, Mar 5, 2016 at 7:22 AM, Dilip Kumar wrote: > On Wed, Mar 2, 2016 at 11:05 AM, Dilip Kumar wrote: >> And this latest result (no regression) is on X86 but on my local machine. >> >> I did not exactly saw what this new version of patch is doing

Re: [HACKERS] Is there a way around function search_path killing SQL function inlining?

2016-03-07 Thread Robert Haas
On Fri, Mar 4, 2016 at 9:29 PM, Regina Obe wrote: > I think the answer to this question is NO, but thought I'd ask. > > A lot of folks in PostGIS land are suffering from restore issues, > materialized view issues etc. because we have functions such as > > ST_Intersects > > Which

Re: [HACKERS] checkpointer continuous flushing - V16

2016-03-07 Thread Andres Freund
On 2016-03-01 16:06:47 +0100, Tomas Vondra wrote: > 1) HP DL380 G5 (old rack server) > - 2x Xeon E5450, 16GB RAM (8 cores) > - 4x 10k SAS drives in RAID-10 on H400 controller (with BBWC) > - RedHat 6 > - shared_buffers = 4GB > - min_wal_size = 2GB > - max_wal_size = 6GB > > 2) workstation with i5

Re: [HACKERS] Freeze avoidance of very large table.

2016-03-07 Thread Masahiko Sawada
On Sat, Mar 5, 2016 at 11:25 PM, Masahiko Sawada wrote: > On Sat, Mar 5, 2016 at 1:25 AM, Robert Haas wrote: >> On Wed, Mar 2, 2016 at 6:41 PM, Tom Lane wrote: >>> Jim Nasby writes: On 3/2/16 4:21

Re: [HACKERS] Fix handling of invalid sockets returned by PQsocket()

2016-03-07 Thread Michael Paquier
On Sun, Mar 6, 2016 at 12:52 PM, Alvaro Herrera wrote: > Peter Eisentraut wrote: >> On 2/17/16 10:52 PM, Michael Paquier wrote: >> > On Thu, Feb 18, 2016 at 1:58 AM, Alvaro Herrera >> > wrote: >> >> Michael Paquier wrote: >> >>> Hi all, >> >>>

Re: [HACKERS] (pgaudit) Audit log is not output after the SET ROLE.

2016-03-07 Thread David Steele
On 3/7/16 4:39 AM, Toshi Harada wrote: > > I am testing the pgaudit(https://commitfest.postgresql.org/9/463/). > (use "http://www.postgresql.org/message-id/56b0101b.6070...@pgmasters.net; > attached patch on 9.6-devel) > > I found strange thing. > > - After SET ROLE, part of the SQL is not the

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Tatsuo Ishii
> The killer there was that the plugin could only alter queries used by > Wordpress core. Nobody just uses Wordpress core. The whole reason > Wordpress became popular is the vast collection of plugins, themes, etc. > 90% of which are written by three stoned monkeys who once saw a PHP 4 > manual

Re: [HACKERS] WIP: Upper planner pathification

2016-03-07 Thread Greg Stark
On Mon, Mar 7, 2016 at 3:37 PM, Robert Haas wrote: > > The currently-committed code generates paths where nested loops and > hash joins get pushed beneath the Gather node, but does not generate > paths where merge joins have been pushed beneath the Gather node. And > the

Re: [HACKERS] WIP: Upper planner pathification

2016-03-07 Thread Tom Lane
Robert Haas writes: > The currently-committed code generates paths where nested loops and > hash joins get pushed beneath the Gather node, but does not generate > paths where merge joins have been pushed beneath the Gather node. And > the reason I didn't try to generate

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-07 Thread Robert Haas
On Mon, Mar 7, 2016 at 7:53 AM, Etsuro Fujita wrote: > Another option to avoid such a hazard would be to remove the two changes > from ExecInitModifyTable and create ExecAuxRowMarks and junk filters even in > the pushdown case. I made the changes because we won't use

Re: [HACKERS] checkpointer continuous flushing - V16

2016-03-07 Thread Andres Freund
On 2016-03-07 09:41:51 -0800, Andres Freund wrote: > > Due to the difference in amount of RAM, each machine used different scales - > > the goal is to have small, ~50% RAM, >200% RAM sizes: > > > > 1) Xeon: 100, 400, 6000 > > 2) i5: 50, 200, 3000 > > > > The commits actually tested are > > > >

Re: [HACKERS] checkpointer continuous flushing - V18

2016-03-07 Thread Andres Freund
On 2016-02-22 20:44:35 +0100, Fabien COELHO wrote: > > >>Random updates on 16 tables which total to 1.1GB of data, so this is in > >>buffer, no significant "read" traffic. > >> > >>(1) with 16 tablespaces (1 per table) on 1 disk : 680.0 tps > >>per second avg, stddev [ min q1 median d3 max ]

Re: [HACKERS] ExecGather() + nworkers

2016-03-07 Thread Peter Geoghegan
On Mon, Mar 7, 2016 at 4:04 AM, Amit Kapila wrote: > Your point is genuine, but OTOH let us say if max_parallel_degree = 1 means > parallelism is disabled then when somebody sets max_parallel_degree = 2, > then it looks somewhat odd to me that, it will mean that 1 worker

Re: [HACKERS] Freeze avoidance of very large table.

2016-03-07 Thread Robert Haas
On Mon, Mar 7, 2016 at 12:41 PM, Masahiko Sawada wrote: > Attached latest version optimisation patch. > I'm still consider regarding pg_upgrade regression test code, so I > will submit that patch later. I was thinking more about this today and I think that we don't

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Joshua D. Drake
On 03/07/2016 11:31 AM, MauMau wrote: Why don't we enrich the catalog? I'd like to hear ideas on how to enrich the catalog efficiently. It's ideal for software vendors and users to voluntarily add to the catalog. I think the product/software directory has vastly outlived its purpose. We

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread MauMau
From: Craig Ringer -- * Make a directory of software/services that can be used with PostgreSQL on the community web site (wiki.postgresql.org or www.postgresql.org). Software/services vendors and PostgreSQL developers/users can edit this

Re: [HACKERS] ExecGather() + nworkers

2016-03-07 Thread Robert Haas
On Mon, Mar 7, 2016 at 2:13 PM, Peter Geoghegan wrote: > On Mon, Mar 7, 2016 at 4:04 AM, Amit Kapila wrote: >> Your point is genuine, but OTOH let us say if max_parallel_degree = 1 means >> parallelism is disabled then when somebody sets

Re: [HACKERS] Proposal: RETURNING primary_key()

2016-03-07 Thread Joshua D. Drake
On 03/07/2016 12:32 PM, Igal @ Lucee.org wrote: The problem is that we do not always know in advance what the Primary Key is, and therefore a solution that was implemented in the pgjdbc I agree that the problem is that you don't always know what the primary key is. I would argue the

Re: [HACKERS] WIP: Upper planner pathification

2016-03-07 Thread Tom Lane
I wrote: >> Attached is a version that addresses today's concerns, and also finishes >> filling in the loose ends I'd left before, such as documentation and >> outfuncs.c support. I think this is in a committable state now, though >> I plan to read through the whole thing again. The extra

Re: [HACKERS] WIP: Upper planner pathification

2016-03-07 Thread David Rowley
On 8 March 2016 at 10:01, Tom Lane wrote: > I've pushed it now; we'll soon see if the buildfarm finds any problems. Fantastic! I'm looking forward to all the future optimisation opportunities that this opens up. Thanks for making this happen. -- David Rowley

Re: [HACKERS] Proposal: RETURNING primary_key()

2016-03-07 Thread Tom Lane
"Igal @ Lucee.org" writes: > On 3/7/2016 12:45 PM, Joshua D. Drake wrote: >> I agree that the problem is that you don't always know what the >> primary key is. >> I would argue the solution is to check before you write the query. Yeah. I'm rather suspicious of this proposal; I

Re: [HACKERS] unexpected result from to_tsvector

2016-03-07 Thread Dmitrii Golub
2016-02-23 20:53 GMT+03:00 Artur Zakirov : > Hello, > > Here is a little patch. It fixes this issue > http://www.postgresql.org/message-id/20160217080048.26357.49...@wrigleys.postgresql.org > > Without patch we get wrong result for the second email 't...@123-reg.ro': > >

Re: [HACKERS] checkpointer continuous flushing - V18

2016-03-07 Thread Andres Freund
On 2016-03-07 21:10:19 +0100, Fabien COELHO wrote: > Now I cannot see how having one context per table space would have a > significant negative performance impact. The 'dirty data' etc. limits are global, not per block device. By having several contexts with unflushed dirty data the total amount

Re: [HACKERS] Proposal: RETURNING primary_key()

2016-03-07 Thread Igal @ Lucee.org
On 3/7/2016 1:20 PM, Tom Lane wrote: Yeah. I'm rather suspicious of this proposal; I do not think it's actually very useful to return a primary-key value without any indication of what the primary key is. There are also corner cases where it seems pretty ill-defined. For example, suppose you

Re: [HACKERS] Proposal: RETURNING primary_key()

2016-03-07 Thread Igal @ Lucee.org
On 3/7/2016 12:45 PM, Joshua D. Drake wrote: I agree that the problem is that you don't always know what the primary key is. I would argue the solution is to check before you write the query. Sure, that would be great, but perhaps I should have give some more context: We have an

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Álvaro Hernández Tortosa
On 07/03/16 11:54, José Luis Tallón wrote: On 03/07/2016 07:30 AM, Tsunakawa, Takayuki wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Mark Kirkwood For cloud - in particular Openstack (which I am working with ATM), the biggest thing

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Josh berkus
MauMau, Crossing this over to pgsql-advocacy list where it really belongs. That's what that list is *for*. Especially since the discussion on -hackers has focused on new PostgreSQL Features, which while also good don't address the general question. On 03/05/2016 09:29 PM, MauMau wrote: > Hello,

Re: [HACKERS] checkpointer continuous flushing - V18

2016-03-07 Thread Fabien COELHO
Hello Andres, (1) with 16 tablespaces (1 per table) on 1 disk : 680.0 tps per second avg, stddev [ min q1 median d3 max ] <=300tps 679.6 ± 750.4 [0.0, 317.0, 371.0, 438.5, 2724.0] 19.5% (2) with 1 tablespace on 1 disk : 956.0 tps per second avg, stddev [ min q1 median d3 max ]

[HACKERS] Proposal: RETURNING primary_key()

2016-03-07 Thread Igal @ Lucee.org
THE ISSUE: In JDBC there is a flag called RETURN_GENERATED_KEYS -- https://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#RETURN_GENERATED_KEYS Which is left quite ambiguous, but in general it is used to return the "generated" Primary Key on INSERT/UPDATE/DELETE -- which is mostly

[HACKERS] New competition from Microsoft?

2016-03-07 Thread Josh berkus
All, http://blogs.microsoft.com/?p=67248 Once SQL Server is available on Linux, we're going to see more people using it as an alternative to PostgreSQL. Especially since they're picking up a lot of our better features, like R support. -- -- Josh Berkus Red Hat OSAS (any opinions are my

[HACKERS] Minor bug affecting ON CONFLICT lock wait log messages

2016-03-07 Thread Peter Geoghegan
Attached patch fixes a bug reported privately by Stephen this morning. He complained about deadlocking ON CONFLICT DO NOTHING statements. There were no exclusion constraints involved, and yet they were incorrectly indicated as being involved in log messages that related to these deadlocks. --

Re: [HACKERS] Parallel Aggregate

2016-03-07 Thread David Rowley
On 5 March 2016 at 07:25, Robert Haas wrote: > On Thu, Mar 3, 2016 at 11:00 PM, David Rowley >> 3. The code never attempts to mix and match Grouping Agg and Hash Agg >> plans. e.g it could be an idea to perform Partial Hash Aggregate -> >> Gather -> Sort -> Finalize Group

Re: [HACKERS] New competition from Microsoft?

2016-03-07 Thread Stephen Frost
* Joe Conway (m...@joeconway.com) wrote: > On 03/07/2016 01:43 PM, Josh berkus wrote: > > http://blogs.microsoft.com/?p=67248 > > > > Once SQL Server is available on Linux, we're going to see more people > > using it as an alternative to PostgreSQL. Especially since they're > > picking up a lot

Re: [HACKERS] Splitting lengthy sgml files

2016-03-07 Thread Tatsuo Ishii
> Surely that's a github bug that you should be complaining to them about? No, it's a known limitation: https://help.github.com/articles/what-are-the-limits-for-viewing-content-and-diffs-in-my-repository/ > I'm disinclined to split existing files because (a) it would complicate > back-patching

Re: [HACKERS] New competition from Microsoft?

2016-03-07 Thread Joe Conway
On 03/07/2016 01:43 PM, Josh berkus wrote: > http://blogs.microsoft.com/?p=67248 > > Once SQL Server is available on Linux, we're going to see more people > using it as an alternative to PostgreSQL. Especially since they're > picking up a lot of our better features, like R support. IANAL, but I

Re: [HACKERS] New competition from Microsoft?

2016-03-07 Thread Josh berkus
On 03/07/2016 01:43 PM, Josh berkus wrote: All, http://blogs.microsoft.com/?p=67248 Once SQL Server is available on Linux, we're going to see more people using it as an alternative to PostgreSQL. Especially since they're picking up a lot of our better features, like R support. Sorry, that

Re: [pgsql-advocacy] [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Adrian Klaver
On 03/07/2016 01:37 PM, Josh berkus wrote: > MauMau, > > Crossing this over to pgsql-advocacy list where it really belongs. > That's what that list is *for*. > > Especially since the discussion on -hackers has focused on new > PostgreSQL Features, which while also good don't address the general

Re: [HACKERS] New competition from Microsoft?

2016-03-07 Thread Joe Conway
On 03/07/2016 02:26 PM, Stephen Frost wrote: > * Joe Conway (m...@joeconway.com) wrote: >> On 03/07/2016 01:43 PM, Josh berkus wrote: >>> http://blogs.microsoft.com/?p=67248 >>> >>> Once SQL Server is available on Linux, we're going to see more people >>> using it as an alternative to PostgreSQL.

Re: [HACKERS] New competition from Microsoft?

2016-03-07 Thread Yorick de Wid
Microsoft is only trying to raise some awareness for their new ASP.NET 5 multi-platform, which works closely with MSSQL. I don't believe they are honestly interested in the *NIX user. They extend their userbase in the hope to get them onto Azure. -Original Message- From:

Re: [HACKERS] New competition from Microsoft?

2016-03-07 Thread Álvaro Hernández Tortosa
On 07/03/16 23:32, Joshua D. Drake wrote: On 03/07/2016 01:43 PM, Josh berkus wrote: All, http://blogs.microsoft.com/?p=67248 Once SQL Server is available on Linux, we're going to see more people using it as an alternative to PostgreSQL. Especially since they're picking up a lot of our

Re: [HACKERS] unexpected result from to_tsvector

2016-03-07 Thread Artur Zakirov
Hello, On 07.03.2016 23:55, Dmitrii Golub wrote: Hello, Should we added tests for this case? I think we should. I have added tests for teo...@123-stack.net and 1...@stack.net emails. 123_reg.ro is not valid domain name, bacause of symbol "_"

Re: [HACKERS] New competition from Microsoft?

2016-03-07 Thread Joshua D. Drake
On 03/07/2016 01:43 PM, Josh berkus wrote: All, http://blogs.microsoft.com/?p=67248 Once SQL Server is available on Linux, we're going to see more people using it as an alternative to PostgreSQL. Especially since they're picking up a lot of our better features, like R support. Yes but: 1.

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-03-07 Thread Jeff Janes
On Mon, Mar 7, 2016 at 3:17 AM, Shulgin, Oleksandr wrote: > On Fri, Mar 4, 2016 at 7:27 PM, Robert Haas wrote: >> >> On Thu, Mar 3, 2016 at 2:48 AM, Shulgin, Oleksandr >> wrote: >> > On Wed, Mar 2, 2016 at 7:33

Re: [HACKERS] WIP: Upper planner pathification

2016-03-07 Thread Robert Haas
On Mon, Mar 7, 2016 at 11:09 AM, Tom Lane wrote: > Robert Haas writes: >> The currently-committed code generates paths where nested loops and >> hash joins get pushed beneath the Gather node, but does not generate >> paths where merge joins have been

Re: [HACKERS] psql completion for ids in multibyte string

2016-03-07 Thread Robert Haas
On Sun, Mar 6, 2016 at 11:24 PM, Kyotaro HORIGUCHI wrote: > At Fri, 4 Mar 2016 12:30:17 -0500, Robert Haas wrote > in >> >>> I committed this and back-patched this but

  1   2   >