Re: [HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Fabien COELHO
About the patch: I'm generally in favor of providing more options to pgbench, especially if it can give optimization ideas to the performance conscious user. I think that the name should be "tpcb-like-plfunc": the script does not implement tpcb per spec, and such a function could be written

Re: [HACKERS] More replication race conditions

2017-08-26 Thread Michael Paquier
On Sun, Aug 27, 2017 at 12:03 PM, Tom Lane wrote: > And *another* replication test race condition just now: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dangomushi&dt=2017-08-26%2019%3A37%3A08 > > As best I can interpret this, it's pointing out that this bit in > src/test/recovery/t

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-08-26 Thread Fabien COELHO
Spending developer time to write code for the hypothetical someone running a psql version 11 linked to a libpq < 7.4, if it can even link, does not look like a very good investment... Anyway, here is required the update. The question is the server's version, not libpq. Ok. Modern psql does

Re: [HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Fabien COELHO
Hello Tom, I dunno, it seems like this proposal involves jacking up the test case and driving a completely different one underneath. There is no reason to consider that you've improved the benchmark results --- you've just substituted a different benchmark, one with no historical basis, and no

Re: [HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Fabien COELHO
Hello, If all the data is in memory and you have a system with fast fsyncs (or are running with fsync off, or unlogged tables, or synchronous_commit off), then the big bottleneck in pgbench is the amount of back and forth between the pgbench program and the backend. Sure. The throughput of

Re: [HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Tom Lane
Jeff Janes writes: > If all the data is in memory and you have a system with fast fsyncs (or are > running with fsync off, or unlogged tables, or synchronous_commit off), > then the big bottleneck in pgbench is the amount of back and forth between > the pgbench program and the backend. There are

Re: [HACKERS] More replication race conditions

2017-08-26 Thread Tom Lane
And *another* replication test race condition just now: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dangomushi&dt=2017-08-26%2019%3A37%3A08 As best I can interpret this, it's pointing out that this bit in src/test/recovery/t/009_twophase.pl: $cur_master->psql( 'postgres', "

Re: [HACKERS] More replication race conditions

2017-08-26 Thread Noah Misch
On Fri, Aug 25, 2017 at 12:09:00PM +0200, Petr Jelinek wrote: > On 24/08/17 19:54, Tom Lane wrote: > > sungazer just failed with > > > > pg_recvlogical exited with code '256', stdout '' and stderr > > 'pg_recvlogical: could not send replication command "START_REPLICATION SLOT > > "test_slot" LOG

[HACKERS] Re: Poor cost estimate with interaction between table correlation and partial indexes

2017-08-26 Thread Michael Malis
Do you think this is a reasonable approach? Should I start working on a patch based on the solution I described or is there some other approach I should look into? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.or

Re: [HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Peter Geoghegan
On Sat, Aug 26, 2017 at 4:59 PM, Jeff Janes wrote: > I still get a 2 fold improvement, from 13668 to 27036, when both > transactions are tested with -M prepared. > > I am surprised, I usually haven't seen that much difference for the default > queries between prepared or not, to the point that I g

Re: [HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Jeff Janes
On Sat, Aug 26, 2017 at 4:28 PM, Peter Geoghegan wrote: > On Sat, Aug 26, 2017 at 3:53 PM, Jeff Janes wrote: > > I get nearly a 3 fold speed up using the new transaction, from 9184 to > 26383 > > TPS, on 8 CPU machine using scale 50 and: > > > > PGOPTIONS="-c synchronous_commit=off" pgbench -c32

Re: [HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Peter Geoghegan
On Sat, Aug 26, 2017 at 3:53 PM, Jeff Janes wrote: > I get nearly a 3 fold speed up using the new transaction, from 9184 to 26383 > TPS, on 8 CPU machine using scale 50 and: > > PGOPTIONS="-c synchronous_commit=off" pgbench -c32 -j32 -T60 -b tpcb-like What about with "-M prepared"? I think that m

[HACKERS] Poor cost estimate with interaction between table correlation and partial indexes

2017-08-26 Thread Michael Malis
Hi. I'm looking to get started contributing code to Postgres. A small issue I'm aware of that I think would make a good first contribution is a poor cost estimate made due to an interaction between table correlation and partial indexes. Currently the planner assumes that when an index is perfectly

[HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-26 Thread Jeff Janes
If all the data is in memory and you have a system with fast fsyncs (or are running with fsync off, or unlogged tables, or synchronous_commit off), then the big bottleneck in pgbench is the amount of back and forth between the pgbench program and the backend. There are 7 commands per transaction.

Re: [HACKERS] subscription worker signalling wal writer too much

2017-08-26 Thread Jeff Janes
On Mon, Jul 3, 2017 at 8:26 PM, Jeff Janes wrote: > On Sat, Jun 24, 2017 at 5:09 PM, Andres Freund wrote: > >> On 2017-06-15 15:06:43 -0700, Jeff Janes wrote: >> >> >> > >> > Wouldn't it >> > better, and much simpler, just to have reverted the first change once >> the >> > second one was done? >

Re: [HACKERS] Patch: add --if-exists to pg_recvlogical

2017-08-26 Thread Rosser Schwarz
On Fri, Aug 25, 2017 at 12:34 PM, Robert Haas wrote: > On Sun, May 21, 2017 at 3:04 PM, Rosser Schwarz > wrote: > > While doing some scripting around pg_recvlogical at $work, I found a need > > for $subject. Attached, find a patch to that effect... Please add this to commitfest.postgresql.org

[HACKERS] Back-branch release notes up for review

2017-08-26 Thread Tom Lane
I've drafted notes for next week's brown-paper-bag releases. If you want to review, see https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=f1b10496a55a64b2872633850e55a2cd9d1c9108 regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] [COMMITTERS] pgsql: pg_test_timing: Add NLS

2017-08-26 Thread Peter Eisentraut
On 7/6/17 14:56, Alvaro Herrera wrote: > We (well, Carlos Chapi, who's doing the translation work now) just > noticed that this has a bug in this line > > + printf("%6s %10s %10s\n", _("< us"), _("% of total"), _("count")); > > _() marks the strings with the c-string flag, which means that th

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-08-26 Thread Tom Lane
Fabien COELHO writes: > Spending developer time to write code for the hypothetical someone running > a psql version 11 linked to a libpq < 7.4, if it can even link, does not > look like a very good investment... Anyway, here is required the update. The question is the server's version, not libp

Re: [HACKERS] psql - add ability to test whether a variable exists

2017-08-26 Thread Fabien COELHO
Here is a version with the :{?varname} syntax. It looks much better for me. I'll admit that it looks better to me as well:-) -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hack

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-08-26 Thread Fabien COELHO
Hello Tom, I think you are taking unreasonable shortcuts here: + SetVariable(pset.vars, "SERVER_VERSION_NAME", PQparameterStatus(pset.db, "server_version")); The existing code in connection_warnings() does this: const char *server_version; /* Try to get full tex

Re: [HACKERS] psql - add ability to test whether a variable exists

2017-08-26 Thread Corey Huinker
On Sat, Aug 26, 2017 at 2:09 PM, Pavel Stehule wrote: > > > 2017-08-26 19:55 GMT+02:00 Fabien COELHO : > >> >> Any colon prefixed syntax can be made to work because it is enough for >>> the lexer to detect and handle... so >>> >>> :{defined varname} >>> >>> may be an option, although I do not li

Re: [HACKERS] psql - add ability to test whether a variable exists

2017-08-26 Thread Pavel Stehule
2017-08-26 19:55 GMT+02:00 Fabien COELHO : > > Any colon prefixed syntax can be made to work because it is enough for the >> lexer to detect and handle... so >> >> :{defined varname} >> >> may be an option, although I do not like the space much because it adds >> some fuzzyness in the lexer which

Re: [HACKERS] psql - add ability to test whether a variable exists

2017-08-26 Thread Fabien COELHO
Any colon prefixed syntax can be made to work because it is enough for the lexer to detect and handle... so :{defined varname} may be an option, although I do not like the space much because it adds some fuzzyness in the lexer which has to process it. It is probably doable, though. I like

Re: [HACKERS] Build failure on thrips

2017-08-26 Thread Michael Meskes
> woodlouse does not do ecpg-check, and Tom is right anyway. I need to > get a large sign nailed to my forehead that says "Postgres is C, and > declarations must come first." I didn't notice either. I guess /me is getting rusty, sigh. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Mich

Re: [HACKERS] Build failure on thrips

2017-08-26 Thread Michael Meskes
> Unless someone has a better idea, I'd suggest working around > that like so: > > #ifdef WIN32 > #ifdef _MSC_VER/* requires MSVC */ >    _configthreadlocale(_ENABLE_PER_THREAD_LOCALE); > #endif > #endif Let's try, committed, thanks again Tom. Michael -- Michael Meskes Michael a

Re: [HACKERS] Build failure on thrips

2017-08-26 Thread Christian Ullrich
* Michael Meskes wrote: >> It's not just thrips. Of the Windows machines that have reported in >> since >> that commit, only currawong and baiji passed. Although lorikeet, >> woodlouse, bowerbird, and brolga are showing green, this proves >> nothing >> because they're all configured to skip the

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-08-26 Thread Tom Lane
Fabien COELHO writes: > So basically the only thing needed from Robert & you seems to change > "11.0" to "11devel", which is fine with me. > The attached v5 does that. I think you are taking unreasonable shortcuts here: + SetVariable(pset.vars, "SERVER_VERSION_NAME", PQparameterStatus(ps

Re: [HACKERS] Build failure on thrips

2017-08-26 Thread Michael Meskes
> It's not just thrips.  Of the Windows machines that have reported in > since > that commit, only currawong and baiji passed.  Although lorikeet, > woodlouse, bowerbird, and brolga are showing green, this proves > nothing > because they're all configured to skip the ecpg check (... I wonder > why)

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-08-26 Thread Fabien COELHO
Hello Tom, I understand that you would prefer VERSION_NAME to show something like "11devel, server 9.6.4" No, that's not what I said. I'm just complaining that as the patch stands it will set SERVER_NAME to "11.0", where I think it should say "11devel" (as of today). Ok. [...] VE

Re: [HACKERS] Build failure on thrips

2017-08-26 Thread Tom Lane
I wrote: > thrips is showing some weird syntax errors that I suspect mean it's > misinterpreting "_configthreadlocale(_ENABLE_PER_THREAD_LOCALE);" > as a declaration. Actually, looking closer, isn't the problem that you've stuck a function call into the midst of declarations? Some compilers will

Re: [HACKERS] Build failure on thrips

2017-08-26 Thread Tom Lane
Michael Meskes writes: > The bug fix I committed for fixing setlocale in ECPG test cases on Windows > seems to work nicely on all buildfarm animals except thrips. Could anyone with > access to a similar systems please check what's going on or send the > precompiled file that creates the compilatio

Re: [HACKERS] psql - add ability to test whether a variable exists

2017-08-26 Thread Fabien COELHO
Hello Pavel, As a follow-up to the \if patch by Corey Huinker, here is a proposal to allow testing whether a client-side variable exists in psql. The syntax is as ugly as the current :'var' and :"var" things, but ISTM that this is the only simple option to have a working SQL-compatible syntax

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-08-26 Thread Tom Lane
Fabien COELHO writes: >> OK, but if human-friendly display is the use-case then it ought to >> duplicate what psql itself would print in, eg, the startup message about >> server version mismatch. The v4 patch does not, in particular it neglects >> PQparameterStatus(pset.db, "server_version"). Th

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2017-08-26 Thread Tomas Vondra
On 08/17/2017 12:06 PM, Adrien Nayrat wrote:> > Hello, > > There is no check of "statistics type/kind" in > pg_stats_ext_mcvlist_items and pg_histogram_buckets. > > select stxname,stxkind from pg_statistic_ext ; stxname | stxkind > ---+- stts3 | {h} stts2 | {m} > > So

[HACKERS] hash partitioning based on v10Beta2

2017-08-26 Thread yang...@highgo.com
Hi all, Now we have had the range / list partition, but hash partitioning is not implemented yet. Attached is a POC patch based on the v10Beta2 to add the hash partitioning feature. Although we will need more discussions about the syntax and other specifications before going ahead the project

[HACKERS] Build failure on thrips

2017-08-26 Thread Michael Meskes
The bug fix I committed for fixing setlocale in ECPG test cases on Windows seems to work nicely on all buildfarm animals except thrips. Could anyone with access to a similar systems please check what's going on or send the precompiled file that creates the compilation errors? Thanks. Michael --

Re: [HACKERS] Not listed as committer

2017-08-26 Thread Michael Meskes
> Ha, that's interesting. > > Should be fixed now, please try again.  Works, thanks Magnus. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Meskes at (Debian|Postgresql) dot Org Jabber: michael at xmpp dot meskes dot org VfL Borussia! Força Barça!

Re: [HACKERS] ECPG: WHENEVER statement with DO CONTINUE action

2017-08-26 Thread Michael Meskes
> Given that it's Friday evening in Europe, I'm betting Michael is gone > for the day.  In the interests of getting the buildfarm back to > green, > I'll see if I can fix this. Correct, thanks Tom. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Mes

[HACKERS] Re: [PATCH] Fix drop replication slot blocking instead of returning error

2017-08-26 Thread Noah Misch
On Thu, Aug 24, 2017 at 03:38:20PM +0200, Simone Gotti wrote: > I noticed that in postgres 10beta3, calling pg_drop_replication_slot on an > active slot will block until it's released instead of returning an error > like > done in pg 9.6. Since this is a change in the previous behavior and the docs

Re: [HACKERS] psql - add ability to test whether a variable exists

2017-08-26 Thread Pavel Stehule
2017-08-26 8:54 GMT+02:00 Fabien COELHO : > > Hello, > > As a follow-up to the \if patch by Corey Huinker, here is a proposal to > allow testing whether a client-side variable exists in psql. > > The syntax is as ugly as the current :'var' and :"var" things, but ISTM > that this is the only simple