Re: [HACKERS] RLS Design

2014-09-11 Thread Erik Rijkers
On Wed, September 10, 2014 23:50, Stephen Frost wrote: > [rls_9-10-2014.patch] I can't get this to apply; I attach the complaints of patch. Erik Rijkers -- git clone git://git.postgresql.org/git/postgresql.git master Cloning into 'master'... -- git branch * master patching file doc/src/s

Re: [HACKERS] about half processes are blocked by btree, btree is bottleneck?

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 06:08 AM, Xiaoyulei wrote: I use benchmarksql with more than 200 clients on pg 9.3.3. when the test is going on, I collect all the process stack. I found about 100 processes are blocked by btree insert. Another 100 are blocked by xloginsert. Does btree has bad performance in conc

Re: [HACKERS] Support for N synchronous standby servers

2014-09-11 Thread Amit Kapila
On Thu, Sep 11, 2014 at 9:10 AM, Michael Paquier wrote: > On Thu, Sep 11, 2014 at 5:21 AM, Heikki Linnakangas > wrote: > > On 08/28/2014 10:10 AM, Michael Paquier wrote: > >> > >> + #synchronous_standby_num = -1 # number of standbys servers using sync > >> rep > > > > > > To be honest, that's a h

Re: [HACKERS] pg_xlogdump --stats

2014-09-11 Thread Abhijit Menon-Sen
At 2014-08-21 10:06:39 +0300, hlinnakan...@vmware.com wrote: > > Committed the patch to add INT64_MODIFIER, with minor fixes. Thank you. > The new rm_identify method needs to be documented. […] > Perhaps add comments to the RmgrData struct, explaining > all of the methods. OK, I'll submit a patc

Re: [HACKERS] pg_xlogdump --stats

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 11:43 AM, Abhijit Menon-Sen wrote: At 2014-08-21 10:06:39 +0300, hlinnakan...@vmware.com wrote: I think the names that rm_identify returns should match those that the rm_desc functions print. I had originally started off trying to keep the output in sync, but it doesn't work very

Re: [HACKERS] inherit support for foreign tables

2014-09-11 Thread Etsuro Fujita
(2014/09/11 4:32), Heikki Linnakangas wrote: I had a cursory look at this patch and the discussions around this. Thank you! ISTM there are actually two new features in this: 1) allow CHECK constraints on foreign tables, and 2) support inheritance for foreign tables. How about splitting it int

Re: [HACKERS] pg_xlogdump --stats

2014-09-11 Thread Andres Freund
On 2014-09-11 12:14:42 +0300, Heikki Linnakangas wrote: > On 09/11/2014 11:43 AM, Abhijit Menon-Sen wrote: > >At 2014-08-21 10:06:39 +0300, hlinnakan...@vmware.com wrote: > >>I think the names that rm_identify returns should match those that the > >>rm_desc functions print. > > > >I had originally

Re: [HACKERS] pg_xlogdump --stats

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 12:22 PM, Andres Freund wrote: On 2014-09-11 12:14:42 +0300, Heikki Linnakangas wrote: On 09/11/2014 11:43 AM, Abhijit Menon-Sen wrote: At 2014-08-21 10:06:39 +0300, hlinnakan...@vmware.com wrote: I think the names that rm_identify returns should match those that the rm_desc func

[HACKERS] Fix MSVC isnan warning from e80252d4

2014-09-11 Thread David Rowley
The attached small patch fixes the following warning: src\backend\utils\adt\arrayfuncs.c(5613): warning C4013: '_isnan' undefined; assuming extern returning int [D:\Postgres\a\postgres.vcxproj] The fix is pretty much just a copy and paste from costsize.c Regards David Rowley isnan_msvc_fix.pa

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Heikki Linnakangas
On 09/10/2014 05:47 PM, Mitsumasa KONDO wrote: Hi, I find typo in your patch. Please confirm. @line 239 - agg->sum2_lag = 0; + agg->sum_lag = 0; Ah thanks, cood catch! And back patch is welcome for me. I've committed and backpatched this, as well as a patch to refactor the way the Poiss

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Heikki Linnakangas
On 08/30/2014 07:16 PM, Fabien COELHO wrote: + if (latency_limit) + printf("number of transactions above the %.1f ms latency limit: " INT64_FORMAT "\n", + latency_limit / 1000.0, latency_late); + Any reason not to report a percentage here? Yes:

Re: [HACKERS] inherit support for foreign tables

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 12:22 PM, Etsuro Fujita wrote: (2014/09/11 4:32), Heikki Linnakangas wrote: I had a cursory look at this patch and the discussions around this. Thank you! ISTM there are actually two new features in this: 1) allow CHECK constraints on foreign tables, and 2) support inheritance

Re: [HACKERS] Fix MSVC isnan warning from e80252d4

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 12:52 PM, David Rowley wrote: The attached small patch fixes the following warning: src\backend\utils\adt\arrayfuncs.c(5613): warning C4013: '_isnan' undefined; assuming extern returning int [D:\Postgres\a\postgres.vcxproj] The fix is pretty much just a copy and paste from costsiz

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
Hi, On 2014-09-10 12:17:34 +0530, Amit Kapila wrote: > include $(top_srcdir)/src/backend/common.mk > diff --git a/src/backend/postmaster/bgreclaimer.c > b/src/backend/postmaster/bgreclaimer.c > new file mode 100644 > index 000..3df2337 > --- /dev/null > +++ b/src/backend/postmaster/bgreclaim

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-11 Thread David Rowley
On Thu, Aug 28, 2014 at 6:23 AM, Tom Lane wrote: > > If the majority of the added code is code that will be needed for > less-bogus optimizations, it might be all right; but I'd kind of want to > see the less-bogus optimizations working first. > > That seems fair. Likely there'd be not a great dea

Re: [HACKERS] Support for N synchronous standby servers

2014-09-11 Thread Amit Kapila
On Thu, Aug 28, 2014 at 12:40 PM, Michael Paquier wrote: > > On Wed, Aug 27, 2014 at 2:46 PM, Rajeev rastogi > wrote: > > I have done some more review, below are my comments: > Thanks! > > > 1. There are currently two loops on *num_sync, Can we simplify the function SyncRepGetSynchronousNodes by

Re: [HACKERS] RLS Design

2014-09-11 Thread Stephen Frost
Erik, * Erik Rijkers (e...@xs4all.nl) wrote: > On Wed, September 10, 2014 23:50, Stephen Frost wrote: > > [rls_9-10-2014.patch] > > I can't get this to apply; I attach the complaints of patch. Thanks for taking a look at this! [...] > patching file src/include/catalog/catversion.h > Hunk #1 FA

Re: [HACKERS] inherit support for foreign tables

2014-09-11 Thread Etsuro Fujita
(2014/09/11 19:46), Heikki Linnakangas wrote: On 09/11/2014 12:22 PM, Etsuro Fujita wrote: (2014/09/11 4:32), Heikki Linnakangas wrote: I had a cursory look at this patch and the discussions around this. Thank you! ISTM there are actually two new features in this: 1) allow CHECK constraints

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Amit Kapila
On Fri, Jul 25, 2014 at 11:41 PM, Robert Haas wrote: > > Patch 4 adds infrastructure that allows one session to save all of its > non-default GUC values and another session to reload those values. > This was written by Amit Khandekar and Noah Misch. It allows > pg_background to start up the backg

Re: [HACKERS] inherit support for foreign tables

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 02:30 PM, Etsuro Fujita wrote: Actually, this patch allows the exact same thing to apply to foreign tables. My explanation was insufficient about that. Sorry for that. Great, that's what I thought. So, should I split the patch into the two? Yeah, please do. - Heikki -- Se

Re: [HACKERS] [TODO] Process pg_hba.conf keywords as case-insensitive

2014-09-11 Thread Robert Haas
On Wed, Sep 10, 2014 at 4:54 AM, Kyotaro HORIGUCHI wrote: > Finally I think that we need case-insensitive version of > get_role_id and() get_database_id() to acoomplish this patch'es > objective. (This runs full-scans on pg_database or pg_authid X() Any such thing is certainly grounds for rejecti

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Stephen Frost
* Pavel Stehule (pavel.steh...@gmail.com) wrote: > Can I help with something, it is there some open question? I had been hoping for a more definitive answer regarding this option for array_to_json, or even a comment about the change to row_to_json. Andrew- any thoughts on this? (that's what the p

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
Hello Heikki, Now that I've finished the detour and committed and backpatched the changes to the way latency is calculated, we can get back to this patch. It needs to be rebased. Before rebasing, I think that there are a few small problems with the modification applied to switch from an int

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Arthur Silva
I agree that there's no reason to fix an algorithm to it, unless maybe it's pglz. There's some initial talk about implementing pluggable compression algorithms for TOAST and I guess the same must be taken into consideration for the WAL. -- Arthur Silva On Thu, Sep 11, 2014 at 2:46 AM, Rahila Sye

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
(3) I wish that the maximum implied multiplier could be explicitely documented in the source code. From pg_rand48 source code, I think that it is 33.27106466687737 Small possibly buggy code attached, to show how I computed the above figure. -- Fabien.#include #include int main(void

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 09:37:07AM -0300, Arthur Silva wrote: > I agree that there's no reason to fix an algorithm to it, unless maybe it's > pglz. There's some initial talk about implementing pluggable compression > algorithms for TOAST and I guess the same must be taken into consideration > for t

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Robert Haas
Thanks for reviewing, Andres. On Thu, Sep 11, 2014 at 7:01 AM, Andres Freund wrote: >> +static void bgreclaim_quickdie(SIGNAL_ARGS); >> +static void BgreclaimSigHupHandler(SIGNAL_ARGS); >> +static void ReqShutdownHandler(SIGNAL_ARGS); >> +static void bgreclaim_sigusr1_handler(SIGNAL_ARGS); > > Th

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Amit Kapila
On Thu, Sep 11, 2014 at 6:32 PM, Robert Haas wrote: > > Thanks for reviewing, Andres. > > On Thu, Sep 11, 2014 at 7:01 AM, Andres Freund wrote: > >> +static void bgreclaim_quickdie(SIGNAL_ARGS); > >> +static void BgreclaimSigHupHandler(SIGNAL_ARGS); > >> +static void ReqShutdownHandler(SIGNAL_ARG

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
On 2014-09-11 09:02:34 -0400, Robert Haas wrote: > Thanks for reviewing, Andres. > > On Thu, Sep 11, 2014 at 7:01 AM, Andres Freund wrote: > >> +static void bgreclaim_quickdie(SIGNAL_ARGS); > >> +static void BgreclaimSigHupHandler(SIGNAL_ARGS); > >> +static void ReqShutdownHandler(SIGNAL_ARGS); >

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
Hello Heikki Now that I've finished the detour and committed and backpatched the changes to the way latency is calculated, we can get back to this patch. It needs to be rebased. Here is the rebase, which seems ok. See also the small issues raised aboud getPoissonRand in another email. -- F

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
> > We really need a more centralized way to handle error cleanup in > > auxiliary processes. The current state of affairs is really pretty > > helter-skelter. But for this patch, I think we should aim to mimic > > the existing style, as ugly as it is. I'm not sure whether Amit's got > > the log

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Robert Haas
On Wed, Sep 10, 2014 at 5:09 PM, Tomas Vondra wrote: > OK. So here's v13 of the patch, reflecting this change. With the exception of ExecChooseHashTableSize() and a lot of stylistic issues along the lines of what I've already complained about, this patch seems pretty good to me. It does three th

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Arthur Silva
On Wed, Sep 10, 2014 at 12:43 PM, Robert Haas wrote: > On Tue, Sep 9, 2014 at 10:08 AM, Arthur Silva wrote: > > I'm continuously studying Postgres codebase. Hopefully I'll be able to > make > > some contributions in the future. > > > > For now I'm intrigued about the extensive use of memory alig

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Mitsumasa KONDO
2014-09-11 22:01 GMT+09:00 k...@rice.edu : > On Thu, Sep 11, 2014 at 09:37:07AM -0300, Arthur Silva wrote: > > I agree that there's no reason to fix an algorithm to it, unless maybe > it's > > pglz. > Yes, it seems difficult to judge only the algorithm performance. We have to start to consider so

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Amit Kapila
On Thu, Sep 11, 2014 at 6:59 PM, Andres Freund wrote: > > > > We really need a more centralized way to handle error cleanup in > > > auxiliary processes. The current state of affairs is really pretty > > > helter-skelter. But for this patch, I think we should aim to mimic > > > the existing styl

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 9:22 AM, Andres Freund wrote: >> It's exactly the same as what bgwriter.c does. > > So what? There's no code in common, so I see no reason to have one > signal handler using underscores and the next one camelcase names. /me shrugs. It's not always possible to have things

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 9:32 AM, Arthur Silva wrote: > I thought all memory alignment was (or at least the bulk of it) handled > using some codebase wide macros/settings, otherwise how could different > parts of the code inter-op? Poking this area might suffice for some initial > testing to check

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Mitsumasa KONDO
2014-09-11 15:47 GMT+09:00 Fabien COELHO : > > Hello Robert, > > I am not objecting to the functionality; I'm objecting to bolting on >> ad-hoc operators one at a time. I think an expression syntax would >> let us do this in a much more scalable way. If I had time, I'd go do >> that, but I don'

Re: [HACKERS] Aussie timezone database changes incoming

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 12:20 AM, Craig Ringer wrote: > I shouldn't be surprised that Australia gets to change. While the cynic > in me thinks this is the usual USA-is-the-center-of-the-universe-ism, in > reality it makes sense given relative population and likely impact. Just because it makes se

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tom Lane
Robert Haas writes: > With the exception of ExecChooseHashTableSize() and a lot of stylistic > issues along the lines of what I've already complained about, this > patch seems pretty good to me. It does three things: > ... > (3) It allows the number of batches to increase on the fly while the > h

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
On 2014-09-11 09:48:10 -0400, Robert Haas wrote: > On Thu, Sep 11, 2014 at 9:22 AM, Andres Freund wrote: > > I wonder if we should recheck the number of freelist items before > > sleeping. As the latch currently is reset before sleeping (IIRC) we > > might miss being woken up soon. It very well mi

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane wrote: > Robert Haas writes: >> With the exception of ExecChooseHashTableSize() and a lot of stylistic >> issues along the lines of what I've already complained about, this >> patch seems pretty good to me. It does three things: >> ... >> (3) It allows t

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane wrote: >> Robert Haas writes: >>> (3) It allows the number of batches to increase on the fly while the >>> hash join is in process. >> Pardon me for not having read the patch yet, but what part of (3) >> wasn't there already? > EI

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 10:03 AM, Andres Freund wrote: > On 2014-09-11 09:48:10 -0400, Robert Haas wrote: >> On Thu, Sep 11, 2014 at 9:22 AM, Andres Freund >> wrote: >> > I wonder if we should recheck the number of freelist items before >> > sleeping. As the latch currently is reset before sleep

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
How should skipped transactions should be taken into account in the log file output, with and without aggregation? I assume we'll want to have some trace of skipped transactions in the logs. The problem with this point is that how to report something "not done" is unclear, especially as the

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Andres Freund
On 2014-09-11 10:32:24 -0300, Arthur Silva wrote: > Unaligned memory access received a lot attention in Intel post-Nehalen era. > So it may very well pay off on Intel servers. You might find this blog post > and it's comments/external-links interesting > http://lemire.me/blog/archives/2012/05/31/da

[HACKERS] gist vacuum seaq access

2014-09-11 Thread Костя Кузнецов
After discussion of gist seaq access in vaccum there are 2 issue: Heikki says :Vacuum needs to memorize the current NSN when it begins1) how i may getting corect NSN. Also i must setting F_DELETED flag on empty page and to clean parent from link on deleted_pages2) how i may getting parent of page f

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11 Září 2014, 15:31, Robert Haas wrote: > On Wed, Sep 10, 2014 at 5:09 PM, Tomas Vondra wrote: >> OK. So here's v13 of the patch, reflecting this change. > > With the exception of ExecChooseHashTableSize() and a lot of stylistic > issues along the lines of what I've already complained about, th

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Stephen Frost
Pavel, * Pavel Stehule (pavel.steh...@gmail.com) wrote: > I removed dynamic allocation and reduced patch size. This is certainly better, imv, though there are a couple of minor issues (extra semi-colons, extraneous whitespace, get_line_style was still changed to non-const, even though it doesn't

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11 Září 2014, 16:11, Tom Lane wrote: > Robert Haas writes: >> On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane wrote: >>> Robert Haas writes: (3) It allows the number of batches to increase on the fly while the hash join is in process. > >>> Pardon me for not having read the patch yet, but

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-09-11 Thread Petr Jelinek
On 10/09/14 13:13, Fujii Masao wrote: On Wed, Sep 10, 2014 at 1:45 AM, Petr Jelinek wrote: Hi, I recently wanted several times to have slave server prepared at certain point in time to reduce the time it takes for it to replay remaining WALs (say I have pg_basebackup -x on busy db for example)

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Merlin Moncure
On Thu, Sep 11, 2014 at 8:32 AM, Arthur Silva wrote: > > On Wed, Sep 10, 2014 at 12:43 PM, Robert Haas wrote: >> >> On Tue, Sep 9, 2014 at 10:08 AM, Arthur Silva wrote: >> > I'm continuously studying Postgres codebase. Hopefully I'll be able to >> > make >> > some contributions in the future. >>

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 10:11 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane wrote: >>> Robert Haas writes: (3) It allows the number of batches to increase on the fly while the hash join is in process. > >>> Pardon me for not having read the pat

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tom Lane
"Tomas Vondra" writes: > On 11 Září 2014, 16:11, Tom Lane wrote: >> Ah. Well, that would mean that we need a heuristic for deciding when to >> increase the number of buckets versus the number of batches ... seems >> like a difficult decision. > That's true, but that's not the aim of this patc

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-09-11 Thread Stephen Frost
* Albe Laurenz (laurenz.a...@wien.gv.at) wrote: > Etsuro Fujita wrote: > > I agree with you on that point. So, I've updated the patch to have the > > explicit flag, as you proposed. Attached is the updated version of the > > patch. In this version, I've also revised code and its comments a bit.

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 7:14 AM, David Rowley wrote: > Here's a quick demo, of the patch at work: > > test=# create table c (id int primary key); > CREATE TABLE > test=# create table b (id int primary key, c_id int not null references > c(id)); > CREATE TABLE > test=# create table a (id int primar

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Tom Lane
Merlin Moncure writes: > Be advised of the difficulties you are going to face here. Assuming > for a second there is no reason not to go unaligned on Intel and there > are material benefits to justify the effort, that doesn't necessarily > hold for other platforms like arm/power. Note that on ma

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Petr Jelinek
On 10/09/14 22:53, Robert Haas wrote: Here's what the other approach looks like. I can't really see doing this way and then only providing hooks for those two functions, so this is with hooks for all the send-side stuff. Original version: 9 files changed, 295 insertions(+), 3 deletions(-) This

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Andres Freund
On 2014-09-11 11:39:12 -0400, Tom Lane wrote: > Even on Intel, I'd wonder what unaligned accesses do to atomicity > guarantees and suchlike. They pretty much kill atomicity guarantees. Atomicity is guaranteed while you're inside a cacheline, but not once you span them. > This is not a big deal fo

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Andrew Dunstan
On 09/11/2014 08:29 AM, Stephen Frost wrote: * Pavel Stehule (pavel.steh...@gmail.com) wrote: Can I help with something, it is there some open question? I had been hoping for a more definitive answer regarding this option for array_to_json, or even a comment about the change to row_to_json. An

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-11 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 11, 2014 at 7:14 AM, David Rowley wrote: >> 5. I've added a flag to pg_class called relhasfkey. Currently this gets set >> to true when a foreign key is added, though I've added nothing to set it >> back to false again. I notice that relhasindex gets set back to

Re: [HACKERS] Support for N synchronous standby servers

2014-09-11 Thread Robert Haas
On Wed, Sep 10, 2014 at 11:40 PM, Michael Paquier wrote: > Currently two nodes can only have the same priority if they have the > same application_name, so we could for example add a new connstring > parameter called, let's say application_group, to define groups of > nodes that will have the same

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Pavel Stehule
Hi 2014-09-11 16:42 GMT+02:00 Stephen Frost : > Pavel, > > * Pavel Stehule (pavel.steh...@gmail.com) wrote: > > I removed dynamic allocation and reduced patch size. > > This is certainly better, imv, though there are a couple of minor > issues (extra semi-colons, extraneous whitespace, get_line_

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-09-11 Thread Tom Lane
Stephen Frost writes: > I have to admit that, while I applaud the effort made to have this > change only be to postgres_fdw, I'm not sure that having the > update/delete happening during the Scan phase and then essentially > no-op'ing the ExecForeignUpdate/ExecForeignDelete is entirely in-line > w

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11 Září 2014, 17:28, Tom Lane wrote: > "Tomas Vondra" writes: >> On 11 Z?? 2014, 16:11, Tom Lane wrote: >>> Ah. Well, that would mean that we need a heuristic for deciding when >>> to >>> increase the number of buckets versus the number of batches ... seems >>> like a difficult decision. >

Re: [HACKERS] RLS Design

2014-09-11 Thread Robert Haas
On Sat, Sep 6, 2014 at 2:54 AM, Brightwell, Adam wrote: > * Add ALTER TABLE { ENABLE | DISABLE } ROW LEVEL SECURITY - set flag > on table to allow for a default-deny capability. If RLS is enabled on a > table and has no policies, then a default-deny policy is automatically > applied. If RLS is

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 1:46 AM, Rahila Syed wrote: >>I will repeat the above tests with high load on CPU and using the benchmark > given by Fujii-san and post the results. > > Average % of CPU usage at user level for each of the compression algorithm > are as follows. > > CompressionMulti

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Andres Freund
On 2014-09-11 12:55:21 -0400, Robert Haas wrote: > I advise supporting pglz only for the initial patch, and adding > support for the others later if it seems worthwhile. The approach > seems to work well enough with pglz that it's worth doing even if we > never add the other algorithms. That appr

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Bruce Momjian
On Thu, Sep 11, 2014 at 12:55:21PM -0400, Robert Haas wrote: > I advise supporting pglz only for the initial patch, and adding > support for the others later if it seems worthwhile. The approach > seems to work well enough with pglz that it's worth doing even if we > never add the other algorithms

Re: [HACKERS] Commitfest status

2014-09-11 Thread Tomas Vondra
On 10.9.2014 22:39, Heikki Linnakangas wrote: > The bad news is that the rest don't seem to moving graduating from the > Needs Review state. ISTM that many patches (a) in 'needs review' actually have a review, or are being thoroughly discussed (b) in 'waiting on author' are not really waitin

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund wrote: > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: >> I advise supporting pglz only for the initial patch, and adding >> support for the others later if it seems worthwhile. The approach >> seems to work well enough with pglz that it's worth

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 2:47 AM, Fabien COELHO wrote: > Ok. I do agree that an expression syntax would be great! > > However, that would not diminish nor change much the amount and kind of code > necessary to add an operator or a function That's not really true. You can't really add abs(x) or ha

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Arthur Silva
On Thu, Sep 11, 2014 at 11:27 AM, Andres Freund wrote: > On 2014-09-11 10:32:24 -0300, Arthur Silva wrote: > > Unaligned memory access received a lot attention in Intel post-Nehalen > era. > > So it may very well pay off on Intel servers. You might find this blog > post > > and it's comments/exte

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Andres Freund
On 2014-09-11 13:04:43 -0400, Robert Haas wrote: > On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund > wrote: > > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: > >> I advise supporting pglz only for the initial patch, and adding > >> support for the others later if it seems worthwhile. The appr

Re: [HACKERS] implement subject alternative names support for SSL connections

2014-09-11 Thread Alexey Klyukin
On Mon, Sep 8, 2014 at 8:04 PM, Heikki Linnakangas wrote: > On 09/05/2014 07:30 PM, Alexey Klyukin wrote: >> The error does not state whether the names comes from the CN or from >> the SAN section. > > > I'd reword that slightly, to: > > psql: server certificate for "example.com" (and 2 other nam

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Arthur Silva
On Thu, Sep 11, 2014 at 12:39 PM, Tom Lane wrote: > Merlin Moncure writes: > > Be advised of the difficulties you are going to face here. Assuming > > for a second there is no reason not to go unaligned on Intel and there > > are material benefits to justify the effort, that doesn't necessarily

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Stephen Frost
* Pavel Stehule (pavel.steh...@gmail.com) wrote: > 2014-09-11 16:42 GMT+02:00 Stephen Frost : > > I don't particularly like this (having these fields set in > > refresh_utf8format to hard-coded strings in the function), why not have > > those handled the same as the rest, where the strings themselv

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Stephen Frost
Andrew, * Andrew Dunstan (and...@dunslane.net) wrote: > On 09/11/2014 08:29 AM, Stephen Frost wrote: > >* Pavel Stehule (pavel.steh...@gmail.com) wrote: > >>Can I help with something, it is there some open question? > >I had been hoping for a more definitive answer regarding this option for > >arr

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 1:17 PM, Andres Freund wrote: > On 2014-09-11 13:04:43 -0400, Robert Haas wrote: >> On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund >> wrote: >> > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: >> >> I advise supporting pglz only for the initial patch, and adding >> >>

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 06:58:06PM +0200, Andres Freund wrote: > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: > > I advise supporting pglz only for the initial patch, and adding > > support for the others later if it seems worthwhile. The approach > > seems to work well enough with pglz that i

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 07:17:42PM +0200, Andres Freund wrote: > On 2014-09-11 13:04:43 -0400, Robert Haas wrote: > > On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund > > wrote: > > > On 2014-09-11 12:55:21 -0400, Robert Haas wrote: > > >> I advise supporting pglz only for the initial patch, and a

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 7:34 AM, Amit Kapila wrote: > Don't we need some way to prohibit changing GUC by launching process, > once it has shared the existing GUC? Nope. I mean, eventually, for true parallelism ... we absolutely will need that. But pg_background itself doesn't need that; it's pe

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 02:54:36PM -0300, Arthur Silva wrote: > Indeed I don't know any other architectures that this would be at an > option. So if this ever moves forward it must be turned on at compile time > for x86-64 only. I wonder how the Mysql handle their rows even on those > architectures

Re: [HACKERS] about half processes are blocked by btree, btree is bottleneck?

2014-09-11 Thread Peter Geoghegan
On Wed, Sep 10, 2014 at 8:08 PM, Xiaoyulei wrote: > Sum:66 > #0 0x7f8273a77627 in semop () from /lib64/libc.so.6 > #1 0x0060cda7 in PGSemaphoreLock () > #2 0x006511a9 in LWLockAcquire () > #3 0x004987f7 in _bt_relandgetbuf () > #4 0x0049c116 in _bt_search (

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Fabien COELHO
However, that would not diminish nor change much the amount and kind of code necessary to add an operator or a function That's not really true. You can't really add abs(x) or hash(x) right now because the current code only supports this syntax: \set varname operand1 [ operator operand2 ] Th

Re: [HACKERS] RLS Design

2014-09-11 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Sat, Sep 6, 2014 at 2:54 AM, Brightwell, Adam > wrote: > > * Add ALTER TABLE { ENABLE | DISABLE } ROW LEVEL SECURITY - set flag > > on table to allow for a default-deny capability. If RLS is enabled on a > > table and has no policies, th

Re: [HACKERS] Commitfest status

2014-09-11 Thread Petr Jelinek
On 11/09/14 18:59, Tomas Vondra wrote: On 10.9.2014 22:39, Heikki Linnakangas wrote: The bad news is that the rest don't seem to moving graduating from the Needs Review state. ISTM that many patches (b) in 'waiting on author' are not really waiting, because the author already responded /

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Petr Jelinek
On 11/09/14 20:37, Robert Haas wrote: 1. This patch generates warning on windows 1>pg_background.obj : error LNK2001: unresolved external symbol StatementTimeout You need to add PGDLLIMPORT for StatementTimeout OK. I still think we should go back and PGDLLIMPORT-ize all the GUC variables. +

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Andres Freund
On 2014-09-11 21:27:15 +0200, Petr Jelinek wrote: > On 11/09/14 20:37, Robert Haas wrote: > >OK. I still think we should go back and PGDLLIMPORT-ize all the GUC > >variables. > > +1 Same here. I think Tom was the only one against it, but pretty much everyone else was for it. We should fix all t

Re: [HACKERS] Aussie timezone database changes incoming

2014-09-11 Thread Gavin Flower
On 12/09/14 01:57, Robert Haas wrote: On Thu, Sep 11, 2014 at 12:20 AM, Craig Ringer wrote: I shouldn't be surprised that Australia gets to change. While the cynic in me thinks this is the usual USA-is-the-center-of-the-universe-ism, in reality it makes sense given relative population and likel

Re: [HACKERS] [v9.5] Custom Plan API

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 11:24 AM, Kouhei Kaigai wrote: >> Don't the changes to src/backend/optimizer/plan/createplan.c belong in >> patch #2? >> > The borderline between #1 and #2 is little bit bogus. So, I moved most of > portion into #1, however, invocation of InitCustomScan (that is a callback

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-11 Thread Peter Geoghegan
On Wed, Sep 10, 2014 at 11:36 AM, Robert Haas wrote: > No, not really. All you have to do is right a little test program to > gather the information. I don't think a little test program is useful - IMV it's too much of a simplification to suppose that a strcoll() has a fixed cost, and a memcmp()

Re: [HACKERS] RLS Design

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost wrote: >> 2. Row level security policies can exist for a table with DISABLE ROW >> LEVEL SECURITY in effect, but they don't do anything until RLS is >> enabled. A possible advantage of this approach is that you could >> *temporarily* shut off RLS for

Re: [HACKERS] RLS Design

2014-09-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost wrote: > > The one thing I'm wondering about with this design is- what happens when > > a policy is initially added? Currently, we automatically turn on RLS > > for the table when that happens. I'm not

Re: [HACKERS] Stating the significance of Lehman & Yao in the nbtree README

2014-09-11 Thread Peter Geoghegan
On Tue, Sep 9, 2014 at 12:01 AM, Heikki Linnakangas wrote: >> + Lehman and Yao don't require read locks, but assume that in-memory >> + copies of tree pages are unshared. > This is the existing paragraph, just moved to different place in the README. That's right - it seemed to make just as much

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 4:13 PM, Peter Geoghegan wrote: > On Wed, Sep 10, 2014 at 11:36 AM, Robert Haas wrote: >> No, not really. All you have to do is right a little test program to >> gather the information. > > I don't think a little test program is useful - IMV it's too much of a > simplific

Re: [HACKERS] pg_upgrade and epoch

2014-09-11 Thread Bruce Momjian
On Wed, Sep 10, 2014 at 02:24:17AM +0100, Greg Stark wrote: > On Tue, Sep 9, 2014 at 4:05 PM, Bruce Momjian wrote: > >> > Yes, I did think about that, but it seems like a behavior change. > >> > However, it is tempting to avoid future bug reports about this. > >> > >> When this came up in March, T

Re: [HACKERS] pg_upgrade and epoch

2014-09-11 Thread Tom Lane
Bruce Momjian writes: > On Wed, Sep 10, 2014 at 02:24:17AM +0100, Greg Stark wrote: >> I think the reason nobody's responding is because nobody has anything >> significant to add. It's a behavior change from not-working to >> working. Why wouldn't it be backpatched? > OK, Greg seems to be passion

Re: [HACKERS] pg_upgrade and epoch

2014-09-11 Thread Andres Freund
On 2014-09-11 16:58:12 -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 10, 2014 at 02:24:17AM +0100, Greg Stark wrote: > >> I think the reason nobody's responding is because nobody has anything > >> significant to add. It's a behavior change from not-working to > >> working. Why wou

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-11 Thread Peter Geoghegan
On Thu, Sep 11, 2014 at 1:50 PM, Robert Haas wrote: > I think I said pretty clearly that it was. I agree that you did, but it wasn't clear exactly what factors you were asking me to simulate. It still isn't. Do you want me to compare the same string a million times in a loop, both with a strcoll(

  1   2   >