[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Pavel Stehule
Hi Stephen Can I help with something, it is there some open question? Regards Pavel 2014-09-08 6:27 GMT+02:00 Stephen Frost sfr...@snowman.net: * Pavel Stehule (pavel.steh...@gmail.com) wrote: ignore_nulls in array_to_json_pretty probably is not necessary. On second hand, the cost is

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Fabien COELHO
Hello Robert, I am not objecting to the functionality; I'm objecting to bolting on ad-hoc operators one at a time. I think an expression syntax would let us do this in a much more scalable way. If I had time, I'd go do that, but I don't. We could add abs(x) and hash(x) and it would all be

Re: [HACKERS] RLS Design

2014-09-11 Thread Erik Rijkers
On Wed, September 10, 2014 23:50, Stephen Frost wrote: [rls_9-10-2014.patch] I can't get this to apply; I attach the complaints of patch. Erik Rijkers -- git clone git://git.postgresql.org/git/postgresql.git master Cloning into 'master'... -- git branch * master patching file

Re: [HACKERS] about half processes are blocked by btree, btree is bottleneck?

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 06:08 AM, Xiaoyulei wrote: I use benchmarksql with more than 200 clients on pg 9.3.3. when the test is going on, I collect all the process stack. I found about 100 processes are blocked by btree insert. Another 100 are blocked by xloginsert. Does btree has bad performance in

Re: [HACKERS] Support for N synchronous standby servers

2014-09-11 Thread Amit Kapila
On Thu, Sep 11, 2014 at 9:10 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Sep 11, 2014 at 5:21 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 08/28/2014 10:10 AM, Michael Paquier wrote: + #synchronous_standby_num = -1 # number of standbys servers using sync rep

Re: [HACKERS] pg_xlogdump --stats

2014-09-11 Thread Abhijit Menon-Sen
At 2014-08-21 10:06:39 +0300, hlinnakan...@vmware.com wrote: Committed the patch to add INT64_MODIFIER, with minor fixes. Thank you. The new rm_identify method needs to be documented. […] Perhaps add comments to the RmgrData struct, explaining all of the methods. OK, I'll submit a patch to

Re: [HACKERS] pg_xlogdump --stats

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 11:43 AM, Abhijit Menon-Sen wrote: At 2014-08-21 10:06:39 +0300, hlinnakan...@vmware.com wrote: I think the names that rm_identify returns should match those that the rm_desc functions print. I had originally started off trying to keep the output in sync, but it doesn't work

Re: [HACKERS] inherit support for foreign tables

2014-09-11 Thread Etsuro Fujita
(2014/09/11 4:32), Heikki Linnakangas wrote: I had a cursory look at this patch and the discussions around this. Thank you! ISTM there are actually two new features in this: 1) allow CHECK constraints on foreign tables, and 2) support inheritance for foreign tables. How about splitting it

Re: [HACKERS] pg_xlogdump --stats

2014-09-11 Thread Andres Freund
On 2014-09-11 12:14:42 +0300, Heikki Linnakangas wrote: On 09/11/2014 11:43 AM, Abhijit Menon-Sen wrote: At 2014-08-21 10:06:39 +0300, hlinnakan...@vmware.com wrote: I think the names that rm_identify returns should match those that the rm_desc functions print. I had originally started off

Re: [HACKERS] pg_xlogdump --stats

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 12:22 PM, Andres Freund wrote: On 2014-09-11 12:14:42 +0300, Heikki Linnakangas wrote: On 09/11/2014 11:43 AM, Abhijit Menon-Sen wrote: At 2014-08-21 10:06:39 +0300, hlinnakan...@vmware.com wrote: I think the names that rm_identify returns should match those that the rm_desc

[HACKERS] Fix MSVC isnan warning from e80252d4

2014-09-11 Thread David Rowley
The attached small patch fixes the following warning: src\backend\utils\adt\arrayfuncs.c(5613): warning C4013: '_isnan' undefined; assuming extern returning int [D:\Postgres\a\postgres.vcxproj] The fix is pretty much just a copy and paste from costsize.c Regards David Rowley

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Heikki Linnakangas
On 09/10/2014 05:47 PM, Mitsumasa KONDO wrote: Hi, I find typo in your patch. Please confirm. @line 239 - agg-sum2_lag = 0; + agg-sum_lag = 0; Ah thanks, cood catch! And back patch is welcome for me. I've committed and backpatched this, as well as a patch to refactor the way the

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Heikki Linnakangas
On 08/30/2014 07:16 PM, Fabien COELHO wrote: + if (latency_limit) + printf(number of transactions above the %.1f ms latency limit: INT64_FORMAT \n, + latency_limit / 1000.0, latency_late); + Any reason not to report a percentage here? Yes: I

Re: [HACKERS] inherit support for foreign tables

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 12:22 PM, Etsuro Fujita wrote: (2014/09/11 4:32), Heikki Linnakangas wrote: I had a cursory look at this patch and the discussions around this. Thank you! ISTM there are actually two new features in this: 1) allow CHECK constraints on foreign tables, and 2) support inheritance

Re: [HACKERS] Fix MSVC isnan warning from e80252d4

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 12:52 PM, David Rowley wrote: The attached small patch fixes the following warning: src\backend\utils\adt\arrayfuncs.c(5613): warning C4013: '_isnan' undefined; assuming extern returning int [D:\Postgres\a\postgres.vcxproj] The fix is pretty much just a copy and paste from

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
Hi, On 2014-09-10 12:17:34 +0530, Amit Kapila wrote: include $(top_srcdir)/src/backend/common.mk diff --git a/src/backend/postmaster/bgreclaimer.c b/src/backend/postmaster/bgreclaimer.c new file mode 100644 index 000..3df2337 --- /dev/null +++ b/src/backend/postmaster/bgreclaimer.c

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-11 Thread David Rowley
On Thu, Aug 28, 2014 at 6:23 AM, Tom Lane t...@sss.pgh.pa.us wrote: If the majority of the added code is code that will be needed for less-bogus optimizations, it might be all right; but I'd kind of want to see the less-bogus optimizations working first. That seems fair. Likely there'd be

Re: [HACKERS] Support for N synchronous standby servers

2014-09-11 Thread Amit Kapila
On Thu, Aug 28, 2014 at 12:40 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Aug 27, 2014 at 2:46 PM, Rajeev rastogi rajeev.rast...@huawei.com wrote: I have done some more review, below are my comments: Thanks! 1. There are currently two loops on *num_sync, Can we simplify

Re: [HACKERS] RLS Design

2014-09-11 Thread Stephen Frost
Erik, * Erik Rijkers (e...@xs4all.nl) wrote: On Wed, September 10, 2014 23:50, Stephen Frost wrote: [rls_9-10-2014.patch] I can't get this to apply; I attach the complaints of patch. Thanks for taking a look at this! [...] patching file src/include/catalog/catversion.h Hunk #1 FAILED

Re: [HACKERS] inherit support for foreign tables

2014-09-11 Thread Etsuro Fujita
(2014/09/11 19:46), Heikki Linnakangas wrote: On 09/11/2014 12:22 PM, Etsuro Fujita wrote: (2014/09/11 4:32), Heikki Linnakangas wrote: I had a cursory look at this patch and the discussions around this. Thank you! ISTM there are actually two new features in this: 1) allow CHECK

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Amit Kapila
On Fri, Jul 25, 2014 at 11:41 PM, Robert Haas robertmh...@gmail.com wrote: Patch 4 adds infrastructure that allows one session to save all of its non-default GUC values and another session to reload those values. This was written by Amit Khandekar and Noah Misch. It allows pg_background to

Re: [HACKERS] inherit support for foreign tables

2014-09-11 Thread Heikki Linnakangas
On 09/11/2014 02:30 PM, Etsuro Fujita wrote: Actually, this patch allows the exact same thing to apply to foreign tables. My explanation was insufficient about that. Sorry for that. Great, that's what I thought. So, should I split the patch into the two? Yeah, please do. - Heikki --

Re: [HACKERS] [TODO] Process pg_hba.conf keywords as case-insensitive

2014-09-11 Thread Robert Haas
On Wed, Sep 10, 2014 at 4:54 AM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Finally I think that we need case-insensitive version of get_role_id and() get_database_id() to acoomplish this patch'es objective. (This runs full-scans on pg_database or pg_authid X() Any such thing is

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Stephen Frost
* Pavel Stehule (pavel.steh...@gmail.com) wrote: Can I help with something, it is there some open question? I had been hoping for a more definitive answer regarding this option for array_to_json, or even a comment about the change to row_to_json. Andrew- any thoughts on this? (that's what the

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
Hello Heikki, Now that I've finished the detour and committed and backpatched the changes to the way latency is calculated, we can get back to this patch. It needs to be rebased. Before rebasing, I think that there are a few small problems with the modification applied to switch from an

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Arthur Silva
I agree that there's no reason to fix an algorithm to it, unless maybe it's pglz. There's some initial talk about implementing pluggable compression algorithms for TOAST and I guess the same must be taken into consideration for the WAL. -- Arthur Silva On Thu, Sep 11, 2014 at 2:46 AM, Rahila

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
(3) I wish that the maximum implied multiplier could be explicitely documented in the source code. From pg_rand48 source code, I think that it is 33.27106466687737 Small possibly buggy code attached, to show how I computed the above figure. -- Fabien.#include math.h #include stdio.h

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 09:37:07AM -0300, Arthur Silva wrote: I agree that there's no reason to fix an algorithm to it, unless maybe it's pglz. There's some initial talk about implementing pluggable compression algorithms for TOAST and I guess the same must be taken into consideration for the

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Robert Haas
Thanks for reviewing, Andres. On Thu, Sep 11, 2014 at 7:01 AM, Andres Freund and...@2ndquadrant.com wrote: +static void bgreclaim_quickdie(SIGNAL_ARGS); +static void BgreclaimSigHupHandler(SIGNAL_ARGS); +static void ReqShutdownHandler(SIGNAL_ARGS); +static void

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Amit Kapila
On Thu, Sep 11, 2014 at 6:32 PM, Robert Haas robertmh...@gmail.com wrote: Thanks for reviewing, Andres. On Thu, Sep 11, 2014 at 7:01 AM, Andres Freund and...@2ndquadrant.com wrote: +static void bgreclaim_quickdie(SIGNAL_ARGS); +static void BgreclaimSigHupHandler(SIGNAL_ARGS); +static

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
On 2014-09-11 09:02:34 -0400, Robert Haas wrote: Thanks for reviewing, Andres. On Thu, Sep 11, 2014 at 7:01 AM, Andres Freund and...@2ndquadrant.com wrote: +static void bgreclaim_quickdie(SIGNAL_ARGS); +static void BgreclaimSigHupHandler(SIGNAL_ARGS); +static void

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
Hello Heikki Now that I've finished the detour and committed and backpatched the changes to the way latency is calculated, we can get back to this patch. It needs to be rebased. Here is the rebase, which seems ok. See also the small issues raised aboud getPoissonRand in another email. --

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
We really need a more centralized way to handle error cleanup in auxiliary processes. The current state of affairs is really pretty helter-skelter. But for this patch, I think we should aim to mimic the existing style, as ugly as it is. I'm not sure whether Amit's got the logic

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Robert Haas
On Wed, Sep 10, 2014 at 5:09 PM, Tomas Vondra t...@fuzzy.cz wrote: OK. So here's v13 of the patch, reflecting this change. With the exception of ExecChooseHashTableSize() and a lot of stylistic issues along the lines of what I've already complained about, this patch seems pretty good to me. It

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Arthur Silva
On Wed, Sep 10, 2014 at 12:43 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Sep 9, 2014 at 10:08 AM, Arthur Silva arthur...@gmail.com wrote: I'm continuously studying Postgres codebase. Hopefully I'll be able to make some contributions in the future. For now I'm intrigued about

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Mitsumasa KONDO
2014-09-11 22:01 GMT+09:00 k...@rice.edu k...@rice.edu: On Thu, Sep 11, 2014 at 09:37:07AM -0300, Arthur Silva wrote: I agree that there's no reason to fix an algorithm to it, unless maybe it's pglz. Yes, it seems difficult to judge only the algorithm performance. We have to start to

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Amit Kapila
On Thu, Sep 11, 2014 at 6:59 PM, Andres Freund and...@2ndquadrant.com wrote: We really need a more centralized way to handle error cleanup in auxiliary processes. The current state of affairs is really pretty helter-skelter. But for this patch, I think we should aim to mimic the

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 9:22 AM, Andres Freund and...@2ndquadrant.com wrote: It's exactly the same as what bgwriter.c does. So what? There's no code in common, so I see no reason to have one signal handler using underscores and the next one camelcase names. /me shrugs. It's not always

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 9:32 AM, Arthur Silva arthur...@gmail.com wrote: I thought all memory alignment was (or at least the bulk of it) handled using some codebase wide macros/settings, otherwise how could different parts of the code inter-op? Poking this area might suffice for some initial

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Mitsumasa KONDO
2014-09-11 15:47 GMT+09:00 Fabien COELHO coe...@cri.ensmp.fr: Hello Robert, I am not objecting to the functionality; I'm objecting to bolting on ad-hoc operators one at a time. I think an expression syntax would let us do this in a much more scalable way. If I had time, I'd go do that,

Re: [HACKERS] Aussie timezone database changes incoming

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 12:20 AM, Craig Ringer cr...@2ndquadrant.com wrote: I shouldn't be surprised that Australia gets to change. While the cynic in me thinks this is the usual USA-is-the-center-of-the-universe-ism, in reality it makes sense given relative population and likely impact. Just

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: With the exception of ExecChooseHashTableSize() and a lot of stylistic issues along the lines of what I've already complained about, this patch seems pretty good to me. It does three things: ... (3) It allows the number of batches to increase on the

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Andres Freund
On 2014-09-11 09:48:10 -0400, Robert Haas wrote: On Thu, Sep 11, 2014 at 9:22 AM, Andres Freund and...@2ndquadrant.com wrote: I wonder if we should recheck the number of freelist items before sleeping. As the latch currently is reset before sleeping (IIRC) we might miss being woken up soon.

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: With the exception of ExecChooseHashTableSize() and a lot of stylistic issues along the lines of what I've already complained about, this patch seems pretty good to me. It does three

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: (3) It allows the number of batches to increase on the fly while the hash join is in process. Pardon me for not having read the patch yet,

Re: [HACKERS] Scaling shared buffer eviction

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 10:03 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-11 09:48:10 -0400, Robert Haas wrote: On Thu, Sep 11, 2014 at 9:22 AM, Andres Freund and...@2ndquadrant.com wrote: I wonder if we should recheck the number of freelist items before sleeping. As the

Re: [HACKERS] pgbench throttling latency limit

2014-09-11 Thread Fabien COELHO
How should skipped transactions should be taken into account in the log file output, with and without aggregation? I assume we'll want to have some trace of skipped transactions in the logs. The problem with this point is that how to report something not done is unclear, especially as the

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Andres Freund
On 2014-09-11 10:32:24 -0300, Arthur Silva wrote: Unaligned memory access received a lot attention in Intel post-Nehalen era. So it may very well pay off on Intel servers. You might find this blog post and it's comments/external-links interesting

[HACKERS] gist vacuum seaq access

2014-09-11 Thread Костя Кузнецов
After discussion of gist seaq access in vaccum there are 2 issue: Heikki says :Vacuum needs to memorize the current NSN when it begins1) how i may getting corect NSN. Also i must setting F_DELETED flag on empty page and to clean parent from link on deleted_pages2) how i may getting parent of page

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11 Září 2014, 15:31, Robert Haas wrote: On Wed, Sep 10, 2014 at 5:09 PM, Tomas Vondra t...@fuzzy.cz wrote: OK. So here's v13 of the patch, reflecting this change. With the exception of ExecChooseHashTableSize() and a lot of stylistic issues along the lines of what I've already complained

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Stephen Frost
Pavel, * Pavel Stehule (pavel.steh...@gmail.com) wrote: I removed dynamic allocation and reduced patch size. This is certainly better, imv, though there are a couple of minor issues (extra semi-colons, extraneous whitespace, get_line_style was still changed to non-const, even though it doesn't

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11 Září 2014, 16:11, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: (3) It allows the number of batches to increase on the fly while the hash join is in process.

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-09-11 Thread Petr Jelinek
On 10/09/14 13:13, Fujii Masao wrote: On Wed, Sep 10, 2014 at 1:45 AM, Petr Jelinek p...@2ndquadrant.com wrote: Hi, I recently wanted several times to have slave server prepared at certain point in time to reduce the time it takes for it to replay remaining WALs (say I have pg_basebackup -x on

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Merlin Moncure
On Thu, Sep 11, 2014 at 8:32 AM, Arthur Silva arthur...@gmail.com wrote: On Wed, Sep 10, 2014 at 12:43 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Sep 9, 2014 at 10:08 AM, Arthur Silva arthur...@gmail.com wrote: I'm continuously studying Postgres codebase. Hopefully I'll be able to

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 10:11 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Sep 11, 2014 at 9:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: (3) It allows the number of batches to increase on the fly while the

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes: On 11 Září 2014, 16:11, Tom Lane wrote: Ah. Well, that would mean that we need a heuristic for deciding when to increase the number of buckets versus the number of batches ... seems like a difficult decision. That's true, but that's not the aim of this

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-09-11 Thread Stephen Frost
* Albe Laurenz (laurenz.a...@wien.gv.at) wrote: Etsuro Fujita wrote: I agree with you on that point. So, I've updated the patch to have the explicit flag, as you proposed. Attached is the updated version of the patch. In this version, I've also revised code and its comments a bit.

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 7:14 AM, David Rowley dgrowle...@gmail.com wrote: Here's a quick demo, of the patch at work: test=# create table c (id int primary key); CREATE TABLE test=# create table b (id int primary key, c_id int not null references c(id)); CREATE TABLE test=# create table a

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: Be advised of the difficulties you are going to face here. Assuming for a second there is no reason not to go unaligned on Intel and there are material benefits to justify the effort, that doesn't necessarily hold for other platforms like arm/power.

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Petr Jelinek
On 10/09/14 22:53, Robert Haas wrote: Here's what the other approach looks like. I can't really see doing this way and then only providing hooks for those two functions, so this is with hooks for all the send-side stuff. Original version: 9 files changed, 295 insertions(+), 3 deletions(-) This

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Andres Freund
On 2014-09-11 11:39:12 -0400, Tom Lane wrote: Even on Intel, I'd wonder what unaligned accesses do to atomicity guarantees and suchlike. They pretty much kill atomicity guarantees. Atomicity is guaranteed while you're inside a cacheline, but not once you span them. This is not a big deal for

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Andrew Dunstan
On 09/11/2014 08:29 AM, Stephen Frost wrote: * Pavel Stehule (pavel.steh...@gmail.com) wrote: Can I help with something, it is there some open question? I had been hoping for a more definitive answer regarding this option for array_to_json, or even a comment about the change to row_to_json.

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Sep 11, 2014 at 7:14 AM, David Rowley dgrowle...@gmail.com wrote: 5. I've added a flag to pg_class called relhasfkey. Currently this gets set to true when a foreign key is added, though I've added nothing to set it back to false again. I notice

Re: [HACKERS] Support for N synchronous standby servers

2014-09-11 Thread Robert Haas
On Wed, Sep 10, 2014 at 11:40 PM, Michael Paquier michael.paqu...@gmail.com wrote: Currently two nodes can only have the same priority if they have the same application_name, so we could for example add a new connstring parameter called, let's say application_group, to define groups of nodes

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Pavel Stehule
Hi 2014-09-11 16:42 GMT+02:00 Stephen Frost sfr...@snowman.net: Pavel, * Pavel Stehule (pavel.steh...@gmail.com) wrote: I removed dynamic allocation and reduced patch size. This is certainly better, imv, though there are a couple of minor issues (extra semi-colons, extraneous

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-09-11 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: I have to admit that, while I applaud the effort made to have this change only be to postgres_fdw, I'm not sure that having the update/delete happening during the Scan phase and then essentially no-op'ing the ExecForeignUpdate/ExecForeignDelete is

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-09-11 Thread Tomas Vondra
On 11 Září 2014, 17:28, Tom Lane wrote: Tomas Vondra t...@fuzzy.cz writes: On 11 Z?? 2014, 16:11, Tom Lane wrote: Ah. Well, that would mean that we need a heuristic for deciding when to increase the number of buckets versus the number of batches ... seems like a difficult decision.

Re: [HACKERS] RLS Design

2014-09-11 Thread Robert Haas
On Sat, Sep 6, 2014 at 2:54 AM, Brightwell, Adam adam.brightw...@crunchydatasolutions.com wrote: * Add ALTER TABLE name { ENABLE | DISABLE } ROW LEVEL SECURITY - set flag on table to allow for a default-deny capability. If RLS is enabled on a table and has no policies, then a default-deny

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 1:46 AM, Rahila Syed rahilasyed...@gmail.com wrote: I will repeat the above tests with high load on CPU and using the benchmark given by Fujii-san and post the results. Average % of CPU usage at user level for each of the compression algorithm are as follows.

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Andres Freund
On 2014-09-11 12:55:21 -0400, Robert Haas wrote: I advise supporting pglz only for the initial patch, and adding support for the others later if it seems worthwhile. The approach seems to work well enough with pglz that it's worth doing even if we never add the other algorithms. That

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Bruce Momjian
On Thu, Sep 11, 2014 at 12:55:21PM -0400, Robert Haas wrote: I advise supporting pglz only for the initial patch, and adding support for the others later if it seems worthwhile. The approach seems to work well enough with pglz that it's worth doing even if we never add the other algorithms.

Re: [HACKERS] Commitfest status

2014-09-11 Thread Tomas Vondra
On 10.9.2014 22:39, Heikki Linnakangas wrote: The bad news is that the rest don't seem to moving graduating from the Needs Review state. ISTM that many patches (a) in 'needs review' actually have a review, or are being thoroughly discussed (b) in 'waiting on author' are not really

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-11 12:55:21 -0400, Robert Haas wrote: I advise supporting pglz only for the initial patch, and adding support for the others later if it seems worthwhile. The approach seems to work well enough with pglz

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 2:47 AM, Fabien COELHO coe...@cri.ensmp.fr wrote: Ok. I do agree that an expression syntax would be great! However, that would not diminish nor change much the amount and kind of code necessary to add an operator or a function That's not really true. You can't really

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Arthur Silva
On Thu, Sep 11, 2014 at 11:27 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-11 10:32:24 -0300, Arthur Silva wrote: Unaligned memory access received a lot attention in Intel post-Nehalen era. So it may very well pay off on Intel servers. You might find this blog post and it's

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Andres Freund
On 2014-09-11 13:04:43 -0400, Robert Haas wrote: On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-11 12:55:21 -0400, Robert Haas wrote: I advise supporting pglz only for the initial patch, and adding support for the others later if it seems

Re: [HACKERS] implement subject alternative names support for SSL connections

2014-09-11 Thread Alexey Klyukin
On Mon, Sep 8, 2014 at 8:04 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 09/05/2014 07:30 PM, Alexey Klyukin wrote: The error does not state whether the names comes from the CN or from the SAN section. I'd reword that slightly, to: psql: server certificate for example.com (and

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread Arthur Silva
On Thu, Sep 11, 2014 at 12:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: Be advised of the difficulties you are going to face here. Assuming for a second there is no reason not to go unaligned on Intel and there are material benefits to justify the

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-09-11 Thread Stephen Frost
* Pavel Stehule (pavel.steh...@gmail.com) wrote: 2014-09-11 16:42 GMT+02:00 Stephen Frost sfr...@snowman.net: I don't particularly like this (having these fields set in refresh_utf8format to hard-coded strings in the function), why not have those handled the same as the rest, where the

[HACKERS] Re: proposal: ignore null fields in not relation type composite type based constructors

2014-09-11 Thread Stephen Frost
Andrew, * Andrew Dunstan (and...@dunslane.net) wrote: On 09/11/2014 08:29 AM, Stephen Frost wrote: * Pavel Stehule (pavel.steh...@gmail.com) wrote: Can I help with something, it is there some open question? I had been hoping for a more definitive answer regarding this option for

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 1:17 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-11 13:04:43 -0400, Robert Haas wrote: On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-11 12:55:21 -0400, Robert Haas wrote: I advise supporting pglz only for the

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 06:58:06PM +0200, Andres Freund wrote: On 2014-09-11 12:55:21 -0400, Robert Haas wrote: I advise supporting pglz only for the initial patch, and adding support for the others later if it seems worthwhile. The approach seems to work well enough with pglz that it's

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 07:17:42PM +0200, Andres Freund wrote: On 2014-09-11 13:04:43 -0400, Robert Haas wrote: On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-11 12:55:21 -0400, Robert Haas wrote: I advise supporting pglz only for the initial

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 7:34 AM, Amit Kapila amit.kapil...@gmail.com wrote: Don't we need some way to prohibit changing GUC by launching process, once it has shared the existing GUC? Nope. I mean, eventually, for true parallelism ... we absolutely will need that. But pg_background itself

Re: [HACKERS] Memory Alignment in Postgres

2014-09-11 Thread k...@rice.edu
On Thu, Sep 11, 2014 at 02:54:36PM -0300, Arthur Silva wrote: Indeed I don't know any other architectures that this would be at an option. So if this ever moves forward it must be turned on at compile time for x86-64 only. I wonder how the Mysql handle their rows even on those architectures as

Re: [HACKERS] about half processes are blocked by btree, btree is bottleneck?

2014-09-11 Thread Peter Geoghegan
On Wed, Sep 10, 2014 at 8:08 PM, Xiaoyulei xiaoyu...@huawei.com wrote: Sum:66 #0 0x7f8273a77627 in semop () from /lib64/libc.so.6 #1 0x0060cda7 in PGSemaphoreLock () #2 0x006511a9 in LWLockAcquire () #3 0x004987f7 in _bt_relandgetbuf () #4 0x0049c116

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-11 Thread Fabien COELHO
However, that would not diminish nor change much the amount and kind of code necessary to add an operator or a function That's not really true. You can't really add abs(x) or hash(x) right now because the current code only supports this syntax: \set varname operand1 [ operator operand2 ]

Re: [HACKERS] RLS Design

2014-09-11 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: On Sat, Sep 6, 2014 at 2:54 AM, Brightwell, Adam adam.brightw...@crunchydatasolutions.com wrote: * Add ALTER TABLE name { ENABLE | DISABLE } ROW LEVEL SECURITY - set flag on table to allow for a default-deny capability. If RLS is enabled

Re: [HACKERS] Commitfest status

2014-09-11 Thread Petr Jelinek
On 11/09/14 18:59, Tomas Vondra wrote: On 10.9.2014 22:39, Heikki Linnakangas wrote: The bad news is that the rest don't seem to moving graduating from the Needs Review state. ISTM that many patches (b) in 'waiting on author' are not really waiting, because the author already responded

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Petr Jelinek
On 11/09/14 20:37, Robert Haas wrote: 1. This patch generates warning on windows 1pg_background.obj : error LNK2001: unresolved external symbol StatementTimeout You need to add PGDLLIMPORT for StatementTimeout OK. I still think we should go back and PGDLLIMPORT-ize all the GUC variables.

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-09-11 Thread Andres Freund
On 2014-09-11 21:27:15 +0200, Petr Jelinek wrote: On 11/09/14 20:37, Robert Haas wrote: OK. I still think we should go back and PGDLLIMPORT-ize all the GUC variables. +1 Same here. I think Tom was the only one against it, but pretty much everyone else was for it. We should fix all the

Re: [HACKERS] Aussie timezone database changes incoming

2014-09-11 Thread Gavin Flower
On 12/09/14 01:57, Robert Haas wrote: On Thu, Sep 11, 2014 at 12:20 AM, Craig Ringer cr...@2ndquadrant.com wrote: I shouldn't be surprised that Australia gets to change. While the cynic in me thinks this is the usual USA-is-the-center-of-the-universe-ism, in reality it makes sense given

Re: [HACKERS] [v9.5] Custom Plan API

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 11:24 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Don't the changes to src/backend/optimizer/plan/createplan.c belong in patch #2? The borderline between #1 and #2 is little bit bogus. So, I moved most of portion into #1, however, invocation of InitCustomScan (that

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-11 Thread Peter Geoghegan
On Wed, Sep 10, 2014 at 11:36 AM, Robert Haas robertmh...@gmail.com wrote: No, not really. All you have to do is right a little test program to gather the information. I don't think a little test program is useful - IMV it's too much of a simplification to suppose that a strcoll() has a fixed

Re: [HACKERS] RLS Design

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost sfr...@snowman.net wrote: 2. Row level security policies can exist for a table with DISABLE ROW LEVEL SECURITY in effect, but they don't do anything until RLS is enabled. A possible advantage of this approach is that you could *temporarily* shut

Re: [HACKERS] RLS Design

2014-09-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost sfr...@snowman.net wrote: The one thing I'm wondering about with this design is- what happens when a policy is initially added? Currently, we automatically turn on RLS for the table when that

Re: [HACKERS] Stating the significance of Lehman Yao in the nbtree README

2014-09-11 Thread Peter Geoghegan
On Tue, Sep 9, 2014 at 12:01 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: + Lehman and Yao don't require read locks, but assume that in-memory + copies of tree pages are unshared. This is the existing paragraph, just moved to different place in the README. That's right - it seemed to

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-11 Thread Robert Haas
On Thu, Sep 11, 2014 at 4:13 PM, Peter Geoghegan p...@heroku.com wrote: On Wed, Sep 10, 2014 at 11:36 AM, Robert Haas robertmh...@gmail.com wrote: No, not really. All you have to do is right a little test program to gather the information. I don't think a little test program is useful - IMV

Re: [HACKERS] pg_upgrade and epoch

2014-09-11 Thread Bruce Momjian
On Wed, Sep 10, 2014 at 02:24:17AM +0100, Greg Stark wrote: On Tue, Sep 9, 2014 at 4:05 PM, Bruce Momjian br...@momjian.us wrote: Yes, I did think about that, but it seems like a behavior change. However, it is tempting to avoid future bug reports about this. When this came up in March,

Re: [HACKERS] pg_upgrade and epoch

2014-09-11 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: On Wed, Sep 10, 2014 at 02:24:17AM +0100, Greg Stark wrote: I think the reason nobody's responding is because nobody has anything significant to add. It's a behavior change from not-working to working. Why wouldn't it be backpatched? OK, Greg seems to

  1   2   >