Re: [HACKERS] Asynchronous execution on FDW

2015-07-10 Thread Kyotaro HORIGUCHI
Hi, Currently there's no means to observe what it is doing from outside, so the additional sixth patch is to output debug messages about asynchronous execution. The sixth patch did not contain one message shown in the example. Attached is the revised version. Other patches are not changed.

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-10 Thread Sawada Masahiko
On Fri, Jul 10, 2015 at 3:42 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, Jul 8, 2015 at 10:10 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Thu, Jul 9, 2015 at 4:31 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Fri, Jul 3, 2015 at 1:25 AM, Sawada Masahiko sawada.m...@gmail.com

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-10 Thread Heikki Linnakangas
On 07/10/2015 02:06 AM, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2015-07-06 11:49:54 -0400, Tom Lane wrote: Rather than reverting cab9a0656c36739f, which would re-introduce a different performance problem, perhaps we could have COPY create a new relfilenode when it does

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-10 Thread Andres Freund
On 2015-07-09 19:06:11 -0400, Tom Lane wrote: What evidence have you got to base that value judgement on? cab9a0656c36739f was based on an actual user complaint, so we have good evidence that there are people out there who care about the cost of truncating a table many times in one

Re: [HACKERS] configure can't detect proper pthread flags

2015-07-10 Thread Heikki Linnakangas
On 07/08/2015 08:50 PM, Tom Lane wrote: Heikki Linnakangas hlinn...@iki.fi writes: The only scenario where you might now get warnings if we switch to upstream version, and didn't before, is if one of the flags makes pthreads to work, but also creates compiler warnings, while another flag later

[HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2015-07-10 Thread Etsuro Fujita
To save cycles, I modified create_foreignscan_plan so that it detects whether any system columns are requested if scanning a base relation. Also, I revised other code there a little bit. For ExecInitForeignScan, I simplified the code there to determine the scan tuple type, whith seems to me

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-10 Thread Simon Riggs
On 10 July 2015 at 09:49, Sawada Masahiko sawada.m...@gmail.com wrote: It is a minor thing, but if there is no other use for this fourth bit-space, it seems a shame to waste it when there is some use for it. I haven't looked at the code around this area to know how hard it would be to

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-10 Thread Andres Freund
On 2015-07-10 19:23:28 +0900, Fujii Masao wrote: Maybe I'm missing something. But I start wondering why TRUNCATE and INSERT (or even all the operations on the table created at the current transaction) need to be WAL-logged while COPY can be optimized. If no WAL records are generated on that

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-10 Thread Beena Emerson
Hello, Tue, Jul 7, 2015 at 02:56 AM, Josh Berkus wrote: pro-JSON: * standard syntax which is recognizable to sysadmins and devops. * can use JSON/JSONB functions with ALTER SYSTEM SET to easily make additions/deletions from the synch rep config. * can add group labels (see below)

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Michael Paquier
On Fri, Jul 10, 2015 at 7:13 PM, Alexander Korotkov aekorot...@gmail.com wrote: On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jul 9, 2015 at 10:33 PM, Alexander Korotkov wrote: [...] + /* Caclculate max data size on page according to

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-10 Thread Andres Freund
On 2015-07-10 13:38:50 +0300, Heikki Linnakangas wrote: In the long-term, I'd like to refactor this whole thing so that we never WAL-log any operations on a relation that's created in the same transaction (when wal_level=minimal). Instead, at COMMIT, we'd fsync() the relation, or if it's

Re: [HACKERS] Re: Removing SSL renegotiation (Was: Should we back-patch SSL renegotiation fixes?)

2015-07-10 Thread Andres Freund
On 2015-07-01 23:32:23 -0400, Noah Misch wrote: We'd need to be triply confident that we know better than the DBA before removing flexibility in back branches. +1 for just changing the default. I think we do. But I also think that I pretty clearly lost this argument, so let's just change the

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Heikki Linnakangas
On 07/10/2015 01:13 PM, Alexander Korotkov wrote: On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier michael.paqu...@gmail.com wrote: + #define GIN_MIN_FILLFACTOR20 + #define GIN_DEFAULT_FILLFACTOR90 I am still worrying about using a default fillfactor at 90, as we did a lot

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2015-07-10 Thread David Rowley
On 10 July 2015 at 21:40, Etsuro Fujita fujita.ets...@lab.ntt.co.jp wrote: To save cycles, I modified create_foreignscan_plan so that it detects whether any system columns are requested if scanning a base relation. Also, I revised other code there a little bit. For ExecInitForeignScan, I

Re: [HACKERS] WIP: Enhanced ALTER OPERATOR

2015-07-10 Thread Heikki Linnakangas
On 07/06/2015 07:21 PM, Uriy Zhuravlev wrote: Hello hackers. This is the fifth version of the patch (the fourth was unsuccessful :)). I added documentation and was held a small refactoring. I edited the formatting of the syntax page a bit, and came up with this: ALTER OPERATOR name ( {

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Alexander Korotkov
On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jul 9, 2015 at 10:33 PM, Alexander Korotkov wrote: [...] + /* Caclculate max data size on page according to fillfactor */ s/Caclculate/Calculate When creating a simple gin index, I am seeing

[HACKERS] security labels on databases are bad for dump restore

2015-07-10 Thread Andres Freund
Hi, pg_dump dumps security labels on databases. Which makes sense. The problem is that they're dumped including the database name. Which means that if you dump a database and restore it into a differently named one you'll either get a failure because the database does not exist, or worse you'll

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-10 Thread Pavel Stehule
forgotten attachment Regards Pavel 2015-07-10 14:34 GMT+02:00 Pavel Stehule pavel.steh...@gmail.com: Hi I am sending review of this patch: 1. I reread a previous discussion and almost all are for this patch (me too) 2. I have to fix a typo in hstore_io.c function (update attached),

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-10 Thread Fujii Masao
On Fri, Jul 10, 2015 at 2:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: Fujii Masao masao.fu...@gmail.com writes: On Tue, Jul 7, 2015 at 12:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: One idea I had was to allow the COPY optimization only if the heap file is physically zero-length at the time the

Re: [HACKERS] creating extension including dependencies

2015-07-10 Thread Heikki Linnakangas
On 07/09/2015 07:05 PM, Petr Jelinek wrote: On 2015-07-07 15:41, Andres Freund wrote: On 2015-07-07 22:36:29 +0900, Fujii Masao wrote: On Mon, Jun 15, 2015 at 7:50 AM, Petr Jelinek p...@2ndquadrant.com wrote: Hi, I am getting tired installing manually required extensions manually. I was

Re: [HACKERS] more RLS oversights

2015-07-10 Thread Joe Conway
On 07/03/2015 10:03 AM, Noah Misch wrote: (1) CreatePolicy() and AlterPolicy() omit to call assign_expr_collations() on the node trees. Test case: begin; set row_security = force; create table t (c) as values ('bar'::text); create policy p on t using (c ('foo'::text COLLATE C)); alter

Re: [HACKERS] pg_upgrade + Extensions

2015-07-10 Thread Andrew Dunstan
On Fri, Jul 10, 2015 at 5:05 PM, David E. Wheeler da...@justatheory.com wrote: On Jul 10, 2015, at 11:32 AM, Smitha Pamujula smitha.pamuj...@iovation.com wrote: Your installation references loadable libraries that are missing from the new installation. You can add these libraries to the

Re: [HACKERS] more RLS oversights

2015-07-10 Thread Noah Misch
On Fri, Jul 10, 2015 at 03:08:53PM -0700, Joe Conway wrote: On 07/03/2015 10:03 AM, Noah Misch wrote: (1) CreatePolicy() and AlterPolicy() omit to call assign_expr_collations() on the node trees. The attached fixes this issue for me, but I am unsure whether we really need/want the

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-10 Thread Jim Nasby
On 7/10/15 4:46 AM, Simon Riggs wrote: On 10 July 2015 at 09:49, Sawada Masahiko sawada.m...@gmail.com mailto:sawada.m...@gmail.com wrote: It is a minor thing, but if there is no other use for this fourth bit-space, it seems a shame to waste it when there is some use for it. I

Re: [HACKERS] polymorphic types - enforce casting to most common type automatically

2015-07-10 Thread Pavel Stehule
2015-07-10 23:19 GMT+02:00 Pavel Stehule pavel.steh...@gmail.com: Hi 2015-07-10 18:43 GMT+02:00 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: now a functions with more than one polymorphic arguments are relative fragile due missing casting to most common

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-10 Thread Pavel Stehule
2015-07-10 16:16 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de : On Fri, Jul 10, 2015 at 4:04 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2. why you did indirect call via JsonOutContext? What is benefit dst.value(dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid,

Re: [HACKERS] LANGUAGE sql functions don't use the custom plan logic

2015-07-10 Thread Tom Lane
Andres Freund and...@anarazel.de writes: I recently was forcfully reminded that sql functions don't use the plancache.c infrastructure. I was sort of aware of that, but I didn't think far enough ahead that that also implies we'll not use custom plans for statements with parameters when it'd be

Re: [HACKERS] LANGUAGE sql functions don't use the custom plan logic

2015-07-10 Thread Andres Freund
On 2015-07-10 09:52:30 -0400, Tom Lane wrote: There's a whole lot of ways in which the current implementation of SQL-language functions leaves things to be desired. Yea. What did you run into exactly? A generic plan was used on a partitioned table. Normally the function could be inlined so

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-10 Thread Sawada Masahiko
On Fri, Jul 10, 2015 at 10:43 PM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Jul 10, 2015 at 2:41 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Fri, Jul 10, 2015 at 3:05 AM, Sawada Masahiko sawada.m...@gmail.com wrote: Also something for pg_upgrade is also not yet. TODO -

Re: [HACKERS] creating extension including dependencies

2015-07-10 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On July 10, 2015 4:16:59 PM GMT+02:00, Tom Lane t...@sss.pgh.pa.us wrote: Would that propagate down through multiple levels of CASCADE? (Although I'm not sure it would be sensible for a non-relocatable extension to depend on a relocatable one, so maybe

[HACKERS] LANGUAGE sql functions don't use the custom plan logic

2015-07-10 Thread Andres Freund
Hi, I recently was forcfully reminded that sql functions don't use the plancache.c infrastructure. I was sort of aware of that, but I didn't think far enough ahead that that also implies we'll not use custom plans for statements with parameters when it'd be helpful. That's imo quite the trap.

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-10 Thread Pavel Stehule
Hi I am sending review of this patch: 1. I reread a previous discussion and almost all are for this patch (me too) 2. I have to fix a typo in hstore_io.c function (update attached), other (patching, regress tests) without problems My objections: 1. comments - missing comment for some basic

Re: ctidscan as an example of custom-scan (Re: [HACKERS] [v9.5] Custom Plan API)

2015-07-10 Thread Heikki Linnakangas
On 07/02/2015 08:21 AM, Kouhei Kaigai wrote: Folks, Moved this patch to next CF 2015-02 because of lack of review(ers). Do we still need this patch as contrib module? It was originally required it as example of custom-scan interface last summer, however, here was no strong requirement

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-10 Thread Fujii Masao
On Fri, Jul 10, 2015 at 2:41 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Fri, Jul 10, 2015 at 3:05 AM, Sawada Masahiko sawada.m...@gmail.com wrote: Also something for pg_upgrade is also not yet. TODO - Test case for this feature - pg_upgrade support. I had forgotten to change

Re: [HACKERS] creating extension including dependencies

2015-07-10 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: On Fri, Jul 10, 2015 at 10:09 PM, Heikki Linnakangas hlinn...@iki.fi This seems quite reasonable, but I have to ask: How many extensions are there out there that depend on another extension? Off the top of my head, I can't think of any.. With

Re: [HACKERS] creating extension including dependencies

2015-07-10 Thread Andres Freund
On July 10, 2015 4:16:59 PM GMT+02:00, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@anarazel.de writes: I think we should copy the SCHEMA option here and document that we use the same schema. But it needs to be done in a way that doesn't error out if the extension is not

Re: [HACKERS] creating extension including dependencies

2015-07-10 Thread Andres Freund
On 2015-06-15 00:50:08 +0200, Petr Jelinek wrote: + /* Create and execute new CREATE EXTENSION statement. */ + ces = makeNode(CreateExtensionStmt); + ces-extname = curreq; +

Re: [HACKERS] creating extension including dependencies

2015-07-10 Thread Vladimir Borodin
10 июля 2015 г., в 16:09, Heikki Linnakangas hlinn...@iki.fi написал(а): On 07/09/2015 07:05 PM, Petr Jelinek wrote: On 2015-07-07 15:41, Andres Freund wrote: On 2015-07-07 22:36:29 +0900, Fujii Masao wrote: On Mon, Jun 15, 2015 at 7:50 AM, Petr Jelinek p...@2ndquadrant.com wrote: Hi,

Re: [HACKERS] WIP: Enhanced ALTER OPERATOR

2015-07-10 Thread Uriy Zhuravlev
Hello. On Friday 10 July 2015 15:46:35 you wrote: * I think it'd be better to use NONE instead of NULL above, to remove the estimator. It seems inconsistent when we've used NONE to mean missing earlier in the same statement. Ok, you right. * The way you check for the NULL in OperatorUpd is

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-10 Thread Shulgin, Oleksandr
2015-07-10 14:34 GMT+02:00 Pavel Stehule pavel.steh...@gmail.com: Hi I am sending review of this patch: 1. I reread a previous discussion and almost all are for this patch (me too) 2. I have to fix a typo in hstore_io.c function (update attached), other (patching, regress tests)

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-10 Thread Pavel Stehule
2015-07-10 15:57 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de : 2015-07-10 14:34 GMT+02:00 Pavel Stehule pavel.steh...@gmail.com: Hi I am sending review of this patch: 1. I reread a previous discussion and almost all are for this patch (me too) 2. I have to fix a typo in

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-10 Thread Shulgin, Oleksandr
On Fri, Jul 10, 2015 at 4:04 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2. why you did indirect call via JsonOutContext? What is benefit dst.value(dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid, false); instead json_out_value(dst, ) For consistency. Even though we

Re: [HACKERS] creating extension including dependencies

2015-07-10 Thread Tom Lane
Andres Freund and...@anarazel.de writes: I think we should copy the SCHEMA option here and document that we use the same schema. But it needs to be done in a way that doesn't error out if the extension is not relocatable... Would that propagate down through multiple levels of CASCADE?

[HACKERS] pg_upgrade + Ubuntu

2015-07-10 Thread Joshua D. Drake
Hackers, Simple problem (I think): 9.4 version of pg_upgrade said: /usr/lib/postgresql/9.4/bin/pg_ctl -w -l pg_upgrade_server.log -D /var/lib/postgresql/9.4/main -o -p 9400 -b -c synchronous_commit=off -c fsync=off -c full_page_writes=off -c listen_addresses='' -c

Re: [HACKERS] pg_upgrade + Extensions

2015-07-10 Thread Tom Lane
David E. Wheeler da...@justatheory.com writes: My co-workers tell me that pg_upgrade told them to drop the colnames and hostname extensions before upgrading from 9.3 to 9.4. Really? I see nothing in the source code that would print any such advice. There *is* a check on whether .so libraries

Re: [HACKERS] pg_upgrade + Ubuntu

2015-07-10 Thread Mike Blackwell
Does pg_config show the correct location? If so, perhaps pg_upgrade could get the .conf location the same way rather than requiring a command line option. __ *Mike Blackwell | Technical Analyst, Distribution

Re: [HACKERS] pg_upgrade + Extensions

2015-07-10 Thread Smitha Pamujula
Tom, I just tested and yes that worked. Once we have the new library for the hostname, pg_upgrade is not complaining about the hostname extension. Another thing we found is this. We needed to drop json_build extension before the upgrade. However the upgrade fails with the following error and the

[HACKERS] pg_upgrade + Extensions

2015-07-10 Thread David E. Wheeler
Hackers, My co-workers tell me that pg_upgrade told them to drop the colnames and hostname extensions before upgrading from 9.3 to 9.4. Fortunately, Postgres had not recorded any dependencies on functions from these extensions (not sure why not, since we do user them, but for the moment

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-10 Thread Alexander Korotkov
On Fri, Jun 26, 2015 at 6:39 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 25, 2015 at 9:23 AM, Peter Eisentraut pete...@gmx.net wrote: On 6/22/15 1:37 PM, Robert Haas wrote: Currently, the only time we report a process as waiting is when it is waiting for a heavyweight lock.

Re: [HACKERS] polymorphic types - enforce casting to most common type automatically

2015-07-10 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: now a functions with more than one polymorphic arguments are relative fragile due missing casting to most common type. Some our functions like coalesce can do it, so it is surprising for our users. our custom polymorphic function foo(anyelement,

Re: [HACKERS] pg_upgrade + Ubuntu

2015-07-10 Thread Alvaro Herrera
Joshua D. Drake wrote: On 07/10/2015 11:01 AM, Mike Blackwell wrote: Does pg_config show the correct location? If so, perhaps pg_upgrade could get the .conf location the same way rather than requiring a command line option. Good idea but: postgres@ly19:~$ pg_config You need to

Re: [HACKERS] pg_upgrade + Ubuntu

2015-07-10 Thread Mike Blackwell
On Fri, Jul 10, 2015 at 2:10 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Joshua D. Drake wrote: On 07/10/2015 11:01 AM, Mike Blackwell wrote: Does pg_config show the correct location? Good idea but: postgres@ly19:~$ pg_config You need to install postgresql-server-dev-X.Y

Re: [HACKERS] pg_upgrade + Ubuntu

2015-07-10 Thread Joshua D. Drake
On 07/10/2015 12:10 PM, Alvaro Herrera wrote: It seems to me that this is a Debian packaging issue, not an upstream issue, isn't it? If you want to fix the problem in this way, then surely whatever package contains pg_upgrade should also contain pg_config. Why are you not using

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-07-10 Thread Tom Lane
Tomas Vondra tomas.von...@2ndquadrant.com writes: currently partial indexes end up not using index only scans in most cases, because check_index_only() is overly conservative, as explained in this comment: ... I've done a bunch of tests, and I do see small (hardly noticeable) increase in

Re: [HACKERS] polymorphic types - enforce casting to most common type automatically

2015-07-10 Thread Pavel Stehule
Hi 2015-07-10 18:43 GMT+02:00 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: now a functions with more than one polymorphic arguments are relative fragile due missing casting to most common type. Some our functions like coalesce can do it, so it is surprising

Re: [HACKERS] pg_upgrade + Extensions

2015-07-10 Thread David E. Wheeler
On Jul 10, 2015, at 11:32 AM, Smitha Pamujula smitha.pamuj...@iovation.com wrote: I just tested and yes that worked. Once we have the new library for the hostname, pg_upgrade is not complaining about the hostname extension. Great, thank you Smitha -- and Tom for the pointer. Your

Re: [HACKERS] Additional role attributes superuser review

2015-07-10 Thread Heikki Linnakangas
On 05/08/2015 07:35 AM, Stephen Frost wrote: Gavin, * Gavin Flower (gavinflo...@archidevsys.co.nz) wrote: What if I had a company with several subsidiaries using the same database, and want to prefix roles and other things with the subsidiary's initials? (I am not saying this would be a good

Re: [HACKERS] pg_upgrade + Ubuntu

2015-07-10 Thread Jim Nasby
On 7/10/15 2:20 PM, Mike Blackwell wrote: postgres@ly19:~$ pg_config You need to install postgresql-server-dev-X.Y for building a server-side extension or libpq-dev for building a client-side application. Which is worse having to install yet another package or having a

[HACKERS] Reusing abbreviated keys during second pass of ordered [set] aggregates

2015-07-10 Thread Peter Geoghegan
Currently, there are certain aggregates that sort some tuples before making a second pass over their memtuples array (through the tuplesort gettuple interface), calling one or more attribute equality operator functions as they go. For example, this occurs during the execution of the following