Re: [HACKERS] using arrays within structure in ECPG
On Wed, Apr 02, 2014 at 09:33:15AM +0530, Ashutosh Bapat wrote: So, you are saying that we should try to catch such errors and report during pre-compile time. That's better than silently corrupting the data. Well, I think this goes without saying. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: michael.meskes at gmail dot com VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Including replication slot data in base backups
--On 1. April 2014 11:26:08 -0400 Robert Haas robertmh...@gmail.com wrote: As a general comment, I think that replication slots, while a great feature, have more than the usual potential for self-inflicted injury. A replication slot prevents the global xmin from advancing (so your tables will bloat) and WAL from being removed (so your pg_xlog directory will fill up and take down the server). The very last thing you want to do is to keep around a replication slot that should have been dropped, and I suspect a decent number of users are going to make that mistake, just as they do with prepared transactions and backends left idle in transaction. Oh yes, i saw this happening uncountless times now by customers when restoring a basebackup with in-progress prepared xacts (and was indeed fooled myself a few times, too). I always was under the impression that there should be a big big warning at least in the logs to hint the user to check any remaining prepared xacts... -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Including replication slot data in base backups
On 2014-04-02 09:59:28 +0900, Michael Paquier wrote: On Tue, Apr 1, 2014 at 11:59 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-01 16:45:46 +0200, Magnus Hagander wrote: On Tue, Apr 1, 2014 at 2:24 PM, Michael Paquier michael.paqu...@gmail.comwrote: As of now, pg_basebackup creates an empty repository for pg_replslot/ in a base backup, forcing the user to recreate slots on other nodes of the cluster with pg_create_*_replication_slot, or copy pg_replslot from another node. This is not really user-friendly especially after a failover where a given slave may not have the replication slot information of the master that it is replacing. What exactly is your use case for copying the slots? I had in mind users that want to keep around base backups that could be used for recovery operations like PITR using a base backup and archives. It does not apply directly to a live standby, as it would mean that this standby would be defined to retain WAL for other slaves connected to the master. I honestly can't follow why that implies copying the slots? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GSoC 2014 proposal
On Tue, Apr 1, 2014 at 2:23 PM, Heikki Linnakangas hlinnakan...@vmware.comwrote: The BIRCH algorithm as described in the paper describes building a tree in memory. If I understood correctly, you're suggesting to use a pre-built GiST index instead. Interesting idea! There are a couple of signifcant differences between the CF tree described in the paper and GiST: 1. In GiST, a leaf item always represents one heap tuple. In the CF tree, a leaf item represents a cluster, which consists of one or more tuples. So the CF tree doesn't store an entry for every input tuple, which makes it possible to keep it in memory. 2. In the CF tree, all entries in a leaf node must satisfy a threshold requirement, with respect to a threshold value T: the diameter (or radius) has to be less than T. GiST imposes no such restrictions. An item can legally be placed anywhere in the tree; placing it badly will just lead to degraded search performance, but it's still a legal GiST tree. 3. A GiST index, like any other index in PostgreSQL, holds entries also for deleted tuples, until the index is vacuumed. So you cannot just use information from a non-leaf node and use it in the result, as the information summarized at a non-leaf level includes noise from the dead tuples. Can you elaborate how you are planning to use a GiST index to implement BIRCH? You might also want to take a look at SP-GiST; SP-GiST is more strict in where in the tree an item can be stored, and lets the operator class to specify exactly when a node is split etc. Hmmm, it's likely I've imagined something quite outside of this paper, and even already suggested it to Ivan... :) I need a little time to rethink it. -- With best regards, Alexander Korotkov.
Re: [HACKERS] Including replication slot data in base backups
On Wed, Apr 2, 2014 at 6:58 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-02 09:59:28 +0900, Michael Paquier wrote: On Tue, Apr 1, 2014 at 11:59 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-01 16:45:46 +0200, Magnus Hagander wrote: On Tue, Apr 1, 2014 at 2:24 PM, Michael Paquier michael.paqu...@gmail.comwrote: As of now, pg_basebackup creates an empty repository for pg_replslot/ in a base backup, forcing the user to recreate slots on other nodes of the cluster with pg_create_*_replication_slot, or copy pg_replslot from another node. This is not really user-friendly especially after a failover where a given slave may not have the replication slot information of the master that it is replacing. What exactly is your use case for copying the slots? I had in mind users that want to keep around base backups that could be used for recovery operations like PITR using a base backup and archives. It does not apply directly to a live standby, as it would mean that this standby would be defined to retain WAL for other slaves connected to the master. I honestly can't follow why that implies copying the slots? You simply do not need to recreate manually the slots on the new master. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] using arrays within structure in ECPG
I have one more doubt, regarding offsets. In ECPGdump_a_simple() we have code if (siz == NULL || strlen(siz) == 0 || strcmp(arrsize, 0) == 0 || strcmp(arrsize, 1) == 0) fprintf(o, \n\t%s,%s,(long)%s,(long)%s,%s, , get_type(type), variable, varcharsize, arrsize, offset); else fprintf(o, \n\t%s,%s,(long)%s,(long)%s,%s, , get_type(type), variable, varcharsize, arrsize, siz); If the caller has passed siz, it means that this variable is part of the a structure. Remember in dump_variables(), this function is called with struct_sizeof = NULL. So, once we know that siz != NULL and strlen(siz) != 0, it's evident that the simple variable we are dumping is part of a structure and hence we should be using siz instead of offset. Why then we still check arrsize? In a case, where we are dumping a pointer to a structure, this code dumps each member with offset = size of that member, thus again corrupting memory, if there are more than one rows being saved through pointer. On Wed, Apr 2, 2014 at 3:10 PM, Michael Meskes mes...@postgresql.orgwrote: On Wed, Apr 02, 2014 at 09:33:15AM +0530, Ashutosh Bapat wrote: So, you are saying that we should try to catch such errors and report during pre-compile time. That's better than silently corrupting the data. Well, I think this goes without saying. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: michael.meskes at gmail dot com VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company
[HACKERS] Fwd: SSL auth question
Hello, I set certificate auth on postgresql 9.3. I generate SSL certificate with my custom extension. So, OpenSSL read it, PostgreSQL accept it if this extension is not critical, but if I set this extension critical, PostgreSQL deny connection. How can I prevent it? Where PostgreSQL try to read SSL extension? Best regards, Dmitry Voronin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Including replication slot data in base backups
On 2014-04-02 20:59:03 +0900, Michael Paquier wrote: On Wed, Apr 2, 2014 at 6:58 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-02 09:59:28 +0900, Michael Paquier wrote: On Tue, Apr 1, 2014 at 11:59 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-01 16:45:46 +0200, Magnus Hagander wrote: On Tue, Apr 1, 2014 at 2:24 PM, Michael Paquier michael.paqu...@gmail.comwrote: As of now, pg_basebackup creates an empty repository for pg_replslot/ in a base backup, forcing the user to recreate slots on other nodes of the cluster with pg_create_*_replication_slot, or copy pg_replslot from another node. This is not really user-friendly especially after a failover where a given slave may not have the replication slot information of the master that it is replacing. What exactly is your use case for copying the slots? I had in mind users that want to keep around base backups that could be used for recovery operations like PITR using a base backup and archives. It does not apply directly to a live standby, as it would mean that this standby would be defined to retain WAL for other slaves connected to the master. I honestly can't follow why that implies copying the slots? You simply do not need to recreate manually the slots on the new master. That doesn't seem like a good justification. The new master won't necessarily have all the neccessary WAL available, no? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_fn_expr_variadic considered harmful
Robert Haas robertmh...@gmail.com writes: On Tue, Apr 1, 2014 at 2:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: There's also the point that even if we changed ruleutils' behavior now, this would not fix existing dump files that have considered the two forms interchangeable ever since VARIADIC existed. And we generally try hard to not break existing dump files. To be even more to the point: what you propose is incapable of fixing the precise problem stated in the bug report, because it's complaining about a dump taken from 9.1, and there is *no* way to make 9.1 produce a dump that only uses VARIADIC if the original call did. It hasn't got the information. Even using a newer version of pg_dump wouldn't help that. Well, that argues for the choice of trying to make them equivalent again, I suppose, but it sounds like there are some nasty edge cases that won't easily be filed down. I think your idea of redefining funcvariadic to be true only for VARIADIC ANY is probably a promising approach to that solution, but as you say it leaves some problems unsolved. I think what we'll have to do is tell complainants to recreate any affected indexes or rules after installing 9.3.5. Given the relatively small number of complaints, I don't think it's worth working harder, nor taking risks like inserting catalog lookups into readfuncs.c. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inheritance of foregn key constraints - dropping isinternal triggers by a
On Tuesday 01 of April 2014 11:06:00 you wrote: On Tue, Apr 1, 2014 at 9:13 AM, Andrzej Mazurkiewicz andr...@mazurkiewicz.org wrote: It seems that if the trigger is internal (tgisinternal = true) it is not visible to the DROP TRIGGER command. So it cannot be deleted using DROP TRIGGER command, although the dependency type is DEPENDENCY_AUTOMATIC Try using a quoted identifier: DROP TRIGGER RI_ConstraintTrigger_c_19322 ON master; Regards, -- Fabrízio de Royes Mello Consultoria/Coaching PostgreSQL Good Afternoon. Sorry for my mistake. I have forgotten about modifications of names to lower case. The owner of a table can drop trigger using the command DROP TRIGGER. However, where I can find information on philosophy of that approach. For me internal triggers are marked isinternal not to be accessed using SQL commands. Removing internals, f. e. FK triggers, by hand by a programmer and even by an administrator seems not to be a recommended practice. Kind Regards Andrzej Mazurkiewicz -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] using arrays within structure in ECPG
On Wed, Apr 02, 2014 at 05:49:03PM +0530, Ashutosh Bapat wrote: I have one more doubt, regarding offsets. ... This is actually a very good question. Parts of this code are older than my work on ecpg, meaning they were already in version 0.1. It could very well be that with some changes over the years this test isn't needed anymore. The regression suite works without just nicely. To be honest, the whole type.c code needs a real rewrite to be better legible. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: michael.meskes at gmail dot com VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] polymorphic SQL functions has a problem with domains
Hello I was informed about impossibility to use a polymorphic functions together with domain types see create domain xx as numeric(15); create or replace function g(anyelement, anyelement) returns anyelement as $$ select $1 + $2 $$ language sql immutable; postgres=# select g(1::xx, 2::xx); ERROR: return type mismatch in function declared to return xx DETAIL: Actual return type is numeric. CONTEXT: SQL function g during inlining is this bug? Regards Pavel Stehule
Re: [HACKERS] polymorphic SQL functions has a problem with domains
Pavel Stehule pavel.steh...@gmail.com writes: I was informed about impossibility to use a polymorphic functions together with domain types see create domain xx as numeric(15); create or replace function g(anyelement, anyelement) returns anyelement as $$ select $1 + $2 $$ language sql immutable; postgres=# select g(1::xx, 2::xx); ERROR: return type mismatch in function declared to return xx DETAIL: Actual return type is numeric. CONTEXT: SQL function g during inlining That example doesn't say you can't use polymorphic functions with domains. It says that this particular polymorphic function definition is wrong: it is not making sure its result is of the expected data type. I don't recall right now whether SQL functions will apply an implicit cast on the result for you, but even if they do, an upcast from numeric to some domain over numeric wouldn't be implicit. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] polymorphic SQL functions has a problem with domains
2014-04-02 17:19 GMT+02:00 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: I was informed about impossibility to use a polymorphic functions together with domain types see create domain xx as numeric(15); create or replace function g(anyelement, anyelement) returns anyelement as $$ select $1 + $2 $$ language sql immutable; postgres=# select g(1::xx, 2::xx); ERROR: return type mismatch in function declared to return xx DETAIL: Actual return type is numeric. CONTEXT: SQL function g during inlining That example doesn't say you can't use polymorphic functions with domains. It says that this particular polymorphic function definition is wrong: it is not making sure its result is of the expected data type. I don't recall right now whether SQL functions will apply an implicit cast on the result for you, but even if they do, an upcast from numeric to some domain over numeric wouldn't be implicit. But I am not able to enforce a casting in polymorphic function or there is some possibility? Regards Pavel regards, tom lane
Re: [HACKERS] polymorphic SQL functions has a problem with domains
Tom Lane-2 wrote Pavel Stehule lt; pavel.stehule@ gt; writes: I was informed about impossibility to use a polymorphic functions together with domain types see create domain xx as numeric(15); create or replace function g(anyelement, anyelement) returns anyelement as $$ select $1 + $2 $$ language sql immutable; postgres=# select g(1::xx, 2::xx); ERROR: return type mismatch in function declared to return xx DETAIL: Actual return type is numeric. CONTEXT: SQL function g during inlining That example doesn't say you can't use polymorphic functions with domains. It says that this particular polymorphic function definition is wrong: it is not making sure its result is of the expected data type. I don't recall right now whether SQL functions will apply an implicit cast on the result for you, but even if they do, an upcast from numeric to some domain over numeric wouldn't be implicit. How would that be possible though? Since any number of domains could be defined over numeric as soon as the + operator causes the domain to be lost there is no way to get it back manually - you cannot just make it SELECT ($1 + $2)::xx. Does something like: SELECT ($1 + $2)::$1%TYPE exist where you can explicitly cast to the type of the input argument? David J. -- View this message in context: http://postgresql.1045698.n5.nabble.com/polymorphic-SQL-functions-has-a-problem-with-domains-tp5798349p5798356.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] polymorphic SQL functions has a problem with domains
David Johnston pol...@yahoo.com writes: Does something like: SELECT ($1 + $2)::$1%TYPE exist where you can explicitly cast to the type of the input argument? I don't think SQL-language functions have such a notation, but it's possible in plpgsql, if memory serves. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] polymorphic SQL functions has a problem with domains
2014-04-02 18:27 GMT+02:00 Tom Lane t...@sss.pgh.pa.us: David Johnston pol...@yahoo.com writes: Does something like: SELECT ($1 + $2)::$1%TYPE exist where you can explicitly cast to the type of the input argument? I don't think SQL-language functions have such a notation, but it's possible in plpgsql, if memory serves. No, this possibility doesn't there, what I know. but you can do assignment to some output variable - what is effective same Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] polymorphic SQL functions has a problem with domains
On 2014-04-02 12:27:30 -0400, Tom Lane wrote: David Johnston pol...@yahoo.com writes: Does something like: SELECT ($1 + $2)::$1%TYPE exist where you can explicitly cast to the type of the input argument? I don't think SQL-language functions have such a notation, but it's possible in plpgsql, if memory serves. Sometimes you can play nasty tricks using COALESCE() to force a cast like that. E.g. SELECT COALESCE(NULLIF($1, $1), $1 + $2); Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] polymorphic SQL functions has a problem with domains
Tom Lane-2 wrote David Johnston lt; polobo@ gt; writes: Does something like: SELECT ($1 + $2)::$1%TYPE exist where you can explicitly cast to the type of the input argument? I don't think SQL-language functions have such a notation, but it's possible in plpgsql, if memory serves. Indeed. http://www.postgresql.org/docs/9.3/interactive/plpgsql-declarations.html#PLPGSQL-DECLARATION-TYPE Section 40.3.3 You lose inlining but at least it (should) work. David J. -- View this message in context: http://postgresql.1045698.n5.nabble.com/polymorphic-SQL-functions-has-a-problem-with-domains-tp5798349p5798367.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] polymorphic SQL functions has a problem with domains
2014-04-02 18:34 GMT+02:00 David Johnston pol...@yahoo.com: Tom Lane-2 wrote David Johnston polobo@ writes: Does something like: SELECT ($1 + $2)::$1%TYPE exist where you can explicitly cast to the type of the input argument? I don't think SQL-language functions have such a notation, but it's possible in plpgsql, if memory serves. Indeed. http://www.postgresql.org/docs/9.3/interactive/plpgsql-declarations.html#PLPGSQL-DECLARATION-TYPE This casting is indirect via assignment Section 40.3.3 You lose inlining but at least it (should) work. David J. -- View this message in context: http://postgresql.1045698.n5.nabble.com/polymorphic-SQL-functions-has-a-problem-with-domains-tp5798349p5798367.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
On 04/02/2014 12:25 AM, Andrew Dunstan wrote: On 04/01/2014 09:22 PM, Andrew Dunstan wrote: On 04/01/2014 08:53 PM, Tom Lane wrote: The current typedefs list seems to be lacking any Windows-only typedefs. Noticed while trying to pgindent postmaster.c. Hmm. odd. will check. It's apparently causing the buildfarm to crash, which is why I must have disabled it. I'll chase that down tomorrow. OK, we're back: http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=frogmouthdt=2014-04-02%2016%3A08%3A12stg=typedefs BTW, three animals are currently trying to contribute typedefs but aren't in fact contributing anything: okapi, dromedary and prairiedog. See http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list=1 I can't really help much on these as my Gentoo facilities are non-existent, and my OSX facilities are not much better. I do recall trying to find a way to get typedefs on OSX a few years ago, without success. cheers andrew cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] jsonb is also breaking the rule against nameless unions
Same issue as in http://www.postgresql.org/message-id/31718.1394059...@sss.pgh.pa.us In file included from jsonb.c:19: ../../../../src/include/utils/jsonb.h:195: warning: unnamed struct/union that defines no instances jsonb.c: In function `jsonb_in_object_field_start': jsonb.c:250: structure has no member named `string' jsonb.c:251: structure has no member named `string' jsonb.c:251: structure has no member named `string' jsonb.c:252: structure has no member named `string' jsonb.c: In function `jsonb_put_escaped_value': jsonb.c:266: structure has no member named `string' jsonb.c:266: structure has no member named `string' jsonb.c:271: structure has no member named `numeric' jsonb.c:274: structure has no member named `boolean' jsonb.c: In function `jsonb_in_scalar': jsonb.c:301: structure has no member named `string' jsonb.c:302: structure has no member named `string' jsonb.c:302: structure has no member named `string' jsonb.c:303: structure has no member named `string' ... etc etc etc ... We really need to get a buildfarm member going that complains about this. I had hoped to install a sufficiently old gcc version on prairiedog or dromedary, but didn't have much luck rebuilding ancient gcc releases on OS X. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jsonb is also breaking the rule against nameless unions
Tom Lane wrote: Same issue as in http://www.postgresql.org/message-id/31718.1394059...@sss.pgh.pa.us In file included from jsonb.c:19: ../../../../src/include/utils/jsonb.h:195: warning: unnamed struct/union that defines no instances jsonb.c: In function `jsonb_in_object_field_start': jsonb.c:250: structure has no member named `string' jsonb.c:251: structure has no member named `string' jsonb.c:251: structure has no member named `string' jsonb.c:252: structure has no member named `string' jsonb.c: In function `jsonb_put_escaped_value': jsonb.c:266: structure has no member named `string' jsonb.c:266: structure has no member named `string' jsonb.c:271: structure has no member named `numeric' jsonb.c:274: structure has no member named `boolean' jsonb.c: In function `jsonb_in_scalar': jsonb.c:301: structure has no member named `string' jsonb.c:302: structure has no member named `string' jsonb.c:302: structure has no member named `string' jsonb.c:303: structure has no member named `string' ... etc etc etc ... We really need to get a buildfarm member going that complains about this. I had hoped to install a sufficiently old gcc version on prairiedog or dromedary, but didn't have much luck rebuilding ancient gcc releases on OS X. Complain how? I find that gcc -std=c90 -pedantic emits these warnings about it: def.c:3:24: warning: ISO C90 doesn’t support unnamed structs/unions [-pedantic] def.c:1:8: warning: struct has no named members [-pedantic] -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jsonb is also breaking the rule against nameless unions
On 2014-04-02 13:56:40 -0400, Tom Lane wrote: We really need to get a buildfarm member going that complains about this. I had hoped to install a sufficiently old gcc version on prairiedog or dromedary, but didn't have much luck rebuilding ancient gcc releases on OS X. Some experimentation shows that clang's -Wc11-extensions warns about this... If we could get the build on clang warnings free we could use that together with -Werror... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jsonb is also breaking the rule against nameless unions
On 2014-04-02 15:17:16 -0300, Alvaro Herrera wrote: Tom Lane wrote: Same issue as in http://www.postgresql.org/message-id/31718.1394059...@sss.pgh.pa.us In file included from jsonb.c:19: ../../../../src/include/utils/jsonb.h:195: warning: unnamed struct/union that defines no instances jsonb.c: In function `jsonb_in_object_field_start': jsonb.c:250: structure has no member named `string' jsonb.c:251: structure has no member named `string' jsonb.c:251: structure has no member named `string' jsonb.c:252: structure has no member named `string' jsonb.c: In function `jsonb_put_escaped_value': jsonb.c:266: structure has no member named `string' jsonb.c:266: structure has no member named `string' jsonb.c:271: structure has no member named `numeric' jsonb.c:274: structure has no member named `boolean' jsonb.c: In function `jsonb_in_scalar': jsonb.c:301: structure has no member named `string' jsonb.c:302: structure has no member named `string' jsonb.c:302: structure has no member named `string' jsonb.c:303: structure has no member named `string' ... etc etc etc ... We really need to get a buildfarm member going that complains about this. I had hoped to install a sufficiently old gcc version on prairiedog or dromedary, but didn't have much luck rebuilding ancient gcc releases on OS X. Complain how? I find that gcc -std=c90 -pedantic emits these warnings about it: def.c:3:24: warning: ISO C90 doesn’t support unnamed structs/unions [-pedantic] def.c:1:8: warning: struct has no named members [-pedantic] Last time I checked gcc builds of postgres using -pedantic are so verbose that warnings don't have an effect anymore. Is that not the case anymore? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jsonb is also breaking the rule against nameless unions
Andres Freund and...@2ndquadrant.com writes: On 2014-04-02 15:17:16 -0300, Alvaro Herrera wrote: Tom Lane wrote: We really need to get a buildfarm member going that complains about this. Complain how? I find that gcc -std=c90 -pedantic emits these warnings about it: def.c:3:24: warning: ISO C90 doesnât support unnamed structs/unions [-pedantic] def.c:1:8: warning: struct has no named members [-pedantic] Last time I checked gcc builds of postgres using -pedantic are so verbose that warnings don't have an effect anymore. Is that not the case anymore? Well, in any case, people very seldom check to see if any buildfarm members are producing compiler warnings. You need the build to actually go red to get anyone's attention reliably. I concur that -pedantic is pretty much useless for our purposes anyway. The non-C89 feature that I've been really worried about is flexible array members (which we intend to start using more heavily, so we need a complaint if someone leaves out the FLEXIBLE_ARRAY_MEMBER macro). Based on the last month or so I guess that anonymous unions are a big issue as well. I'd like to have a buildfarm member whose compiler doesn't recognize either of those ... and AFAICT, -pedantic is no help for the array case. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jsonb is also breaking the rule against nameless unions
On 2014-04-02 14:36:28 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-04-02 15:17:16 -0300, Alvaro Herrera wrote: Tom Lane wrote: We really need to get a buildfarm member going that complains about this. Complain how? I find that gcc -std=c90 -pedantic emits these warnings about it: def.c:3:24: warning: ISO C90 doesn’t support unnamed structs/unions [-pedantic] def.c:1:8: warning: struct has no named members [-pedantic] Last time I checked gcc builds of postgres using -pedantic are so verbose that warnings don't have an effect anymore. Is that not the case anymore? Well, in any case, people very seldom check to see if any buildfarm members are producing compiler warnings. You need the build to actually go red to get anyone's attention reliably. Yea, we'd need to be able to turn on -Werror if it's going to have any effect. I don't think our configure currently copes with that unfortunately... I just tried it on clang. It builds clean with -Wc11-extensions except warning about _Static_assert(). That's possibly fixable with some autoconf trickery. The non-C89 feature that I've been really worried about is flexible array members (which we intend to start using more heavily, so we need a complaint if someone leaves out the FLEXIBLE_ARRAY_MEMBER macro). Based on the last month or so I guess that anonymous unions are a big issue as well. I'd like to have a buildfarm member whose compiler doesn't recognize either of those ... and AFAICT, -pedantic is no help for the array case. gcc's -pedantic warns about flexible array members here, but it doesn't solve the problem with it being unusable :( Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jsonb is also breaking the rule against nameless unions
Andres Freund and...@2ndquadrant.com writes: On 2014-04-02 13:56:40 -0400, Tom Lane wrote: We really need to get a buildfarm member going that complains about this. I had hoped to install a sufficiently old gcc version on prairiedog or dromedary, but didn't have much luck rebuilding ancient gcc releases on OS X. Some experimentation shows that clang's -Wc11-extensions warns about this... If we could get the build on clang warnings free we could use that together with -Werror... What's it warning about currently? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jsonb is also breaking the rule against nameless unions
Andres Freund and...@2ndquadrant.com writes: On 2014-04-02 14:36:28 -0400, Tom Lane wrote: Well, in any case, people very seldom check to see if any buildfarm members are producing compiler warnings. You need the build to actually go red to get anyone's attention reliably. Yea, we'd need to be able to turn on -Werror if it's going to have any effect. I don't think our configure currently copes with that unfortunately... I'm pretty sure you can set CFLAGS from the buildfarm configuration options --- see the animals that are building with -DCLOBBER_CACHE_ALWAYS. I just tried it on clang. It builds clean with -Wc11-extensions except warning about _Static_assert(). That's possibly fixable with some autoconf trickery. Ah. That sounds promising. What clang version is that? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jsonb is also breaking the rule against nameless unions
On 2014-04-02 14:42:39 -0400, Tom Lane wrote:
Andres Freund and...@2ndquadrant.com writes:
On 2014-04-02 13:56:40 -0400, Tom Lane wrote:
We really need to get a buildfarm member going that complains about this.
I had hoped to install a sufficiently old gcc version on prairiedog or
dromedary, but didn't have much luck rebuilding ancient gcc releases on
OS X.
Some experimentation shows that clang's -Wc11-extensions warns about
this... If we could get the build on clang warnings free we could use
that together with -Werror...
What's it warning about currently?
So, when I manually put a #undef HAVE__STATIC_ASSERT somewhere relevant,
I can compile pg warning free under clang trunk with:
-std=c89 -Wall -Wextra -pedantic
-Wc11-extensions -Wmissing-declarations
-Wno-unused-parameter -Wno-sign-compare -Wno-missing-field-initializers
-Wno-overlength-strings -Wno-variadic-macros -Wno-long-long
-Wno-gnu-statement-expression
without any warnings.
If I add -Wc99-extensions and remove -pedantic it complains about
FLEXIBLE_ARRAY_MEMBER, commas at the end of enumerator lists, extended
field designators (e.g. offsetof(POLYGON, p[0])).
That it warns about FLEXIBLE_ARRAY_MEMBER suggest our configure test for
that could use some improvement. Commas at the enum of enum list are
easily fixed (patch attached).
The extended offsetof bit is a bit more critical, we use that pretty
widely. Luckily it can be disabled with -Wno-extended-offsetof
There's also the valid warning about:
/home/andres/src/postgresql/src/bin/pg_dump/parallel.c:561:22: warning:
initializer for aggregate is not a compile-time constant
[-Wc99-extensions]
int pipefd[2] =
{pipeMW[PIPE_READ], pipeWM[PIPE_WRITE]};
^~~
1 warning generated.
pedantic also complains about:
/home/andres/src/postgresql/src/bin/psql/tab-complete.c:815:35: warning:
assigning to 'rl_completion_func_t *'
(aka 'char **(*)(const char *, int, int)') from 'void *' converts
between void pointer and function pointer [-Wpedantic]
rl_attempted_completion_function = (void *) psql_completion;
^
which seems valid and solvable.
But there's also:
preproc.y:15039:7: warning: C requires #line number to be less than
32768, allowed as extension [-Wpedantic]
#line 51524 preproc.c
^
which seems pretty pointless.
Some thoughts about -Wno-* flags needed:
-Wno-overlength-strings: I don't care.
-Wno-variadic-macros: suggests the configure test is missing a step.
-Wno-long-long: Looks like sloppy coding in seldomly looked at parts to me.
-Wno-gnu-statement-expression: Haven't looked
This doesn't look too bad. At least without -pedantic the warnings seem
sensible after disabling an option or two...
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()
Normally I'm not for adding gucs that just gate new features. But I think a simple guc to turn this on or off would be fine and alleviate any concerns. I think users would appreciate it quite a lot It would even have a positive effect of helping raise awareness of the feature. I often scan the list of config options to get an idea of new features when I'm installing new software or upgrading. -- greg On 1 Apr 2014 17:38, Jim Nasby j...@nasby.net wrote: On 4/1/14, 1:04 PM, Peter Geoghegan wrote: It strains credulity to think that this patch alone would have that effect, but there might be quite a few similar improvements that are possible. So I think it would be good to consider how far we want to go in this direction and where we think we might want to stop. That's not to say, let's not ever do this, just, let's think carefully about where we want to end up. Fair enough. I agree with the concern, but also have to say that I can't count how many times I could have used this. A big +1, at least in this case. -- Jim C. Nasby, Data Architect j...@nasby.net 512.569.9461 (cell) http://jim.nasby.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()
On 2014-04-02 21:08:47 +0100, Greg Stark wrote: Normally I'm not for adding gucs that just gate new features. But I think a simple guc to turn this on or off would be fine and alleviate any concerns. I think users would appreciate it quite a lot I don't have strong feelings about the feature, but introducing a guc for it feels entirely ridiculous to me. This is a minor detail in an error message, not more. It would even have a positive effect of helping raise awareness of the feature. I often scan the list of config options to get an idea of new features when I'm installing new software or upgrading. Really? Should we now add GUCs for every feature then? Greetings, Andres Freund PS: Could you please start to properly quote again? You seem to have stopped doing that entirely in the last few months. -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()
On Wed, Apr 2, 2014 at 4:16 PM, Andres Freund and...@2ndquadrant.com wrote: I don't have strong feelings about the feature, but introducing a guc for it feels entirely ridiculous to me. This is a minor detail in an error message, not more. I agree. It's just a HINT. It's quite helpful in certain particular contexts, but in the grand scheme of things isn't all that important. I am being quite conservative in trying to anticipate cases where on balance it'll actually hurt more than it will help. I doubt that there actually are any. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)
This is a follow-up to the thread at
http://www.postgresql.org/message-id/4eb5fa1b.1090...@2ndquadrant.com
A quick summary: that thread proposed adding a relation_free_space()
function to the pageinspect extension. Various review comments were
received, among which was the suggestion that the code belonged in
pg_stattuple as a faster way to calculate free_percent.
===
I've attached an extension that produces largely pgstattuple-compatible
numbers for a table without doing a full-table scan.
It scans through the table, skipping blocks that have their visibility
map bit set. For such pages, it gets the free space from the free space
map, and assumes that all remaining space on the page is taken by live
tuples. It scans other pages tuple-by-tuple and counts live and dead
tuples and free space.
Here's a comparison of fastbloat and pgstattuple output on a 50-million
row table with some holes created with a single big DELETE statement:
ams=# select * from fastbloat('x');
table_len | scanned_percent | approx_tuple_count | approx_tuple_len |
approx_tuple_percent | dead_tuple_count | dead_tuple_len | dead_tuple_percent |
free_space | free_percent
+-++--+--+--++++--
6714761216 | 17 | 41318301 | 5483815648 |
81.67 | 8681708 | 258624 | 16.55 |
80972912 | 1.21
(1 row)
Time: 639.455 ms
ams=# select * from pgstattuple('x');
table_len | tuple_count | tuple_len | tuple_percent | dead_tuple_count |
dead_tuple_len | dead_tuple_percent | free_space | free_percent
+-++---+--++++--
6714761216 |41318292 | 5288741376 | 78.76 | 8681708 |
258624 | 16.55 | 91810372 | 1.37
(1 row)
Time: 15610.651 ms
In the above, the table_len is nblocks*BLCKSZ, and the dead_tuple_count,
dead_tuple_len, dead_tuple_percent, free_space, and free_percent are all
exact. scanned_percent shows the percentage of pages that were scanned
tuple-by-tuple (the others having been skipped based on the VM bit).
The live tuple count, size, and percentage are all estimates.
The approx_tuple_count is calculated using vac_estimate_reltuples based
on the pages/tuples that were scanned. The approx_tuple_len is the exact
size of the live tuples on scanned pages, plus the approximate size from
skipped pages (BLCKSZ-GetRecordedFreeSpace()). This is an overestimate,
because it's counting the line pointer array as live tuple space.
Even in the worst case, when every page has dead tuples, fastbloat is
marginally faster than pgstattuple. The same table as the first example,
but with every even-numbered row deleted:
ams=# select * from fastbloat('x');
table_len | scanned_percent | approx_tuple_count | approx_tuple_len |
approx_tuple_percent | dead_tuple_count | dead_tuple_len | dead_tuple_percent |
free_space | free_percent
+-++--+--+--++++--
6714761216 | 100 | 20659150 | 2644371200 |
39.38 | 20659142 | 2644370176 | 39.38 |
1203068996 |17.92
(1 row)
Time: 8924.511 ms
ams=# select * from pgstattuple('x');
table_len | tuple_count | tuple_len | tuple_percent | dead_tuple_count |
dead_tuple_len | dead_tuple_percent | free_space | free_percent
+-++---+--++++--
6714761216 |20659150 | 2644371200 | 39.38 | 20659142 |
2644370176 | 39.38 | 1203068996 |17.92
(1 row)
Time: 13338.712 ms
Since the code depends on the visibility map to determine which pages to
skip, it does not support indexes (which have no visibility map).
(Just drop the attached files into contrib/fastbloat, and make install
should just work. Then just create extension fastbloat.)
Questions and suggestions welcome.
-- Abhijit
/*
* contrib/fastbloat/fastbloat.c
*
* Abhijit Menon-Sen a...@2ndquadrant.com
* Portions Copyright (c) 2001,2002 Tatsuo Ishii (from pg_stattuple)
*
* Permission to use, copy, modify, and distribute this software and
* its documentation for any purpose, without fee, and without a
* written agreement is hereby granted, provided that the above
* copyright notice and this paragraph and the following two
* paragraphs appear in all copies.
*
* IN NO EVENT SHALL THE AUTHOR BE LIABLE TO ANY PARTY FOR DIRECT,
* INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
* LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND
Re: [HACKERS] jsonb is also breaking the rule against nameless unions
On 2014-04-02 15:03:47 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-04-02 14:36:28 -0400, Tom Lane wrote: Well, in any case, people very seldom check to see if any buildfarm members are producing compiler warnings. You need the build to actually go red to get anyone's attention reliably. Yea, we'd need to be able to turn on -Werror if it's going to have any effect. I don't think our configure currently copes with that unfortunately... I'm pretty sure you can set CFLAGS from the buildfarm configuration options --- see the animals that are building with -DCLOBBER_CACHE_ALWAYS. The problem is rather that configure dies somewhere when CFLAGS includes -Werror. Not very nice imo. I just tried it on clang. It builds clean with -Wc11-extensions except warning about _Static_assert(). That's possibly fixable with some autoconf trickery. Ah. That sounds promising. What clang version is that? It's debian's clang-3.5, which is built from trunk IIUC: 1:3.5~svn201651-1 I have some patches that fix the configure tests to properly use -Werror, but I am too tired to check their validity now. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)
On Wed, Apr 2, 2014 at 5:41 PM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: I've attached an extension that produces largely pgstattuple-compatible numbers for a table without doing a full-table scan. It scans through the table, skipping blocks that have their visibility map bit set. For such pages, it gets the free space from the free space map, and assumes that all remaining space on the page is taken by live tuples. It scans other pages tuple-by-tuple and counts live and dead tuples and free space. That's clever. I think it might underestimate free space relative to tuples because the free space map isn't guaranteed to be completely correct. But I guess you knew that already... -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fwd: SSL auth question
On Wed, Apr 2, 2014 at 2:38 AM, carriingfat...@ya.ru wrote: I set certificate auth on postgresql 9.3. I generate SSL certificate with my custom extension. So, OpenSSL read it, PostgreSQL accept it if this extension is not critical, but if I set this extension critical, PostgreSQL deny connection. How can I prevent it? Where PostgreSQL try to read SSL extension? I don't know what it means to set an extension critical. If you provide enough details for someone to reproduce the exact scenario you tried, someone may be able to help. Otherwise, probably not. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: Fwd: [HACKERS] Proposal: variant of regclass
On Wed, Apr 2, 2014 at 1:41 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Mar 31, 2014 at 7:08 PM, Yugo Nagata nag...@sraoss.co.jp wrote: Hi Amit Kapila, Thank you for your reviewing. I updated the patch to v5. I have checked the latest version and found few minor improvements that are required: 1. ! if (!missing_ok) ! ereport(ERROR, ! (errcode(ERRCODE_UNDEFINED_OBJECT), ! errmsg(type \%s\ does not exist, ! TypeNameToString(typeName)), ! parser_errposition(NULL, typeName-location))); pfree(buf.data); seems to be missing in error cases. Eh, surely this is being done in some memory context that an error will reset anyway? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fwd: SSL auth question
On 1 Apr 2014, at 11:38 PM, carriingfat...@ya.ru wrote: I set certificate auth on postgresql 9.3. I generate SSL certificate with my custom extension. So, OpenSSL read it, PostgreSQL accept it if this extension is not critical, but if I set this extension critical, PostgreSQL deny connection. I think that is the correct behavior. The critical bit tells PostgreSQL (or other software) what to do if it does not understand the extension: if there's an unknown extension with the critical bit set, then the certificate can't be validated. If the critical bit is not set, then the unknown extension is ignored, and the certificate is processed as if the extension weren't there. See this section of RFC 5280: http://tools.ietf.org/html/rfc5280#section-4.2 The idea is that you can set the critical bit for extensions that are supposed *restrict* the usability of the certificate, so that the certificate won't be used in undesired ways by software that doesn't understand the extension. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
Andrew Dunstan and...@dunslane.net writes: BTW, three animals are currently trying to contribute typedefs but aren't in fact contributing anything: okapi, dromedary and prairiedog. See http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list=1 Man, that's a short list. I wonder if we need to encourage more people to do this. I can't really help much on these as my Gentoo facilities are non-existent, and my OSX facilities are not much better. I do recall trying to find a way to get typedefs on OSX a few years ago, without success. I poked around a bit, and so far as I can tell, OS X does not store debug symbol tables in executables. It looks like gdb goes to the .o files when it wants debug info. What's in the .o files is good ol' DWARF (at least in reasonably recent OS X releases), so it's not any harder to pull out the typedef names than it is on Linux. The problem is that you gotta iterate over all the .o files in the build tree rather than the installed executables. I looked at fixing find_typedefs but it seems like it's pretty fixated on the latter approach; any thoughts on how to revise it? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
On 04/02/2014 08:43 PM, Tom Lane wrote:
Andrew Dunstan and...@dunslane.net writes:
BTW, three animals are currently trying to contribute typedefs but
aren't in fact contributing anything: okapi, dromedary and prairiedog.
See http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list=1
Man, that's a short list. I wonder if we need to encourage more people
to do this.
I can't really help much on these as my Gentoo facilities are
non-existent, and my OSX facilities are not much better. I do recall
trying to find a way to get typedefs on OSX a few years ago, without
success.
I poked around a bit, and so far as I can tell, OS X does not store debug
symbol tables in executables. It looks like gdb goes to the .o files when
it wants debug info. What's in the .o files is good ol' DWARF (at least
in reasonably recent OS X releases), so it's not any harder to pull out
the typedef names than it is on Linux. The problem is that you gotta
iterate over all the .o files in the build tree rather than the installed
executables. I looked at fixing find_typedefs but it seems like it's
pretty fixated on the latter approach; any thoughts on how to revise it?
Well, the reason it's that way is that that's the way it was done before
the buildfarm took over the task. But it's not holy writ. Doing
something else would be a SMOC.
Essentially, I think the part that would need to change is this:
foreach my $bin (
glob($installdir/bin/*),
glob($installdir/lib/*),
glob($installdir/lib/postgresql/*)
)
For OSX we'd construct the list via File::Find to recurse through the
directories.
So, something like this:
my $using_osx = [some test for OSX];
my @testfiles;
my $obj_wanted = sub {
/^.*\.o\z/s
push(@testfiles, $File::Find::name);
};
if ($using_osx)
{
File::Find::find($obj_wanted,$pgsql);
}
else
{
@testfiles = (
glob($installdir/bin/*),
glob($installdir/lib/*),
glob($installdir/lib/postgresql/*);
}
foreach my $bin (@testfiles)
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
On 2 Apr 2014, at 5:43 PM, Tom Lane wrote: I poked around a bit, and so far as I can tell, OS X does not store debug symbol tables in executables. It looks like gdb goes to the .o files when it wants debug info. What's in the .o files is good ol' DWARF (at least in reasonably recent OS X releases), so it's not any harder to pull out the typedef names than it is on Linux. The problem is that you gotta iterate over all the .o files in the build tree rather than the installed executables. I looked at fixing find_typedefs but it seems like it's pretty fixated on the latter approach; any thoughts on how to revise it? The Apple development tools gather the debug information during the final link stage (the one that produces the executable or shared object) using dsymutil, which simply iterates over all of the .o files and links the debug info into a separate object, foo.dSYM. Apple's gdb and lldb then find the relevant .dSYM file using a per-build UUID embedded in the executable/library/debug symbol file. The dSYM file is a normal object file which has the DWARF sections but not the usual text/data sections, so if it can be generated/found, it should be possible to dump the DWARF data from it and look for typedefs that way. (I'm pretty sure that if you were to run dsymutil and then merge the resulting object file's sections into the executable/shlib, you'd get a perfectly functional and debuggable result without having to look for or cart around the extra dSYM file--- I haven't tried this though.) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
Wim Lewis w...@omnigroup.com writes: On 2 Apr 2014, at 5:43 PM, Tom Lane wrote: I poked around a bit, and so far as I can tell, OS X does not store debug symbol tables in executables. The Apple development tools gather the debug information during the final link stage (the one that produces the executable or shared object) using dsymutil, which simply iterates over all of the .o files and links the debug info into a separate object, foo.dSYM. Apple's gdb and lldb then find the relevant .dSYM file using a per-build UUID embedded in the executable/library/debug symbol file. Ah. I've forgotten the details, but I'm pretty sure that we have deliberately arranged our build process so that the .dSYM files don't get built during link steps. Debugging seems to work all right anyway, at least if the build tree is available, so I think Apple's gdb is able to work from the symbol tables in the .o files. While perhaps that approach should be rethought, I'm disinclined to mess with it just for the benefit of find_typedefs. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GSoC proposal - make an unlogged table logged
On Tue, Apr 1, 2014 at 1:40 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-01 13:37:57 -0300, Fabrízio de Royes Mello wrote: In the GSoC proposal page [1] I received some suggestions to strech goals: * ALTER TABLE name SET UNLOGGED. This is essentially the reverse of the core proposal, which is ALTER TABLE name SET LOGGED. Yes, I think that should definitely be included. It would be weird to have SET LOGGED but not SET UNLOGGED. Yes, that makes sense. * Allow unlogged indexes on logged tables. I don't think it's realistic to build the infrastructure necessary for that as part of gsoc. The reasons have been explained somewhere in this thread. * Implement ALTER TABLE name SET LOGGED without rewriting the whole table, when wal_level = minimal. Yea, maybe. * Allow unlogged materialized views. I don't think that's realistic either. Thanks Andres for your comments. Anyway I added this additional goals to proposal and as Heikki said: It's actually nice to have several separate goals like this, it means that if you finish the task faster than expected, you can move on to the next goal, and if one task takes longer than expected so that you don't finish all the work, we'll still have something useful. I hope you can help me in some way with this project too. :-) Regards, -- Fabrízio de Royes Mello Consultoria/Coaching PostgreSQL Timbira: http://www.timbira.com.br Blog sobre TI: http://fabriziomello.blogspot.com Perfil Linkedin: http://br.linkedin.com/in/fabriziomello Twitter: http://twitter.com/fabriziomello
Re: Fwd: [HACKERS] Proposal: variant of regclass
On Thu, Apr 3, 2014 at 5:43 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 2, 2014 at 1:41 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Mar 31, 2014 at 7:08 PM, Yugo Nagata nag...@sraoss.co.jp wrote: Hi Amit Kapila, Thank you for your reviewing. I updated the patch to v5. I have checked the latest version and found few minor improvements that are required: 1. ! if (!missing_ok) ! ereport(ERROR, ! (errcode(ERRCODE_UNDEFINED_OBJECT), ! errmsg(type \%s\ does not exist, ! TypeNameToString(typeName)), ! parser_errposition(NULL, typeName-location))); pfree(buf.data); seems to be missing in error cases. Eh, surely this is being done in some memory context that an error will reset anyway? Right, it will get reset in error. However still we need to free for missing_ok case and when it is successful in getting typeid. So don't you think it is better to just free once before calling LookupTypeName()? The code is right in it's current form as well, it's just a minor suggestion for improvement, so if you think current way the code written is okay, then ignore this suggestion. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
Andrew Dunstan and...@dunslane.net writes:
For OSX we'd construct the list via File::Find to recurse through the
directories.
So, something like this:
Thanks for the tips. The attached patch against buildfarm 4.11 seems
to work, cf
http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=dromedarydt=2014-04-03%2003%3A33%3A16stg=typedefs
I tried it on current OSX (10.9.2) as well as dromedary's 10.6.8.
It does not work on prairiedog (10.4.11) --- the debug info format
seems to be different that far back. Can't say that's worth worrying
about.
regards, tom lane
*** build-farm-4.11/run_build.pl~ Fri Jun 14 09:05:52 2013
--- build-farm-4.11/run_build.pl Wed Apr 2 23:31:27 2014
***
*** 1506,1520
my $objdump = $host || 'objdump';
my @err = `$objdump -W 21`;
my @readelferr = `readelf -w 21`;
my %syms;
my @dumpout;
my @flds;
! foreach my $bin (
! glob($installdir/bin/*),
! glob($installdir/lib/*),
! glob($installdir/lib/postgresql/*)
! )
{
next if $bin =~ m!bin/(ipcclean|pltcl_)!;
next unless -f $bin;
--- 1506,1536
my $objdump = $host || 'objdump';
my @err = `$objdump -W 21`;
my @readelferr = `readelf -w 21`;
+ my $using_osx = (`uname` eq Darwin\n);
+ my @testfiles;
my %syms;
my @dumpout;
my @flds;
! if ($using_osx)
! {
! # On OS X, we need to examine the .o files
! my $obj_wanted = sub {
! /^.*\.o\z/s push(@testfiles, $File::Find::name);
! };
!
! File::Find::find($obj_wanted,$pgsql);
! }
! else
! {
! # Elsewhere, look at the installed executables and shared libraries
! @testfiles = (
! glob($installdir/bin/*),
! glob($installdir/lib/*),
! glob($installdir/lib/postgresql/*)
! );
! }
! foreach my $bin (@testfiles)
{
next if $bin =~ m!bin/(ipcclean|pltcl_)!;
next unless -f $bin;
***
*** 1534,1540
}
elsif ( @readelferr 10 )
{
-
# FreeBSD, similar output to Linux
@dumpout =
`readelf -w $bin 2/dev/null | egrep -A3 DW_TAG_typedef 2/dev/null`;
--- 1550,1555
***
*** 1546,1551
--- 1561,1579
$syms{$flds[-1]} =1;
}
}
+ elsif ($using_osx)
+ {
+ @dumpout =
+ `dwarfdump $bin 2/dev/null | egrep -A2 TAG_typedef 2/dev/null`;
+ foreach (@dumpout)
+ {
+ @flds = split;
+ next unless (@flds == 3);
+ next unless ($flds[0] eq AT_name();
+ next unless ($flds[1] =~ m/^(.*)$/);
+ $syms{$1} =1;
+ }
+ }
else
{
@dumpout = `$objdump --stabs $bin 2/dev/null`;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
I wrote: Thanks for the tips. The attached patch against buildfarm 4.11 seems to work, cf http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=dromedarydt=2014-04-03%2003%3A33%3A16stg=typedefs BTW, after looking a bit more closely at what this added to the typedefs list, I realize that this mechanism also collects typedefs that appear in files that are compiled but never installed, for example src/timezone/zic and the ecpg regression tests. This seems like a Good Thing, since certainly pgindent is going to hit the source files for those programs too. I wonder if we ought to switch over to scanning the .o files on all platforms? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] a fast bloat measurement tool
At 2014-04-02 20:10:54 -0400, robertmh...@gmail.com wrote: I think it might underestimate free space relative to tuples because the free space map isn't guaranteed to be completely correct. But I guess you knew that already... Yes, and tuple_len is already a slight overestimate (because it counts the line pointer array as live tuple space for skipped pages). But for the speedup it provides, I believe the result is still useful. I'll mention the potential for free-space inaccuracy in the README. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()
On Wed, Apr 2, 2014 at 4:16 PM, Andres Freund and...@2ndquadrant.comwrote: PS: Could you please start to properly quote again? You seem to have stopped doing that entirely in the last few months. I've been responding a lot from the phone. Unfortunately the Gmail client on the phone makes it nearly impossible to format messages well. I'm beginning to think it would be better to just not quote at all any more. I'm normally not doing a point-by-point response anyways. -- greg
