Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Marko Kreen
On 9/21/08, Tom Lane [EMAIL PROTECTED] wrote: Joe Conway [EMAIL PROTECTED] writes: Good point -- I'll look into that and post something tomorrow. How does requirepassword sound for the option? It is consistent with requiressl but a bit long and hard to read. Maybe require_password?

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Simon Riggs
On Sun, 2008-09-21 at 04:38 +0400, Dmitry Koterov wrote: Is it possible to create a foreign key constraint for ALL elements of an array field? CREATE TABLE a(id INTEGER); CREATE TABLE b(id INTEGER, a_ids INTEGER[]); Field b.a_ids contains a list of ID's of a table. I want to ensure

[HACKERS] Toasted table not deleted when no out of line columns left

2008-09-21 Thread Zoltan Boszormenyi
Hi, we came across a database where a table had a toasted table, keeping huge amounts of disk space allocated. However, the table's current definition didn't explain why there was a toasted table. Then upon some experiments, it struck me. There _was_ a toasted field but as the schema was

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-21 Thread Tom Lane
Zoltan Boszormenyi [EMAIL PROTECTED] writes: we came across a database where a table had a toasted table, keeping huge amounts of disk space allocated. However, the table's current definition didn't explain why there was a toasted table. Then upon some experiments, it struck me. There _was_ a

Re: [HACKERS] Assert Levels

2008-09-21 Thread Peter Eisentraut
Simon Riggs wrote: Well, we don't. That's why I'd suggest to do it slowly and classify everything as medium weight until proven otherwise. Once you have classified all asserts, what do we do with the result? What would be the practical impact? What would be your recommendation about who

[HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
I had a thought while looking over the column-level privileges patch that Stephen Frost is working on. To wit, that the only reason that column default expressions are stored in a separate catalog pg_attrdef is the historical assumption in some parts of the code that pg_attribute rows are

Re: [HACKERS] Assert Levels

2008-09-21 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: Simon Riggs wrote: Well, we don't. That's why I'd suggest to do it slowly and classify everything as medium weight until proven otherwise. Once you have classified all asserts, what do we do with the result? What would be the practical impact?

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Joshua D. Drake
Tom Lane wrote: A possible objection to this plan is that if the column-level privileges patch doesn't get in, then we're left with a useless column in pg_attribute. But an always-null column doesn't cost much of anything, and we know that sooner or later we will support per-column ACLs: they

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Marko Kreen wrote: On 9/21/08, Tom Lane [EMAIL PROTECTED] wrote: Joe Conway [EMAIL PROTECTED] writes: Good point -- I'll look into that and post something tomorrow. How does requirepassword sound for the option? It is consistent with requiressl but a bit long and hard to read. Maybe

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
Joshua D. Drake [EMAIL PROTECTED] writes: Tom Lane wrote: A possible objection to this plan is that if the column-level privileges patch doesn't get in, then we're left with a useless column in pg_attribute. But an always-null column doesn't cost much of anything, and we know that sooner or

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread dpage
pgadmin has some umm, interesting queries over pg_depends. It sounds like this change could complicate those. I doubt it's an insurmountable problem of course. On 9/21/08, Tom Lane [EMAIL PROTECTED] wrote: Joshua D. Drake [EMAIL PROTECTED] writes: Tom Lane wrote: A possible objection to this

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
[EMAIL PROTECTED] writes: pgadmin has some umm, interesting queries over pg_depends. It sounds like this change could complicate those. I doubt it's an insurmountable problem of course. Yeah. But the only real point of the change is cleanliness, and if it's injecting ugliness into clients

[HACKERS] pg_settings.sourcefile patch is a security breach

2008-09-21 Thread Tom Lane
We go to some lengths to prevent non-superusers from examining data_directory and other values that would tell them exactly where the PG data directory is in the server's filesystem. The recently applied patch to expose full pathnames of GUC variables' source files blows a hole a mile wide in

Re: [HACKERS] pg_settings.sourcefile patch is a security breach

2008-09-21 Thread Magnus Hagander
Tom Lane wrote: We go to some lengths to prevent non-superusers from examining data_directory and other values that would tell them exactly where the PG data directory is in the server's filesystem. The recently applied patch to expose full pathnames of GUC variables' source files blows a

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Dmitry Koterov
Normalization is not a panacea here. Sometimes such normalization creates too much overeat and a lot of additional code (especially if there are a lot of such dependencies). Array support in Postgres is quite handy; in my practive, moving from a_b_map to arrays economizes hundreds of lines of

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Andrew Dunstan
Simon Riggs wrote: No, its not possible. Need a trigger. I think we should support it though. If we extend the relational model with arrays then it would be sensible if we support this aspect as well. Implementation would be fairly straightforward. ri_triggers currently assumes a non-array

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread David Fetter
On Sun, Sep 21, 2008 at 10:49:56PM +0400, Dmitry Koterov wrote: Normalization is not a panacea here. Sometimes such normalization creates too much overeat and a lot of additional code (especially if there are a lot of such dependencies). Array support in Postgres is quite handy; in my

Re: [HACKERS] Predictable order of SQL commands in pg_dump

2008-09-21 Thread Dmitry Koterov
Unfortunately, I cannot reproduce this with 100% effect. But, time to time I execute diff utility for a database and notice that two or more trigger or constraint definitions (or something else) are permuted. Something like this: +ALTER TABLE ONLY a +ADD CONSTRAINT fk_b_Id FOREIGN KEY

[HACKERS] parallel pg_restore

2008-09-21 Thread Andrew Dunstan
I am working on getting parallel pg_restore working. I'm currently getting all the scaffolding working, and hope to have a naive prototype posted within about a week. The major question is how to choose the restoration order so as to maximize efficiency both on the server and in reading the

Re: [HACKERS] Assert Levels

2008-09-21 Thread Tom Lane
Greg Smith [EMAIL PROTECTED] writes: The next time I'm doing some performance testing I'll try to quantify how much damage the expensive ones do by playing with pg_config_manual.h. Normally I'm testing with 1GB+ of shared_buffers which makes the current assert scheme unusable. There is a

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Marko Kreen
On 9/21/08, Joe Conway [EMAIL PROTECTED] wrote: Marko Kreen wrote: You need to ignore pg_service also. (And PGPASSWORD) Why? pg_service does not appear to support wildcards, so what is the attack vector? service=foo host=custom And on PGPASSWORD, the fine manual says the following:

Re: [HACKERS] Assert Levels

2008-09-21 Thread Greg Smith
On Fri, 19 Sep 2008, Tom Lane wrote: Well, there are certain things that --enable-cassert turns on that are outrageously expensive...I don't think anyone knows what the performance impact of just the regular Asserts is; it's been too long since these other things were stuck in there. The

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-21 Thread Hans-Jürgen Schönig
*snip* Judging from that, the toasted table cleanup may be part of ALTER TABLE DROP COLUMN. That would only help if you were dropping the last potentially- toastable column of a table. And implementing it would require introducing weird corner cases into the tuple toaster, because it

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Marko Kreen [EMAIL PROTECTED] writes: On 9/21/08, Joe Conway [EMAIL PROTECTED] wrote: Why? pg_service does not appear to support wildcards, so what is the attack vector? service=foo host=custom The proposal to require a password = foo entry in the conn string seems to

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: BTW, a possible hole in this scheme would be if a user could supply a conninfo string that was intentionally malformed in a way that would cause a tacked-on pgpassfile option to be ignored by libpq. We might need to add some validity checks to dblink, or tighten libpq's own

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
Alex Hunsaker [EMAIL PROTECTED] writes: Hrm, I thought if anything we wanted to put them in pg_constraints (at least inherited ones). Now maybe I have defaults confused with NOT NULLs... But don't we want to be able to give defaults names and and such? No, I think you're thinking of NOT

Re: [HACKERS] parallel pg_restore

2008-09-21 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: I am working on getting parallel pg_restore working. I'm currently getting all the scaffolding working, and hope to have a naive prototype posted within about a week. The major question is how to choose the restoration

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Marko Kreen [EMAIL PROTECTED] writes: On 9/21/08, Joe Conway [EMAIL PROTECTED] wrote: Why? pg_service does not appear to support wildcards, so what is the attack vector? service=foo host=custom The proposal to require a password = foo entry in the conn string seems to resolve all of these,

Re: [HACKERS] parallel pg_restore

2008-09-21 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: I am working on getting parallel pg_restore working. I'm currently getting all the scaffolding working, and hope to have a naive prototype posted within about a week. The major question is how to choose the restoration order so as to maximize

Re: [HACKERS] Predictable order of SQL commands in pg_dump

2008-09-21 Thread Tom Lane
Dmitry Koterov [EMAIL PROTECTED] writes: CREATE TRIGGER t000_set_id -BEFORE INSERT OR DELETE OR UPDATE ON a +BEFORE INSERT OR DELETE OR UPDATE ON b FOR EACH ROW EXECUTE PROCEDURE i_trg(); CREATE TRIGGER t000_set_id -BEFORE INSERT OR DELETE OR UPDATE ON b +

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Dmitry Koterov
I strongly suspect you'd benefit a lot more by learning database best practices rather than assuming, as you appear to be doing, that you are dealing with a new field and that you know it best. Neither is true. Of course, you absolutely right. I venerate you! O! :-) -- Sent via

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Alex Hunsaker
On Sun, Sep 21, 2008 at 11:09 AM, Tom Lane [EMAIL PROTECTED] wrote: A possible objection to this plan is that if the column-level privileges patch doesn't get in, then we're left with a useless column in pg_attribute. But an always-null column doesn't cost much of anything, and we know that

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Simon Riggs
On Sun, 2008-09-21 at 15:07 -0400, Andrew Dunstan wrote: Simon Riggs wrote: No, its not possible. Need a trigger. I think we should support it though. If we extend the relational model with arrays then it would be sensible if we support this aspect as well. Implementation would

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: If we push the responsibility back to dblink, we might as well export conninfo_parse() or some wrapper thereof and let dblink simply check for a non-null password from the very beginning. That's not totally unreasonable, since we already export the

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-21 Thread Tom Lane
=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= [EMAIL PROTECTED] writes: ... And implementing it would require introducing weird corner cases into the tuple toaster, because it might now come across TOAST pointers that point to a no-longer-existent table, and have to consider that to be a no-op

Re: [HACKERS] Predictable order of SQL commands in pg_dump

2008-09-21 Thread Dmitry Koterov
Great! Would it be implemented in a next version? Seems it would be very helpful, especially for people who commit database structure to CVS/SVN once per minute to track changes history (or similar)... On Sun, Sep 21, 2008 at 11:57 PM, Tom Lane [EMAIL PROTECTED] wrote: Dmitry Koterov [EMAIL

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Marko Kreen [EMAIL PROTECTED] writes: On 9/21/08, Joe Conway [EMAIL PROTECTED] wrote: Why? pg_service does not appear to support wildcards, so what is the attack vector? service=foo host=custom The proposal to require a password = foo entry in the conn string seems to

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Tom Lane wrote: So that seems to tilt the decision towards exposing the conninfo_parse function. Joe, do you want to have a go at it, or shall I? Here's a first shot. Hmm ... one problem with this is that the caller can't tell

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Hmm ... one problem with this is that the caller can't tell failure-because-out-of-memory from failure-because-string-is-bogus. snip Is it worth having the PQconninfoParse function pass back the error message to avoid this corner case? I thought briefly about it, and wasn't

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: If the return value is NULL, use errmsg if you'd like. I'd guess in most instances you don't even need to bother freeing errmsg as it is in a limited life memory context. Uh, you're confusing the backend environment with libpq's much more spartan

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: If the return value is NULL, use errmsg if you'd like. I'd guess in most instances you don't even need to bother freeing errmsg as it is in a limited life memory context. Uh, you're confusing the backend environment with libpq's much more

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Tom Lane wrote: Uh, you're confusing the backend environment with libpq's much more spartan lifestyle. errmsg will be malloc'd and it will *not* go away unless the caller free()s it. Yup, just figured that out. Otherwise OK with it? Yeah. We could make

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: I can think of a way around that: represent a default expression using classid = OID of pg_attribute, objid = OID of table, objsubid = column attnum. This is distinct from the column itself, which is represented with classid = OID of pg_class. It seems

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
Stephen Frost [EMAIL PROTECTED] writes: Honestly, I really disliked the code which assumed pg_attribute had no NULLable/toastable columns and used what seemed like pretty gruesome hacks to create pg_attribute structures. Agreed, but that seems orthogonal to the point here, which is that a

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Yeah. We could make one further refinement: callers that don't care about acquiring an error string can pass NULL for the errmsg parameter. That tells PQconninfoParse to throw away the errmsg string anyway. With that, the minimal case isn't much uglier than your original: just

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: Stephen Frost [EMAIL PROTECTED] writes: If we were to accept the pg_attrdef approach, why aren't we doing a pg_attracl table instead of adding a column to pg_attribute? That's actually not an unreasonable question. If you were to do that then you

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: New patch attached. This is close, but you're failing to guard against a few out-of-memory corner cases (and now that I look, PQconndefaults() is too). The libpq documentation needs more work than this, too. I'll make a cleanup pass and commit. BTW, I'm

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: New patch attached. erm ... wait a minute. This approach doesn't actually solve the problem at all, because conninfo_parse is responsible for filling in various sorts of default values. In particular it would happily pull a password from the services file

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: New patch attached. erm ... wait a minute. This approach doesn't actually solve the problem at all, because conninfo_parse is responsible for filling in various sorts of default values. In particular it would happily pull a password from

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Tom Lane wrote: Refactoring doesn't seem like an easy way to fix this, because of the problem that the behavior of pulling up defaults is part of the API specification for PQconndefaults(). conninfo_parse() is presently only called from a few places --

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Joe Conway wrote: Tom Lane wrote: Refactoring doesn't seem like an easy way to fix this, because of the problem that the behavior of pulling up defaults is part of the API specification for PQconndefaults(). Thoughts? Hmm, I could have sworn I looked for that, and saw it elsewhere. Anyway,

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Maybe better: static PQconninfoOption * conninfo_parse(const char *conninfo, PQExpBuffer errorMessage, bool fill_defaults, bool *password_from_string) I'm thinking a separate conninfo_fill_defaults function is better, though it's not a big

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: What do you think about getting rid of the password_from_string state variable? It was always a bit of a kluge, and we don't seem to need it anymore with this approach. It is still used in PQconnectionUsedPassword(). That is still needed to prevent a non-superuser from

[HACKERS] pg_dump feature

2008-09-21 Thread Naz
Hi all, I brought this up a few years ago in the 7.4 days, and since there is still no satisfactory solution to this I thought I'd raise it again. When dumping a schema, it is often necessary to dump the tables separately to the constraints and other non-structural metadata. The most obvious

Re: [HACKERS] pg_dump feature

2008-09-21 Thread David Fetter
On Mon, Sep 22, 2008 at 03:25:35AM +1000, Naz wrote: Hi all, I brought this up a few years ago in the 7.4 days, and since there is still no satisfactory solution to this I thought I'd raise it again. When dumping a schema, it is often necessary to dump the tables separately to the