Re: [HACKERS] (WIP) VACUUM REWRITE - CLUSTER by ctid

2009-10-23 Thread Heikki Linnakangas
Itagaki Takahiro wrote: I'm working on alternative version of VACUUM FULL, which is like CLUSTER but sort tuples in ctid order without index. The original discussion is here: [HACKERS] Feedback on getting rid of VACUUM FULL

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Dimitri Fontaine dfonta...@hi-media.com writes: But it will set GET DIAGNOSTIC ... = ROW_COUNT, am I being told on IRC. This has been discussed before, please read archives. The thread I found is pretty content free as far as answering my question goes.

Re: [HACKERS] Controlling changes in plpgsql variable resolution

2009-10-23 Thread Peter Eisentraut
On Tue, 2009-10-20 at 10:32 -0400, Tom Lane wrote: That's only sane if you are 100% certain that there could not be a security issue arising from the change of behavior. Otherwise someone could for instance subvert a security-definer function by running it under the setting it wasn't written

Re: [HACKERS] table corrupted

2009-10-23 Thread Dimitri Fontaine
João Eugenio Marynowski joa...@gmail.com writes: 2009-10-16 16:11:47 BRT 192.168.0.29 ERROR:  invalid page header in block 462821 of relation ... It could be that the following will prove helpful: http://archives.postgresql.org/pgsql-general/2007-07/msg00506.php

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Heikki Linnakangas
Marc Munro wrote: Here is a typical veil secured view definition: create view parties as SELECT party_id, client_id, party_type_id, username, party_name FROM parties.parties WHERE api.user_has_client_or_personal_privilege(client_id, party_id,

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Simon Riggs
On Fri, 2009-10-23 at 11:30 +0300, Heikki Linnakangas wrote: The most user-friendly and backwards-compatible (though not necessarily back-patchable) approach I can see is: 1. If the user has read access to all the underlying tables, plan it like we do today. For me, it would make most

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Pavel Stehule
2009/10/23 Simon Riggs si...@2ndquadrant.com: On Fri, 2009-10-23 at 11:30 +0300, Heikki Linnakangas wrote: The most user-friendly and backwards-compatible (though not necessarily back-patchable) approach I can see is: 1. If the user has read access to all the underlying tables, plan it like

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Cédric Villemain
Le vendredi 23 octobre 2009 01:08:15, Joshua D. Drake a écrit : On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: On Thu, Oct 22, 2009 at 11:16 AM, Cédric Villemain cedric.villem...@dalibo.com wrote: Le lundi 19 octobre 2009 23:27:20, Greg Stark a écrit : On Mon, Oct 19, 2009 at

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread KaiGai Kohei
Simon Riggs wrote: On Fri, 2009-10-23 at 11:30 +0300, Heikki Linnakangas wrote: The most user-friendly and backwards-compatible (though not necessarily back-patchable) approach I can see is: 1. If the user has read access to all the underlying tables, plan it like we do today. For me,

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Simon Riggs
On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: Also, we should presume that any function created with SECURITY DEFINER and created by a superuser would have plan security, so we don't need to annotate lots of old code to work securely. Annotating the built-in functions is a lot

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread KaiGai Kohei
Simon Riggs wrote: On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: Also, we should presume that any function created with SECURITY DEFINER and created by a superuser would have plan security, so we don't need to annotate lots of old code to work securely. Annotating the built-in

Re: [HACKERS] SE-PgSQL developer documentation (Re: Reworks for Access Control facilities (r2363))

2009-10-23 Thread KaiGai Kohei
KaiGai Kohei wrote: After the long trial-and-errors, we learned a few approaches which use common entry points for both of DAC and MAC were rocky-path more than what we initially imagined. So, we came back to the original design. It deploys MAC hooks on the strategic points of core routines.

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Heikki Linnakangas
Simon Riggs wrote: Also, we should presume that any function created with SECURITY DEFINER and created by a superuser would have plan security, so we don't need to annotate lots of old code to work securely. Annotating the built-in functions is a lot easier. SECURITY DEFINER is an orthogonal

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Heikki Linnakangas
Rod Taylor wrote: This still allow many optimizations to be applied in complex cases. The planner CREATE VIEW phone_number AS SELECT person, phone, company FROM phone_data USING SECURITY FILTER(phone NOT LIKE '6%') JOIN person USING (person_id) JOIN company USING

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Richard Huxton
Heikki Linnakangas wrote: The most useful automatic annotation I can see is to treat functions implementing B-tree operators as safe. I *think* that's safe, anyway. Index lookups and single-type comparisons were the only things I could come up with as safe. Unless there is some way to generate

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 5:23 AM, Cédric Villemain cedric.villem...@dalibo.com wrote: Le vendredi 23 octobre 2009 01:08:15, Joshua D. Drake a écrit : On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: On Thu, Oct 22, 2009 at 11:16 AM, Cédric Villemain cedric.villem...@dalibo.com wrote:

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Cédric Villemain
Le vendredi 23 octobre 2009 14:23:09, Robert Haas a écrit : On Fri, Oct 23, 2009 at 5:23 AM, Cédric Villemain cedric.villem...@dalibo.com wrote: Le vendredi 23 octobre 2009 01:08:15, Joshua D. Drake a écrit : On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: On Thu, Oct 22, 2009 at

Re: [HACKERS] EvalPlanQual seems a tad broken

2009-10-23 Thread Tom Lane
I wrote: On further review it seems that a better way to do this is to make things happen inside the EPQ machinery. We need to freeze the rows returned by *all* scan nodes, not only the ones referencing real tables --- for example, a join against a VALUES scan node would still be a problem

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Tom Lane
Dimitri Fontaine dfonta...@hi-media.com writes: I'll go search for more, meantime I'll just add the main goal of this new thread is to have -hackers know there is a real user demand for having EXECUTE set FOUND the same way it sets GET DIAGNOSTIC. [shrug...] There is also real user demand for

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: Sorry, what is happen if function is marked as plan security? I was suggesting an intelligent default by which we could determine function marking implicitly, if it was not explicitly stated on

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Dimitri Fontaine dfonta...@hi-media.com writes: I'll go search for more, meantime I'll just add the main goal of this new thread is to have -hackers know there is a real user demand for having EXECUTE set FOUND the same way it

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread David Fetter
On Fri, Oct 23, 2009 at 10:04:29AM -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: Sorry, what is happen if function is marked as plan security? I was suggesting an intelligent default by which we could determine

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Andrew Dunstan
Robert Haas wrote: On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Dimitri Fontaine dfonta...@hi-media.com writes: I'll go search for more, meantime I'll just add the main goal of this new thread is to have -hackers know there is a real user demand for having

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: For instance, one feature that I'm considering now is a temporal join which is a join on overlaps rather than equals, e.g.: SELECT * FROM a, b WHERE a.x b.x; I might try to provide a modified merge join algorithm to implement that more efficiently in

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Heikki Linnakangas
Tom Lane wrote: The thought that's been in the back of my mind is that you could solve 99% of the performance problem if you trusted all builtin functions and nothing else. This avoids the question of who gets to mark functions as trustable. Except that all builtin functions are not

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: [shrug...]  There is also real user demand for not silently breaking code that works now, which is what we risk anytime we change the set of statements that can set FOUND. We've had

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Any change here is *not* a bug fix, it is a change of clearly documented and not-obviously-unreasonable behavior. We have to take seriously the likelihood that it will break existing code. Perhaps plpgsql could support tests of SQLSTATE, and recognize

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Kevin Grittner
David Fetter da...@fetter.org wrote: One of the things the security community has learned is that the only way it's even possible to get an information leak rate of zero is to have a system which does nothing at all. It's a fact we need to bear in mind when addressing this or any other

Re: [HACKERS] client_lc_messages

2009-10-23 Thread Alvaro Herrera
Peter Eisentraut wrote: Note that only glibc supports switching the language at run time. And doing it is probably very expensive if you want to do it twice per message. Ouch :-( I think you could probably get much of the use case out of this if you concentrate on making two switches for

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Merlin Moncure
On Fri, Oct 23, 2009 at 11:07 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Tom Lane t...@sss.pgh.pa.us wrote: Any change here is *not* a bug fix, it is a change of clearly documented and not-obviously-unreasonable behavior.  We have to take seriously the likelihood that it will break

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 10:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: [shrug...]  There is also real user demand for not silently breaking code that works now, which is what we risk

Re: [HACKERS] client_lc_messages

2009-10-23 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: So we'd go with a single setting to define language, which would be the current lc_messages, and two new settings, say translate_log_messages and translate_client_messages, the latter being USERSET. Does that sound reasonable? How do we get

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Simon Riggs
On Fri, 2009-10-23 at 10:04 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: Sorry, what is happen if function is marked as plan security? I was suggesting an intelligent default by which we could determine function

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Pavel Stehule
2009/10/23 Robert Haas robertmh...@gmail.com: On Fri, Oct 23, 2009 at 10:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: [shrug...]  There is also real user demand for not silently breaking

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 12:05 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2009/10/23 Robert Haas robertmh...@gmail.com: On Fri, Oct 23, 2009 at 10:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane

Re: [HACKERS] client_lc_messages

2009-10-23 Thread Magnus Hagander
2009/10/23 Alvaro Herrera alvhe...@commandprompt.com: Peter Eisentraut wrote: Note that only glibc supports switching the language at run time.  And doing it is probably very expensive if you want to do it twice per message. Ouch :-( I think you could probably get much of the use case out

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Greg Stark
On Fri, Oct 23, 2009 at 7:34 AM, Tom Lane t...@sss.pgh.pa.us wrote: The way I see it, we have two approaches:  1. Try to make the current system work by standardizing the strategy numbers for GiST somehow, and then use the default GiST operator class, if available. This proposal is a

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: I always thought it was strange that the GIST strategy numbers were completely meaningless. It does seem like assigning meaning to strategy numbers gradually as we learn new interrelated indexable strategies. We would still have a range of values for new

Re: [HACKERS] table corrupted

2009-10-23 Thread João Eugenio Marynowski
2009/10/23 Robert Haas robertmh...@gmail.com Well, that's not really the problem. Your data is corrupted - increasing the index row size is not going to fix it. I'm not really knowledgeable enough about the guts of the database to know whether there are lower-level tools that could be used

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: I'm not sure that solves the problem because the default gist operator class isn't necessarily going to be the one with the strategies this needs, Forgot to mention: I do not think default-ness of opclasses enters into it at all. The meaning of the query is

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Jeff Davis
On Fri, 2009-10-23 at 16:25 -0400, Tom Lane wrote: Forgot to mention: I do not think default-ness of opclasses enters into it at all. The meaning of the query is fully defined by the operator that is in it. All we need to know is what are the semantics of that operator. If we can find it in

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Fri, 2009-10-23 at 16:25 -0400, Tom Lane wrote: Conceivably we could get different left-of operators out of different opclasses, but if they don't behave effectively the same, the user has messed up the opclasses. It would probably be worthwhile to

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Josh Berkus
Cedric, ase is a table containing 29 GB of bytea in a database of 52 GB. Every row on the 29GB table is grab only few times. And it will just renew OS cache memory every time (the server have only 8GB of ram). So when I remove this table (not the index) from the OS cache memory, I keep

[HACKERS] Statement Level Deferred Triggers

2009-10-23 Thread Andres Freund
Hi all, I would like to hear some opinions before starting to take a stab at implementing $subject. My current use case is updating materialized views at the end of the transaction so that they appear consistent to the outside. Updating them on every row changed is far too expensive - so every

Re: [HACKERS] table corrupted

2009-10-23 Thread João Eugenio Marynowski
2009/10/23 Dimitri Fontaine dfonta...@hi-media.com João Eugenio Marynowski joa...@gmail.com writes: 2009-10-16 16:11:47 BRT 192.168.0.29 ERROR: invalid page header in block 462821 of relation ... It could be that the following will prove helpful:

Re: [HACKERS] Statement Level Deferred Triggers

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 7:29 PM, Andres Freund and...@anarazel.de wrote: Hi all, I would like to hear some opinions before starting to take a stab at implementing $subject. My current use case is updating materialized views at the end of the transaction so that they appear consistent to the

[HACKERS] misleading comments in pgbench

2009-10-23 Thread Jeff Janes
From contrib/pgbench/pgbench.c starting in revision 1.77 * Note: TPC-B requires at least 100 bytes per row, and the filler * fields in these table declarations were intended to comply with that. * But because they default to NULLs, they don't actually take any * space. We