Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Stefan Moeding
Hi! Josh Berkus writes: > Now, if we had an OS which could be convinced to handle caching > differently for different physical devices, then I could see wanting > this setting to be per-tablespace. For example, it would make a lot of > sense not to FS-cache any data which is on a ramdisk or supe

[HACKERS] misleading comments in pgbench

2009-10-23 Thread Jeff Janes
>From contrib/pgbench/pgbench.c starting in revision 1.77 * Note: TPC-B requires at least 100 bytes per row, and the "filler" * fields in these table declarations were intended to comply with that. * But because they default to NULLs, they don't actually take any * space. We c

Re: [HACKERS] Statement Level Deferred Triggers

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 7:29 PM, Andres Freund wrote: > Hi all, > > I would like to hear some opinions before starting to take a stab at > implementing $subject. > My current use case is updating materialized views at the end of the > transaction so that they appear consistent to the outside. > Up

Re: [HACKERS] table corrupted

2009-10-23 Thread João Eugenio Marynowski
2009/10/23 Dimitri Fontaine > João Eugenio Marynowski writes: > > 2009-10-16 16:11:47 BRT 192.168.0.29 ERROR: invalid page header in > > block 462821 of relation "..." > > It could be that the following will prove helpful: > http://archives.postgresql.org/pgsql-general/2007-07/msg00506.php >

[HACKERS] Statement Level Deferred Triggers

2009-10-23 Thread Andres Freund
Hi all, I would like to hear some opinions before starting to take a stab at implementing $subject. My current use case is updating materialized views at the end of the transaction so that they appear consistent to the outside. Updating them on every row changed is far too expensive - so every c

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Josh Berkus
Cedric, > ase is a table containing 29 GB of bytea in a database of 52 GB. Every row > on the 29GB table is grab only few times. And it will just renew OS cache > memory every time (the server have only 8GB of ram). > So when I remove this table (not the index) from the OS cache memory, I keep

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Tom Lane
Jeff Davis writes: > On Fri, 2009-10-23 at 16:25 -0400, Tom Lane wrote: >> Conceivably we could get different left-of >> operators out of different opclasses, but if they don't behave >> effectively the same, the user has messed up the opclasses. > It would probably be worthwhile to make an attem

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Jeff Davis
On Fri, 2009-10-23 at 16:25 -0400, Tom Lane wrote: > Forgot to mention: I do not think default-ness of opclasses enters > into it at all. The meaning of the query is fully defined by the > operator that is in it. All we need to know is what are the > semantics of that operator. If we can find it

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Tom Lane
Greg Stark writes: > I'm not sure that solves the problem because the "default" gist > operator class isn't necessarily going to be the one with the > strategies this needs, Forgot to mention: I do not think default-ness of opclasses enters into it at all. The meaning of the query is fully defin

Re: [HACKERS] table corrupted

2009-10-23 Thread João Eugenio Marynowski
2009/10/23 Robert Haas > Well, that's not really the problem. Your data is corrupted - > increasing the index row size is not going to fix it. > > I'm not really knowledgeable enough about the guts of the database to > know whether there are lower-level tools that could be used to rescue > your

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Tom Lane
Greg Stark writes: > I always thought it was strange that the GIST strategy numbers were > completely meaningless. It does seem like assigning meaning to > strategy numbers gradually as we learn new interrelated indexable > strategies. We would still have a range of values for new non-standard > s

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Greg Stark
On Fri, Oct 23, 2009 at 7:34 AM, Tom Lane wrote: >> The way I see it, we have two approaches: >>  1. Try to make the current system work by standardizing the strategy >> numbers for GiST somehow, and then use the default GiST operator class, >> if available. > > This proposal is a non-starter, bec

Re: [HACKERS] client_lc_messages

2009-10-23 Thread Magnus Hagander
2009/10/23 Alvaro Herrera : > Peter Eisentraut wrote: > >> Note that only glibc supports switching the language at run time.  And >> doing it is probably very expensive if you want to do it twice per >> message. > > Ouch :-( > >> I think you could probably get much of the use case out of this if yo

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 12:05 PM, Pavel Stehule wrote: > 2009/10/23 Robert Haas : >> On Fri, Oct 23, 2009 at 10:50 AM, Tom Lane wrote: >>> Robert Haas writes: On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane wrote: > [shrug...]  There is also real user demand for not silently breaking > c

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Pavel Stehule
2009/10/23 Robert Haas : > On Fri, Oct 23, 2009 at 10:50 AM, Tom Lane wrote: >> Robert Haas writes: >>> On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane wrote: [shrug...]  There is also real user demand for not silently breaking code that works now, which is what we risk anytime we change the

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Simon Riggs
On Fri, 2009-10-23 at 10:04 -0400, Tom Lane wrote: > Simon Riggs writes: > > On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: > >> Sorry, what is happen if function is marked as "plan security"? > > > I was suggesting an intelligent default by which we could determine > > function marking i

Re: [HACKERS] client_lc_messages

2009-10-23 Thread Tom Lane
Alvaro Herrera writes: > So we'd go with a single setting to define language, which would be the > current lc_messages, and two new settings, say translate_log_messages > and translate_client_messages, the latter being USERSET. > Does that sound reasonable? How do we get to the point where indiv

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 10:50 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane wrote: >>> [shrug...]  There is also real user demand for not silently breaking >>> code that works now, which is what we risk anytime we change the set >>> of statements that can

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Merlin Moncure
On Fri, Oct 23, 2009 at 11:07 AM, Kevin Grittner wrote: > Tom Lane wrote: > >> Any change here is *not* a bug fix, it is a change of clearly >> documented and not-obviously-unreasonable behavior.  We have to take >> seriously the likelihood that it will break existing code. > > Perhaps plpgsql co

Re: [HACKERS] client_lc_messages

2009-10-23 Thread Alvaro Herrera
Peter Eisentraut wrote: > Note that only glibc supports switching the language at run time. And > doing it is probably very expensive if you want to do it twice per > message. Ouch :-( > I think you could probably get much of the use case out of this if you > concentrate on making two switches

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Kevin Grittner
David Fetter wrote: > One of the things the security community has learned is that the > only way it's even possible to get an information leak rate of zero > is to have a system which does nothing at all. It's a fact we need > to bear in mind when addressing this or any other issue of access >

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Kevin Grittner
Tom Lane wrote: > Any change here is *not* a bug fix, it is a change of clearly > documented and not-obviously-unreasonable behavior. We have to take > seriously the likelihood that it will break existing code. Perhaps plpgsql could support tests of SQLSTATE, and recognize '02000' (the standa

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Tom Lane
Robert Haas writes: > On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane wrote: >> [shrug...]  There is also real user demand for not silently breaking >> code that works now, which is what we risk anytime we change the set >> of statements that can set FOUND. > We've had this discussion before and I'm s

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Heikki Linnakangas
Tom Lane wrote: > The thought that's been in the back of my mind is that you could solve > 99% of the performance problem if you trusted all builtin functions and > nothing else. This avoids the question of who gets to mark functions > as trustable. Except that all builtin functions are not trust

Re: [HACKERS] pre-proposal: type interfaces

2009-10-23 Thread Tom Lane
Jeff Davis writes: > For instance, one feature that I'm considering now is a "temporal join" > which is a join on "overlaps" rather than "equals", e.g.: > SELECT * FROM a, b WHERE a.x && b.x; > I might try to provide a modified merge join algorithm to implement that > more efficiently in some

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Andrew Dunstan
Robert Haas wrote: On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane wrote: Dimitri Fontaine writes: I'll go search for more, meantime I'll just add the main goal of this new thread is to have -hackers know there is a real user demand for having EXECUTE set FOUND the same way it sets GET DI

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread David Fetter
On Fri, Oct 23, 2009 at 10:04:29AM -0400, Tom Lane wrote: > Simon Riggs writes: > > On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: > >> Sorry, what is happen if function is marked as "plan security"? > > > I was suggesting an intelligent default by which we could > > determine function ma

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 9:52 AM, Tom Lane wrote: > Dimitri Fontaine writes: >> I'll go search for more, meantime I'll just add the main goal of this >> new thread is to have -hackers know there is a real user demand for >> having EXECUTE set FOUND the same way it sets GET DIAGNOSTIC. > > [shrug..

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Tom Lane
Simon Riggs writes: > On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: >> Sorry, what is happen if function is marked as "plan security"? > I was suggesting an intelligent default by which we could determine > function marking implicitly, if it was not explicitly stated on the > CREATE FUNC

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Tom Lane
Dimitri Fontaine writes: > I'll go search for more, meantime I'll just add the main goal of this > new thread is to have -hackers know there is a real user demand for > having EXECUTE set FOUND the same way it sets GET DIAGNOSTIC. [shrug...] There is also real user demand for not silently breaki

Re: [HACKERS] EvalPlanQual seems a tad broken

2009-10-23 Thread Tom Lane
I wrote: > On further review it seems that a better way to do this is to make > things happen inside the EPQ machinery. We need to "freeze" the rows > returned by *all* scan nodes, not only the ones referencing real tables > --- for example, a join against a VALUES scan node would still be a > pro

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Cédric Villemain
Le vendredi 23 octobre 2009 14:23:09, Robert Haas a écrit : > On Fri, Oct 23, 2009 at 5:23 AM, Cédric Villemain > > wrote: > > Le vendredi 23 octobre 2009 01:08:15, Joshua D. Drake a écrit : > >> On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: > >> > On Thu, Oct 22, 2009 at 11:16 AM, Cédric

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 5:23 AM, Cédric Villemain wrote: > Le vendredi 23 octobre 2009 01:08:15, Joshua D. Drake a écrit : >> On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: >> > On Thu, Oct 22, 2009 at 11:16 AM, Cédric Villemain >> > >> > wrote: >> > > Le lundi 19 octobre 2009 23:27:20, Gr

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Richard Huxton
Heikki Linnakangas wrote: > The most useful "automatic" annotation I can see is to treat functions > implementing B-tree operators as safe. I *think* that's safe, anyway. Index lookups and single-type comparisons were the only things I could come up with as safe. Unless there is some way to genera

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Heikki Linnakangas
Rod Taylor wrote: > This still allow many optimizations to be applied in complex cases. The > planner > > CREATE VIEW phone_number AS > SELECT person, phone, company > FROM phone_data USING SECURITY FILTER(phone NOT LIKE '6%') >JOIN person USING (person_id) >JOIN company USING (co

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Heikki Linnakangas
Simon Riggs wrote: > Also, we should presume that any function created with SECURITY DEFINER > and created by a superuser would have plan security, so we don't need to > annotate lots of old code to work securely. Annotating the built-in > functions is a lot easier. SECURITY DEFINER is an orthogon

Re: [HACKERS] SE-PgSQL developer documentation (Re: Reworks for Access Control facilities (r2363))

2009-10-23 Thread KaiGai Kohei
KaiGai Kohei wrote: > After the long trial-and-errors, we learned a few approaches which > use common entry points for both of DAC and MAC were rocky-path more > than what we initially imagined. > So, we came back to the original design. It deploys MAC hooks on > the strategic points of core routin

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread KaiGai Kohei
Simon Riggs wrote: > On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: >>> Also, we should presume that any function created with SECURITY DEFINER >>> and created by a superuser would have plan security, so we don't need to >>> annotate lots of old code to work securely. Annotating the built-i

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Simon Riggs
On Fri, 2009-10-23 at 19:38 +0900, KaiGai Kohei wrote: > > Also, we should presume that any function created with SECURITY DEFINER > > and created by a superuser would have plan security, so we don't need to > > annotate lots of old code to work securely. Annotating the built-in > > functions is a

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread KaiGai Kohei
Simon Riggs wrote: > On Fri, 2009-10-23 at 11:30 +0300, Heikki Linnakangas wrote: > >> The most user-friendly and backwards-compatible (though not necessarily >> back-patchable) approach I can see is: >> >> 1. If the user has read access to all the underlying tables, plan it >> like we do today. >

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Cédric Villemain
Le vendredi 23 octobre 2009 01:08:15, Joshua D. Drake a écrit : > On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: > > On Thu, Oct 22, 2009 at 11:16 AM, Cédric Villemain > > > > wrote: > > > Le lundi 19 octobre 2009 23:27:20, Greg Stark a écrit : > > >> On Mon, Oct 19, 2009 at 2:08 PM, marcin

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Pavel Stehule
2009/10/23 Simon Riggs : > On Fri, 2009-10-23 at 11:30 +0300, Heikki Linnakangas wrote: > >> The most user-friendly and backwards-compatible (though not necessarily >> back-patchable) approach I can see is: >> >> 1. If the user has read access to all the underlying tables, plan it >> like we do tod

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Simon Riggs
On Fri, 2009-10-23 at 11:30 +0300, Heikki Linnakangas wrote: > The most user-friendly and backwards-compatible (though not necessarily > back-patchable) approach I can see is: > > 1. If the user has read access to all the underlying tables, plan it > like we do today. For me, it would make most

Re: [HACKERS] Using views for row-level access control is leaky

2009-10-23 Thread Heikki Linnakangas
Marc Munro wrote: > Here is a typical veil secured view definition: > > create view parties as > SELECT party_id, client_id, party_type_id, username, party_name > FROM parties.parties > WHERE api.user_has_client_or_personal_privilege(client_id, > par

Re: [HACKERS] table corrupted

2009-10-23 Thread Dimitri Fontaine
João Eugenio Marynowski writes: > 2009-10-16 16:11:47 BRT 192.168.0.29 ERROR:  invalid page header in > block 462821 of relation "..." It could be that the following will prove helpful: http://archives.postgresql.org/pgsql-general/2007-07/msg00506.php http://pgsql.tapoueh.org/site/html/news/2

Re: [HACKERS] Controlling changes in plpgsql variable resolution

2009-10-23 Thread Peter Eisentraut
On Tue, 2009-10-20 at 10:32 -0400, Tom Lane wrote: > That's only sane if you are 100% certain that there could not be a > security issue arising from the change of behavior. Otherwise someone > could for instance subvert a security-definer function by running it > under the setting it wasn't writt

Re: [HACKERS] plpgsql EXECUTE will not set FOUND

2009-10-23 Thread Dimitri Fontaine
Tom Lane writes: > Dimitri Fontaine writes: >> But it will set GET DIAGNOSTIC ... = ROW_COUNT, am I being told on IRC. > > This has been discussed before, please read archives. The thread I found is pretty content free as far as answering my question goes. http://archives.postgresql.org/pgsql