Re: [HACKERS] ERROR: GIN indexes do not support whole-index scans

2010-05-21 Thread Kevin Flanagan
Ah - you mentioning index definitions has suddenly made it clearer just what that error message might mean. The source_lang_code and target_lang_code columns didn't yet each have an index. If I create an index for either one of them, the error then goes away, I'm guessing because the query processo

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

2010-05-21 Thread Florian Pflug
On May 21, 2010, at 4:20 , Florian Pflug wrote: > On May 19, 2010, at 2:15 , Florian Pflug wrote: >> On May 17, 2010, at 3:30 , Robert Haas wrote: >>> On Sun, May 16, 2010 at 9:07 PM, Florian Pflug wrote: On May 14, 2010, at 22:54 , Robert Haas wrote: > On Thu, May 13, 2010 at 5:39 PM, To

[HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
Folks, I feel dumb. I have been looking for a document which specifies what trusted and untrusted PLs must do and forbid, so far without result. Where do we document this, and if we don't where *should* we document this? Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* David Fetter (da...@fetter.org) wrote: > I have been looking for a document which specifies what trusted and > untrusted PLs must do and forbid, so far without result. I think you might have been missing the tree for the forest in this case.. :) I'm sure you've seen this, but perhaps you weren'

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Peter Geoghegan
> That's about it- a language is TRUSTED if there's no way for a user to > be able to write a function which will give them access to things > they're not supposed to have.  Practically, this includes things like > any kind of direct I/O (files, network, etc). The fact that plpythonu used to be pl

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
Peter Geoghegan writes: >> That's about it- a language is TRUSTED if there's no way for a user to >> be able to write a function which will give them access to things >> they're not supposed to have.  Practically, this includes things like >> any kind of direct I/O (files, network, etc). > The fa

Re: [HACKERS] Snapshot Materialized Views - GSoC

2010-05-21 Thread Robert Haas
2010/5/20 Pavel : > For this summer I have plan to make patch inplementing snapshot materialized > views (MV). I believe it will not be end of effort to implement more of MV. > But I / we need discuss MV syntax and exact behaviour so I have some > questions about that for all of you: > > a) relkind

Re: [HACKERS] Snapshot Materialized Views - GSoC

2010-05-21 Thread Massa, Harald Armin
Pavel, b) create MV syntax? > - CREATE MATERIALIZED VIEW mvname AS ..., I think it is quite > obvious to do so, but I had to ask > please do not fortget the: create or replace MATERIALIZED VIEW option. And also the DROP if exists for the drop-command Best wishes Harald -- GHUM Harald

Re: [HACKERS] Snapshot Materialized Views - GSoC

2010-05-21 Thread Tom Lane
"Massa, Harald Armin" writes: > please do not fortget the: > create or replace MATERIALIZED VIEW > option. Please do. For something as complex as a table or view, CREATE OR REPLACE is a lot more complicated than it is for simple objects like functions. (See flamewar just a couple weeks ago abou

[HACKERS] changed source files.

2010-05-21 Thread MMK
Hello: How does one find out what source files were modified between two releases? For example, what .c and .h changed between 8.4.3 and 8.4.4? The Notes do not mention specific source files. Thanks, MMK.

Re: [HACKERS] changed source files.

2010-05-21 Thread Stephen Frost
MMK, * MMK (bom...@yahoo.com) wrote: > How does one find out what source files were modified between two releases? > For example, what .c and .h changed between 8.4.3 and 8.4.4? > The Notes do not mention specific source files. You can pull down the CVS tags for each and then run a recursive diff

[HACKERS] unnailing shared relations (was Re: global temporary tables)

2010-05-21 Thread Robert Haas
On Sat, Apr 24, 2010 at 6:53 PM, Robert Haas wrote (in reply to Tom Lane): > If we create, e.g. pg_shared_class and > pg_shared_attribute, then we can un-nail the catalogs you just nailed > to make the authentication process able to work without selecting a > database. Actually, there's another w

Re: [HACKERS] Snapshot Materialized Views - GSoC

2010-05-21 Thread Florian Pflug
On May 21, 2010, at 15:59 , Robert Haas wrote: > 2010/5/20 Pavel : >> For this summer I have plan to make patch inplementing snapshot materialized >> views (MV). I believe it will not be end of effort to implement more of MV. >> But I / we need discuss MV syntax and exact behaviour so I have some >

Re: [HACKERS] unnailing shared relations (was Re: global temporary tables)

2010-05-21 Thread Pavel Stehule
2010/5/21 Robert Haas : > On Sat, Apr 24, 2010 at 6:53 PM, Robert Haas > wrote (in reply to Tom Lane): >> If we create, e.g. pg_shared_class and >> pg_shared_attribute, then we can un-nail the catalogs you just nailed >> to make the authentication process able to work without selecting a >> databa

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Josh Berkus
So, here's a working definition: 1) cannot directly read or write files on the server. 2) cannot bind network ports 3) uses only the SPI interface to interact with postgresql tables etc. 4) does any logging only using elog to the postgres log Questions: a) it seems like there should be some kin

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Magnus Hagander
On Fri, May 21, 2010 at 11:55 AM, Josh Berkus wrote: > So, here's a working definition: > > 1) cannot directly read or write files on the server. > 2) cannot bind network ports To make that more covering, don't yu really need something like "cannot communicate with outside processes"? -- Magnu

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Josh Berkus
On 05/21/2010 11:57 AM, Magnus Hagander wrote: On Fri, May 21, 2010 at 11:55 AM, Josh Berkus wrote: So, here's a working definition: 1) cannot directly read or write files on the server. 2) cannot bind network ports To make that more covering, don't yu really need something like "cannot comm

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 11:57:33AM -0400, Magnus Hagander wrote: > On Fri, May 21, 2010 at 11:55 AM, Josh Berkus wrote: > > So, here's a working definition: > > > > 1) cannot directly read or write files on the server. > > 2) cannot bind network ports > > To make that more covering, don't yu real

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* David Fetter (da...@fetter.org) wrote: > These need to be testable conditions, and new tests need to get added > any time we find that we've missed something. Making this concept > fuzzier is exactly the wrong direction to go. I'm really not sure that we want to be in the business of writing a

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Magnus Hagander
On Fri, May 21, 2010 at 12:22 PM, David Fetter wrote: > On Fri, May 21, 2010 at 11:57:33AM -0400, Magnus Hagander wrote: >> On Fri, May 21, 2010 at 11:55 AM, Josh Berkus wrote: >> > So, here's a working definition: >> > >> > 1) cannot directly read or write files on the server. >> > 2) cannot bin

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 12:26:24PM -0400, Stephen Frost wrote: > * David Fetter (da...@fetter.org) wrote: > > These need to be testable conditions, and new tests need to get > > added any time we find that we've missed something. Making this > > concept fuzzier is exactly the wrong direction to go

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* David Fetter (da...@fetter.org) wrote: > That is *precisely* the business we need to be in, at least for the > languages we ship, and it would behoove us to test languages we don't > ship so we can warn people when they don't pass. k, let's start with something simpler first tho- I'm sure we can

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 01:45:45PM -0400, Stephen Frost wrote: > * David Fetter (da...@fetter.org) wrote: > > That is *precisely* the business we need to be in, at least for the > > languages we ship, and it would behoove us to test languages we don't > > ship so we can warn people when they don't

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Florian Pflug
On May 21, 2010, at 18:26 , Stephen Frost wrote: > * David Fetter (da...@fetter.org) wrote: >> These need to be testable conditions, and new tests need to get added >> any time we find that we've missed something. Making this concept >> fuzzier is exactly the wrong direction to go. > > I'm really

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* David Fetter (da...@fetter.org) wrote: > On Fri, May 21, 2010 at 01:45:45PM -0400, Stephen Frost wrote: > > k, let's start with something simpler first tho- I'm sure we can pull in > > the glibc regression tests and run them too. You know, just in case > > there's a bug there, somewhere. > > Th

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
David Fetter writes: > On Fri, May 21, 2010 at 12:26:24PM -0400, Stephen Frost wrote: >> I'm really not sure that we want to be in the business of writing a >> ton of regression tests to see if languages which claim to be >> trusted really are.. > That is *precisely* the business we need to be in

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Robert Haas
On Fri, May 21, 2010 at 1:58 PM, David Fetter wrote: > On Fri, May 21, 2010 at 01:45:45PM -0400, Stephen Frost wrote: >> * David Fetter (da...@fetter.org) wrote: >> > That is *precisely* the business we need to be in, at least for the >> > languages we ship, and it would behoove us to test languag

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
Robert Haas writes: > So... can we get back to coming up with a reasonable > definition, (1) no access to system calls (including file and network I/O) (2) no access to process memory, other than variables defined within the PL. What else? regards, tom lane -- Sent vi

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Robert Haas
On Fri, May 21, 2010 at 2:21 PM, Tom Lane wrote: > Robert Haas writes: >> So... can we get back to coming up with a reasonable >> definition, > > (1) no access to system calls (including file and network I/O) > > (2) no access to process memory, other than variables defined within the > PL. > > W

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 > Well, the best way to define what a trusted language can do is to > define a *whitelist* of what it can do, not a blacklist of what it > can't do. That's the only way to get a complete definition. It's then > up to the implementation step to f

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > So... can we get back to coming up with a reasonable > definition, and Guess I'm wondering if we could steal such a definition from one of the languages we allow as trusted already.. Just a thought. I certainly think we should make sure that we doc

[HACKERS] small exclusion constraints patch

2010-05-21 Thread Jeff Davis
Currently, the check for exclusion constraints performs a sanity check that's slightly too strict -- it assumes that a tuple will conflict with itself. That is not always the case: the operator might be "<>", in which case it's perfectly valid for the search for conflicts to not find itself. This

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
"Greg Sabino Mullane" writes: >> Well, the best way to define what a trusted language can do is to >> define a *whitelist* of what it can do, not a blacklist of what it >> can't do. > No, that's exactly backwards. We can't define all the things a language > can do, but we can certainly lay out t

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
Robert Haas writes: > On Fri, May 21, 2010 at 2:21 PM, Tom Lane wrote: >> (1) no access to system calls (including file and network I/O) >> (2) no access to process memory, other than variables defined within the >> PL. >> What else? > Doesn't subvert the general PostgreSQL security mechanisms?

Re: [HACKERS] changed source files.

2010-05-21 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 >> How does one find out what source files were modified between two releases? >> For example, what .c and .h changed between 8.4.3 and 8.4.4? >> The Notes do not mention specific source files. > > You can pull down the CVS tags for each and the

Re: [HACKERS] small exclusion constraints patch

2010-05-21 Thread Tom Lane
Jeff Davis writes: > Currently, the check for exclusion constraints performs a sanity check > that's slightly too strict -- it assumes that a tuple will conflict with > itself. That is not always the case: the operator might be "<>", in > which case it's perfectly valid for the search for conflict

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 03:15:27PM -0400, Tom Lane wrote: > Robert Haas writes: > > On Fri, May 21, 2010 at 2:21 PM, Tom Lane wrote: > >> (1) no access to system calls (including file and network I/O) > >> (2) no access to process memory, other than variables defined within the > >> PL. > >> What

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 12:36:50PM -0700, David Fetter wrote: > On Fri, May 21, 2010 at 03:15:27PM -0400, Tom Lane wrote: > > Robert Haas writes: > > > On Fri, May 21, 2010 at 2:21 PM, Tom Lane wrote: > > >> (1) no access to system calls (including file and network I/O) > > >> (2) no access to pr

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Joshua Tolley
On Fri, May 21, 2010 at 1:36 PM, David Fetter wrote: > On Fri, May 21, 2010 at 03:15:27PM -0400, Tom Lane wrote: >> As long as you can't do database access except via SPI, that should >> be covered.  So I guess the next item on the list is no, or at least >> restricted, access to functions outside

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
Joshua Tolley writes: > Agreed. As long as a trusted language can do things outside the > database only by going through a database and calling some function to > which the user has rights, in an untrusted language, that seems decent > to me. A user with permissions to launch_missiles() would have

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Jonathan Leto
Howdy, On Fri, May 21, 2010 at 11:21 AM, Tom Lane wrote: > Robert Haas writes: >> So... can we get back to coming up with a reasonable >> definition, > > (1) no access to system calls (including file and network I/O) > > (2) no access to process memory, other than variables defined within the >

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Joshua Tolley
On Fri, May 21, 2010 at 2:04 PM, Tom Lane wrote: > Joshua Tolley writes: >> Agreed. As long as a trusted language can do things outside the >> database only by going through a database and calling some function to >> which the user has rights, in an untrusted language, that seems decent >> to me.

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Jan Wieck
The original idea was that a trusted language does not allow an unprivileged user to gain access to any object or data, he does not have access to without that language. This does not include data transformation functionality, like string processing or the like. As long as the user had legitim

Re: [HACKERS] unnailing shared relations (was Re: global temporary tables)

2010-05-21 Thread Robert Haas
On Fri, May 21, 2010 at 11:10 AM, Pavel Stehule wrote: > 2010/5/21 Robert Haas : >> On Sat, Apr 24, 2010 at 6:53 PM, Robert Haas >> wrote (in reply to Tom Lane): >>> If we create, e.g. pg_shared_class and >>> pg_shared_attribute, then we can un-nail the catalogs you just nailed >>> to make the au

[HACKERS] (9.1) btree_gist support for searching on "not equals"

2010-05-21 Thread Jeff Davis
This patch adds support to btree_gist for searching on <> ("not equals"). This allows an interesting use of exclusion constraints: Say you have a table: create table zoo ( cage int, animal text, exclude using gist (cage with =, animal with <>) ); That will permit you to add a

[HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages

2010-05-21 Thread Josh Berkus
From a discussion at dinner at pgcon, I wanted to send this to the list for people to poke holes in it: Problem: currently, if your database has a large amount of "cold" data, such as 350GB of 3-year-old sales transactions, in 8.4 vacuum no longer needs to touch it thanks to the visibility map

Re: [HACKERS] (9.1) btree_gist support for searching on "not equals"

2010-05-21 Thread Marko Tiikkaja
On 5/21/10 11:47 PM +0300, Jeff Davis wrote: It also allows you to enforce the constraint that only one tuple exists in a table by doing something like: create table a ( i int, exclude using gist (i with<>), unique (i) ); FWIW, this is achievable a lot more easily: CREA

[HACKERS] beta testing - planner bug - ERROR: XX000: failed to build any 2-way joins

2010-05-21 Thread Pavel Stehule
Hello there are maybe planner bug. test case: CREATE TABLE person ( id SERIAL PRIMARY KEY, name VARCHAR(64) ) WITHOUT OIDS; CREATE TABLE person_data ( id SERIAL PRIMARY KEY ) WITHOUT OIDS; SELECT name FROM person WHERE name IN (

Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages

2010-05-21 Thread Florian Pflug
On May 21, 2010, at 23:57 , Josh Berkus wrote: > From a discussion at dinner at pgcon, I wanted to send this to the list for > people to poke holes in it: > > Problem: currently, if your database has a large amount of "cold" data, such > as 350GB of 3-year-old sales transactions, in 8.4 vacuum n

[HACKERS] beta testing - pg_upgrade bug fix - double free

2010-05-21 Thread Pavel Stehule
Hello it fixes bug pg_upgrade(13359) malloc: *** error for object 0x801600: non-page-aligned, non-allocated pointer being freed *** set a breakpoint in malloc_error_break to debug arget 03:31 /usr/local/src/postgresql/contrib/pg_upgrade git diff . diff --git a/contrib/pg_upgrade/check.c b/contr