Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Josh Berkus
Simon, Use Case: VLDB with tons of (now) read only data, some not. Data needs to be accessible, but data itself is rarely touched, allowing storage costs to be minimised via a storage hierarchy of progressively cheaper storage. There's actually 2 cases to optimize for: 1)

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Simon Riggs
On Tue, 2007-12-11 at 20:30 -0800, Josh Berkus wrote: Simon, Use Case: VLDB with tons of (now) read only data, some not. Data needs to be accessible, but data itself is rarely touched, allowing storage costs to be minimised via a storage hierarchy of progressively cheaper storage.

Re: [HACKERS] [EMAIL PROTECTED]: Re: [pgsql-es-ayuda] SLL error 100% cpu]

2007-12-12 Thread Magnus Hagander
On Wed, Dec 12, 2007 at 12:30:50AM -0500, Tom Lane wrote: Trevor Talbot [EMAIL PROTECTED] writes: On 12/11/07, Tom Lane [EMAIL PROTECTED] wrote: I dunno anything about how to fix the real problem (what's winsock error 10004?), WSAEINTR, A blocking operation was interrupted by a call to

Re: [HACKERS] VLDB Features

2007-12-12 Thread Dimitri Fontaine
Hi, Le mercredi 12 décembre 2007, Josh Berkus a écrit : I'm curious what you feel is missing that pgloader doesn't fill that requirement: http://pgfoundry.org/projects/pgloader/ Because pgloader is implemented in middleware, it carries a very high overhead if you have bad rows. As little

[HACKERS] little correction

2007-12-12 Thread Oleg Bartunov
Bruce, in http://www.postgresql.org/docs/8.3/static/textsearch-dictionaries.html#TEXTSEARCH-THESAURUS I think ALTER TEXT SEARCH CONFIGURATION russian ADD MAPPING FOR asciiword, asciihword, hword_asciipart WITH thesaurus_simple; should be ALTER TEXT SEARCH CONFIGURATION russian

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Zeugswetter Andreas ADI SD
There are a number of nasty limitations for partitions currently (not the least of which is that real uniqueness guarantees are impractical), Just to add an other opinion to this statement, because it imho sounds overly pessimistic: Uniqueness is currently perfectly practical, when the

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Zeugswetter Andreas ADI SD
Getting partitioning/read-only right will allow 70+TB of that to be on tape or similar, which with compression can be reduced to maybe 20TB? I don't want to promise any particular compression ratio, but it will make a substantial difference, as I'm sure you realise. Wouldn't one very

[HACKERS] ScalarArrayOp advancing array keys

2007-12-12 Thread Gregory Stark
Consider a situation where there's an index on x,y and we're processing a where clause like: WHERE x IN (1,2,3,4,5) AND y IN ('A','B','C','D') Assuming we use the index we loop through doing an index lookup for every combination of the two (generated) arrays. Except if I understand

[HACKERS] foreign keys

2007-12-12 Thread Sam Mason
Hi, How hard/generally useful would it be to allow the target of a foreign key to be on a set of columns where only a subset of them actually have a unique constraint. For example: CREATE TABLE base ( id INTEGER NOT NULL PRIMARY KEY, type INTEGER NOT NULL ); CREATE TABLE

Re: [HACKERS] VLDB Features

2007-12-12 Thread Simon Riggs
On Tue, 2007-12-11 at 15:31 -0800, Josh Berkus wrote: Simon, we should start a VLDB-Postgres developer wiki page. http://developer.postgresql.org/index.php/DataWarehousing -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Andrew Sullivan
On Wed, Dec 12, 2007 at 12:14:43PM +0100, Zeugswetter Andreas ADI SD wrote: Uniqueness is currently perfectly practical, when the unique index contains the column[s] that is/are used in a non overlapping partitioning scheme. Well, yes, assuming you have no bugs. Part of the reason I want the

[HACKERS] result of convert_to is bytea

2007-12-12 Thread Pavel Stehule
Hello documentation fix. result of convert_to is bytea, not text. Regards Pavel Stehule *** ./src/sgml/func.sgml.orig 2007-12-12 17:18:55.0 +0100 --- ./src/sgml/func.sgml 2007-12-12 17:19:56.0 +0100 *** *** 1386,1392

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Andrew Sullivan
On Wed, Dec 12, 2007 at 12:58:11PM +0100, Zeugswetter Andreas ADI SD wrote: Wouldn't one very substantial requirement of such storage be to have it independent of db version, or even db product? Keeping old hardware and software around can be quite expensive. This was one of the explicit

[HACKERS] test

2007-12-12 Thread Joshua D. Drake
test ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [HACKERS] VLDB Features

2007-12-12 Thread Markus Schiltknecht
Hi, Josh Berkus wrote: Here's the other VLDB features we're missing: Parallel Query Uh.. this only makes sense in a distributed database, no? I've thought about parallel querying on top of Postgres-R. Does it make sense implementing some form of parallel querying apart from the

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Simon Riggs
On Wed, 2007-12-12 at 11:22 -0500, Andrew Sullivan wrote: On Wed, Dec 12, 2007 at 12:14:43PM +0100, Zeugswetter Andreas ADI SD wrote: Uniqueness is currently perfectly practical, when the unique index contains the column[s] that is/are used in a non overlapping partitioning scheme. Well,

Re: [HACKERS] VLDB Features

2007-12-12 Thread Josh Berkus
Markus, Parallel Query Uh.. this only makes sense in a distributed database, no? I've thought about parallel querying on top of Postgres-R. Does it make sense implementing some form of parallel querying apart from the distribution or replication engine? Sure. Imagine you have a 5TB

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Simon Riggs
On Wed, 2007-12-12 at 10:48 -0800, Josh Berkus wrote: Andrew, The requirement was, anyway, that we be able to read old versions of archived rows. IIRC there was an implementation choice, whether we would _never_ allow such rows to be SET READ WRITE or whether they'd be immediately

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Josh Berkus
Andrew, The requirement was, anyway, that we be able to read old versions of archived rows. IIRC there was an implementation choice, whether we would _never_ allow such rows to be SET READ WRITE or whether they'd be immediately upgraded to the present format on SET READ WRITE. Well, in

Re: [HACKERS] VLDB Features

2007-12-12 Thread Markus Schiltknecht
Hi Josh, Josh Berkus wrote: Sure. Imagine you have a 5TB database on a machine with 8 cores and only one concurrent user. You'd like to have 1 core doing I/O, and say 4-5 cores dividing the scan and join processing into 4-5 chunks. Ah, right, thank for enlightenment. Heck, I'm definitely

Re: [HACKERS] test

2007-12-12 Thread Alvaro Herrera
Joshua D. Drake wrote: test Does anybody see any value in having [EMAIL PROTECTED] be an alias for pgsql-hackers? -- Alvaro Herrera http://www.flickr.com/photos/alvherre/ Postgres is bloatware by design: it was built to house PhD theses. (Joey Hellerstein, SIGMOD

Re: [HACKERS] test

2007-12-12 Thread Andrew Hammond
On Dec 12, 2007 11:37 AM, Alvaro Herrera [EMAIL PROTECTED] wrote: Joshua D. Drake wrote: test Does anybody see any value in having [EMAIL PROTECTED] be an alias for pgsql-hackers? No, but I see some mild irritation in having to modify my rules to tag a second address with the

[HACKERS] Trigger problem - conclusion

2007-12-12 Thread Nikolay Grebnev
Good Day, I recently posted a message here ( http://archives.postgresql.org/pgsql-hackers/2007-12/msg00340.php) that the trigger does not work as it should from time to time. Now the trigger works on C, before It was on TCL and it had the same problem. As the trigger works all right in 99.999 %

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 12 Dec 2007 19:07:57 + Simon Riggs [EMAIL PROTECTED] wrote: I seem to be the only one saying global indexes are bad, so if people that want them can do the math and honestly say they want them, then I will listen. global indexes are

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Josh Berkus
Simon, Who was working on it? Zdenec and Dhanaraj from Sun, and someone from EDB (I'm not sure who, maybe Korry?). Unfortunately, both companies have shifted staff around and we need to re-start work. Of course, if hackers other than those from EDB Sun want to attack the problem, the more

[HACKERS] PGparam proposal v2

2007-12-12 Thread Andrew Chernow
Here is our updated PGparam extension to the libpq api: http://archives.postgresql.org/pgsql-hackers/2007-12/msg00356.php We have a patch implementing the following which we are cleaning up. We are also kicking around some ideas for arrays and possibly composite types which we may consider if

Re: [HACKERS] VLDB Features

2007-12-12 Thread Gavin Sherry
On Wed, Dec 12, 2007 at 08:26:16PM +0100, Markus Schiltknecht wrote: Isn't Gavin Sherry working on this? Haven't read anything from him lately... Me neither. Swallowed by Greenplum and France. Hm.. good for him, I guess! Yes, I'm around -- just extremely busy with a big release at

Re: [HACKERS] Trigger problem - conclusion

2007-12-12 Thread Andrew Dunstan
Nikolay Grebnev wrote: Good Day, I recently posted a message here (http://archives.postgresql.org/pgsql-hackers/2007-12/msg00340.php) that the trigger does not work as it should from time to time. Now the trigger works on C, before It was on TCL and it had the same problem. As the trigger

Re: [HACKERS] VLDB Features

2007-12-12 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Greenplum as well as other Real Life stuff. For those of us here who have no idea what you are talking about can you define what Real Life is like? Joshua D. Drake - -- The PostgreSQL Company: Since 1997, http://www.commandprompt.com/

Re: [HACKERS] Current code for function j2date does not have the same correct dynamic range as older code.

2007-12-12 Thread Dann Corbit
From: Dann Corbit Sent: Tuesday, December 11, 2007 5:58 PM To: pgsql-hackers@postgresql.org Subject: Current code for function j2date does not have the same correct dynamic range as older code. It may not matter to the PostgreSQL group, since nothing goes wrong until the year is 1,465,002 or

[HACKERS] Recreating archive_status

2007-12-12 Thread Simon Riggs
During recovery procedures, there is a step that says If you didn't archive pg_xlog/ at all, then recreate it, and be sure to recreate the subdirectory pg_xlog/archive_status/ as well. If you forget to do this, you may not realise until the recovering server comes up and tries writing to the

Re: [HACKERS] Recreating archive_status

2007-12-12 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 12 Dec 2007 22:28:58 + Simon Riggs [EMAIL PROTECTED] wrote: We could check this just as the server comes up and then re-create it if necessary. So we have one less step in the process to remember. Existing scripts which perform this

Re: [HACKERS] [DOCS] distributed checkpoint

2007-12-12 Thread Josh Berkus
All, Just FYI, it's going to be difficult to replace the name of the feature in the PR docs at this point; I already have 11 translations. What's *wrong* with Load Distributed Checkpoint, which is what we've been calling it for 6 months? -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco

Re: [HACKERS] Recreating archive_status

2007-12-12 Thread Peter Eisentraut
Simon Riggs wrote: We could check this just as the server comes up and then re-create it if necessary. So we have one less step in the process to remember. Existing scripts which perform this automatically will not need changing. Oh please yes -- Peter Eisentraut

Re: [HACKERS] WORM and Read Only Tables (v0.1)

2007-12-12 Thread Andrew Sullivan
On Wed, Dec 12, 2007 at 07:07:57PM +, Simon Riggs wrote: Enforcing uniqueness with a global index has a number of disadvantages. This is why I was trying to talk about constraints rather than global indexes. Just because we happen to implement them that way today does not mean that such

Re: [HACKERS] VLDB Features

2007-12-12 Thread Gregory Stark
Josh Berkus [EMAIL PROTECTED] writes: Markus, Parallel Query Uh.. this only makes sense in a distributed database, no? I've thought about parallel querying on top of Postgres-R. Does it make sense implementing some form of parallel querying apart from the distribution or replication

Re: [HACKERS] [DOCS] distributed checkpoint

2007-12-12 Thread Bruce Momjian
Josh Berkus wrote: All, Just FYI, it's going to be difficult to replace the name of the feature in the PR docs at this point; I already have 11 translations. What's *wrong* with Load Distributed Checkpoint, which is what we've been calling it for 6 months? There was nothing *wrong*

Re: [HACKERS] [DOCS] distributed checkpoint

2007-12-12 Thread Greg Smith
On Wed, 12 Dec 2007, Josh Berkus wrote: What's *wrong* with Load Distributed Checkpoint, which is what we've been calling it for 6 months? One issue was that distributed has some association with distributed computing, which isn't actually the case. Spread is also more descriptive of what

Re: [HACKERS] little correction

2007-12-12 Thread Tom Lane
Oleg Bartunov [EMAIL PROTECTED] writes: in http://www.postgresql.org/docs/8.3/static/textsearch-dictionaries.html#TEXTSEARCH-THESAURUS I think ALTER TEXT SEARCH CONFIGURATION russian ADD MAPPING FOR asciiword, asciihword, hword_asciipart WITH thesaurus_simple; should be ALTER TEXT

Re: [HACKERS] [DOCS] distributed checkpoint

2007-12-12 Thread Magnus Hagander
Just FYI, it's going to be difficult to replace the name of the feature in the PR docs at this point; I already have 11 translations. What's *wrong* with Load Distributed Checkpoint, which is what we've been calling it for 6 months? Are you saying the PR was 'string freezed' before rc1?