Re: [PERFORM] not exists clause

2008-01-10 Thread Guy Rouillier
S Golly wrote: I cannot get the "not exists" clause of ANSI SQL to execute correctly. select t.col11, t.col1... from table1 t where not exists (select 1 from table2 where col2 = t.col1); table1 has 40M + rows. if that matters. OS is FreeBSD 6.2, postgresql version 8.2.6 Is it not supported

Re: [PERFORM] Best way to index IP data?

2008-01-10 Thread Jonah H. Harris
On Jan 10, 2008 6:25 PM, Steve Atkins <[EMAIL PROTECTED]> wrote: > http://pgfoundry.org/projects/ip4r/ > > That has the advantage over using integers, or the built-in inet type, > of being indexable for range and overlap queries. Agreed. ip4r is da bomb. -- Jonah H. Harris, Sr. Software Archite

Re: [PERFORM] Best way to index IP data?

2008-01-10 Thread Steve Atkins
On Jan 10, 2008, at 3:14 PM, Kevin Kempter wrote: Hi List; We'll be loading a table with begining & ending I.P.'s - the table will likely have upwards of 30million rows. Any thoughts on how to get the best performance out of queries that want to look for IP ranges or the use of between

Re: [PERFORM] Best way to index IP data?

2008-01-10 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 10 Jan 2008 16:14:54 -0700 Kevin Kempter <[EMAIL PROTECTED]> wrote: > Hi List; > > We'll be loading a table with begining & ending I.P.'s - the table > will likely have upwards of 30million rows. Any thoughts on how to > get the best perform

[PERFORM] Best way to index IP data?

2008-01-10 Thread Kevin Kempter
Hi List; We'll be loading a table with begining & ending I.P.'s - the table will likely have upwards of 30million rows. Any thoughts on how to get the best performance out of queries that want to look for IP ranges or the use of between queries? Should these be modeled as integers? Thanks in

Re: [PERFORM] not exists clause

2008-01-10 Thread Josh Berkus
Golly, > I cannot get the "not exists" clause of ANSI SQL to execute correctly. > select t.col11, t.col1... from table1 t where not exists (select 1 from > table2 where col2 = t.col1); > table1 has 40M + rows. if that matters. > > OS is FreeBSD 6.2, postgresql version 8.2.6 You'll have to post th

[PERFORM] not exists clause

2008-01-10 Thread S Golly
I cannot get the "not exists" clause of ANSI SQL to execute correctly. select t.col11, t.col1... from table1 t where not exists (select 1 from table2 where col2 = t.col1); table1 has 40M + rows. if that matters. OS is FreeBSD 6.2, postgresql version 8.2.6 Is it not supported or a bug ? thank you

Re: [PERFORM] big database performance

2008-01-10 Thread Jared Mauch
On Thu, Jan 10, 2008 at 12:08:39PM +0100, Stephane Bailliez wrote: > Jared Mauch wrote: >> I do large databases in Pg, like 300GB/day of new data. > > That's impressive. Would it be possible to have details on your hardware, > schema and configuration and type of usage ? > > I'm sure there's

Re: [PERFORM] big database performance

2008-01-10 Thread Jared Mauch
On Thu, Jan 10, 2008 at 10:57:46AM +0200, Adrian Moisey wrote: > What sort of information do you need from me ? Ratio of read vs write operations (select vs insert/copy). average number of indicies per table average table size. (analyze verbose if you want to get into m

Re: [PERFORM] Search for fixed set of keywords

2008-01-10 Thread Oleg Bartunov
On Thu, 10 Jan 2008, J?rg Kiegeland wrote: Did you try integer arrays with GIN (inverted index) ? I now tried this, and GIN turned out to be linear time, compared with GIST which was acceptable time. However I tested this only for Z=infinity, for Z=1000, GIST/GIN are both not acceptable. Sor

Re: [PERFORM] big database performance

2008-01-10 Thread Stephane Bailliez
Jared Mauch wrote: I do large databases in Pg, like 300GB/day of new data. That's impressive. Would it be possible to have details on your hardware, schema and configuration and type of usage ? I'm sure there's something to learn in there for a lot of people (or at least for me) Ch

Re: [PERFORM] Search for fixed set of keywords

2008-01-10 Thread Jörg Kiegeland
Did you try integer arrays with GIN (inverted index) ? I now tried this, and GIN turned out to be linear time, compared with GIST which was acceptable time. However I tested this only for Z=infinity, for Z=1000, GIST/GIN are both not acceptable. ---(end of broadcast)--

Re: [PERFORM] big database performance

2008-01-10 Thread Adrian Moisey
Hi I do large databases in Pg, like 300GB/day of new data. Need a lot more data on what you're having issues with. That is big! What sort of information do you need from me ? Is your problem with performance database reads? writes? (insert/copy?) How many indicies do you have?

Re: [PERFORM] big database performance

2008-01-10 Thread Adrian Moisey
Hi Also, we're running the db on ext3 with noatime. Should I look at changing or getting rid of journaling ? No (unless you like really long fsck times). data=writeback is safe with PostgreSQL, though. I tested that on a dev box, and I didn't notice a difference when using pgbench -- A