[HACKERS] Performance while loading data and indexing

2002-09-26 Thread Shridhar Daithankar
Hello all, Some time back I posted a query to build a site with 150GB of database. In last couple of weeks, lots of things were tested at my place and there are some results and again some concerns. This is a long post. Please be patient and read thr. If we win this, I guess we have a good

Re: [HACKERS] Performance while loading data and indexing

2002-09-26 Thread Shridhar Daithankar
On 26 Sep 2002 at 14:05, Shridhar Daithankar wrote: Some time back I posted a query to build a site with 150GB of database. In last couple of weeks, lots of things were tested at my place and there are some results and again some concerns. 2) Creating index takes huge amount of time.

Re: [HACKERS] Performance while loading data and indexing

2002-09-26 Thread Shridhar Daithankar
On 26 Sep 2002 at 10:51, [EMAIL PROTECTED] wrote: Hi, it seems you have to cluster it, I don't think you have another choise. Hmm.. That didn't occur to me...I guess some real time clustering like usogres would do. Unless it turns out to be a performance hog.. But this is just insert

Re: [HACKERS] Performance while loading data and indexing

2002-09-26 Thread Mario Weilguni
On 26 Sep 2002 at 14:05, Shridhar Daithankar wrote: Some time back I posted a query to build a site with 150GB of database. In last couple of weeks, lots of things were tested at my place and there are some results and again some concerns. 2) Creating index takes huge amount of time.

Re: [HACKERS] Performance while loading data and indexing

2002-09-26 Thread Shridhar Daithankar
On 26 Sep 2002 at 11:17, Mario Weilguni wrote: On 26 Sep 2002 at 14:05, Shridhar Daithankar wrote: Some time back I posted a query to build a site with 150GB of database. In last couple of weeks, lots of things were tested at my place and there are some results and again some concerns.

Re: [HACKERS] Performance while loading data and indexing

2002-09-26 Thread Mario Weilguni
Well the test runs were for 10GB of data. Schema is attached. Read in fixed fonts..Last nullable fields are dummies but may be used in fututre and varchars are not acceptable(Not my requirement). Tuple size is around 100 bytes.. The index creation query was CREATE INDEX index1 ON tablename

Re: [GENERAL] [HACKERS] Performance while loading data and indexing

2002-09-26 Thread Martijn van Oosterhout
On Thu, Sep 26, 2002 at 03:01:35PM +0530, Shridhar Daithankar wrote: Content-Description: Mail message body The index creation query was CREATE INDEX index1 ON tablename (esn,min,datetime); What if I put datetime ahead? It's likely the the datetime field will have high degree of locality

Re: [HACKERS] Performance while loading data and indexing

2002-09-26 Thread Tom Lane
Shridhar Daithankar [EMAIL PROTECTED] writes: On 26 Sep 2002 at 11:50, Mario Weilguni wrote: Just an idea, I noticed you use char(10) for esn and min, and use this as index. Are these really fixed len fields all having 10 bytes? 10 bytes. Those are id numbers.. like phone numbers always have

Re: [GENERAL] [HACKERS] Performance while loading data and indexing

2002-09-26 Thread scott.marlowe
If you are seeing very slow performance on a drive set, check dmesg to see if you're getting SCSI bus errors or something similar. If your drives aren't properly terminated then the performance will suffer a great deal. ---(end of broadcast)---