Re: [GENERAL] optimizing daily data storage in Pg

2010-07-23 Thread Andy Colson
On 7/23/2010 12:39 AM, P Kishor wrote: On Thu, Jul 22, 2010 at 4:56 PM, Andy Colson wrote: On 7/22/2010 9:41 AM, P Kishor wrote: I have been struggling with this for a while now, have even gone down a few paths but struck out, so I turn now to the community for ideas. First, the problem: Stor

Re: [GENERAL] optimizing daily data storage in Pg

2010-07-22 Thread Scott Marlowe
Does ordering the data in the table by day, cell_id help at all? How big ARE the indexes we're talking about? If the total size of all the indexes you need to do your queries run into the hundreds of gigabytes, going from 12 to 32 Gigs of RAM may be like flapping your arms our the window of your c

Re: [GENERAL] optimizing daily data storage in Pg

2010-07-22 Thread P Kishor
On Thu, Jul 22, 2010 at 4:56 PM, Andy Colson wrote: > On 7/22/2010 9:41 AM, P Kishor wrote: >> >> I have been struggling with this for a while now, have even gone down >> a few paths but struck out, so I turn now to the community for ideas. >> First, the problem: Store six daily variables for ~ 25

Re: [GENERAL] optimizing daily data storage in Pg

2010-07-22 Thread Andy Colson
On 7/22/2010 9:41 AM, P Kishor wrote: I have been struggling with this for a while now, have even gone down a few paths but struck out, so I turn now to the community for ideas. First, the problem: Store six daily variables for ~ 25 years for cells in a grid. * Number of vars = 6 * Number of

[GENERAL] optimizing daily data storage in Pg

2010-07-22 Thread P Kishor
I have been struggling with this for a while now, have even gone down a few paths but struck out, so I turn now to the community for ideas. First, the problem: Store six daily variables for ~ 25 years for cells in a grid. * Number of vars = 6 * Number of cells ~ 13 million * Number of days ~ 91