[HACKERS] [feature] cached index to speed up specific queries on extremely large data sets

2014-04-11 Thread lkcl .
hi folks, please cc me direct on responses as i am subscribed on digest. i've been asked to look at how to deal with around 7 billion records (appx 30 columns, appx data size total 1k) and this might have to be in a single system (i will need to Have Words with the client about that). the data

Re: [HACKERS] [feature] cached index to speed up specific queries on extremely large data sets

2014-04-11 Thread Heikki Linnakangas
On 04/11/2014 03:20 PM, lkcl . wrote: so i had an idea. there already exists the concept of indexes. there already exists the concept of cached queries. question: would it be practical to*merge* those two concepts such that specific queries could be*updated* as new records are added, such

Re: [HACKERS] [feature] cached index to speed up specific queries on extremely large data sets

2014-04-11 Thread lkcl .
On Fri, Apr 11, 2014 at 1:26 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 04/11/2014 03:20 PM, lkcl . wrote: so i had an idea. there already exists the concept of indexes. there already exists the concept of cached queries. question: would it be practical to*merge* those two

Re: [HACKERS] [feature] cached index to speed up specific queries on extremely large data sets

2014-04-11 Thread Michael Paquier
On Fri, Apr 11, 2014 at 9:53 PM, lkcl . luke.leigh...@gmail.com wrote: On Fri, Apr 11, 2014 at 1:26 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 04/11/2014 03:20 PM, lkcl . wrote: so i had an idea. there already exists the concept of indexes. there already exists the concept of

Re: [HACKERS] [feature] cached index to speed up specific queries on extremely large data sets

2014-04-11 Thread lkcl .
On Fri, Apr 11, 2014 at 2:12 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Apr 11, 2014 at 9:53 PM, lkcl . luke.leigh...@gmail.com wrote: section: https://wiki.postgresql.org/wiki/Materialized_Views. When updating a materialized view, or refreshing it, you need as well to be