Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> writes: >> I would suggest that *all* of those TODOs are premature in the absence >> of experimental evidence about the effect of varying the parameters. > Isn't it obvious that the "right" value is going to depend extraord

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > I would suggest that *all* of those TODOs are premature in the absence > of experimental evidence about the effect of varying the parameters. > If we end up finding out that the existing settings are about right > anyway across a range of test cases, who n

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Luke Lonergan
Hi Tom, On 4/12/07 1:40 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > I would suggest that *all* of those TODOs are premature in the absence > of experimental evidence about the effect of varying the parameters. > If we end up finding out that the existing settings are about right > anyway across a

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Tom Lane
"Luke Lonergan" <[EMAIL PROTECTED]> writes: > On 4/12/07 9:24 AM, "Bruce Momjian" <[EMAIL PROTECTED]> wrote: >> At this point I would be happy just to set the TOAST threshold to a >> value defined as optimal, rather than as the most minimal use of TOAST >> possible. > I agree that's a good startin

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Luke Lonergan
Hi Bruce, On 4/12/07 9:24 AM, "Bruce Momjian" <[EMAIL PROTECTED]> wrote: > Luke Lonergan wrote: >> Hi Bruce, >> >> How about these: >> >> - Allow specification of TOAST size threshold in bytes on a per column basis >> - Enable storage of columns in separate TOAST tables >> - Enable use of multi

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Bruce Momjian
Luke Lonergan wrote: > Hi Bruce, > > How about these: > > - Allow specification of TOAST size threshold in bytes on a per column basis > - Enable storage of columns in separate TOAST tables > - Enable use of multi-row compression method(s) for TOAST tables At this point I would be happy just to

Re: [HACKERS] TOASTing smaller things

2007-04-10 Thread Luke Lonergan
Hi Bruce, How about these: - Allow specification of TOAST size threshold in bytes on a per column basis - Enable storage of columns in separate TOAST tables - Enable use of multi-row compression method(s) for TOAST tables - Luke On 3/26/07 5:39 PM, "Bruce Momjian" <[EMAIL PROTECTED]> wrote: >

Re: [HACKERS] TOASTing smaller things

2007-03-26 Thread Bruce Momjian
Luke Lonergan wrote: > I advocate the following: > > - Enable specification of TOAST policy on a per column basis > > As a first step, then: > > - Enable vertical partitioning of tables using per-column specification of > storage policy. > How are these different from ALTER TABLE SET STORAGE?

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Luke Lonergan
Hi Hannu, On 3/22/07 3:21 PM, "Hannu Krosing" <[EMAIL PROTECTED]> wrote: > Maybe monetdb ( http://monetdb.cwi.nl/ ) can be of some inspiration ? > > btw, it may be a good idea to have a verion of bizgresMPP which has > monetdb as partition db, if monetdb is as efficient as they tell it is . Yep

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Hannu Krosing
Ühel kenal päeval, N, 2007-03-22 kell 10:19, kirjutas Luke Lonergan: > Andreas, > > On 3/22/07 9:40 AM, "Andreas Pflug" <[EMAIL PROTECTED]> wrote: > > > Wouldn't it be enough to enable having the toast table on a different > > table space? > > Yes, but the ultimate goal would allow the allocatio

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Luke Lonergan
Chris, > Hmm. Are you trying to do something sort of like CStore? > > http://db.csail.mit.edu/projects/cstore/ > > That seems to have some relevant ideas... I think "something like" is a good way to put it. As you know Cstore was a prototype for Vertica and these are in the same class as Syba

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Chris Browne
[EMAIL PROTECTED] ("Luke Lonergan") writes: > Andreas, > > On 3/22/07 9:40 AM, "Andreas Pflug" <[EMAIL PROTECTED]> wrote: > >> Wouldn't it be enough to enable having the toast table on a different >> table space? > > Yes, but the ultimate goal would allow the allocation of a storage mechanism > tha

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Luke Lonergan
Andreas, On 3/22/07 9:40 AM, "Andreas Pflug" <[EMAIL PROTECTED]> wrote: > Wouldn't it be enough to enable having the toast table on a different > table space? Yes, but the ultimate goal would allow the allocation of a storage mechanism that is unique to each column. The most frequently used mec

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Andreas Pflug
Luke Lonergan wrote: > I advocate the following: > > - Enable specification of TOAST policy on a per column basis > > As a first step, then: > > - Enable vertical partitioning of tables using per-column specification of > storage policy. > Wouldn't it be enough to enable having the toast table o

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Luke Lonergan
I advocate the following: - Enable specification of TOAST policy on a per column basis As a first step, then: - Enable vertical partitioning of tables using per-column specification of storage policy. - Luke On 3/21/07 1:12 PM, "Bruce Momjian" <[EMAIL PROTECTED]> wrote: > > Is this a TODO?

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Jim C. Nasby
On Wed, Mar 21, 2007 at 12:37:36PM -0400, Chris Browne wrote: > 4. A different mechanism would be to add a fifth storage column > strategy (the present four are PLAIN, EXTENDED, EXTERNAL, MAIN), let's > say, TOAST. > > At present, the 4 values are essentially advisory; columns get TOASTed > if th

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Chris Browne
[EMAIL PROTECTED] (Tom Lane) writes: > Chris Browne <[EMAIL PROTECTED]> writes: >> #define TOAST_DENOMINATOR 17 >>/* Use this as the divisor; current default behaviour falls from >> TOAST_DENOMINATOR = 4 */ > >> #define TOAST_TUPLE_THRESHOLD^I\ >> ^IMAXALIGN_DOWN((BLCKSZ - \ >> ^I^I^I^I MA

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Bruce Momjian
Is this a TODO? --- Jan Wieck wrote: > On 3/21/2007 2:05 PM, Tom Lane wrote: > > Chris Browne <[EMAIL PROTECTED]> writes: > >> #define TOAST_DENOMINATOR 17 > >>/* Use this as the divisor; current default behaviour fall

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Jan Wieck
On 3/21/2007 2:05 PM, Tom Lane wrote: Chris Browne <[EMAIL PROTECTED]> writes: #define TOAST_DENOMINATOR 17 /* Use this as the divisor; current default behaviour falls from TOAST_DENOMINATOR = 4 */ #define TOAST_TUPLE_THRESHOLD^I\ ^IMAXALIGN_DOWN((BLCKSZ - \ ^I^I^I^I MAXALIGN(sizeof(Pag

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Tom Lane
Chris Browne <[EMAIL PROTECTED]> writes: > #define TOAST_DENOMINATOR 17 >/* Use this as the divisor; current default behaviour falls from > TOAST_DENOMINATOR = 4 */ > #define TOAST_TUPLE_THRESHOLD^I\ > ^IMAXALIGN_DOWN((BLCKSZ - \ > ^I^I^I^I MAXALIGN(sizeof(PageHeaderData) + 3 * sizeof(Ite