On Jul 23, 2009, at 6:22 AM, Laurent Laborde wrote:
On Wed, Jul 22, 2009 at 10:54 AM, Laurent
Labordekerdez...@gmail.com wrote:
My 1st applied patch is the safest and simpliest :
in pg_lzcompress.c :
static const PGLZ_Strategy strategy_default_data = {
256,/* Data chunks less than
On Thu, Jul 23, 2009 at 4:45 PM, Kevin
Grittnerkevin.gritt...@wicourts.gov wrote:
Laurent Laborde kerdez...@gmail.com wrote:
(iostat show a 5~25MB/s bandwidth at 100%util instead of 2~5MB/s at
100%util).
Any numbers for overall benefit at the application level?
So... now i'm not sure
I'm currently rewriting the whole toaster stuff to simply define :
- a compression threshold (size limit to compress, in Nth of page)
- an external threshold (size limit to externalize compressed data, in
Nth of page)
i keep the TOAST_INDEX_TARGET and EXTERN_TUPLES_PER_PAGE.
I expect a lot of
On Tue, Jul 28, 2009 at 2:36 PM, Laurent Labordekerdez...@gmail.com wrote:
I'm currently rewriting the whole toaster stuff to simply define :
- a compression threshold (size limit to compress, in Nth of page)
- an external threshold (size limit to externalize compressed data, in
Nth of page)
Laurent Laborde kerdez...@gmail.com wrote:
If it works, and if you're interested, i may try to write a patch
for 8.5.
Cool! I can help with it if you wish.
If you haven't already done so, be sure to read this carefully:
http://wiki.postgresql.org/wiki/Developer_FAQ
Also, be sure you
On Tue, Jul 28, 2009 at 12:44 PM, Kevin
Grittnerkevin.gritt...@wicourts.gov wrote:
Finally, you should probably consider volunteering to review a patch
or two for the next commitfest. :-) To ensure timely review of
submitted patches, while still allowing the reviewers some guarantee
of
On Wed, Jul 22, 2009 at 10:54 AM, Laurent Labordekerdez...@gmail.com wrote:
My 1st applied patch is the safest and simpliest :
in pg_lzcompress.c :
static const PGLZ_Strategy strategy_default_data = {
256, /* Data chunks less than 256 are not compressed */
256, /* force
Laurent Laborde kerdez...@gmail.com wrote:
(iostat show a 5~25MB/s bandwidth at 100%util instead of 2~5MB/s at
100%util).
Any numbers for overall benefit at the application level?
So... now i'm not sure anymore about lowering the tuple per page
om 4 to 8.
Doing that would mean more
My 1st applied patch is the safest and simpliest :
in pg_lzcompress.c :
static const PGLZ_Strategy strategy_default_data = {
256,/* Data chunks less than 256 are not compressed */
256,/* force compression on data chunks on record = 256bytes */
1, /*
Tom Lane t...@sss.pgh.pa.us wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
It seems like it might be reasonable to have a separate threshold
for compression from that for out-of-line storage. Since I've been
in that code recently, I would be pretty comfortable doing
something
I wrote:
If we want to allow a compression target external storage target,
I guess we would have to allow the smaller target to go first
Scratch that part -- even with a compression target the external
storage target, it would make sense use the same sequence of steps.
-Kevin
--
Sent
Sorry I responded that quickly this early. I keep having additional
thoughts
Kevin Grittner kevin.gritt...@wicourts.gov wrote:
Tom Lane t...@sss.pgh.pa.us wrote:
And especially, how could you have per-column targets?
Yeah, this would have to be done by table, not by column.
If we
Kevin Grittner kevin.gritt...@wicourts.gov writes:
It seems like it might be reasonable to have a separate threshold for
compression from that for out-of-line storage. Since I've been in
that code recently, I would be pretty comfortable doing something
about that; but, as is so often the
On Fri, Jul 17, 2009 at 11:10 PM, Kevin
Grittnerkevin.gritt...@wicourts.gov wrote:
Laurent Laborde kerdez...@gmail.com wrote:
What about SET STORAGE MAIN then ? To prevent out-of-line storage ?
Well, that doesn't try as hard as you might think to keep from storing
data out-of-line. It uses
Hi again !
I also take a look at another possibility to improve compression.
There is two compression strategy :
static const PGLZ_Strategy strategy_default_data = {
256,/* Data chunks less than 256 bytes are not
* compressed */
Laurent Laborde kerdez...@gmail.com wrote:
There is two compression strategy :
static const PGLZ_Strategy strategy_default_data = {
static const PGLZ_Strategy strategy_always_data = {
1) strategy_always_data seems to never be used.
A quick grep sure makes it look that way. I will
On Mon, Jul 20, 2009 at 6:04 PM, Kevin
Grittnerkevin.gritt...@wicourts.gov wrote:
What about setting PGLZ_strategy_always as the default strategy
(insane cpu cost ?) ?
Or something in-between ?
That goes beyond what I was trying to do with my recent patch. What
you propose may be useful,
Friendly greetings !
I'd like to have a higher compression ratio on our base.
From pg 8.3 documentation :
The TOAST code is triggered only when a row value to be stored in a
table is wider than TOAST_TUPLE_THRESHOLD bytes (normally 2 kB).
The TOAST code will compress and/or move field values
Laurent Laborde kerdez...@gmail.com wrote:
What about trying to change the TOAST_TUPLE_TARGET to get a higher
compression (by having more toasted record) ?
I'd like to change the TOAST_TUPLES_PER_PAGE. Maybe from 4 to 8 ?
Is that correct ? Did i missed something ?
I did some statistics
On Fri, 2009-07-17 at 12:50 -0500, Kevin Grittner wrote:
Laurent Laborde kerdez...@gmail.com wrote:
(3) Allow override of the thresholds for individual columns.
Are any of these non-controversial? What do people like there? What
did I miss?
I would skip 1 and 2 and have (3).
Joshua
Joshua D. Drake j...@commandprompt.com wrote:
On Fri, 2009-07-17 at 12:50 -0500, Kevin Grittner wrote:
(3) Allow override of the thresholds for individual columns.
I would skip 1 and 2 and have (3).
Sure, pick the one which requires new syntax! ;-)
How about two new ALTER TABLE
On Fri, 2009-07-17 at 14:21 -0500, Kevin Grittner wrote:
Joshua D. Drake j...@commandprompt.com wrote:
On Fri, 2009-07-17 at 12:50 -0500, Kevin Grittner wrote:
(3) Allow override of the thresholds for individual columns.
I would skip 1 and 2 and have (3).
Sure, pick the one
On Fri, Jul 17, 2009 at 9:21 PM, Kevin
Grittnerkevin.gritt...@wicourts.gov wrote:
Joshua D. Drake j...@commandprompt.com wrote:
On Fri, 2009-07-17 at 12:50 -0500, Kevin Grittner wrote:
(3) Allow override of the thresholds for individual columns.
I would skip 1 and 2 and have (3).
Sure,
Laurent Laborde kerdez...@gmail.com wrote:
Kevin Grittnerkevin.gritt...@wicourts.gov wrote:
How about two new ALTER TABLE actions:
ALTER [ COLUMN ] column SET COMPRESSION_THRESHOLD integer
ALTER [ COLUMN ] column SET EXTERNAL_THRESHOLD integer
Laurent, would something like this
On Fri, Jul 17, 2009 at 10:40 PM, Kevin
Grittnerkevin.gritt...@wicourts.gov wrote:
Laurent Laborde kerdez...@gmail.com wrote:
But... on which version are you planning to do that ?
The patch, if there's consensus that it's a good idea, would be for
8.5. Since it is new functionality, there
Laurent Laborde kerdez...@gmail.com wrote:
What about SET STORAGE MAIN then ? To prevent out-of-line storage ?
Well, that doesn't try as hard as you might think to keep from storing
data out-of-line. It uses the same threshold as the default EXTENDED
storage, but saves the out-of-line
26 matches
Mail list logo
