This was a follow up to a problem I raised to another proposal. I thought
the context was clear, bu apparently not... So here's the context.
Data compression has a huge effect on performance. Reducing the physical
size of the data reduces database size and the number of pages to be read
to fetch a given number of records. Given that cpu cycles are nearly free
but disk performance is close to stable (disregarding SSDs, which is a
different topic), the easiest way to double performance is to half the
number of pages that need to be read.
Unfortunately, data compression is one of those areas where the cost of
compression has a non-linear relationship with density while decompression
is linear. Consequently there are huge tradeoffs between density and
computational cost that are directly related to the compression algorithm
and the expected ratio of reads to writes.
In other words, Adriano, compression a a big effect on performance but "one
size does not fit all." The run length encoding I designed 30 years is
pretty good are compression binary zeros and trailing blanks but very
little else. There are far better compression algoritms but they come at a
much greater cost.
Providing user control over choice of compression algorithm on a table by
table and application by application basis would give a knowledgeable user
control over a range if tradeoffs where there cannot be a general purpose
best.
Personally, I'd go for value based encoding and leave it with that, but you
guys like your technology to age for a decade before you will call it your
own, so here is a way to muddle while the clock runs out on value based
encoding.
On Saturday, February 28, 2015, Adriano dos Santos Fernandes <
adrian...@gmail.com> wrote:
> What problem are you trying to solve?
>
> What's the benefit? Are you trying to transfer inability of the product
> designers to the user? How he wisely can chose that?
>
> Can you start from the requirements instead of the design?
>
>
> Adriano
>
>
> On 28-02-2015 22:27, Jim Starkey wrote:
> > Sorry to continue to meddle, but once ideas start flowing, it's hard to
> stop them.
> >
> > If anyone wanted to implement user defined record encodings, here is one
> way to do it:
> >
> > 1. Add a clause to "create table" to specify an encoding name
> > 2. Add a parameter to the configuration files to map encoding name
> to loadable library name
> > 3. Extend RDB$FORMATS to include an encoding name
> > 4. Extend RDB$RELATIONS accordingly
> > 5. Define a encoding/decoding API
> > 6. Etc.
> >
> > Since the encoding name would be registered in RDB$FORMATS, a change in
> encoding would be upwards compatible -- existing records would retain the
> scheme in effect when they were last updated, but new records would be
> encoded with whatever was in effect.
> >
> > It would require a minor ODS for an initial implementation (major unless
> you guys take the time to allow extensions to system tables), but
> thereafter encoding schemes could be added at will for either experimental
> purposes or for table specific special schema.
> >
> > Whether this is worth doing is a decision I will leave to others.
> >
> > Jim Starkey
> >
> >
> >
> ------------------------------------------------------------------------------
> > Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> > by Intel and developed in partnership with Slashdot Media, is your hub
> for all
> > things parallel software development, from weekly thought leadership
> blogs to
> > news, videos, case studies, tutorials and more. Take a look and join the
> > conversation now. http://goparallel.sourceforge.net/
> > Firebird-Devel mailing list, web interface at
> https://lists.sourceforge.net/lists/listinfo/firebird-devel
> >
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> Firebird-Devel mailing list, web interface at
> https://lists.sourceforge.net/lists/listinfo/firebird-devel
>
--
Jim Starkey
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
