Re: [Firebird-devel] Record level compression for V4

Hi Dmitry, I will send latest version, that have some improvements. In any case, feel free to contact privately (E-mail or skype). Slavek Ing. Slavomir Skopalik Executive Head Elekt Labs s.r.o. Collection and evaluation of data from machines and laboratories by means of system MASA

Re: [Firebird-devel] Record level compression for V4

Slavomir, > is it good time (V4) to commit new record level compression? > > http://elektlabs.cz/fbrle/ Is this download the latest version? http://elektlabs.cz/fbrle/FirebirdWin64_ElektLabs.zip > If yes, will you need my help for this task? I will start with replacing the current RLE with the

Re: [Firebird-devel] Record level compression for V4

Hi, to be able to do it, I need to switch to record level compression. And for this task I need your help. For first, you can use a memory move instead compression, I will replace by ELekt Labs RLE (+LZ4) for fist iteration. Next I will change to value encoding or value encoding + LZ4 (or LZ4

Re: [Firebird-devel] Record level compression for V4

04.03.2016 18:49, Slavomir Skopalik wrote: > > if you help me with integration, will do it. I will. Just consider it a trial development. I cannot promise anything before seeing the test results. Dmitry --

Re: [Firebird-devel] Record level compression for V4

Hi Dmitry, if you help me with integration, will do it. Slavek > I have nothing against a value-based encoding. But our v4 development > tasks currently do not include inventing one. If someone jumps in with a > prototype implementation, I'd be definitely willing to review/consider it. > > >

Re: [Firebird-devel] Record level compression for V4

03.03.2016 22:49, Slavomir Skopalik wrote: > > Is it right time or will be postponed to V5 So far I see no serious reason to postpone it. Just please wait a couple of weeks, I will post to this thread when ready. Dmitry

Re: [Firebird-devel] Record level compression for V4

03.03.2016 19:52, James Starkey wrote: > > I was not talking about value-based encoding (which surely requires a > different approach). > > And why not? I have nothing against a value-based encoding. But our v4 development tasks currently do not include inventing one. If someone jumps in

Re: [Firebird-devel] Record level compression for V4

1. Data can (already did) at any point, not only at specific, that you marks by '!'. 2. If packed record doesn't fit into one page, control sequence is parsed from end. 3. When rest of sequence will fit, compression is started again from beginning. 4. If fragment has odd length, zero is added.

Re: [Firebird-devel] Record level compression for V4

Hi Jim, Elekt Labs RLE is ready to use and is better then current one. The best one have to be developed and I will help if I can. The question is still same: Is it right time or will be postponed to V5 (2024+) ? Slavek > > Excuse me, but faster than the current one is not the question. The

Re: [Firebird-devel] Record level compression for V4

On Thursday, March 3, 2016, Dmitry Yemanov wrote: > 03.03.2016 19:16, Jim Starkey wrote: > > > On 3/3/2016 11:10 AM, Dmitry Yemanov wrote: > >> Am I missing something obvious? > > > > Yeah. Data. > > I was not talking about value-based encoding (which surely requires a >

Re: [Firebird-devel] Record level compression for V4

03.03.2016 19:16, Jim Starkey wrote: > On 3/3/2016 11:10 AM, Dmitry Yemanov wrote: >> Am I missing something obvious? > > Yeah. Data. I was not talking about value-based encoding (which surely requires a different approach). Dmitry

Re: [Firebird-devel] Record level compression for V4

On 3/3/2016 11:10 AM, Dmitry Yemanov wrote: > Am I missing something obvious? Yeah. Data. > Dmitry > -- > > Site24x7 APM Insight: Get Deep Visibility into Application Performance > APM + Mobile APM + RUM: Monitor 3

Re: [Firebird-devel] Record level compression for V4

03.03.2016 18:34, Slavomir Skopalik wrote: > > But value encoding cannot be implemented until we switch from fragment > compression to true record level compression. I was not speaking about any encoding, just about compacting the record. Fragments are not compressed independently. The whole

Re: [Firebird-devel] Record level compression for V4

All, This is how I see the things. RLE is good for record compression due to the following facts: 1) In-memory record has all the fields expanded up to their maximum size. Shorter values are padded with zeroes (that are easy to compress). 2) In-memory record stores the NULL bitmap and at the

Re: [Firebird-devel] Record level compression for V4

> they are usually poorly documented, which creates a large burden on > driver implementers having to follow all intricacies of the code (in my > case in a language that I'm not fluent in) to understand what changes > they need to implement. Especially with code involving encryption or >

Re: [Firebird-devel] Record level compression for V4

On 2016-03-03 1:35, Slavomir Skopalik wrote: > Hi, > this is about record encoding . > My private test with RLE + LZ4 shows, that from combination RLE, LZ4, > RLE + LZ4 the last is the best. I'm perfectly ok with doing that for records. > I believe, that better record encoding will help wire

Re: [Firebird-devel] Record level compression for V4

Hi, this is about record encoding . My private test with RLE + LZ4 shows, that from combination RLE, LZ4, RLE + LZ4 the last is the best. I believe, that better record encoding will help wire transfer in any case. From other side, record encoding can be used in all situation (in average speed

Re: [Firebird-devel] Record level compression for V4

On 2-3-2016 18:41, Slavomir Skopalik wrote: > Hi Jim, > I know, but in current situation is little problematic. > > Current compression is doing on record fragment, not on full record. > I was hacked Elekt Labs RLE to be able work in this situation. > My tests show big advantage compared to

Re: [Firebird-devel] Record level compression for V4

Hi Jim, I know, but in current situation is little problematic. Current compression is doing on record fragment, not on full record. I was hacked Elekt Labs RLE to be able work in this situation. My tests show big advantage compared to current V3 RLE. If we switch to real record level