Hi Karol, this is not windows compresion. This is record fragment compresion, that is done during when record is putting into DB page. New RLE has similar complexity as current RLE for compress, but more efficient for decompres because do not split large block of data into small pieces (yes, it is depended on data). Finally, because of more dense data, speed was increased (for read and write because less IO, more cache hits, ...).
I was indetified that Firebird killer is UTF8 charset. In my example is TMemo that is declared as VARCHAR(2000) UTF8. But in engine is allocated 8000 bytes for this field and old RLE will pack it as 126 byte. New RLE will pack it into 2 bytes if this field will be NULL. Value encoding as Jim suposed will encode this into 1 byte :). Also, my plan is move compresion from fragment level to record level and use value encoding instead RLE and LZ4 (if helps). This changes will increase data density and will be faster than my actual RLE. Also, if you want to create our self test, I can send you sources or compiled engine dll. I post this here as note, that before using some complex method we can use some more stupid (come from Z80 era) and already done improovements. If somebody have a fork of Firebird, I will participate on development. Slavek Ing. Slavomir Skopalik Executive Head Elekt Labs s.r.o. Collection and evaluation of data from machines and laboratories by means of system MASA (http://www.elektlabs.cz/m2demo) ----------------------------------------------------------------- Address: Elekt Labs s.r.o. Chaloupky 158 783 72 Velky Tynec Czech Republic --------------------------------------------------------------- Mobile: +420 724 207 851 icq:199 118 333 skype:skopaliks e-mail:skopa...@elektlabs.cz http://www.elektlabs.cz On 23.3.2015 19:11, liviusliv...@poczta.onet.pl wrote: >>> Decrease from ~150s(any run) -> 52s for first run and 36s another run. >>> If are you interested, I can send you source code or publish compiled >>> FB3 for Windows x64. >>> Slavek > Hi, > > results are optimistics but > can you make different test - compare write times not only reads? > > i ask because i see the same results when i turn file compression on Windows > system. > and db size decrease form 91GB to 48GB > > regards, > Karol Bieniaszewski > > > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming The Go Parallel Website, sponsored > by Intel and developed in partnership with Slashdot Media, is your hub for all > things parallel software development, from weekly thought leadership blogs to > news, videos, case studies, tutorials and more. Take a look and join the > conversation now. http://goparallel.sourceforge.net/ > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel > ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
