On 5/5/2014 11:53 AM, Dmitry Yemanov wrote:
05.05.2014 19:06, Jim Starkey wrote:
My sole reservation is that there can be information lost when a large
valued 64 integer is converted into a double to generate the index key.
AFAIR, int64 integers are not converted into double, they're compressed
using their native format.
If so, it's a bug. The purpose of the index encoding is that all
numbers have the same representation, based on actual value rather than
declaration. It also screws up big time when the scale factor changes.
If you declare an integer field as 64 bit, store some numbers, then
change either field scale factor or the field type to double, you index
lookups will fail.
Not good. I hope you're wrong.
My second concern is CPU performance. Index bucket traversal has always
been the single greatest hot spot in the architecture. Arno's jump
table, for example, was a net success by reducing CPU at the expense of
index density. The CPU cost of skip over a (variable length?)
transaction id, determine whether there is a second transaction id, and
is so, skip over the second could easily double or triple the cost of
index lookups.
In ODS11 we already have variably-length encoded prefix/length and
record number elements of the index node. My old tests showed that it
costed us about 20-30% in CPU time. The same for txn ids will surely add
even more overhead, but it's unlikely to reach 2x-3x levels.
I don't know what encoding you used, but the encoding:
1. The (canonical length - 1) of the number is stored in the high order
4 bits of the first byte
2. The first 4 significant bits of the number is stored in the low
order 4 bits of the first byte
3. The remain bytes follow, most significant byte first.
This encoding can be compared bytewise. If rolled into an index entry,
the complete entry could be compared naturally without decode. As a
special benefit, encoding known to be distinct can be compared without
regard for length.
I don't know what the total performance hit would be. I suggest
somebody prototype an implementation and get a solid handle on the cost
before making a decision. Alternatively, you could make a per-index
option and let the user pay his money and take his chances.
------------------------------------------------------------------------------
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
• 3 signs your SCM is hindering your productivity
• Requirements for releasing software faster
• Expert tips and advice for migrating your SCM now
http://p.sf.net/sfu/perforce
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
