D. Richard Hipp wrote: > Most varints are "type varints" and type varints are almost always a > single byte (the only exceptions being for large blobs or strings). > Varints are also used to store the total number of bytes in a row > (also usually one byte). Most varints are a single byte. > > We, too, have profiled, and we agree that a lot of time is spent > decoding varints. As you have already observed, the common case of a > single-byte varint is usually handled by by a macro and so never calls > the sqlite3GetVarint() decoder function. And sqlite3GetVarint() is > very carefully coded to be fast even when it is called. The varint > decoder is one of the more carefully scrutinized parts of SQLite. > > I'm scanning through some profiling output now and I'm seeing that > some varints are almost always a single byte (only 60 multibyte > varints out of 474350, in one example) while others are multibyte > about half the time. I'm not seeing any cases where more then half > the varints are multibyte. > > Right. I observed that in the single byte case, the macro prevents the calling of the varint funtions. The last time I profiled this was after Shane worked it over last year. I believe the actual functions (not code generated by the macro) accounted for about 6% of the time spent in SQLite during our performance test suite. Obviously, that is going to be highly variable depending on the type of data contained, and the types of operations performed, etc. Clearly the varints in some cases, will be almost exclusively single byte. What I'm more curious about is their overall usage. Are we looking at 60/474350 for the entire database (in your example) or just one particular use within the database?
FWIW, I experimented with several different encoding schemes that preserved the single byte properties and was quickly able to cut the time consumed in our profiling test in half. Unfortunately, they all broke compatibility. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
