Hi Slava,
I'm building (what amounts to) an olap database engine in factor, and
part of that involves linear searching mmapped files. Basically the
faster I can do linear searching operations, the less I need to use
sorted indexes and the more data I can fit into memory (esp with
compression). I've currently got a bunch of C code doing searches, but
ideally I'd like to not rely on a separate C dll in the released code or
require people to have the gcc tool chain to compile it.
I decided to experiment with using asm intrinsics to implement the tight
loops, and took 'count' as a simple testcase. I'm searching a 48M file
of 32bit integers.
By comparison some C code (minus the mmaping stuff):
int count_items(int *from,int *to,int item) {
int count = 0;
while(from != to){
if(*from == item) count++;
from ++;
}
return count;
}
Which when compiled with -O3 takes about 80ms to search the file on a
processor locked at 800MHZ with the filebuffers running warm. The best I
could do with factor in this setup was a few seconds (which I think is
still pretty good).
So as an experiment I wrote an x86 asm version of count-items (see
below) and spliced it into factor using word intrinsics. This is fast
and really cool, but I was wondering if maybe there are any ways to
better factor this.
E.g. within a word definition is it possible to put unboxed items on the
stack? (maybe by temporarily disabling gc or something?). If everything
is inlined within a word definition, can the gc still get invoked?
I'm just experimenting here, but any thoughts would be much appreciated!
Many thanks,
Phil
USING: cpu.x86.assembler cpu.x86.architecture cpu.architecture words
prettyprint generator.registers generator tools.disassembler
generator.fixup alien.c-types alien fry io.mmap ;
: (asmcount) ( from to value count -- count )
drop drop drop ; ! dummy impl
! operands: from to value count
: %asmcount ( -- )
"myloop" define-label "noincr" define-label
"from" get dup %unbox-alien ! unbox input params
"to" get dup %unbox-alien
"value" operand %untag-fixnum
"count" operand %untag-fixnum
"myloop" resolve-label ! loop start
"from" operand [] "value" operand CMP ! incr count if matches
"noincr" get JNE
"count" operand 1 ADD
"noincr" resolve-label
"from" operand 4 ADD ! next
"to" operand "from" operand CMP
"myloop" get JNE
"count" operand %tag-fixnum ! box output params
;
\ (asmcount)
{
{
[ %asmcount ]
H{
{ +input+ { { f "from" } { f "to" } { f "value" } { f
"count" } } }
{ +output+ { "count" } }
}
}
} define-intrinsics
: count-items ( from to value -- count ) 0 (asmcount) ;
: mmap>from,to ( mmap -- alien alien )
[ address>> dup alien-address ] [ length>> ] bi + <alien> ;
: with-whole-mapped-file ( fname quot -- )
[ dup file-info size>> ] dip with-mapped-file ;
: test-count-items ( fname value-to-match -- count )
'[ mmap>from,to , count-items ] with-whole-mapped-file ;
: test "mytestfile" 12345 test-count-items ;
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Factor-talk mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/factor-talk
