Erik Max Francis <[EMAIL PROTECTED]> writes: > Alexander Schmolck wrote: > >> Is this any faster? >> >> ordSum, orsSumSq = (lambda c:c.real,c.imag)(sum(complex(ord(x),ord(x)<<1) >> for x in data)) > > That's pretty clever, but I neglected to mention that I need to accumulate the > sums as ints/longs to avoid losing precision, so converting to floating point > isn't an optional.
I think you can safely use this trick (provided it really makes things faster) provided you sum blocks no larger than 2**(53-8) bytes; if your files are really that big you'd certainly want to split summing into several blocks anyway, because otherwise you'll be doing *lots* of extra bignum arithmetic instead of int32/int64 addition (I'd assume this will slow things noticably down even in python). Another trick you could try, again using table-lookup: work on words (i.e. 2bytes) instead of single bytes again using a table (from word->(byte-sum,sq-byte-sum) tuples) ; this will half the function calls and the table size of is hopefully still small enough to not to ruin your cache-hit rate (you might want to try array). 'as -- http://mail.python.org/mailman/listinfo/python-list