[cmucl-help] Re: big strings
> "WP" == Walter C Pelissero <[EMAIL PROTECTED]> writes: WP> I suppose one could compare the generated code (disassembling it) and WP> see if there is any degradation. Some further investigation indicates that max-bits really can be #x. And the word-offset deftype is wrong. AFAICT, word-offset really is the word-offset (32-bit words) of a particular bit. So the type is more like (integer 0 (2^32/32)). I briefly looked at the disassembly for both functions. There are only 20 differences and they seem minor. I think the difference is because the offsets don't fit in a fixnum anymore. WP> Wouldn't a 32-bit integer be too big to fit in a "simple" Lisp object WP> thus requiring a further indirection? That is correct. But bit-bash-copy carefully declares things so that the compiler knows the bit offsets will remain a 32-bit unsigned object, and hence no consing is needed. (I think. The disassembly doesn't show any more consing than the previous version.) I've done a few more tests (ansi-tests) and it seems ok. Using your test-string-stream (but with only 15 iterations instead of 17) for timing, both versions have the same execution times. I think bit-bash-copy still has some issues since it won't be able to copy every possible object, but that should only be a problem with really big heaps and objects. I'll check this in very shortly. Thanks, Ray
[cmucl-help] Re: big strings
Raymond Toy (RT/EUS) writes: > > "WP" == Walter C Pelissero <[EMAIL PROTECTED]> writes: > > WP> Raymond Toy (RT/EUS) writes: > >> Thanks for the replacement. The only possible issue with the > >> replacement would be performance. BIT-BASH-COPY tries pretty hard > >> to do word copies. And BIT-BASH-COPY is intended to be able to > >> copy arbitrary bit strings. I don't know why it's limited to > >> (unsigned-byte 27) offsets. Perhaps to make sure the bit index > >> stays as a 32-bit int? > > WP> This is what came to mind reading the error. What I didn't > understand > WP> is why we get an error at all, being: > > WP> 2^27 > 17M (way larger) > > But the 17M is converted to a bit index and the bit index is greater > than 2^27. Sorry, I failed to grasp the meaning of the "bit-" part. > As an experiment, I changed the constant max-bits in > code/bit-bash.lisp from 2^27 to most-positive-fixnum. The comment > there says it's the maximum number of bits that can be delt [sic] with > during a single call. I suppose one could compare the generated code (disassembling it) and see if there is any degradation. > I don't see why it can't really handle at least > most-positive-fixnum number of bits or even #x number of > bits. [I assume you meant 2^32 and not 2^2^32] Wouldn't a 32-bit integer be too big to fit in a "simple" Lisp object thus requiring a further indirection? -- walter pelissero http://www.pelissero.de
[cmucl-help] Re: big strings
> "WP" == Walter C Pelissero <[EMAIL PROTECTED]> writes: WP> Raymond Toy (RT/EUS) writes: >> Thanks for the replacement. The only possible issue with the >> replacement would be performance. BIT-BASH-COPY tries pretty hard >> to do word copies. And BIT-BASH-COPY is intended to be able to >> copy arbitrary bit strings. I don't know why it's limited to >> (unsigned-byte 27) offsets. Perhaps to make sure the bit index >> stays as a 32-bit int? WP> This is what came to mind reading the error. What I didn't understand WP> is why we get an error at all, being: WP> 2^27 > 17M (way larger) But the 17M is converted to a bit index and the bit index is greater than 2^27. As an experiment, I changed the constant max-bits in code/bit-bash.lisp from 2^27 to most-positive-fixnum. The comment there says it's the maximum number of bits that can be delt [sic] with during a single call. I don't see why it can't really handle at least most-positive-fixnum number of bits or even #x number of bits. This compiles everything and your test function finishes fine. I think I'll do a few more tests on this. Ray
[cmucl-help] Re: big strings
Raymond Toy (RT/EUS) writes: > Thanks for the replacement. The only possible issue with the > replacement would be performance. BIT-BASH-COPY tries pretty hard > to do word copies. And BIT-BASH-COPY is intended to be able to > copy arbitrary bit strings. I don't know why it's limited to > (unsigned-byte 27) offsets. Perhaps to make sure the bit index > stays as a 32-bit int? This is what came to mind reading the error. What I didn't understand is why we get an error at all, being: 2^27 > 17M (way larger) and anyway array-dimension-limit = 2^29 - 1 -- walter pelissero http://www.pelissero.de
[cmucl-help] Re: big strings
> "WP" == Walter C Pelissero <[EMAIL PROTECTED]> writes: WP> On 19D the following code will cause an error: WP> * (defun test-string-stream () WP> (with-output-to-string (stream) WP> (dotimes (x 17) WP> (write-string (make-string (* 1024 1024)) stream))) WP> (values)) Thanks for the neat bug. [snip] WP> The following code gets rid of the bug in the string streams, although WP> there may be a more serious error somewhere else, as I haven't been WP> able to track down the call to BIT-BASH-COPY. BIT-BASH-COPY is called because of a DEFTRANSFORM for REPLACE, which is used in string-sout. WP> (in-package :lisp) Thanks for the replacement. The only possible issue with the replacement would be performance. BIT-BASH-COPY tries pretty hard to do word copies. And BIT-BASH-COPY is intended to be able to copy arbitrary bit strings. I don't know why it's limited to (unsigned-byte 27) offsets. Perhaps to make sure the bit index stays as a 32-bit int? And since the transform for REPLACE operates on simple strings, we could probably just call out to memcpy and depend on the OS to have an efficient implementation of memcpy, which is probably true these days. Ray
