There are definitely some issues with string calculations. I've had
some issues dealing
with allegro's unicode support.
Look in memutil.lisp:
The byte-length macro
Look in serializer.lisp:
(method serialize %serialize) the string case statements
Serialization of strings involves:
1) Serialize appropriate string tag based on Lisp character width for
string type
In SBCL this is 1 byte or 4 bytes depending on Unicode support
2) Serialize the total string length (bytes * num chars). The
byte-length macro
is used for this
3) Serialize the array of characters using buffer-write-string
Hopefully that will help you figure out where the mismatch occurs.
Cheers,
Ian
Petter Egesund wrote:
> I am not sure, I have been looking for good documentation. I might exist
> somewhere, though? I am guessing int in FFI is closer connected to int
> as is defined by C and thereby the OS? Somebody can probably tell us...
>
> It seems also to be a problem computing the size of a buffer, as well,
> which causes problems for strings. I will take a closer look tomorrow,
> tonight I have to struggle with some deadlines...
>
> Petter
>
>
> -----Opprinnelig melding-----
> Fra: Ian Eslick [mailto:[EMAIL PROTECTED]
> Sendt: 22. august 2006 15:43
> Til: Petter Egesund; [EMAIL PROTECTED]
> Kopi: Robert L. Read; Ian Eslick; elephant-devel@common-lisp.net
> Emne: RE: SV: SV: SV: SV: SV: SV: [elephant-devel] Memory error with
> Elephant
>
> I don't have internet at the moment, but what is the definition of int
> vs integer in the sb-alien foreign interface? Just want to make sure we
> aren't asking for trouble in the future. (i.e. is int signed or
> unsigned vs integer? Is it 32bit for compatibility or whatever the clib
> it's linked to says it is?)
>
> Ian
>
> -----Original Message-----
> From: "Petter Egesund" <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc: "Robert L. Read" <[EMAIL PROTECTED]>; "Ian Eslick"
> <[EMAIL PROTECTED]>; elephant-devel@common-lisp.net
> Sent: 8/22/06 8:18 AM
> Subject: SV: SV: SV: SV: SV: SV: [elephant-devel] Memory error with
> Elephant
>
> Hi;
>
> I have done some debugging. It seems the problem of serializing integers
> is caused by mixing of the types int and integer in the foreign
> interface?
>
> I change the function read-int in the package memutil.lisp to:
>
> (defun pread-int (buf offset)
> "Read a 32-bit signed integer from a foreign char buffer."
> (declare (optimize speed (safety 1) (debug 1))
> (type (alien (* char)) buf)
> (type fixnum offset))
> (print "Offset: ") (print offset)
> (the (signed-byte 32)
> (deref (cast (sap-alien (sap+ (alien-sap buf) offset) (* char))
> (* int))))) ;; !!!!!!!!!!!!! Changed from (* integer)
>
>
> After this the integer-test runs fine. I will do some more testing.
>
> Cheers,
>
> Petter Egesund
>
>
>
>
>
> -----Opprinnelig melding-----
> Fra: Ian Eslick [mailto:[EMAIL PROTECTED]
> Sendt: 21. august 2006 16:11
> Til: Petter Egesund
> Kopi: Robert L. Read; [EMAIL PROTECTED]; Ian Eslick
> Emne: Re: SV: SV: SV: SV: SV: [elephant-devel] Memory error with
> Elephant
>
> Interesting. That explains the source of our problem (serializing
> integers) but I still don't understand why. Even if the 4 byte
> assumption in lisp keeps us from expanding the buffer streams
> appropriately the write should still write the first 8 bytes correctly
> and thus be able to read it correctly. It almost looks like it is
> writing 4 bytes but reading 8.
>
> 1) Try replacing the #x45 with #x00 in my test code
>
> The other test is to see what is actually being put into the buffer
> stream.
>
> 2) replace #x45 with #x40302010 and add the following code instead of
> the buffer-read-int at the end:
>
> (loop for i from 0 upto 16 do
> (format t "byte ~A: ~A~%" i (buffer-read-byte bs)))
>
> Ian
>
> Petter Egesund wrote:
>
>>
>>
>> ----------------------------------------------------------------------
>> --
>> *Fra:* Robert L. Read [mailto:[EMAIL PROTECTED]
>> *Sendt:* 19. august 2006 22:26
>> *Til:* Petter Egesund
>> *Kopi:* [EMAIL PROTECTED]; Ian Eslick
>> *Emne:* Re: SV: SV: SV: SV: [elephant-devel] Memory error with
>> Elephant
>>
>> Personally, the fact that buffer-write-int in memutil.lisp assumes a
>> 4-byte integer:
>> (defun buffer-write-int (i bs)
>> "Write a 32-bit signed integer."
>> (declare (optimize (speed 3) (safety 0))
>> (type buffer-stream bs)
>> (type (signed-byte 32) i))
>> (with-struct-slots ((buf buffer-stream-buffer)
>> (size buffer-stream-size)
>> (len buffer-stream-length))
>> bs
>> (let ((needed (+ size 4)))
>> (when (> needed len)
>> (resize-buffer-stream bs needed))
>> (write-int buf i size)
>> (setf size needed)
>> nil)))
>> While the c-code version of write-int in libmemutil.c uses the
>> "sizeof(int)" paradigm seems fragile on a 64-bit architecture:
>>
>> void write_int(char *buf, int num, int offset) {
>> memcpy(buf+offset, &num, sizeof(int)); }
>>
>> I would like to know if "sizeof(int)" is 8 or 4 as compiled.
>> A simple C program (or reading the f-ing manual) would answer that
>> question.
>> An awful hack that would let us test things would be to hard-wire "4"
>> in place of
>> "sizeof(int)" in the libmemutil.c file; I don't know enough about
>> UFFI and sap-alien to know if that would work.
>>
>> A better solution is to expose a C-function that simply gives the size
>>
>
>
>> of an integer in bytes to LISP (it could be added to libmemutil.c
>> file) and then have memutil.lisp use that in place of the assumption
>> "4".)
>>
>> How this changes the typing in memutil.lisp, I'm not sure ---- can
>> we/should we force the
>> AMD64 to use 32-bit integers, or should we rewrite memutil.lisp to
>> determine based on its compilation environment the size of an integer?
>>
>
>
>> Either solution is better than what we have now.
>>
>>
>> On Sat, 2006-08-19 at 20:53 +0200, Petter Egesund wrote:
>>
>>> It looks like below. Certainly not 35.
>>>
>>> Petter
>>>
>>> -------
>>>
>>> ELE> (defun fixnum-test ()
>>> (with-buffer-streams (bs)
>>> (buffer-write-int #x23 bs)
>>> (buffer-write-int #x45 bs)
>>> (reset-buffer-stream bs)
>>> (buffer-read-int bs)))
>>> ; in: LAMBDA NIL
>>> ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT 69 ELEPHANT::BS)
>>> ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET
>>> WHEN ; --> COND IF PROGN ; ==>
>>> ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS
>>> ; ELEPHANT-MEMUTIL::NEEDED)
>>> ;
>>> ; note: doing signed word to integer coercion (cost 20)
>>>
>>> ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT 35 ELEPHANT::BS)
>>> ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET
>>> WHEN ; --> COND IF PROGN ; ==>
>>> ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS
>>> ; ELEPHANT-MEMUTIL::NEEDED)
>>> ;
>>> ; note: doing signed word to integer coercion (cost 20) ; ;
>>> compilation unit finished
>>> ; printed 2 notes
>>> FIXNUM-TEST
>>> ELE> (fixnum-test)
>>> 296352743459
>>> ELE>
>>>
>>>
>>>
>>>
>>>
>>>
>
>
_______________________________________________
elephant-devel site list
elephant-devel@common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel
