> I would look at a Gray Streams solution. You could have a specialized
> stream class where a READ-SEQUENCE method specialized to your
> class could directly read double-floats into the lisp array and also
> READ could do its thing on the meta data. You would implement
> the class methods using unix:unix-open, unix:unix-read, unix:unix-write
> etc.
>
> Paul
I don't entirely understand your suggestion. In this scheme, what
would I have to implement? Would I have to implement myself all the
lower-level things (stream-read-char, stream-unread-char, etc.), and
then put the CL reader on top of that somehow?
At first, I thought that unix-read and unix-write might help me even
without Gray streams, but they seem to suffer from the same problems
as my earlier approaches --- I don't seem to be able to interleave C-
and CL-level IO operations.
For instance, if I define:
(defun unsigned-array (list)
(make-array (length list)
:element-type '(unsigned-byte 32)
:initial-contents list))
(defun read-unsigned-array (length stream)
(let ((result (make-array length :element-type '(unsigned-byte 32))))
(unix:unix-read (system:fd-stream-fd stream)
(system:vector-sap result)
(* 4 length))
result))
Then this:
(let ((a (unsigned-array '(1 2 3))))
(with-open-file (f "/home/rif/Tmp/tmp1.data" :direction :output)
(unix:unix-write (system:fd-stream-fd f) a 0 12)))
(with-open-file (f "/home/rif/Tmp/tmp1.data")
(read-unsigned-array 3 f))
does what I expect, but:
(let ((a (unsigned-array '(1 2 3)))
(b (unsigned-array '(4 5 6)))
(c (unsigned-array '(7 8 9))))
(with-open-file (f "/home/rif/Tmp/tmp3.data" :direction :output)
(write a :stream f)
(unix:unix-write (system:fd-stream-fd f) b 0 12)
(write c :stream f)))
(with-open-file (f "/home/rif/Tmp/tmp3.data")
(values (read f)
(read-unsigned-array 3 f)
(read f)))
does not. I have also not found any way to do it by opening the file
with unix-open, and then making it into a normal stream.
I don't yet see how Gray streams help me get around this issue. I
admit I'm not an expert on the issue (I've never really used Gray
streams), but I don't want to sink a lot of time in if there's no real
chance they can actually address this problem.
Cheers,
rif
