I
inadvertently sent it to Davids address instead of the forums.
Thanks for the links, I'll look into it. Talking about large amounts of data,
do you happen to have an example of using HashArray? This would be
great.
Regards,
Michael
Am 10/17/2012 12:05 PM, schrieb
Makarius:
On Wed, 17 Oct 2012, David Matthews wrote:
Michael,
It's probably a good idea to send questions like this to the
Poly/ML mailing list and I'm copying this reply to the list.
There are people on the list who may have experience of what
you're trying to do and also any replies get archived and reduce
the chances of the same question being asked repeatedly.
To reply to your specific question: there's no difference in
efficiency between TextIO and BinIO on Unix. On Windows TextIO
is more complicated because of the need to convert between CRLF
and "\n". I'm not clear how you are accessing the file, whether
it is random or sequential. To read individual bytes you can
always use BinIO.input1 or TextIO.input1. This would save you
having to explode a vector.
BinIO deals with values of type Word8.word and the vector is
Word8Vector.vector. These are unsigned byte values.
Regards,
David
On 16/10/2012 20:59, Michael Moeller wrote:
Hi,
I've got a problem regarding huge files as input. I need to
get access to
the single byteslike with fgetc() in C. TextIO and
subsequently exploding
the string does the jobbut performance matters and it seems to
me like
BinIOsaves me from the 'explode' step.Unfortunately I can't
figure out
how to access theresulting BinIO.vector. I'd be appreciative
to seean
exampleand please tell me if this is not the right approach.
Regards,
Michael
David probably intended to send this back to Michael Moeller, and
just got the "M" completion wrong.
Nonetheless, I can offer two hints from the Isabelle/ML code base:
(1) Combinators fold_lines and fold_pages to access large text
files
in a scalable manner:
http://isabelle.in.tum.de/repos/isabelle/file/Isabelle2012/src/Pure/General/file.ML#l132
(2) Some applications of BinIO.vector over sockets, which had to
be
trashed for compatibility with old Poly/ML 5.4.x versions:
http://isabelle.in.tum.de/repos/isabelle/file/63144ea111f7/src/Pure/System/system_channel.ML
http://isabelle.in.tum.de/repos/isabelle/file/8b20be429cf3/src/Pure/System/system_channel.ML
Makarius
_______________________________________________
polyml mailing list
polyml@inf.ed.ac.uk
http://lists.inf.ed.ac.uk/mailman/listinfo/polyml
|
_______________________________________________
polyml mailing list
polyml@inf.ed.ac.uk
http://lists.inf.ed.ac.uk/mailman/listinfo/polyml
