Re: [Chicken-users] Is there interest in a new egg (fast-io)?
On Apr 6, 2010, at 9:34 AM, Jeronimo Pellegrini wrote: BTW, I've benchmarked complex reading ad writing too. I suppose the difference is huge for reading complexes because the read procedure doesn't know that it's a complex number. Hi there. Your numbers looked odd to me so I downloaded and ran your benchmarks. I believe you got such poor performance on reads due to (use numbers). Here are your numbers again: schemefast-io speedup --- write fixnum14.602 5.3032.7535 read fixnum58.208 11.7064.9725 write flonum37.003 21.6471.7094 read flonum 100.367 15.6266.4231 write complex 89.361 56.0201.5952 read complex 3076.952 59.47651.734 --- Here are my numbers: scheme fast-io line numbers 1.8 2.1 -- write fixnum26 2126 26 read fixnum61 2022 211 193 write flonum48 2548 50 read flonum86 2425 276 246 where scheme is your code without numbers loaded, fast-io is your fast-io code, numbers is your code with numbers version 1.8 or 2.1 loaded, line is my fast scheme implementation. CPU performance disparity aside, I don't get nearly the slowdown that you do with plain scheme (read), so my guess is that either you loaded the numbers egg or that fast-io is inexplicably slow on my machine. I presume you loaded numbers because your complex test did not work without it. Furthermore the line implementation is basically on par with fast-io, as long as numbers is not loaded. To get the line implementation for readers all you need to do is replace (read port) with (string-number (read-line port)). Furthermore when you generate the files, use a newline delimiter instead of a space. line is faster than scheme by 3-4x because read is more general and more importantly, because (read) is character based and char-based I/O from stream ports is really slow on Chicken. On the other hand, line and string based I/O is fast. Based on this admittedly quick test, there's not much advantage to using fast-io as long as your data is newline-delimited. What might be useful is a read-delimited-string (or whatever) which would read until a specified set of delimiters. This wouldn't replace read-line--as newlines are handled specially due to \r--nor would it handle escaped chars, but it would handle the basic case of reading strings until NUL or SPACE. Or you could preprocess your space-delimited files with tr ' ' '\012' :) Jim ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Is there interest in a new egg (fast-io)?
Hello! On Thu, Apr 22, 2010 at 02:25:00PM -0500, Jim Ursetto wrote: Here are my numbers: scheme fast-io line numbers 1.8 2.1 -- write fixnum26 2126 26 read fixnum61 2022 211 193 write flonum48 2548 50 read flonum86 2425 276 246 where scheme is your code without numbers loaded, fast-io is your fast-io code, numbers is your code with numbers version 1.8 or 2.1 loaded, line is my fast scheme implementation. CPU performance disparity aside, I don't get nearly the slowdown that you do with plain scheme (read), so my guess is that either you loaded the numbers egg or that fast-io is inexplicably slow on my machine. I presume you loaded numbers because your complex test did not work without it. Yes, I used the numbers egg, that's right. I wasn't expecting that it would slow down reading and writing of flonums and fixnums that much. Furthermore the line implementation is basically on par with fast-io, as long as numbers is not loaded. To get the line implementation for readers all you need to do is replace (read port) with (string-number (read-line port)). Furthermore when you generate the files, use a newline delimiter instead of a space. line is faster than scheme by 3-4x because read is more general and more importantly, because (read) is character based and char-based I/O from stream ports is really slow on Chicken. On the other hand, line and string based I/O is fast. That makes sense -- by reading a sequence of characters and then using string-number you used a somewhat similar idea that I used myself (if you *know* it's a number, don't try to parse a string or S-expression or anything else). Based on this admittedly quick test, there's not much advantage to using fast-io as long as your data is newline-delimited. Yes, it looks like that. What might be useful is a read-delimited-string (or whatever) which would read until a specified set of delimiters. It's on the latest version of the egg! (read-string-until CHAR PORT) (read-string-between LEFTCHAR RIGHTCHAR PORT) I didn't yet include read-string-n because it's an identifier used in R6RS, and may (who knows) end up in the next standard (R7RS or whatever it will be called -- as far as I know there is no name for it yet) when it comes out. This wouldn't replace read-line--as newlines are handled specially due to \r--nor would it handle escaped chars, but it would handle the basic case of reading strings until NUL or SPACE. Or you could preprocess your space-delimited files with tr ' ' '\012' :) Thanks for pointing this out! I'll still use specialized-io myself (my files have several numbers per line, with some strings there too). For human-unreadable fast I/O maybe converting numbers into binary and writing blobs would be faster. J. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Is there interest in a new egg (fast-io)?
On Mon, Apr 05, 2010 at 09:04:47PM -0700, Shawn Rutledge wrote: On Mon, Apr 5, 2010 at 7:18 PM, Jeronimo Pellegrini j...@aleph0.info wrote: Implemented and tested: - write-fixnum, read-fixnum - write-flonum, read-flonum For those I have been making good use of the endian-port egg (not released for chicken 4 yet though); I wonder how the benchmarks would turn out, but the implementation seems like it should be fast. Sounds like you have some unique ideas for strings though. I'll work on the string stuff soon. I won't write full lexers, but those functions should at least help a bit when parsing simple files. :-) I don't have Chicken 3 installed, so I can't compare performance. Anyway, I've uploaded it here if you'd like to give it a try: http://aleph0.info/fast-io.tar.gz J. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Is there interest in a new egg (fast-io)?
On Tue, Apr 06, 2010 at 11:24:20AM -0300, Jeronimo Pellegrini wrote: Anyway, I've uploaded it here if you'd like to give it a try: http://aleph0.info/fast-io.tar.gz I forgot -- whatch out for the file size and time to be spent in benchmarks! You may want to change the value of +max+ in benchmarks/run.scm to a smaller value and increase it gradually. BTW, I've benchmarked complex reading ad writing too. I suppose the difference is huge for reading complexes because the read procedure doesn't know that it's a complex number. One caveat: fast-io reads and writes exactly as printf/scanf do, so the external representation may not be the one used by your other Scheme programs! J. schemefast-io speedup --- write fixnum14.602 5.3032.7535 read fixnum58.208 11.7064.9725 write flonum37.003 21.6471.7094 read flonum 100.367 15.6266.4231 write complex 89.361 56.0201.5952 read complex 3076.952 59.47651.734 --- ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Is there interest in a new egg (fast-io)?
Hi Jeronimo On Mon, 5 Apr 2010 23:18:57 -0300 Jeronimo Pellegrini j...@aleph0.info wrote: I've been working on some unsafe fast I/O procedures for my own use, and I thought I'd make them an extension, if more people would like to use them. These are for reading and writing values from/to large files. I'll include the benchmarking code with the egg, as well as tests. Would you like an SVN account, so you can check in your code and manage it? If so, just send me a private message indicating a username and a hash of your password, which can be generated with: $ openssl passwd -apr1 I'd suggest naming the egg in a way it doesn't sound as general as faster I/O primitives. Maybe it's just me, but when I read fast-io, I instinctively think that Chicken's I/O primitives are slow and by using fast-io I'll get faster I/O for any case. Again, it's only a suggestion, maybe it's just me. Best wishes. Mario ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Is there interest in a new egg (fast-io)?
On Tue, Apr 06, 2010 at 03:41:36PM -0400, Mario Domenech Goulart wrote: Hi Jeronimo Hi Mario! I'd suggest naming the egg in a way it doesn't sound as general as faster I/O primitives. Maybe it's just me, but when I read fast-io, I instinctively think that Chicken's I/O primitives are slow and by using fast-io I'll get faster I/O for any case. Again, it's only a suggestion, maybe it's just me. Right! What would be more appropriate? unsafe-io fast-unsafe-io unsafe-io-procedures nonstandard-fst-io J. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Is there interest in a new egg (fast-io)?
On Tue, 6 Apr 2010 17:11:25 -0300 Jeronimo Pellegrini j...@aleph0.info wrote: I'd suggest naming the egg in a way it doesn't sound as general as faster I/O primitives. Maybe it's just me, but when I read fast-io, I instinctively think that Chicken's I/O primitives are slow and by using fast-io I'll get faster I/O for any case. Again, it's only a suggestion, maybe it's just me. Right! What would be more appropriate? unsafe-io fast-unsafe-io unsafe-io-procedures nonstandard-fst-io How about type-specific-io? Best wishes. Mario ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
[Chicken-users] Is there interest in a new egg (fast-io)?
Hello, I've been working on some unsafe fast I/O procedures for my own use, and I thought I'd make them an extension, if more people would like to use them. The fast-io extension would have these procedures: Implemented and tested: - write-fixnum, read-fixnum - write-flonum, read-flonum - write-one-char, read-one-char (I don't know if the names are OK; I didn't want them to conflict with R5RS procedures) Implemented, not tested: - write-complex, read-complex Planned but not yet implemented: - write-string - read-string-n - read-string-until (reads until a delimiter is found) - read-string-between (ignores characters until left delimiter, then reads until right delimiter) All of these are implemented using (as little as possible) foreign-lambda* with some inline calls to fprintf, fscanf, and fgetc. The only kind of error checked for is the return from the C functions (and if, for example, fscanf fails to read, then a (configurable) scheme error procedure is called). The two last string-reading procedures will be triple-checked for security. :-) Why: For everyday use, I really think this isn't necessary. But if you need (as I think I'll need) to read and write very large amounts of numbers from/to files, several times. Then this becomes useful. If you *know* that your file only has fixnums or flonums (written by your program), then you can use these procedures -- so there's no need to make the program go checking if it looks like an exact or inexact number, or as a string, or whatever. Just tell it to read an integer number!) If there is interest, it's already packaged as an egg (installed on my machine). The speedup I got is on the following table: schemefast-io speedup --- write fixnum14.602 5.3032.7535 read fixnum58.208 11.7064.9725 write flonum37.003 21.6471.7094 read flonum 100.367 15.6266.4231 --- These are for reading and writing values from/to large files. I'll include the benchmarking code with the egg, as well as tests. Thanks, J. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Is there interest in a new egg (fast-io)?
Jeronimo Pellegrini scripsit: - write-fixnum, read-fixnum - write-flonum, read-flonum Does this refer to a textual or a binary format file? -- John Cowan co...@ccil.org http://ccil.org/~cowan In computer science, we stand on each other's feet. --Brian K. Reid ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Is there interest in a new egg (fast-io)?
On Mon, Apr 05, 2010 at 10:34:03PM -0400, John Cowan wrote: Jeronimo Pellegrini scripsit: - write-fixnum, read-fixnum - write-flonum, read-flonum Does this refer to a textual or a binary format file? Textual. I had no intention of using binary files, although that could be nice too (saves space and could be somewhat faster, since disk access is the bottleneck). J. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Is there interest in a new egg (fast-io)?
On Mon, Apr 5, 2010 at 7:18 PM, Jeronimo Pellegrini j...@aleph0.info wrote: Implemented and tested: - write-fixnum, read-fixnum - write-flonum, read-flonum For those I have been making good use of the endian-port egg (not released for chicken 4 yet though); I wonder how the benchmarks would turn out, but the implementation seems like it should be fast. Sounds like you have some unique ideas for strings though. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
