On Thursday, November 19, 2015 at 7:45:45 PM UTC-5, Neil Van Dyke wrote: > Brian Adkins wrote on 11/19/2015 07:08 PM: > > The example that provided the initial motivation is extremely simple - > > process 45M+ text records with some minor transformations (including > > computing a couple soundex values on first/last names). Racket came in at > > 2.4x faster than Ruby, but still CPU bound, and it really should be I/O > > bound, so I figure I'll process N records in parallel. But I'm really > > thinking more generally - I know I'll occasionally encounter these types of > > issues. > > BTW, I don't know how much energy you've already put into > hand-optimizing the Racket code for that example, but just a general > comment for the list... There are often big gains to be had through > hand-optimizing. For this particular example, if one is seeing lots of > GC cost (which could be contributing to CPU-bound), and can't suppress > GC for the duration of the run, one might start by focusing on reducing > allocations (e.g., by reusing byte strings for read buffer, and maybe > even using same buffer for writing after you look at the pipeline). > Then, once I/O method is decided, profile and look at the operations and > the pipeline (e.g., maybe you find it's a win to have Soundex routines > working from zero-copy byte string I/O buffer directly, assuming > single-byte chars and handling any non-ASCII/Latin-1 as they go). In > this example, a fast implementation might be somewhat closer to how > you'd do it in C than a first-shot Racket implementation is, only it's > still easier than C overall, and it works more smoothly with other > Racket code than intermixing C and Racket does. > > Neil V.
I did *some* hand optimizing. Here's the code for the soundex and some string helper functions I created: https://gist.github.com/lojic/1deba97f2e2eb2fe3fc0 I fired up the profiler (in error trace mode) and worked on the critical paths. string-replace string-trim showed up. I hand rolled non-upcase-alpha and remove-zeros because they were faster than string-replace with regexes. I'm sure more could be done, but I'd be a bit surprised if I can go from 2.4x to even 3x the speed of Ruby given the optimizing I've already done. Brian -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
