I have a few minor stile comments:
*** I'd replace
(define backslash 92)
with
(define backslash (char->integer #\\))
to improve legibility.
And do the same replacement for other magic numbers. After
optimization, both versions are identical.
[The only site where this would cause a difference is in code like
(for ([i (in-range 92)]) ...)
but you are not using something like this.]
*** In few spots, you use `null` as "do nothing in this case", for
example in line 38. I think it's more idiomatic to use `(void)`.
*** In line 328, I it possible to do this? Replace
;;[[[
;; 5. Remove all occurrences of AEIOU, except first letter
(define len2 (bytes-length str4))
(define str5(bytes-append
(subbytes str4 0 1)
(bytes-delete-pred
(if (> len2 1) (subbytes str4 1 len2) #"")
(λ (b) ...))))
;; 6. If first symbol is a digit replace it with letter saved on step 1.
(when (byte-numeric? (bytes-ref str5 0))
(bytes-set! str5 0 first_letter))
;;;]]]
with this
;;[[[
;; 5. Remove all occurrences of AEIOU, except first letter
(define len2 (bytes-length str4))
(define str5(bytes-append
#"?"
(bytes-delete-pred
(if (> len2 1) (subbytes str4 1 len2) #"")
(λ (b) ...))))
;; 6. Replace firt symbol with letter saved on step 1.
(bytes-set! str5 0 first_letter)
;;;]]]
or this:
;;[[[
;; 5. Remove all occurrences
(define len2 (bytes-length str4))
(when (>= len2 1)
(bytes-set! str4 0 (char->integer #\?))
(define str5
(bytes-delete-pred
str4
(λ (b) ...))))
;; 6. Replace firt symbol with letter saved on step 1.
(when (>= len2 1)
(bytes-set! str5 0 first_letter))
;;;]]]
I hope I'm not missing a corner case. I estimate that with the 200K
fields this will reduce the time from 2.5s to 2.4s.
*** I tried a few more changes that avoid allocating intermediate
bytes, but they are more complex and with 200K rows I estimate that
the difference in run time would be only 0.02s. If I find something
bigger, I'll write again.
Gustavo
PS: How many "JOHN"s do you have? Have you considered caching the
soundex of the names. But I'm not sure if this would be faster.
On Sun, Jan 17, 2016 at 10:13 PM, Brian Adkins <[email protected]> wrote:
> On Sunday, January 17, 2016 at 2:54:39 PM UTC-5, Brian Adkins wrote:
>> On Sunday, January 17, 2016 at 2:50:19 PM UTC-5, Brian Adkins wrote:
>> >
>> > With built-in string-trim, the lowest of three runs was 10293. Using your
>> > string-trim the lowest of three runs was 7618, so it reduced the runtime
>> > by 26%.
>>
>> Although, I probably should've mentioned that I'm not particularly
>> interested in unsafe optimizations. I already have a very fast C program if
>> I'm willing to risk unsafe behavior, so for Racket, I'd like to retain
>> safety.
>>
>> Having said that, I'm pretty sure a combination of using Byte Strings and
>> manually optimizing string-trim & string-replace (or skipping them in some
>> cases) will get under the Ruby time.
>
> Yay - success! :)
>
> I changed all strings to byte strings while leaving the style of the code
> very similar. It made a significant difference. Of course, I also gained the
> benefit if handwritten bytes-split, bytes-replace, bytes-delete, bytes-trim,
> etc. which were narrowly defined just for this app.
>
> Timings on a 200K line file are now:
>
> Ruby = 7.53s
> Racket = 2.52s (was 10.3s)
>
> The string version of the Racket program was over 4x slower. I'm quite
> satisfied with being 3x faster than Ruby with a similar coding style given
> this is really in Ruby's sweet spot i.e. text munging.
>
> New code is here:
>
> https://gist.github.com/lojic/892049e617637903f982
>
> I think my next step will be to create a version that uses places.
> make-shared-bytes may be useful for that. I'll report back with timings. I
> have a 4 core macbook pro w/ 8 hyperthreads - no idea whether the
> hyperthreads are actually useful, but if I can get a 3x speedup with 4 cores,
> I'd be pretty pleased.
>
> I suppose the following is a reasonable architecture:
>
> 1 place for reading the input file and placing a record in a input queue
> N places (one per core) to read from the input queue, process and place in an
> output queue
> 1 place for reading the output queue and writing to either of two output files
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
