Re: [racket-users] Racket performance tips
My string-trim uses unsafe ops, but I'm pretty sure it's safe. The (safe) string-length at the start ensures we're using a string. The rest are indexing and fixnum arithmetic on integers that are guaranteed to be valid indices of the string. Still, if you don't like this, replace the unsafe ops with the corresponding safe ones. It will still be much faster than the built-in version. > On Jan 17, 2016, at 2:54 PM, Brian Adkinswrote: > >> On Sunday, January 17, 2016 at 2:50:19 PM UTC-5, Brian Adkins wrote: >> >> With built-in string-trim, the lowest of three runs was 10293. Using your >> string-trim the lowest of three runs was 7618, so it reduced the runtime by >> 26%. > > Although, I probably should've mentioned that I'm not particularly interested > in unsafe optimizations. I already have a very fast C program if I'm willing > to risk unsafe behavior, so for Racket, I'd like to retain safety. > > Having said that, I'm pretty sure a combination of using Byte Strings and > manually optimizing string-trim & string-replace (or skipping them in some > cases) will get under the Ruby time. > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Racket performance tips
However, I don't think string representation is the issue, so long as we're talking about the performance of string-trim. Racket's string-trim is written for flexibility. It allows you to trim the left side, the right side, or both sides of the string, and it allows you to trim characters matching an arbitrary regexp. Ruby's String#strip, on the other hand, trims whitespace characters from both sides of the string. If you want to trim only the left side, you use String#lstrip. The right side? String#rstrip. (And then there are in-place and non-mutating versions of each.) If you want to trim something other than whitespace you use something else. My string-trim is much faster than Racket's built-in one, because it's completely inflexible, like Ruby's String#strip. Also, because it doesn't use regexps, it doesn't have another layer of interpretation -- and the whole thing is visible to the JIT (though I don't know how much difference that makes in this case). -Jon On Sun, Jan 17, 2016 at 3:08 PM, Robby Findlerwrote: > Do we know if ruby represents strings the same way Racket does? The > representation in C clearly admits more efficient implementations of > relevant operations here, and Ruby's might too. > > Robby > > > On Sun, Jan 17, 2016 at 2:00 PM, Jon Zeppieri wrote: > > My string-trim uses unsafe ops, but I'm pretty sure it's safe. The > (safe) string-length at the start ensures we're using a string. The rest > are indexing and fixnum arithmetic on integers that are guaranteed to be > valid indices of the string. > > > > Still, if you don't like this, replace the unsafe ops with the > corresponding safe ones. It will still be much faster than the built-in > version. > > > >> On Jan 17, 2016, at 2:54 PM, Brian Adkins > wrote: > >> > >>> On Sunday, January 17, 2016 at 2:50:19 PM UTC-5, Brian Adkins wrote: > >>> > >>> With built-in string-trim, the lowest of three runs was 10293. Using > your string-trim the lowest of three runs was 7618, so it reduced the > runtime by 26%. > >> > >> Although, I probably should've mentioned that I'm not particularly > interested in unsafe optimizations. I already have a very fast C program if > I'm willing to risk unsafe behavior, so for Racket, I'd like to retain > safety. > >> > >> Having said that, I'm pretty sure a combination of using Byte Strings > and manually optimizing string-trim & string-replace (or skipping them in > some cases) will get under the Ruby time. > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "Racket Users" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email to racket-users+unsubscr...@googlegroups.com. > >> For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google > Groups "Racket Users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to racket-users+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Racket performance tips
On Sunday, January 17, 2016 at 10:09:36 AM UTC-5, Jon Zeppieri wrote: > Oops: that final else was wrong. If all we encounter in the string is > whitespace, the result is the empty string, not the input string, so: > > > ;; === > > (require racket/unsafe/ops) > > > (define (string-trim s) > (define len (string-length s)) > > (let loop ([i 0]) > (cond [(unsafe-fx< i len) > (cond [(char-whitespace? (unsafe-string-ref s i)) > (loop (unsafe-fx+ i 1))] > [else > (let inner ([j (unsafe-fx- len 1)]) > (cond [(char-whitespace? (unsafe-string-ref s j)) > (inner (unsafe-fx- j 1))] > [else > (substring s i (unsafe-fx+ j 1))]))])] > [else > ""]))) > ;; === > > > > > > > > On Sun, Jan 17, 2016 at 1:24 AM, Jon Zeppieriwrote: > > > > > > On Sat, Jan 16, 2016 at 11:29 PM, Brian Adkins wrote: > > > I'm happy to run experiments and report timings though. > > > > > > > > Since the profile suggests that string-trim is the biggest culprit (followed > by fprintf), try using this specialized version of string-trim locally: > > > ;; === > > (require racket/unsafe/ops) > > > (define (string-trim s) > (define len (string-length s)) > > (let loop ([i 0]) > (cond [(unsafe-fx< i len) > (cond [(char-whitespace? (unsafe-string-ref s i)) > (loop (unsafe-fx+ i 1))] > [else > (let inner ([j (unsafe-fx- len 1)]) > (cond [(char-whitespace? (unsafe-string-ref s j)) > (inner (unsafe-fx- j 1))] > [else > (substring s i (unsafe-fx+ j 1))]))])] > [else > s]))) > ;; === > > > Instead of fprintf-ing the tabbed values, you might try (displayln > (string-join fields "\t")). Of course, that requires building a list of > strings, which has its own cost. With built-in string-trim, the lowest of three runs was 10293. Using your string-trim the lowest of three runs was 7618, so it reduced the runtime by 26%. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Racket performance tips
On Sunday, January 17, 2016 at 2:50:19 PM UTC-5, Brian Adkins wrote: > > With built-in string-trim, the lowest of three runs was 10293. Using your > string-trim the lowest of three runs was 7618, so it reduced the runtime by > 26%. Although, I probably should've mentioned that I'm not particularly interested in unsafe optimizations. I already have a very fast C program if I'm willing to risk unsafe behavior, so for Racket, I'd like to retain safety. Having said that, I'm pretty sure a combination of using Byte Strings and manually optimizing string-trim & string-replace (or skipping them in some cases) will get under the Ruby time. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Racket performance tips
> On Jan 17, 2016, at 2:50 PM, Brian Adkinswrote: > > With built-in string-trim, the lowest of three runs was 10293. Using your > string-trim the lowest of three runs was 7618, so it reduced the runtime by > 26%. Would converting this into a `bytes-trim` function that only works with byte-strings help more? -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Racket performance tips
MRI (the main ruby interpreter) has an odd string representation that's optimized for shorter strings. There's some info here: [ http://patshaughnessy.net/2012/1/4/never-create-ruby-strings-longer-than-23-characters]. The type is defined here: [ https://github.com/ruby/ruby/blob/af18eafc44bb3bb6aff78f244a67b807500e3e9f/include/ruby/ruby.h#L979 ]. -J On Sun, Jan 17, 2016 at 3:08 PM, Robby Findlerwrote: > Do we know if ruby represents strings the same way Racket does? The > representation in C clearly admits more efficient implementations of > relevant operations here, and Ruby's might too. > > Robby > > > On Sun, Jan 17, 2016 at 2:00 PM, Jon Zeppieri wrote: > > My string-trim uses unsafe ops, but I'm pretty sure it's safe. The > (safe) string-length at the start ensures we're using a string. The rest > are indexing and fixnum arithmetic on integers that are guaranteed to be > valid indices of the string. > > > > Still, if you don't like this, replace the unsafe ops with the > corresponding safe ones. It will still be much faster than the built-in > version. > > > >> On Jan 17, 2016, at 2:54 PM, Brian Adkins > wrote: > >> > >>> On Sunday, January 17, 2016 at 2:50:19 PM UTC-5, Brian Adkins wrote: > >>> > >>> With built-in string-trim, the lowest of three runs was 10293. Using > your string-trim the lowest of three runs was 7618, so it reduced the > runtime by 26%. > >> > >> Although, I probably should've mentioned that I'm not particularly > interested in unsafe optimizations. I already have a very fast C program if > I'm willing to risk unsafe behavior, so for Racket, I'd like to retain > safety. > >> > >> Having said that, I'm pretty sure a combination of using Byte Strings > and manually optimizing string-trim & string-replace (or skipping them in > some cases) will get under the Ruby time. > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "Racket Users" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email to racket-users+unsubscr...@googlegroups.com. > >> For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google > Groups "Racket Users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to racket-users+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Racket performance tips
Do we know if ruby represents strings the same way Racket does? The representation in C clearly admits more efficient implementations of relevant operations here, and Ruby's might too. Robby On Sun, Jan 17, 2016 at 2:00 PM, Jon Zeppieriwrote: > My string-trim uses unsafe ops, but I'm pretty sure it's safe. The (safe) > string-length at the start ensures we're using a string. The rest are > indexing and fixnum arithmetic on integers that are guaranteed to be valid > indices of the string. > > Still, if you don't like this, replace the unsafe ops with the corresponding > safe ones. It will still be much faster than the built-in version. > >> On Jan 17, 2016, at 2:54 PM, Brian Adkins wrote: >> >>> On Sunday, January 17, 2016 at 2:50:19 PM UTC-5, Brian Adkins wrote: >>> >>> With built-in string-trim, the lowest of three runs was 10293. Using your >>> string-trim the lowest of three runs was 7618, so it reduced the runtime by >>> 26%. >> >> Although, I probably should've mentioned that I'm not particularly >> interested in unsafe optimizations. I already have a very fast C program if >> I'm willing to risk unsafe behavior, so for Racket, I'd like to retain >> safety. >> >> Having said that, I'm pretty sure a combination of using Byte Strings and >> manually optimizing string-trim & string-replace (or skipping them in some >> cases) will get under the Ruby time. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Racket Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to racket-users+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Store value with unsupported type in Postgres?
How about: (query-exec conn (format "INSERT INTO some_table (ip) VALUES (inet '~a')" client-ip)) On Sun, Jan 17, 2016 at 6:35 PM, Alexis Kingwrote: > The DB docs for SQL type conversions[1] note that not all Postgres types > are supported by Racket, and it recommends using a cast to work around > this. It even uses the inet type as an example right at the start of the > page. However, I want to store an inet value in my database, not query > an inet value out, and I can’t figure out what set of casts I need for > the right coercion to take place. > > My query looks like this: > > (query-exec conn "INSERT INTO some_table (ip) VALUES ($1)" client-ip) > > Attempting to execute that query gives me the following error: > > query-exec: unsupported type > type: inet > typeid: 869 > > Is there any way to annotate this so that I can insert into that table? > > [1]: http://docs.racket-lang.org/db/sql-types.html > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Store value with unsupported type in Postgres?
You can cast first to a supported type: (query-exec conn "INSERT INTO some_table (ip) VALUES (inet ($1 ::text))" client-ip) On 2016-01-17 7:35 PM, Alexis King wrote: I would like to avoid interpolating into a query if at all possible, given that this string is not something I control. I could be very careful about validating or sanitizing it, but this is a pretty textbook use case for parameterized queries. On Jan 17, 2016, at 16:19, Jon Zeppieriwrote: How about: (query-exec conn (format "INSERT INTO some_table (ip) VALUES (inet '~a')" client-ip)) -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Racket performance tips
On Sunday, January 17, 2016 at 2:54:39 PM UTC-5, Brian Adkins wrote: > On Sunday, January 17, 2016 at 2:50:19 PM UTC-5, Brian Adkins wrote: > > > > With built-in string-trim, the lowest of three runs was 10293. Using your > > string-trim the lowest of three runs was 7618, so it reduced the runtime by > > 26%. > > Although, I probably should've mentioned that I'm not particularly interested > in unsafe optimizations. I already have a very fast C program if I'm willing > to risk unsafe behavior, so for Racket, I'd like to retain safety. > > Having said that, I'm pretty sure a combination of using Byte Strings and > manually optimizing string-trim & string-replace (or skipping them in some > cases) will get under the Ruby time. Yay - success! :) I changed all strings to byte strings while leaving the style of the code very similar. It made a significant difference. Of course, I also gained the benefit if handwritten bytes-split, bytes-replace, bytes-delete, bytes-trim, etc. which were narrowly defined just for this app. Timings on a 200K line file are now: Ruby = 7.53s Racket = 2.52s (was 10.3s) The string version of the Racket program was over 4x slower. I'm quite satisfied with being 3x faster than Ruby with a similar coding style given this is really in Ruby's sweet spot i.e. text munging. New code is here: https://gist.github.com/lojic/892049e617637903f982 I think my next step will be to create a version that uses places. make-shared-bytes may be useful for that. I'll report back with timings. I have a 4 core macbook pro w/ 8 hyperthreads - no idea whether the hyperthreads are actually useful, but if I can get a 3x speedup with 4 cores, I'd be pretty pleased. I suppose the following is a reasonable architecture: 1 place for reading the input file and placing a record in a input queue N places (one per core) to read from the input queue, process and place in an output queue 1 place for reading the output queue and writing to either of two output files -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[racket-users] Store value with unsupported type in Postgres?
The DB docs for SQL type conversions[1] note that not all Postgres types are supported by Racket, and it recommends using a cast to work around this. It even uses the inet type as an example right at the start of the page. However, I want to store an inet value in my database, not query an inet value out, and I can’t figure out what set of casts I need for the right coercion to take place. My query looks like this: (query-exec conn "INSERT INTO some_table (ip) VALUES ($1)" client-ip) Attempting to execute that query gives me the following error: query-exec: unsupported type type: inet typeid: 869 Is there any way to annotate this so that I can insert into that table? [1]: http://docs.racket-lang.org/db/sql-types.html -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Store value with unsupported type in Postgres?
Ah, that is a better idea. > On Jan 17, 2016, at 7:39 PM, Marc Burnswrote: > > You can cast first to a supported type: > > (query-exec conn "INSERT INTO some_table (ip) VALUES (inet ($1 ::text))" > client-ip) > >> On 2016-01-17 7:35 PM, Alexis King wrote: >> I would like to avoid interpolating into a query if at all possible, >> given that this string is not something I control. I could be very >> careful about validating or sanitizing it, but this is a pretty textbook >> use case for parameterized queries. >> >>> On Jan 17, 2016, at 16:19, Jon Zeppieri wrote: >>> >>> How about: (query-exec conn (format "INSERT INTO some_table (ip) VALUES >>> (inet '~a')" client-ip)) > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Store value with unsupported type in Postgres?
I sympathize, but using a prepared statement parameter requires support for the type of the parameter. If the library doesn't support it, you'll need to use strings (and escape them appropriately, though it looks like the library doesn't provide a string-escaping function), or else patch the library to provide support. > On Jan 17, 2016, at 7:35 PM, Alexis Kingwrote: > > I would like to avoid interpolating into a query if at all possible, > given that this string is not something I control. I could be very > careful about validating or sanitizing it, but this is a pretty textbook > use case for parameterized queries. > >> On Jan 17, 2016, at 16:19, Jon Zeppieri wrote: >> >> How about: (query-exec conn (format "INSERT INTO some_table (ip) VALUES >> (inet '~a')" client-ip)) > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Store value with unsupported type in Postgres?
Perfect, that works great, thank you! It looks like the precedence works out such that I could do $1::text::inet and have it work properly, which is clean enough for my needs. > On Jan 17, 2016, at 16:39, Marc Burnswrote: > > You can cast first to a supported type: > > (query-exec conn "INSERT INTO some_table (ip) VALUES (inet ($1 ::text))" > client-ip) -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Store value with unsupported type in Postgres?
I would like to avoid interpolating into a query if at all possible, given that this string is not something I control. I could be very careful about validating or sanitizing it, but this is a pretty textbook use case for parameterized queries. > On Jan 17, 2016, at 16:19, Jon Zeppieriwrote: > > How about: (query-exec conn (format "INSERT INTO some_table (ip) VALUES (inet > '~a')" client-ip)) -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[racket-users] How can I build up the #:usage-help string dynamically in racket lang's (command-line …) function?
Hi All, (I'm cross-posting this from StackOverflow) http://stackoverflow.com/q/34837318/24172 I have this code: (define s1 "aoeu") (define command (command-line #:usage-help s1 #:args (op) op)) It fails with "command-line: #:usage-help clause contains non-string." If I replace the reference to s1 with an actual string (eg "aoeu") then it works just fine. I would like to build up that string (s1 in this example) dynamically, but I can't figure out how to do that. P -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] How can I build up the #:usage-help string dynamically in racket lang's (command-line …) function?
I think you'll need to use parse-command-line, instead of command-line. -J On Sun, Jan 17, 2016 at 7:51 AM, Pieter Breedwrote: > Hi All, > > (I'm cross-posting this from StackOverflow) > http://stackoverflow.com/q/34837318/24172 > > I have this code: > > (define s1 "aoeu") > > (define command > (command-line >#:usage-help s1 >#:args (op) op)) > > It fails with "command-line: #:usage-help clause contains non-string." > > If I replace the reference to s1 with an actual string (eg "aoeu") then it > works just fine. I would like to build up that string (s1 in this example) > dynamically, but I can't figure out how to do that. > > P > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.