On Wednesday, January 20, 2016 at 11:28:59 PM UTC-5, Brian Adkins wrote: > My initial experiment with places is a bit disappointing: > > Sequential version: cpu time: 2084 real time: 2091 gc time: 91 > > Places version: cpu time: 16895 real time: 3988 gc time: 4244 > > Using 8x the CPU time seems quite high. > > And more importantly, the places version only wrote 128,541 lines to the > output file vs. the correct number of 198,480 (65%). I suspect the program is > ending while some of the worker places are still active, but I think that > implies that if I correctly waited for all activity to finish, the numbers > would be even worse. Putting a sleep after the main input reading loop gets > me all but 5 of the records. > > My simplistic design was to have the main place read lines from an input > file, write the lines to the place channels of N workers for processing (in a > round robin manner using modulo # workers). The workers write the parsed > lines to a single output place for writing to the output file. > > Is it possible to limit the number of messages on a place channel? I suspect > the input place is moving faster than the workers, so the workers' place > channels may be getting huge. > > Someone mentioned buffered asynchronous channels on IRC since you can set a > limit, but it appears you can't send them across a place channel, so I'm > unsure how to make use of them with places. > > Sequential & parallel code is here: > > https://gist.github.com/lojic/283aa3eec777e4810efc > > Relevant lines are lines 44 to 105 of the parallel version. > > Are there any projects, papers, etc. of best practices with respect to places > that I can look at? > > Thanks, > Brian
FYI - after some trial and error, I was able to specify a sleep time of ~ 500ms after the input file has been read to allow sufficient time for the workers to write to the output place and the output place to populate the file (minus ~ 5 records). The output from time for various number of workers is: 2 workers => cpu time: 9396 real time: 3279 gc time: 2233 3 workers => cpu time: 11835 real time: 3543 gc time: 2137 4 workers => cpu time: 17078 real time: 4461 gc time: 4234 5 workers => cpu time: 21610 real time: 4988 gc time: 5366 I have 4 cores and 8 hyperthreads, but adding workers just means more CPU time and more elapsed time :( -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
