I have not yet done much investigating on this, but: - on Racket BC, operations like `+` do indeed block, and effectively you need to replace them with lower-level operations that don't (such as `unsafe-fl+`). Typed Racket can help with this, or you can do it all by hand. As you note, that makes the code more painful to read. - on Racket CS, operations like `+` do not block, and I see much better speedup. I changed the third range to 720-800 to get answers quicker, and I got numbers like:
[samth@homer:/tmp/cp3-exeriments (master) plt] racketcs main.rkt cp3-baseline: cpu time: 1947 real time: 1947 gc time: 10 cp3-precomputed: cpu time: 399 real time: 399 gc time: 4 cp3-precomputed-more: cpu time: 475 real time: 475 gc time: 3 cp3-futures: cpu time: 4285 real time: 740 gc time: 11 cp3-precomputed-futures: cpu time: 785 real time: 138 gc time: 3 cp3-precomputed-more-futures: cpu time: 876 real time: 153 gc time: 4 So a more than 2x increase in cpu time, but a more than 2x decrease in wall-clock time. Certainly more investigation is needed to figure out why things take so much longer total, but this seems like a promising speedup. - The futures-visualizer uses logging and a functional graphics library, both of which will allocate a lot more. You can use `trace-futures` and `show-visualizer` to separate out the gui display from execution, which might help. Sam On Wed, Jun 17, 2020 at 4:50 AM Alex Harsanyi <alexharsa...@gmail.com> wrote: > > > I am trying to speed up an algorithm using futures, but I am getting some > unexpected results (and no real speed improvements), and I was wondering if > someone more experienced could have a look a the code and tell me what am I > doing wrong. > > I put up the code in this repository: > https://github.com/alex-hhh/cp3-exeriments, unfortunately it is the simplest > meaningful example that I can come up with. Most of the functions, however > are just support functions and there are six implementation of the same > algorithm. > > Basically, the problem I am trying to solve is to fit a model to existing > data and this is done by evaluating 2.5 million combinations of parameters. > My best, non-futures based, algorithm can do this in about 3 seconds (8 > seconds in DrRacket). > > Given that each of this 2.5 million combination is completely independent > from the others, they could all be done in parallel. Given this, I "sliced" > the combinations into 30 groups and tried to perform each "slice" in its own > future and select the best among the 30 results produced by these futures. > > Unfortunately, while the futures versions of the algorithm produce the > correct result, the algorithm runs at the same speed as the non-futures > version. My `processor-count` returns 8, so I would expect at least some > level of parallelism. > > As a first step, I tried using `would-be-future`, to see if it reported any > operations which might block, but nothing was printed out. > > I also tried using the futures visualized, and I found the following: > > * the code appears to be blocking on primitive operations, such as +, -, < > etc. > * I suspect these operations are inside the code generated by the `for` > loops, so I am not sure how to remove them without making the code even more > difficult to read. > * there seems to be a lot more time spent in the garbage collector when > running the futures visualizer than without it (DrRacket runs with unlimited > memory) > > So I am wondering if someone who is more familiar with futures can look at > the code and provide some hints about what can be done to make this code run > in parallel (or if it cannot, I would like to understand why). > > This is already a long message, so I will not add further details here, but > the repository at https://github.com/alex-hhh/cp3-exeriments has an > explanation of what every function does, and I am happy to provide further > clarifications if needed. > > Thanks, > Alex. > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/racket-users/8bf6f7c4-3b2f-4b86-9a8a-be68e82d09cfo%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/CAK%3DHD%2BaKpxBhaRs64yfmats3j_V5kF7KTe3Mb4OsgUpaDvk_WQ%40mail.gmail.com.
