Having learned about agents recently, I've created a pretty contrived
program, which I believed would easily lend itself to parallel
processing. The program takes a list of MD5 sums and then does brute
force comparison the find the corresponding four character strings
they were generated from. The brute force character generation and
MD5'ing are CPU intensive, so I assumed I'd see a big performance
improvement by spawning one agent per CPU and then dividing the
workload evenly across them.
I'm testing on a quad-core box, and I'm seeing virtually identical
performance numbers switching between one agent and four agents. I've
also tried larger numbers like 12 agents, but the results are the
same. The load average also stays the same hovering around 1.0, and
the execution time is static at approximately 15 seconds. To make sure
Clojure, Java, and agents were at least working, I also tested a very
small program to just do an infinite loop per agent, and it instantly
maxed out all four CPUs, so I know agents work ;-).
(ns agentmd5
(:refer-clojure)
(:import
(java.security
NoSuchAlgorithmException
MessageDigest)
(java.math BigInteger)))
; computes an md5sum of a string
(defn md5-sum
"Compute the hex MD5 sum of a string."
[#^String str]
(let [alg (doto (MessageDigest/getInstance "MD5")
(.reset)
(.update (.getBytes str)))]
(try
(.toString (new BigInteger 1 (.digest alg)) 16)
(catch NoSuchAlgorithmException e
(throw (new RuntimeException e))))))
; starts at "aaaa" and rotates string based on `n'
; "aaaa", "aaab", "aaac", etc...
(defn base26 [n]
(let [seed-string "aaaa"
s (new StringBuilder seed-string)
a_val (int \a)]
(loop [pos 3 x (int n)]
(when (pos? x)
(let [digit (char (+ a_val (unchecked-remainder x
26)))]
(.setCharAt s pos digit)
(when (pos? pos)
(recur (int (dec pos)) (unchecked-divide x
26))))))
(.toString s)))
; cycles through character combinations for a given md5 string
; until the matching result is found
(defn decode-md5 [md5]
(loop [x 0]
(let [gen-string (base26 x)]
(if (= md5 (md5-sum gen-string))
gen-string
(if (< x 456976) ; 26^4
(recur (inc x)))))))
; decodes a bucket
(defn decode [bucket]
(map decode-md5 bucket))
; returns a collection of agents for a given set of work buckets
(defn spawn-agents [agents buckets]
(if (empty? buckets)
agents
(recur (conj agents (agent (first buckets)))
(rest buckets))))
; number of tasks to doll out to the agents
(def num-work-units 60)
; generate requested number of work units
(def work-units (map md5-sum (for [x (range num-work-units)]
(base26 50000)))) ; (base26 50000)
== "cvzc"
; number of agents to spawn
(def num-agents 4)
; divide the units into buckets for each agent
(def work-buckets (partition (int (/ (count work-units)
num-agents)) work-
units))
; create an agent for each bucket
(def agents (spawn-agents [] work-buckets))
; send each agent a job to do
(doseq [agent agents]
(send agent decode))
; ensure successful dispatch
(apply await agents)
; view the results
(doseq [agent agents]
(doseq [result @agent]
(println result)))
; clean up
(shutdown-agents)
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---
