Hi akka users, I'm new to scala/akka/JVM platform, and frankly I had a hard time since I tried to implement a simple web crawler using actor model with akka. In detail, this crawler reads a list of URLs from a local file, dispatch each URL to workers from master actor, and finally return webpage content to master. Seems to be a very simple project.
At first I ran my crawler with 3 workers and feed them with dozens of URLs and the crawlers runs fine except some 403 errors. Then I applied the crawler to a 5k-lines-of-url with 1000 workers, the master actor can't print "acknowledge" in its receive function as well in 3-worker situation. Is this because of the master actor had been flooded? And how to solve this? Regards, Todd -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
