Question on best practices on handling SQL database concurrency issues I am "pmap"ping a evaluation to a long list (which calls a computationally intense script) from within clojure. The script itself is designed to be completely free of concurrency side-effects. During the evaluation, several calculations are made that are then written to a SQLite database. My approach is to test for concurrency issues on a dual core system prior to moving to a cluster.
What I find is that on occasion there is a database locking issue when sub-processes try to write to the database at the same time (java.sql.SQLException: database is locked). The side effect is that one of the evaluations is not written to the database (bad, cause it takes 3min to compute). I can fix it by catching the exception, and then calling (Thread/sleep) before trying to rewrite again. This is an ugly fix, and I am concerned that this may not scale properly. What is the best practices to handle such an issue in a concurrent and scalable way? Is it as simple as moving to a better database, such as mySQL? I could use Threads/Locks and move the db transaction outside the evaluation loop, or save all transactions and then commit after all evaluations are done. I can't help but feel both solns seem like cheating when working with a conncurrent language such as clojure. Any Advice? I am using contrib.sql and java.sql (org.sqlite.JDBC) -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
