Hi, Alex has reported a thread-safety issue (issue 79) that we should consider as a general problem for Pygr. Currently, there are a number of places in Pygr that are not thread-safe (i.e. running multiple threads may lead to error messages in these parts of Pygr). Specifically, Pygr uses caching on a variety of "database interfaces" (e.g. SequenceDB, SQLTable, various places in metabase etc.). To make these caches thread-safe would require adding locking to protect the pieces of code that change or lookup the cache's content (having two threads trying to change the cache at the same time would cause bugs). This seems pretty straightforward to add, but is something we haven't discussed for Pygr.
So: should we make Pygr thread-safe? Should we fix this as part of the 0.8 release? How important is multi-threading for bioinformatics users? Do you think I am underestimating the difficulty of making Pygr thread-safe (i.e. my proposal to just put locks around cache lookup / update operations)? This came up now because Alex's requested feature (run multiple sequences as a single blast execution) uses threading, and as a result exposed the lack of thread-safety in one of our cache classes. The only reason to use a thread here is that we use two-way pipes. That is, we are both reading and writing pipes to / from a separate process (blastall); with a large volume of sequence there is the possibility of a read-write deadlock (i.e. our write operation blocks because it has filled the write buffer, but blastall blocks also because its write buffer has filled too (because we aren't continuing to read the buffer, since we're blocked in our write operation). For more details, see http://code.google.com/p/pygr/issues/detail?id=79 A trivial way around this whole issue is to use a temporary file for sending our data to blastall (instead of a pipe). Thus, another choice is to say we will not make Pygr 0.8 thread-safe, and to simply convert two-way pipes (such as used by process_blast()) to use a temporary file instead of the write pipe. -- Chris --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pygr-dev" group. To post to this group, send email to pygr-dev@googlegroups.com To unsubscribe from this group, send email to pygr-dev+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pygr-dev?hl=en -~----------~----~----~----~------~----~------~--~---
