Hi All, My question relates to how the downloader module knows whether or not to download pygr resources when local resources are present (have been previously downloaded). Some background:
A few days ago I was doing some development work with the data at Bio.MSA.UCSC.anoCar1_pairwiseGasAcu1. I decided that rather than connect to UCSC each time I was developing, that I would cache the data locally by specifying the download=True option when I construct my resource. This worked fine, and the downloader grabbed a couple of GB of data. Further development that day was a lot quicker, since the data was sitting on my local workstation. I found that I could still leave the download=True option there, and pygr seemed to magically know that the data was local and did not in fact need to be downloaded. So today, after a weekend away, I have come back to work on the same project, and I found that the same construction line does indeed cause pygr to attempt to download the data again - a lengthy process I don't want to go through again if I don't have to. Furthermore it overwrites the .gz files it downloaded previously (luckily I had backups). I had not deleted any files or moved any of pygr's data "out of the way", although I did do a directory rename higher up the directory tree, which may or may not cause pygr to think something has changed if it hashes the cwd() for example. I haven't looked at the code - perhaps I should :) I thought it might be quicker to just ask the experts. My question is: How does pygr make the decision on whether or not to download resources? Do local resources automatically expire after a certain amount of time? Is it possible that there was some sort of update to the UCSC data that has caused me to need to redownload it? Or has my upstream directory name change inadvertently triggered some internal pygr check and caused it to want to download fresh data?. I did notice the .pygr_data binary file, for example, which makes me think that some sort of metadata is being stored about the state of the downloads. Hopefully someone has a quick answer for me, otherwise it is time to take a look at the downloader source ;) Thanks all -- You received this message because you are subscribed to the Google Groups "pygr-dev" group. To post to this group, send email to pygr-...@googlegroups.com. To unsubscribe from this group, send email to pygr-dev+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pygr-dev?hl=en.
