Hi Matthew, Running the seeder in parallel to incoming user requests is definitely something that wasn't planned for, and may potentially end up in corrupted tiles being written to the caches. The heart of the problem is that the seeder deletes the existing lockfiles at startup as it expects it will be the only instance creating tiles; clearly this isn't the case if there are other seeding instances running, or if live web users are requesting tiles. The reason the locks are deleted is to avoid a deadlock when old lockfiles are still present after e.g. a server crash.
I'd say the risk of tile corruption is pretty low, however the error messages (another thread/process failed to create the tile ...) are to be regularly expected (namely each time you launch a seeder process when a lockfile created by a web-user request is present). I'd say your options are - to live with the small risk of tile corruption, and the fact that your users will be receiving transient error messages, as is the case today - to trivially patch mapcache to deactivate the lockfile deletion when launching a seeder. note that in that case this will end up in a deadlock if there where leftover lockfiles, and they would need to be manually deleted. - to sponsor the implementation of some kind of lockfile deadlock detection/recovery mechanism, either by checking that the lockfile creator is still alive, and/or that the lockfile creation date is more recent than a certain threshold. cheers, thomas On 18 February 2013 08:09, Matthew Doyle <[email protected]> wrote: > Hi Thomas and MapCache users, > > Is there a way to raise the limit on the number of lockfiles which MapCache > uses by default? > > I am seeding a *lot* of layers twice a day, and when there are many parallel > processes at once, i get a steady stream of messages in my apache error_log > like: > > [Mon Feb 18 06:16:14 2013] [error] [client 134.178.63.147] found old > lockfile /tiletmp/scratch/_gc_lock3-4-26-ADFDWEATHER1820130218061203.lck, > deleting itfound old lockfile > /tiletmp/scratch/_gc_lock3-4-28-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock3-5-29-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock3-4-27-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock3-4-29-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock3-5-26-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock3-6-27-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock0-0-0-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock2-1-6-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock1-0-2-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock3-5-28-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock3-5-27-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock3-5-25-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock3-6-26-ADFDWEATHER1820130218061203.lck, deleting > itfound old lockfile > /tiletmp/scratch/_gc_lock2-1-7-ADFDWEATHER1820130218061203.lck, deleting it > > Subsequently, i also see a lot of messages like: > [Mon Feb 18 06:17:08 2013] [error] [client 134.178.63.167] tileset > ADFDWEATHER: unknown error (another thread/process failed to create the tile > I was waiting for), referer: http://www.bom.gov.au/australia/meteye/ > when a user tries to access a tile which is in the process of being seeded. > > Is this in any way related to a limit on the lockfiles? Is it possible to > avoid these errors, or are they false positives and something to ignore? > > Regards, > Matt > > _______________________________________________ mapserver-users mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/mapserver-users
