Hi Steve, On 26 February 2013 17:34, Stephen Woodbridge <[email protected]> wrote: > Hi Thomas, > > Does the seeder delete the lock files once on startup? > If so then would it be safe to stop apache, start the seeder, then restart > apache and allow the seeder and apache to generate tiles? Unfortunately no, as the apache instance also deletes existing lockfiles for the same reason. In practice this *would* however help a bit as the seeder is never supposed to lock on a lock it has created itself, however you might still end up with corruptions in the case where multiple seeding instances have been launched. Given Matthew's usecase, I also suspect that restarting apache is not an acceptable behavior.
cheers, -- thomas > > Thanks, > -Steve W > > > On 2/26/2013 11:29 AM, thomas bonfort wrote: >> >> Hi Matthew, >> >> Running the seeder in parallel to incoming user requests is definitely >> something that wasn't planned for, and may potentially end up in >> corrupted tiles being written to the caches. The heart of the problem >> is that the seeder deletes the existing lockfiles at startup as it >> expects it will be the only instance creating tiles; clearly this >> isn't the case if there are other seeding instances running, or if >> live web users are requesting tiles. The reason the locks are deleted >> is to avoid a deadlock when old lockfiles are still present after e.g. >> a server crash. >> >> I'd say the risk of tile corruption is pretty low, however the error >> messages (another thread/process failed to create the tile ...) are to >> be regularly expected (namely each time you launch a seeder process >> when a lockfile created by a web-user request is present). I'd say >> your options are >> >> - to live with the small risk of tile corruption, and the fact that >> your users will be receiving transient error messages, as is the case >> today >> - to trivially patch mapcache to deactivate the lockfile deletion when >> launching a seeder. note that in that case this will end up in a >> deadlock if there where leftover lockfiles, and they would need to be >> manually deleted. >> - to sponsor the implementation of some kind of lockfile deadlock >> detection/recovery mechanism, either by checking that the lockfile >> creator is still alive, and/or that the lockfile creation date is more >> recent than a certain threshold. >> >> cheers, >> thomas >> >> On 18 February 2013 08:09, Matthew Doyle <[email protected]> wrote: >>> >>> Hi Thomas and MapCache users, >>> >>> Is there a way to raise the limit on the number of lockfiles which >>> MapCache >>> uses by default? >>> >>> I am seeding a *lot* of layers twice a day, and when there are many >>> parallel >>> processes at once, i get a steady stream of messages in my apache >>> error_log >>> like: >>> >>> [Mon Feb 18 06:16:14 2013] [error] [client 134.178.63.147] found old >>> lockfile /tiletmp/scratch/_gc_lock3-4-26-ADFDWEATHER1820130218061203.lck, >>> deleting itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-4-28-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-5-29-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-4-27-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-4-29-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-5-26-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-6-27-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock0-0-0-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock2-1-6-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock1-0-2-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-5-28-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-5-27-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-5-25-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock3-6-26-ADFDWEATHER1820130218061203.lck, deleting >>> itfound old lockfile >>> /tiletmp/scratch/_gc_lock2-1-7-ADFDWEATHER1820130218061203.lck, deleting >>> it >>> >>> Subsequently, i also see a lot of messages like: >>> [Mon Feb 18 06:17:08 2013] [error] [client 134.178.63.167] tileset >>> ADFDWEATHER: unknown error (another thread/process failed to create the >>> tile >>> I was waiting for), referer: http://www.bom.gov.au/australia/meteye/ >>> when a user tries to access a tile which is in the process of being >>> seeded. >>> >>> Is this in any way related to a limit on the lockfiles? Is it possible to >>> avoid these errors, or are they false positives and something to ignore? >>> >>> Regards, >>> Matt >>> >>> >> _______________________________________________ >> mapserver-users mailing list >> [email protected] >> http://lists.osgeo.org/mailman/listinfo/mapserver-users >> > > _______________________________________________ > mapserver-users mailing list > [email protected] > http://lists.osgeo.org/mailman/listinfo/mapserver-users _______________________________________________ mapserver-users mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/mapserver-users
