Did anyone try this yet? I haven’t had chance but would very much like to get this fixed as it’s the number one bug in biber right now due to such disappearing cache files.
PK > On 20 Jan 2015, at 6:58 am, Shawn Laffan <shawn.laf...@unsw.edu.au> wrote: > > A candidate patch for this issue is now ready for review. > > https://github.com/shawnlaffan/perl_par_reinstate_cache/blob/master/canary.patch > > Key points are more or less as per my previous email (below) except that I > have added a sub called PAR::get_canary_file_name to avoid any hard coding. > I have also decreased the verbosity threshold for packing files into /inc, as > previously they needed double verbosity flags in pp to trigger. > > Thoughts? > > > Regards, > Shawn. > > > > On 21/12/2014 21:05, Shawn Laffan wrote: >> I've had a go at implementing the rearguard approach, albeit with the >> working name PAR_CANARY.txt instead of REARGUARD.txt. >> >> It's still in progress, but if people would like to comment or critique then >> changes are at https://github.com/shawnlaffan/perl_par_reinstate_cache >> >> Some key points are: >> >> 1. The approach I have taken is to add a method >> PAR::Packer::_add_canary_file, but possibly it should simply be appended to >> the -a file list. >> >> 2. Name clashes are unlikely, but perhaps the canary file name needs to be >> a bit more unique, or maybe it can be located in the script folder instead >> of the top level of the inc dir. >> >> 3. There also needs to be a method to access the canary file name as it is >> currently hard coded in both PAR::Packer and PAR.pm, but I'm not sure where >> best to locate that. >> >> 4. The test file is under the Par-Packer folder, >> t/21-pp_reinstate_cached_files.t. It could perhaps later be merged into >> 20-pp.t since it copies large amounts of code from that file. It needs >> "perl Makefile.PL; make" to have been run so it can access the relevant >> files. >> >> >> I also included the added files in the manifest under >> https://github.com/shawnlaffan/perl_par_reinstate_cache/commit/6cad3a0562aca287834a8b926ce2ed3f9b7552e1, >> assuming that was a bug as Roderich suggested. >> >> >> >> Regards, >> Shawn. >> >> >> >> On 19/12/2014 22:57, Roderich Schupp wrote: >>> >>> On Thu, Dec 18, 2014 at 10:22 PM, Shawn Laffan <shawn.laf...@unsw.edu.au> >>> wrote: >>> >>> The extractions currently use Archive::Unzip::Burst as the first attempt. >>> I don't know its behaviour, but I assume that if some files exist and are >>> locked then it will fail. >>> >>> It's also protected by the $inc.lock "mutex", just like the slow path >>> (using Archive::Zip). >>> >>> In that event the approach switches to iterating over $zip->memberNames, so >>> it will still work. (Actually, Archive::Unzip::Burst seems not to install >>> on Windows, so the latter approach will be the norm on that OS). >>> >>> Probably very few people use it. It's just a Perl small wrapper around >>> InfoZip unzip. >>> >>> Of course, this won't fix the issues where files are cleaned up while a PAR >>> is running process, >>> >>> Obviously we will only re-extract *missing* files, so no problem with >>> locked files here. >>> >>> so some sort of API sub would still be useful to allow scripts to >>> re-extract if file opens fail. >>> >>> That would imply that you are able to know when opening a file from "deep >>> inside third party modules" fails - how? >>> >>> WRT the different cache areas, PAR::_extract_inc currently spends up to 10 >>> seconds trying to create a lock file, so that line needs to be modified. >>> >>> Nope. It's not a lock file, it's a lock *directory*. That's because >>> creation of a directory is the only portable (even network filesystem safe) >>> filesystem mutex operation. The up to 10 second delay comes only into play >>> in the contended case. >>> >>> Adding a check for -w on the target temp location should be enough to avoid >>> that when the exe file is in a non-writable directory. >>> >>> The parent directory of $inc in _extract_inc is writable by construction. >>> >>> Cheers, Roderich >>> >>> >> >> -- >> Assoc Prof Shawn Laffan >> School of Biological, Earth and Environmental Sciences >> UNSW, Sydney 2052, Australia >> Tel +61 2 9385 8093 Fax +61 2 9385 1558 >> >> http://www.bees.unsw.edu.au/staff/shawn-laffan >> >> >> http://www.purl.org/biodiverse >> (free diversity analysis software) >> >> http://www.tandf.co.uk/journals/ijgis >> >> >> UNSW CRICOS Provider Code 00098G >> > > -- > Assoc Prof Shawn Laffan > School of Biological, Earth and Environmental Sciences > UNSW, Sydney 2052, Australia > Tel +61 2 9385 8093 Fax +61 2 9385 1558 > > http://www.bees.unsw.edu.au/staff/shawn-laffan > > > http://www.purl.org/biodiverse > (free diversity analysis software) > > http://www.tandf.co.uk/journals/ijgis > > > UNSW CRICOS Provider Code 00098G > -- Dr Philip Kime
