I'm getting a little antsy to land this. It's a pretty invasive series and I'd like to land it sooner rather than later to save myself rebasing pains.
On Friday, September 26, 2014 09:28:17 PM Dylan Baker wrote: > bump. > > On Tuesday, September 23, 2014 05:55:47 PM Dylan Baker wrote: > > For a long time piglit has had a considerable problem with resuming > > interrupted runs. This problem comes from the recovery tool, a very > > fragile piece of code that relies on the formatting of the json to walk > > backwards throwing away code. The problem with this approach are two > > fold, first if the indentation changes the tool breaks. Second, it can > > only handle corruption at the end of the file. > > > > There is a second problem with our resume method, and that is that the > > json file is read, truncated, and completely re-written on every resume, > > this can be quite expensive. > > > > Initially I began writing a complicated tool that actually could parse > > json and clean it, but this quickly became a significant amount of code, > > and the complexity was overwhelming, it was a complete project in > > itself. > > > > What I have implemented is a much simpler idea, that I borrowed from > > databases, Atomic writes. By atomic we mean that a transaction should > > either completely succeed or completely fail. Using a file backend there > > are two basic ways to go about doing this. One method Copy the file to a > > temporary location, write to the temporary file, verify the results, and > > copy it back over the original. This has a couple of distinct drawbacks. > > First it requires a lot of copying, second it means that the file needs > > to be locked for long periods, and finally it requires reading the file. > > > > What I implemented instead is a system that writes each test to it's own > > file using built-in methods for both json and xml. At the end the runner > > reads in each of these files, and creates a combined file from the > > results. This has advantages and drawbacks. Because each test gets it's > > own file there is no locking required at all, and flushing and syncing > > is cheaper. It also means that there is no need to verify writes, when > > the final file is being assembled json.load and xml.ElementTree.parse > > are called on each file, and if they fail the result is considered > > failed and ignored. > > > > This series also takes a few more steps toward abstracting backends, by > > creating a separate package for backends, and including a small subset > > of the classes and functions from the package in the __init__ module, > > allowing them to be accessed as if backends itself was a module. This is > > sufficient for most applications, with mainly unit tests needing to > > bypass this and access the modules directly. > > > > Patch 1 removes a broken executable script that has been broken for some > > time > > Patch 2 does most of the abstraction > > Patches 3-6 do some code cleanup > > Patches 7 and 8 implement atomic writes in the two backends > > Patch 9 does some additional refactoring that is possible becasue of > > patches 7 and 8 > >
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Piglit mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/piglit
