On 2011-10-05 10:45, Niels Thykier wrote:
> On 2011-09-13 18:04, Niels Thykier wrote:
>> Package: lintian
>> Severity: important
>>
>>
>> Jakub realized the source of a lot of our errors on lintian.d.o are
>> caused by limitations in the file-system. We should probably use
>> a pool or something similar to reduce the amount of elements in
>> each dirs.
>>
>> ~Niels
>>
>>
>>
>
> I guess it might be a good time for a little status update here.
>
Since no one has commented so far I have applied do-cracy and done stuff...
> The lab-refactor branch is now working for "simple" use cases[1].
> However, the lintian.d.o-style usage needs some attention.
>
> In the master branch we use $lab/info/* as a list of "what was in the
> mirror last time we checked". Those files have been repurposed in the
> lab-refactor branch, where their new meaning is "what is currently in
> the lab". This means that "dist" search[2] is currently broken.
> To my knowledge there are *2* known cases where "dist" searches make
> sense - lintian.d.o and lintian.debathena.o. I feel we should move that
> functionality to a new frontend (such as the "lintian-harness"[3]) that
> would focus lintian.d.o-like setups.
>
> Note that "repurposing" is not entirely complete and therefore
> reporting/harness is more or less broken right now. One of the issues
> is that unpack/* still use the files in info/* as a dist list and not a
> lab list.
>
"dist" search is now removed from lintian - the reporting stuff left
untouched and is therefore still broken. Yay for progress! :)
When I was looking at this, I realised two things. First a lot of
variables (and cmd-options) now appear to be redundant in
frontend/lintian. Namely all of LINTIAN_{ARCHIVEDIR,AREA,DIST} and
possibly also LINTIAN_ARCH. It has not been double checked, but I
strongly suspect them of being unused now.
Secondly, the current "search" rules were not sufficient. Basically, it
was only possible to match "all" packages with a given name. I have
solved this by creating a simple way of referring to packages in the Lab.
Originially I planned to accept both the current "britney-style"
format[1] and the "filename-style"[2]. However it occurred to me that
the "filename-style" is (for obvious reasons) impossible to reliably
distinguish from a normal file. As this could lead to confusion for the
users (i.e. principe of "least surprise"), I decided to not include the
"filename-style". The "britney-style" format is described in
man/lintian.pod.in.
[1] [type:]package[/version[/arch]]
[2] package_version[_arch].ext
>
> I also considered adding a file in info/ to keep track of lab-wide
> (meta)data, such as the lab-format. In the old lab format, this is
> stored in every entry. This makes is slightly more difficult to check
> if we are dealing with a compatible lab.
> Consider if you use an "old" lintian to use the new lab style - they
> do not store the entries the same place, so it has no reliable way to
> detect it is not compatible. I would prefer that an old lintian would
> always be able to say "The lab uses a newer lab-format that this version
> of lintian supports" - even if this case will "probably never happen".
>
I have added a lab-wide data file stored as "$LAB/info/lab-info". It
uses the deb822-style syntax and has two fields in the first paragraph:
"""
Lab-Format: $format
Layout: pool
"""
The Lab-Format field describes the current format of the lab[3]. The
Layout field describes how the packages are placed in the lab.
Currently only one layout exists (namely "pool"), which reflects the
layout in the current branch.
The Layout field allows us to implement and play with a new layout
side-by-side with the current one. Hopefully we will never need this
feature, but probably we will.
[3] Will be 11 when the development is done. Currently it is 10.1.
>
> I am also wondering what we need in the "per-entry" lintian-status file.
> In the master branch, we store Lintian-Version, Lab-Format, Package
> (name), Version (package), Type (package) and Timestamp.
> When we read the status file, we compare lab-format, package version
> and timestamp. With the changes in lab-refactor branch, the lab always
> supports multiple versions of the same package, thus the package version
> comparision is a no-op.
>
> As I understand it, the timestamp is there to make lintian "re-unpack"
> the package if it changed since the last run. Currently it completely
> removes the entry if the timestamp does not "match". Though this code
> only makes sense for "personal" static labs - on the lintian.d.o case,
> the version of a package can not be reused (at least not in general).
> The timestamp-part is not in the lab-refactor branch (yet?).
>
> I am considering to replace the "Lab-format" value with an
> "entry-format-version". Not sure it makes sense, but I thinking it may
> make migration to newer formats easier.
> If I had not (ab)used the oppertunity to do optimizations in the
> .lintian-status file (see below), the migration from the current to the
> lab-format would basically just have been a bunch of "mv X Y" + updating
> info/*.
>
I have decided to keep the redundant package information (package,
version and type), plus I added architecture (for non-source packages).
I realised that it may allows us to implement consistency checks later,
which is probably a "good thing".
The per-entry files also carry a new version, namely "Lab-Entry-Format".
This version describes how entries are stored and is "generally"
disjoined from the Layout and the Lab-Format.
The "Lab-Format" and the "Timestamp" field have been removed.
> [...]
>
> Any comments? If not I will (hopefully) get the branch ready to be
> merged into master within 2-3 weeks - so if you have not reviewed the
> branch yet, now would be a good time to start. :)
>
Yeah, so "2-3 weeks" is slightly off, but there is still progress.
> ~Niels
> [...]
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
