On 2018-09-21, Raphaël Hertzog wrote: > Version 0.6.5 introduced a checksum check based on the data from > the "extrafiles" file at the root of the mirror. Now when that > file doesn't exist, simple-cdd fails with a stacktrace and is > unable to build any image. > > Arguably, the lack of this file is not a showstopper, it should > just generate a warning... and effectively there are many custom > Debian repositories without this file and you still want to be able > to run simple-cdd on them.
Before simple-cdd used extrafiles, it would blindly download
debian-installer files from the mirror with no verification, and
recursively get entire directory trees...
Using extrafiles enables a signed trust path to checksums of various
non-package files, and conveniently lists the files available to
download on the archive as an added bonus.
So it's non-trivial to add support for arbitrary files in arbitrary
directories in a secure manner...
> In my specific case, the Debian mirror is created with "debmirror"
> and this tool doesn't include that file. But I could also mention the
> case of many derivatives that just use reprepro.
Maybe these other tools could add support for extrafiles?
It's unfortunate that it may not work in all environments, though
simple-cdd has always targeted building images with files from
debian.org, and not arbitrary locations.
A patch to enable support without extrafiles would, of course, be
considered if it didn't risk degrading the trust path by default.
> Also it would be nice if simple-cdd documented somewhere its requirements for
> the mirror and repositories that it can use.
That's surely doable.
> FTR here's the stacktrace:
> 2018-09-18 14:36:26,005 DEBUG Building local Debian mirror for debian-cd...
> 2018-09-18 14:36:26,007 DEBUG downloading: .../tmp/mirror/extrafiles
> Traceback (most recent call last):
> File "/usr/bin/build-simple-cdd", line 658, in <module>
> scdd.build_mirror()
> File "/usr/bin/build-simple-cdd", line 270, in build_mirror
> self.run_tool("mirror", tool)
> File "/usr/bin/build-simple-cdd", line 367, in run_tool
> tool.run()
> File "/usr/lib/python3/dist-packages/simple_cdd/tools/mirror_wget.py", line
> 64, in
> run
> _download(download_extrafiles_file, extrafiles_file_inlinesig)
> File "/usr/lib/python3/dist-packages/simple_cdd/tools/mirror_wget.py", line
> 55, in
> _download
> request.urlretrieve(url, filename=output)
> File "/usr/lib/python3.5/urllib/request.py", line 188, in urlretrieve
> with contextlib.closing(urlopen(url, data)) as fp:
> File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
> return opener.open(url, data, timeout)
> File "/usr/lib/python3.5/urllib/request.py", line 472, in open
> response = meth(req, response)
> File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
> 'http', request, response, code, msg, hdrs)
> File "/usr/lib/python3.5/urllib/request.py", line 510, in error
> return self._call_chain(*args)
> File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
> result = func(*args)
> File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
> raise HTTPError(req.full_url, code, msg, hdrs, fp)
> urllib.error.HTTPError: HTTP Error 404: Not Found
Yeah, simple-cdd should at least handle this rather than spitting out a
backtrace.
Thanks for the report, sorry I don't have better news for this issue!
live well,
vagrant
signature.asc
Description: PGP signature
