Hi everyone, a week or so ago me and Kim were talking about the distfiles mirror, and we began to drift to the topic of checksumming and how we could prevent bad downloads and such. Due to to it being 5:30 AM at the time, I believe I managed to come up with an idea of how to possibly work checksumming into our system in a way that doesn't interrupt our One True Workflow.
Essentially, we'd offload checksum generation to the mirror by adding a manifest generation to run_accerso.sh. Every day when accerso is ran on distfiles.e.o, it would do a few sanity checks on the fetched files, delete bad ones, and create a list of checksums which would be fetched at some point in time, perhaps sync, and then used during installation. A few obvious issues with this is how it would work with locally added packages (such as when bumping a package and testing it), and how it puts trust in the mirror to generate good checksums. The first issue could be taken care of by allowing fetching to continue with a warning or something that the integrity can't be checked, and the second... the second I don't believe is that much of an issue, because if the mirror has a bad checksum, then that means bad fetches, which means we have a problem anyway and someone needs to fix that package's bad DOWNLOADS. Benedikt proposed for the sanity checks, we could just check the file extension of what we've fetched against what the actual file type is; ex. `for file in *.tar.gz;do [[ $(file -b --mime-type) == application/x-gzip ]] || rm -v "$file";done` or something similar. That would be able to protect the checksum generation from at least things like badly configured servers serving 404 pages without sending an actual 404 status back to us. An actual IRC log is here for your enjoyment: http://ix.io/jNO What does the list think? -- Kylie McClain, Exherbo Linux developer and Musician https://somasis.com - https://github.com/Somasis
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Exherbo-dev mailing list [email protected] http://lists.exherbo.org/mailman/listinfo/exherbo-dev
