Re: [gentoo-alt] gemato in prefix
On 05-03-2018 18:45:00 +0100, Michael Weiser wrote: > > > gemato is down to 24 seconds for verification, hashverify now takes 13 > > > seconds. > > Darn, and how many cpu cores does it use throughout? Just two? > > I think you can relax. :) After upgrade from 10.3 to 11.2 it now runs > more than a minute again. In four runs I've seen 66, 87, 56 and 72 > seconds. > > From the PIDs, process tree and CPU load it seems to start one > controller and four workers (the i7-5650U has Hyperthreading). Work > doesn't seem to be distributed equally between the workers because they > finish at different times. The first finishes after about twenty > seconds. The other follow in 10 second intervals. At the end the > controller seems to do some more work for another 10 seconds. > > It gets slower with just two workers (-j2). For some reason I can't seem > to saturate the CPUs. It can't be I/O per se because right afterwards > hashverify goes up to 272% CPU usage and finishes in 8 to 11 seconds. So > the fs cache must be fully primed. Hmmm, I fear it's IO-bound somehow. Thanks for your detailed observations. > > > I'll go on and try to integrate gemato and hashverify with emaint sync > > > in prefix. > > Gemato should be already pretty much hooked, isn't it? I assume for now > > we could use a hook? > > It is in mainline portage. There it now lives as a hardcoded call to > gemato in rsync.py as seen with 2.3.22 on Linux. Prefix portage 2.3.18 > seems to lack it. > > Is it a lot of work to make a prefix portage release in line with > mainline portage? It's usually not that bad, but because they hardwired gemato, I don't want to push that out. > I've installed the repos.postsync.d hook from the source distribution > for now and it seems to work fine. Same for hashverify. So I now have > extra-pranoia double-verification using two implementations. :) > > How are chances to get the Prefix signing key into gentoo-keys? Good suggestion, I should see to that. Perhaps they're willing to do it. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-alt] gemato in prefix
Hi Fabian, On Sun, Mar 04, 2018 at 09:20:31PM +0100, Fabian Groffen wrote: > > Were you able to pin down the actual cause of the verification failures? > Yes. > (BRACE BRACE) Now that's what I call a tough nut to crack. Awesome work to sort it out. > > gemato is down to 24 seconds for verification, hashverify now takes 13 > > seconds. > Darn, and how many cpu cores does it use throughout? Just two? I think you can relax. :) After upgrade from 10.3 to 11.2 it now runs more than a minute again. In four runs I've seen 66, 87, 56 and 72 seconds. >From the PIDs, process tree and CPU load it seems to start one controller and four workers (the i7-5650U has Hyperthreading). Work doesn't seem to be distributed equally between the workers because they finish at different times. The first finishes after about twenty seconds. The other follow in 10 second intervals. At the end the controller seems to do some more work for another 10 seconds. It gets slower with just two workers (-j2). For some reason I can't seem to saturate the CPUs. It can't be I/O per se because right afterwards hashverify goes up to 272% CPU usage and finishes in 8 to 11 seconds. So the fs cache must be fully primed. > > I'll go on and try to integrate gemato and hashverify with emaint sync > > in prefix. > Gemato should be already pretty much hooked, isn't it? I assume for now > we could use a hook? It is in mainline portage. There it now lives as a hardcoded call to gemato in rsync.py as seen with 2.3.22 on Linux. Prefix portage 2.3.18 seems to lack it. Is it a lot of work to make a prefix portage release in line with mainline portage? I've installed the repos.postsync.d hook from the source distribution for now and it seems to work fine. Same for hashverify. So I now have extra-pranoia double-verification using two implementations. :) How are chances to get the Prefix signing key into gentoo-keys? -- thanks, Micha
Re: [gentoo-alt] gemato in prefix
On 04-03-2018 17:58:57 +0100, Michael Weiser wrote: > Hi Fabian, > > On Sun, Mar 04, 2018 at 10:34:27AM +0100, Fabian Groffen wrote: > > > I think I finally cracked it. I basically rewrote hashgen because it > > was needlessly complex and interwoven for some reason. I just did an > > emerge --sync (against rsync1) and ran hashverify after it (emerge > > hashgen). It fails because of found excess file: srf-ip-conn-srv.pid > > but that's the only complaint it has. > > Yes, hashverify as well as gemato can now verify a freshly synced prefix > tree without error (apart from the pid file). > > Were you able to pin down the actual cause of the verification failures? Yes. (BRACE BRACE) In order to reduce the amount of files to be rsynced on every sync, I play tricks with mtimes of generated Manifests and the directories they are in. The idea is that if I manage to keep the mtime the same, even though the file was re-generated, then rsync won't "see" the file as needing to be transferred. The source of all problems, is wrongly setting the mtimes, IOW, rsync didn't transfer the Manifest, while it should. An extra added difficulty here is that I try to get a re-producible rsync generation -- that is, rsync1 and rsync2 generate the tree on their own, and the output should be the same. Some of the things that went wrong: - I falsely believed opendir/readdir returned entries sorted. I don't know why I thought this was true, but I thought it was the case. Perhaps this is filesystem dependant, in any case rsync1 and rsync2 (which run on completely different hardware, even arch), the file order isn't the same, so the same Manifest on rsync1 had a different order of its contents than rsync2. This way, you have the same filesize (and mtime, because we set it), but the hashes are actually different. I didn't check if opendir/readdir is consistent between runs, but in any case, an emerge --sync that does deltas on the result of a previous sync on the other rsync slave results in all wrong hashes because category manifests almost always change due to updates. - I wrongly assumed that git recorded dates would make sense. This actually is absolutely bogus, weeks into the future, eons in the past. To get rsync1 and rsync2 generate the same mtimes, I decided to touch files after a git pull with their git recorded date. Because this date is highly unreliable timestamps got out of order easily. I salvaged this by touching the files with the timestamp of the start of the rsync generation process instead. On both machines this is triggered from cron at the same time, so this gives an equal mtime. - For the above, rsync1 and 2 can get out of mtime-sync if for some reason either misses a cycle. If someone touches eutils.eclass this the regen process might take a long time, so this legitimately happens. During the night when the generation is down for capacity reasons on rsync1 and 2, I rsync the states of the git repos, such that at the start of the day, both start from the same "seed". - I was setting the mtime for the top level Manifest, but this file contains a TIMESTAMP entry which changes every generation, so this was a stupid thing to do. - Originally, I was basing the mtime off of the existing Manifest in the tree. This was reasonable when repoman would have to make a change there on every commit, but nowadays it doesn't, so it can easily be older than the ebuilds in the same directory. As a result, many Manifests were having too-old mtimes, and I cascased them down to the upper levels, causing major outdatedness. - Next strategy was to take the mtime of the oldest file the Manifest refers to. This is a good approach, except I didn't push this down to all the sub-Manifests. In the latest rewrite, I fixed a missed case. > gemato is down to 24 seconds for verification, hashverify now takes 13 > seconds. Darn, and how many cpu cores does it use throughout? Just two? > I'll go on and try to integrate gemato and hashverify with emaint sync > in prefix. Gemato should be already pretty much hooked, isn't it? I assume for now we could use a hook? Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-alt] gemato in prefix
Hi Fabian, On Sun, Mar 04, 2018 at 10:34:27AM +0100, Fabian Groffen wrote: > I think I finally cracked it. I basically rewrote hashgen because it > was needlessly complex and interwoven for some reason. I just did an > emerge --sync (against rsync1) and ran hashverify after it (emerge > hashgen). It fails because of found excess file: srf-ip-conn-srv.pid > but that's the only complaint it has. Yes, hashverify as well as gemato can now verify a freshly synced prefix tree without error (apart from the pid file). Were you able to pin down the actual cause of the verification failures? gemato is down to 24 seconds for verification, hashverify now takes 13 seconds. I'll go on and try to integrate gemato and hashverify with emaint sync in prefix. -- Michael
Re: [gentoo-alt] gemato in prefix
I think I finally cracked it. I basically rewrote hashgen because it was needlessly complex and interwoven for some reason. I just did an emerge --sync (against rsync1) and ran hashverify after it (emerge hashgen). It fails because of found excess file: srf-ip-conn-srv.pid but that's the only complaint it has. I hope the same happens for you when you try. Fabian On 01-03-2018 20:02:00 +0100, Michael Weiser wrote: > Hi Fabian, > > On Thu, Mar 01, 2018 at 02:12:16PM +0100, Fabian Groffen wrote: > > > > I synced at 11:56 from rsync2. > > By the way. Syncing at 11:56 means you need to finish within 25 seconds > > Good that I put the time in there. I totally failed to > connect the sync time to the regen time. But: It was only this one time. > The other tries before I was careful to stay away from 26,56 of the > hour. > > I just tried again at 19:15 and 19:21 and hashverify is now back to > reporting hash mismatches on basically every single Manifest. > > I've also verified that my compiler output isn't totally broken by > hashgen-ing my local tree and hashverify-ing as well as gemato-verifying > it. gemato croaks on srf-ip-conn-srv.pid being there. After deleting and > re-hashgen-ing it, gemato verifies the tree fine. Then I gemato-create-d > it and hashverify-d successfully. > > So it all seems consistent and valid locally but to break somehow in > transit. > > BTW: gemato verify takes 90s on my MacBook Air where hasverify takes 7 > to 9 seconds. So I'm still sold on the speed. :) > -- > Thanks, > Michael > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-alt] gemato in prefix
Hi Fabian, On Thu, Mar 01, 2018 at 02:12:16PM +0100, Fabian Groffen wrote: > > I synced at 11:56 from rsync2. > By the way. Syncing at 11:56 means you need to finish within 25 seconds Good that I put the time in there. I totally failed to connect the sync time to the regen time. But: It was only this one time. The other tries before I was careful to stay away from 26,56 of the hour. I just tried again at 19:15 and 19:21 and hashverify is now back to reporting hash mismatches on basically every single Manifest. I've also verified that my compiler output isn't totally broken by hashgen-ing my local tree and hashverify-ing as well as gemato-verifying it. gemato croaks on srf-ip-conn-srv.pid being there. After deleting and re-hashgen-ing it, gemato verifies the tree fine. Then I gemato-create-d it and hashverify-d successfully. So it all seems consistent and valid locally but to break somehow in transit. BTW: gemato verify takes 90s on my MacBook Air where hasverify takes 7 to 9 seconds. So I'm still sold on the speed. :) -- Thanks, Michael
Re: [gentoo-alt] gemato in prefix
On 01-03-2018 14:12:16 +0100, Fabian Groffen wrote: > So, for your purposes of getting a consistent sync, you better sync > around 50 and 20 (assuming you can get the sync to finish within 6 > minutes). > > > I get: > > > > # time /usr/local/gentoo/usr/portage/scripts/rsync-generation/hashgen > > hashverify /usr/local/gentoo/usr/portage > > verifying /usr/local/gentoo/usr/portage... > > RSA key fingerprint 0204 A8AB D003 E57A 9558 850D BA08 091E C631 7B3C > > good signature made 2018-03-01 10:26:41 UTC by > > Gentoo Prefix Portage Snapshot Signing Key (Automated Signing Key) > > Manifest.files.gz:scripts/Manifest.gz: > > - file size mismatch > >got: 1948 > > expected: 1949 For the record: I ran verify during the moment it rsyncs over, and got many of these errors. So I'll probably need to come up with something atomic (although I wonder now what main rsync slaves do). -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-alt] gemato in prefix
On 01-03-2018 12:13:14 +0100, Michael Weiser wrote: > Hi Fabian, > > On Thu, Mar 01, 2018 at 08:46:26AM +0100, Fabian Groffen wrote: > > > I ran a --sync on my macbook this morning: > > I synced at 11:56 from rsync2. By the way. Syncing at 11:56 means you need to finish within 25 seconds or so. On rsync1 this is 110 seconds. (This is the difference of a Core i5 4x3.4GHz versus a G5 2x2.5GHz.) So, for your purposes of getting a consistent sync, you better sync around 50 and 20 (assuming you can get the sync to finish within 6 minutes). > I get: > > # time /usr/local/gentoo/usr/portage/scripts/rsync-generation/hashgen > hashverify /usr/local/gentoo/usr/portage > verifying /usr/local/gentoo/usr/portage... > RSA key fingerprint 0204 A8AB D003 E57A 9558 850D BA08 091E C631 7B3C > good signature made 2018-03-01 10:26:41 UTC by > Gentoo Prefix Portage Snapshot Signing Key (Automated Signing Key) > Manifest.files.gz:scripts/Manifest.gz: > - file size mismatch >got: 1948 > expected: 1949 Yay, rsync1 and rsync2 disagreed on the size for both files. After unpacking and sorting the contents, it was equal though. Lesson learnt: the contents should be sorted to ensure equality. Now I'll put this up so I can see if I can track the reference error (in Manifest.files.gz). > manifest verification failed > 6.36 real 4.20 user 4.12 sys > > So it looks a lot better but still not fully fixed. > > > hashverify is too fast to be correct (on a > > 6-core machine it does the tree in less than a second) at the moment, > > but I'm adding counters and shizzle to verify that it's actually doing > > something. > > The speed is unbelievable in the sense of the word. A quick dtruss shows > that it is in fact opening and reading files at least. Yes, also, I noticed that q (from portage-utils) updates the cache usually in below-second times, which walks the entire tree as well. So perhaps it is all not that impossible. When generating manifests, hashgen is also quite fast. Of course it's operating on a warm tree, but it's doing writes all the time, so... Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-alt] gemato in prefix
Hi Fabian, On Thu, Mar 01, 2018 at 08:46:26AM +0100, Fabian Groffen wrote: > I ran a --sync on my macbook this morning: I synced at 11:56 from rsync2. > [febe:~/Gentoo-10.10/usr/portage] fabian% cd scripts/rsync-generation > [febe:portage/scripts/rsync-generation] fabian% clang -o hashgen > -fopenmp -Wall -Werror -O3 -pipe -lssl -lcrypto -lb2 -lz `gpgme-config > --libs` hashgen.c > [febe:portage/scripts/rsync-generation] fabian% time ./hashgen hashverify > $EPREFIX/usr/portage > If you could at least reproduce this, then we'd be on the same page. I get: # time /usr/local/gentoo/usr/portage/scripts/rsync-generation/hashgen hashverify /usr/local/gentoo/usr/portage verifying /usr/local/gentoo/usr/portage... RSA key fingerprint 0204 A8AB D003 E57A 9558 850D BA08 091E C631 7B3C good signature made 2018-03-01 10:26:41 UTC by Gentoo Prefix Portage Snapshot Signing Key (Automated Signing Key) Manifest.files.gz:scripts/Manifest.gz: - file size mismatch got: 1948 expected: 1949 Manifest.files.gz:metadata/Manifest.gz: - file size mismatch got: 6667274 expected: 6667102 Manifest.files.gz:net-libs/Manifest.gz: - file size mismatch got: 33983 expected: 33500 net-misc/srf-ip-conn-srv/Manifest: stray file not in Manifest: srf-ip-conn-srv.pid Manifest.files.gz:net-proxy/Manifest.gz: - file size mismatch got: 6377 expected: 6380 Manifest.files.gz:profiles/Manifest.gz: - file size mismatch got: 180939 expected: 180944 manifest verification failed 6.36 real 4.20 user 4.12 sys So it looks a lot better but still not fully fixed. > hashverify is too fast to be correct (on a > 6-core machine it does the tree in less than a second) at the moment, > but I'm adding counters and shizzle to verify that it's actually doing > something. The speed is unbelievable in the sense of the word. A quick dtruss shows that it is in fact opening and reading files at least. -- bye, Michael
Re: [gentoo-alt] gemato in prefix
On 28-02-2018 12:34:31 -0600, R0b0t1 wrote: > Can you not use webrsync-gpg for the time being? I'm affraid not, we do "sign" the snapshots, but they are just tarred up versions of the rsync tree as generated. The same tree we're talking about here. > Incremental updates of authenticated files would be best, but until > that can be done in a completely foolproof way I would wait so as to > not give yourself a false sense of security. Honestly I never understood why Portage doesn't just verify the paths to the ebuilds it eventually wants to install. Anyway, for me the goal is to get some sense of verification, the ultimate sense of security is kind of pointless, since you can point it at any random host, and any random joe can generate any random, but valid(ating) tree. Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-alt] gemato in prefix
I ran a --sync on my macbook this morning: [febe:~/Gentoo-10.10/usr/portage] fabian% cd scripts/rsync-generation [febe:portage/scripts/rsync-generation] fabian% clang -o hashgen -fopenmp -Wall -Werror -O3 -pipe -lssl -lcrypto -lb2 -lz `gpgme-config --libs` hashgen.c [febe:portage/scripts/rsync-generation] fabian% time ./hashgen hashverify $EPREFIX/usr/portage verifying /Users/fabian/Gentoo-10.10/usr/portage... RSA key fingerprint 0204 A8AB D003 E57A 9558 850D BA08 091E C631 7B3C good signature made 2018-03-01 06:58:17 UTC by Gentoo Prefix Portage Snapshot Signing Key (Automated Signing Key) scripts/Manifest.gz: stray file not in Manifest: hashgen net-misc/srf-ip-conn-srv/Manifest: stray file not in Manifest: srf-ip-conn-srv.pid manifest verification failed 7.561u 18.643s 0:24.97 104.9% 0+0k 4+14143io 0pf+0w [febe:portage/scripts/rsync-generation] fabian% (both stray files are known issues at the moment, hashgen obviously I just compiled in the tree, the .pid file is most probably a mistakenly added file to git that should get resolved soon after some consensus is reached) If you could at least reproduce this, then we'd be on the same page. Fabian On 28-02-2018 19:50:34 +0100, Fabian Groffen wrote: > Ok, the SphinxTrain you report, I had that too, and that was a bad sync. > > So it seems I managed to break everyone's trees, and I need to bump > mtime to get this fixed. > > I'll see to incorporating your patch for Darwin. I've just added more > parallelism, and I believe hashverify is too fast to be correct (on a > 6-core machine it does the tree in less than a second) at the moment, > but I'm adding counters and shizzle to verify that it's actually doing > something. > > Fabian > > > On 28-02-2018 19:38:58 +0100, Michael Weiser wrote: > > Hi Fabian, > > > > On Wed, Feb 28, 2018 at 04:58:34PM +0100, Fabian Groffen wrote: > > > > > Can you give this another try now? Many Manifests were not in sync, I'm > > > trying to see if that is persistent or not, but I got a clean run here > > > now. > > > > I synced at about 19:05 CET from rsync2. gemato still fails: > > > > ERROR:root:Manifest mismatch for app-pda/Manifest.gz > > BLAKE2B: expected: > > d8a3b53e76aff2866c747bf039ecad8961fad33c04c6f425f7ed826d8c4731c8baad5946747c131804b4e5c1f6c90bdbd49c0fa58f1921ed7f4c9e548fbc6631, > > have: > > 49cb3f4aa903adf238cb58f6590138b74f46277da12e0666da6983310663c25b659fa5d60658f8b6fd0285a9c6202bb521daa4d46e96b3469c764346794b4a59 > > SHA512: expected: > > 429dde656d914b77698199a956b0b302fe96e55476022654384058840dbc1775ffae8d3a0f8953789ccefe1c47f6c1f9a26889858695c06b521dc965eaba8c13, > > have: > > c19984dab99d11b8b44b997dd275a5fac2d53c69420b1be27ef475b42241976d13eea0b24db5a7a510a2c60ebf1b43b943b5eb5b230dd313615886028744b6f8 > > > > And what's more: By accident I found that the Manifest of gentoo-keys > > (of all the possible ebuilds) doesn't match the actual filesize: > > > > !!! Digest verification failed: > > !!! > > /usr/local/gentoo/usr/portage/app-crypt/gentoo-keys/gentoo-keys-201607021514-r2.ebuild > > !!! Reason: Filesize does not match recorded size > > !!! Got: 575 > > !!! Expected: 563 > > > > I also tried hashverify. It needed some minor darwin touches: > > > > /usr/local/gentoo/usr/portage/scripts/rsync-generation/hashgen.c:102:24: > > error: no member named 'st_mtim' in 'struct stat' > > if (tv[1].tv_sec < s->st_mtim.tv_sec || > > > > st_mtim and st_atim are st_mtimespec and st_atimespec on darwin. > > > > With that it compiles, verifies the GPG signature fine and then reports > > hash mismatches like mad. I added a debug printf that shows the file > > being processed and here's an example of what comes out: > > > > RSA key fingerprint 0204 A8AB D003 E57A 9558 850D BA08 091E C631 7B3C > > good signature made 2018-02-28 17:56:44 UTC by > > Gentoo Prefix Portage Snapshot Signing Key (Automated Signing Key) > > Processing /usr/local/gentoo/usr/portage//Manifest.files.gz > > Processing /usr/local/gentoo/usr/portage//app-accessibility/Manifest.gz > > Processing > > /usr/local/gentoo/usr/portage//app-accessibility/SphinxTrain/Manifest > > - BLAKE2B hash mismatch > > computed: > > 'ecf65f1f0f2cb75c2e8299819b14a9587869f7c3fb3ddbfe0dde6b4620cc4ecfedd3dfd62285f900c8f11f8b1ebcae68356a9e9371b0f20ed94eab50b3a09d59' > > recorded in manifest: > > 'e714ed5d44165ecc96e115f225e3fe1e63d45f12c59f1a96fcd2533b3cf39e7db3d329c71ca7445e16cff97b177d18520231f08303f65383fcb6c2f215c8113c' > > - SHA512 hash mismatch > > computed: > > 'aa1a0a47856a8d05a6d94aa0622fd94ac528800f1d83d8909cc7372c05ad133689d02ea9851df41d25593bf3422c8c6ead83dd9ee286535856b6cd5699ef9514' > > recorded in manifest: > > 'a62a301c8d5486f4a29ad528ad2610cf681a9ec94769e2479403f302a256aad8f7f8773aa6a76f7aff10f490a1d100c21c2aa4fceffa214e6d1de3d544cf3f79' > > Processing > > /usr/local/gentoo/usr/portage//app-accessibility/accerciser/Manifest > > Processing > > /usr/local/gentoo/usr/portage//app-a
Re: [gentoo-alt] gemato in prefix
Ok, the SphinxTrain you report, I had that too, and that was a bad sync. So it seems I managed to break everyone's trees, and I need to bump mtime to get this fixed. I'll see to incorporating your patch for Darwin. I've just added more parallelism, and I believe hashverify is too fast to be correct (on a 6-core machine it does the tree in less than a second) at the moment, but I'm adding counters and shizzle to verify that it's actually doing something. Fabian On 28-02-2018 19:38:58 +0100, Michael Weiser wrote: > Hi Fabian, > > On Wed, Feb 28, 2018 at 04:58:34PM +0100, Fabian Groffen wrote: > > > Can you give this another try now? Many Manifests were not in sync, I'm > > trying to see if that is persistent or not, but I got a clean run here > > now. > > I synced at about 19:05 CET from rsync2. gemato still fails: > > ERROR:root:Manifest mismatch for app-pda/Manifest.gz > BLAKE2B: expected: > d8a3b53e76aff2866c747bf039ecad8961fad33c04c6f425f7ed826d8c4731c8baad5946747c131804b4e5c1f6c90bdbd49c0fa58f1921ed7f4c9e548fbc6631, > have: > 49cb3f4aa903adf238cb58f6590138b74f46277da12e0666da6983310663c25b659fa5d60658f8b6fd0285a9c6202bb521daa4d46e96b3469c764346794b4a59 > SHA512: expected: > 429dde656d914b77698199a956b0b302fe96e55476022654384058840dbc1775ffae8d3a0f8953789ccefe1c47f6c1f9a26889858695c06b521dc965eaba8c13, > have: > c19984dab99d11b8b44b997dd275a5fac2d53c69420b1be27ef475b42241976d13eea0b24db5a7a510a2c60ebf1b43b943b5eb5b230dd313615886028744b6f8 > > And what's more: By accident I found that the Manifest of gentoo-keys > (of all the possible ebuilds) doesn't match the actual filesize: > > !!! Digest verification failed: > !!! > /usr/local/gentoo/usr/portage/app-crypt/gentoo-keys/gentoo-keys-201607021514-r2.ebuild > !!! Reason: Filesize does not match recorded size > !!! Got: 575 > !!! Expected: 563 > > I also tried hashverify. It needed some minor darwin touches: > > /usr/local/gentoo/usr/portage/scripts/rsync-generation/hashgen.c:102:24: > error: no member named 'st_mtim' in 'struct stat' > if (tv[1].tv_sec < s->st_mtim.tv_sec || > > st_mtim and st_atim are st_mtimespec and st_atimespec on darwin. > > With that it compiles, verifies the GPG signature fine and then reports > hash mismatches like mad. I added a debug printf that shows the file > being processed and here's an example of what comes out: > > RSA key fingerprint 0204 A8AB D003 E57A 9558 850D BA08 091E C631 7B3C > good signature made 2018-02-28 17:56:44 UTC by > Gentoo Prefix Portage Snapshot Signing Key (Automated Signing Key) > Processing /usr/local/gentoo/usr/portage//Manifest.files.gz > Processing /usr/local/gentoo/usr/portage//app-accessibility/Manifest.gz > Processing > /usr/local/gentoo/usr/portage//app-accessibility/SphinxTrain/Manifest > - BLAKE2B hash mismatch > computed: > 'ecf65f1f0f2cb75c2e8299819b14a9587869f7c3fb3ddbfe0dde6b4620cc4ecfedd3dfd62285f900c8f11f8b1ebcae68356a9e9371b0f20ed94eab50b3a09d59' > recorded in manifest: > 'e714ed5d44165ecc96e115f225e3fe1e63d45f12c59f1a96fcd2533b3cf39e7db3d329c71ca7445e16cff97b177d18520231f08303f65383fcb6c2f215c8113c' > - SHA512 hash mismatch > computed: > 'aa1a0a47856a8d05a6d94aa0622fd94ac528800f1d83d8909cc7372c05ad133689d02ea9851df41d25593bf3422c8c6ead83dd9ee286535856b6cd5699ef9514' > recorded in manifest: > 'a62a301c8d5486f4a29ad528ad2610cf681a9ec94769e2479403f302a256aad8f7f8773aa6a76f7aff10f490a1d100c21c2aa4fceffa214e6d1de3d544cf3f79' > Processing > /usr/local/gentoo/usr/portage//app-accessibility/accerciser/Manifest > Processing > /usr/local/gentoo/usr/portage//app-accessibility/accerciser/accerciser-3.22.0.ebuild > Processing > /usr/local/gentoo/usr/portage//app-accessibility/accerciser/metadata.xml > Processing > /usr/local/gentoo/usr/portage//app-accessibility/at-spi2-atk/Manifest > - BLAKE2B hash mismatch > computed: > '55b456c982be1452a9aec9cb5ba9cd0b406cb55d4ace6aba4b1387c2208fc6ddbaca3d14158d61d938dec3442d6b535a209198a080ee2585753461b3b2e36685' > recorded in manifest: > '4d9670351132d8f7716f841a1e7052e7517c0638e9f9a95a2f4a2de4d6af928994fc31f9aa796d33b3ae894c5f42c29a2eae4e69748f36203220e6d83bba5ed7' > - SHA512 hash mismatch > computed: > '368448746af5beec6778151629746488eb9ed6bd3f32cf032df662aeeec77b3e82e222adcde4c6ada3dea44e6ee8604b90bbaef17a2c7f9bbef9e7d4fbcf273a' > recorded in manifest: > 'bb2aa40d3871481a3c021d7ffb79c44ec0d350e1977cc29e946b06ad1a91a0f0a514324b4095f35250fbc4f95d5e12f7a0b526bcc3772a2ca6f23bddef26fe74' > Processing > /usr/local/gentoo/usr/portage//app-accessibility/at-spi2-core/Manifest > - BLAKE2B hash mismatch > computed: > '47ca01b5215034b3166fa32f8f88837973aeb144806eb2e33c43a6aa495bc6337303f87289eb3ecc2838059702f8e89459560812f479106369d38f4851bdaf3f' > recorded in manifest: > '168fa7e833a3974820925cd84e68863ad64b3edc4b3b4ff9f78c59e1c6d49463c831493053756f039b5fc562a3ad2dfd37e29d26731510e3b8741c0ab6f5025c' > > It seems most Manifests but n
Re: [gentoo-alt] gemato in prefix
Hi Fabian, On Wed, Feb 28, 2018 at 04:58:34PM +0100, Fabian Groffen wrote: > Can you give this another try now? Many Manifests were not in sync, I'm > trying to see if that is persistent or not, but I got a clean run here > now. I synced at about 19:05 CET from rsync2. gemato still fails: ERROR:root:Manifest mismatch for app-pda/Manifest.gz BLAKE2B: expected: d8a3b53e76aff2866c747bf039ecad8961fad33c04c6f425f7ed826d8c4731c8baad5946747c131804b4e5c1f6c90bdbd49c0fa58f1921ed7f4c9e548fbc6631, have: 49cb3f4aa903adf238cb58f6590138b74f46277da12e0666da6983310663c25b659fa5d60658f8b6fd0285a9c6202bb521daa4d46e96b3469c764346794b4a59 SHA512: expected: 429dde656d914b77698199a956b0b302fe96e55476022654384058840dbc1775ffae8d3a0f8953789ccefe1c47f6c1f9a26889858695c06b521dc965eaba8c13, have: c19984dab99d11b8b44b997dd275a5fac2d53c69420b1be27ef475b42241976d13eea0b24db5a7a510a2c60ebf1b43b943b5eb5b230dd313615886028744b6f8 And what's more: By accident I found that the Manifest of gentoo-keys (of all the possible ebuilds) doesn't match the actual filesize: !!! Digest verification failed: !!! /usr/local/gentoo/usr/portage/app-crypt/gentoo-keys/gentoo-keys-201607021514-r2.ebuild !!! Reason: Filesize does not match recorded size !!! Got: 575 !!! Expected: 563 I also tried hashverify. It needed some minor darwin touches: /usr/local/gentoo/usr/portage/scripts/rsync-generation/hashgen.c:102:24: error: no member named 'st_mtim' in 'struct stat' if (tv[1].tv_sec < s->st_mtim.tv_sec || st_mtim and st_atim are st_mtimespec and st_atimespec on darwin. With that it compiles, verifies the GPG signature fine and then reports hash mismatches like mad. I added a debug printf that shows the file being processed and here's an example of what comes out: RSA key fingerprint 0204 A8AB D003 E57A 9558 850D BA08 091E C631 7B3C good signature made 2018-02-28 17:56:44 UTC by Gentoo Prefix Portage Snapshot Signing Key (Automated Signing Key) Processing /usr/local/gentoo/usr/portage//Manifest.files.gz Processing /usr/local/gentoo/usr/portage//app-accessibility/Manifest.gz Processing /usr/local/gentoo/usr/portage//app-accessibility/SphinxTrain/Manifest - BLAKE2B hash mismatch computed: 'ecf65f1f0f2cb75c2e8299819b14a9587869f7c3fb3ddbfe0dde6b4620cc4ecfedd3dfd62285f900c8f11f8b1ebcae68356a9e9371b0f20ed94eab50b3a09d59' recorded in manifest: 'e714ed5d44165ecc96e115f225e3fe1e63d45f12c59f1a96fcd2533b3cf39e7db3d329c71ca7445e16cff97b177d18520231f08303f65383fcb6c2f215c8113c' - SHA512 hash mismatch computed: 'aa1a0a47856a8d05a6d94aa0622fd94ac528800f1d83d8909cc7372c05ad133689d02ea9851df41d25593bf3422c8c6ead83dd9ee286535856b6cd5699ef9514' recorded in manifest: 'a62a301c8d5486f4a29ad528ad2610cf681a9ec94769e2479403f302a256aad8f7f8773aa6a76f7aff10f490a1d100c21c2aa4fceffa214e6d1de3d544cf3f79' Processing /usr/local/gentoo/usr/portage//app-accessibility/accerciser/Manifest Processing /usr/local/gentoo/usr/portage//app-accessibility/accerciser/accerciser-3.22.0.ebuild Processing /usr/local/gentoo/usr/portage//app-accessibility/accerciser/metadata.xml Processing /usr/local/gentoo/usr/portage//app-accessibility/at-spi2-atk/Manifest - BLAKE2B hash mismatch computed: '55b456c982be1452a9aec9cb5ba9cd0b406cb55d4ace6aba4b1387c2208fc6ddbaca3d14158d61d938dec3442d6b535a209198a080ee2585753461b3b2e36685' recorded in manifest: '4d9670351132d8f7716f841a1e7052e7517c0638e9f9a95a2f4a2de4d6af928994fc31f9aa796d33b3ae894c5f42c29a2eae4e69748f36203220e6d83bba5ed7' - SHA512 hash mismatch computed: '368448746af5beec6778151629746488eb9ed6bd3f32cf032df662aeeec77b3e82e222adcde4c6ada3dea44e6ee8604b90bbaef17a2c7f9bbef9e7d4fbcf273a' recorded in manifest: 'bb2aa40d3871481a3c021d7ffb79c44ec0d350e1977cc29e946b06ad1a91a0f0a514324b4095f35250fbc4f95d5e12f7a0b526bcc3772a2ca6f23bddef26fe74' Processing /usr/local/gentoo/usr/portage//app-accessibility/at-spi2-core/Manifest - BLAKE2B hash mismatch computed: '47ca01b5215034b3166fa32f8f88837973aeb144806eb2e33c43a6aa495bc6337303f87289eb3ecc2838059702f8e89459560812f479106369d38f4851bdaf3f' recorded in manifest: '168fa7e833a3974820925cd84e68863ad64b3edc4b3b4ff9f78c59e1c6d49463c831493053756f039b5fc562a3ad2dfd37e29d26731510e3b8741c0ab6f5025c' It seems most Manifests but not all and only Manifests are failing. -- Michael
Re: [gentoo-alt] gemato in prefix
On Tue, Feb 20, 2018 at 2:24 PM, Michael Weiser wrote: > Hi Fabian, > > On Tue, Feb 20, 2018 at 08:41:57PM +0100, Fabian Groffen wrote: > >> Thing is I once believed Portage checked manifest and all, but it seems >> not to do anything any more, so my idea of things being OK may have been > > I also was a bit surprised to find that portage didn't authenticate and > verify the tree at all. Stumbling over webrsync more or less by > accident, I've been using it as the next best thing in the interim. > > From what I was able to find on the net, there's never been any > actual implementation before Michal Gorny started gemato (see > https://www.gentoo.org/glep/glep-0074.html#motivation wrt GLEP-58 from > 2008 never being implemented). After using gemato on Gentoo Linux as a > very early adopter I'm eager to get something comparable going in Prefix > Mac. > Can you not use webrsync-gpg for the time being? Incremental updates of authenticated files would be best, but until that can be done in a completely foolproof way I would wait so as to not give yourself a false sense of security. Cheers, R0b0t1
Re: [gentoo-alt] gemato in prefix
Can you give this another try now? Many Manifests were not in sync, I'm trying to see if that is persistent or not, but I got a clean run here now. Fabian On 20-02-2018 20:25:33 +0100, Michael Weiser wrote: > Hi Fabian, > > On Fri, Feb 02, 2018 at 09:06:34PM +0100, Fabian Groffen wrote: > > > > does it make sense to look into using gemato for repo verification or is > > > there a reason this cannot work currently? > > > It should, but I didn't get around to it. > > I finally got around to trying gemato on my Mac. It sets off fine but > immediately fails on sys-apps/Manifest.gz: > > $ gemato verify -K /Users/michael/b/pubring.gpg /usr/local/gentoo/usr/portage/ > INFO:root:Refreshing keys from keyserver... > INFO:root:Keys refreshed. > INFO:root:Manifest timestamp: 2018-02-19 17:28:21 UTC > INFO:root:Valid OpenPGP signature found: > INFO:root:- primary key: 0204A8ABD003E57A9558850DBA08091EC6317B3C > INFO:root:- subkey: 0204A8ABD003E57A9558850DBA08091EC6317B3C > INFO:root:- timestamp: 2018-02-19 17:28:21 UTC > INFO:root:Verifying /usr/local/gentoo/usr/portage/... > ERROR:root:Manifest mismatch for sys-apps/Manifest.gz > BLAKE2B: expected: > 304895d779741fedeaac05df18857d5b0c1afa23220e6e578bd7ddca53f6d4781751881f13c59c361f3a225c7c8290cfa2ae278c779ad4c68a938b9336ebc999, > have: > e2260c115b7886ac16e74a8c981f3830650c018aa2d1566947b7eee2463eb8c56a5d5be3d30f324b239b3f9899b1781fe6f11c3bfb482bebb8df48e09e15ef43 > SHA512: expected: > 0985d753fcb39735651606c30dbe9335d6d82569ca0e6ac766f268f5fd8d3df40e9f2664c145c752bb9c7c09a06f7766bc9fdb42a37809e62ea6462743bde2c6, > have: > 9d60081f638b5678780c21f698f0ee56cd4fa4dfe3d89a6c38403a37bd6cd782181fe0368af597d316f110e82c61cc8770346007a2a63dad90b7bac555c277eb > > I can reproduce the discrepancy with sha512sum and b2sum. > > Is it possible that prefix's tree isn't fully rehashed and resigned > after changes? > > > Instead I want to use my own > > C-based tool, but I also didn't get around to getting it ready. > > Is it available somewhere to try out? > -- > Thanks, Michael > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-alt] gemato in prefix
Hi Fabian, On Wed, Feb 21, 2018 at 10:04:42AM +0100, Fabian Groffen wrote: > To give you an idea; the current tree is getting its Manifests from > hashgen.c, which you can find in scripts/rsync-generation/hashgen.c. > The hashverify tool, which I'm currently working on, is basically an > addition to that file (doing argv[0] detection) to perform the > verification. At this time of writing, I have the gpg-verification and > single file entry verification in place. I'm still trying to close the > gap in checking the dirs in particular looking for files that are not > listed in the Manifest. I've been following or rather post-reading the discussion about GLEP-74 and it seems a lot of thought and security considerations have gone into it. Are you following it for your implementation? I see you're using OpenMP. That looks very neat and could certainly make it scale better than gemato. BTW: I remember OpenMP being plain missing from clang until recently. Ah, https://clang-omp.github.io/ says 3.7 and onwards have native support. > hashgen currently runs in 30s or so on the tree to generate manifests. > I hope it can verify in the same amount of time (we're talking about a > Quad G5 PowerPC machine here with rusty old spinning disks), leaving it > in a much better position to be used for Prefix, since we tend to have > slower/older machines around. I can only believe that number if the portage tree fits into the fs cache (RAM). Then it would be purely CPU-bound, I guess, and C could play out its advantage. On a machine that has just synced the tree using rsync it would also still be in RAM as a working set (if there was sufficient RAM) and the same consideration would apply. On my MacBook Air I get 625012k from du for the tree. So this should be alright for any machine with 1GB+ of RAM that's not doing anything else at the time. As soon as the tree doesn't completely fit into RAM, the later files of the rsync will push the first files out of the cache. So for verification, the reading will start from scratch, making the verification I/O-bound and the advantage of a C implementation mostly irrelevant, won't it? As a quick number: On an ARM SBC of mine with a Dual-Core 1.2GHz Cortex-A7, 1GB of RAM and fs on a microSD card that does 22MB/s sequential read I get 400 seconds flat for the first run of gemato verify and 386 for the second when the fs cache is primed already. I do see a bit of I/O-boundness on the first run and on the second, gemato fully hogs the CPU. It would be interesting to compare how much of that is python overhead and how much the actual crypto. Unfortunately the gcc on that board doesn't have openmp support (yet). gemato create runs 667s on the same board and falls back to a single thread for some reason about a minute in. So I'd certainly take a 22fold speedup if I could get it. :) > We're not really looking forward to 15 > minutes of verification as some bugs have been reported to with gemato. Well, with webrsync I was looking at around three minutes of download and five minutes of unpacking and local rsync on my ARM SBCs. With gemato it's now about 30 seconds to three minutes of rsyncing depending on the amount of changes and six minutes of gemato checking. So basically I've neither gained nor lost anything but feel much more efficient by once again not downloading what hasn't changed. It also shows that I have a high pain threshold in dealing with Gentoo on small and old machines. Considering how long Gentoo users typically wait for compiles to finish I wouldn't have expected a bit of a wait for gemato or hashverify to be much of an issue. > Portage used to do 1) checking the digests of ebuilds and 2) checking > for missing and extra files. I noticed that at least 1) is no longer > present, which I find weird. I need checking this on normal Gentoo, > (simply edit an ebuild and try to emerge it without updating its > digest), but I have the suspicion this got disabled because the full > tree verification should catch this. Needless to say, that's > suboptimal, and not very secure IMO. Ah, now I understand what you were getting at. I just verified: Both checks are working on Gentoo Linux but not in Prefix Mac. It seems to have to do with setting "thin-manifests" in layout.conf of the prefix tree. Once I set that to false, prefix portage behaves the same as Gentoo Linux portage. Gentoo Linux: [root@linux:/usr/portage/xfce-base/garcon] sha512sum garcon-0.6.1.ebuild dfa80c3e8c766af3d170536f6d7c48793da6bccd4f1647ac8194130a521eb852bf6872514a75e8da7d0eec02eca53d76e431c4371c6f9c891a68fa516fdca8b7 garcon-0.6.1.ebuild [root@linux:/usr/portage/xfce-base/garcon] grep EBUILD.*0.6.1.*SHA512 Manifest EBUILD garcon-0.6.1.ebuild 1018 SHA512 dfa80c3e8c766af3d170536f6d7c48793da6bccd4f1647ac8194130a521eb852bf6872514a75e8da7d0eec02eca53d76e431c4371c6f9c891a68fa516fdca8b7 [root@linux:/usr/portage/xfce-base/garcon] sed -i -e "s,econf,fooconf," garcon-0.6.1.ebuild [root@linux:/usr/portage
Re: [gentoo-alt] gemato in prefix
Hi Michael, To give you an idea; the current tree is getting its Manifests from hashgen.c, which you can find in scripts/rsync-generation/hashgen.c. The hashverify tool, which I'm currently working on, is basically an addition to that file (doing argv[0] detection) to perform the verification. At this time of writing, I have the gpg-verification and single file entry verification in place. I'm still trying to close the gap in checking the dirs in particular looking for files that are not listed in the Manifest. hashgen currently runs in 30s or so on the tree to generate manifests. I hope it can verify in the same amount of time (we're talking about a Quad G5 PowerPC machine here with rusty old spinning disks), leaving it in a much better position to be used for Prefix, since we tend to have slower/older machines around. We're not really looking forward to 15 minutes of verification as some bugs have been reported to with gemato. Portage used to do 1) checking the digests of ebuilds and 2) checking for missing and extra files. I noticed that at least 1) is no longer present, which I find weird. I need checking this on normal Gentoo, (simply edit an ebuild and try to emerge it without updating its digest), but I have the suspicion this got disabled because the full tree verification should catch this. Needless to say, that's suboptimal, and not very secure IMO. Now I may be all wrong with trying to implement the verification myself, but that's a separate topic. gemato should work fine. I've checked some of the digests you mentioned and they look ok. So I'm wondering whether perhaps you got caught in the middle of a sync. This used to be much less of a problem because of per-ebuild-dir integrity, but now the entire tree requires to be consistent. I'll look into re-activating my symlink-flip, which should make the switch atomic, but I don't know what rsync is doing if the symlink is flipped during a sync. It reduces the invalid window somewhat I guess. Thanks, Fabian On 20-02-2018 21:24:03 +0100, Michael Weiser wrote: > Hi Fabian, > > On Tue, Feb 20, 2018 at 08:41:57PM +0100, Fabian Groffen wrote: > > > Thing is I once believed Portage checked manifest and all, but it seems > > not to do anything any more, so my idea of things being OK may have been > > I also was a bit surprised to find that portage didn't authenticate and > verify the tree at all. Stumbling over webrsync more or less by > accident, I've been using it as the next best thing in the interim. > > From what I was able to find on the net, there's never been any > actual implementation before Michal Gorny started gemato (see > https://www.gentoo.org/glep/glep-0074.html#motivation wrt GLEP-58 from > 2008 never being implemented). After using gemato on Gentoo Linux as a > very early adopter I'm eager to get something comparable going in Prefix > Mac. > > How can I help? > -- > Thanks, > Michael > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-alt] gemato in prefix
Hi Fabian, On Tue, Feb 20, 2018 at 08:41:57PM +0100, Fabian Groffen wrote: > Thing is I once believed Portage checked manifest and all, but it seems > not to do anything any more, so my idea of things being OK may have been I also was a bit surprised to find that portage didn't authenticate and verify the tree at all. Stumbling over webrsync more or less by accident, I've been using it as the next best thing in the interim. >From what I was able to find on the net, there's never been any actual implementation before Michal Gorny started gemato (see https://www.gentoo.org/glep/glep-0074.html#motivation wrt GLEP-58 from 2008 never being implemented). After using gemato on Gentoo Linux as a very early adopter I'm eager to get something comparable going in Prefix Mac. How can I help? -- Thanks, Michael
Re: [gentoo-alt] gemato in prefix
Well, yeah, I have the feeling that until I'm done with the verification (it's a work in progress, the problem is mostly in walking the entire tree a bit efficient) I can see if what we have actually makes sense. Thing is I once believed Portage checked manifest and all, but it seems not to do anything any more, so my idea of things being OK may have been false appearances because Portage no longer gives a . Fabian On 20-02-2018 20:25:33 +0100, Michael Weiser wrote: > Hi Fabian, > > On Fri, Feb 02, 2018 at 09:06:34PM +0100, Fabian Groffen wrote: > > > > does it make sense to look into using gemato for repo verification or is > > > there a reason this cannot work currently? > > > It should, but I didn't get around to it. > > I finally got around to trying gemato on my Mac. It sets off fine but > immediately fails on sys-apps/Manifest.gz: > > $ gemato verify -K /Users/michael/b/pubring.gpg /usr/local/gentoo/usr/portage/ > INFO:root:Refreshing keys from keyserver... > INFO:root:Keys refreshed. > INFO:root:Manifest timestamp: 2018-02-19 17:28:21 UTC > INFO:root:Valid OpenPGP signature found: > INFO:root:- primary key: 0204A8ABD003E57A9558850DBA08091EC6317B3C > INFO:root:- subkey: 0204A8ABD003E57A9558850DBA08091EC6317B3C > INFO:root:- timestamp: 2018-02-19 17:28:21 UTC > INFO:root:Verifying /usr/local/gentoo/usr/portage/... > ERROR:root:Manifest mismatch for sys-apps/Manifest.gz > BLAKE2B: expected: > 304895d779741fedeaac05df18857d5b0c1afa23220e6e578bd7ddca53f6d4781751881f13c59c361f3a225c7c8290cfa2ae278c779ad4c68a938b9336ebc999, > have: > e2260c115b7886ac16e74a8c981f3830650c018aa2d1566947b7eee2463eb8c56a5d5be3d30f324b239b3f9899b1781fe6f11c3bfb482bebb8df48e09e15ef43 > SHA512: expected: > 0985d753fcb39735651606c30dbe9335d6d82569ca0e6ac766f268f5fd8d3df40e9f2664c145c752bb9c7c09a06f7766bc9fdb42a37809e62ea6462743bde2c6, > have: > 9d60081f638b5678780c21f698f0ee56cd4fa4dfe3d89a6c38403a37bd6cd782181fe0368af597d316f110e82c61cc8770346007a2a63dad90b7bac555c277eb > > I can reproduce the discrepancy with sha512sum and b2sum. > > Is it possible that prefix's tree isn't fully rehashed and resigned > after changes? > > > Instead I want to use my own > > C-based tool, but I also didn't get around to getting it ready. > > Is it available somewhere to try out? > -- > Thanks, Michael > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-alt] gemato in prefix
Hi Fabian, On Fri, Feb 02, 2018 at 09:06:34PM +0100, Fabian Groffen wrote: > > does it make sense to look into using gemato for repo verification or is > > there a reason this cannot work currently? > It should, but I didn't get around to it. I finally got around to trying gemato on my Mac. It sets off fine but immediately fails on sys-apps/Manifest.gz: $ gemato verify -K /Users/michael/b/pubring.gpg /usr/local/gentoo/usr/portage/ INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-02-19 17:28:21 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: 0204A8ABD003E57A9558850DBA08091EC6317B3C INFO:root:- subkey: 0204A8ABD003E57A9558850DBA08091EC6317B3C INFO:root:- timestamp: 2018-02-19 17:28:21 UTC INFO:root:Verifying /usr/local/gentoo/usr/portage/... ERROR:root:Manifest mismatch for sys-apps/Manifest.gz BLAKE2B: expected: 304895d779741fedeaac05df18857d5b0c1afa23220e6e578bd7ddca53f6d4781751881f13c59c361f3a225c7c8290cfa2ae278c779ad4c68a938b9336ebc999, have: e2260c115b7886ac16e74a8c981f3830650c018aa2d1566947b7eee2463eb8c56a5d5be3d30f324b239b3f9899b1781fe6f11c3bfb482bebb8df48e09e15ef43 SHA512: expected: 0985d753fcb39735651606c30dbe9335d6d82569ca0e6ac766f268f5fd8d3df40e9f2664c145c752bb9c7c09a06f7766bc9fdb42a37809e62ea6462743bde2c6, have: 9d60081f638b5678780c21f698f0ee56cd4fa4dfe3d89a6c38403a37bd6cd782181fe0368af597d316f110e82c61cc8770346007a2a63dad90b7bac555c277eb I can reproduce the discrepancy with sha512sum and b2sum. Is it possible that prefix's tree isn't fully rehashed and resigned after changes? > Instead I want to use my own > C-based tool, but I also didn't get around to getting it ready. Is it available somewhere to try out? -- Thanks, Michael
Re: [gentoo-alt] gemato in prefix
It should, but I didn't get around to it. Instead I want to use my own C-based tool, but I also didn't get around to getting it ready. Fabian On 02-02-2018 14:27:52 +0100, Michael Weiser wrote: > Hi, > > does it make sense to look into using gemato for repo verification or is > there a reason this cannot work currently? > -- > Thanks, > Michael > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
