Re: [gentoo-user] Re: All sorts of digest verification failures

[Neil Bothwick]

On Mon, 16 Nov 2015 03:48:52 -0600, Dale wrote:

> > According to https://bugs.gentoo.org/show_bug.cgi?id=565694 it has
> > been fixed, I synced too early this morning to pick that up.

> I synced again just a hour or two ago and I must have missed it too.  It
> spit out the same error as before.  I'll try again in a couple days. 
> Maybe things will have settled by then.  Maybe.  ;-)

I was a little too quick off the mark there, but I have just synced again
and it finally works.


-- 
Neil Bothwick

Q: What's the proper plural of a 'Net-connected Windows machine?
A: A Botnet


pgppgvoxbQll9.pgp
Description: OpenPGP digital signature

[gentoo-user] Re: All sorts of digest verification failures

[Martin Vaeth]

cov...@ccs.covici.com  wrote:
>
> I have thinmanifests=true as specified in some news item or post, I
> think this was a mandatory change some time ago using rsync.

If you really use rsync/webrsync and not git, this is unlikely:
The file containing this line (metadata/layout.conf) should be
overridden at every rsync (unless you took special measures,
but this was certainly never recommended).

> They figured the ebuilds sync anyway so no reason for the
> manifests to have them.

It is not about syncing but about security (checksums with
signatures should safe you from MITM and even compromised
servers). Thin-manifests was only meant for git, because git
already contains checksums ('though only less secure sha1,
but that's a different story), so it was decided that no
duplicate checksums are needed for git.
For *rsync* the situation is different.

Re: [gentoo-user] Re: All sorts of digest verification failures

[Marc Joliet]

On Monday 16 November 2015 17:21:07 Martin Vaeth wrote:
>cov...@ccs.covici.com  wrote:
>> I have thinmanifests=true as specified in some news item or post, I
>> think this was a mandatory change some time ago using rsync.
>
>If you really use rsync/webrsync and not git, this is unlikely:
>The file containing this line (metadata/layout.conf) should be
>overridden at every rsync (unless you took special measures,
>but this was certainly never recommended).
>
>> They figured the ebuilds sync anyway so no reason for the
>> manifests to have them.
>
>It is not about syncing but about security (checksums with
>signatures should safe you from MITM and even compromised
>servers). Thin-manifests was only meant for git, because git
>already contains checksums ('though only less secure sha1,
>but that's a different story), so it was decided that no
>duplicate checksums are needed for git.
>For *rsync* the situation is different.

Don't forget that in Gentoo all commits are also GPG signed.

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Re: All sorts of digest verification failures

[Rich Freeman]

On Mon, Nov 16, 2015 at 4:49 PM, Marc Joliet  wrote:
>
> Don't forget that in Gentoo all commits are also GPG signed.
>

Sure, but to be fair those signatures are only bound to the content of
the commit by an sha1 hash.

-- 
Rich

Re: [gentoo-user] Re: All sorts of digest verification failures

[Dale]

Grant Edwards wrote:
> On 2015-11-15, Dale  wrote:
>
>> Wasn't there a emerge option to ignore the manifest?  I seem to recall
>> there used to be one but it was a LONG time ago.  Of course, if someone
>> has tinkered with something that would be a bad thing to do.
> If the digest failures are for packages you don't have installed, just
> removing the entire directory for each of the broken manifests is one
> temporary work-around.
>


Thing is, one of the ones that fails is a package that I have
installed.  That would be busybox and I think that is likely on
everyone's system so this would affect every Gentoo system out there.  I
think I read on -dev somewhere a while back that busybox ended up being
pulled in as part of the @system profile/set. 

Anyway, I figure they will have a fix before to long.  I just find it
hard to believe that with all the planning that was put into this, the
changelogs and these issues were missed.  It's not like they decided to
change this over a short time frame.  They been working on this for
ages.  Live and learn I guess.  ;-)

Dale

:-)  :-) 

Re: [gentoo-user] Re: All sorts of digest verification failures

[Dale]

Neil Bothwick wrote:
> On Sun, 15 Nov 2015 17:39:29 -0800, walt wrote:
>
>
>> I hope the gentoo devs will fix this bug before you have a chance to
>> test my advice :)
> According to https://bugs.gentoo.org/show_bug.cgi?id=565694 it has been
> fixed, I synced too early this morning to pick that up.
>
>


I synced again just a hour or two ago and I must have missed it too.  It
spit out the same error as before.  I'll try again in a couple days. 
Maybe things will have settled by then.  Maybe.  ;-)

Dale

:-)  :-) 

Re: [gentoo-user] Re: All sorts of digest verification failures

[Neil Bothwick]

On Sun, 15 Nov 2015 17:39:29 -0800, walt wrote:

> > I have the same wrong size recorded in the Manifest!
> > No idea why portage didn't yell at me - there seems
> > to be another bug involved...  
> 
> I did the same thing today (15 Nov) and it succeeded.
> 
> However, I ran the ebuild command on a non-broken ebuild.  Try
> repeating the same command on busybox 1.23.x or 1.24.x

Do you mean ebuild manifest? It doesn't matter which ebuild you run that
on, it creates a manifest for all ebuilds and files in the firsctory.

> I hope the gentoo devs will fix this bug before you have a chance to
> test my advice :)

According to https://bugs.gentoo.org/show_bug.cgi?id=565694 it has been
fixed, I synced too early this morning to pick that up.


-- 
Neil Bothwick

Be regular. Eat cron flakes.


pgpiPWRjH234O.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Re: All sorts of digest verification failures

[Marc Joliet]

On Monday 16 November 2015 17:32:56 Rich Freeman wrote:
>On Mon, Nov 16, 2015 at 4:49 PM, Marc Joliet  wrote:
>> Don't forget that in Gentoo all commits are also GPG signed.
>
>Sure, but to be fair those signatures are only bound to the content of
>the commit by an sha1 hash.

Ah! Thanks, I didn't know that.

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Re: All sorts of digest verification failures

[Alan Mackenzie]

On Thu, Nov 12, 2015 at 06:45:44PM +0200, Alan McKinnon wrote:
> On 12/11/2015 18:42, Grant Edwards wrote:
> > On 2015-11-12,   wrote:
> >> Grant Edwards  wrote:

> >>> After an emerge --sync that appeared to work with no problems, my
> >>> "emerge -auvND world" command is reporting that the Changelong files
> >>> are broken for about 2/3 of the packages it wants to update:

[  ]

> The dev are doing some $MAGIC to reinstate ChangeLogs and the first run
> is expected to take a while (i.e. several hours). I suppose you can
> expect some breakage till it finishes.

> It's being discussed and tracked on gentoo-dev, you can drop a mail
> there with specifics to let the devs know what's happening.

Three days later.  I'm still getting this error message, but with a nasty
twist in the tail.  emerge -puND @world reports (amongst others) the
following update:

[ebuild   R] sys-apps/busybox-1.23.1-r1  USE="-pam*"

, and the error message I get on actually trying to start the update is

!!! Digest verification failed:
!!! /usr/portage/sys-apps/busybox/busybox-.ebuild
!!! Reason: Filesize does not match recorded size
!!! Got: 8493
!!! Expected: 8580

.  Why is the build system looking at the digest for version  when it
should be rebuilding version 1.23.1-r1?

> -- 
> Alan McKinnon
> alan.mckin...@gmail.com

-- 
Alan Mackenzie (Nuremberg, Germany).

[gentoo-user] Re: All sorts of digest verification failures

[Martin Vaeth]

Alan Mackenzie  wrote:
>!!! /usr/portage/sys-apps/busybox/busybox-.ebuild
>!!! Got: 8493
>!!! Expected: 8580

Do you use the default (rsync) for syncing, or have you changed
the method?

I have the above claimed filesize (8493), but the Manifest
I obtained from rsync is correct.

The timestamp of the Manifest file in that folder is
Nov 13, 08:17:31 (UTC)
(do not forget to export TZ=UTC when you check the date).
Exactly the same date has the ChangeLog file in that folder
(which is not surprsing).

If in your case ChangeLog and Manifest *both* have an older date,
you got the data perhaps from an outdated mirror.

If ChangeLog has the same date, but Manifest is older (or newer
and still wrong), something strange is going on. In such a case,
one should also consider reasons like local filesystem errors.

Re: [gentoo-user] Re: All sorts of digest verification failures

[Matti Nykyri]

> On Nov 15, 2015, at 12:43, Peter Humphrey  wrote:
> 
> On Sunday 15 Nov 2015 12:22:39 Matti Nykyri wrote:
>>> On Nov 15, 2015, at 11:59, Alan Mackenzie  wrote:
> --->8
>>> Three days later.  I'm still getting this error message, but with a
>>> nasty
>>> twist in the tail.  emerge -puND @world reports (amongst others) the
>>> 
>>> following update:
>>>   [ebuild   R] sys-apps/busybox-1.23.1-r1  USE="-pam*"
>>> 
>>> , and the error message I get on actually trying to start the update is
>>> 
>>>   !!! Digest verification failed:
>>>   !!! /usr/portage/sys-apps/busybox/busybox-.ebuild
>>>   !!! Reason: Filesize does not match recorded size
>>>   !!! Got: 8493
>>>   !!! Expected: 8580
>>> 
>>> .  Why is the build system looking at the digest for version  when
>>> it
>>> should be rebuilding version 1.23.1-r1?
>> 
>> Well it's not. It just checks all the manifests and complains about
>> errors. It doesn't affect the building of 1.23.1-r1.
> 
> I'm getting the same thing as Alan, and have been for several days.
> 
>> If I were you I'ld download the latest portage snapshot. That should take
>> care of any remaining issues. Is your portage upto date?
> 
> My portage is sync'd daily, so it shouldn't need a whole new snapshot.

Sunc doesn't necessarily sync everything and if some random files are out of 
sync you either need to update them manually or get a new snapshot. I doubt the 
server/mirror is messed up, but you could try changing that. Downloading a new 
snapshot isn't that big a deal and that should definitely fix everything, so 
why not try that first?

You may also inspect each package and remanifest those that have problems 
(ebuild manifest). If you are certain that nobody has been fingering your 
ebuilds.

-- 
-M

Re: [gentoo-user] Re: All sorts of digest verification failures

[Alan Mackenzie]

Hello, Matti

On Sun, Nov 15, 2015 at 12:22:39PM +0200, Matti Nykyri wrote:
> > On Nov 15, 2015, at 11:59, Alan Mackenzie  wrote:

> >> On Thu, Nov 12, 2015 at 06:45:44PM +0200, Alan McKinnon wrote:
> >>> On 12/11/2015 18:42, Grant Edwards wrote:
>  On 2015-11-12,   wrote:
>  Grant Edwards  wrote:

> > After an emerge --sync that appeared to work with no problems, my
> > "emerge -auvND world" command is reporting that the Changelong files
> > are broken for about 2/3 of the packages it wants to update:

> > [  ]

> >> The dev are doing some $MAGIC to reinstate ChangeLogs and the first run
> >> is expected to take a while (i.e. several hours). I suppose you can
> >> expect some breakage till it finishes.

> >> It's being discussed and tracked on gentoo-dev, you can drop a mail
> >> there with specifics to let the devs know what's happening.

> > Three days later.  I'm still getting this error message, but with a nasty
> > twist in the tail.  emerge -puND @world reports (amongst others) the
> > following update:

> >[ebuild   R] sys-apps/busybox-1.23.1-r1  USE="-pam*"

> > , and the error message I get on actually trying to start the update is

> >!!! Digest verification failed:
> >!!! /usr/portage/sys-apps/busybox/busybox-.ebuild
> >!!! Reason: Filesize does not match recorded size
> >!!! Got: 8493
> >!!! Expected: 8580

> > .  Why is the build system looking at the digest for version  when it
> > should be rebuilding version 1.23.1-r1?

> Well it's not. It just checks all the manifests and complains about
> errors. It doesn't affect the building of 1.23.1-r1.

Ah, OK.  But it causes emerge to bail out, so never gets round to
building 1.23.1-r1.

> If I were you I'ld download the latest portage snapshot. That should
> take care of any remaining issues. Is your portage upto date?

I'd just synched my portage tree immediately before attempting the build.
I'll try again tomorrow, in the hope everything will have been sorted out
by then.

> -- 
> -M

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [gentoo-user] Re: All sorts of digest verification failures

[Matti Nykyri]

> On Nov 15, 2015, at 11:59, Alan Mackenzie  wrote:
> 
>> On Thu, Nov 12, 2015 at 06:45:44PM +0200, Alan McKinnon wrote:
>>> On 12/11/2015 18:42, Grant Edwards wrote:
 On 2015-11-12,   wrote:
 Grant Edwards  wrote:
> 
> After an emerge --sync that appeared to work with no problems, my
> "emerge -auvND world" command is reporting that the Changelong files
> are broken for about 2/3 of the packages it wants to update:
> 
> [  ]
> 
>> The dev are doing some $MAGIC to reinstate ChangeLogs and the first run
>> is expected to take a while (i.e. several hours). I suppose you can
>> expect some breakage till it finishes.
> 
>> It's being discussed and tracked on gentoo-dev, you can drop a mail
>> there with specifics to let the devs know what's happening.
> 
> Three days later.  I'm still getting this error message, but with a nasty
> twist in the tail.  emerge -puND @world reports (amongst others) the
> following update:
> 
>[ebuild   R] sys-apps/busybox-1.23.1-r1  USE="-pam*"
> 
> , and the error message I get on actually trying to start the update is
> 
>!!! Digest verification failed:
>!!! /usr/portage/sys-apps/busybox/busybox-.ebuild
>!!! Reason: Filesize does not match recorded size
>!!! Got: 8493
>!!! Expected: 8580
> 
> .  Why is the build system looking at the digest for version  when it
> should be rebuilding version 1.23.1-r1?

Well it's not. It just checks all the manifests and complains about errors. It 
doesn't affect the building of 1.23.1-r1.

If I were you I'ld download the latest portage snapshot. That should take care 
of any remaining issues. Is your portage upto date?

-- 
-M

Re: [gentoo-user] Re: All sorts of digest verification failures

[Matti Nykyri]

> On Nov 15, 2015, at 13:01, Alan Mackenzie  wrote:
> 
>> Well it's not. It just checks all the manifests and complains about
>> errors. It doesn't affect the building of 1.23.1-r1.

Ok. I must be using some different switch then.

> Ah, OK.  But it causes emerge to bail out, so never gets round to
> building 1.23.1-r1.
> 
>> If I were you I'ld download the latest portage snapshot. That should
>> take care of any remaining issues. Is your portage upto date?
> 
> I'd just synched my portage tree immediately before attempting the build.
> I'll try again tomorrow, in the hope everything will have been sorted out
> by then.

Syncing and getting a new snapshot is a bit different. With a snapshot you get 
a new clean start. Download it and unpack.

After that check that your portage package is upto date (emerge -av portage).

-- 
-Matti

Re: [gentoo-user] Re: All sorts of digest verification failures

[Neil Bothwick]

On Sun, 15 Nov 2015 09:59:28 +, Alan Mackenzie wrote:

> [ebuild   R] sys-apps/busybox-1.23.1-r1  USE="-pam*"
> 
> , and the error message I get on actually trying to start the update is
> 
> !!! Digest verification failed:
> !!! /usr/portage/sys-apps/busybox/busybox-.ebuild
> !!! Reason: Filesize does not match recorded size
> !!! Got: 8493
> !!! Expected: 8580
> 
> .  Why is the build system looking at the digest for version  when
> it should be rebuilding version 1.23.1-r1?

The manifest covers all files in the ebuild's directory. If one fails the
test, the entire directory is considered suspect so the ebuild has to be
skipped.

This has been going on a for a few days, there is an open bug but no
apparent solution. The manifests are now generated automatically when a
git commit is made, but there appears to be something going wrong with
the process in this instance.


-- 
Neil Bothwick

Megabyte: (n.) more than you can comprehend and less than you'll need.


pgp0FOXOSlOL6.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Re: All sorts of digest verification failures

[Peter Humphrey]

On Sunday 15 Nov 2015 12:22:39 Matti Nykyri wrote:
> > On Nov 15, 2015, at 11:59, Alan Mackenzie  wrote:
--->8
> > Three days later.  I'm still getting this error message, but with a
> > nasty
> > twist in the tail.  emerge -puND @world reports (amongst others) the
> > 
> > following update:
> >[ebuild   R] sys-apps/busybox-1.23.1-r1  USE="-pam*"
> > 
> > , and the error message I get on actually trying to start the update is
> > 
> >!!! Digest verification failed:
> >!!! /usr/portage/sys-apps/busybox/busybox-.ebuild
> >!!! Reason: Filesize does not match recorded size
> >!!! Got: 8493
> >!!! Expected: 8580
> > 
> > .  Why is the build system looking at the digest for version  when
> > it
> > should be rebuilding version 1.23.1-r1?
> 
> Well it's not. It just checks all the manifests and complains about
> errors. It doesn't affect the building of 1.23.1-r1.

I'm getting the same thing as Alan, and have been for several days.

> If I were you I'ld download the latest portage snapshot. That should take
> care of any remaining issues. Is your portage upto date?

My portage is sync'd daily, so it shouldn't need a whole new snapshot.

-- 
Rgds
Peter

Re: [gentoo-user] Re: All sorts of digest verification failures

[Peter Humphrey]

On Sunday 15 Nov 2015 12:56:16 Matti Nykyri wrote:
> > On Nov 15, 2015, at 12:43, Peter Humphrey  
wrote:
> > I'm getting the same thing as Alan, and have been for several days.
> > 
> >> If I were you I'ld download the latest portage snapshot. That should
> >> take care of any remaining issues. Is your portage upto date?
> > 
> > My portage is sync'd daily, so it shouldn't need a whole new snapshot.
> 
> Sunc doesn't necessarily sync everything

Eh? How can --sync not sync?

> ... and if some random files are out of sync you either need to update
> them manually or get a new snapshot. I doubt the server/mirror is messed
> up, but you could try changing that. Downloading a new snapshot isn't that
> big a deal and that should definitely fix everything, so why not try that
> first?

I may do so, but not today.

-- 
Rgds
Peter

Re: [gentoo-user] Re: All sorts of digest verification failures

[Alan Mackenzie]

Hello, Martin.

On Sun, Nov 15, 2015 at 02:56:42PM +, Martin Vaeth wrote:
> Alan Mackenzie  wrote:
> >!!! /usr/portage/sys-apps/busybox/busybox-.ebuild
> >!!! Got: 8493
> >!!! Expected: 8580

> Do you use the default (rsync) for syncing, or have you changed
> the method?

rsync all the time.

> I have the above claimed filesize (8493), but the Manifest
> I obtained from rsync is correct.

> The timestamp of the Manifest file in that folder is
> Nov 13, 08:17:31 (UTC)

I have 07:17:31 + for Manifest and ChangeLog.  Are you sure your
08:17:31 isn't central European time?

> (do not forget to export TZ=UTC when you check the date).
> Exactly the same date has the ChangeLog file in that folder
> (which is not surprsing).

It does indeed.

> If in your case ChangeLog and Manifest *both* have an older date,
> you got the data perhaps from an outdated mirror.

> If ChangeLog has the same date, but Manifest is older (or newer
> and still wrong), something strange is going on. In such a case,
> one should also consider reasons like local filesystem errors.

I don't think I've got a filesystem error: ext3 is tried and tested.
After downloading /etc/portage via rsync again, I still get the same
error.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [gentoo-user] Re: All sorts of digest verification failures

[Alan Mackenzie]

Hello, Matti.

On Sun, Nov 15, 2015 at 01:15:16PM +0200, Matti Nykyri wrote:
> > On Nov 15, 2015, at 13:01, Alan Mackenzie  wrote:

> >> Well it's not. It just checks all the manifests and complains about
> >> errors. It doesn't affect the building of 1.23.1-r1.

> Ok. I must be using some different switch then.

> > Ah, OK.  But it causes emerge to bail out, so never gets round to
> > building 1.23.1-r1.

> >> If I were you I'ld download the latest portage snapshot. That should
> >> take care of any remaining issues. Is your portage upto date?

I deleted my /usr/portage (with the exception of
/usr/portage/distfiles), and ran emerge --sync again.  I get precisely
the same error message, still.

> > I'd just synched my portage tree immediately before attempting the build.
> > I'll try again tomorrow, in the hope everything will have been sorted out
> > by then.

> Syncing and getting a new snapshot is a bit different. With a snapshot you 
> get a new clean start. Download it and unpack.

> After that check that your portage package is upto date (emerge -av portage).

Yes, my portage is fully up to date.

I think the Gentoo maintainers haven't fixed this problem, yet.

> -- 
> -Matti

-- 
Alan Mackenzie (Nuremberg, Germany).

[gentoo-user] Re: All sorts of digest verification failures

[Martin Vaeth]

Neil Bothwick  wrote:
>
> I deleted the busybox directory from the tree then ran emerge --sync.
> The error is still there

You have the same files that I have.
Unfortunately, only now I actually did:

$ grep busybox- Manifest
EBUILD busybox-.ebuild 8580 [...]

???
I have the same wrong size recorded in the Manifest!
No idea why portage didn't yell at me - there seems
to be another bug involved.

So, sorry, my previous conjecture was wrong.

The only information which I can give you currently,
is that probably all mirrors contain the same broken Manifest
in the moment.

*Perhaps* (a very wild conjecture!) the reason why this Manifest
got not updated is the same bug which caused portage not to yell
at me, despite the broken Manifest.

As a temporary hack, calling "ebuild manifest" on that file is
probably the only reasonable way to proceed, but this problem
must be fixed, of course...

Re: [gentoo-user] Re: All sorts of digest verification failures

[Dale]

Martin Vaeth wrote:
> Neil Bothwick  wrote:
>> I deleted the busybox directory from the tree then ran emerge --sync.
>> The error is still there
> You have the same files that I have.
> Unfortunately, only now I actually did:
>
> $ grep busybox- Manifest
> EBUILD busybox-.ebuild 8580 [...]
>
> ???
> I have the same wrong size recorded in the Manifest!
> No idea why portage didn't yell at me - there seems
> to be another bug involved.
>
> So, sorry, my previous conjecture was wrong.
>
> The only information which I can give you currently,
> is that probably all mirrors contain the same broken Manifest
> in the moment.
>
> *Perhaps* (a very wild conjecture!) the reason why this Manifest
> got not updated is the same bug which caused portage not to yell
> at me, despite the broken Manifest.
>
> As a temporary hack, calling "ebuild manifest" on that file is
> probably the only reasonable way to proceed, but this problem
> must be fixed, of course...
>
>
>


Wasn't there a emerge option to ignore the manifest?  I seem to recall
there used to be one but it was a LONG time ago.  Of course, if someone
has tinkered with something that would be a bad thing to do. 

Dale

:-)  :-)

Re: [gentoo-user] Re: All sorts of digest verification failures

[covici]

Dale  wrote:

> Martin Vaeth wrote:
> > Neil Bothwick  wrote:
> >> I deleted the busybox directory from the tree then ran emerge --sync.
> >> The error is still there
> > You have the same files that I have.
> > Unfortunately, only now I actually did:
> >
> > $ grep busybox- Manifest
> > EBUILD busybox-.ebuild 8580 [...]
> >
> > ???
> > I have the same wrong size recorded in the Manifest!
> > No idea why portage didn't yell at me - there seems
> > to be another bug involved.
> >
> > So, sorry, my previous conjecture was wrong.
> >
> > The only information which I can give you currently,
> > is that probably all mirrors contain the same broken Manifest
> > in the moment.
> >
> > *Perhaps* (a very wild conjecture!) the reason why this Manifest
> > got not updated is the same bug which caused portage not to yell
> > at me, despite the broken Manifest.
> >
> > As a temporary hack, calling "ebuild manifest" on that file is
> > probably the only reasonable way to proceed, but this problem
> > must be fixed, of course...
> >
> >
> >
> 
> 
> Wasn't there a emerge option to ignore the manifest?  I seem to recall
> there used to be one but it was a LONG time ago.  Of course, if someone
> has tinkered with something that would be a bad thing to do. 

I thought I remembered something where the manifest no longer checks the
ebuild, just the file in distfiles -- I thought I remembered seeing this
somewhere.  It was at a time when we got all the manifests all at once.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

 John Covici
 cov...@ccs.covici.com

[gentoo-user] Re: All sorts of digest verification failures

[Grant Edwards]

On 2015-11-15, Dale  wrote:

> Wasn't there a emerge option to ignore the manifest?  I seem to recall
> there used to be one but it was a LONG time ago.  Of course, if someone
> has tinkered with something that would be a bad thing to do.

If the digest failures are for packages you don't have installed, just
removing the entire directory for each of the broken manifests is one
temporary work-around.

-- 
Grant

Re: [gentoo-user] Re: All sorts of digest verification failures

[Simon Thelen]

On 15-11-15 at 15:29, cov...@ccs.covici.com wrote:
> Dale  wrote:
> > Martin Vaeth wrote:
> > > Neil Bothwick  wrote:
[..]
> > > As a temporary hack, calling "ebuild manifest" on that file is
> > > probably the only reasonable way to proceed, but this problem
> > > must be fixed, of course...
> > Wasn't there a emerge option to ignore the manifest?  I seem to recall
> > there used to be one but it was a LONG time ago.  Of course, if someone
> > has tinkered with something that would be a bad thing to do. 
> I thought I remembered something where the manifest no longer checks the
> ebuild, just the file in distfiles -- I thought I remembered seeing this
> somewhere.  It was at a time when we got all the manifests all at once.
I sync from git and none of my Manifests track the ebuilds, so this
could be a thing. Note how the Manifest for busybox in the gentoo.git
tree doesn't have manifests for any of the ebuilds. [1]

[1]: https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-apps/busybox/Manifest

-- 
Simon Thelen

[gentoo-user] Re: All sorts of digest verification failures

[walt]

On Sun, 15 Nov 2015 19:05:26 + (UTC)
Martin Vaeth  wrote:

> Neil Bothwick  wrote:
> >
> > I deleted the busybox directory from the tree then ran emerge
> > --sync. The error is still there  
> 
> You have the same files that I have.
> Unfortunately, only now I actually did:
> 
> $ grep busybox- Manifest
> EBUILD busybox-.ebuild 8580 [...]
> 
> ???
> I have the same wrong size recorded in the Manifest!
> No idea why portage didn't yell at me - there seems
> to be another bug involved...

I did the same thing today (15 Nov) and it succeeded.

However, I ran the ebuild command on a non-broken ebuild.  Try
repeating the same command on busybox 1.23.x or 1.24.x

I hope the gentoo devs will fix this bug before you have a chance to
test my advice :)

Re: [gentoo-user] Re: All sorts of digest verification failures

[Matti Nykyri]

> On Nov 15, 2015, at 21:33, Alan Mackenzie  wrote:
> 
> Hello, Matti.
> 
> I deleted my /usr/portage (with the exception of
> /usr/portage/distfiles), and ran emerge --sync again.  I get precisely
> the same error message, still.

I rsync with rsync.europe.gentoo.org and with me the manifest with busybox is 
in order.  Size for .ebuild is 8493 and the Manifest has the same size. 
Also the hashes match.

I hope you get it right :)

-- 
-Matti

Re: [gentoo-user] Re: All sorts of digest verification failures

[Neil Bothwick]

On 15 November 2015 09:59:28 GMT+00:00, Alan Mackenzie  wrote:
> On Thu, Nov 12, 2015 at 06:45:44PM +0200, Alan McKinnon wrote:
> > On 12/11/2015 18:42, Grant Edwards wrote:
> > > On 2015-11-12,   wrote:
> > >> Grant Edwards  wrote:
> 
> > >>> After an emerge --sync that appeared to work with no problems,
> my
> > >>> "emerge -auvND world" command is reporting that the Changelong
> files
> > >>> are broken for about 2/3 of the packages it wants to update:
> 
> [  ]
> 
> > The dev are doing some $MAGIC to reinstate ChangeLogs and the first
> run
> > is expected to take a while (i.e. several hours). I suppose you can
> > expect some breakage till it finishes.
> 
> > It's being discussed and tracked on gentoo-dev, you can drop a mail
> > there with specifics to let the devs know what's happening.
> 
> Three days later.  I'm still getting this error message, but with a
> nasty
> twist in the tail.  emerge -puND @world reports (amongst others) the
> following update:
> 
> [ebuild   R] sys-apps/busybox-1.23.1-r1  USE="-pam*"
> 
> , and the error message I get on actually trying to start the update
> is
> 
> !!! Digest verification failed:
> !!! /usr/portage/sys-apps/busybox/busybox-.ebuild
> !!! Reason: Filesize does not match recorded size
> !!! Got: 8493
> !!! Expected: 8580
> 
> .  Why is the build system looking at the digest for version  when
> it
> should be rebuilding version 1.23.1-r1?
> 
> > -- 
> > Alan McKinnon
> > alan.mckin...@gmail.com
> 
> -- 
> Alan Mackenzie (Nuremberg, Germany).

Because the manifest covers all files in the ebuild directory, if any is 
changed the ebuilds are considered untrustworthy. This one has been around for 
a few days and there is an open bug report. 
-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[gentoo-user] Re: All sorts of digest verification failures

[Martin Vaeth]

Simon Thelen  wrote:
> I sync from git and none of my Manifests track the ebuilds, so this
> could be a thing.

No. git has (probably, I didn't check)
thin-manifests = true
in its metadata/layout.conf, but for rsync this should
not be the case for security reasons. I double-checked,
and I have indeed thin-manifests = false there, as it should be.
Nevertheless, the conflict was not reported.

Re: [gentoo-user] Re: All sorts of digest verification failures

[covici]

Martin Vaeth  wrote:

> Simon Thelen  wrote:
> > I sync from git and none of my Manifests track the ebuilds, so this
> > could be a thing.
> 
> No. git has (probably, I didn't check)
> thin-manifests = true
> in its metadata/layout.conf, but for rsync this should
> not be the case for security reasons. I double-checked,
> and I have indeed thin-manifests = false there, as it should be.
> Nevertheless, the conflict was not reported.

I have thinmanifests=true as specified in some news item or post, I
think this was a mandatory change some time ago using rsync.  They
figured the ebuilds sync anyway so no reason for the manifests to have
them.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

 John Covici
 cov...@ccs.covici.com

Re: [gentoo-user] Re: All sorts of digest verification failures

[Neil Bothwick]

On Sun, 15 Nov 2015 14:56:42 + (UTC), Martin Vaeth wrote:

> >!!! /usr/portage/sys-apps/busybox/busybox-.ebuild
> >!!! Got: 8493
> >!!! Expected: 8580  
> 
> Do you use the default (rsync) for syncing, or have you changed
> the method?
> 
> I have the above claimed filesize (8493), but the Manifest
> I obtained from rsync is correct.
> 
> The timestamp of the Manifest file in that folder is
> Nov 13, 08:17:31 (UTC)
> (do not forget to export TZ=UTC when you check the date).
> Exactly the same date has the ChangeLog file in that folder
> (which is not surprsing).
> 
> If in your case ChangeLog and Manifest *both* have an older date,
> you got the data perhaps from an outdated mirror.
> 

I deleted the busybox directory from the tree then ran emerge --sync. The
error is still there

-rw-r--r-- 1 root root  8388 Aug  9 21:34 busybox-1.21.0.ebuild
-rw-r--r-- 1 root root  8573 Aug  9 21:34 busybox-1.23.1.ebuild
-rw-r--r-- 1 root root  8611 Aug  9 21:34 busybox-1.23.1-r1.ebuild
-rw-r--r-- 1 root root  8724 Aug  9 21:34 busybox-1.23.2.ebuild
-rw-r--r-- 1 root root  8577 Nov 13 06:01 busybox-1.24.1.ebuild
-rw-r--r-- 1 root root  8493 Nov 13 06:01 busybox-.ebuild
-rw-r--r-- 1 root root  4402 Nov 13 07:17 ChangeLog
-rw-r--r-- 1 root root 60228 Nov  9 05:01 ChangeLog-2015
drwxr-xr-x 1 root root  1528 Nov 15 17:15 files
-rw-r--r-- 1 root root 19287 Nov 13 07:17 Manifest
-rw-r--r-- 1 root root   788 Aug 24 22:01 metadata.xml

-- 
Neil Bothwick

Our bikinis are exciting. They are simply the tops.


pgpc21vIjKxJm.pgp
Description: OpenPGP digital signature

[gentoo-user] Re: All sorts of digest verification failures

[Grant Edwards]

On 2015-11-12,   wrote:
> Grant Edwards  wrote:
>
>> After an emerge --sync that appeared to work with no problems, my
>> "emerge -auvND world" command is reporting that the Changelong files
>> are broken for about 2/3 of the packages it wants to update:
>> 
>> !!! Digest verification failed:
>> !!! /usr/portage/dev-libs/libxml2/ChangeLog
>> !!! Reason: Filesize does not match recorded size
>> !!! Got: 5221
>> !!! Expected: 5038
>> 
>> !!! Digest verification failed:
>> !!! /usr/portage/app-text/iso-codes/ChangeLog
>> !!! Reason: Filesize does not match recorded size
>> !!! Got: 4195
>> !!! Expected: 4014
>> 
>> [ ... and so on for another dozen or so packages ... ]
>> 
>> I removed the emerge timestamp, sync'ed again, and got the same
>> result.  Based on past experiences, I'm guessing that if I wait a day
>> or two and sync again the problems will go away.
>> 
>> But I am curious what causes these temporary breakages.  Does anybody
>> know how this happens?
>
> Try it again. I just synced and received new Manifest and Changelog
> files for every(?) package of the portage tree. But no report of bad 
> digest.

Just for fun, I removed the files from the portage tree that were
reported as bad, and did another sync.  Apparently, the rsync host
that got chosen that time had just been updated, because it downloaded
313 files (I didn't pay much attention to which files exactly), and
now emerge is happy again.

I assume that the portage trees on the rsync servers from my first two
attempts were in some intermediate state with new manifist files and
old ChangeLog files or vice versa.

In the past I've wondered how portage tree updates and rsync servers
are managed so that people don't run into problems like this more
often.

-- 
Grant Edwards   grant.b.edwardsYow! Is it 1974?  What's
  at   for SUPPER?  Can I spend
  gmail.commy COLLEGE FUND in one
   wild afternoon??

Re: [gentoo-user] Re: All sorts of digest verification failures

[Peter Humphrey]

On Thursday 12 Nov 2015 18:45:44 Alan McKinnon wrote:
> On 12/11/2015 18:42, Grant Edwards wrote:
> > On 2015-11-12,   wrote:
> >> Grant Edwards  wrote:
> >>> After an emerge --sync that appeared to work with no problems, my
> >>> "emerge -auvND world" command is reporting that the Changelong files
> >>> are broken for about 2/3 of the packages it wants to update:
> >>> 
> >>> !!! Digest verification failed:
> >>> !!! /usr/portage/dev-libs/libxml2/ChangeLog
> >>> !!! Reason: Filesize does not match recorded size
> >>> !!! Got: 5221
> >>> !!! Expected: 5038
> >>> 
> >>> !!! Digest verification failed:
> >>> !!! /usr/portage/app-text/iso-codes/ChangeLog
> >>> !!! Reason: Filesize does not match recorded size
> >>> !!! Got: 4195
> >>> !!! Expected: 4014
> >>> 
> >>> [ ... and so on for another dozen or so packages ... ]
> >>> 
> >>> I removed the emerge timestamp, sync'ed again, and got the same
> >>> result.  Based on past experiences, I'm guessing that if I wait a day
> >>> or two and sync again the problems will go away.
> >>> 
> >>> But I am curious what causes these temporary breakages.  Does anybody
> >>> know how this happens?
> >> 
> >> Try it again. I just synced and received new Manifest and Changelog
> >> files for every(?) package of the portage tree. But no report of bad
> >> digest.
> > 
> > Just for fun, I removed the files from the portage tree that were
> > reported as bad, and did another sync.  Apparently, the rsync host
> > that got chosen that time had just been updated, because it downloaded
> > 313 files (I didn't pay much attention to which files exactly), and
> > now emerge is happy again.
> > 
> > I assume that the portage trees on the rsync servers from my first two
> > attempts were in some intermediate state with new manifist files and
> > old ChangeLog files or vice versa.
> > 
> > In the past I've wondered how portage tree updates and rsync servers
> > are managed so that people don't run into problems like this more
> > often.
> 
> The dev are doing some $MAGIC to reinstate ChangeLogs and the first run
> is expected to take a while (i.e. several hours). I suppose you can
> expect some breakage till it finishes.

Yes, and I got 74,000 files transferred this morning. And yesterday a couple 
of manifests were wrong, but they were corrected soon enough.

> It's being discussed and tracked on gentoo-dev, you can drop a mail
> there with specifics to let the devs know what's happening.

-- 
Rgds
Peter

Re: [gentoo-user] Re: All sorts of digest verification failures

[Alan McKinnon]

On 12/11/2015 18:42, Grant Edwards wrote:
> On 2015-11-12,   wrote:
>> Grant Edwards  wrote:
>>
>>> After an emerge --sync that appeared to work with no problems, my
>>> "emerge -auvND world" command is reporting that the Changelong files
>>> are broken for about 2/3 of the packages it wants to update:
>>>
>>> !!! Digest verification failed:
>>> !!! /usr/portage/dev-libs/libxml2/ChangeLog
>>> !!! Reason: Filesize does not match recorded size
>>> !!! Got: 5221
>>> !!! Expected: 5038
>>>
>>> !!! Digest verification failed:
>>> !!! /usr/portage/app-text/iso-codes/ChangeLog
>>> !!! Reason: Filesize does not match recorded size
>>> !!! Got: 4195
>>> !!! Expected: 4014
>>>
>>> [ ... and so on for another dozen or so packages ... ]
>>>
>>> I removed the emerge timestamp, sync'ed again, and got the same
>>> result.  Based on past experiences, I'm guessing that if I wait a day
>>> or two and sync again the problems will go away.
>>>
>>> But I am curious what causes these temporary breakages.  Does anybody
>>> know how this happens?
>>
>> Try it again. I just synced and received new Manifest and Changelog
>> files for every(?) package of the portage tree. But no report of bad 
>> digest.
> 
> Just for fun, I removed the files from the portage tree that were
> reported as bad, and did another sync.  Apparently, the rsync host
> that got chosen that time had just been updated, because it downloaded
> 313 files (I didn't pay much attention to which files exactly), and
> now emerge is happy again.
> 
> I assume that the portage trees on the rsync servers from my first two
> attempts were in some intermediate state with new manifist files and
> old ChangeLog files or vice versa.
> 
> In the past I've wondered how portage tree updates and rsync servers
> are managed so that people don't run into problems like this more
> often.
> 

The dev are doing some $MAGIC to reinstate ChangeLogs and the first run
is expected to take a while (i.e. several hours). I suppose you can
expect some breakage till it finishes.

It's being discussed and tracked on gentoo-dev, you can drop a mail
there with specifics to let the devs know what's happening.

-- 
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] Re: All sorts of digest verification failures

[Dale]

Grant Edwards wrote:
> On 2015-11-12,   wrote:
>> Grant Edwards  wrote:
>>
>>> After an emerge --sync that appeared to work with no problems, my
>>> "emerge -auvND world" command is reporting that the Changelong files
>>> are broken for about 2/3 of the packages it wants to update:
>>>
>>> !!! Digest verification failed:
>>> !!! /usr/portage/dev-libs/libxml2/ChangeLog
>>> !!! Reason: Filesize does not match recorded size
>>> !!! Got: 5221
>>> !!! Expected: 5038
>>>
>>> !!! Digest verification failed:
>>> !!! /usr/portage/app-text/iso-codes/ChangeLog
>>> !!! Reason: Filesize does not match recorded size
>>> !!! Got: 4195
>>> !!! Expected: 4014
>>>
>>> [ ... and so on for another dozen or so packages ... ]
>>>
>>> I removed the emerge timestamp, sync'ed again, and got the same
>>> result.  Based on past experiences, I'm guessing that if I wait a day
>>> or two and sync again the problems will go away.
>>>
>>> But I am curious what causes these temporary breakages.  Does anybody
>>> know how this happens?
>> Try it again. I just synced and received new Manifest and Changelog
>> files for every(?) package of the portage tree. But no report of bad 
>> digest.
> Just for fun, I removed the files from the portage tree that were
> reported as bad, and did another sync.  Apparently, the rsync host
> that got chosen that time had just been updated, because it downloaded
> 313 files (I didn't pay much attention to which files exactly), and
> now emerge is happy again.
>
> I assume that the portage trees on the rsync servers from my first two
> attempts were in some intermediate state with new manifist files and
> old ChangeLog files or vice versa.
>
> In the past I've wondered how portage tree updates and rsync servers
> are managed so that people don't run into problems like this more
> often.
>


There's a page on g.o somewhere that explains this.  I read it a long
time ago, not sure how much has changed.  I think it updates like every
30 minutes or something but I seem to recall that some servers can
adjust that to hours or even just once a day if they need to.  It's been
a while but I bet it is still over there somewhere.  Got curious so I
went and found it.

https://wiki.gentoo.org/wiki/Project:Infrastructure/Rsync 

There are two links there.  Top one should give you a general idea. 

Dale

:-)  :-)