Re: Failed to synchronize cache for repo 'fedora', disabling.

[Adrian Reber]

On Fri, May 27, 2016 at 11:12:39AM -0600, Kevin Fenzi wrote:
> On Fri, 27 May 2016 10:35:43 +0200
> Adrian Reber  wrote:
> 
> > I wrote details about this in the above mentioned issue. I am not
> > convinced it is a MirrorManager bug. It actually exactly did what it
> > is supposed to do. The problem was that it had a newer checksum and
> > date of a repomd.xml file which does not exist on disk anymore. It
> > would be interesting to know if this newer repomd.xml file was at
> > some point actually available on the filesystem. Which seems hard to
> > find out.
> 
> Weird. I cannot think of a reason why it would have updated then
> un-updated. 

Yes. No idea. It's just strange that metalink checksum timestamp was
newer than the file and the checksum was different.

> > Ah, I could have a look at the netapp snapshots. But I don't seem
> > them. Are there netapps snapshots enable for
> > 
> > ntap-phx2-c01-fedora01-nfs.storage.phx2.redhat.com:/fedora_ftp/fedora.redhat.com/pub
> > 
> > Any way to look at older versions of that directory?
> 
> There are snapshots, but currently there's 7 of them one every 4 hours,
> so thats only 28 hours ago. This happened longer than that right?

No. It would have been interesting to see the state at:

$ date -ud @1464076094
Tue 24 May 07:48:14 UTC 2016

That was the timestamp in the metalink.

According to the current propagation result:

https://admin.fedoraproject.org/mirrormanager/propgation

Something changed between 2016-05-24-10-28-14 and 2016-05-24-14-28-14

Ah, the propagation logs might be a good place to check.

The following is the timestamp and the value of the sha256sum of
repomd.xml of
/srv/pub/fedora/linux/development/rawhide/Everything/x86_64/os/repodata
in the database. Shortened to only include the timestamps when it
changes:

2016-05-23-10-28-20 --- 
4b7495d42f9661ee9f1dcab1f235bba883d619cc5185b09e191c19429aa61e9e
2016-05-24-14-28-15 --- 
6c00530839492c598ba2daa41f068f9f673d39063645debb69e97a5a06dadc1a
2016-05-24-18-27-36 --- 
4414edff46c7a2d9754a5b711d23227e7840031871f0049d5a546062592964f7
2016-05-26-08-28-38 --- 
6c00530839492c598ba2daa41f068f9f673d39063645debb69e97a5a06dadc1a
2016-05-26-16-27-41 --- 
4414edff46c7a2d9754a5b711d23227e7840031871f0049d5a546062592964f7

last propagation check in the log:

2016-05-27-16-27-28 --- 
4414edff46c7a2d9754a5b711d23227e7840031871f0049d5a546062592964f7

So it seems to change sometimes back and forth. Which is strange.

Patrick, would did you do to fix the metalink errors on 2016-05-26? Did you wait
for a new compose or did you touch the directory and repomd.xml file so that
umdl picks up the 'changes'.

Adrian


signature.asc
Description: PGP signature
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/infrastructure@lists.fedoraproject.org

Re: Failed to synchronize cache for repo 'fedora', disabling.

[Kevin Fenzi]

On Fri, 27 May 2016 10:35:43 +0200
Adrian Reber  wrote:

> I wrote details about this in the above mentioned issue. I am not
> convinced it is a MirrorManager bug. It actually exactly did what it
> is supposed to do. The problem was that it had a newer checksum and
> date of a repomd.xml file which does not exist on disk anymore. It
> would be interesting to know if this newer repomd.xml file was at
> some point actually available on the filesystem. Which seems hard to
> find out.

Weird. I cannot think of a reason why it would have updated then
un-updated. 

> Ah, I could have a look at the netapp snapshots. But I don't seem
> them. Are there netapps snapshots enable for
> 
> ntap-phx2-c01-fedora01-nfs.storage.phx2.redhat.com:/fedora_ftp/fedora.redhat.com/pub
> 
> Any way to look at older versions of that directory?

There are snapshots, but currently there's 7 of them one every 4 hours,
so thats only 28 hours ago. This happened longer than that right?

kevin


pgpTw5haZfy7E.pgp
Description: OpenPGP digital signature
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/infrastructure@lists.fedoraproject.org

Re: Failed to synchronize cache for repo 'fedora', disabling.

[Adrian Reber]

On Thu, May 26, 2016 at 09:14:55AM -0600, Kevin Fenzi wrote:
> On Thu, 26 May 2016 06:11:57 -0400 (EDT)
> Jun Aruga  wrote:
> 
> > Hi,
> > 
> > I got an error in mock building in my environment for fedora-rawhide
> > from today. Yesterday I could do mock building. My colleague also got
> > same error for another package's mock building today.
> > 
> > Is anyone working in the system now?
> 
> This seems to be a mirrormanager bug.
> 
> ( https://github.com/fedora-infra/mirrormanager2/issues/96 )
> 
> Basically it's supposed to each time rawhide updates add the new
> repomd.xml in and keep the previous 2 around for slow updating mirrors.
> However, we haven't had any rawhide composes that worked in the last 3
> days, so it rolled off the last known good repomd.xml leaving
> nothing. ;( 

I wrote details about this in the above mentioned issue. I am not
convinced it is a MirrorManager bug. It actually exactly did what it is
supposed to do. The problem was that it had a newer checksum and date of
a repomd.xml file which does not exist on disk anymore. It would be
interesting to know if this newer repomd.xml file was at some point
actually available on the filesystem. Which seems hard to find out.

Ah, I could have a look at the netapp snapshots. But I don't seem them.
Are there netapps snapshots enable for

ntap-phx2-c01-fedora01-nfs.storage.phx2.redhat.com:/fedora_ftp/fedora.redhat.com/pub

Any way to look at older versions of that directory?

Adrian


signature.asc
Description: PGP signature
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/infrastructure@lists.fedoraproject.org

Re: Failed to synchronize cache for repo 'fedora', disabling.

[Jun Aruga]

Hi,

Thanks for your message and your fixing it.
Now I could do mock building for fedora-rawhide with success.
I wish that the issue referred on below github URL is going to be fixed. 

Thanks.

Jun


- Original Message -
> From: "Kevin Fenzi" <ke...@scrye.com>
> To: infrastructure@lists.fedoraproject.org
> Sent: Thursday, May 26, 2016 5:14:55 PM
> Subject: Re: Failed to synchronize cache for repo 'fedora', disabling.
> 
> On Thu, 26 May 2016 06:11:57 -0400 (EDT)
> Jun Aruga <jar...@redhat.com> wrote:
> 
> > Hi,
> > 
> > I got an error in mock building in my environment for fedora-rawhide
> > from today. Yesterday I could do mock building. My colleague also got
> > same error for another package's mock building today.
> > 
> > Is anyone working in the system now?
> 
> This seems to be a mirrormanager bug.
> 
> ( https://github.com/fedora-infra/mirrormanager2/issues/96 )
> 
> Basically it's supposed to each time rawhide updates add the new
> repomd.xml in and keep the previous 2 around for slow updating mirrors.
> However, we haven't had any rawhide composes that worked in the last 3
> days, so it rolled off the last known good repomd.xml leaving
> nothing. ;(
> 
> We have manually fixed it up for now... and will keep an eye on it.
> 
> Sorry for the trouble.
> 
> kevin
> 
> ___
> infrastructure mailing list
> infrastructure@lists.fedoraproject.org
> https://lists.fedoraproject.org/admin/lists/infrastructure@lists.fedoraproject.org
> 
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/infrastructure@lists.fedoraproject.org

Re: Failed to synchronize cache for repo 'fedora', disabling.

[Kevin Fenzi]

On Thu, 26 May 2016 06:11:57 -0400 (EDT)
Jun Aruga  wrote:

> Hi,
> 
> I got an error in mock building in my environment for fedora-rawhide
> from today. Yesterday I could do mock building. My colleague also got
> same error for another package's mock building today.
> 
> Is anyone working in the system now?

This seems to be a mirrormanager bug.

( https://github.com/fedora-infra/mirrormanager2/issues/96 )

Basically it's supposed to each time rawhide updates add the new
repomd.xml in and keep the previous 2 around for slow updating mirrors.
However, we haven't had any rawhide composes that worked in the last 3
days, so it rolled off the last known good repomd.xml leaving
nothing. ;( 

We have manually fixed it up for now... and will keep an eye on it. 

Sorry for the trouble. 

kevin


pgpTxW7x5nkLM.pgp
Description: OpenPGP digital signature
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/infrastructure@lists.fedoraproject.org

Failed to synchronize cache for repo 'fedora', disabling.

[Jun Aruga]

Hi,

I got an error in mock building in my environment for fedora-rawhide from 
today. Yesterday I could do mock building.
My colleague also got same error for another package's mock building today.

Is anyone working in the system now?

Thanks.

Below is for detail.

$ mock -r fedora-rawhide-x86_64 rubygem-bundler_ext-0.4.0-3.fc25.src.rpm
Run? [y/n]: y
INFO: mock.py version 1.2.17 starting (python version = 3.4.3)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
INFO: Start(rubygem-bundler_ext-0.4.0-3.fc25.src.rpm)  
Config(fedora-rawhide-x86_64)
Start: clean chroot
Finish: clean chroot
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled dnf cache
Start: cleaning dnf metadata
Finish: cleaning dnf metadata
Mock Version: 1.2.17
INFO: Mock Version: 1.2.17
Start: dnf install
Failed to synchronize cache for repo 'fedora', disabling.
Warning: Group 'buildsys-build' does not exist.
Error: Nothing to do.
ERROR: Exception(rubygem-bundler_ext-0.4.0-3.fc25.src.rpm) 
Config(fedora-rawhide-x86_64) 0 minutes 51 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-rawhide-x86_64/result
ERROR: Command failed. See logs for output.
 # /usr/bin/dnf --installroot /var/lib/mock/fedora-rawhide-x86_64/root/ 
--releasever 25 --disableplugin=local --setopt=deltarpm=false install 
@buildsys-build --setopt=tsflags=nocontexts

$ cat /etc/yum.repos.d/fedora.repo 
[fedora]
name=Fedora $releasever - $basearch
failovermethod=priority
#baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/os/
metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever=$basearch
enabled=1
metadata_expire=28d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$releasever-$basearch
skip_if_unavailable=False
...
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/infrastructure@lists.fedoraproject.org