-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On
Behalf Of Troy Dawson
Sent: 06 December 2007 20:18
To: [EMAIL PROTECTED]
Cc: Bly, MJ (Martin); [EMAIL PROTECTED]
Subject: Re: Problem with Mirroring
Hi Stephan,
Thank you for the explanation, especially about how frequent
it is. To be honest, your reports were the only one's we'd
gotten about this problem, so we weren't aware of how bad it
is. I'm also glad that Martin spoke up, because if all the
reports come from the same place it sort of seems like it's a
possible communication between places problem.
And if he isn't reading this, please send my apologies to
Kai. I guess I did snap at him, and I'm sorry.
Here at Fermilab I've had to explain to people at least 6
times in the past week that we haven't yet released SLF 5.1
and explain to them what that means.
So I guess my temper has gotten a bit thin on that subject.
Troy
[EMAIL PROTECTED] wrote:
Hi Troy,
this problem has become so frequent that I stopped
reporting it, since
I was sure your storage experts are chasing it.
I usually run a dryrun rsync whenever I expect some change
(it's not a
cron job). Lately, at least one out of four times the
dryrun told me
it would delete some files it clearly shouldn't, typically with the
first character of the filename from some range in the
alphabet, as in
the reports your received.
Every time I checked (maybe the first five times), using an
ftp client
confirmed what rsync said: the server didn't know about the files.
Given that an rsync dryrun takes about a minute, the
problem must be
present a pretty significant fraction of the day. I usually have to
wait for O(30m) until the files "are back". Now we
typically run this
in the morning. I've oserved the around noon as well (all times
GMT+1). Hence it may be due to some maintenace task you or
your storage guys run at night.
It should be fairly simple to gain more statistics: just running an
"ls <release>/SL/RPMS/" (or /SL/ for SL5) every 15 minutes on the
server, for all stable releases, an comparing with the
first result,
should provide you with evidence that the problem is real
and an idea
of how serious it is, without any risk of false positives or
significant load on the server.
And please cut Kai a little slack. He's in charge of
mirroring the SL
tree here (not trivial lately due to the problem this is
all about),
and deciding which updates make it onto the systems and which don't
yet, while I'm on leave. For example, right now he has to
put the xen
updates for 5.0 on hold. As he said, he's new to the SL
business, but
he's not a fool. 5.1 beta is integrated into our installation and
configuration management under the 5.1 label, which is
probably why he
confused 5rolling for 5.1.
Cheers,
Stephan
On Thu, 6 Dec 2007, Troy Dawson wrote:
Hi Martin,
Could you let me know what time this happened, and what machine is
doing the transfer.
It's actually much easier for us to trace ftp transactions than
rsync, so maybe this will give us a glimpse into what is happening
better than tracking down DESY's rsync.
Troy
Bly, MJ (Martin) wrote:
I am seeing this too, on a long-standing mirror using lftp direct
from ftp.scientificlinux.org. Clobbered some installations I was
doing of our copy on Sunday - we lost some of the SL4.4 i386 base
stuff, mostly files starting 'o' (openssl, openoffice,
openssh, ... list on request).
A re-sync of the mirror brought the files back.
Martin.
--
Martin Bly
RAL Tier1 Fabric Team
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]
On Behalf
Of Kai Leffhalm
Sent: 06 December 2007 11:38
To: [EMAIL PROTECTED]
Subject: Problem with Mirroring
Hi,
I am new here and have one problem when mirroring the
repositories:
sometimes the packages disappear and reappear after
some time. I
was told that this is a problem with the fileserver. Now
there are
some old packages, which seems to be deleted on purpose and some
seems to be reorganized.
Is there any possibility to see, which files would be deleted on
purpose and which are just temporarily deleted?
Cheers
Kai Leffhalm
Examples:
SL5.1: (Files are moved to different directory)
x86_64/SL/firefox-1.5.0.12-7.el5.i386.rpm
x86_64/SL/firefox-1.5.0.12-7.el5.x86_64.rpm
x86_64/SL/firefox-devel-1.5.0.12-7.el5.i386.rpm
x86_64/SL/firefox-devel-1.5.0.12-7.el5.x86_64.rpm
deleting
x86_64/updates/security/firefox-1.5.0.12-7.el5.x86_64.rpm
deleting x86_64/updates/security/firefox-1.5.0.12-7.el5.i386.rpm
deleting
x86_64/updates/security/firefox-devel-1.5.0.12-7.el5.x86_64.rpm
deleting
x86_64/updates/security/firefox-devel-1.5.0.12-7.el5.i386.rpm
SL4.4: (Old and current packages are deleted) deleting
i386/errata/SL/RPMS/postgresql-libs-7.4.17-1.RHEL4.1.i386.rpm
deleting
i386/errata/SL/RPMS/postgresql-jdbc-7.4.17-1.RHEL4.1.i386.rpm
deleting
i386/errata/SL/RPMS/openssh-server-3.9p1-8.RHEL4.20.i386.rpm
deleting
i386/errata/SL/RPMS/openssh-clients-3.9p1-8.RHEL4.20.i386.rpm
--
Stephan Wiesand
DESY - DV -
Platanenallee 6
15738 Zeuthen, Germany
--
__________________________________________________
Troy Dawson [EMAIL PROTECTED] (630)840-6468 Fermilab
ComputingDivision/LCSI/CSI DSS Group
__________________________________________________
