I've done some limited tests on an Win2k client where I created roughly
1.5 million 100KB files and backed it up with the 4.1.x client. After
the initial full backup, it took roughly 55-60 minutes to do a backup
due to the fact that it had to traverse the tree. This Win2K server box
was pretty slow. Pentium 500 cpu, 256MB ram.
Although this is unrelated to the problem, I then put on the 4.2.x
client and installed the TSM journaling service. Backup time went from
about an hour to about 5-7 minutes. A great savings for the NT/2k
platform.
If your system was able to match the earlier figure performance wise, I
would expect 8.3 million files to take roughly 3 hours to scan the
drive. Of course these are different platforms and my box had no load on
it at the time of the tests other then what TSM was doing.
Also that's only scanning the filesystem and not really sending much
over the pipe. Your report shows you backed up 2.83GB of data.
One obvious problem judging by your output is your network throughput.
It's horrible. Though as other's have pointed out there's probably more
to the problem then just that but it looks like you're only getting
about 10Mbps throughput. I would recommend you test your network
throughput independent of TSM by FTP'ing a reasonably sized file between
the client box and the TSM server to see what kind of throughput you
see. If it matches what TSM is getting, you probably have a network
issue (I've seen many times where AIX boxes set to auto-negotiate speed
on their NIC's don't perform well with particular switch/routers. The
solution is to force both the NIC and router to full duplex 100mbps no
auto negotiation). If it doesn't match what TSM is doing then perhaps
some client tuning would help.
At your report's network transfer rate of 522KB/s your 2.83GB of data
would've taken.. let see.. 522KB = 522,000 Bytes.. 2.83GB =
2,830,000,000 Bytes.. 2,830,000,000/522 = 5,421,455 seconds.. = 90,357
minutes.. = 1,505 hours.. = 62 days.. hrm that can't be right.
You sure it's finishing backups in 16 hours?
I would say looking at the numbers that your major problem is your
network throughput here.. NOT the number of files you're backing up.
Check with the FTP test above and check NIC/network settings. I bet your
problem lies there.
Gerald Wichmann
System Engineer
StorageLink
408-844-8893 (v)
408-844-9801 (f)
-----Original Message-----
From: Zoltan Forray/AC/VCU [mailto:[EMAIL PROTECTED]]
Sent: Friday, August 24, 2001 8:25 AM
To: [EMAIL PROTECTED]
Subject: Long, long, long backup sessions
Looking for some guidance to improve backup session times.
The time it takes to backup our e-mail servers (4-way processor H70 AIX
4.3.3 with 2GB RAM. TSM client is 4.1.2.0) is becoming unacceptable and
unworkable.
Currently, backup sessions are sometimes taking more than 16-hours (as I
send this, one session that started yesterday evening is still running),
yet the total amount of data backed up is only around 2.5-4GB. The
current session stats shows the first data transfer wasn't until around
3:45 this morning, yet the backup session started around 6PM, YESTERDAY.
Yes, I understand the LARGE amount of OBJECTS INSPECTED is an issue.
But, the current data tells me it took over 8-hours from the start of
the schedule to when it actually transfered any data from the client to
the server (OS390 TSM 4.1.3).
Here is a end-of-schedule snippet from a previous days run. Note, it
started at 6PM and finished at 9AM the next day. When it actually did
any work, it went pretty fast:
08/23/01 09:00:37 --- SCHEDULEREC STATUS BEGIN
08/23/01 09:00:37 Total number of objects inspected: 8,521,197
08/23/01 09:00:37 Total number of objects backed up: 105,231
08/23/01 09:00:37 Total number of objects updated: 0
08/23/01 09:00:37 Total number of objects rebound: 0
08/23/01 09:00:37 Total number of objects deleted: 0
08/23/01 09:00:37 Total number of objects expired: 32,662
08/23/01 09:00:37 Total number of objects failed: 7
08/23/01 09:00:37 Total number of bytes transferred: 2.83 GB
08/23/01 09:00:37 Data transfer time: 5,685.47 sec
08/23/01 09:00:37 Network data transfer rate: 522.69 KB/sec
08/23/01 09:00:37 Aggregate data transfer rate: 55.16 KB/sec
08/23/01 09:00:37 Objects compressed by: 51%
08/23/01 09:00:37 Elapsed processing time: 00:02:48
08/23/01 09:00:37 --- SCHEDULEREC STATUS END
08/23/01 09:00:37 --- SCHEDULEREC OBJECT END BACKUP.MAIL 08/22/01
18:00:00
The "Elapsed processing time" is confusing. Is it trying to tell me it
only took 3-minutes to actually transfer the 2.83GB ??
What can we do to improve these times ? Is INCRBYDATE going to make
that
much of a different, considering we will still have to do some kinda
full backup, anyway ?
Any and all suggestions will be seriously considered !
===========================
Zoltan Forray
Virginia Commonwealth University
University Computing Center
e-mail: [EMAIL PROTECTED]
voice: 804-828-4807
