On Fri, 2007-06-15 at 09:01 -0400, Bob Shannon wrote:
> Some years ago Barry Merrill wrote a paper on the history of SMF. He
> described an SMF-processing system IBM developed but never released
> (SMFMAN?).
He presented at SHARE 73 in August of 1989. Here are notes of that
session from my trip report of the time:
===
This was an outstanding presentation by Barry Merrill on the history of
SMF. He assembled the information from recently declassified IBM
documents, and through interviews with the original architects of the
facility.
OS/360 was announced on April 7, 1964 with no SMF. Barely three months
later the SHARE Systems Objectives and Requirements Committee (SORC)
presented IBM with a report that said: IBM S/360 must provide as a
standard feature job accounting information. The SORC report included
most of the data items that you have seen in SMF data over the past 20
years. The SHARE SORC also asked for a couple of data items that we
still don't have after 20 years:
* operator intervention time (time awaiting tape/paper mounts,
etc.)
* core used by program, core used by data
The closest thing to SMF provided by IBM at the time was the IEFACTRT
exit. IEFACTRT was provided pointers to data items such as the jobname
and stepname. OS/360 also provided a system dataset - SYS1.ACCT - to
which IEFACTRT could write user records.
The SORC report basically lay dormant until June 1966 when Bob Brockish
joined IBM (he had previously been active in SHARE). He looked seriously
at the SORC report and saw two architectural goals of a future SMF
project. Users needed to account for time and resource usage on their
expensive mainframes. IBM needed to know how OS/360 was being used in
the field! In fact, there was an active project at the time within IBM -
called the SYSOUT Project - which was devising programs to look at
SYSOUT tapes from customer sites and try to analyze system usage from
the output that customers produced.
On July 6, 1966, the name SMF - System Management Facility - was picked
in a brainstorming session. The philosophy of the project was that IBM
would provide a way to collect data, but the customer must actually
report on the collected data (IBM couldn't see how to second-guess
customers on their use of accounting data).
After surveying data requirements, it was determined that SMF would have
to modify 160 modules of the operating system. This caused a great deal
of worry over system performance, and so IBM planned for
installation-selectable "packages" of information.
GUIDE finally got involved in the SMF project on May 22, 1967. On August
25, SHARE went to its membership with a survey which asked "How much
degradation is OK for collection of SMF data?"
The SMF exits and intercepts were originally called the "Dynamic Control
Capability". It consisted of a system management dataset, five data
collection packages, and "dynamic control entries" (which presumably
controlled which packages were to be activated). The data collection
packages each were to record a certain level of detail:
Package 1:
job level data
Package 2:
step level data
Package 3:
additional job/step data
Package 4:
still more data
Package 5:
sample data
Packages 1-4 were for accounting purposes. Package 5 was for performance
measurement (and was never implemented until MF/1 came around in MVS).
SMF was to provide utility reports based on data recorded in SYS1.MAN -
system, job and step profiles.
There was to be no performance degradation when no SMF is employed.
Packages 1-3 should not degrade the system more than 3%. Package 4 was
allowed to degrade the system 4%. Package 5 time/sample was to be no
more than 500 milliseconds on a 360/50 CPU.
Subset 2 of the SMF definition contained a "dataset management utility".
It provided an archive for the SYS1.MAN database. It also reported on
dataset OPENs and CLOSEs, and provided additional billing information.
On July 27, 1967, volume and PDS reorganization utilities were defined
as part of SMF (the system was to write a message to the console when a
PDS required reorganization)! IBM at the time had 550 users of OS/360
and they forecasted an eventual total of 1,590 users. With 75,000 CPUs
delivered worldwide, this estimate may have been low...
One of the SMF architects - Bart Page - was designing a SRM component
for SMF, but he was a better technician than he was a salesman. He
subsequently left SMF to go work on the MVM - Multiple Virtual Memories
- project.
In January 1968 the final specification for the management utility was
internally published, and included DASD and tape inventory functions.
The specification included utility syntax, JCL to execute and output
messages (numbered IEH901 through IEH916). The program name: IEHMAN! It
was tested on a 360/50, and took 19 minutes to update the archive from a
SYS1.MAN with 60 jobs on it.
On June 25, 1969 the Resource Management Utility was deleted with no
explanation. This was indeed a last-minute decision - there was already
an IEHMAN Planning Guide awaiting publication. Apparently signals were
crossed in Mechanicsburg, and some of the Planning Guides were actually
shipped (and subsequently recalled).
Type 16 records were eliminated and incorporated into types 14 and 15.
SMF was released.
IBM had estimated that SMF would get five APARs over the first six
months. Actually, 40 PTFs addressed 80 APARs.
SAS began its spectacular climb to success as an SMF analyzer (it
originally cost $100).
In the mid '70s, SMF began to develop some real performance problems.
Each write to the MANX/Y dataset required a TCLOSE, an ENQ and a write
verify. The "final" implementation of SMF (MVS/SE2) took 12 people
almost two years, four times the original estimate. It finally
implemented what SHARE people asked for 20 years earlier:
* Performance measurement
* Started task measurement
* Record selectivity
Its write performance was also enhanced when it went to VSAM: a BSAM
write takes about 60,000 instructions. A VSAM ICIP write in SRB mode
takes about 1,500.
Throughout the talk, Barry emphasized the working relationship between
IBM and SHARE during the SMF definition. He also repeatedly pointed out
the robust qualities of the architecture, that data requirements thought
of 20 years ago are still valid today.
--
David Andrews
A. Duda and Sons, Inc.
[EMAIL PROTECTED]
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
