You started with this: "Any program change requires full regression testing,
including "just a recompile"." I'm saying that paying "lip-service" to audit
requirements, and not confirming that the programs are exactly the same, is
heading (potentially) for exactly what you don't want. If the program is
supposed to be identical ("just a recompile") then, for me, "testing" is
proving that it is identical.
Fine, if the auditors don't accept that an identical program produces an
identical outcome, you are stymied. But I for sure would not just do
regression-tests when the program is supposed to be identical.
"ABO is different, the executable code changes. The changes are can be
"reconciled", through the listing. An automatic verification is going to give
way better results than any regression-test. Yes, for sure, that is a harder
argument with an auditor, but even if the regression test is still forced, I'd
go with the verification every time to "prove" the system, and, yes, that again
means the regression-test (applied for the entire process) is again degraded."
Again, I'm saying that independent verification will give you better than your
regression-tests. If the auditors still want regression-tests, I'm just saying
I'd do both, and the one I'd rely on for finding any theoretical potential
problem is not the regression-tests.
I'm not sure how you come to your conclusion about me when as far as I'm
concerned it provides for a more stable Production system. To summarise:
automate consumption and verification of the audit trail which ABO provides in
its listing: talk to the auditors; if it is not possible to win the argument
with the auditors (and I've both won and lost arguments with auditors) then you
have to follow the audit procedures laid down *in addition to verifying beyond
what is supplied by your regression tests*.
On top of that, from earlier, where there are "known problems" and "flaky
programs" take extra care, and perhaps exclude them from ABO. Plus the "Bust
ABO and You Win a T-Shirt Party"-type thing.
Now, wouldn't it be great if the American Institute of Auditors (I've invented
the name, but I'm sure there is at least one such organisation) and IBM got
together and certified ABO. Probably not going to happen.
Perhaps more feasible is an RFE to make the ABO output more readily
"consumable" by automation (although I don't know that it is difficult now).
As to "rolling your own", that's been IBM's response to at least one RFE
(unrelated to this) I voted for. "RFE: How about you add this functionality to
SDSF so everyone doesn't have to roll-their own with the REXX/SDSS?" Response
"Rejected, everyone can roll their own with REXX/SDSF".
For the ABO, yes, it sounds like you'll be regression-testing if you use it.
Those costs have to be factored in, but such is life. Your regression-tests
will make the auditors (and those who consume audit reports, which is lots of
important people, some of whom could theoretically close you down) happy. Keep
an eye on the ABO fix list.
I don't think your systems will be worse for the process (apart from the
potential "morale" issue of doing regression just for the sake of regression).
With the "just a recompile" you are dead wrong. Mitigated by your processes,
almost certainly. But if a program that is supposed to be identical is not, it
can quite easily pass your regression and bite you in tender portions. If it is
supposed to be the same, *check it is the same*. I seriously can't imagine any
other reasonable way to do it.
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN