Hi, Paul....
Paul Gilmartin wrote:
Looking for data validation, I find the CRC stage. But 16-bit checksums
are much in disrepute -- probably good enough for detecting noise, but
not sound cryptographically. (Heck, even the 128-bit MD5 is now
suspect.)
Is better available somewhere? SHA-1? Or even the lowly MD5? Weakness
notwithstanding, MD5 has the advantage of being readily available on
other platforms.
I don't happen to have a PIPE MD5 stage, but I do have a small Rexx
function that computes the MD5 checksum of a stream of data...it could
very easily be used from PIPES via a bit of Rexx code. Let me know if
you would like to get the VMARC file....
And I readily suspected and proved a weakness in PIPE CRC: files
with content:
a
bc
and:
ab
c
show identical CRC checksums -- apparently the algorithm ignores record
boundaries. This is unfortunate because such framing errors are readily
introduced by transfer to a stream-oriented workstation and back. And
consider Melinda's(?) assertion that CMS Pipelines is superior to other
pipe utilities because not all data should be treated as streams. I
suppose PIPE PACK | CRC can be used to overcome the record boundary
deficiency.
There should be identical checksums for your two example files; I
believe that the CRC algorithm is defined to work on data that is
represented by a simple string of bytes, so, yes, it should ignore
record boundaries. I don't think this reveals a weakness in PIPES CRC
stage; I think it shows that it is working as designed.
If you want the two example files shown above to yield different CRC
values, one approach might be to denote the record boundaries
explicitly, e.g., adding a NL (x'15') to the end of each record before
it is passed to the CRC stage:
....| specs 1-* 1 x15 n | crc....
For my testing, I would have liked a stage comparable to the Rexx C2X
function. Of course, it's easy to write, but has someone else already
invented that wheel?
I am sure I am not really understanding your question here, but what's
wrong with a simple "| specs 1-* c2x 1" stage inserted wherever you
wanted to see the data flowing through the pipe in hex?
Thanks,
gil
--
StorageTek
INFORMATION made POWERFUL
--
DJ V/Soft
