I feel your pain, John. For details of my shock period, see the end of this reply. Long note.
Before you get too annoyed ... the 0x15 thing is actually a blessing. It turns out that since USS plain text uses EBCDIC NL and all other Unix use ASCII LF, you can programatically recognize one or the other and adjust reliably. In both character sets, 0x15 and 0x0a are non-printable. But 0x85, which is the official ASCII "newline", might be mistaken as an EBCDIC lower case "e". And the EBCDIC "linefeed" is 0x25, which could be an ASCII percent sign. On USS, PAX gets it right. Dunno how to beat 'iconv' into submission. When you find out, please let me know. I have some old and very crude code based on this surprising characteristic. It scans an input stream that is supposed to be plain text. When it sees a 0x0a, it treats the preceding content as ASCII. When it sees a 0x15, it treats the preceding content as EBCDIC. It then either translates or not based on the local environment. Unix made the mistake of calling 0x0a "newline". It's not. Never was. 0x0a is "linefeed". The mixup probably came from abstraction semantics (eg: backslash n in C programs). But this is just a guess. EBCDIC also has both linefeed (0x25) and carriage return (0x0d, surprisingly the same as for ASCII). When ASCII grew up from 7-bit to 8-bit, it got a formal newline, which is 0x85. Welcome to the world of control characters. Back in the late 80s, some good people put a lot of time into normalizing all this stuff. As for me ... when I first saw OpenVM, I was stoked! Mount the root FS, then 'openvm shell', and voi-la! All the minimal standard stuff was there. (Of course there was a compiler to build the add-ons everyone demanded.) So I brought in a tarball with my stuff, cracked it open, tried to run one of my handy shell scripts ... bbbzzzttt!!! Didn't work. Not just didn't work, it was as if something had gotten really trashed. Looking at the file, I realized that it was ASCII (as expected) but that the byte filesystem side of CMS was EBCDIC (not expected). I was totally bummed out and didn't touch OpenVM for like 7 months. I was spittin bullets mad at IBM for this stupidity. When you cross the line from record oriented to byte oriented, add that "newline" but also translate the rest. Yes? No. Then I realized that they HAD to do it the way they did (stream, yes, but EBCDIC). There would have been all kinds of issues with compilers and things like that if USS (and OpenVM) spoke ASCII. (Not that I would not personally be delighted.) I do wish that all USS and OpenVM utils which expect "plain text" would include and automatic 0x15 / 0x0a sense on their input. I'm not asking for much. :-) -- R; <>< On Fri, Jun 4, 2010 at 08:39, McKown, John <[email protected]> wrote: > Thanks to all for these scripts. I've now got another whine for IBM about > z/OS UNIX. z/OS UNIX uses "new line" (0x15) as a line ending instead of LF > (line feed). So, when I use iconv on Linux, the 0x15 gets translated to 0x85. > EBCDIC is becoming a royal PITA for me! The z/OS version of pax has a > "properly perverted" translate table to translate 0x15 to 0x0a and vice versa > on its translation. > > -- > John McKown > Systems Engineer IV > IT > > Administrative Services Group > > HealthMarkets(r) > > 9151 Boulevard 26 * N. Richland Hills * TX 76010 > (817) 255-3225 phone * (817)-961-6183 cell > [email protected] * www.HealthMarkets.com > > Confidentiality Notice: This e-mail message may contain confidential or > proprietary information. If you are not the intended recipient, please > contact the sender by reply e-mail and destroy all copies of the original > message. HealthMarkets(r) is the brand name for products underwritten and > issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake > Life Insurance Company(r), Mid-West National Life Insurance Company of > TennesseeSM and The MEGA Life and Health Insurance Company.SM > > ---------------------------------------------------------------------- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
