Re: FTP converting between UTF-8 and EBCDIC
On Mon, 16 Nov 2020 02:28:06 +, Frank Swarbrick wrote: >We don't use Unix for any of our production business applications. If we were >"starting from scratch" I imagine we might choose to use Unix files for many >things, but I can't see us going this direction now. > >This is an existing process with an existing MVS data set, and up until a >couple of days ago it was treated (on the remote side) as just a standard >"ASCII" file. A couple of days ago a user entered as last name with a >lower-case 'a' with an accent, causing a two-byte character to be used where >it had always been a single byte before. > Did that (â?) cause the last name to overflow a field? If not, what's the problem? No single SBCS code page can accommodate all characters you're likely to encounter. You might filter on input to acceptable, SBCS-translatable characters. Expect accusations of ethnic bias if you do that. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: FTP converting between UTF-8 and EBCDIC
We don't use Unix for any of our production business applications. If we were "starting from scratch" I imagine we might choose to use Unix files for many things, but I can't see us going this direction now. This is an existing process with an existing MVS data set, and up until a couple of days ago it was treated (on the remote side) as just a standard "ASCII" file. A couple of days ago a user entered as last name with a lower-case 'a' with an accent, causing a two-byte character to be used where it had always been a single byte before. From: IBM Mainframe Discussion List on behalf of Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu> Sent: Sunday, November 15, 2020 4:15 PM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: FTP converting between UTF-8 and EBCDIC On Sun, 15 Nov 2020 22:44:05 +, Frank Swarbrick wrote: >I couldn't find this actually stated anywhere, but it looks like if you want >to use z/OS FTP to transmit a UTF-8 encoded file to (and I assume from) an >EBCDIC codepage you have to use multi-byte encoding instead of single byte >encoding. The z/OS file also has to be variable length, not fixed length. > Why bother? Just transfer in binary to a z/OS UNIX file. ISPF Edit, for example, handles that format splendidly, subject to the display capability of the attached terminal; even automatically if the file is tagged as CCSID 1208. >Here is an example using curl. Note that this works even on standard Windows >10, as curl.exe is now provided with Windows 10. If you are using PowerShell >instead of CMD.exe you need to specify "curl.exe" instead of just "curl". >Search the internet for why, if you care. > >curl --user DVFJS --ftp-ssl --insecure --use-ascii --quote "site >mbdataconn=(ibm-1140,utf-8) encoding=mbcs" --upload-file >20201106180101ICMUpload.txt ftp://ZOSD:8443/'DEV.RXMTIN.ICM.ICMM2M.VB' > >"--insecure" is necessary for me because we have a self-signed FTP server >certificate. It may not be necessary for you. > >Note that this is only required if the source file actually has one or more >multi-byte UTF-8 codepoints. A UTF-8 file that only has single-byte >codepoints is identical to a standard 7-bit ASCII file and can be transmitted >using a default sbcs encoding. > >I honestly was surprised this work, as the documentation (as I read it, >anyway) didn't seem to indicate that conversion between a multi-byte codeset >(UTF-8) and a single-byte EBCDIC codeset (IBM-1140, in this case) is possible. > But apparently it is. Note that I didn't test anything where the EBCDIC >codepage does not have a corresponding codepoint for a particular character. >The character in question in this example is "�", which is code-sequence >x'C3A1' in UTF-8 and x'45' in ECBDIC-1140. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: FTP converting between UTF-8 and EBCDIC
On Sun, 15 Nov 2020 22:44:05 +, Frank Swarbrick wrote: >I couldn't find this actually stated anywhere, but it looks like if you want >to use z/OS FTP to transmit a UTF-8 encoded file to (and I assume from) an >EBCDIC codepage you have to use multi-byte encoding instead of single byte >encoding. The z/OS file also has to be variable length, not fixed length. > Why bother? Just transfer in binary to a z/OS UNIX file. ISPF Edit, for example, handles that format splendidly, subject to the display capability of the attached terminal; even automatically if the file is tagged as CCSID 1208. >Here is an example using curl. Note that this works even on standard Windows >10, as curl.exe is now provided with Windows 10. If you are using PowerShell >instead of CMD.exe you need to specify "curl.exe" instead of just "curl". >Search the internet for why, if you care. > >curl --user DVFJS --ftp-ssl --insecure --use-ascii --quote "site >mbdataconn=(ibm-1140,utf-8) encoding=mbcs" --upload-file >20201106180101ICMUpload.txt ftp://ZOSD:8443/'DEV.RXMTIN.ICM.ICMM2M.VB' > >"--insecure" is necessary for me because we have a self-signed FTP server >certificate. It may not be necessary for you. > >Note that this is only required if the source file actually has one or more >multi-byte UTF-8 codepoints. A UTF-8 file that only has single-byte >codepoints is identical to a standard 7-bit ASCII file and can be transmitted >using a default sbcs encoding. > >I honestly was surprised this work, as the documentation (as I read it, >anyway) didn't seem to indicate that conversion between a multi-byte codeset >(UTF-8) and a single-byte EBCDIC codeset (IBM-1140, in this case) is possible. > But apparently it is. Note that I didn't test anything where the EBCDIC >codepage does not have a corresponding codepoint for a particular character. >The character in question in this example is "�", which is code-sequence >x'C3A1' in UTF-8 and x'45' in ECBDIC-1140. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Improve OMVS cp performance?
On Sun, 15 Nov 2020 16:28:02 -0600, Mike Schwab wrote: >They only had 7 track tapes at the time. > >9 track tapes introduced with S/360 in 1964. >https://en.wikipedia.org/wiki/9_track_tape > Which was no obstacle. There were two modes of I/O to 7-track tapes: TRTCH=T, which mapped each tape frame of 6 data bits plus 1 parity to one 8-bit byte. TRTCH=C, which mapped the 24 data bits of 4 tape frames to 3 bytes I know; I successfully used the latter. I had to argue with ops who tried to tell me that I shouldn't use that; it would corrupt mu data. It didn't. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
FTP converting between UTF-8 and EBCDIC
I couldn't find this actually stated anywhere, but it looks like if you want to use z/OS FTP to transmit a UTF-8 encoded file to (and I assume from) an EBCDIC codepage you have to use multi-byte encoding instead of single byte encoding. The z/OS file also has to be variable length, not fixed length. Here is an example using curl. Note that this works even on standard Windows 10, as curl.exe is now provided with Windows 10. If you are using PowerShell instead of CMD.exe you need to specify "curl.exe" instead of just "curl". Search the internet for why, if you care. curl --user DVFJS --ftp-ssl --insecure --use-ascii --quote "site mbdataconn=(ibm-1140,utf-8) encoding=mbcs" --upload-file 20201106180101ICMUpload.txt ftp://ZOSD:8443/'DEV.RXMTIN.ICM.ICMM2M.VB' "--insecure" is necessary for me because we have a self-signed FTP server certificate. It may not be necessary for you. Note that this is only required if the source file actually has one or more multi-byte UTF-8 codepoints. A UTF-8 file that only has single-byte codepoints is identical to a standard 7-bit ASCII file and can be transmitted using a default sbcs encoding. I honestly was surprised this work, as the documentation (as I read it, anyway) didn't seem to indicate that conversion between a multi-byte codeset (UTF-8) and a single-byte EBCDIC codeset (IBM-1140, in this case) is possible. But apparently it is. Note that I didn't test anything where the EBCDIC codepage does not have a corresponding codepoint for a particular character. The character in question in this example is "á", which is code-sequence x'C3A1' in UTF-8 and x'45' in ECBDIC-1140. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Improve OMVS cp performance?
Telefunken TR 4, designed 1958, first delivered in 1962. This predates
IBM/360
by at least 4 years. The fastest mainframe built in Europe at that time.
The internal code ("Zentralcode") was an 8-bit code using 256 characters.
Word structure, a word had 48 bits plus 2 tag bits (tagged architecture)
plus some parity bits, not seen by the programmer.
Mostly used with ALGOL; the TR 4 was "a Hardware implementation of ALGOL"
(quote from E.J. Dijkstra).
A word could hold up to six characters, but some later languages
(like Fortran) decided to store only 4 characters in one word,
to be more compatible with IBM Fortran.
Kind regards
Bernd
Am 15.11.2020 um 03:25 schrieb Mike Schwab:
You have to remember that S/360 was the first 8 bit computer. Prior
computers used 4 bits for a digit and 6 bits for a character. They
designed EBCDIC to be easily converted for use with existing 7 track
tape drives, printers, card and tape readers and punches. There was a
proposed ASCII code that was put on documentation but dropped for the
370 virtual memory bit in the PSW.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Improve OMVS cp performance?
They only had 7 track tapes at the time. 9 track tapes introduced with S/360 in 1964. https://en.wikipedia.org/wiki/9_track_tape On Sun, Nov 15, 2020 at 3:36 PM Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu> wrote: > > On Sun, 15 Nov 2020 15:13:24 -0600, Mike Schwab wrote: > > >Sorry. First computer to use 8 bits per character. > > > Wikipedia, not necessarily an authority, says: > https://en.wikipedia.org/wiki/IBM_7030_Stretch#Data_formats > Alphanumeric characters are variable length and can use any character > code of 8 bits or less. > > ... so, if true, 8 bits was at least possible. Was it widely in "use"? > > >On Sun, Nov 15, 2020 at 11:09 AM Seymour J Metz wrote: > >> > >> > You have to remember that S/360 was the first 8 bit computer. > >> > >> What is the 7030, chopped liver? > > -- gil > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Improve OMVS cp performance?
On Sun, 15 Nov 2020 15:13:24 -0600, Mike Schwab wrote: >Sorry. First computer to use 8 bits per character. > Wikipedia, not necessarily an authority, says: https://en.wikipedia.org/wiki/IBM_7030_Stretch#Data_formats Alphanumeric characters are variable length and can use any character code of 8 bits or less. ... so, if true, 8 bits was at least possible. Was it widely in "use"? >On Sun, Nov 15, 2020 at 11:09 AM Seymour J Metz wrote: >> >> > You have to remember that S/360 was the first 8 bit computer. >> >> What is the 7030, chopped liver? -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Improve OMVS cp performance?
Sorry. First computer to use 8 bits per character. On Sun, Nov 15, 2020 at 11:09 AM Seymour J Metz wrote: > > > You have to remember that S/360 was the first 8 bit computer. > > What is the 7030, chopped liver? > > > -- > Shmuel (Seymour J.) Metz > http://mason.gmu.edu/~smetz3 > > > > From: IBM Mainframe Discussion List on behalf of > Mike Schwab > Sent: Saturday, November 14, 2020 9:25 PM > To: IBM-MAIN@LISTSERV.UA.EDU > Subject: Re: Improve OMVS cp performance? > > You have to remember that S/360 was the first 8 bit computer. Prior > computers used 4 bits for a digit and 6 bits for a character. They > designed EBCDIC to be easily converted for use with existing 7 track > tape drives, printers, card and tape readers and punches. There was a > proposed ASCII code that was put on documentation but dropped for the > 370 virtual memory bit in the PSW. > > On Sat, Nov 14, 2020 at 6:39 PM Seymour J Metz wrote: > > > > I doubt that IBM custumers would have been happy with an 8-bit code page > > with only 128 valid code points. International considerations would still > > have forced IBM to device incompatible code pages for different countries. > > > > Obviously 8859 is another Tower of Babel; why do you think I described it > > as "a dollar short"? > > > > No,, IBM could not have implemented full Unicode, or even the full MLP, > > back in the 1960s. But it could certainly have implemented a basic subset > > for all customers and selected additional pages for international > > customers. Had Unicode and UTF-8 been around at the time, I'm certain that > > IBM would have gone that route. > > > > > > -- > > Shmuel (Seymour J.) Metz > > http://mason.gmu.edu/~smetz3 > > > > > > > > From: IBM Mainframe Discussion List on behalf of > > Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu> > > Sent: Saturday, November 14, 2020 6:22 PM > > To: IBM-MAIN@LISTSERV.UA.EDU > > Subject: Re: Improve OMVS cp performance? > > > > On Sat, 14 Nov 2020 23:00:00 +, Seymour J Metz wrote: > > > > >Because there was no standard 8-bit code at the time. IBM did push for an > > >8-bit ASCII, > > > > > That's not an obstacle. DEC PDP-8 stored ASCII characters one per > > 12-bit word. IBM could have simply declared the top bit "reserved" > > as they are so often wont to do. > > > > >but it never happened except for a mapping between octets and punch > > >combinations on cards. Had Unicode been around at the time they would > > >probably have jumped at it. > > > > > >ISO 8859 was a day late and a dollar short. > > > > > ISO-8859-* is afflicted with the same babel as EBCDIC code pages > > because of the "*" you elided. > > > > UTF-8 is the norm nowadays because of a peculiar upward compatibility > > with ASCII. But the mebibytes and megahertz to support it came a day late. > > > > -- gil > > > > -- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > > -- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > > -- > Mike A Schwab, Springfield IL USA > Where do Forest Rangers go to get away from it all? > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Improve OMVS cp performance?
> You have to remember that S/360 was the first 8 bit computer. What is the 7030, chopped liver? -- Shmuel (Seymour J.) Metz http://mason.gmu.edu/~smetz3 From: IBM Mainframe Discussion List on behalf of Mike Schwab Sent: Saturday, November 14, 2020 9:25 PM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: Improve OMVS cp performance? You have to remember that S/360 was the first 8 bit computer. Prior computers used 4 bits for a digit and 6 bits for a character. They designed EBCDIC to be easily converted for use with existing 7 track tape drives, printers, card and tape readers and punches. There was a proposed ASCII code that was put on documentation but dropped for the 370 virtual memory bit in the PSW. On Sat, Nov 14, 2020 at 6:39 PM Seymour J Metz wrote: > > I doubt that IBM custumers would have been happy with an 8-bit code page with > only 128 valid code points. International considerations would still have > forced IBM to device incompatible code pages for different countries. > > Obviously 8859 is another Tower of Babel; why do you think I described it as > "a dollar short"? > > No,, IBM could not have implemented full Unicode, or even the full MLP, back > in the 1960s. But it could certainly have implemented a basic subset for all > customers and selected additional pages for international customers. Had > Unicode and UTF-8 been around at the time, I'm certain that IBM would have > gone that route. > > > -- > Shmuel (Seymour J.) Metz > http://mason.gmu.edu/~smetz3 > > > > From: IBM Mainframe Discussion List on behalf of > Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu> > Sent: Saturday, November 14, 2020 6:22 PM > To: IBM-MAIN@LISTSERV.UA.EDU > Subject: Re: Improve OMVS cp performance? > > On Sat, 14 Nov 2020 23:00:00 +, Seymour J Metz wrote: > > >Because there was no standard 8-bit code at the time. IBM did push for an > >8-bit ASCII, > > > That's not an obstacle. DEC PDP-8 stored ASCII characters one per > 12-bit word. IBM could have simply declared the top bit "reserved" > as they are so often wont to do. > > >but it never happened except for a mapping between octets and punch > >combinations on cards. Had Unicode been around at the time they would > >probably have jumped at it. > > > >ISO 8859 was a day late and a dollar short. > > > ISO-8859-* is afflicted with the same babel as EBCDIC code pages > because of the "*" you elided. > > UTF-8 is the norm nowadays because of a peculiar upward compatibility > with ASCII. But the mebibytes and megahertz to support it came a day late. > > -- gil > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Have I misunderstood TOD clock & leap seconds?
I submitted an RCF for clarification of STCKCONV a couple years ago. I got an (automated?) notification of receipt; no resolution. I'll wait to see what the 2.5 (when?) doc says. You asked for this particular clarification on June 16 of this year. You were sent an email on July 3 describing the update. The z/OS 2.4 book shows the update made to CONVTOD and STCKCONV, in both cases adding "The leap second offset is not considered when determining the output.". Your query from 2018 was basically with respect to LOCAL vs GMT time. As we have discussed multiple times, we do not intend to change to mention that in the book. Peter Relson z/OS Core Technology Design -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Improve OMVS cp performance?
CMS/TSO Pipes!!! On Sat, 14 Nov 2020, 8:26 pm Mike Schwab, wrote: > You have to remember that S/360 was the first 8 bit computer. Prior > computers used 4 bits for a digit and 6 bits for a character. They > designed EBCDIC to be easily converted for use with existing 7 track > tape drives, printers, card and tape readers and punches. There was a > proposed ASCII code that was put on documentation but dropped for the > 370 virtual memory bit in the PSW. > > On Sat, Nov 14, 2020 at 6:39 PM Seymour J Metz wrote: > > > > I doubt that IBM custumers would have been happy with an 8-bit code page > with only 128 valid code points. International considerations would still > have forced IBM to device incompatible code pages for different countries. > > > > Obviously 8859 is another Tower of Babel; why do you think I described > it as "a dollar short"? > > > > No,, IBM could not have implemented full Unicode, or even the full MLP, > back in the 1960s. But it could certainly have implemented a basic subset > for all customers and selected additional pages for international > customers. Had Unicode and UTF-8 been around at the time, I'm certain that > IBM would have gone that route. > > > > > > -- > > Shmuel (Seymour J.) Metz > > http://mason.gmu.edu/~smetz3 > > > > > > > > From: IBM Mainframe Discussion List on > behalf of Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu> > > Sent: Saturday, November 14, 2020 6:22 PM > > To: IBM-MAIN@LISTSERV.UA.EDU > > Subject: Re: Improve OMVS cp performance? > > > > On Sat, 14 Nov 2020 23:00:00 +, Seymour J Metz wrote: > > > > >Because there was no standard 8-bit code at the time. IBM did push for > an 8-bit ASCII, > > > > > That's not an obstacle. DEC PDP-8 stored ASCII characters one per > > 12-bit word. IBM could have simply declared the top bit "reserved" > > as they are so often wont to do. > > > > >but it never happened except for a mapping between octets and punch > combinations on cards. Had Unicode been around at the time they would > probably have jumped at it. > > > > > >ISO 8859 was a day late and a dollar short. > > > > > ISO-8859-* is afflicted with the same babel as EBCDIC code pages > > because of the "*" you elided. > > > > UTF-8 is the norm nowadays because of a peculiar upward compatibility > > with ASCII. But the mebibytes and megahertz to support it came a day > late. > > > > -- gil > > > > -- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > > -- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > > -- > Mike A Schwab, Springfield IL USA > Where do Forest Rangers go to get away from it all? > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Have I misunderstood TOD clock & leap seconds?
I disagree with both. z/OS TOD clock is unaffected; user processes are suspended as necessary to preclude duplicate/anachronistic TIME readings. I rarely remember the exact details, but I remember enough to know that the above is not correct in general. Spinning with nothing running (which you could consider to be "suspended"), until you could not get things appearing out of order, is one choice z/OS provides. It is not the only choice that z/OS provides, depending on your hardware configuration. Clock steering does happen. That might not mean that the machine slows down. Peter Relson z/OS Core Technology Design -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
