Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Peter Sylvester

On 06/12/2012 10:56 PM, John Gilmore wrote:

The term 'bijective' is a fairly old one, the earliest reference I
found in a Mathematical Reviews index was for 1939..  Anyone who has
had a college course in  mathematical logic or, yes, set theory is
likely to know or have forgotten its meaning.

It does, however, have a bad reputation because of its use to
intimidate non-mathematicians.  The example in Quine's Quiddities
under the topic mathematosis illustrates why.

Still, it is perhaps better than 'roundtrip', which in ordinary use
does not really imply bijective.  My wife and I both found that we
were measurably heavier after a recent Boston-Roma-Boston trip.
Additional efforts were required to restore our status quo ante
weights.


you have two mappings between unicode and ebcdic,
you need an injection from ebcdic to unicode,
and a surjection from the image of that injection
which must be the inverse of the injection.
thus both mappings are bijections between their
field and image and inverse to each other.

in fact no one needs mappings, codes, computers at all,
you just take the universal library, i.e. the bit.

field and image of you two mappings of your roundtrip
are different, still, at least almost, applying a regime
may create a projection that preserve your integrities.
you don't become your wife for example or have your
head cut off.

a mapping that exchanges your with your with is
perfectly bijective and even using itself as inverse is so,
but still, you may have a problem when + is interpreted
as - and vice versa, so you also want preservation of
the somewhat implied semantics of a character/person.

have fun

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Shmuel Metz (Seymour J.)
In 00f101cd48c5$195bdba0$4c1392e0$@mcn.org, on 06/12/2012
   at 10:59 AM, Charles Mills charl...@mcn.org said:

Fair enough. Unicode services reports however that it supports
roundtrip conversion in many of these cases, including for example,
37 to 850 (pretty basic ASCII).

850 is not ASCII; it includes code points from 128 to 244.
 
-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 ISO position; see http://patriot.net/~shmuel/resume/brief.html 
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Bill Godfrey
The same glossary says this about code point: A unique bit pattern defined in 
a code. Depending on the code, a code point can be 7-bit, 8-bit, 16-bit, or 
other. Code points are assigned to a graphic character in a code page.

Note the last sentence of that.

Also, you earlier quoted a section of a DB2 website about round trip 
conversion, including this:

Example: In ASCII CCSID 1252, the trademark symbol T is code point X'99'. In 
EBCDIC CCSID 37, this code point does not exist.

So a code point is not just any possible combination of bits, it's a 
combination of bits that is assigned to a character. It is possible to say that 
a code point does not exist in a CCSID. Therefore the description of round 
trip that you quoted:

Round trip. Encoding that occurs when *every* code point in the source CCSID 
maps to a unique code point in the target CCSID

can be read to apply only to certain bit combinations, those to which a 
character is assigned, and not to all bit combinations.

The glossary in the DB2 website that you referenced earlier has this for code 
point:

In CDRA, a unique bit pattern that represents a character in a code page.

Bill

On Tue, 12 Jun 2012 21:07:51 -0700, Charles Mills wrote:

1027 was a test case and the data is arbitrary. I think the point of round
trip is supposed to be that *every* character is recoverable.

A good write-up in of all places g the Glossary of the Z Unicode manual.
Emphasis is mine.

Round trip. Encoding that occurs when *every* code
point in the source CCSID maps to a unique code point
in the target CCSID. Using round trip tables ensure the
capability of reversing the conversion, and recovering
the complete original source datastream.

I'm going to see what they say on the PMR.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Bill Godfrey
Sent: Tuesday, June 12, 2012 7:12 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Maybe it will turn out that round trip only applies to the characters that
are defined in the Source CCSID (and I'm not sure if defined is the right
word). When I look at a chart of CCSID 1027 here:

ftp://ftp.software.ibm.com/software/globalization/gcoc/attachments/CP01027.p
df

I see that all of the characters that you see being being converted to X'1A'
are either outside the chart (which starts at X'40') or have no identified
value in the chart. If your data contains these unidentified characters,
what is their purpose, and is it really appropriate to consider the data to
be associated with CCSID 1027, or just something related to but not the same
as CCSID 1027?

Bill

On Tue, 12 Jun 2012 16:42:02 -0700, Charles Mills wrote:

Not sure what you mean. Here's the PMR:

Problem Details
.
Product or Service: Support for Unicode Component ID: 5752SCUNI .
Operating System: z/OS
.
Problem title
Round trip conversion not working as expected .
Problem description
My understanding of round trip conversion is that every code point in
the Source CCSID should convert to a unique code point in the Target
CCSID (so that conversion in the reverse direction will restore the
original code points). However, when I use CUNLCNV to convert between
CCSIDs 1027 and 1208 I see more than one character converted to x'1A'
(UTF8 SUB). Is round trip conversion working correctly? (Both the
documentation and CUNLINFO report that 'R' is a supported conversion
technique for these two CCSIDs.)

Characters converted to X'1A' include 3F, 41, 6A, 80, 90, etc.

Thank you for your consideration.
.
Business impact ( BusImpact )
Negatively impacting product design decisions.

Charles
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On
Behalf Of Tony Harminc
Sent: Tuesday, June 12, 2012 4:15 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

On 12 June 2012 18:55, Charles Mills charl...@mcn.org wrote:
 I have opened a PMR.

For the doc, or the behaviour of the service? Or did you choose the
let us decide for you option...?

Tony H.


--
For IBM-MAIN subscribe / signoff / archive access instructions, send email
to lists...@bama.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Charles Mills
I got a response to the PMR. Taking the liberty of paraphrasing a long
reply, the essence of it seemed to be that -- per the CCSID pair lists in
the manual -- they support round trip conversion from 1027 to 1208 but not
from 1208 to 1027. Here is what I wrote back:

It sounds like you are saying (for the CCSIDs in question) we support  
round trip, but in one direction only. It would be like if I bought a  
round trip ticket on Delta between San Francisco and Atlanta, and after 
I got to Atlanta, they explained that it was a round trip ticket only   
in one direction.   

I would kind of question also whether what you are doing conforms to
your definition of round trip in the Unicode manual glossary: Round 
trip. Encoding that occurs when every code point in the source CCSID
maps to a unique code point in the target CCSID. Using round trip   
tables ensure the capability of reversing the conversion, and   
recovering the complete original source datastream. 

I would question every code point in the source CCSID maps to a unique 
code point in the target CCSID when both 3F and 41 map to the same 
code point, and I wonder how I would recover the original source
datastream.

Charles

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread McKown, John
I __think__ I understand what IBM is saying. You must differentiate between a 
hex value (0x00..0xFF) and a code point (a subset of hex values). Not 
every hex value is a code point in every CCSID. I.e. a single byte CCSID 
may have less than 256 code points, or a double byte CCSID may have less than 
65336 code points. A round trip conversion from one CCSID to a different 
CCSID says that every possible code point in CCSID#1 will map to a unique 
code point in CCSID#1. I.e. the trip is on a two way street and you can 
turn around to go back to where you started. However, there may be a one 
way street from CCSID#2 to CCSID#1. So when you start in CCSID#2, you cannot 
just turn around when you get do CCSID#1 and return to your starting 
location. 

Think of the code points in a CCSID as streets in a city. If the street is 
two way (round trip), then you can go back the way you came. If the street is 
one way, then you can't. So CCSID#2 may have a one way street to CCSID#1. 
You can make the trip, but if you go back, you must use a different street. So 
when you go back to CCSID#2, you end up somewhere else in CCSID#2 from where 
you started. 

A round trip route from CCSID#1 to CCSID#2 means every street from CCSID#1 to 
CCSID#2 is two way. But that does not mean that every street from CCSID#2 to 
CCSID#1 must be a two way street. There may be a one way trip from #2 to 
#1. So you cannot say that #2 is round trip to #1. That is, every code 
point in CCSID#2 must go to a code point in CCSID#1, but if more than one 
does, then the conversion is not classified round trip.

And it means that, so far as these conversions are concerned, if you use a hex 
value not in the set of code points, nothing at all is guarnateed (you went 
off road so to speak). So round trip does not say that every hex value 
maps to a unique destination. In fact a hex value which is not a code point 
in a source CCSID could actually translate to a hex value which is a code 
point in the destination CCSID.

I now have a headache and am going to have a lay down.

-- 
John McKown 
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone * 
john.mck...@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or 
proprietary information. If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message. 
HealthMarkets(r) is the brand name for products underwritten and issued by the 
insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance 
Company(r), Mid-West National Life Insurance Company of TennesseeSM and The 
MEGA Life and Health Insurance Company.SM


 -Original Message-
 From: IBM Mainframe Discussion List 
 [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Charles Mills
 Sent: Wednesday, June 13, 2012 1:00 PM
 To: IBM-MAIN@bama.ua.edu
 Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion
 
 I got a response to the PMR. Taking the liberty of paraphrasing a long
 reply, the essence of it seemed to be that -- per the CCSID 
 pair lists in
 the manual -- they support round trip conversion from 1027 to 
 1208 but not
 from 1208 to 1027. Here is what I wrote back:
 
 It sounds like you are saying (for the CCSIDs in question) 
 we support  
 round trip, but in one direction only. It would be like if I 
 bought a  
 round trip ticket on Delta between San Francisco and Atlanta, 
 and after 
 I got to Atlanta, they explained that it was a round trip 
 ticket only   
 in one direction. 
   
   
   
 I would kind of question also whether what you are doing 
 conforms to
 your definition of round trip in the Unicode manual glossary: 
 Round 
 trip. Encoding that occurs when every code point in the 
 source CCSID
 maps to a unique code point in the target CCSID. Using round 
 trip   
 tables ensure the capability of reversing the conversion, and 
   
 recovering the complete original source datastream.   
   
   
   
 I would question every code point in the source CCSID maps 
 to a unique 
 code point in the target CCSID when both 3F and 41 map to 
 the same 
 code point, and I wonder how I would recover the original 
 source
 datastream.
 
 Charles
 
 --
 For IBM-MAIN subscribe / signoff / archive access instructions,
 send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
 
 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Mike Schwab
On Wed, Jun 13, 2012 at 12:59 PM, Charles Mills charl...@mcn.org wrote:
 I got a response to the PMR. Taking the liberty of paraphrasing a long
 reply, the essence of it seemed to be that -- per the CCSID pair lists in
 the manual -- they support round trip conversion from 1027 to 1208 but not
 from 1208 to 1027. Here is what I wrote back:

What this means is:
Roundtrip example:  Every defined character in 1027, excluding values
that do not have a character defined, exist in 1208, is successfully
translated from 1027 to 1208 and back to 1027.  All codepoints that do
not have a character defined will be translated to (one?) non-valid
value.

Non-roundtrip example:  Some defined characters in 1208, and all
codepoints that do not have a character defined, do not exist in 1027.
 If you translate text from 1208 to 1027, the characters not defined
in 1027, and all undefined codepoints will be translated to (one?)
non-valid codepoint.

In any translation, a codepoint that does not encode a character will
be translated to (one?) non-valid codepoint.

If, while the text is in 1208 is changed to add a character not in
1027, upon translation that value will be changed to an invalid value
in 1027.

 It sounds like you are saying (for the CCSIDs in question) we support
 round trip, but in one direction only. It would be like if I bought a
 round trip ticket on Delta between San Francisco and Atlanta, and after
 I got to Atlanta, they explained that it was a round trip ticket only
 in one direction.

 I would kind of question also whether what you are doing conforms to
 your definition of round trip in the Unicode manual glossary: Round
 trip. Encoding that occurs when every code point in the source CCSID
 maps to a unique code point in the target CCSID.

You missed the word defined.  If a codepoint does not have a defined
character, it is translated to (one?) invalid value.

 Using round trip
 tables ensure the capability of reversing the conversion, and
 recovering the complete original source datastream.

 I would question every code point in the source CCSID maps to a unique
 code point in the target CCSID when both 3F and 41 map to the same
 code point, and I wonder how I would recover the original source
 datastream.

Again, you missed the word defined.  If a codepoint does not have a
defined character, it is translated to (one?) invalid value.

 Charles


-- 
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Shmuel Metz (Seymour J.)
In
93891f43642f3c419a7d75acc2b1db6f3c68cc9...@exchangemb2.dhs.state.ia.us,
on 06/12/2012
   at 03:38 PM, Roberts, John J jrobe...@dhs.state.ia.us said:

Because bijective is not so well understood by anyone born before
1952 or so. 

I was born before 1952 and understand bijective just fine. OTOH,
round trip is something I specify when I am buying travel tickets.

OTOH, a mapping from an EBCDIC code page to a Unicode code page can't
be bijective, only injective, and there is no round trip for, e.g.,
Unicode-EBCDIC-Unicode, regardless of which EBCDIC code page and
Unicode transform you use. Contrast left inverse with right
inverse.
 
-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 ISO position; see http://patriot.net/~shmuel/resume/brief.html 
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Charles Mills
I did not miss the word defined. It's not in IBM's definition.

I hear everyone who is saying the term 'code point' *really* means a bit
value with a glyph assigned to it, but that's not what the definitions out
there say. Wikipedia:

In character encoding terminology, a code point or code position is any of
the numerical values that make up the code space (or code page).[1] For
example, ASCII comprises 128 code points in the range 0hex to 7Fhex,
Extended ASCII comprises 256 code points in the range 0hex to FFhex, and
Unicode comprises 1,114,112 code points in the range 0hex to 10hex. The
Unicode code space is divided into seventeen planes (the basic multilingual
plane, and 16 supplementary planes), each with 65,536 (= 216) code points.
Thus the total size of the Unicode code space is 17 × 65,536 = 1,114,112.

ASCII comprises 128 code points. Not all 128 of those have glyphs.

Unicode comprises 1,114,112 code points. Not all of those million-plus
code points have glyphs.

every code point in the source CCSID maps to a unique code point in the
target CCSID (note no defined)

That says every one of the 128 ASCII code points maps to a unique bit
combination. But it ain't so.

If one is going to define round trip conversion as applying only to
corresponding glyphs then the definition loses any meaning. Any rational
translation is round trip with regard to corresponding glyphs.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Mike Schwab
Sent: Wednesday, June 13, 2012 12:02 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

On Wed, Jun 13, 2012 at 12:59 PM, Charles Mills charl...@mcn.org wrote:
 I got a response to the PMR. Taking the liberty of paraphrasing a long 
 reply, the essence of it seemed to be that -- per the CCSID pair lists 
 in the manual -- they support round trip conversion from 1027 to 1208 
 but not from 1208 to 1027. Here is what I wrote back:

What this means is:
Roundtrip example:  Every defined character in 1027, excluding values that
do not have a character defined, exist in 1208, is successfully translated
from 1027 to 1208 and back to 1027.  All codepoints that do not have a
character defined will be translated to (one?) non-valid value.

Non-roundtrip example:  Some defined characters in 1208, and all codepoints
that do not have a character defined, do not exist in 1027.
 If you translate text from 1208 to 1027, the characters not defined in
1027, and all undefined codepoints will be translated to (one?) non-valid
codepoint.

In any translation, a codepoint that does not encode a character will be
translated to (one?) non-valid codepoint.

If, while the text is in 1208 is changed to add a character not in 1027,
upon translation that value will be changed to an invalid value in 1027.

 It sounds like you are saying (for the CCSIDs in question) we support 
 round trip, but in one direction only. It would be like if I bought a 
 round trip ticket on Delta between San Francisco and Atlanta, and 
 after I got to Atlanta, they explained that it was a round trip ticket 
 only in one direction.

 I would kind of question also whether what you are doing conforms to 
 your definition of round trip in the Unicode manual glossary: Round 
 trip. Encoding that occurs when every code point in the source CCSID 
 maps to a unique code point in the target CCSID.

You missed the word defined.  If a codepoint does not have a defined
character, it is translated to (one?) invalid value.

 Using round trip
 tables ensure the capability of reversing the conversion, and 
 recovering the complete original source datastream.

 I would question every code point in the source CCSID maps to a 
 unique code point in the target CCSID when both 3F and 41 map to the 
 same code point, and I wonder how I would recover the original source 
 datastream.

Again, you missed the word defined.  If a codepoint does not have a defined
character, it is translated to (one?) invalid value.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Tony Harminc
On 13 June 2012 15:02, Mike Schwab mike.a.sch...@gmail.com wrote:

 Roundtrip example:  Every defined character in 1027, excluding values
 that do not have a character defined, exist in 1208, is successfully
 translated from 1027 to 1208 and back to 1027.  All codepoints that do
 not have a character defined will be translated to (one?) non-valid
 value.

 Non-roundtrip example:  Some defined characters in 1208, and all
 codepoints that do not have a character defined, do not exist in 1027.
  If you translate text from 1208 to 1027, the characters not defined
 in 1027, and all undefined codepoints will be translated to (one?)
 non-valid codepoint.

 In any translation, a codepoint that does not encode a character will
 be translated to (one?) non-valid codepoint.

Not in *any* translation. IBM does define this kind of translation,
but it isn't round trippable - it's enforced subset match.

From the CDRA book: The enforced subset match criterion guarantees
the preservation of the subset of characters that are common to both
the input and output character sets. Any character not in this common
subset will be replaced with a unique character that indicates that a
substitution has occurred. They go on to recommend use of X'1A' in
ASCII-ish environments, and X'3F' in EBCDIC.

 You missed the word defined.  If a codepoint does not have a defined
 character, it is translated to (one?) invalid value.

I can't make it parse that way.

CDRA defines only the graphic character data conversion part of the
overall data conversion process. A limited number of control
characters are addressed as part of handling different string types
(see Types of Strings) and as part of control character mappings
(see Pairings of Code Points). Other control characters are treated
as bytes, and are dealt with according to mismatch management
criteria.

and under Criteria for Character Set Mismatch Management, describing
Pairing of Code Points Using Round Trip:

 - An input graphic code point outside the common set is mapped to an
output graphic code point outside the common set
 - An input control code point is mapped to an output control code
point outside the mnemonic-based common set
 - If the graphic encoding space of the source is larger than the
graphic encoding space of the target, some graphic code points will be
mapped to control code points, and vice versa.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Paul Gilmartin
On Wed, 13 Jun 2012 13:50:17 -0500, McKown, John wrote:

I __think__ I understand what IBM is saying. You must differentiate between a 
hex value (0x00..0xFF) and a code point (a subset of hex values). Not 
every hex value is a code point in every CCSID. I.e. a single byte CCSID 
may have less than 256 code points, or a double byte CCSID may have less 
than 65336 code points...
 
In that case, a conscientious utility _ought_ to indicate an error on 
encountering
an octet which is not a code point rather than mapping it to some arbitrary
value.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Paul Gilmartin
On Wed, 13 Jun 2012 08:31:12 -0400, Shmuel Metz (Seymour J.) wrote:

In
... a Unicode code page ...
 
Oxymoron?  Unicode is unicode.  There's only one code point
for any given character.  Unicode was designed to avoid the
babel of code pages.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Charles Mills
FWIW, z/OS Unicode Services does indicate that at least one SUB character
was output. It's not an error (RC still = 0) but it is a documented output
status bit flag.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Paul Gilmartin
Sent: Wednesday, June 13, 2012 5:20 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

On Wed, 13 Jun 2012 13:50:17 -0500, McKown, John wrote:

I __think__ I understand what IBM is saying. You must differentiate between
a hex value (0x00..0xFF) and a code point (a subset of hex values).
Not every hex value is a code point in every CCSID. I.e. a single byte
CCSID may have less than 256 code points, or a double byte CCSID may have
less than 65336 code points...
 
In that case, a conscientious utility _ought_ to indicate an error on
encountering an octet which is not a code point rather than mapping it to
some arbitrary value.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-13 Thread Paul Gilmartin
On Wed, 13 Jun 2012 17:30:41 -0700, Charles Mills wrote:

FWIW, z/OS Unicode Services does indicate that at least one SUB character
was output. It's not an error (RC still = 0) but it is a documented output
status bit flag.
 
What avail is this if SUB is a valid character in the input code page?

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN



Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Farley, Peter x23353
This is a false assumption: ... every code point in the from CCSID translates 
to a unique (possibly meaningless) code point in the to CCSID 

There is no guarantee that all code points in a given CCSID map to a unique 
code point in any other CCSID.

Peter

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of 
Charles Mills
Sent: Tuesday, June 12, 2012 11:32 AM
To: IBM-MAIN@bama.ua.edu
Subject: Anyone a Unicode Services expert? -- roundtrip conversion

My understanding of roundtrip conversion is that every code point in the
from CCSID translates to a unique (possibly meaningless) code point in the
to CCSID so that if for example a customer is so foolish as to transmit, for
example, an object deck from z/OS to a PC in text format, and then
transfer it back to z/OS the same way, he will have a good object deck. 

I am running Kirk Wolf's excellent showtrtab with the command showtrtab -s
1027 -t 1208 -q R

The results I see include the following:

3F:  1A 
40:  20 
41:  1A

Several other EBCDIC code points also translate to ASCII sub, 0x1a. So the
question is if multiple EBCDIC code points all translate to 0x1a, how the
heck is this a round trip translation?

(I'm not alleging a bug in showtrtab. I am getting the same results with my
code and ran showtrtab as a reality check and a clearer example.)

Where am I going wrong?

Charles 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
Well, Peter, that's certainly consistent with what I see.

I'm looking, however, at slide 11 of
http://publib.boulder.ibm.com/infocenter/ieduasst/stgv1r0/index.jsp?topic=/c
om.ibm.iea.zos/zos/1.9/IntegratingNewAppOnzOS/zOSV1R9_Integrating_newAppl_LE
UnicodeServices/player.html . (You may have to unfold that URL.)

It says R - Roundtrip conversion. Roundtrip conversions between two CCSIDs
assure that all characters making the roundtrip arrive as they were
originally. How is that going to be accomplished if both 3F and 41
translate to 1A? How will they make the round trip back to what they were?

What does technique R mean?

I certainly understand that not every character in a given CCSID maps
meaningfully to another CCSID. That's why I said possibly meaningless in
my original question.

Charles
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 8:41 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

This is a false assumption: ... every code point in the from CCSID
translates to a unique (possibly meaningless) code point in the to CCSID


There is no guarantee that all code points in a given CCSID map to a
unique code point in any other CCSID.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Farley, Peter x23353
I believe that slide refers to round trips within the z/OS world only.  There 
is no statement that CCSID conversion by a system other than z/OS (such as the 
PC ftp client in your example) will be covered in the 'round trip guarantee.

You have to be transmitting and receiving with the same or a very compatible 
iconv() instance to be able to make such a guarantee.  It's remotely possible 
that transmission to a linux ftp client with an iconv() that is compatible with 
the z/OS iconv() *might* support such a guarantee (e.g., z/Linux), but I 
wouldn't bet on it.  And Win systems are almost guaranteed NOT to support such 
a guarantee.

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of 
Charles Mills
Sent: Tuesday, June 12, 2012 11:49 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Well, Peter, that's certainly consistent with what I see.

I'm looking, however, at slide 11 of
http://publib.boulder.ibm.com/infocenter/ieduasst/stgv1r0/index.jsp?topic=/c
om.ibm.iea.zos/zos/1.9/IntegratingNewAppOnzOS/zOSV1R9_Integrating_newAppl_LE
UnicodeServices/player.html . (You may have to unfold that URL.)

It says R - Roundtrip conversion. Roundtrip conversions between two CCSIDs
assure that all characters making the roundtrip arrive as they were
originally. How is that going to be accomplished if both 3F and 41
translate to 1A? How will they make the round trip back to what they were?

What does technique R mean?

I certainly understand that not every character in a given CCSID maps
meaningfully to another CCSID. That's why I said possibly meaningless in
my original question.

Charles
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 8:41 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

This is a false assumption: ... every code point in the from CCSID
translates to a unique (possibly meaningless) code point in the to CCSID


There is no guarantee that all code points in a given CCSID map to a
unique code point in any other CCSID.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Paul Gilmartin
On Tue, 12 Jun 2012 12:03:55 -0400, Farley, Peter x23353 wrote:

I believe that slide refers to round trips within the z/OS world only.  
There is no statement that CCSID conversion by a system other than z/OS (such 
as the PC ftp client in your example) will be covered in the 'round trip 
guarantee.

You have to be transmitting and receiving with the same or a very compatible 
iconv() instance to be able to make such a guarantee.  It's remotely possible 
that transmission to a linux ftp client with an iconv() that is compatible 
with the z/OS iconv() *might* support such a guarantee (e.g., z/Linux), but I 
wouldn't bet on it.  And Win systems are almost guaranteed NOT to support such 
a guarantee.
 
(More at:

http://publib.boulder.ibm.com/infocenter/zos/v1r11/topic/com.ibm.zos.r11.halz001/mbdata.htm
)
Which apparently implies that FTP doesn't support conversion between single-byte
code pages and multi-byte code pages.  Why not!?

Otherwise, I'd be inclined to try from a desktop client:

BINARY
QUOTE SITE ASCII
QUOTE SITE MBDATACONN(IBM-1047,UTF-8)

or whatever.  Let z/OS perform all the conversion.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
Thanks, Peter. 

Please understand I am not criticizing or faulting z/OS Unicode Services. I
am just trying to explain to a customer the output they may expect to see
when they use our product that in turn uses Unicode Services (USS? -- LOL --
never mind).

Let's leave PCs out of it. If I were going z/OS to z/OS to z/OS, how could
both 3F and 41 make the round trip successfully?

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 9:04 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

I believe that slide refers to round trips within the z/OS world only.
There is no statement that CCSID conversion by a system other than z/OS
(such as the PC ftp client in your example) will be covered in the 'round
trip guarantee.

You have to be transmitting and receiving with the same or a very compatible
iconv() instance to be able to make such a guarantee.  It's remotely
possible that transmission to a linux ftp client with an iconv() that is
compatible with the z/OS iconv() *might* support such a guarantee (e.g.,
z/Linux), but I wouldn't bet on it.  And Win systems are almost guaranteed
NOT to support such a guarantee.

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Charles Mills
Sent: Tuesday, June 12, 2012 11:49 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Well, Peter, that's certainly consistent with what I see.

I'm looking, however, at slide 11 of
http://publib.boulder.ibm.com/infocenter/ieduasst/stgv1r0/index.jsp?topic=/c
om.ibm.iea.zos/zos/1.9/IntegratingNewAppOnzOS/zOSV1R9_Integrating_newAppl_LE
UnicodeServices/player.html . (You may have to unfold that URL.)

It says R - Roundtrip conversion. Roundtrip conversions between two CCSIDs
assure that all characters making the roundtrip arrive as they were
originally. How is that going to be accomplished if both 3F and 41
translate to 1A? How will they make the round trip back to what they were?

What does technique R mean?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Farley, Peter x23353
Well, I don't have my green card with me at the moment, but IIRC 3F is the 
EBCDIC SUB character, which means mapping it to ASCII 1A is correct.  EBCDIC 41 
I think is hard space or something like that, and classic 7-bit ASCII does 
not support such a character, so there is no way for EBCDIC 41 to round trip 
through classic 7-bit ASCII (which is what I believe CCSID 1208 is, no?).  Now 
if you are transferring through ISO8879 (Latin-1) or UTF-8 instead, there may 
be a code point in one of those CCSID's that maps to hard space.  I don't 
have time for the necessary code point research at the moment, but you may want 
to look into those CCSID's as an alternative to CCSID 1208.

HTH

Peter

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of 
Charles Mills
Sent: Tuesday, June 12, 2012 1:10 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Thanks, Peter. 

Please understand I am not criticizing or faulting z/OS Unicode Services. I
am just trying to explain to a customer the output they may expect to see
when they use our product that in turn uses Unicode Services (USS? -- LOL --
never mind).

Let's leave PCs out of it. If I were going z/OS to z/OS to z/OS, how could
both 3F and 41 make the round trip successfully?

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 9:04 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

I believe that slide refers to round trips within the z/OS world only.
There is no statement that CCSID conversion by a system other than z/OS
(such as the PC ftp client in your example) will be covered in the 'round
trip guarantee.

You have to be transmitting and receiving with the same or a very compatible
iconv() instance to be able to make such a guarantee.  It's remotely
possible that transmission to a linux ftp client with an iconv() that is
compatible with the z/OS iconv() *might* support such a guarantee (e.g.,
z/Linux), but I wouldn't bet on it.  And Win systems are almost guaranteed
NOT to support such a guarantee.

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Charles Mills
Sent: Tuesday, June 12, 2012 11:49 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Well, Peter, that's certainly consistent with what I see.

I'm looking, however, at slide 11 of
http://publib.boulder.ibm.com/infocenter/ieduasst/stgv1r0/index.jsp?topic=/c
om.ibm.iea.zos/zos/1.9/IntegratingNewAppOnzOS/zOSV1R9_Integrating_newAppl_LE
UnicodeServices/player.html . (You may have to unfold that URL.)

It says R - Roundtrip conversion. Roundtrip conversions between two CCSIDs
assure that all characters making the roundtrip arrive as they were
originally. How is that going to be accomplished if both 3F and 41
translate to 1A? How will they make the round trip back to what they were?

What does technique R mean?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
Peter, thanks. Believe me, I have done a *lot* of CCSID research. 

CCSID 1208 is Encoding scheme 7807 - UTF-8, UCS-2 transform; Name UTF-8 WITH
IBM PUA. It is *the* basic UTF-8 CCSID.

 there is no way for EBCDIC 41 to round trip through classic 7-bit ASCII

Fair enough. Unicode services reports however that it supports roundtrip
conversion in many of these cases, including for example, 37 to 850 (pretty
basic ASCII). What does roundtrip conversion mean? That is my fundamental
question.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 10:48 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Well, I don't have my green card with me at the moment, but IIRC 3F is the
EBCDIC SUB character, which means mapping it to ASCII 1A is correct.  EBCDIC
41 I think is hard space or something like that, and classic 7-bit ASCII
does not support such a character, so there is no way for EBCDIC 41 to
round trip through classic 7-bit ASCII (which is what I believe CCSID 1208
is, no?).  Now if you are transferring through ISO8879 (Latin-1) or UTF-8
instead, there may be a code point in one of those CCSID's that maps to
hard space.  I don't have time for the necessary code point research at
the moment, but you may want to look into those CCSID's as an alternative to
CCSID 1208.

HTH

Peter

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Charles Mills
Sent: Tuesday, June 12, 2012 1:10 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Thanks, Peter. 

Please understand I am not criticizing or faulting z/OS Unicode Services. I
am just trying to explain to a customer the output they may expect to see
when they use our product that in turn uses Unicode Services (USS? -- LOL --
never mind).

Let's leave PCs out of it. If I were going z/OS to z/OS to z/OS, how could
both 3F and 41 make the round trip successfully?

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 9:04 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

I believe that slide refers to round trips within the z/OS world only.
There is no statement that CCSID conversion by a system other than z/OS
(such as the PC ftp client in your example) will be covered in the 'round
trip guarantee.

You have to be transmitting and receiving with the same or a very compatible
iconv() instance to be able to make such a guarantee.  It's remotely
possible that transmission to a linux ftp client with an iconv() that is
compatible with the z/OS iconv() *might* support such a guarantee (e.g.,
z/Linux), but I wouldn't bet on it.  And Win systems are almost guaranteed
NOT to support such a guarantee.

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Charles Mills
Sent: Tuesday, June 12, 2012 11:49 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Well, Peter, that's certainly consistent with what I see.

I'm looking, however, at slide 11 of
http://publib.boulder.ibm.com/infocenter/ieduasst/stgv1r0/index.jsp?topic=/c
om.ibm.iea.zos/zos/1.9/IntegratingNewAppOnzOS/zOSV1R9_Integrating_newAppl_LE
UnicodeServices/player.html . (You may have to unfold that URL.)

It says R - Roundtrip conversion. Roundtrip conversions between two CCSIDs
assure that all characters making the roundtrip arrive as they were
originally. How is that going to be accomplished if both 3F and 41
translate to 1A? How will they make the round trip back to what they were?

What does technique R mean?

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email
to lists...@bama.ua.edu with the message: INFO IBM-MAIN

This message and any attachments are intended only for the use of the
addressee and may contain information that is privileged and confidential.
If the reader of the message is not the intended recipient or an authorized
representative of the intended recipient, you are hereby notified that any
dissemination of this communication is strictly prohibited. If you have
received this communication in error, please notify us immediately by e-mail
and delete the message and any attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email
to lists...@bama.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Steve Comstock

On 6/12/2012 11:59 AM, Charles Mills wrote:

Peter, thanks. Believe me, I have done a *lot* of CCSID research.

CCSID 1208 is Encoding scheme 7807 - UTF-8, UCS-2 transform; Name UTF-8 WITH
IBM PUA. It is *the* basic UTF-8 CCSID.


there is no way for EBCDIC 41 to round trip through classic 7-bit ASCII


Fair enough. Unicode services reports however that it supports roundtrip
conversion in many of these cases, including for example, 37 to 850 (pretty
basic ASCII). What does roundtrip conversion mean? That is my fundamental
question.


Roundtrip conversion means what you think: translate string A to string B then
back from string B to string A' then A = A'

Clearly you are not experiencing that.

Just as clearly, there are many mappings for which roundtrip
conversion is not possible. I've lost the original post, what
is the mapping you are trying to achieve?




Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 10:48 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Well, I don't have my green card with me at the moment, but IIRC 3F is the
EBCDIC SUB character, which means mapping it to ASCII 1A is correct.  EBCDIC
41 I think is hard space or something like that, and classic 7-bit ASCII
does not support such a character, so there is no way for EBCDIC 41 to
round trip through classic 7-bit ASCII (which is what I believe CCSID 1208
is, no?).  Now if you are transferring through ISO8879 (Latin-1) or UTF-8
instead, there may be a code point in one of those CCSID's that maps to
hard space.  I don't have time for the necessary code point research at
the moment, but you may want to look into those CCSID's as an alternative to
CCSID 1208.

HTH

Peter

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Charles Mills
Sent: Tuesday, June 12, 2012 1:10 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Thanks, Peter.

Please understand I am not criticizing or faulting z/OS Unicode Services. I
am just trying to explain to a customer the output they may expect to see
when they use our product that in turn uses Unicode Services (USS? -- LOL --
never mind).

Let's leave PCs out of it. If I were going z/OS to z/OS to z/OS, how could
both 3F and 41 make the round trip successfully?

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 9:04 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

I believe that slide refers to round trips within the z/OS world only.
There is no statement that CCSID conversion by a system other than z/OS
(such as the PC ftp client in your example) will be covered in the 'round
trip guarantee.

You have to be transmitting and receiving with the same or a very compatible
iconv() instance to be able to make such a guarantee.  It's remotely
possible that transmission to a linux ftp client with an iconv() that is
compatible with the z/OS iconv() *might* support such a guarantee (e.g.,
z/Linux), but I wouldn't bet on it.  And Win systems are almost guaranteed
NOT to support such a guarantee.

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Charles Mills
Sent: Tuesday, June 12, 2012 11:49 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Well, Peter, that's certainly consistent with what I see.

I'm looking, however, at slide 11 of
http://publib.boulder.ibm.com/infocenter/ieduasst/stgv1r0/index.jsp?topic=/c
om.ibm.iea.zos/zos/1.9/IntegratingNewAppOnzOS/zOSV1R9_Integrating_newAppl_LE
UnicodeServices/player.html . (You may have to unfold that URL.)

It says R - Roundtrip conversion. Roundtrip conversions between two CCSIDs
assure that all characters making the roundtrip arrive as they were
originally. How is that going to be accomplished if both 3F and 41
translate to 1A? How will they make the round trip back to what they were?

What does technique R mean?

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email
to lists...@bama.ua.edu with the message: INFO IBM-MAIN

This message and any attachments are intended only for the use of the
addressee and may contain information that is privileged and confidential.
If the reader of the message is not the intended recipient or an authorized
representative of the intended recipient, you are hereby notified that any
dissemination of this communication is strictly prohibited. If you have
received this communication in error, please notify us immediately by e-mail
and delete the message and any attachments from your system

Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Farley, Peter x23353
I will have to leave the answer to your fundamental question for wiser and more 
experienced heads to answer.  I would think you could file a PMR on this and 
see what IBM has to say about it.  You might only get a doc change out of it, 
but at least then there would be doc on what is and is not supported for a 
guaranteed round trip.

Peter

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of 
Charles Mills
Sent: Tuesday, June 12, 2012 1:59 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Peter, thanks. Believe me, I have done a *lot* of CCSID research. 

CCSID 1208 is Encoding scheme 7807 - UTF-8, UCS-2 transform; Name UTF-8 WITH
IBM PUA. It is *the* basic UTF-8 CCSID.

 there is no way for EBCDIC 41 to round trip through classic 7-bit ASCII

Fair enough. Unicode services reports however that it supports roundtrip
conversion in many of these cases, including for example, 37 to 850 (pretty
basic ASCII). What does roundtrip conversion mean? That is my fundamental
question.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 10:48 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Well, I don't have my green card with me at the moment, but IIRC 3F is the
EBCDIC SUB character, which means mapping it to ASCII 1A is correct.  EBCDIC
41 I think is hard space or something like that, and classic 7-bit ASCII
does not support such a character, so there is no way for EBCDIC 41 to
round trip through classic 7-bit ASCII (which is what I believe CCSID 1208
is, no?).  Now if you are transferring through ISO8879 (Latin-1) or UTF-8
instead, there may be a code point in one of those CCSID's that maps to
hard space.  I don't have time for the necessary code point research at
the moment, but you may want to look into those CCSID's as an alternative to
CCSID 1208.

HTH

Peter

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Charles Mills
Sent: Tuesday, June 12, 2012 1:10 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Thanks, Peter. 

Please understand I am not criticizing or faulting z/OS Unicode Services. I
am just trying to explain to a customer the output they may expect to see
when they use our product that in turn uses Unicode Services (USS? -- LOL --
never mind).

Let's leave PCs out of it. If I were going z/OS to z/OS to z/OS, how could
both 3F and 41 make the round trip successfully?

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 9:04 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

I believe that slide refers to round trips within the z/OS world only.
There is no statement that CCSID conversion by a system other than z/OS
(such as the PC ftp client in your example) will be covered in the 'round
trip guarantee.

You have to be transmitting and receiving with the same or a very compatible
iconv() instance to be able to make such a guarantee.  It's remotely
possible that transmission to a linux ftp client with an iconv() that is
compatible with the z/OS iconv() *might* support such a guarantee (e.g.,
z/Linux), but I wouldn't bet on it.  And Win systems are almost guaranteed
NOT to support such a guarantee.

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Charles Mills
Sent: Tuesday, June 12, 2012 11:49 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Well, Peter, that's certainly consistent with what I see.

I'm looking, however, at slide 11 of
http://publib.boulder.ibm.com/infocenter/ieduasst/stgv1r0/index.jsp?topic=/c
om.ibm.iea.zos/zos/1.9/IntegratingNewAppOnzOS/zOSV1R9_Integrating_newAppl_LE
UnicodeServices/player.html . (You may have to unfold that URL.)

It says R - Roundtrip conversion. Roundtrip conversions between two CCSIDs
assure that all characters making the roundtrip arrive as they were
originally. How is that going to be accomplished if both 3F and 41
translate to 1A? How will they make the round trip back to what they were?

What does technique R mean?
--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system

Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
 I've lost the original post, what is the mapping you are trying to
achieve?

The original example was somewhat extreme.

CCSID 01027: Encoding scheme 1100 - EBCDIC, SBCS; Name JAPAN LATIN

CCSID 01208: Encoding scheme 7807 - UTF-8, UCS-2 transform; Name UTF-8 WITH
IBM 

Both the Unicode manual for z/OS V1R13 and the returned information from
CUNLINFO indicate that roundtrip conversion is supported for this CCSID
pair.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Steve Comstock
Sent: Tuesday, June 12, 2012 11:06 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

On 6/12/2012 11:59 AM, Charles Mills wrote:
 Peter, thanks. Believe me, I have done a *lot* of CCSID research.

 CCSID 1208 is Encoding scheme 7807 - UTF-8, UCS-2 transform; Name 
 UTF-8 WITH IBM PUA. It is *the* basic UTF-8 CCSID.

 there is no way for EBCDIC 41 to round trip through classic 7-bit 
 ASCII

 Fair enough. Unicode services reports however that it supports 
 roundtrip conversion in many of these cases, including for example, 37 
 to 850 (pretty basic ASCII). What does roundtrip conversion mean? That 
 is my fundamental question.

Roundtrip conversion means what you think: translate string A to string B
then back from string B to string A' then A = A'

Clearly you are not experiencing that.

Just as clearly, there are many mappings for which roundtrip conversion is
not possible. I've lost the original post, what is the mapping you are
trying to achieve?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
Right. I don't have a functional problem. There is no fix that I am
looking for. I have an understanding and documentation (mine!) problem.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Farley, Peter x23353
Sent: Tuesday, June 12, 2012 11:21 AM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

I will have to leave the answer to your fundamental question for wiser and
more experienced heads to answer.  I would think you could file a PMR on
this and see what IBM has to say about it.  You might only get a doc change
out of it, but at least then there would be doc on what is and is not
supported for a guaranteed round trip.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Tony Harminc
On 12 June 2012 13:59, Charles Mills charl...@mcn.org wrote:

 Fair enough. Unicode services reports however that it supports roundtrip
 conversion in many of these cases, including for example, 37 to 850 (pretty
 basic ASCII). What does roundtrip conversion mean? That is my fundamental
 question.

The CDRA Reference and Registry book (SC09-2190-00 - admittedly rather
old...) says this about Round Trip Integrity (in a larger section on
Criteria for Character Set Mismatch Management):

The objective of this criterion is to send data from one system to
another one that has different representations of character data, and
retrieve it without loss. Often the do not convert choice is not
available. For example, data stored in a System/370 database is
configured to have all its graphic character data in one CCSID. If it
acts as a remote repository for data from a PC application, or from an
application in another System/370 using a different CCSID, the data
must be converted to the configured CCSID. The data is intended to be
retrieved by the same application without loss when it is converted
back for use in its original CCSID.

There's more, but I think the notion of Round Trip Integrity is clear enough.

And having typed that in with the recollection that this book was not
available online, I see that there is an updated version at

http://www-01.ibm.com/software/globalization/cdra/

The bit I quoted is in chapter 6.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
Thanks. I think either your and my understanding of roundtrip is flawed, or
Unicode Services understands it differently, or I am missing something.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Tony Harminc
Sent: Tuesday, June 12, 2012 12:02 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

On 12 June 2012 13:59, Charles Mills charl...@mcn.org wrote:

 Fair enough. Unicode services reports however that it supports roundtrip
 conversion in many of these cases, including for example, 37 to 850
(pretty
 basic ASCII). What does roundtrip conversion mean? That is my fundamental
 question.

The CDRA Reference and Registry book (SC09-2190-00 - admittedly rather
old...) says this about Round Trip Integrity (in a larger section on
Criteria for Character Set Mismatch Management):

The objective of this criterion is to send data from one system to
another one that has different representations of character data, and
retrieve it without loss. Often the do not convert choice is not
available. For example, data stored in a System/370 database is
configured to have all its graphic character data in one CCSID. If it
acts as a remote repository for data from a PC application, or from an
application in another System/370 using a different CCSID, the data
must be converted to the configured CCSID. The data is intended to be
retrieved by the same application without loss when it is converted
back for use in its original CCSID.

There's more, but I think the notion of Round Trip Integrity is clear
enough.

And having typed that in with the recollection that this book was not
available online, I see that there is an updated version at

http://www-01.ibm.com/software/globalization/cdra/

The bit I quoted is in chapter 6.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
FYI:

 A round-trip conversion works only in a two-tier homogenous environment
where the data makes the complete round trip. For example, if you pass data
from DB2 for Linux, UNIX, and Windows to DB2 for z/OS and then back to DB2
for Linux, UNIX, and Windows with a round-trip conversion, no data is lost.
The data was converted back to its original format. However, if you have a
more complicated environment, a round-trip conversion does not necessarily
preserve data integrity. For example, if you pass data from DB2 for z/OS to
DB2 for Linux, UNIX, and Windows and then to Linux on a Java client, two
conversions have potentially occurred. Because the data was not converted
back to its original format before the second conversion, data might have
been lost even if round-trip conversions are used.

Example: In ASCII CCSID 1252, the trademark symbol T is code point X'99'. In
EBCDIC CCSID 37, this code point does not exist. During character conversion
from ASCII CCSID 1252 to EBCDIC CCSID 37, the trademark symbol is converted
to a control character, X'39'. If you use SPUFI to select the data, the data
displays as the unprintable character that you specified in your emulator,
which is generally a quotation mark () or a period (.). If you issue a
SELECT HEX(column) statement, the data displays as X'39', which is the DEL
control character. When the client uses a round-trip conversion to convert
this character back to ASCII CCSID 1252, the trademark symbol is preserved
as code point X'99'. Notice that this conversion is not lossless unless it
is converted back to the original format.

--
http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=%2Fco
m.ibm.db2z9.doc.char%2Fsrc%2Ftpc%2Fdb2z_roundtripconversion.htm 

Charles

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Paul Gilmartin
On Tue, 12 Jun 2012 15:02:17 -0400, Tony Harminc wrote:

The objective of this criterion is to send data from one system to
another one that has different representations of character data, and
retrieve it without loss. Often the do not convert choice is not
available. For example, data stored in a System/370 database is
configured to have all its graphic character data in one CCSID. If it
acts as a remote repository for data from a PC application, or from an
application in another System/370 using a different CCSID, the data
must be converted to the configured CCSID. The data is intended to be
retrieved by the same application without loss when it is converted
back for use in its original CCSID.

There's more, but I think the notion of Round Trip Integrity is clear enough.

Why does IBM have this compulsion to be different from everyone else
and invent its own terminology rather than using a conventional,
well-understood word such as bijective?

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Roberts, John J
Why does IBM have this compulsion to be different from everyone else
and invent its own terminology rather than using a conventional,
well-understood word such as bijective?

Because bijective is not so well understood by anyone born before 1952 or so. 
 The term relates to SET Theory, which was a big part of the New Math.  My 
younger brother, born in 1953 was exposed to this, but I was not.  Actually, I 
only learned of the term (and the noun bijection) on this forum, this past 
year.

But, of course we have digressed far from Charles original question.  Sorry.

John

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread zMan
On Tue, Jun 12, 2012 at 4:03 PM, Paul Gilmartin paulgboul...@aim.comwrote:

 Why does IBM have this compulsion to be different from everyone else
 and invent its own terminology rather than using a conventional,
 well-understood word such as bijective?


Harumph. It may be an established and well-defined term, and your point is
valid, but conventional and well-understood are arguable. Sure, the
math PhDs use that term, but reg'lar folks don't.

However, it would have behooved IBM to define and use it. Or maybe the
people who wrote that don't know it...
-- 
zMan -- I've got a mainframe and I'm not afraid to use it

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Paul Gilmartin
On Tue, 12 Jun 2012 12:59:13 -0700, Charles Mills wrote:

 A round-trip conversion works only in a two-tier homogenous environment
where the data makes the complete round trip. For example, if you pass data
from DB2 for Linux, UNIX, and Windows to DB2 for z/OS and then back to DB2
for Linux, UNIX, and Windows with a round-trip conversion, no data is lost.
The data was converted back to its original format. However, if you have a
more complicated environment, a round-trip conversion does not necessarily
preserve data integrity. For example, if you pass data from DB2 for z/OS to
DB2 for Linux, UNIX, and Windows and then to Linux on a Java client, two
conversions have potentially occurred. Because the data was not converted
back to its original format before the second conversion, data might have
been lost even if round-trip conversions are used.
 
???  Errr...

The composition of two (or more) bijective function is a bijective
function.  This can't violate data integrity.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Kirk Wolf
I would say that bijection (a one-one correspondence) is not exactly the
same as roundtripping.

For example:

IBM-1047 (single byte EBCDIC)  -  UTF-8

- you can round-trip this, since you can take any character in the source
code page, and get a UTF-8 character.   If you take any of *those* 256
UTF-8 characters, they will map uniquely back to where they started.

But, you cannot have a round-trip from UTF-8 - IBM-1047  (since the source
set is MUCH bigger).

But, I still don't understand the results that Charles is seeing; doesn't
look like round-tripping  :-(

Kirk Wolf
Dovetailed Technologies
http://dovetail.com

On Tue, Jun 12, 2012 at 3:38 PM, Roberts, John J
jrobe...@dhs.state.ia.uswrote:

 Why does IBM have this compulsion to be different from everyone else
 and invent its own terminology rather than using a conventional,
 well-understood word such as bijective?

 Because bijective is not so well understood by anyone born before 1952
 or so.  The term relates to SET Theory, which was a big part of the New
 Math.  My younger brother, born in 1953 was exposed to this, but I was
 not.  Actually, I only learned of the term (and the noun bijection) on
 this forum, this past year.

 But, of course we have digressed far from Charles original question.
  Sorry.

 John

 --
 For IBM-MAIN subscribe / signoff / archive access instructions,
 send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread John Gilmore
The term 'bijective' is a fairly old one, the earliest reference I
found in a Mathematical Reviews index was for 1939..  Anyone who has
had a college course in  mathematical logic or, yes, set theory is
likely to know or have forgotten its meaning.

It does, however, have a bad reputation because of its use to
intimidate non-mathematicians.  The example in Quine's Quiddities
under the topic mathematosis illustrates why.

Still, it is perhaps better than 'roundtrip', which in ordinary use
does not really imply bijective.  My wife and I both found that we
were measurably heavier after a recent Boston-Roma-Boston trip.
Additional efforts were required to restore our status quo ante
weights.


--jg

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
They are saying that if you go from Linux to z/OS to Linux then '%' might
become '?' and then back to '%'. (Those are just example characters -- don't
take them literally.)

But if you go from z/OS to Linux to Java then '?' might become '%' and then
'!'. 

They are saying that round-trip does not mean you can just translate it
twice and be back to the original character. You have to go round trip --
back to where (the CCSID?) you came from.

It's not the clearest example. g

Charles
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Paul Gilmartin
Sent: Tuesday, June 12, 2012 1:45 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

On Tue, 12 Jun 2012 12:59:13 -0700, Charles Mills wrote:

 A round-trip conversion works only in a two-tier homogenous 
environment where the data makes the complete round trip. For example, 
if you pass data from DB2 for Linux, UNIX, and Windows to DB2 for z/OS 
and then back to DB2 for Linux, UNIX, and Windows with a round-trip
conversion, no data is lost.
The data was converted back to its original format. However, if you 
have a more complicated environment, a round-trip conversion does not 
necessarily preserve data integrity. For example, if you pass data from 
DB2 for z/OS to
DB2 for Linux, UNIX, and Windows and then to Linux on a Java client, 
two conversions have potentially occurred. Because the data was not 
converted back to its original format before the second conversion, 
data might have been lost even if round-trip conversions are used.
 
???  Errr...

The composition of two (or more) bijective function is a bijective function.
This can't violate data integrity.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
I have opened a PMR.

Charles
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Charles Mills
Sent: Tuesday, June 12, 2012 12:50 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Thanks. I think either your and my understanding of roundtrip is flawed, or
Unicode Services understands it differently, or I am missing something.

Charles

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Tony Harminc
On 12 June 2012 18:55, Charles Mills charl...@mcn.org wrote:
 I have opened a PMR.

For the doc, or the behaviour of the service? Or did you choose the
let us decide for you option...?

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
Not sure what you mean. Here's the PMR:

Problem Details 
.   
Product or Service: Support for Unicode 
Component ID: 5752SCUNI 
.   
Operating System: z/OS  
.   
Problem title   
Round trip conversion not working as expected   
.   
Problem description 
My understanding of round trip conversion is that every code point in 
the Source CCSID should convert to a unique code point in the Target
CCSID (so that conversion in the reverse direction will restore the 
original code points). However, when I use CUNLCNV to convert between   
CCSIDs 1027 and 1208 I see more than one character converted to x'1A'   
(UTF8 SUB). Is round trip conversion working correctly? (Both the   
documentation and CUNLINFO report that 'R' is a supported conversion
technique for these two CCSIDs.)

Characters converted to X'1A' include 3F, 41, 6A, 80, 90, etc.  

Thank you for your consideration.   
.   
Business impact ( BusImpact )   
Negatively impacting product design decisions.  

Charles
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Tony Harminc
Sent: Tuesday, June 12, 2012 4:15 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

On 12 June 2012 18:55, Charles Mills charl...@mcn.org wrote:
 I have opened a PMR.

For the doc, or the behaviour of the service? Or did you choose the let us
decide for you option...?

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email
to lists...@bama.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN


Re: Anyone a Unicode Services expert? -- roundtrip conversion

2012-06-12 Thread Charles Mills
1027 was a test case and the data is arbitrary. I think the point of round
trip is supposed to be that *every* character is recoverable.

A good write-up in of all places g the Glossary of the Z Unicode manual.
Emphasis is mine.

Round trip. Encoding that occurs when *every* code
point in the source CCSID maps to a unique code point
in the target CCSID. Using round trip tables ensure the
capability of reversing the conversion, and recovering
the complete original source datastream.

I'm going to see what they say on the PMR.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf
Of Bill Godfrey
Sent: Tuesday, June 12, 2012 7:12 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

Maybe it will turn out that round trip only applies to the characters that
are defined in the Source CCSID (and I'm not sure if defined is the right
word). When I look at a chart of CCSID 1027 here:

ftp://ftp.software.ibm.com/software/globalization/gcoc/attachments/CP01027.p
df

I see that all of the characters that you see being being converted to X'1A'
are either outside the chart (which starts at X'40') or have no identified
value in the chart. If your data contains these unidentified characters,
what is their purpose, and is it really appropriate to consider the data to
be associated with CCSID 1027, or just something related to but not the same
as CCSID 1027?

Bill 

On Tue, 12 Jun 2012 16:42:02 -0700, Charles Mills wrote:

Not sure what you mean. Here's the PMR:

Problem Details
.
Product or Service: Support for Unicode Component ID: 5752SCUNI .
Operating System: z/OS
.
Problem title
Round trip conversion not working as expected .
Problem description
My understanding of round trip conversion is that every code point in 
the Source CCSID should convert to a unique code point in the Target 
CCSID (so that conversion in the reverse direction will restore the 
original code points). However, when I use CUNLCNV to convert between 
CCSIDs 1027 and 1208 I see more than one character converted to x'1A'
(UTF8 SUB). Is round trip conversion working correctly? (Both the 
documentation and CUNLINFO report that 'R' is a supported conversion 
technique for these two CCSIDs.)

Characters converted to X'1A' include 3F, 41, 6A, 80, 90, etc.

Thank you for your consideration.
.
Business impact ( BusImpact )
Negatively impacting product design decisions.

Charles
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On 
Behalf Of Tony Harminc
Sent: Tuesday, June 12, 2012 4:15 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: Anyone a Unicode Services expert? -- roundtrip conversion

On 12 June 2012 18:55, Charles Mills charl...@mcn.org wrote:
 I have opened a PMR.

For the doc, or the behaviour of the service? Or did you choose the 
let us decide for you option...?

Tony H.


--
For IBM-MAIN subscribe / signoff / archive access instructions, send email
to lists...@bama.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN