I (Marco Cimarosti) wrote: > Mark Davis wrote: > > Feedback that is received before the UTC meeting (starting > > August 20) can be made available for the discussion of > > TR29 at that meeting. > > The handling of apostrophe is not satisfactory for French and > Italian, as the document itself acknowledges. > > The solution proposed in UTR#29 is tailoring, but French and > Italian are so widespread languages, and so frequently quoted > in other languages, that I think it could make sense to try > and fit them in the *default* definition of the algorithm. > > I know that having this accepted won't be so easy, but I am > so tired of system which cannot properly move cursor or count > words in my own language, that I thought it was worth trying. > > The attached HTML file describes the proposed changes.
Ehmm..., I already found some major errors in my proposal. :-( Please ignore my previous HTML file and get the one attached to this e-mail. Sorry about that: I am in a hurry finishing up things before a long national holiday, so I haven't had much time to prepare this feedback and submit it before the August 20 deadline. _ Marco
Proposal to accomodate French/Italian apostrophes in Unicode's UTR#29author: Marco Cimarosti Rationale - The existing word-boundary rules of UTR#29 are designed to capture the meaning of apostrophes in English (and many other languages). I.e., apostrophes normally are inside a word, as in "don't" or "Marco's". The behavior of apostrophes is quite different in Italian and French (and other languages, e.g. Esperanto), where an apostrophe normally marks the deletion of the last vowel of a word which occur before a word starting with a vowel, e.g. "d'Unicode" (d' from de = "of"), or "l'Angleterre" (l' from la = "the"). The two words are graphically joined (no space before or after the apostrophe). The apostrophe is part of the first word, and an implicit word break comes after it. Implementing this behavior in the default definition of UTR#29 is important to accomodate the needs of the large French and Italian speaking communities, as well as the needs of the people writing in other languages, who often use loanwords or quotations from these popular languages. Proposed euristic - The present proposal is based on the observation that French-style "splitting" apostrophes are always followed by a vowel, whereas English-style "joining" apostrophes are normally followed by a consonant. The issue is complicated by the fact that both French and Italian have mute H's which can interfere in the algorithm. The proposal defines three new character classes: LatinVowels (containing all vowels meaningful in French, Italian, and Esperanto), LatinH (containing only the letter H in the two cases), and Apostrophe (containing the two characters used for apostrophe). The characters contained in the new classes are removed from the classes where the they used to belong (ALetter and MidLetter). The new classes are used to define two new rules (before current rule 6) for French-style apostrophes, which cover the cases"<consonant>'<vowel>" and "<consonant>'h<vowel>". Rules from 5 downwards are slightly changed because the former classes ALetter and MidLetter are now split in two or more classes. Open issues - Although this proposal might enhance the handling of some common cases in two common languages, there still are many remaining edge cases which can only be solved by tailoring the algorithm. For instance, the "c'h" trigraph of the Breton language would unduely be splitted by the default definition, when followed by a vowel. Note - The proposed changes are concentrated in Table 2 (Default Word Boundaries). Proposed additions are colored in green, proposed deletions are colored in red, and existing text remains in black. ... Table 2. Default Word Boundaries
... | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
