Tom Christiansen wrote:
> Has anybody specifically looked at how Perl6 regexes might map to
> the various requirements of UTS#18, Unicode Regular Expressions?

Roughly Perl6 supports Level 2 (I did a fast check of UTS#18 and the specs).

> I ask because to my inexperienced eye, quite a few perl6isms are
> *much* better at this than in perl5 obtain, and so I wondered
> whether this was by conscious intent and design.  Is/Was it?

Seems intended.

> I'm also curious whether there are active plans to address the
> tr18 requirements in perl6 regexes.  It would be a wonderful
> feather in perl6's cap to be able to legitimately claim Level 2
> or even Level 3 compliance, since besides perl5, only ICU right
> now manages even Level 1, with everybody else *very* far behind.

I would like to have:
- all Unicode features of Perl5 (UCD, charnames, Normalize, properties)
- most of the features of ICU (e.g. transforms, localisation)
- normalization form, local-support and tailored $features on string
  level (_not_ lexical context)

This means that any string can be in or can transformed into the form
- Byte
- NFG (Default Grapheme Clusters)
- NFGT (Tailored Grapheme Cluster)

Tailored means, that the Graphem (NFG) needs a Language-Local (e.g. German-Swiss-Spelling_1996, or German-Austrian-PhoneBookCollation). Without a Language-Local a NFG-string is handled as NFG-string (default).

Tailored also means, that the user (Perl6 programmer) can tailor the relevant mechanisms (formatting, normalization, collation, properties, case folding etc.).

> TR18 specifies three levels of support (Basic, Extended, and Tailored),
> with each having specific, reasonably well-defined requirements:

There is a lot of work in the UNICODE standard - using it costs nothing, but saves time. E.g. the allowed characters for identifiers can be defined with the Unicode properties 'ID_Start' and 'ID_Continue', Grapheme with Grapheme_Base, Grapheme_Extend etc.

>   =Level 1: Basic Unicode Support
>    RL1.3    Subtraction and Intersection

IMHO not complete

>    RL1.5    Simple Loose Matches

Hmm ...

>    RL1.6    Line Boundaries

can be defined

>    RL1.7    Supplementary Code Points

IMHO not specced

>   =Level 2: Extended Unicode Support
>    RL2.1    Canonical Equivalents

IMHO not specced

> RL2.4 Default Loose Matches RL2.5 Name Properties RL2.6 Wildcard Properties

IMHO not specced

>   =Level 3: Tailored Unicode Support

IMHO not specced

It would be easier to reference the appropriate chapters of the Unicode standard in the specification of Perl6. This would make Unicode test-cases reusable. And an implementation should always declare, which features of Unicode are implemented (and which not) in which version of Unicode.

Helmut Wollmersdorfer

Reply via email to