Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan Hull via swift-evolution]

Just to play devil’s advocate (since I am honestly ok with the less draconian 
form of the proposal now), wouldn’t it be possible to declare the relevant 
unicode characters in the module’s plist (or similar file)?  I know it isn’t 
the most elegant solution, but it does move the bar on using those characters 
from impossible to possible, and it would no longer mix up the compilation 
layers.

Thanks,
Jon

> On Oct 24, 2016, at 10:10 PM, Chris Lattner  wrote:
> 
> 
>> On Oct 20, 2016, at 5:03 PM, Jonathan S. Shapiro via swift-evolution 
>> > wrote:
>> 
>> On Thu, Oct 20, 2016 at 12:25 PM, Jonathan Hull via swift-evolution 
>> > wrote:
>> Would it be possible to do the following:
>> 
>> • Have 1 group which are always used as identifiers.  This would probably be 
>> the identifiers from this proposal.
>> 
>> • Have a 2nd group which are always used as operators (quite a bit larger 
>> than the group proposed by this proposal).  At a minimum, the following 
>> ascii + set operators + math operators:
>> (± ≠ ≤ ≥ ¿ ¡ ™ ¢ ¶ • ° ƒ © √ ∆ ◊ § ≈  ∫ ÷ ¬ ).
>> 
>> • Everything else is in a 3rd group. 
>> 
>> No. This is far, far too complicated, and it mixes up the layers of the 
>> compilation process.
> 
> Right.  Any proposal that changes parser behavior based on a visible operator 
> declaration will break the ability for Swift to separately compile files.  
> This will have massive tooling ramifications that are almost certainly a 
> non-starter.
> 
> -Chris
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan Hull via swift-evolution]

True, but that doesn’t mean we have to resign ourselves the lowest common 
denominator.  We need a way to operate with older systems, but we also want to 
embrace what has been made possible now.  Within a few years, it is quite 
likely that this technology will be common both across Apple’s lineup and that 
of competitors.

For example, it is much more realistic to have something like the union 
operator in the standard library now, because those with the TouchBar can 
(theoretically) easily type it, and those of us without a TouchBar can continue 
to use the much less concise formUnion.

I also have several ideas on how to make typing these things easier 
sans-touchbar. There are lots of potential UI which make it not-too-difficult, 
we have just have to have a reason to do them.

Let’s move forward and develop something for the modern age instead of limiting 
ourselves to the state of the art from the 1970s (e.g. I personally find it 
barbaric that we have to type an arrow ‘<=' instead of ‘≤’).  Sure, we have to 
make sure there is a bridge for those who aren’t on the bleeding edge, but that 
doesn’t mean we shouldn’t push forward at all.

Thanks,
Jon


> On Oct 28, 2016, at 3:13 PM, Erica Sadun  wrote:
> 
> 
>> On Oct 27, 2016, at 11:46 AM, Jonathan Hull via swift-evolution 
>>  wrote:
>> 
>> It looks like the issue of typing unicode symbols was solved much sooner 
>> than any of us were predicting…
>> 
>> Thanks,
>> Jon
> 
> Swift as a language extends beyond Apple's platform and Unicode symbology.
> 
> -- E
> 
> 
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Russ Bishop via swift-evolution]

> On Oct 25, 2016, at 3:19 AM, Xiaodi Wu  wrote:
> 
> Unfortunately, Joe is correct on this point. As I stated earlier in the 
> thread, there are a series of characters that can be either text or emoji in 
> presentation, where the default presentation differs depending on platform, 
> technology, use case, or context. This is also not a bug, but explicitly 
> contemplated by Unicode technical recommendations. You can convince yourself 
> of this fact by looking up the Wikipedia page on the Unicode "dingbats" block 
> and comparing the rendering on Safari on iOS and Safari on macOS. You will 
> see that they are different.
> 
> Unfortunately, you are incorrect about the behavior of missing glyphs. 
> Unlike, say, Chinese displayed on a machine without the necessary fonts, 
> there is a security concern that Unicode 9 emoji not yet supported by Apple 
> are non-displaying on that platform. No placeholder appears. This includes 
> what is according to Emojipedia the #1 most popular emoji, the shrug. (Check 
> out Emojipedia on a Mac.) It appears that there is no required placeholder 
> glyph for unsupported emoji, so any of them can legitimately disappear on a 
> non-supported platform. This is an issue worth serious consideration.

IMHO I don’t think Swift needs to be designed around rendering bugs with 
specific fonts on specific platforms. We can file a radar to have this 
corrected. I’m not aware of anything in Unicode that says it is acceptable to 
just drop unknown characters. I think some ZJW sequences or modifiers can be 
ignored; anything that can be ignored for rendering should be ignored for 
uniqueness of identifiers too.


Russ

> 
> On Tue, Oct 25, 2016 at 00:41 Russ Bishop via swift-evolution 
> > wrote:
>> On Oct 24, 2016, at 9:43 AM, Joe Groff via swift-evolution 
>> > wrote:
>> 
>> 
>>> On Oct 23, 2016, at 9:41 PM, Chris Lattner via swift-evolution 
>>> > wrote:
>>> 
>>> 
 On Oct 18, 2016, at 11:34 PM, Jacob Bandes-Storch via swift-evolution 
 > wrote:
 
 Dear Swift-Evolution community,
 
 A few of us have been preparing a proposal to refine the definitions of 
 identifiers & operators. This includes some changes to the permitted 
 Unicode characters.
 
 The latest (perhaps final?) draft is available here:
 
 
 https://github.com/jtbandes/swift-evolution/blob/unicode-id-op/proposals/-refining-identifier-and-operator-symbology.md
  
 
 
 We'd welcome your initial thoughts, and will probably submit a PR soon to 
 the swift-evolution repo for a formal review. Full text follows below.
>>> 
>>> I haven’t had a chance to read the entire proposal, nor the tons of great 
>>> discussion down thread, but here are a few thoughts, just MHO:
>>> 
>>> - I’m loving that you’re taking a detail oriented approach to the problem.  
>>> I agree with you that our current approach is unprincipled, and we need to 
>>> get this right for Swift 4.
>>> - I think that it is perfectly fine to err on the side of conservatism: if 
>>> it isn’t clear how to classify something (e.g. Braille patterns), we should 
>>> just reject them in both operators and identifiers (make them be 
>>> unassigned).  If these unclear cases are important to someone, then we can 
>>> consider (as a separate additive proposal) adding them back later.
>>> - As to conservatism, explicitly reserving “..” (for possible future 
>>> language directions) seems reasonable to me.  Are there any other similar 
>>> things we should consider reserving?
>>> 
>>> - I applaud the creativity keeping  a valid identifier :-), but it is 
>>> really missing the point.  *All* of the non-symbol-like emoji’s should be 
>>> valid in identifiers.  With a quick unscientific look at Apple’s character 
>>> picker, all the emojis other than a few in “Symbols” seem like they should 
>>> be identifiers.  It would be fine to conservatively leave all emoji 
>>> “symbols” as unassigned.
>> 
>> The problem with this is that "emoji" is not a well-defined category by 
>> Unicode. Whether a character is rendered as emoji or a traditional symbol in 
>> a given font on a given platform can depend on variation selectors, and the 
>> exact variation selectors (or lack thereof) that choose emoji or traditional 
>> representation are non-portable, even among different text rendering APIs on 
>> the same platform (e.g. ATSUI vs TextKit vs CoreText vs WebKit on Darwin).
>> 
>> -Joe
>> 
> 
> I’m not sure that is true. Unicode gives the list: 
> http://unicode.org/emoji/charts/full-emoji-list.html 
> 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Benjamin Spratling via swift-evolution]

And there’s our confirmation of hardware emoji keyboard.
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

I'm not aware of any Unicode stipulations on rendering of unassigned code
points. In any case, Swift _n_ doesn't need to be designed around specific
platforms that exist in 2016, but it'd be perfectly sensible to say that
Swift 4 should restrict valid identifiers to those characters that are
displayed consistently on widely used platforms existing in 2016.

On Thu, Oct 27, 2016 at 19:35 Russ Bishop  wrote:

> On Oct 25, 2016, at 3:19 AM, Xiaodi Wu  wrote:
>
> Unfortunately, Joe is correct on this point. As I stated earlier in the
> thread, there are a series of characters that can be either text or emoji
> in presentation, where the default presentation differs depending on
> platform, technology, use case, or context. This is also not a bug, but
> explicitly contemplated by Unicode technical recommendations. You can
> convince yourself of this fact by looking up the Wikipedia page on the
> Unicode "dingbats" block and comparing the rendering on Safari on iOS and
> Safari on macOS. You will see that they are different.
>
>
> Unfortunately, you are incorrect about the behavior of missing glyphs.
> Unlike, say, Chinese displayed on a machine without the necessary fonts,
> there is a security concern that Unicode 9 emoji not yet supported by Apple
> are non-displaying on that platform. No placeholder appears. This includes
> what is according to Emojipedia the #1 most popular emoji, the shrug.
> (Check out Emojipedia on a Mac.) It appears that there is no required
> placeholder glyph for unsupported emoji, so any of them can legitimately
> disappear on a non-supported platform. This is an issue worth serious
> consideration.
>
>
> IMHO I don’t think Swift needs to be designed around rendering bugs with
> specific fonts on specific platforms. We can file a radar to have this
> corrected. I’m not aware of anything in Unicode that says it is acceptable
> to just drop unknown characters. I think some ZJW sequences or modifiers
> can be ignored; anything that can be ignored for rendering should be
> ignored for uniqueness of identifiers too.
>
>
> Russ
>
>
> On Tue, Oct 25, 2016 at 00:41 Russ Bishop via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> On Oct 24, 2016, at 9:43 AM, Joe Groff via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>
> On Oct 23, 2016, at 9:41 PM, Chris Lattner via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>
> On Oct 18, 2016, at 11:34 PM, Jacob Bandes-Storch via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> Dear Swift-Evolution community,
>
> A few of us have been preparing a proposal to refine the definitions of
> identifiers & operators. This includes some changes to the permitted
> Unicode characters.
>
> The latest (perhaps final?) draft is available here:
>
>
> https://github.com/jtbandes/swift-evolution/blob/unicode-id-op/proposals/-refining-identifier-and-operator-symbology.md
>
> We'd welcome your initial thoughts, and will probably submit a PR soon to
> the swift-evolution repo for a formal review. Full text follows below.
>
>
> I haven’t had a chance to read the entire proposal, nor the tons of great
> discussion down thread, but here are a few thoughts, just MHO:
>
> - I’m loving that you’re taking a detail oriented approach to the
> problem.  I agree with you that our current approach is unprincipled, and
> we need to get this right for Swift 4.
> - I think that it is perfectly fine to err on the side of conservatism: if
> it isn’t clear how to classify something (e.g. Braille patterns), we should
> just reject them in both operators and identifiers (make them be
> unassigned).  If these unclear cases are important to someone, then we can
> consider (as a separate additive proposal) adding them back later.
> - As to conservatism, explicitly reserving “..” (for possible future
> language directions) seems reasonable to me.  Are there any other similar
> things we should consider reserving?
>
> - I applaud the creativity keeping  a valid identifier :-), but it is
> really missing the point.  *All* of the non-symbol-like emoji’s should be
> valid in identifiers.  With a quick unscientific look at Apple’s character
> picker, all the emojis other than a few in “Symbols” seem like they should
> be identifiers.  It would be fine to conservatively leave all emoji
> “symbols” as unassigned.
>
>
> The problem with this is that "emoji" is not a well-defined category by
> Unicode. Whether a character is rendered as emoji or a traditional symbol
> in a given font on a given platform can depend on variation selectors, and
> the exact variation selectors (or lack thereof) that choose emoji or
> traditional representation are non-portable, even among different text
> rendering APIs on the same platform (e.g. ATSUI vs TextKit vs CoreText vs
> WebKit on Darwin).
>
> -Joe
>
>
> I’m not sure that is true. Unicode gives the list:
> http://unicode.org/emoji/charts/full-emoji-list.html.
>
> If a platform can’t 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Erica Sadun via swift-evolution]

> On Oct 27, 2016, at 11:46 AM, Jonathan Hull via swift-evolution 
>  wrote:
> 
> It looks like the issue of typing unicode symbols was solved much sooner than 
> any of us were predicting…
> 
> Thanks,
> Jon

Swift as a language extends beyond Apple's platform and Unicode symbology.

-- E


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Thorsten Seitz via swift-evolution]

+1

-Thorsten 

> Am 20.10.2016 um 02:32 schrieb Nevin Brackett-Rozinsky via swift-evolution 
> :
> 
> I strongly oppose the proposed mass-removal of operator characters. It would 
> be a major loss to the Swift language if that were to occur even temporarily.
> 
> The long-term goal is for Swift to adopt the official Unicode guidance for 
> operator characters, which is still under development. Therefore I believe we 
> should make only minor and obvious changes right now, because there is no 
> sense in “jumping the gun” and causing unnecessary source-breaking changes.
> 
> In particular, we should make it clear that Swift will most likely adopt the 
> Unicode operator conventions when they become available, so people are aware 
> and prepared.
> 
> When the time comes, we should deprecate any operator characters that Unicode 
> recommends against (unless we have a good reason not to), before removing 
> them in the next major release. The deprecation period ensures that 
> source-breaking changes result in a warning at first, so developers have time 
> to adapt.
> 
> I just went through all the valid operator characters in Swift, and the only 
> ones I would recommend to eliminate at this time are:
> U+2800 – U+28FF (Braille patterns)
> U+3021 – U+3029 (Hangzhou numerals)
> U+2205 and U+221E (Empty set and Infinity)
> 
> Additionally, I propose to *add* one operator that is missing:
> U+214B (Turned ampersand)
> 
> • • •
> 
> As for the rest of the proposal, I suppose normalizing identifiers and 
> dealing with confusable characters in a sensible way.
> 
> Regarding emoji, I look at them rather like the “I’m feeling lucky” button on 
> Google—essentially nobody uses it, but when they tried getting rid of it 
> people didn’t like the result. So I agree with Brent about that we should 
> keep them for cultural, not technical, reasons.
> 
> • • •
> 
> Returning to the discussion of operators, I am reminded of what happened when 
> we eliminated argument labels from functions passed as parameters. The intent 
> was and still is to reinstate them in a more robust manner.
> 
> However, during the interim the result has been a regression that goes 
> against the core tenets and philosophy of Swift. I would not want to repeat 
> that while waiting for official Unicode operator guidelines.
> 
> So I am strongly—adamantly—opposed to the operator-eviscerating portion of 
> this proposal.
> 
> We should make Braille characters, Hangzhou numerals, the empty set and the 
> infinity sign into identifiers. All other operators should remain as they are 
> until official Unicode recommendations exist, at which point we should 
> deprecate as necessary.
> 
> Nevin
> 
> 
> 
>> On Wed, Oct 19, 2016 at 5:53 PM, Matthew Johnson via swift-evolution 
>>  wrote:
 
 
>>> 
 IMO, the best argument against using unicode symbols for operators defined 
 by mathematics is that they are currently difficult to type.
>>> 
>>> And there is no realistic hope of that changing. This issue is so 
>>> compelling that C and C++ introduced standardized text-ascii alternatives 
>>> for the punctuation operators to relieve stress on non-english keyboard 
>>> users.
>> 
>> I don’t agree that there is no realistic hope of that changing.  It appears 
>> to be pretty reasonable to anticipate that we’ll all be using 
>> software-driven keyboards that can display software-defined symbols on the 
>> keys in the relatively near future (probably 5 years, certainly 10).  All 
>> kinds of interesting things become possible when that happens, including the 
>> ability to make unicode operators much easier to discover and type in a 
>> programmer’s editor.
>> 
>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan Hull via swift-evolution]

It looks like the issue of typing unicode symbols was solved much sooner than 
any of us were predicting…

Thanks,
Jon
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[David Sweeris via swift-evolution]

Sent from my iPhone
> On Oct 25, 2016, at 10:24, Joe Groff via swift-evolution 
>  wrote:
> 
>> Ok, but to clarify the requirement, *every* file would have to declare the 
>> operators it is using at the top of the file.  It isn’t enough for them to 
>> be declared in some file within the current module.  Not having this 
>> property breaks the ability to do a quick parse of a file without doing name 
>> lookup.
> 
> Yeah, that's a tradeoff. I think that requiring non-standard operator use to 
> be explicitly declared could be a good thing, though, since I don't think 
> that we can realistically expect users to learn or intuitively agree on what 
> glyphs are "operator" or "identifier", no matter what character set we design.
I could get behind having to explicitly import operators:
import CoolLib
import operators CoolLib
or
import CoolLib {types functions vars operators}
But having to re-declare every "non-standard" operator for every file really 
limits their usefulness, IMHO.

> As long as { } aren't in the operator character set, we should still be able 
> to skip function bodies without parsing, so operator use declarations could 
> still be order-independent at the top level of declarations. (Whether it's a 
> good idea to bury your import declarations in the middle of your other decls 
> is another story.)
Oh, is using {} as operators on the table? There's gotta be some interesting 
syntax someone could make with those...

- Dave Sweeris___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Matt Whiteside via swift-evolution]

> On Oct 24, 2016, at 22:33, Chris Lattner via swift-evolution 
>  wrote:
> would it be possible to carve off some obvious blocks of emoji support as 
> identifiers (e.g. not symbols, flags, or anything else complicated), and 
> carve off the most obvious blocks of the math operators as operators?  For 
> the operator set, maybe we could start with some small subset of 100 (totally 
> random number here) operators that are commonly requested and seem obvious, 
> then expand it out to a principled set once UAX31 is resolved?  

+1 to this approach. 

Regarding emoji, I don’t really use them much myself, so I’m favor of the 
minimal amount of work needed to get them under control (for now).  I’d rather 
see the effort spent elsewhere.  The operator stuff is interesting though.

-Matt___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Chris Lattner via swift-evolution]

> On Oct 25, 2016, at 3:08 PM, Anton Zhilin via swift-evolution 
>  wrote:
> 
> Why can't we just remove distinction between operator and identifier symbols? 
> I'd be fine with the following:

This has been discussed in prior threads: it is core to the behavior of the 
parser.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Johannes Neubauer via swift-evolution]

> Am 26.10.2016 um 00:08 schrieb Anton Zhilin via swift-evolution 
> :
> 
> Why can't we just remove distinction between operator and identifier symbols? 
> I'd be fine with the following:
> 
> ```swift
> infix operator map
> infix func map(lhs: [Int], rhs: (Int) -> Int) { ... }
> [1,2,3] map {$0*2}
> ```

Kotlin allows arbitrary infix functions. Nice feature. +1


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Anton Zhilin via swift-evolution]

Why can't we just remove distinction between operator and identifier
symbols? I'd be fine with the following:

```swift
infix operator map
infix func map(lhs: [Int], rhs: (Int) -> Int) { ... }
[1,2,3] map {$0*2}
```

No explicit imports required, plus we can create nice DSLs. Of course, it's
an additive change, but it's worth considering right now, because it can
remove some headache caused by current distinction.
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Adrian Kashivskyy via swift-evolution]

I’m also -1 on disallowing emojis as identifiers. As it was stated may times 
before, emojis are an important part of modern communication, especially 
between young people and kids.

I understand the complexity of keeping them around, especially if they are not 
well-defined by Unicode and if they are not rendered correctly in certain 
environments, but that seems like a valid argument to defer this discussion and 
not to make rash decision.

In the end, as a compromise, I would vote to perhaps restrict the range of 
allowed emojis until they become more standardized.

– Adrian___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Joe Groff via swift-evolution]

> On Oct 24, 2016, at 10:40 PM, Russ Bishop  wrote:
> 
>> 
>> On Oct 24, 2016, at 9:43 AM, Joe Groff via swift-evolution 
>> > wrote:
>> 
>> 
>>> On Oct 23, 2016, at 9:41 PM, Chris Lattner via swift-evolution 
>>> > wrote:
>>> 
>>> 
 On Oct 18, 2016, at 11:34 PM, Jacob Bandes-Storch via swift-evolution 
 > wrote:
 
 Dear Swift-Evolution community,
 
 A few of us have been preparing a proposal to refine the definitions of 
 identifiers & operators. This includes some changes to the permitted 
 Unicode characters.
 
 The latest (perhaps final?) draft is available here:
 
 
 https://github.com/jtbandes/swift-evolution/blob/unicode-id-op/proposals/-refining-identifier-and-operator-symbology.md
  
 
 
 We'd welcome your initial thoughts, and will probably submit a PR soon to 
 the swift-evolution repo for a formal review. Full text follows below.
>>> 
>>> I haven’t had a chance to read the entire proposal, nor the tons of great 
>>> discussion down thread, but here are a few thoughts, just MHO:
>>> 
>>> - I’m loving that you’re taking a detail oriented approach to the problem.  
>>> I agree with you that our current approach is unprincipled, and we need to 
>>> get this right for Swift 4.
>>> - I think that it is perfectly fine to err on the side of conservatism: if 
>>> it isn’t clear how to classify something (e.g. Braille patterns), we should 
>>> just reject them in both operators and identifiers (make them be 
>>> unassigned).  If these unclear cases are important to someone, then we can 
>>> consider (as a separate additive proposal) adding them back later.
>>> - As to conservatism, explicitly reserving “..” (for possible future 
>>> language directions) seems reasonable to me.  Are there any other similar 
>>> things we should consider reserving?
>>> 
>>> - I applaud the creativity keeping  a valid identifier :-), but it is 
>>> really missing the point.  *All* of the non-symbol-like emoji’s should be 
>>> valid in identifiers.  With a quick unscientific look at Apple’s character 
>>> picker, all the emojis other than a few in “Symbols” seem like they should 
>>> be identifiers.  It would be fine to conservatively leave all emoji 
>>> “symbols” as unassigned.
>> 
>> The problem with this is that "emoji" is not a well-defined category by 
>> Unicode. Whether a character is rendered as emoji or a traditional symbol in 
>> a given font on a given platform can depend on variation selectors, and the 
>> exact variation selectors (or lack thereof) that choose emoji or traditional 
>> representation are non-portable, even among different text rendering APIs on 
>> the same platform (e.g. ATSUI vs TextKit vs CoreText vs WebKit on Darwin).
>> 
>> -Joe
>> 
> 
> I’m not sure that is true. Unicode gives the list: 
> http://unicode.org/emoji/charts/full-emoji-list.html 
> . 
> 
> If a platform can’t render the ZJW sequences it can render them as separate 
> Emoji, but Swift can still treat that as the same identifier.
> 
>   ==  
>  
> If you don’t have a font capable of displaying the character at all that 
> isn’t any different from not having a Chinese font available. You should get 
> the missing character glyph. The list of emoji base characters is not 
> unrestricted - there is a specific and limited list of valid base characters 
> that accept modifiers. 
> 
> If we wanted to go further and say that all Emoji modifiers are preserved and 
> rendered if possible but not considered part of the identifier that would be 
> OK with me. Same for variation selectors.

Good to hear that there's some work being done into standardizing emoji—this 
wasn't the case three years ago when we adopted the N1518 character set. If 
there's a Unicode standard we can cite for what constitutes an emoji, that'd be 
a good foundation for "supporting emoji" in Swift syntax, though as Xiaodi 
notes, implementations still vary quite a bit in what they present as emoji in 
practice, and the fact that unsupported emoji are allowed to rendered 
completely invisible is troubling.

-Joe

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Joe Groff via swift-evolution]

> On Oct 24, 2016, at 10:15 PM, Chris Lattner  wrote:
> 
>> 
>> On Oct 24, 2016, at 9:40 AM, Joe Groff via swift-evolution 
>>  wrote:
>> 
>> 
>>> On Oct 22, 2016, at 5:53 PM, Jonathan S. Shapiro 
>>>  wrote:
>>> 
>>> I missed this earlier posting from Joe Groff, who wrote:
>>> 
>>> In the discussion about operators, I wonder whether it makes sense to 
>>> formally separate "identifier" and "operator" characters at all. ...
>>> 
>>> The consequence if we do not formally separate the operators (verbs) from 
>>> the identifiers (nouns) is that white space will be needed around all 
>>> operators. That's not necessarily a bad thing, but it would be a 
>>> significant and incompatible departure from today's Swift, both in terms of 
>>> actual source code breakage and in terms of the "look and feel" that many 
>>> people feel passionate about.
>> 
>> That's not a strict requirement. If we require operator usage to be declared 
>> explicitly, the lexer can accommodate those declarations. Since operators 
>> only appear as part of expressions inside bodies, the operator import or 
>> declaration doesn't even necessarily have to be ordered at the top of the 
>> file since we can still skip function bodies when parsing declarations 
>> (though I think we'd want to encourage imports on top anyway for the benefit 
>> of readers). This wouldn't be unprecedented—operators as they stand already 
>> effectively require an extra pass of parsing.
> 
> Ok, but to clarify the requirement, *every* file would have to declare the 
> operators it is using at the top of the file.  It isn’t enough for them to be 
> declared in some file within the current module.  Not having this property 
> breaks the ability to do a quick parse of a file without doing name lookup.

Yeah, that's a tradeoff. I think that requiring non-standard operator use to be 
explicitly declared could be a good thing, though, since I don't think that we 
can realistically expect users to learn or intuitively agree on what glyphs are 
"operator" or "identifier", no matter what character set we design.

> In addition to the tooling impact, going with such an approach would be very 
> inconsistent with the rest of Swift’s grammar, which aims to be order 
> independent (except in script files / top level code).

As long as { } aren't in the operator character set, we should still be able to 
skip function bodies without parsing, so operator use declarations could still 
be order-independent at the top level of declarations. (Whether it's a good 
idea to bury your import declarations in the middle of your other decls is 
another story.)

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

Unfortunately, Joe is correct on this point. As I stated earlier in the
thread, there are a series of characters that can be either text or emoji
in presentation, where the default presentation differs depending on
platform, technology, use case, or context. This is also not a bug, but
explicitly contemplated by Unicode technical recommendations. You can
convince yourself of this fact by looking up the Wikipedia page on the
Unicode "dingbats" block and comparing the rendering on Safari on iOS and
Safari on macOS. You will see that they are different.

Unfortunately, you are incorrect about the behavior of missing glyphs.
Unlike, say, Chinese displayed on a machine without the necessary fonts,
there is a security concern that Unicode 9 emoji not yet supported by Apple
are non-displaying on that platform. No placeholder appears. This includes
what is according to Emojipedia the #1 most popular emoji, the shrug.
(Check out Emojipedia on a Mac.) It appears that there is no required
placeholder glyph for unsupported emoji, so any of them can legitimately
disappear on a non-supported platform. This is an issue worth serious
consideration.

Finally, the issue remains live as to whether we can have some way of
confronting the issue that Apple platforms now have emoji that differ both
in name and appearance from other platforms. In the latest version of macOS
Sierra, a wide swath of emoji have been renamed to diverge from Unicode
guidelines. Some, like "pile of poo" have been renamed only subtly ("pile
of poop"). However, others like "imp" have been completely renamed, so that
where formerly the Apple rendering was a stretch as compared to Unicode
recommendations, now Apple platforms are literally drawing and describing a
completely different thing at the same codepoint. We in the Swift community
obviously cannot tell Apple how to draw or name their emoji. But, Apple is
clearly willing to unilaterally revise arbitrary numbers of emoji in a
single release of their OS and may continue to do so. Is this appropriate
for programming language identifiers? I think not. But I can't square that
with the clear demand for emoji identifiers in the community.

On Tue, Oct 25, 2016 at 00:41 Russ Bishop via swift-evolution <
swift-evolution@swift.org> wrote:

> On Oct 24, 2016, at 9:43 AM, Joe Groff via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>
> On Oct 23, 2016, at 9:41 PM, Chris Lattner via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>
> On Oct 18, 2016, at 11:34 PM, Jacob Bandes-Storch via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> Dear Swift-Evolution community,
>
> A few of us have been preparing a proposal to refine the definitions of
> identifiers & operators. This includes some changes to the permitted
> Unicode characters.
>
> The latest (perhaps final?) draft is available here:
>
>
> https://github.com/jtbandes/swift-evolution/blob/unicode-id-op/proposals/-refining-identifier-and-operator-symbology.md
>
> We'd welcome your initial thoughts, and will probably submit a PR soon to
> the swift-evolution repo for a formal review. Full text follows below.
>
>
> I haven’t had a chance to read the entire proposal, nor the tons of great
> discussion down thread, but here are a few thoughts, just MHO:
>
> - I’m loving that you’re taking a detail oriented approach to the
> problem.  I agree with you that our current approach is unprincipled, and
> we need to get this right for Swift 4.
> - I think that it is perfectly fine to err on the side of conservatism: if
> it isn’t clear how to classify something (e.g. Braille patterns), we should
> just reject them in both operators and identifiers (make them be
> unassigned).  If these unclear cases are important to someone, then we can
> consider (as a separate additive proposal) adding them back later.
> - As to conservatism, explicitly reserving “..” (for possible future
> language directions) seems reasonable to me.  Are there any other similar
> things we should consider reserving?
>
> - I applaud the creativity keeping  a valid identifier :-), but it is
> really missing the point.  *All* of the non-symbol-like emoji’s should be
> valid in identifiers.  With a quick unscientific look at Apple’s character
> picker, all the emojis other than a few in “Symbols” seem like they should
> be identifiers.  It would be fine to conservatively leave all emoji
> “symbols” as unassigned.
>
>
> The problem with this is that "emoji" is not a well-defined category by
> Unicode. Whether a character is rendered as emoji or a traditional symbol
> in a given font on a given platform can depend on variation selectors, and
> the exact variation selectors (or lack thereof) that choose emoji or
> traditional representation are non-portable, even among different text
> rendering APIs on the same platform (e.g. ATSUI vs TextKit vs CoreText vs
> WebKit on Darwin).
>
> -Joe
>
>
> I’m not sure that is true. Unicode gives the list:
> 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Russ Bishop via swift-evolution]

> On Oct 24, 2016, at 9:43 AM, Joe Groff via swift-evolution 
>  wrote:
> 
> 
>> On Oct 23, 2016, at 9:41 PM, Chris Lattner via swift-evolution 
>> > wrote:
>> 
>> 
>>> On Oct 18, 2016, at 11:34 PM, Jacob Bandes-Storch via swift-evolution 
>>> > wrote:
>>> 
>>> Dear Swift-Evolution community,
>>> 
>>> A few of us have been preparing a proposal to refine the definitions of 
>>> identifiers & operators. This includes some changes to the permitted 
>>> Unicode characters.
>>> 
>>> The latest (perhaps final?) draft is available here:
>>> 
>>> 
>>> https://github.com/jtbandes/swift-evolution/blob/unicode-id-op/proposals/-refining-identifier-and-operator-symbology.md
>>>  
>>> 
>>> 
>>> We'd welcome your initial thoughts, and will probably submit a PR soon to 
>>> the swift-evolution repo for a formal review. Full text follows below.
>> 
>> I haven’t had a chance to read the entire proposal, nor the tons of great 
>> discussion down thread, but here are a few thoughts, just MHO:
>> 
>> - I’m loving that you’re taking a detail oriented approach to the problem.  
>> I agree with you that our current approach is unprincipled, and we need to 
>> get this right for Swift 4.
>> - I think that it is perfectly fine to err on the side of conservatism: if 
>> it isn’t clear how to classify something (e.g. Braille patterns), we should 
>> just reject them in both operators and identifiers (make them be 
>> unassigned).  If these unclear cases are important to someone, then we can 
>> consider (as a separate additive proposal) adding them back later.
>> - As to conservatism, explicitly reserving “..” (for possible future 
>> language directions) seems reasonable to me.  Are there any other similar 
>> things we should consider reserving?
>> 
>> - I applaud the creativity keeping  a valid identifier :-), but it is 
>> really missing the point.  *All* of the non-symbol-like emoji’s should be 
>> valid in identifiers.  With a quick unscientific look at Apple’s character 
>> picker, all the emojis other than a few in “Symbols” seem like they should 
>> be identifiers.  It would be fine to conservatively leave all emoji 
>> “symbols” as unassigned.
> 
> The problem with this is that "emoji" is not a well-defined category by 
> Unicode. Whether a character is rendered as emoji or a traditional symbol in 
> a given font on a given platform can depend on variation selectors, and the 
> exact variation selectors (or lack thereof) that choose emoji or traditional 
> representation are non-portable, even among different text rendering APIs on 
> the same platform (e.g. ATSUI vs TextKit vs CoreText vs WebKit on Darwin).
> 
> -Joe
> 

I’m not sure that is true. Unicode gives the list: 
http://unicode.org/emoji/charts/full-emoji-list.html 
. 

If a platform can’t render the ZJW sequences it can render them as separate 
Emoji, but Swift can still treat that as the same identifier.

  ==  
 
If you don’t have a font capable of displaying the character at all that isn’t 
any different from not having a Chinese font available. You should get the 
missing character glyph. The list of emoji base characters is not unrestricted 
- there is a specific and limited list of valid base characters that accept 
modifiers. 

If we wanted to go further and say that all Emoji modifiers are preserved and 
rendered if possible but not considered part of the identifier that would be OK 
with me. Same for variation selectors.


Russ


>> - I really think we should keep symbols as operators, including much of the 
>> math symbols (e.g. ∪).  In a later separate proposal, we can consider 
>> whether it makes sense for emoji symbols (like ✖️to be usable as operators), 
>> I can see arguments both ways.
>> 
>> -Chris
>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Chris Lattner via swift-evolution]

> On Oct 21, 2016, at 9:38 PM, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
> Well, it seems that I jumped the gun and sent my document link to 
> swift-evolution by mistake. Since I can't take it back, it might be good to 
> say what it's about. Because of my mistake, this version has not had any 
> review by the rest of the author group, which probably would have improved it 
> a lot. I had intended one more round of edits to deal with a few things that 
> still say "FIX" and a few minor cleanups, but I think the substance of the 
> proposal is sound.
> 
> This revised proposal 
> 
>  tries to take into account most of the feedback we have received here. Lots 
> of nitty-gritty changes, but here's the big picture of the changes:

This is looking like great progress over the original proposal and (to restate 
the obvious) I’m a huge fan of your (and Jacob, Erica, and Xiaodi’s) work on 
this proposal.  Thank you ALL for driving this forward, it is a very messy but 
critical task to get right for Swift 4.

My biggest concern is the Operator Definition section 
(https://github.com/jsshapiro/swift-evolution/blob/unicode-id-op/proposals/-refining-identifiers-and-operators.md#operator-definition)

The issue is that this breaks the ability to separately parse files.  The C 
family of languages generally requires the parser to reason about a 
“translation unit”, and used a technique colloquially known as the “lexer hack” 
to determine whether an identifier token refers to a type or a value 
definition.  Once the lexer determines this, the parser behaves differently.  
Your proposal introduces an analogous “lexer hack”, for the purposes of 
determining whether an token is an identifier or an operator.

This is deeply concerning to me, because this would break the ability to parse 
a file without parsing all of its dependencies. Our current support for this 
has some well known problems (e.g. you can’t resolve expressions to a 
particularly useful representation: you get SequenceExprs unless you can 
perform name lookup) but we do have the ability parse *declarations* with high 
fidelity.  

This ability is extremely important for us to be able to get incremental 
compilation of an individual source file (which typically depends on 
declarations from other source files, but does not depend on the contents of 
their bodies) and is important for various latency-critical IDE features.  For 
example, if you perform a code completion within a function, you generally only 
need to fully type check the function body that you’re inside, everything else 
in the file can be ignored, and only referenced declarations need to be 
(recursively) checked.


I think you have a good set of goals and motivations, and I agree that aligning 
with UAX31’s ultimate resolution is important.  That said, I hope that we can 
tackle this in Swift 4 by taking a small but reasonable subset of what UAX31 is 
“certainly” going to support, and then wait for UAX31 to be finalized before 
expanding the rest out.

I’m not a unicode expert by any stretch of the imagination, but would it be 
possible to carve off some obvious blocks of emoji support as identifiers (e.g. 
not symbols, flags, or anything else complicated), and carve off the most 
obvious blocks of the math operators as operators?  For the operator set, maybe 
we could start with some small subset of 100 (totally random number here) 
operators that are commonly requested and seem obvious, then expand it out to a 
principled set once UAX31 is resolved?  This would avoid regressing too much in 
Swift 4 from Swift 3, but also dramatically limit the risk of painting 
ourselves into a corner that turns out to be incompatible with UAX31.

-Chris


> Emojis are admitted, subject to reasonable sanity conditions.
> A (significantly) broader, but still conservative set of code points are 
> admitted for symbol identifiers. Hopefully this addresses current need, but I 
> remain open to adopting full-on [:Sm:] and [:So:] if there is a strong push 
> for that.
> Operator definition is orthogonal to identifier specification, which deals 
> with the noun/verb confusion and also addresses the widely-expressed feeling 
> that some symbols aren't operators and their conventional meaning should be 
> usable. The term "operator" no longer has anything to do with identifiers.
> A laundry list of potential parsing gotchas are addressed. The previous 
> proposal would have broken the generics syntax and also the binding syntax. 
> This isn't a substantive conceptual change, but it's important if the 
> proposal is going to, you know, actually work. :-)
> Dollar is admitted in identifiers.
> Explicitly addresses anonymous closure parameters in a way that reflects how 
> the compiler actually needs to deal with such things. Might be I've written a 
> 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Chris Lattner via swift-evolution]

> On Oct 24, 2016, at 9:40 AM, Joe Groff via swift-evolution 
>  wrote:
> 
> 
>> On Oct 22, 2016, at 5:53 PM, Jonathan S. Shapiro 
>> > wrote:
>> 
>> I missed this earlier posting from Joe Groff, who wrote:
>> 
>> In the discussion about operators, I wonder whether it makes sense to 
>> formally separate "identifier" and "operator" characters at all. ...
>> 
>> The consequence if we do not formally separate the operators (verbs) from 
>> the identifiers (nouns) is that white space will be needed around all 
>> operators. That's not necessarily a bad thing, but it would be a significant 
>> and incompatible departure from today's Swift, both in terms of actual 
>> source code breakage and in terms of the "look and feel" that many people 
>> feel passionate about.
> 
> That's not a strict requirement. If we require operator usage to be declared 
> explicitly, the lexer can accommodate those declarations. Since operators 
> only appear as part of expressions inside bodies, the operator import or 
> declaration doesn't even necessarily have to be ordered at the top of the 
> file since we can still skip function bodies when parsing declarations 
> (though I think we'd want to encourage imports on top anyway for the benefit 
> of readers). This wouldn't be unprecedented—operators as they stand already 
> effectively require an extra pass of parsing.

Ok, but to clarify the requirement, *every* file would have to declare the 
operators it is using at the top of the file.  It isn’t enough for them to be 
declared in some file within the current module.  Not having this property 
breaks the ability to do a quick parse of a file without doing name lookup.

In addition to the tooling impact, going with such an approach would be very 
inconsistent with the rest of Swift’s grammar, which aims to be order 
independent (except in script files / top level code).

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Chris Lattner via swift-evolution]

> On Oct 20, 2016, at 5:03 PM, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
> On Thu, Oct 20, 2016 at 12:25 PM, Jonathan Hull via swift-evolution 
> > wrote:
> Would it be possible to do the following:
> 
> • Have 1 group which are always used as identifiers.  This would probably be 
> the identifiers from this proposal.
> 
> • Have a 2nd group which are always used as operators (quite a bit larger 
> than the group proposed by this proposal).  At a minimum, the following ascii 
> + set operators + math operators:
> (± ≠ ≤ ≥ ¿ ¡ ™ ¢ ¶ • ° ƒ © √ ∆ ◊ § ≈  ∫ ÷ ¬ ).
> 
> • Everything else is in a 3rd group. 
> 
> No. This is far, far too complicated, and it mixes up the layers of the 
> compilation process.

Right.  Any proposal that changes parser behavior based on a visible operator 
declaration will break the ability for Swift to separately compile files.  This 
will have massive tooling ramifications that are almost certainly a non-starter.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Chris Lattner via swift-evolution]

> On Oct 20, 2016, at 7:03 AM, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
> On Thu, Oct 20, 2016 at 12:12 AM, Austin Zheng via swift-evolution 
> > wrote:
> Is there a compromise we can come up with, maybe?
> 
> So speaking just for myself, I strongly oppose emojis because every example 
> of emoji code I have seen has been truly obfuscated. Emojis therefore present 
> very serious and active source-level security risks that will require 
> significant engineering investment to manage and will never be fully managed 
> successfully.
> 
> That said, I'm very glad that some people here have pointed out the "kid use 
> case", because I had not considered that one. I think that's actually pretty 
> compelling.
> 
> Let me ask a question: would single-character emoji identifiers be enough, or 
> do we need multi-character emojis? Single-character emoji identifiers would 
> go a long way toward limiting the capacity for obfuscation, but I'm guessing 
> it won't be enough for a bunch of people here.

I don’t think it is a goal to “prevent” or “limit” obfuscation.  Operator 
overloading and using weird symbols is inherently going to obfuscate code for 
some people, and if it were a goal, we’d prevent operator overloading entirely. 
 We can’t legislate in the language that code is readable and maintainable.  We 
need to trust people to use the tools the language provides in a sane way, and 
rely on team feedback and coding standards to set the norm in their environment.

Besides that, I think we’ve all seen code where it would be most honest to name 
a variable or function .  That’s the sometimes sad reality of software, and 
Swift should aim to support honest expression of these realities. :-) :-)

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Chris Lattner via swift-evolution]

> On Oct 19, 2016, at 1:46 PM, Brent Royal-Gordon via swift-evolution 
>  wrote:
> 
> I was in the middle of writing about my opposition to the original proposal 
> when I went to bed last night, and was going to advocate something like this:
> 
>> Given the current state of the discussion over in Unicode land, I think it 
>> would probably be safe from a compatibility standpoint to admit code points 
>> that fall into the following (Unicode-style) code point set:
>> 
>> [:S:] - [:Sc:] - [:xidcontinue:] - [:nfcqc=n:] & [:scx=Common:] - 
>> pictographics - emoji
> 
> I suspect we can probably also do something about emoji, since I doubt UAX 
> #31 is going to. Given that they are all static pictures of people or things, 
> I think we can decide they are all nouns and thus all identifier characters. 
> If we think there are some which might be declared operators later, we can 
> exclude them for now, but I'd like to at least see the bulk of them brought 
> in.
> 
> I think addressing emoji is important not for any technical reason, but for 
> nontechnical ones. Emoji are a statement about Swift's modern approach; 
> modernity is important. They are fun and whimsical; whimsy is important.
> 
> And most importantly, emoji identifiers are part of Swift's culture. It's 
> widely understood that you don't use them in real code, but they are very 
> common in examples. Just as we worry about source compatibility and binary 
> compatibility, so we should worry about culture compatibility. Removing emoji 
> would cause a gratuitous cultural regression.

Very well said Brent: +1 from me.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Joe Groff via swift-evolution]

> On Oct 22, 2016, at 5:53 PM, Jonathan S. Shapiro 
>  wrote:
> 
> I missed this earlier posting from Joe Groff, who wrote:
> 
> In the discussion about operators, I wonder whether it makes sense to 
> formally separate "identifier" and "operator" characters at all. ...
> 
> The consequence if we do not formally separate the operators (verbs) from the 
> identifiers (nouns) is that white space will be needed around all operators. 
> That's not necessarily a bad thing, but it would be a significant and 
> incompatible departure from today's Swift, both in terms of actual source 
> code breakage and in terms of the "look and feel" that many people feel 
> passionate about.

That's not a strict requirement. If we require operator usage to be declared 
explicitly, the lexer can accommodate those declarations. Since operators only 
appear as part of expressions inside bodies, the operator import or declaration 
doesn't even necessarily have to be ordered at the top of the file since we can 
still skip function bodies when parsing declarations (though I think we'd want 
to encourage imports on top anyway for the benefit of readers). This wouldn't 
be unprecedented—operators as they stand already effectively require an extra 
pass of parsing.

-Joe

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Joe Groff via swift-evolution]

> On Oct 23, 2016, at 9:41 PM, Chris Lattner via swift-evolution 
>  wrote:
> 
> 
>> On Oct 18, 2016, at 11:34 PM, Jacob Bandes-Storch via swift-evolution 
>> > wrote:
>> 
>> Dear Swift-Evolution community,
>> 
>> A few of us have been preparing a proposal to refine the definitions of 
>> identifiers & operators. This includes some changes to the permitted Unicode 
>> characters.
>> 
>> The latest (perhaps final?) draft is available here:
>> 
>> 
>> https://github.com/jtbandes/swift-evolution/blob/unicode-id-op/proposals/-refining-identifier-and-operator-symbology.md
>>  
>> 
>> 
>> We'd welcome your initial thoughts, and will probably submit a PR soon to 
>> the swift-evolution repo for a formal review. Full text follows below.
> 
> I haven’t had a chance to read the entire proposal, nor the tons of great 
> discussion down thread, but here are a few thoughts, just MHO:
> 
> - I’m loving that you’re taking a detail oriented approach to the problem.  I 
> agree with you that our current approach is unprincipled, and we need to get 
> this right for Swift 4.
> - I think that it is perfectly fine to err on the side of conservatism: if it 
> isn’t clear how to classify something (e.g. Braille patterns), we should just 
> reject them in both operators and identifiers (make them be unassigned).  If 
> these unclear cases are important to someone, then we can consider (as a 
> separate additive proposal) adding them back later.
> - As to conservatism, explicitly reserving “..” (for possible future language 
> directions) seems reasonable to me.  Are there any other similar things we 
> should consider reserving?
> 
> - I applaud the creativity keeping  a valid identifier :-), but it is 
> really missing the point.  *All* of the non-symbol-like emoji’s should be 
> valid in identifiers.  With a quick unscientific look at Apple’s character 
> picker, all the emojis other than a few in “Symbols” seem like they should be 
> identifiers.  It would be fine to conservatively leave all emoji “symbols” as 
> unassigned.

The problem with this is that "emoji" is not a well-defined category by 
Unicode. Whether a character is rendered as emoji or a traditional symbol in a 
given font on a given platform can depend on variation selectors, and the exact 
variation selectors (or lack thereof) that choose emoji or traditional 
representation are non-portable, even among different text rendering APIs on 
the same platform (e.g. ATSUI vs TextKit vs CoreText vs WebKit on Darwin).

-Joe

> - I really think we should keep symbols as operators, including much of the 
> math symbols (e.g. ∪).  In a later separate proposal, we can consider whether 
> it makes sense for emoji symbols (like ✖️to be usable as operators), I can 
> see arguments both ways.
> 
> -Chris
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Chris Lattner via swift-evolution]

> On Oct 18, 2016, at 11:34 PM, Jacob Bandes-Storch via swift-evolution 
>  wrote:
> 
> Dear Swift-Evolution community,
> 
> A few of us have been preparing a proposal to refine the definitions of 
> identifiers & operators. This includes some changes to the permitted Unicode 
> characters.
> 
> The latest (perhaps final?) draft is available here:
> 
> 
> https://github.com/jtbandes/swift-evolution/blob/unicode-id-op/proposals/-refining-identifier-and-operator-symbology.md
>  
> 
> 
> We'd welcome your initial thoughts, and will probably submit a PR soon to the 
> swift-evolution repo for a formal review. Full text follows below.

I haven’t had a chance to read the entire proposal, nor the tons of great 
discussion down thread, but here are a few thoughts, just MHO:

- I’m loving that you’re taking a detail oriented approach to the problem.  I 
agree with you that our current approach is unprincipled, and we need to get 
this right for Swift 4.
- I think that it is perfectly fine to err on the side of conservatism: if it 
isn’t clear how to classify something (e.g. Braille patterns), we should just 
reject them in both operators and identifiers (make them be unassigned).  If 
these unclear cases are important to someone, then we can consider (as a 
separate additive proposal) adding them back later.
- As to conservatism, explicitly reserving “..” (for possible future language 
directions) seems reasonable to me.  Are there any other similar things we 
should consider reserving?

- I applaud the creativity keeping  a valid identifier :-), but it is really 
missing the point.  *All* of the non-symbol-like emoji’s should be valid in 
identifiers.  With a quick unscientific look at Apple’s character picker, all 
the emojis other than a few in “Symbols” seem like they should be identifiers.  
It would be fine to conservatively leave all emoji “symbols” as unassigned.
- I really think we should keep symbols as operators, including much of the 
math symbols (e.g. ∪).  In a later separate proposal, we can consider whether 
it makes sense for emoji symbols (like ✖️to be usable as operators), I can see 
arguments both ways.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Martin Waitz via swift-evolution]

Jonathan, that’s a really nice proposal! :-)

You also already achieved some points from your „future directions“ ;-)

— Martin
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Ben Rimmington via swift-evolution]

Re: 


### Conventional Identifiers

`conventional-ident-head → [:Emoji:]`
will include the keycap base characters ('#', '*', ASCII digits).



Does the NFC format prevent emoji ZWJ sequences?


### Symbol Identifiers

You could include both blocks of mathematical operators.
* U+2200 ... U+22FF [:Block=Mathematical_Operators:]
* U+2A00 ... U+2AFF [:Block=Supplemental_Mathematical_Operators:]

-- Ben

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Colin Barrett via swift-evolution]

> On Oct 22, 2016, at 8:53 PM, Jonathan S. Shapiro 
>  wrote:
> 
> I missed this earlier posting from Joe Groff, who wrote:
> 
> In the discussion about operators, I wonder whether it makes sense to 
> formally separate "identifier" and "operator" characters at all. ...
> 
> The consequence if we do not formally separate the operators (verbs) from the 
> identifiers (nouns) is that white space will be needed around all operators. 
> That's not necessarily a bad thing, but it would be a significant and 
> incompatible departure from today's Swift, both in terms of actual source 
> code breakage and in terms of the "look and feel" that many people feel 
> passionate about.

That’s one way yeah. Or you’d could to make the grammar context sensitive / 
apply a "lexer hack”. Probably other ways to deal w/ context sensitivity as 
well. Joe’s proposed syntax seems pretty explicit, and hopefully it’sdsimple to 
plumb / capture that info in the lexer. (I’m ignorant of the implementation of 
Swift’s front-end unfortunately!)

-Colin

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

I missed this earlier posting from Joe Groff, who wrote:

In the discussion about operators, I wonder whether it makes sense to
> formally separate "identifier" and "operator" characters at all. ...


The consequence if we do not formally separate the operators (verbs) from
the identifiers (nouns) is that white space will be needed around all
operators. That's not necessarily a bad thing, but it would be a
significant and incompatible departure from today's Swift, both in terms of
actual source code breakage and in terms of the "look and feel" that many
people feel passionate about.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Colin Barrett via swift-evolution]

I’m a -1 on the original proposal. I can see the logic in doing things that 
way, but it’s really unclear to me why we need to act *now*. In fact it seems 
like waiting might be a better option, given the things mentioned upthread 
about revisions to the Unicode standard.

Also, I think the message quoted below is a promising direction worth 
exploring. How would something like this work in the front-end? Swift’s grammar 
currently distinguishes between operators and identifiers right?

-Colin

> On Oct 19, 2016, at 12:17 PM, Joe Groff via swift-evolution 
>  wrote:
> 
> I think this is a promising direction. Getting us in line with Unicode 
> recommendations is an important first step, and being conservative about the 
> treatment of operator characters and emoji is a good engineering approach, 
> though certainly unfortunate in the short term for users who've adopted 
> custom operators or found interesting uses for emoji identifiers in Swift 3 
> and earlier.
> 
> In the discussion about operators, I wonder whether it makes sense to 
> formally separate "identifier" and "operator" characters at all. My hunch is 
> that there isn't going to be any perfect categorization; there are so many 
> symbols and scripts out there that it's going to be difficult to definitively 
> characterize many symbols as "obviously" an operator or identifier. Not every 
> developer has the mathematical background to even recognize common math 
> operators beyond the elementary arithmetic ones. Something to consider would 
> be to change the way operators work in the language so that they can use 
> *any* symbols (subject to canonicalization, visibility, and confusability 
> constraints), but require their use to always be explicitly declared in a 
> source file that uses an operator outside of the standard library. For 
> example, you would have to say something like:
> 
> import Sets
> import operator Sets.∪
> 
> to make the '∪' symbol available as an operator in the import declaration's 
> scope. This would provide more obvious evidence in the source code of what 
> tokens are being employed as operators, and lessen the need to have formally 
> distinct identifier and operator character sets.
> 
> -Joe
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

That, I think, is where we're headed. Take a look at Jonathan Shapiro's
latest draft and see what you think :)
On Sat, Oct 22, 2016 at 17:46 Matt Whiteside via swift-evolution <
swift-evolution@swift.org> wrote:

>
> On Oct 19, 2016, at 19:07, Jonathan Hull via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> 
>
> There are a bunch of symbols in unicode which are hard to tell apart, and
> those are bad for recognition, and we should deal with that, but this
> proposal is throwing the baby out with the bathwater, then lighting the
> baby on fire.  Honestly, I would propose we find a way to have Swift see
> certain classes of characters as identical.  Can’t decide which of the
> thousand + symbols should be the one true + symbol?  Have them all map to
> one of them.  That emoji X symbol looks like X, so map it to X.
>
>
>
> Agreed.
>
> I really like the option of using unicode operators in swift.  Removing
> them would be disappointing.  But I do see the problem that there are a lot
> of redundancies and sources of confusion if there are no restrictions.
> Aside from the redundancies, some other cases that I find sub-optimal are:
> 'top-half of integral' ⌠, and  'left parenthesis upper hook’ ⎛ ;  these don’t
> really seem like symbols we want to allow as operators.  Perhaps
> *one* general rule here, is that symbols specifically meant for 2d math
> expressions aren't good candidates for inclusion, at least at the present.
> Another difficult one, I think, are the bracket-like glyphs.  While
> these would be useful in certain math and physics related code, it’s not
> clear at the moment how these could be used in swift, until and unless some
> kind of ‘bracket overloading’ is made possible.
>
> I’m in favor of what Johnathan Hull has suggested above.  I’m also in
> favor of what some others have suggested, i.e., restricting the allowed
> operator symbols to some uncontroversial subset, mainly from the unicode
> ‘math’ category of symbols, and then possibly adding more as needed.
>
> -Matt
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Matt Whiteside via swift-evolution]

> On Oct 19, 2016, at 19:07, Jonathan Hull via swift-evolution 
>  wrote:

> There are a bunch of symbols in unicode which are hard to tell apart, and 
> those are bad for recognition, and we should deal with that, but this 
> proposal is throwing the baby out with the bathwater, then lighting the baby 
> on fire.  Honestly, I would propose we find a way to have Swift see certain 
> classes of characters as identical.  Can’t decide which of the thousand + 
> symbols should be the one true + symbol?  Have them all map to one of them.  
> That emoji X symbol looks like X, so map it to X.


Agreed.

I really like the option of using unicode operators in swift.  Removing them 
would be disappointing.  But I do see the problem that there are a lot of 
redundancies and sources of confusion if there are no restrictions.  Aside from 
the redundancies, some other cases that I find sub-optimal are: 'top-half of 
integral' ⌠, and  'left parenthesis upper hook’ ⎛ ;  these don’t really seem 
like symbols we want to allow as operators.  Perhaps *one* general rule here, 
is that symbols specifically meant for 2d math expressions aren't good 
candidates for inclusion, at least at the present.  Another difficult one, I 
think, are the bracket-like glyphs.  While these would be useful in certain 
math and physics related code, it’s not clear at the moment how these could be 
used in swift, until and unless some kind of ‘bracket overloading’ is made 
possible.

I’m in favor of what Johnathan Hull has suggested above.  I’m also in favor of 
what some others have suggested, i.e., restricting the allowed operator symbols 
to some uncontroversial subset, mainly from the unicode ‘math’ category of 
symbols, and then possibly adding more as needed.

-Matt

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Daniel Duan via swift-evolution]

It’s worth pointing out that the proposal to add ‘$’ to identifiers is still 
under active review and have generated much controversy. I wouldn’t put much 
weight in “backward compatibility” for that proposal.

> On Oct 21, 2016, at 9:38 PM, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
> Well, it seems that I jumped the gun and sent my document link to 
> swift-evolution by mistake. Since I can't take it back, it might be good to 
> say what it's about. Because of my mistake, this version has not had any 
> review by the rest of the author group, which probably would have improved it 
> a lot. I had intended one more round of edits to deal with a few things that 
> still say "FIX" and a few minor cleanups, but I think the substance of the 
> proposal is sound.
> 
> This revised proposal 
> 
>  tries to take into account most of the feedback we have received here. Lots 
> of nitty-gritty changes, but here's the big picture of the changes:
> Emojis are admitted, subject to reasonable sanity conditions.
> A (significantly) broader, but still conservative set of code points are 
> admitted for symbol identifiers. Hopefully this addresses current need, but I 
> remain open to adopting full-on [:Sm:] and [:So:] if there is a strong push 
> for that.
> Operator definition is orthogonal to identifier specification, which deals 
> with the noun/verb confusion and also addresses the widely-expressed feeling 
> that some symbols aren't operators and their conventional meaning should be 
> usable. The term "operator" no longer has anything to do with identifiers.
> A laundry list of potential parsing gotchas are addressed. The previous 
> proposal would have broken the generics syntax and also the binding syntax. 
> This isn't a substantive conceptual change, but it's important if the 
> proposal is going to, you know, actually work. :-)
> Dollar is admitted in identifiers.
> Explicitly addresses anonymous closure parameters in a way that reflects how 
> the compiler actually needs to deal with such things. Might be I've written a 
> compiler or two in my career. :-)
> Consistent with the current direction of UAX31 on these issues.
> Susan Kare's legacy is preserved. :-) If you don't know who Susan is, look 
> her up and learn why Chris loves the dogcow emoji pair.
> The new proposal remains entirely compatible with Swift 3, except where 
> existing source runs up against the narrower symbol identifier space. It's a 
> specific goal to avoid breaking reasonable current practice where possible, 
> though we're surely going to break something with this one.
> 
> I was trained to write specifications in a school that favored rigorous 
> writing. In order to make sure I didn't lose track of something I rewrote the 
> proposal in a form that I know how to use effectively. Any loss of "fun" in 
> the text is my fault alone.
> 
> Interested to see how this will be received.
> 
> 
> Jonathan
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[David Sweeris via swift-evolution]

> On Oct 20, 2016, at 9:37 AM, Jonathan S. Shapiro 
>  wrote:
> 
> On Thu, Oct 20, 2016 at 7:30 AM, David Sweeris  > wrote:
> Sent from my iPhone
> 
> On Oct 20, 2016, at 09:03, Jonathan S. Shapiro via swift-evolution 
> > wrote:
> 
>> On Thu, Oct 20, 2016 at 12:12 AM, Austin Zheng via swift-evolution 
>> > wrote:
>> 
>> Freeze the set of allowed emoji to whatever the current version of the 
>> Unicode spec defines...
>> 
>> UAX31 won't include emojis in either space, because there is no clear 
>> consensus about where they belong (identifiers or operators). Individual 
>> languages can certainly add them to one space or the other, but should take 
>> care not to cross-contaminate. So if we add them to operators, we need to 
>> exclude any that are already part of normal identifiers and vice versa. That 
>> sanity restriction is technically necessary, but it shouldn't be an 
>> inconvenience in practical terms.
> 
> My understanding (which is admittedly fuzzy) is that the distinction between 
> operators and identifiers is only "technically necessary" because allowing 
> characters to be both causes the parsing algorithm lose its virtual mind, and 
> it takes a century for it to figure out what's going on. What I don't recall 
> being discussed before is whether that's a blanket penalty or if the compile 
> times increases are proportional to the amount overlap between the two 
> character sets.
> 
> The hard requirements are:
> Nothing in identifier start can be in operator start or operator continue. [*]
> Nothing in operator start can be in identifier start or identifier continue. 
> [*]
> Nothing in syntactic punctuation (period, brackets, parens, and so forth) can 
> be in either type of identifier without creating a lot of serious hair. You 
> can see one example of hair in the "double dots" rule.
> If these requirements are not preserved, the consequence is that white space 
> becomes required between identifiers and operators. So, for example, without 
> these rules:
> 
> a+b// gets broken
> a + b  // works
> 
> The presence of dots in operators is actually causing a whole bunch of 
> constraints to get introduced that I'm going to talk about in a moment.


Ah, ok. Your explanation sounds very familiar… Clearly I’ve read it before and 
forgot. I must be going senile in my old age (37).

Would this whitespace rule affect pre/postfix operators, or just the infix 
ones? Personally, I’m fine with requiring whitespace around infix operators, 
but pre/postfix operators would completely lose readability if they needed it, 
too. Have we previously discussed this on the mailing list?

Given my apparent forgetfulness, I have no doubt that we discussed it at length 
two weeks ago, and someone will probably reply to this, quoting some forgotten 
3-page email I sent on the topic :-)

- Dave Sweeris___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

On Sat, Oct 22, 2016 at 1:37 AM, Nevin Brackett-Rozinsky via
swift-evolution  wrote:

> I just read through your new proposal, and I have to say it is extremely
> well-written. There is a vast quantity of information presented quite
> clearly, and it gives me a lot to think about.
>
>
> On Fri, Oct 21, 2016 at 5:38 PM, Jonathan S. Shapiro  gmail.com> wrote:
>
>> On Fri, Oct 21, 2016 at 1:54 PM, Nevin Brackett-Rozinsky via
>> swift-evolution  wrote:
>>
>>> I think it is plainly evident that the well-defined criteria you would
>>> like to use *have not yet been defined* by Unicode. That is a large part of
>>> why I recommend that we postpone a major overhaul of our operator
>>> characters.
>>>
>>
>> That's a feasible way to go, but keep in mind that the UAX31 changes are
>> being co-designed with and informed by the current discussion. There are a
>> bunch of things that have come up here that will allow UAX31 to side-step
>> some "might have happened" mistakes, so this discussion has been very
>> useful.
>>
>> The Swift community can and should make its own decision about whether to
>> remain engaged. The risk of disengagement is that messy compatibility
>> issues will probably have to be faced later that we can easily head-off now.
>>
>
> Ah, I had not previously understood that. Well then, in light of the fact
> that the Unicode recommendations may be influenced by our decisions, and
> given that Swift is an opinionated language, it follows that we ought to
> make our best effort at separating out what we have been calling “operator
> characters” (and your revised proposal calls “symbol identifier”
> characters).
>
> In particular, since there does not yet exist a categorization of symbols
> which fits our needs, and since our needs may help shape such a
> categorization as it forms, it behooves us to fully undertake the endeavor
> of defining which symbols we would like to see in which roles for Swift.
>
> Your proposal mentions and links to a set of 650 code points
> 
>  that
> your group identified by hand as operators. It also links to the combined Sm
> and So categories
> .
> However what you actually propose is the far-more-limited Mathematical
> Operators block
> 
> .
>
> I will take it upon myself to go through code-points by hand and see what
> I can find.
>
> It is worth noting that your proposed “symbol identifier” category, by its
> very name, suggests it should have broader membership than just operators.
> I am not sure if that was intentional, however I will restrict my attention
> to symbols that may reasonably function as operators.
>
> After a preliminary glance through the code blocks, I believe there are
> operator-like characters in these blocks
> 
> :
> Basic Latin
> Latin-1 Supplement
> General Punctuation
> Letterlike Symbols
> Arrows
> Mathematical Operators
> 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Nevin Brackett-Rozinsky via swift-evolution]

I just read through your new proposal, and I have to say it is extremely
well-written. There is a vast quantity of information presented quite
clearly, and it gives me a lot to think about.


On Fri, Oct 21, 2016 at 5:38 PM, Jonathan S. Shapiro <
jonathan.s.shap...@gmail.com> wrote:

> On Fri, Oct 21, 2016 at 1:54 PM, Nevin Brackett-Rozinsky via
> swift-evolution  wrote:
>
>> I think it is plainly evident that the well-defined criteria you would
>> like to use *have not yet been defined* by Unicode. That is a large part of
>> why I recommend that we postpone a major overhaul of our operator
>> characters.
>>
>
> That's a feasible way to go, but keep in mind that the UAX31 changes are
> being co-designed with and informed by the current discussion. There are a
> bunch of things that have come up here that will allow UAX31 to side-step
> some "might have happened" mistakes, so this discussion has been very
> useful.
>
> The Swift community can and should make its own decision about whether to
> remain engaged. The risk of disengagement is that messy compatibility
> issues will probably have to be faced later that we can easily head-off now.
>

Ah, I had not previously understood that. Well then, in light of the fact
that the Unicode recommendations may be influenced by our decisions, and
given that Swift is an opinionated language, it follows that we ought to
make our best effort at separating out what we have been calling “operator
characters” (and your revised proposal calls “symbol identifier”
characters).

In particular, since there does not yet exist a categorization of symbols
which fits our needs, and since our needs may help shape such a
categorization as it forms, it behooves us to fully undertake the endeavor
of defining which symbols we would like to see in which roles for Swift.

Your proposal mentions and links to a set of 650 code points

that
your group identified by hand as operators. It also links to the combined Sm
and So categories
.
However what you actually propose is the far-more-limited Mathematical
Operators block

.

I will take it upon myself to go through code-points by hand and see what I
can find.

It is worth noting that your proposed “symbol identifier” category, by its
very name, suggests it should have broader membership than just operators.
I am not sure if that was intentional, however I will restrict my attention
to symbols that may reasonably function as operators.

After a preliminary glance through the code blocks, I believe there are
operator-like characters in these blocks

:
Basic Latin
Latin-1 Supplement
General Punctuation
Letterlike Symbols
Arrows
Mathematical Operators
Miscellaneous Technical
Miscellaneous Mathematical Symbols-A
Supplemental Arrows-A
Supplemental Arrows-B
Miscellaneous Mathematical Symbols-B
Supplemental Mathematical Operators
Miscellaneous Symbols and Arrows
Supplemental Punctuation

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

All:

Xiaodi has pointed out that the existing Unicode emoji set pulls in a host
of stuff we don't want. My bad for failing to adequately understand what
that file represented.

We can either pull in selected blocks with appropriate sanity filters or
possibly consider adopting the Unicode pictographics. I'll look at cleaning
this up in the morning. If somebody has put forth a clean proposal by then,
so much the better.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Fri, Oct 21, 2016 at 9:26 PM, Xiaodi Wu  wrote:

> You kinda sent that to swift-evolution :P
>

Another painful reminder of why "oops" is a four-letter word...

Sorry!


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

Well, it seems that I jumped the gun and sent my document link to
swift-evolution by mistake. Since I can't take it back, it might be good to
say what it's about. Because of my mistake, this version has *not* had any
review by the rest of the author group, which probably would have improved
it a lot. I had intended one more round of edits to deal with a few things
that still say "FIX" and a few minor cleanups, but I think the substance of
the proposal is sound.

This revised proposal

tries
to take into account most of the feedback we have received here. Lots of
nitty-gritty changes, but here's the big picture of the changes:

   - Emojis are admitted, subject to reasonable sanity conditions.
   - A (significantly) broader, but still conservative set of code points
   are admitted for symbol identifiers. Hopefully this addresses current need,
   but I remain open to adopting full-on [:Sm:] and [:So:] if there is a
   strong push for that.
   - Operator definition is orthogonal to identifier specification, which
   deals with the noun/verb confusion and also addresses the widely-expressed
   feeling that some symbols aren't operators and their conventional meaning
   should be usable. The term "operator" no longer has anything to do with
   identifiers.
   - A laundry list of potential parsing gotchas are addressed. The
   previous proposal would have broken the generics syntax and also the
   binding syntax. This isn't a substantive conceptual change, but it's
   important if the proposal is going to, you know, *actually work*. :-)
   - Dollar is admitted in identifiers.
   - Explicitly addresses anonymous closure parameters in a way that
   reflects how the compiler actually needs to deal with such things. Might be
   I've written a compiler or two in my career. :-)
   - Consistent with the current direction of UAX31 on these issues.
   - Susan Kare's legacy is preserved. :-) If you don't know who Susan is,
   look her up and learn why Chris loves the dogcow emoji pair.

The new proposal remains entirely compatible with Swift 3, except where
existing source runs up against the narrower symbol identifier space. It's
a specific goal to avoid breaking reasonable current practice where
possible, though we're surely going to break *something* with this one.

I was trained to write specifications in a school that favored rigorous
writing. In order to make sure I didn't lose track of something I rewrote
the proposal in a form that I know how to use effectively. Any loss of
"fun" in the text is my fault alone.

Interested to see how this will be received.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

On Fri, Oct 21, 2016 at 4:32 PM, Jonathan S. Shapiro <
jonathan.s.shap...@gmail.com> wrote:

> On Fri, Oct 21, 2016 at 1:10 PM, Xiaodi Wu via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>> A few other thoughts:
>>
>> * One of our (or at least my) overarching goals for this proposal is to
>> refine identifier and operator sets by moving away from an ad-hoc
>> character-by-character approach to a systematic treatment that relies on
>> well-defined criteria to select blocks of characters for inclusion.
>>
>
> I completely agree with this, up to the point where you wrote "select
> blocks". That doesn't seem to be the way things are selected in the Unicode
> universe.
>
> We can choose to adopt blocks as an interim measure, but we should not
> loose sight of the notion that this should eventually be property-driven.
>
> * Particularly if the community insists on emoji being identifiers, we
>> will have to critically evaluate how to proceed on that in tandem with
>> operators, because there exist emojified operators and arrows, and certain
>> emoji are encoded in blocks that are otherwise symbols, which Unicode is
>> likely to deem as operators in the future.
>>
>
> This is not correct. The current view in the UAX31 discussion is that
> emojis and pictographics should be excluded from *both* types of
> identifiers so that individual programming languages can make
> language-specific choices.
>

This proposed approach raises some issues with apparent inconsistency as
well as forward compatibility issues. What I'm saying is that today, among
a chunk of symbols which Unicode may deem to be operators, there will be
some with emoji variants and others without. It seems kinda arbitrary to
exclude specific arrows or specific dingbats from valid operator characters
on the criterion that they have an emoji variant. For instance, if curly
leftwards arrow has an emoji variant but curly upwards arrow does not, one
is considered an invalid operator but the other is valid? Now, going
forward, if an existing codepoint is today part of IDC_Start but tomorrow
gains an emoji variant, what happens then?

The main current problem is that there is no clear-cut Unicode property
> covering Emojis at the present time, which is something that needs to be
> resolved over in Unicode-land. There is a list given in the antique texty
> UCD file format, but it isn't part of the XML formulation of the UCD
> database. I'll be generating a proposed update shortly, and when I have
> that I can provide a working list in the form of a C file that can be used
> by Swift.
>
> I have a mild preference that emojis should live in conventional
> identifiers if they are adopted.
>
>
>>  Moreover, IIUC, certain codepoints can be either emoji or non-emoji
>> symbols and variant selectors can specify which they are...
>>
>
> Can you expand on this and (hopefully) point me at the appropriate spot in
> one of the Unicode TRs?
>
>
> Thanks!
>
>
> Jonathan
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

On Fri, Oct 21, 2016 at 4:32 PM, Jonathan S. Shapiro <
jonathan.s.shap...@gmail.com> wrote:

> On Fri, Oct 21, 2016 at 1:10 PM, Xiaodi Wu via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>> A few other thoughts:
>>
>> * One of our (or at least my) overarching goals for this proposal is to
>> refine identifier and operator sets by moving away from an ad-hoc
>> character-by-character approach to a systematic treatment that relies on
>> well-defined criteria to select blocks of characters for inclusion.
>>
>
> I completely agree with this, up to the point where you wrote "select
> blocks". That doesn't seem to be the way things are selected in the Unicode
> universe.
>
> We can choose to adopt blocks as an interim measure, but we should not
> loose sight of the notion that this should eventually be property-driven.
>
> * Particularly if the community insists on emoji being identifiers, we
>> will have to critically evaluate how to proceed on that in tandem with
>> operators, because there exist emojified operators and arrows, and certain
>> emoji are encoded in blocks that are otherwise symbols, which Unicode is
>> likely to deem as operators in the future.
>>
>
> This is not correct. The current view in the UAX31 discussion is that
> emojis and pictographics should be excluded from *both* types of
> identifiers so that individual programming languages can make
> language-specific choices.
>
> The main current problem is that there is no clear-cut Unicode property
> covering Emojis at the present time, which is something that needs to be
> resolved over in Unicode-land. There is a list given in the antique texty
> UCD file format, but it isn't part of the XML formulation of the UCD
> database. I'll be generating a proposed update shortly, and when I have
> that I can provide a working list in the form of a C file that can be used
> by Swift.
>
> I have a mild preference that emojis should live in conventional
> identifiers if they are adopted.
>
>
>>  Moreover, IIUC, certain codepoints can be either emoji or non-emoji
>> symbols and variant selectors can specify which they are...
>>
>
> Can you expand on this and (hopefully) point me at the appropriate spot in
> one of the Unicode TRs?
>

Indeed. This issue first popped up on my radar when John Gruber wrote that
he had issues with his website displaying the curly arrow symbol as emoji
in recent browsers:

https://twitter.com/gruber/status/590355262281744384

Turns out, certain characters have emoji variants and text variants, which
can be selected for explicitly by appending a variant selector:

http://mts.io/2015/04/21/unicode-symbol-render-text-emoji/

However, whether the *default* presentation in the absence of a variant
selector is text or emoji can change from platform to platform, or from use
case to use case. This is explicitly spelled out in UTR#51:

"The presentation of a given emoji character depends on the environment,
whether or not there is an emoji or text variation selector, and the
default presentation style (emoji vs text). In informal environments like
texting and chats, it is more appropriate for most emoji characters to
appear with a colorful emoji presentation, and only get a text presentation
with a text variation selector. Conversely, in formal environments such as
word processing, it is generally better for emoji characters to appear with
a text presentation, and only get the colorful emoji presentation with the
emoji variation selector."

http://unicode.org/reports/tr51/#Presentation_Style


>
>
> Thanks!
>
>
> Jonathan
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Fri, Oct 21, 2016 at 1:51 PM, Martin Waitz via swift-evolution <
swift-evolution@swift.org> wrote:

> With the current difficulty to come up with an agreed set of identifies
> vs. operators, maybe we should really try to sidestep this issue entirely
> and study Jonathan Shapiro’s idea of using identifiers as operators.
>

Martin:

My "operators" proposal relies on resolving what will be a symbol
identifier, so I'm afraid we can't duck this issue.

I'm writing up a revised proposal as we speak. I'm hopeful that it will
address *most* of the concerns that have been expressed here, but it adopts
a slightly different conceptual approach than the previous one - enough so
that I don't think of it as a revision.

Because of the concern that Xiaodi just raised about emojis and modifiers,
I'm going to identify that as an open issue with an "if feasible" proposed
resolution to place them in the traditional identifier space.

I hope to be able to send a link to this list in a few hours.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Fri, Oct 21, 2016 at 1:54 PM, Nevin Brackett-Rozinsky via
swift-evolution  wrote:

> I think it is plainly evident that the well-defined criteria you would
> like to use *have not yet been defined* by Unicode. That is a large part of
> why I recommend that we postpone a major overhaul of our operator
> characters.
>

That's a feasible way to go, but keep in mind that the UAX31 changes are
being co-designed with and informed by the current discussion. There are a
bunch of things that have come up here that will allow UAX31 to side-step
some "might have happened" mistakes, so this discussion has been very
useful.

The Swift community can and should make its own decision about whether to
remain engaged. The risk of disengagement is that messy compatibility
issues will probably have to be faced later that we can easily head-off now.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Fri, Oct 21, 2016 at 1:10 PM, Xiaodi Wu via swift-evolution <
swift-evolution@swift.org> wrote:

> A few other thoughts:
>
> * One of our (or at least my) overarching goals for this proposal is to
> refine identifier and operator sets by moving away from an ad-hoc
> character-by-character approach to a systematic treatment that relies on
> well-defined criteria to select blocks of characters for inclusion.
>

I completely agree with this, up to the point where you wrote "select
blocks". That doesn't seem to be the way things are selected in the Unicode
universe.

We can choose to adopt blocks as an interim measure, but we should not
loose sight of the notion that this should eventually be property-driven.

* Particularly if the community insists on emoji being identifiers, we will
> have to critically evaluate how to proceed on that in tandem with
> operators, because there exist emojified operators and arrows, and certain
> emoji are encoded in blocks that are otherwise symbols, which Unicode is
> likely to deem as operators in the future.
>

This is not correct. The current view in the UAX31 discussion is that
emojis and pictographics should be excluded from *both* types of
identifiers so that individual programming languages can make
language-specific choices.

The main current problem is that there is no clear-cut Unicode property
covering Emojis at the present time, which is something that needs to be
resolved over in Unicode-land. There is a list given in the antique texty
UCD file format, but it isn't part of the XML formulation of the UCD
database. I'll be generating a proposed update shortly, and when I have
that I can provide a working list in the form of a C file that can be used
by Swift.

I have a mild preference that emojis should live in conventional
identifiers if they are adopted.


>  Moreover, IIUC, certain codepoints can be either emoji or non-emoji
> symbols and variant selectors can specify which they are...
>

Can you expand on this and (hopefully) point me at the appropriate spot in
one of the Unicode TRs?


Thanks!


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Nevin Brackett-Rozinsky via swift-evolution]

I think it is plainly evident that the well-defined criteria you would like
to use *have not yet been defined* by Unicode. That is a large part of why
I recommend that we postpone a major overhaul of our operator characters.

Furthermore, just like during the Great Renaming—when we used some general
rules to automate the name changes, then went through and fine-tuned by
hand to deal with cases where the automated rules produced suboptimal
results—I think it will be great if we can classify the majority of
characters all at once with certain criteria, but we should expect and plan
to go through by hand afterward to clean up the edge cases that humans can
tell were misplaced by the broad rules.

It is more important that we get the definitions of operator and identifier
characters *right* than that we make them *rigidly conform to a certain
rule*. If our rule says “∞” should be an operator, but we as humans
recognize that to be a mistake, then we should change it. Heck, maybe we
should make “∞” parse as a floating-point literal!

The point is, bikeshedding over operators has been and continues to be
diverting our collective attention away from the rest of the proposal, such
as using the IDC_Start and IDC_Continue categories that you mentioned.

Nevin



On Fri, Oct 21, 2016 at 4:10 PM, Xiaodi Wu  wrote:

> A few other thoughts:
>
> * One of our (or at least my) overarching goals for this proposal is to
> refine identifier and operator sets by moving away from an ad-hoc
> character-by-character approach to a systematic treatment that relies on
> well-defined criteria to select blocks of characters for inclusion. My
> personal opinion is that reverting to discussions of individual characters,
> such as the turned ampersand, means we're simply re-shuffling the current
> set of identifier and operator characters to a different one, having failed
> to discover any systematic basis for doing so. I think one of the strongest
> parts of this proposal is the straight-up statement that identifier start
> characters shall be IDC_Start and identifier continuation characters shall
> be IDC_Continue, modulo a few ASCII characters that require special
> treatment in Swift.
>
> * Particularly if the community insists on emoji being identifiers, we
> will have to critically evaluate how to proceed on that in tandem with
> operators, because there exist emojified operators and arrows, and certain
> emoji are encoded in blocks that are otherwise symbols, which Unicode is
> likely to deem as operators in the future. Moreover, IIUC, certain
> codepoints can be either emoji or non-emoji symbols and variant selectors
> can specify which they are, but in the absence of a variant selector
> *either* the emoji or non-emoji version can be correctly displayed
> depending on the platform. For some of these, the non-emoji version is
> either clearly or plausible an operator character. Therefore, without
> dealing *very* carefully with emoji and with operators at the same time, we
> are failing to address a key motivation of this proposal, which is to fix
> the incorrect separation between emoji operators and emoji identifiers.
>
>
> On Fri, Oct 21, 2016 at 2:36 PM, Xiaodi Wu  wrote:
>
>> I disagree pretty strongly with this approach. Firstly because as you've
>> pointed out, changes to the operator characters will have effects on valid
>> operator character sets, secondly because a major question about operators
>> is whether it is feasible or no to manually exclude empty set and infinity
>> given that Unicode is likely to reject that approach. For these reasons I
>> feel very strongly that the reforming the operator and identifier
>> characters must move in tandem.
>>
>> On Fri, Oct 21, 2016 at 14:28 Nevin Brackett-Rozinsky via swift-evolution
>>  wrote:
>>
>>> I think it is important that we as a community discuss the non-operator
>>> portion of this proposal. And, given the strong opinions about operators
>>> that have been expressed, I think it is unlikely we will do so while major
>>> operator changes are on the table.
>>>
>>> Thus I would suggest that either the operator changes should be
>>> separated out into their own proposal, or we should make only minor (and
>>> generally consensus-agreed) changes to the operator set as part of this one.
>>>
>>> Here is what I propose:
>>>
>>> Emoji shall be identifiers, not operators.
>>> The turned ampersand shall be an operator, not an identifier.
>>> The empty set and infinity symbols shall be identifiers, not operators.
>>>
>>> All other potential changes to the set of operator characters then go in
>>> their own proposal, which I am sure will receive a lot of attention.
>>>
>>> It may turn out that the non-operator portion of this proposal
>>> nonetheless touches characters that Swift has designated for operators, in
>>> which case we may address those as they arise.
>>>
>>> Does that sound like a reasonable way 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Martin Waitz via swift-evolution]

With the current difficulty to come up with an agreed set of identifies vs. 
operators, maybe we should really try to sidestep this issue entirely and study 
Jonathan Shapiro’s idea of using identifiers as operators.

— Martin

> Am 21.10.2016 um 22:10 schrieb Xiaodi Wu via swift-evolution 
> :
> 
> A few other thoughts:
> 
> * One of our (or at least my) overarching goals for this proposal is to 
> refine identifier and operator sets by moving away from an ad-hoc 
> character-by-character approach to a systematic treatment that relies on 
> well-defined criteria to select blocks of characters for inclusion. My 
> personal opinion is that reverting to discussions of individual characters, 
> such as the turned ampersand, means we're simply re-shuffling the current set 
> of identifier and operator characters to a different one, having failed to 
> discover any systematic basis for doing so. I think one of the strongest 
> parts of this proposal is the straight-up statement that identifier start 
> characters shall be IDC_Start and identifier continuation characters shall be 
> IDC_Continue, modulo a few ASCII characters that require special treatment in 
> Swift.
> 
> * Particularly if the community insists on emoji being identifiers, we will 
> have to critically evaluate how to proceed on that in tandem with operators, 
> because there exist emojified operators and arrows, and certain emoji are 
> encoded in blocks that are otherwise symbols, which Unicode is likely to deem 
> as operators in the future. Moreover, IIUC, certain codepoints can be either 
> emoji or non-emoji symbols and variant selectors can specify which they are, 
> but in the absence of a variant selector *either* the emoji or non-emoji 
> version can be correctly displayed depending on the platform. For some of 
> these, the non-emoji version is either clearly or plausible an operator 
> character. Therefore, without dealing *very* carefully with emoji and with 
> operators at the same time, we are failing to address a key motivation of 
> this proposal, which is to fix the incorrect separation between emoji 
> operators and emoji identifiers.
> 
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

A few other thoughts:

* One of our (or at least my) overarching goals for this proposal is to
refine identifier and operator sets by moving away from an ad-hoc
character-by-character approach to a systematic treatment that relies on
well-defined criteria to select blocks of characters for inclusion. My
personal opinion is that reverting to discussions of individual characters,
such as the turned ampersand, means we're simply re-shuffling the current
set of identifier and operator characters to a different one, having failed
to discover any systematic basis for doing so. I think one of the strongest
parts of this proposal is the straight-up statement that identifier start
characters shall be IDC_Start and identifier continuation characters shall
be IDC_Continue, modulo a few ASCII characters that require special
treatment in Swift.

* Particularly if the community insists on emoji being identifiers, we will
have to critically evaluate how to proceed on that in tandem with
operators, because there exist emojified operators and arrows, and certain
emoji are encoded in blocks that are otherwise symbols, which Unicode is
likely to deem as operators in the future. Moreover, IIUC, certain
codepoints can be either emoji or non-emoji symbols and variant selectors
can specify which they are, but in the absence of a variant selector
*either* the emoji or non-emoji version can be correctly displayed
depending on the platform. For some of these, the non-emoji version is
either clearly or plausible an operator character. Therefore, without
dealing *very* carefully with emoji and with operators at the same time, we
are failing to address a key motivation of this proposal, which is to fix
the incorrect separation between emoji operators and emoji identifiers.


On Fri, Oct 21, 2016 at 2:36 PM, Xiaodi Wu  wrote:

> I disagree pretty strongly with this approach. Firstly because as you've
> pointed out, changes to the operator characters will have effects on valid
> operator character sets, secondly because a major question about operators
> is whether it is feasible or no to manually exclude empty set and infinity
> given that Unicode is likely to reject that approach. For these reasons I
> feel very strongly that the reforming the operator and identifier
> characters must move in tandem.
>
> On Fri, Oct 21, 2016 at 14:28 Nevin Brackett-Rozinsky via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>> I think it is important that we as a community discuss the non-operator
>> portion of this proposal. And, given the strong opinions about operators
>> that have been expressed, I think it is unlikely we will do so while major
>> operator changes are on the table.
>>
>> Thus I would suggest that either the operator changes should be separated
>> out into their own proposal, or we should make only minor (and generally
>> consensus-agreed) changes to the operator set as part of this one.
>>
>> Here is what I propose:
>>
>> Emoji shall be identifiers, not operators.
>> The turned ampersand shall be an operator, not an identifier.
>> The empty set and infinity symbols shall be identifiers, not operators.
>>
>> All other potential changes to the set of operator characters then go in
>> their own proposal, which I am sure will receive a lot of attention.
>>
>> It may turn out that the non-operator portion of this proposal
>> nonetheless touches characters that Swift has designated for operators, in
>> which case we may address those as they arise.
>>
>> Does that sound like a reasonable way forward?
>>
>> Nevin
>>
>>
>>
>> On Fri, Oct 21, 2016 at 9:27 AM, Ben Rimmington via swift-evolution <
>> swift-evolution@swift.org> wrote:
>>
>>
>> > On 21 Oct 2016, at 13:42, Benjamin Spratling wrote:
>> >
>> > Brackets and symbols are definitely operators.  Different brackets are
>> used to represent various quantum mechanical forms and operations.
>>
>> The brackets are mostly "bracket pieces":
>>
>> [:Math_Symbol:]
>> - [:name=/\bANGLE\b/:]
>> - [:Emoji:]
>> - [:ID_Continue:]
>> - [:NFC_Quick_Check=No:]
>> & [:Script_Extensions=Common:]
>> & [:Block=Miscellaneous_Technical:]
>>
>> > %5B%3AMath_Symbol%3A%5D%0D%0A-+%5B%3Aname%3D%2F%5CbANGLE%
>> 5Cb%2F%3A%5D%0D%0A-+%5B%3AEmoji%3A%5D%0D%0A-+%5B%3AID_
>> Continue%3A%5D%0D%0A-+%5B%3ANFC_Quick_Check%3DNo%3A%5D%
>> 0D%0A%26+%5B%3AScript_Extensions%3DCommon%3A%5D%0D%0A%26+%5B%3ABlock%
>> 3DMiscellaneous_Technical%3A%5D>
>>
>> > Arrows are also useful as operators, including but not restricted to
>> chemical reactions.
>>
>> Including arrows, there are 740 operators:
>>
>> [:Math_Symbol:]
>> - [:name=/\bANGLE\b/:]
>> - [:name=EMPTY SET:]
>> - [:name=INFINITY:]
>> - [:Emoji:]
>> - [:ID_Continue:]
>> - [:NFC_Quick_Check=No:]
>> & [:Script_Extensions=Common:]
>> & [[:Block=Arrows:]
>>

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

I disagree pretty strongly with this approach. Firstly because as you've
pointed out, changes to the operator characters will have effects on valid
operator character sets, secondly because a major question about operators
is whether it is feasible or no to manually exclude empty set and infinity
given that Unicode is likely to reject that approach. For these reasons I
feel very strongly that the reforming the operator and identifier
characters must move in tandem.
On Fri, Oct 21, 2016 at 14:28 Nevin Brackett-Rozinsky via swift-evolution <
swift-evolution@swift.org> wrote:

> I think it is important that we as a community discuss the non-operator
> portion of this proposal. And, given the strong opinions about operators
> that have been expressed, I think it is unlikely we will do so while major
> operator changes are on the table.
>
> Thus I would suggest that either the operator changes should be separated
> out into their own proposal, or we should make only minor (and generally
> consensus-agreed) changes to the operator set as part of this one.
>
> Here is what I propose:
>
> Emoji shall be identifiers, not operators.
> The turned ampersand shall be an operator, not an identifier.
> The empty set and infinity symbols shall be identifiers, not operators.
>
> All other potential changes to the set of operator characters then go in
> their own proposal, which I am sure will receive a lot of attention.
>
> It may turn out that the non-operator portion of this proposal nonetheless
> touches characters that Swift has designated for operators, in which case
> we may address those as they arise.
>
> Does that sound like a reasonable way forward?
>
> Nevin
>
>
>
> On Fri, Oct 21, 2016 at 9:27 AM, Ben Rimmington via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>
> > On 21 Oct 2016, at 13:42, Benjamin Spratling wrote:
> >
> > Brackets and symbols are definitely operators.  Different brackets are
> used to represent various quantum mechanical forms and operations.
>
> The brackets are mostly "bracket pieces":
>
> [:Math_Symbol:]
> - [:name=/\bANGLE\b/:]
> - [:Emoji:]
> - [:ID_Continue:]
> - [:NFC_Quick_Check=No:]
> & [:Script_Extensions=Common:]
> & [:Block=Miscellaneous_Technical:]
>
> <
> http://www.unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%3AMath_Symbol%3A%5D%0D%0A-+%5B%3Aname%3D%2F%5CbANGLE%5Cb%2F%3A%5D%0D%0A-+%5B%3AEmoji%3A%5D%0D%0A-+%5B%3AID_Continue%3A%5D%0D%0A-+%5B%3ANFC_Quick_Check%3DNo%3A%5D%0D%0A%26+%5B%3AScript_Extensions%3DCommon%3A%5D%0D%0A%26+%5B%3ABlock%3DMiscellaneous_Technical%3A%5D
> >
>
> > Arrows are also useful as operators, including but not restricted to
> chemical reactions.
>
> Including arrows, there are 740 operators:
>
> [:Math_Symbol:]
> - [:name=/\bANGLE\b/:]
> - [:name=EMPTY SET:]
> - [:name=INFINITY:]
> - [:Emoji:]
> - [:ID_Continue:]
> - [:NFC_Quick_Check=No:]
> & [:Script_Extensions=Common:]
> & [[:Block=Arrows:]
>[:Block=General_Punctuation:]
>[:Block=Latin_1_Supplement:]
>[:Block=Mathematical_Operators:]
>[:Block=Miscellaneous_Mathematical_Symbols_A:]
>[:Block=Miscellaneous_Symbols_And_Arrows:]
>[:Block=Supplemental_Arrows_A:]
>[:Block=Supplemental_Arrows_B:]
>[:Block=Supplemental_Mathematical_Operators:]]
>
> <
> http://www.unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%3AMath_Symbol%3A%5D%0D%0A-+%5B%3Aname%3D%2F%5CbANGLE%5Cb%2F%3A%5D%0D%0A-+%5B%3Aname%3DEMPTY+SET%3A%5D%0D%0A-+%5B%3Aname%3DINFINITY%3A%5D%0D%0A-+%5B%3AEmoji%3A%5D%0D%0A-+%5B%3AID_Continue%3A%5D%0D%0A-+%5B%3ANFC_Quick_Check%3DNo%3A%5D%0D%0A%26+%5B%3AScript_Extensions%3DCommon%3A%5D%0D%0A%26+%5B%5B%3ABlock%3DArrows%3A%5D%0D%0A+++%5B%3ABlock%3DGeneral_Punctuation%3A%5D%0D%0A+++%5B%3ABlock%3DLatin_1_Supplement%3A%5D%0D%0A+++%5B%3ABlock%3DMathematical_Operators%3A%5D%0D%0A+++%5B%3ABlock%3DMiscellaneous_Mathematical_Symbols_A%3A%5D%0D%0A+++%5B%3ABlock%3DMiscellaneous_Symbols_And_Arrows%3A%5D%0D%0A+++%5B%3ABlock%3DSupplemental_Arrows_A%3A%5D%0D%0A+++%5B%3ABlock%3DSupplemental_Arrows_B%3A%5D%0D%0A+++%5B%3ABlock%3DSupplemental_Mathematical_Operators%3A%5D%5D
> >
>
> -- Ben
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Nevin Brackett-Rozinsky via swift-evolution]

I think it is important that we as a community discuss the non-operator
portion of this proposal. And, given the strong opinions about operators
that have been expressed, I think it is unlikely we will do so while major
operator changes are on the table.

Thus I would suggest that either the operator changes should be separated
out into their own proposal, or we should make only minor (and generally
consensus-agreed) changes to the operator set as part of this one.

Here is what I propose:

Emoji shall be identifiers, not operators.
The turned ampersand shall be an operator, not an identifier.
The empty set and infinity symbols shall be identifiers, not operators.

All other potential changes to the set of operator characters then go in
their own proposal, which I am sure will receive a lot of attention.

It may turn out that the non-operator portion of this proposal nonetheless
touches characters that Swift has designated for operators, in which case
we may address those as they arise.

Does that sound like a reasonable way forward?

Nevin



On Fri, Oct 21, 2016 at 9:27 AM, Ben Rimmington via swift-evolution <
swift-evolution@swift.org> wrote:

>
> > On 21 Oct 2016, at 13:42, Benjamin Spratling wrote:
> >
> > Brackets and symbols are definitely operators.  Different brackets are
> used to represent various quantum mechanical forms and operations.
>
> The brackets are mostly "bracket pieces":
>
> [:Math_Symbol:]
> - [:name=/\bANGLE\b/:]
> - [:Emoji:]
> - [:ID_Continue:]
> - [:NFC_Quick_Check=No:]
> & [:Script_Extensions=Common:]
> & [:Block=Miscellaneous_Technical:]
>
>  %5B%3AMath_Symbol%3A%5D%0D%0A-+%5B%3Aname%3D%2F%5CbANGLE%
> 5Cb%2F%3A%5D%0D%0A-+%5B%3AEmoji%3A%5D%0D%0A-+%5B%3AID_
> Continue%3A%5D%0D%0A-+%5B%3ANFC_Quick_Check%3DNo%3A%5D%
> 0D%0A%26+%5B%3AScript_Extensions%3DCommon%3A%5D%0D%0A%26+%5B%3ABlock%
> 3DMiscellaneous_Technical%3A%5D>
>
> > Arrows are also useful as operators, including but not restricted to
> chemical reactions.
>
> Including arrows, there are 740 operators:
>
> [:Math_Symbol:]
> - [:name=/\bANGLE\b/:]
> - [:name=EMPTY SET:]
> - [:name=INFINITY:]
> - [:Emoji:]
> - [:ID_Continue:]
> - [:NFC_Quick_Check=No:]
> & [:Script_Extensions=Common:]
> & [[:Block=Arrows:]
>[:Block=General_Punctuation:]
>[:Block=Latin_1_Supplement:]
>[:Block=Mathematical_Operators:]
>[:Block=Miscellaneous_Mathematical_Symbols_A:]
>[:Block=Miscellaneous_Symbols_And_Arrows:]
>[:Block=Supplemental_Arrows_A:]
>[:Block=Supplemental_Arrows_B:]
>[:Block=Supplemental_Mathematical_Operators:]]
>
>  %5B%3AMath_Symbol%3A%5D%0D%0A-+%5B%3Aname%3D%2F%5CbANGLE%
> 5Cb%2F%3A%5D%0D%0A-+%5B%3Aname%3DEMPTY+SET%3A%5D%0D%
> 0A-+%5B%3Aname%3DINFINITY%3A%5D%0D%0A-+%5B%3AEmoji%3A%5D%
> 0D%0A-+%5B%3AID_Continue%3A%5D%0D%0A-+%5B%3ANFC_Quick_
> Check%3DNo%3A%5D%0D%0A%26+%5B%3AScript_Extensions%3DCommon%
> 3A%5D%0D%0A%26+%5B%5B%3ABlock%3DArrows%3A%5D%0D%0A+++%5B%
> 3ABlock%3DGeneral_Punctuation%3A%5D%0D%0A+++%5B%3ABlock%
> 3DLatin_1_Supplement%3A%5D%0D%0A+++%5B%3ABlock%
> 3DMathematical_Operators%3A%5D%0D%0A+++%5B%3ABlock%
> 3DMiscellaneous_Mathematical_Symbols_A%3A%5D%0D%0A+++%5B%
> 3ABlock%3DMiscellaneous_Symbols_And_Arrows%3A%5D%0D%0A+++%5B%3ABlock%
> 3DSupplemental_Arrows_A%3A%5D%0D%0A+++%5B%3ABlock%
> 3DSupplemental_Arrows_B%3A%5D%0D%0A+++%5B%3ABlock%
> 3DSupplemental_Mathematical_Operators%3A%5D%5D>
>
> -- Ben
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Ben Rimmington via swift-evolution]

> On 21 Oct 2016, at 13:42, Benjamin Spratling wrote:
> 
> Brackets and symbols are definitely operators.  Different brackets are used 
> to represent various quantum mechanical forms and operations.

The brackets are mostly "bracket pieces":

[:Math_Symbol:]
- [:name=/\bANGLE\b/:]
- [:Emoji:]
- [:ID_Continue:]
- [:NFC_Quick_Check=No:]
& [:Script_Extensions=Common:]
& [:Block=Miscellaneous_Technical:]



> Arrows are also useful as operators, including but not restricted to chemical 
> reactions.

Including arrows, there are 740 operators:

[:Math_Symbol:]
- [:name=/\bANGLE\b/:]
- [:name=EMPTY SET:]
- [:name=INFINITY:]
- [:Emoji:]
- [:ID_Continue:]
- [:NFC_Quick_Check=No:]
& [:Script_Extensions=Common:]
& [[:Block=Arrows:]
   [:Block=General_Punctuation:]
   [:Block=Latin_1_Supplement:]
   [:Block=Mathematical_Operators:]
   [:Block=Miscellaneous_Mathematical_Symbols_A:]
   [:Block=Miscellaneous_Symbols_And_Arrows:]
   [:Block=Supplemental_Arrows_A:]
   [:Block=Supplemental_Arrows_B:]
   [:Block=Supplemental_Mathematical_Operators:]]



-- Ben

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Benjamin Spratling via swift-evolution]

Brackets and symbols are definitely operators.  Different brackets are used to 
represent various quantum mechanical forms and operations.
Arrows are also useful as operators, including but not restricted to chemical 
reactions.

-Ben

Sent from my iPhone.

> On Oct 21, 2016, at 7:20 AM, Ben Rimmington via swift-evolution 
>  wrote:
> 
> 
>> On 20 Oct 2016, at 15:29, Jonathan S. Shapiro wrote:
>> 
>> Quick poll as a sanity check on a possible alternative for operators:
>> 
>> If we admitted [:Sm:] and [:So:] and the traditional ASCII operator 
>> characters, would that cover the things that people currently feel 
>> passionate about? That would almost certainly be compliant with UAX31 once 
>> it settles, and I think it covers all of the cases people have raised here.
> 
> I'd exclude [:So:], arrows, brackets and "Miscellaneous Mathematical Symbols 
> B".
> 
>[:Math_Symbol:]
>- [:name=/\bANGLE\b/:]
>- [:name=EMPTY SET:]
>- [:name=INFINITY:]
>- [:Emoji:]
>- [:ID_Continue:]
>- [:NFC_Quick_Check=No:]
>& [:Script_Extensions=Common:]
>& [[:Block=Latin_1_Supplement:]
>   [:Block=General_Punctuation:]
>   [:Block=Mathematical_Operators:]
>   [:Block=Miscellaneous_Mathematical_Symbols_A:]
>   [:Block=Supplemental_Mathematical_Operators:]]
> 
> 
> 
> Is there a property to test for "pictographic" characters?
> 
> -- Ben
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Ben Rimmington via swift-evolution]

> On 20 Oct 2016, at 15:29, Jonathan S. Shapiro wrote:
> 
> Quick poll as a sanity check on a possible alternative for operators:
> 
> If we admitted [:Sm:] and [:So:] and the traditional ASCII operator 
> characters, would that cover the things that people currently feel passionate 
> about? That would almost certainly be compliant with UAX31 once it settles, 
> and I think it covers all of the cases people have raised here.

I'd exclude [:So:], arrows, brackets and "Miscellaneous Mathematical Symbols B".

[:Math_Symbol:]
- [:name=/\bANGLE\b/:]
- [:name=EMPTY SET:]
- [:name=INFINITY:]
- [:Emoji:]
- [:ID_Continue:]
- [:NFC_Quick_Check=No:]
& [:Script_Extensions=Common:]
& [[:Block=Latin_1_Supplement:]
   [:Block=General_Punctuation:]
   [:Block=Mathematical_Operators:]
   [:Block=Miscellaneous_Mathematical_Symbols_A:]
   [:Block=Supplemental_Mathematical_Operators:]]



Is there a property to test for "pictographic" characters?

-- Ben

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Thu, Oct 20, 2016 at 12:25 PM, Jonathan Hull via swift-evolution <
swift-evolution@swift.org> wrote:

> Would it be possible to do the following:
>
> • Have 1 group which are always used as identifiers.  This would probably
> be the identifiers from this proposal.
>
> • Have a 2nd group which are always used as operators (quite a bit larger
> than the group proposed by this proposal).  At a minimum, the following
> ascii + set operators + math operators:
> (± ≠ ≤ ≥ ¿ ¡ ™ ¢ ¶ • ° ƒ © √ ∆ ◊ § ≈  ∫ ÷ ¬ ).
>
> • Everything else is in a 3rd group.


No. This is far, far too complicated, and it mixes up the layers of the
compilation process.
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Rob Mayoff via swift-evolution]

On Thu, Oct 20, 2016 at 9:29 AM, Jonathan S. Shapiro via swift-evolution <
swift-evolution@swift.org> wrote:

> Quick poll as a sanity check on a possible alternative for operators:
>
> If we admitted [:Sm:] and [:So:] and the traditional ASCII operator
> characters, would that cover the things that people currently feel
> passionate about? That would almost certainly be compliant with UAX31 once
> it settles, and I *think* it covers all of the cases people have raised
> here.
>
> Useful links if you want to check:
>
> [:Sm:]  Symbol, Math
> 
>
> [:So:]   Symbol, Other
> 
>
>
This would define both ∞ and ∅ as operators.
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Thu, Oct 20, 2016 at 8:22 AM, Alex Blewitt  wrote:

> The "Symbol, Other" category contains "Sign of the Horns" 落 which was
> one of the problems with the identifier/operator that kicked off these
> discussions.
>

Actually not. In the interests of sanity I didn't give the full
specification statement, which excludes anything in XIDC, emojis, or
pictographics. From the UAX31 perspective those are sanity conditions, and
I think they would be prudent for Swift as well.

You can still put the horns on somebody, but you have to use a conventional
identifier :-)

Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Alex Blewitt via swift-evolution]

The "Symbol, Other" category contains "Sign of the Horns" 落 which was one of 
the problems with the identifier/operator that kicked off these discussions.

http://www.fileformat.info/info/unicode/char/1f918/index.htm

So it would break some existing cases, e.g.:

  1> let \U+1F913 = "nerd face"
邏: String = "nerd face"

http://www.fileformat.info/info/unicode/char/1f913/index.htm

On the other hand, there are some symbols in [:So:] that may be useful e.g. the 
APL Functional Symbol * series

It might be easier to have just [:Sm:] to start with, and review the [:So:] 
subsequently (or have those addressed in UAX31).

Alex

> On 20 Oct 2016, at 15:29, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
> Quick poll as a sanity check on a possible alternative for operators:
> 
> If we admitted [:Sm:] and [:So:] and the traditional ASCII operator 
> characters, would that cover the things that people currently feel passionate 
> about? That would almost certainly be compliant with UAX31 once it settles, 
> and I think it covers all of the cases people have raised here.
> 
> Useful links if you want to check:
> 
> [:Sm:]  Symbol, Math 
> 
> [:So:]   Symbol, Other 
> 
> 
> Having looked it over, I'm concerned about including [:Sk:] in UAX31 
> operators, and I'm probably going to recommend in the UAX31 discussion that 
> we shouldn't do so.
> 
> 
> Jonathan
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Thu, Oct 20, 2016 at 8:18 AM, Erica Sadun via swift-evolution <
swift-evolution@swift.org> wrote:

>
> I fully agree. It’s hella presumptuous to decide that I’m not allowed to
> express whimsy, frustration, humor, or any other emotions in my code. Or to
> tell an 8 year old using Playgrounds on the iPad that he/she can’t name a
> variable  purely because they find it *funny*. We don’t have to squash
> the joy out of *everything*.
>
>
> Russ
>
>
> The problem isn't whimsy so much as it's selecting the right set. If you
> can point to a standard (or create one) that provides a good set, which
> does not introduce the issues described in the proposal, that would be a
> great starting step for adapting the proposed approach. The same goes for
> the mathematical operators.
>

The noun/verb distinction was clarifying for me in regards to operators. Is
there a similar human-factors distinction we can identify for emojis that
might usefully inform this part of the discussion?


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Thu, Oct 20, 2016 at 8:08 AM, Erica Sadun  wrote:

> I really liked Jonathan's suggestion that removed the distinction between
> operators and identifiers entirely. You could mark a one-argument function
> as postfix or prefix, and a two-argument function as infix and use them as
> a kind of pseudo keyword.
>

Please excuse me if I booger the syntax here, but here is what this would
look like:

// Bind + as a function name in the usual way:
func +(a, b) -> { ... }

// Say we are treating + as a quasi-keyword with the given precedence.
// Requires that + be bound (before or after) as a two argument function
if infix, or a one argument function if prefix/postfix:
infix operator + : TheUsualPrecedevnce

// Parenthesizing a quasi-keyword lets you use it as an identifier:

+  // is an operator

(+) // is a use-occurrence of the function named +


I understand why collapsing the two may seem appealing, but my personal
opinion is keep separate things separate.

Even if the eventual consensus is to collapse them, let's first discuss
them as separate things so we can understand the two things that are being
said.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Erica Sadun via swift-evolution]

> On Oct 20, 2016, at 12:37 AM, Russ Bishop via swift-evolution 
>  wrote:
> 
> 
>> On Oct 19, 2016, at 1:46 PM, Brent Royal-Gordon via swift-evolution 
>> > wrote:
>> 
>> I was in the middle of writing about my opposition to the original proposal 
>> when I went to bed last night, and was going to advocate something like this:
>> 
>>> Given the current state of the discussion over in Unicode land, I think it 
>>> would probably be safe from a compatibility standpoint to admit code points 
>>> that fall into the following (Unicode-style) code point set:
>>> 
>>> [:S:] - [:Sc:] - [:xidcontinue:] - [:nfcqc=n:] & [:scx=Common:] - 
>>> pictographics - emoji
>> 
>> I suspect we can probably also do something about emoji, since I doubt UAX 
>> #31 is going to. Given that they are all static pictures of people or 
>> things, I think we can decide they are all nouns and thus all identifier 
>> characters. If we think there are some which might be declared operators 
>> later, we can exclude them for now, but I'd like to at least see the bulk of 
>> them brought in.
>> 
>> I think addressing emoji is important not for any technical reason, but for 
>> nontechnical ones. Emoji are a statement about Swift's modern approach; 
>> modernity is important. They are fun and whimsical; whimsy is important.
>> 
>> And most importantly, emoji identifiers are part of Swift's culture. It's 
>> widely understood that you don't use them in real code, but they are very 
>> common in examples. Just as we worry about source compatibility and binary 
>> compatibility, so we should worry about culture compatibility. Removing 
>> emoji would cause a gratuitous cultural regression.
>> 
> 
> I fully agree. It’s hella presumptuous to decide that I’m not allowed to 
> express whimsy, frustration, humor, or any other emotions in my code. Or to 
> tell an 8 year old using Playgrounds on the iPad that he/she can’t name a 
> variable  purely because they find it funny. We don’t have to squash the joy 
> out of everything.
> 
> 
> Russ

The problem isn't whimsy so much as it's selecting the right set. If you can 
point to a standard (or create one) that provides a good set, which does not 
introduce the issues described in the proposal, that would be a great starting 
step for adapting the proposed approach. The same goes for the mathematical 
operators.

-- E

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

Operators, Nouns, and Verbs


There's an issue that I think it's worth bringing it out into the open for
everyone to see so that we all know it is present. Solutions are possible,
but they go beyond the scope of the identifier proposal. Here's the brief
statement of the problem:

   1. Operators are verbs. They *operate* on their arguments.
   2. Math symbols are not always verbs. ∑ is a verb (and an operator). ∞
   is usually understood to be a noun.
   3. Operator *symbols* (that is: identifiers) are just names. They are
   neither inherently verbs nor inherently nouns.

We tend (at first glance) to prefer for nouns to be treated as identifiers
and operators as verbs. Operator identifiers confuse the issue because we
are calling them *operator* identifiers. A better name might be "math
symbol identifiers", because it doesn't have the same
association. Unfortunately there is no Unicode category for "Math symbols
that are verbs". This is true, in part, because there actually isn't
general agreement about how symbols are used in math. Once you get past the
basic stuff, a symbol means whatever you define it to mean in the current
publication, and math authors grab symbols entirely for the convenience of
the authors. Hopefully, but not always, in a way that reflects or suggests
a generally recognized intuition. Often no general agreement exists.

If we actually wanted to solve the noun/verb issue, we need to acknowledge
that being a noun (verb) is orthogonal to being a conventional identifier
(math symbol identifier). Here is one way to separate the concepts in Swift:

   1. Make it true that *any* identifier can be either a conventional
   identifier or a math symbol identifier. We already do this in several
   places.
   2. Make it true that *any* identifier (including a conventional
   identifier) can be treated like a reserved word (that is: like an operator)
   for parse purposes.

From a parse perspective, the thing that makes an identifier into an
operator is that (a) it has been given some status as a reserved
identifier, and (b) it has a defined precedence rule. It would be possible
to re-imagine the meaning of Swift's operator declaration syntax to mean
"this identifier is now being given reserved-word status, and should be
treated for parse purposes as an operator while this declaration is
lexically in scope". No change is required to the current language. This
re-interpretation would allow us to say (for example):

infix operator LazyAnd : *somePrecedence*


which would introduce "LazyAnd" as an operator token *even though the
identifier does not use math symbols as its characters.* Simultaneously, it
would allow us to bind ∞ and use that identifier without forcing a noun (∞)
to be a verb simply because it has symbols in the name.

I personally believe that this would resolve some of the confusion about
operators, because it would separate the "how do we tokenize?" question
from the "what behaves like an operator?" question. It would also allow us
to preserve the existing mathematical use of many math symbols that are (by
convention) nouns. From a lexer/parser perspective, the concrete change is
that we go from "it's an operator because it's made up of math symbols" to
"it's an operator because it's an identifier and it's in the list of things
that are in scope as operators" (effectively a look-up table). That's the
entire change.

Unfortunately every change comes at a cost, and the cost of this one is
that we would once again have to be thoughtful about white space. Why?
Because:

a.!  // selection of a field named "!" in object a
a.!+ // selection of a field named "!+" in object a
a.! + // selection of field named "!" in object a followed by operator (?) +


You can build comparable examples without field names:

! b // two identifiers
!+ b // two identifiers
! + b// two identifiers
a+b // three identifiers


How confusing would this become? We have some limited experience, but only
limtied, in BitC. BitC allowed operator definitions to use conventional
identifiers in the way I sketched above (actually, we did full-up mixfix,
but that's another topic), and it worked very well. BitC did *not* allow
operators to be used as general-purpose identifiers, but in hindsight I
believe that we probably should have done so.

Keep in mind that this is exactly the same "think about white space" issue
that we already know from conventional identifiers.

From a "but would this be too weird?" standpoint, all of the *current*
minglings
of identifiers without white space would be preserved, so "a+b" would
continue to behave like you expect. But just like

a.b__and c
a.b __and c


mean two very different things in C++, it would now be true that

a.!&  // ident dot ident ident
a.! & // ident dot ident ident ident


would mean different things.


I don't know if I'm being helpful or just confusing the issue further, but
I hope this helps people think about this stuff better.


Jonathan

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Erica Sadun via swift-evolution]

> On Oct 19, 2016, at 11:17 PM, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
> On Wed, Oct 19, 2016 at 1:31 PM, David Waite via swift-evolution 
> > wrote:
> The problem is that this set does not just contain mathematical operators, 
> but includes among other examples  \u2205 (Empty Set) and \u221E (infinity).
> 
> Both of which are perfectly reasonable symbols to include.
> 
> From a UAX31 standpoint, the practical problem is that operator symbols are 
> going to get defined largely in terms of the existing symbol category. It's 
> not going to be perfect. Traditionally, Unicode standards have been defined 
> in terms of properties rather than blocks. I do think its worth asking 
> whether "mathematical symbols" is too broad and we may wish to consider only 
> "mathematical operators". I'll take that up with Mark.
> 
> This is one reason that I was briefly exploring whether operator identifiers 
> could actually be used as identifiers generally. The answer boils down to: 
> "not if operator symbols admit . (period)". Unfortunately, the existing Swift 
> standard library is already using .

I really liked Jonathan's suggestion that removed the distinction between 
operators and identifiers entirely. You could mark a one-argument function as 
postfix or prefix, and a two-argument function as infix and use them as a kind 
of pseudo keyword. 

-- E


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Matthew Johnson via swift-evolution]

> On Oct 20, 2016, at 9:29 AM, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
> Quick poll as a sanity check on a possible alternative for operators:
> 
> If we admitted [:Sm:] and [:So:] and the traditional ASCII operator 
> characters, would that cover the things that people currently feel passionate 
> about? That would almost certainly be compliant with UAX31 once it settles, 
> and I think it covers all of the cases people have raised here.
> 
> Useful links if you want to check:
> 
> [:Sm:]  Symbol, Math 
> 
> [:So:]   Symbol, Other 
> 
> 
> Having looked it over, I'm concerned about including [:Sk:] in UAX31 
> operators, and I'm probably going to recommend in the UAX31 discussion that 
> we shouldn't do so.

On a quick glance, I think this would be acceptable to me.

> 
> 
> Jonathan
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Thu, Oct 20, 2016 at 7:30 AM, David Sweeris  wrote:

> Sent from my iPhone
>
> On Oct 20, 2016, at 09:03, Jonathan S. Shapiro via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> On Thu, Oct 20, 2016 at 12:12 AM, Austin Zheng via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>>
>> Freeze the set of allowed emoji to whatever the current version of the
>> Unicode spec defines...
>>
>
> UAX31 won't include emojis in either space, because there is no clear
> consensus about where they belong (identifiers or operators). Individual
> languages can certainly add them to one space or the other, but should take
> care not to cross-contaminate. So if we add them to operators, we need to
> exclude any that are already part of normal identifiers and vice versa.
> That sanity restriction is technically necessary, but it shouldn't be an
> inconvenience in practical terms.
>
>
> My understanding (which is admittedly fuzzy) is that the distinction
> between operators and identifiers is only "technically necessary" because
> allowing characters to be both causes the parsing algorithm lose its
> virtual mind, and it takes a century for it to figure out what's going on.
> What I don't recall being discussed before is whether that's a blanket
> penalty or if the compile times increases are proportional to the amount
> overlap between the two character sets.
>

The hard requirements are:

   1. Nothing in identifier start can be in operator start or operator
   continue. [*]
   2. Nothing in operator start can be in identifier start or identifier
   continue. [*]
   3. Nothing in syntactic punctuation (period, brackets, parens, and so
   forth) can be in either type of identifier without creating a lot of
   serious hair. You can see one example of hair in the "double dots" rule.

If these requirements are not preserved, the consequence is that white
space becomes required between identifiers and operators. So, for example,
without these rules:

a+b// gets broken

a + b  // works


The presence of dots in operators is actually causing a whole bunch of
constraints to get introduced that I'm going to talk about in a moment.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[David Sweeris via swift-evolution]

Sent from my iPhone

> On Oct 20, 2016, at 09:03, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
>> On Thu, Oct 20, 2016 at 12:12 AM, Austin Zheng via swift-evolution 
>>  wrote:
>> 
>> Freeze the set of allowed emoji to whatever the current version of the 
>> Unicode spec defines...
> 
> UAX31 won't include emojis in either space, because there is no clear 
> consensus about where they belong (identifiers or operators). Individual 
> languages can certainly add them to one space or the other, but should take 
> care not to cross-contaminate. So if we add them to operators, we need to 
> exclude any that are already part of normal identifiers and vice versa. That 
> sanity restriction is technically necessary, but it shouldn't be an 
> inconvenience in practical terms.

My understanding (which is admittedly fuzzy) is that the distinction between 
operators and identifiers is only "technically necessary" because allowing 
characters to be both causes the parsing algorithm lose its virtual mind, and 
it takes a century for it to figure out what's going on. What I don't recall 
being discussed before is whether that's a blanket penalty or if the compile 
times increases are proportional to the amount overlap between the two 
character sets. If it's the latter, it might be worth discussing whether we 
should allow a *small* group of characters to be legal as both identifiers or 
operators, while maintaining sub-century compile times.

- Dave Sweeris.___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

Quick poll as a sanity check on a possible alternative for operators:

If we admitted [:Sm:] and [:So:] and the traditional ASCII operator
characters, would that cover the things that people currently feel
passionate about? That would almost certainly be compliant with UAX31 once
it settles, and I *think* it covers all of the cases people have raised
here.

Useful links if you want to check:

[:Sm:]  Symbol, Math


[:So:]   Symbol, Other



Having looked it over, I'm concerned about including [:Sk:] in UAX31
operators, and I'm probably going to recommend in the UAX31 discussion that
we shouldn't do so.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Rien via swift-evolution]

Said differently: A monkey with a tool is still a monkey.
I.e. Swift cannot force somebody to become a good programmer no matter what 
rules it imposes.
As far as limiting personal freedoms goes: everybody (kid’s included) should be 
able to use whatever pleases them - within the possibilities of the language.
But the language should not impose restrictions it does not need.
If somebody out there wants to use emoticons, or whole pages of them… so what?.
Any company or programmer worth its salt has their own rules for what 
constitutes a good identifier or operator.

OTOH: I would not go as far as in optimizing the compiler to deal with anything 
non-ascii. If people want to use emoticons, and this results in sub-par 
performance in compilation or execution speed, so be it.

Rien.

> On 20 Oct 2016, at 16:03, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
> On Thu, Oct 20, 2016 at 12:12 AM, Austin Zheng via swift-evolution 
>  wrote:
> Is there a compromise we can come up with, maybe?
> 
> So speaking just for myself, I strongly oppose emojis because every example 
> of emoji code I have seen has been truly obfuscated. Emojis therefore present 
> very serious and active source-level security risks that will require 
> significant engineering investment to manage and will never be fully managed 
> successfully.
> 
> That said, I'm very glad that some people here have pointed out the "kid use 
> case", because I had not considered that one. I think that's actually pretty 
> compelling.
> 
> Let me ask a question: would single-character emoji identifiers be enough, or 
> do we need multi-character emojis? Single-character emoji identifiers would 
> go a long way toward limiting the capacity for obfuscation, but I'm guessing 
> it won't be enough for a bunch of people here.
>  
> Freeze the set of allowed emoji to whatever the current version of the 
> Unicode spec defines...
> 
> UAX31 won't include emojis in either space, because there is no clear 
> consensus about where they belong (identifiers or operators). Individual 
> languages can certainly add them to one space or the other, but should take 
> care not to cross-contaminate. So if we add them to operators, we need to 
> exclude any that are already part of normal identifiers and vice versa. That 
> sanity restriction is technically necessary, but it shouldn't be an 
> inconvenience in practical terms.
> 
> 
> Jonathan
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

I'm sympathetic to the arguments you raise here. However, they were brought
up during the SetAlgebra API discussion and rejected in favor of the
current API. Now is not the time to re-litigate that thread.

The point I'm making is that it's all right if Swift's operators happen to
support the mathematical set operators, but specifically adding them
because people disagree with the direction of Swift is a non-goal or
anti-goal.
On Thu, Oct 20, 2016 at 04:23 J.E. Schotsman via swift-evolution <
swift-evolution@swift.org> wrote:

> On 19 Oct 2016, at 19:00,Xiaodi Wu wrote:
>
> However, it's specifically the set algebra operators that I have the
> greatest objection to cherrypicking:
>
> * Each of these operators has a specific meaning; it is an anti-goal to
> support repurposing the union operator for any other purpose than forming a
> union of two sets, for instance.
>
> * Thus, the only rationale for inclusion of these operators is to support
> aliasing the set algebra protocol members.
>
>
> That’s called customization and I don’t see what’s wrong with that.
> I bet a sizeable number of developers want the set operators and they
> should be able to custom-define them.
>
> * Now, if it is appropriate for these set algebra operations to be
> accessible through these operators, then the standard library should be
> providing them.
>
> * However, exactly such an API has been trialled on a Swift branch, and the
> set algebra renaming debate of 2015 (or was it early 2016?) on this very
> list resulted in a renaming that *rejected* the use of these operators.
>
> * Given that these operators have been deemed not appropriate for the only
> methods that should use them, we should *not* specifically enable these
> symbols as valid operator characters.
>
>
> If it had been decided to provide the operators, which would it have been:
> the bitwise operator symbols or the mathematical set operation symbols?
> That dilemma alone is enough reason to leave this as a customization
> point, IMHO.
>
> The sheer length of the great Set API renaming thread itself makes clear
> this is a linguistic pain zone which is avoided entirely with the operator
> symbols which are very clear and concise.
> One reason for the difficulty to arrive at good English names for the
> methods was the matter of type method vs instance method.
> For example, x.formUnionWith(y: Element) vs Set.formUnion(of x: Element,
> and y: Element)
> Basically the non-mutating methods feel like type methods but the mutating
> ones must be instance methods.
> This largely academic difference too is avoided with the operator approach.
>
> Jan E.
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[J.E. Schotsman via swift-evolution]

> On 19 Oct 2016, at 19:00,Xiaodi Wu wrote:
> 
> However, it's specifically the set algebra operators that I have the
> greatest objection to cherrypicking:
> 
> * Each of these operators has a specific meaning; it is an anti-goal to
> support repurposing the union operator for any other purpose than forming a
> union of two sets, for instance.
> 
> * Thus, the only rationale for inclusion of these operators is to support
> aliasing the set algebra protocol members.

That’s called customization and I don’t see what’s wrong with that.
I bet a sizeable number of developers want the set operators and they should be 
able to custom-define them.

> * Now, if it is appropriate for these set algebra operations to be
> accessible through these operators, then the standard library should be
> providing them.
> 
> * However, exactly such an API has been trialled on a Swift branch, and the
> set algebra renaming debate of 2015 (or was it early 2016?) on this very
> list resulted in a renaming that *rejected* the use of these operators.
> 
> * Given that these operators have been deemed not appropriate for the only
> methods that should use them, we should *not* specifically enable these
> symbols as valid operator characters.

If it had been decided to provide the operators, which would it have been: the 
bitwise operator symbols or the mathematical set operation symbols?
That dilemma alone is enough reason to leave this as a customization point, 
IMHO.

The sheer length of the great Set API renaming thread itself makes clear this 
is a linguistic pain zone which is avoided entirely with the operator symbols 
which are very clear and concise.
One reason for the difficulty to arrive at good English names for the methods 
was the matter of type method vs instance method.
For example, x.formUnionWith(y: Element) vs Set.formUnion(of x: Element, and y: 
Element)
Basically the non-mutating methods feel like type methods but the mutating ones 
must be instance methods.
This largely academic difference too is avoided with the operator approach.

Jan E.___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[J.E. Schotsman via swift-evolution]

> On 20 Oct 2016, at 08:34,Nevin Brackett-Rozinsky wrote:
> 
> So I am strongly—adamantly—opposed to the operator-eviscerating portion of
> this proposal.
> 
> We should make Braille characters, Hangzhou numerals, the empty set and the
> infinity sign into identifiers. All other operators should remain as they
> are until official Unicode recommendations exist, at which point we should
> deprecate as necessary.

+1

Jan E.

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Austin Zheng via swift-evolution]

Is there a compromise we can come up with, maybe? Allow emoji in identifiers, 
but freeze the set of allowed emoji to whatever the current version of the 
Unicode spec defines with the intention that 'automatic expansion' of the 
allowed character set to accommodate future emoji is a non-goal? (Does Unicode 
even provide a way to express "the set of emoji characters supported by 
Specific Unicode Specification X"?)

Austin

> On Oct 20, 2016, at 12:06 AM, Xiaodi Wu via swift-evolution 
>  wrote:
> 
> Point well taken, but FWIW, there is a large difference between *you* 
> expressing whimsy and committing the language to an open-ended series of 
> continuous revisions for the sole purpose of enabling one particular form of 
> whimsy. It's rather an overstatement to say that we are proposing to "squash 
> the joy out of everything," as though we all lived our lives in states of 
> ascetic deprivation before the advent of emoji.
> 
> On Thu, Oct 20, 2016 at 14:38 Russ Bishop via swift-evolution 
> > wrote:
>> On Oct 19, 2016, at 1:46 PM, Brent Royal-Gordon via swift-evolution 
>> > wrote:
>> 
>> I was in the middle of writing about my opposition to the original proposal 
>> when I went to bed last night, and was going to advocate something like this:
>> 
>>> Given the current state of the discussion over in Unicode land, I think it 
>>> would probably be safe from a compatibility standpoint to admit code points 
>>> that fall into the following (Unicode-style) code point set:
>>> 
>>> [:S:] - [:Sc:] - [:xidcontinue:] - [:nfcqc=n:] & [:scx=Common:] - 
>>> pictographics - emoji
>> 
>> I suspect we can probably also do something about emoji, since I doubt UAX 
>> #31 is going to. Given that they are all static pictures of people or 
>> things, I think we can decide they are all nouns and thus all identifier 
>> characters. If we think there are some which might be declared operators 
>> later, we can exclude them for now, but I'd like to at least see the bulk of 
>> them brought in.
>> 
>> I think addressing emoji is important not for any technical reason, but for 
>> nontechnical ones. Emoji are a statement about Swift's modern approach; 
>> modernity is important. They are fun and whimsical; whimsy is important.
>> 
>> And most importantly, emoji identifiers are part of Swift's culture. It's 
>> widely understood that you don't use them in real code, but they are very 
>> common in examples. Just as we worry about source compatibility and binary 
>> compatibility, so we should worry about culture compatibility. Removing 
>> emoji would cause a gratuitous cultural regression.
>> 
> 
> I fully agree. It’s hella presumptuous to decide that I’m not allowed to 
> express whimsy, frustration, humor, or any other emotions in my code. Or to 
> tell an 8 year old using Playgrounds on the iPad that he/she can’t name a 
> variable  purely because they find it funny. We don’t have to squash the joy 
> out of everything.
> 
> 
> Russ
> ___
> swift-evolution mailing list
> swift-evolution@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-evolution 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

Point well taken, but FWIW, there is a large difference between *you*
expressing whimsy and committing the language to an open-ended series of
continuous revisions for the sole purpose of enabling one particular form
of whimsy. It's rather an overstatement to say that we are proposing to
"squash the joy out of everything," as though we all lived our lives in
states of ascetic deprivation before the advent of emoji.

On Thu, Oct 20, 2016 at 14:38 Russ Bishop via swift-evolution <
swift-evolution@swift.org> wrote:

> On Oct 19, 2016, at 1:46 PM, Brent Royal-Gordon via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> I was in the middle of writing about my opposition to the original
> proposal when I went to bed last night, and was going to advocate something
> like this:
>
> Given the current state of the discussion over in Unicode land, I think it
> would probably be safe from a compatibility standpoint to admit code points
> that fall into the following (Unicode-style) code point set:
>
> [:S:] - [:Sc:] - [:xidcontinue:] - [:nfcqc=n:] & [:scx=Common:] -
> pictographics - emoji
>
>
> I suspect we can probably also do something about emoji, since I doubt UAX
> #31 is going to. Given that they are all static pictures of people or
> things, I think we can decide they are all nouns and thus all identifier
> characters. If we think there are some which might be declared operators
> later, we can exclude them for now, but I'd like to at least see the bulk
> of them brought in.
>
> I think addressing emoji is important not for any technical reason, but
> for nontechnical ones. Emoji are a statement about Swift's modern approach;
> modernity is important. They are fun and whimsical; whimsy is important.
>
> And most importantly, emoji identifiers are part of Swift's culture. It's
> widely understood that you don't use them in real code, but they are very
> common in examples. Just as we worry about source compatibility and binary
> compatibility, so we should worry about culture compatibility. Removing
> emoji would cause a gratuitous cultural regression.
>
>
> I fully agree. It’s hella presumptuous to decide that I’m not allowed to
> express whimsy, frustration, humor, or any other emotions in my code. Or to
> tell an 8 year old using Playgrounds on the iPad that he/she can’t name a
> variable  purely because they find it *funny*. We don’t have to squash
> the joy out of *everything*.
>
>
> Russ
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jean-Daniel via swift-evolution]

> Le 20 oct. 2016 à 02:11, Xiaodi Wu via swift-evolution 
>  a écrit :
> 
> I actually take the opposite view of emoji, and I was convinced of this by 
> arguments from some of the other authors (though they may not come to the 
> same conclusions as I do):
> 
> The real and very weighty reason Swift should support Unicode identifiers is 
> that naming things is hard, and it is serious, and we should be adamant about 
> this one thing:
> 
> Even if your primary language is not English, and even if your use of Swift 
> takes you beyond emulating the narrow set of standard library and Foundation 
> API names, you can still take all the care and attention in naming things 
> that we would want to promote in Swift by using your own primary language. We 
> want this to be the case wherever you were born, whatever language your 
> mother taught you, and we want to support this on principle, whether or not 
> we can find empiric evidence of open-source projects on GitHub that make use 
> of any particular language which we know to be used in the world.
> 
> Previously, as we tackled this Unicode problem, a not-illegitimate critique 
> was that Swift's support of Unicode identifiers appeared to be frivolous, 
> because the only examples given in documentation are of emoji, and as you 
> say, it is there to be cute or whimsical. This appearance undermines that 
> very serious idea described above.

And removing emoji remove the possibility to write simple sample code using an 
universal language that is understandable whatever language your mother taught 
you. If you want to have a car variable or a dog variable, just use the emoji 
and everybody can read it without hesitation.

> 
> UAX#31 makes room for the removal of obsolete scripts such as Egyptian 
> hieroglyphics from the range of valid identifier characters on the basis (at 
> least in my reading of the document) that it adds to the burden of a 
> programming language without serving the weighty purpose of expressing 
> themselves in their primary language. By analogy, emoji similarly do not 
> serve that purpose, and since their parsing changes with every Unicode 
> update, we would be making changes to Swift every minor release for the 
> purpose of chasing a modish whimsy.
> 
> 
> On Thu, Oct 20, 2016 at 04:46 Brent Royal-Gordon via swift-evolution 
> > wrote:
> I was in the middle of writing about my opposition to the original proposal 
> when I went to bed last night, and was going to advocate something like this:
> 
> > Given the current state of the discussion over in Unicode land, I think it 
> > would probably be safe from a compatibility standpoint to admit code points 
> > that fall into the following (Unicode-style) code point set:
> >
> > [:S:] - [:Sc:] - [:xidcontinue:] - [:nfcqc=n:] & [:scx=Common:] - 
> > pictographics - emoji
> 
> I suspect we can probably also do something about emoji, since I doubt UAX 
> #31 is going to. Given that they are all static pictures of people or things, 
> I think we can decide they are all nouns and thus all identifier characters. 
> If we think there are some which might be declared operators later, we can 
> exclude them for now, but I'd like to at least see the bulk of them brought 
> in.
> 
> I think addressing emoji is important not for any technical reason, but for 
> nontechnical ones. Emoji are a statement about Swift's modern approach; 
> modernity is important. They are fun and whimsical; whimsy is important.
> 
> And most importantly, emoji identifiers are part of Swift's culture. It's 
> widely understood that you don't use them in real code, but they are very 
> common in examples. Just as we worry about source compatibility and binary 
> compatibility, so we should worry about culture compatibility. Removing emoji 
> would cause a gratuitous cultural regression.
> 
> --
> Brent Royal-Gordon
> Architechies
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-evolution 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Russ Bishop via swift-evolution]

> On Oct 19, 2016, at 1:46 PM, Brent Royal-Gordon via swift-evolution 
>  wrote:
> 
> I was in the middle of writing about my opposition to the original proposal 
> when I went to bed last night, and was going to advocate something like this:
> 
>> Given the current state of the discussion over in Unicode land, I think it 
>> would probably be safe from a compatibility standpoint to admit code points 
>> that fall into the following (Unicode-style) code point set:
>> 
>> [:S:] - [:Sc:] - [:xidcontinue:] - [:nfcqc=n:] & [:scx=Common:] - 
>> pictographics - emoji
> 
> I suspect we can probably also do something about emoji, since I doubt UAX 
> #31 is going to. Given that they are all static pictures of people or things, 
> I think we can decide they are all nouns and thus all identifier characters. 
> If we think there are some which might be declared operators later, we can 
> exclude them for now, but I'd like to at least see the bulk of them brought 
> in.
> 
> I think addressing emoji is important not for any technical reason, but for 
> nontechnical ones. Emoji are a statement about Swift's modern approach; 
> modernity is important. They are fun and whimsical; whimsy is important.
> 
> And most importantly, emoji identifiers are part of Swift's culture. It's 
> widely understood that you don't use them in real code, but they are very 
> common in examples. Just as we worry about source compatibility and binary 
> compatibility, so we should worry about culture compatibility. Removing emoji 
> would cause a gratuitous cultural regression.
> 

I fully agree. It’s hella presumptuous to decide that I’m not allowed to 
express whimsy, frustration, humor, or any other emotions in my code. Or to 
tell an 8 year old using Playgrounds on the iPad that he/she can’t name a 
variable  purely because they find it funny. We don’t have to squash the joy 
out of everything.


Russ

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Russ Bishop via swift-evolution]

Strong -1 from me as currently written.

There is no reason to remove Emoji from identifiers, nor to restrict operators 
to ASCII only (especially since the corresponding UAX spec is still under 
construction). Emoji are just as much a part of modern communication as the 
Latin alphabet. Swift should not seek to restrict a user’s ability to express 
themselves.

Given the problems with operators restricting Emoji from operators seems 
reasonable. Prohibiting non-printing characters also makes sense.


Russ


> On Oct 18, 2016, at 11:34 PM, Jacob Bandes-Storch via swift-evolution 
>  wrote:
> 
> Refining Identifier and Operator Symbology
> 
> Proposal: SE- 
> 
> Authors: Jacob Bandes-Storch , Erica Sadun 
> , Xiaodi Wu , Jonathan 
> Shapiro
> Review Manager: TBD
> Status: Awaiting review
>  
> Introduction
> 
> This proposal seeks to refine and rationalize Swift's identifier and operator 
> symbology. Specifically, this proposal:
> 
> adopts the Unicode recommendation for identifier characters, with some minor 
> exceptions;
> restricts the legal operator set to the current ASCII operator characters;
> changes where dots may appear in operators; and
> disallows Emoji from identifiers and operators.
>  
> 
> Proposed solution
> 
> For identifiers, adopt the recommendations made in UAX #31 Identifier and 
> Pattern Syntax , deriving the sets of valid 
> characters from ID_Start and ID_Continue. Normalize identifiers using 
> Normalization Form C (NFC).
> 
> (For operators, no such recommendation currently exists, although active work 
> is in progress to update UAX #31 to address "operator identifiers".)
> 
> Restrict operators to those ASCII characters which are currently operators. 
> All other operator characters are removed from the language.
> 
> Allow dots in operators in any location, but only in runs of two or more.
> 
> (Overall, this proposal is aggressive in its removal of problematic 
> characters. We are not attempting to prevent the addition or re-addition of 
> characters in the future, but by paring the set down now, we require any 
> future changes to pass the high bar of the Swift Evolution process.)
> 
>  
> Detailed
>  design
> 
>  
> Identifiers
> 
> Swift identifier characters will conform to UAX #31 
>  as follows:
> 
> UAX31-C1.  The conformance described 
> herein refers to the Unicode 9.0.0 version of UAX #31 (dated 2016-05-31 and 
> retrieved 2016-10-09).
> 
> UAX31-C2.  Swift shall observe the 
> following requirements:
> 
> UAX31-R1.  Swift shall augment the 
> definition of "Default Identifiers" with the following profiles:
> 
> ID_Start and ID_Continue shall be used for Start and Continue (replacing 
> XID_Start and XID_Continue). This excludes characters in Other_ID_Start and 
> Other_ID_Continue.
> 
> _ 005F LOW LINE shall additionally be allowed as a Start character.
> 
> The emoji characters  1F436 DOG FACE and  1F42E COW FACE shall be allowed 
> as Start and Continue characters.
> 
> (UAX31-R1a. ) The join-control 
> characters ZWJ and ZWNJ are strictly limited to the special cases A1, A2, and 
> B described in UAX #31. (This requirement is covered in the Normalize Unicode 
> Identifiers proposal .)
> 
> UAX31-R4.  Swift shall consider two 
> identifiers equivalent when they have the same normalized form under NFC 
> . (This requirement is covered in the 
> Normalize Unicode Identifiers proposal 
> .)
> 
> These changes 
> 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan Hull via swift-evolution]

If someone wants to write their variable names in Spanish or Japanese, they 
should be able to.  That is one of the main reasons for including unicode.  
(Note it might make it more difficult for me to read, but it makes it easier 
for them and their team).

If a group of mathematicians find it useful to write code that looks like the 
language they use in their everyday life, why do we need to stop them?  It 
doesn’t have to be useful to everyone to be incredibly useful (and 
communicative) for them.

And it’s not just math. I was able to use custom operators in Swift to mimic 
the standard PEG grammar (with only a few small differences), and it is 
incredibly useful.  The ability to transcribe a grammar directly from a 
research paper without having to translate to another (much less clear) 
language is enormously beneficial to me.  I am able to see the structure 
directly instead of having to “read” it from code (and translate back to my 
working language) the way you have to in most languages. That results in being 
able to put together much more complicated and full featured systems and still 
be able to understand them better at a glance.  That is actually one of the 
main reasons I have been using Swift over Objective C.


> It's more practical to make breaking changes now and introduce the "right 
> set" (that is, a standards-based set of mathematical operators) at a future 
> date, than to justify keeping things as is and removing operators at a future 
> date.
Define practical. How is breaking shipping code to solve a theoretical problem 
practical?  I have yet to see any evidence that the current system is actually 
causing ANY real world problems.

You can break EVERYTHING now, or break only a small subset of that (which is 
the part which most people probably aren’t using anyway) when you know the 
standards for certain. I don’t understand the mindset that breaking more is 
better.  This is one of those cases, where if the eventual standard breaks too 
many things, then you have chosen your standard incorrectly, because it doesn’t 
reflect reality.  The market will speak about what is useful.

Why not say: “These are definitely going to be identifiers. These are 
definitely going to be operators.  Everything else is currently undefined and 
will stay where it is, but may change in the future to match X spec”.  I don’t 
see how that is any more harmful to the growth of Swift, and it is certainly 
less harmful to the rest of us while we wait.

> Beginning with Swift 4, there will be a major push to ensure that backwards
> compatibility with existing code is not broken. It will be possible to
> expand the operator character set, but very difficult to shrink it.
IMO that point has been crossed already. Swift 3 was for all the breaking 
changes, and now for Swift 4 there is a much higher barrier. We are past the 
point of breaking everything for the sake of breaking them because adding is 
easier.

Honestly, there are a lot of helpful improvements in Swift 3, but I would be 
hard pressed to find any of the panicked “we have to break it now or we won’t 
be able to later” changes that were made among them.  Those things are just 
broken, waiting for someone to fix them in an undefined future time.  Given 
that I am still waiting for Apple to fix bugs from Panther, I am not holding my 
breath.

> Personally, a *very* limited cherrypicking might be OK, on the
> understanding that it must be done with very stringent inclusion criteria.
> However, it's specifically the set algebra operators that I have the
> greatest objection to cherrypicking:
> 
> * Each of these operators has a specific meaning; it is an anti-goal to
> support repurposing the union operator for any other purpose than forming a
> union of two sets, for instance.
There are many different branches of math which use symbols in different ways.  
Higher order logic, for instance, uses the union symbol as an operation (which 
is not the union of two sets). I have an old book on computer chip design which 
uses the union symbol as a way to optimally place logic gates.   

I wrote a program (just for myself) that helped me solve a math problem in 
graph theory that I had been working on for 10 years.  10 years… and the 
program helped me solve it in a matter of days.  If I find that symbol useful 
for my computations, but it isn’t the union of two sets, why is that a problem?

There are also many other fields besides math which use symbols for short hand. 
 The ‘+’ symbol is used for addition, but it is also used for concatenation of 
Strings.

My point is that as much as you think you know my goals or context, you don’t.  
It is an anti-goal for us to be the language style police… restricting people’s 
expression because we don’t understand their context, and limiting them to 
problem spaces which we find interesting. In my experience, that sort of thing 
is the root of all evil (and the root of bad UI/UX).

I used to work on a language 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Wed, Oct 19, 2016 at 10:26 PM, Nevin Brackett-Rozinsky via
swift-evolution  wrote:

> Thinking about it further, I am not convinced we need to make *any* change
> to the set of operator characters at this time. It’s not like people are
> clamoring to have Braille variable names after all. And as much as I’d like
> to see the upside-down ampersand (⅋) as an operator, that too can wait.
>

Unfortunately we *do* need to make changes. At the very least, the current
definition of operators includes completely undefined codepoints. That's
just not OK. There are also *many* elements that are unlikely to be
incorporated in UAX31, and we want to be careful about backwards
compatibility issues and also cross-language interop issues. Including too
much risks incompatibility with future evolutions of UAX31 that other
languages are likely to adopt as a gold standard of interop.

I am hopeful that this proposal will be revised to focus solely on adopting
> UAX-31.
>

That's definitely the goal, but we don't yet have a draft "operator
identifier" proposal in UAX31 to adopt. I think Xiaodi's goal here was to
arrive at a subset that would be future proof.

I've put in an email to the proposing group, and I expect there will be a
response, but I don't want to speak as if I'm representing the consensus
until I hear back from them. Can I ask everyone to engage patience on this
issue until some time tomorrow?

You are all very definitely being heard!


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Nevin Brackett-Rozinsky via swift-evolution]

Thinking about it further, I am not convinced we need to make *any* change
to the set of operator characters at this time. It’s not like people are
clamoring to have Braille variable names after all. And as much as I’d like
to see the upside-down ampersand (⅋) as an operator, that too can wait.

I am hopeful that this proposal will be revised to focus solely on adopting
UAX-31. I am not yet familiar with the specifics of that document, and I
expect I am not alone in that regard. Since the proposal indicates several
thousand characters will no longer be valid in identifiers, it seems quite
possible that some of them may be controversial.

I think it is far more productive to spend our collective efforts on making
sure we get identifiers right for Swift 4. We can deal with operators in a
similar manner once official Unicode guidelines are put forth, so we should
not spend time on them now.

Nevin



On Thu, Oct 20, 2016 at 12:44 AM, Jacob Bandes-Storch via swift-evolution <
swift-evolution@swift.org> wrote:

>
> On Wed, Oct 19, 2016 at 10:12 AM, Alex Martini  wrote:
>
>> Grammar changes
>>
>> operator → operator-head operator-characters[opt]
>>
>> operator-head → ! % & * + - / < = > ? ^ | ~
>> operator-head → operator-dot operator-dots
>> operator-character → operator-head
>> operator-characters → operator-character operator-character[opt]
>>
>> operator-dot → .
>> operator-dots → operator-dot operator-dots[opt]
>>
>>
>> 
>>
>>
>> I think there's a mismatch between the English and grammar.  For example,
>> is +..+ allowed or not?
>>
>> The English rule does allow +..+ because its dots appear in a run of two.
>>
>> The grammar allows a run of one or more dots as an operator head, but
>> never allows dots as characters appearing in the middle of an operator,
>> regardless of how many dots appear next to each other.  The grammar
>> wouldn't allow +..+ because the dots don't come at the beginning.
>>
>>
>>
>> Here's an alternate version of the grammar that matches the "two or more"
>> rule.  Because we no longer distinguish between which characters are
>> allowed as the first character of an operator vs a character inside,
>> there's no longer a need for a separate operator-head.
>>
>> operator --> operator-character operator-OPT
>>
>> operator-character --> ! % & * + - / < = > ? ^ | ~
>> operator-character --> operator-dots
>>
>> operator-dots --> .. operator-additional-dots-OPT
>> operator-additional-dots --> . operator-additional-dots-OPT
>>
>
> There is a typo in that operator-character[opt] should be
> operator-characters[opt]. Aside from that, though, I believe the grammar as
> written accepts +..+ already. Take a look at the following series of
> substitutions based on the grammar rules:
>
> operator
> operator-head operator-characters
> + operator-characters
> + operator-character operator-characters[opt]
> + operator-head operator-head
> + operator-dot operator-dots operator-head
> + . . +
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan Hull via swift-evolution]

An EXTREME -1 from me as well.  The restrictions on operators is FAR too 
severe.  At the very least, I need the upside-down question mark and 
exclamation points ¿ ¡, as well as the ≤,≥,≠ operators.  I mean, you are 
getting rid of the division sign ÷.  How could that ever be controversial as an 
operator?

This is the first proposal I have seen, where if it goes through, I will 
actually have to stop using Swift (or find a way not to upgrade)!

Seriously, this proposal would break the vast majority of my code.  One of the 
first things I wrote for Swift was a packrat parser which lets me define 
rulesets that look like standard PEG using swift operators.  I have now written 
several different projects on top of that.  I also have a private extension for 
≤,≥,≠ that I use in my projects.

What happened to needing extra justification for breaking changes? We can add 
them back later, you say. This seems to be breaking things just for the fun of 
it then…. If you are going to need to break things later to become compatible 
with something, then break them later.  Don’t break my production code now to 
avoid a 1% chance you might have to break my production code in the future.

> [begin quote]
> 
> • Operators suffer from low discoverability and difficult readability. They
> use symbols, not names. This places a cognitive cost on users with respect
> to both recall ("What is the operator that applies the behavior I need?")
> and recognition ("What does the operator in this code do?").
> • This cost is obviously highest when symbols are not tied to conventional
> standards like `∪` for union and `⊇` for superset. `∪` is a standard,
> mathematical representation. It’s widely accepted and widely used. Even so,
>  recognizing `formUnion(with:)` may work better for many coders than
> recalling what the `∪` (or, worse, `⊇`) operator does, even when you end up
> having to create suites of specialized selectors. As operators become more
> self-defined or esoteric, costs rise.
> 
> [end quote]
With all due respect, my day job is in the cognitive sciences (I have a 
master’s degree in Cognitive Psychology) and this is pure bullshit.  The entire 
reason we use symbols is that the opposite is true.  Yes, there is temporarily 
a greater burden on people who are new to a symbol, but once they learn it, it 
has much faster recall and recognition. That is why we use icons all over the 
place. It is why mathematicians use symbols.  It is why you see all those 
symbols on signs at the airport.

There are a bunch of symbols in unicode which are hard to tell apart, and those 
are bad for recognition, and we should deal with that, but this proposal is 
throwing the baby out with the bathwater, then lighting the baby on fire.  
Honestly, I would propose we find a way to have Swift see certain classes of 
characters as identical.  Can’t decide which of the thousand + symbols should 
be the one true + symbol?  Have them all map to one of them.  That emoji X 
symbol looks like X, so map it to X.

This proposal is lazy and unreasonable.  I think we can all agree that there 
needs to be thought around confusable symbols… but most of us just aren’t using 
the confusable symbols.  The argument seems to be “Let’s break EVERYTHING now 
so we don’t have to break it later”, which is pure ridiculousness.  Don’t break 
anything now… and then when you are ready to deal with confusables (which, 
again, most people aren’t actually using) do it surgically then.  You will be 
breaking a strictly smaller amount of code…




___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Nevin Brackett-Rozinsky via swift-evolution]

I strongly oppose the proposed mass-removal of operator characters. It
would be a major loss to the Swift language if that were to occur even
temporarily.

The long-term goal is for Swift to adopt the official Unicode guidance for
operator characters, which is still under development. Therefore I believe
we should make only minor and obvious changes right now, because there is
no sense in “jumping the gun” and causing unnecessary source-breaking
changes.

In particular, we should make it clear that Swift will most likely adopt
the Unicode operator conventions when they become available, so people are
aware and prepared.

When the time comes, we should deprecate any operator characters that
Unicode recommends against (unless we have a good reason not to), before
removing them in the next major release. The deprecation period ensures
that source-breaking changes result in a warning at first, so developers
have time to adapt.

I just went through all the valid operator characters in Swift, and the
only ones I would recommend to eliminate at this time are:
U+2800 – U+28FF (Braille patterns)
U+3021 – U+3029 (Hangzhou numerals)
U+2205 and U+221E (Empty set and Infinity)

Additionally, I propose to *add* one operator that is missing:
U+214B (Turned ampersand)

• • •

As for the rest of the proposal, I suppose normalizing identifiers and
dealing with confusable characters in a sensible way.

Regarding emoji, I look at them rather like the “I’m feeling lucky” button
on Google—essentially nobody uses it, but when they tried getting rid of it
people didn’t like the result. So I agree with Brent about that we should
keep them for cultural, not technical, reasons.

• • •

Returning to the discussion of operators, I am reminded of what happened
when we eliminated argument labels from functions passed as parameters. The
intent was and still is to reinstate them in a more robust manner.

However, during the interim the result has been a regression that goes
against the core tenets and philosophy of Swift. I would not want to repeat
that while waiting for official Unicode operator guidelines.

So I am strongly—adamantly—opposed to the operator-eviscerating portion of
this proposal.

We should make Braille characters, Hangzhou numerals, the empty set and the
infinity sign into identifiers. All other operators should remain as they
are until official Unicode recommendations exist, at which point we should
deprecate as necessary.

Nevin



On Wed, Oct 19, 2016 at 5:53 PM, Matthew Johnson via swift-evolution <
swift-evolution@swift.org> wrote:

>
>
> IMO, the best argument against using unicode symbols for operators defined
>> by mathematics is that they are currently difficult to type.
>>
>
> And there is no realistic hope of that changing. This issue is so
> compelling that C and C++ introduced standardized text-ascii alternatives
> for the punctuation operators to relieve stress on non-english keyboard
> users.
>
>
> I don’t agree that there is no realistic hope of that changing.  It
> appears to be pretty reasonable to anticipate that we’ll all be using
> software-driven keyboards that can display software-defined symbols on the
> keys in the relatively near future (probably 5 years, certainly 10).  All
> kinds of interesting things become possible when that happens, including
> the ability to make unicode operators much easier to discover and type in a
> programmer’s editor.
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

I actually take the opposite view of emoji, and I was convinced of this by
arguments from some of the other authors (though they may not come to the
same conclusions as I do):

The real and very weighty reason Swift should support Unicode identifiers
is that naming things is hard, and it is serious, and we should be adamant
about this one thing:

Even if your primary language is not English, and even if your use of Swift
takes you beyond emulating the narrow set of standard library and
Foundation API names, you can still take all the care and attention in
naming things that we would want to promote in Swift by using your own
primary language. We want this to be the case wherever you were born,
whatever language your mother taught you, and we want to support this on
principle, whether or not we can find empiric evidence of open-source
projects on GitHub that make use of any particular language which we know
to be used in the world.

Previously, as we tackled this Unicode problem, a not-illegitimate critique
was that Swift's support of Unicode identifiers appeared to be frivolous,
because the only examples given in documentation are of emoji, and as you
say, it is there to be cute or whimsical. This appearance undermines that
very serious idea described above.

UAX#31 makes room for the removal of obsolete scripts such as Egyptian
hieroglyphics from the range of valid identifier characters on the basis
(at least in my reading of the document) that it adds to the burden of a
programming language without serving the weighty purpose of expressing
themselves in their primary language. By analogy, emoji similarly do not
serve that purpose, and since their parsing changes with every Unicode
update, we would be making changes to Swift every minor release for the
purpose of chasing a modish whimsy.


On Thu, Oct 20, 2016 at 04:46 Brent Royal-Gordon via swift-evolution <
swift-evolution@swift.org> wrote:

I was in the middle of writing about my opposition to the original proposal
when I went to bed last night, and was going to advocate something like
this:

> Given the current state of the discussion over in Unicode land, I think
it would probably be safe from a compatibility standpoint to admit code
points that fall into the following (Unicode-style) code point set:
>
> [:S:] - [:Sc:] - [:xidcontinue:] - [:nfcqc=n:] & [:scx=Common:] -
pictographics - emoji

I suspect we can probably also do something about emoji, since I doubt UAX
#31 is going to. Given that they are all static pictures of people or
things, I think we can decide they are all nouns and thus all identifier
characters. If we think there are some which might be declared operators
later, we can exclude them for now, but I'd like to at least see the bulk
of them brought in.

I think addressing emoji is important not for any technical reason, but for
nontechnical ones. Emoji are a statement about Swift's modern approach;
modernity is important. They are fun and whimsical; whimsy is important.

And most importantly, emoji identifiers are part of Swift's culture. It's
widely understood that you don't use them in real code, but they are very
common in examples. Just as we worry about source compatibility and binary
compatibility, so we should worry about culture compatibility. Removing
emoji would cause a gratuitous cultural regression.

--
Brent Royal-Gordon
Architechies

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Matthew Johnson via swift-evolution]

> 
> 
> IMO, the best argument against using unicode symbols for operators defined by 
> mathematics is that they are currently difficult to type.
> 
> And there is no realistic hope of that changing. This issue is so compelling 
> that C and C++ introduced standardized text-ascii alternatives for the 
> punctuation operators to relieve stress on non-english keyboard users.

I don’t agree that there is no realistic hope of that changing.  It appears to 
be pretty reasonable to anticipate that we’ll all be using software-driven 
keyboards that can display software-defined symbols on the keys in the 
relatively near future (probably 5 years, certainly 10).  All kinds of 
interesting things become possible when that happens, including the ability to 
make unicode operators much easier to discover and type in a programmer’s 
editor.

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Brent Royal-Gordon via swift-evolution]

I was in the middle of writing about my opposition to the original proposal 
when I went to bed last night, and was going to advocate something like this:

> Given the current state of the discussion over in Unicode land, I think it 
> would probably be safe from a compatibility standpoint to admit code points 
> that fall into the following (Unicode-style) code point set:
> 
> [:S:] - [:Sc:] - [:xidcontinue:] - [:nfcqc=n:] & [:scx=Common:] - 
> pictographics - emoji

I suspect we can probably also do something about emoji, since I doubt UAX #31 
is going to. Given that they are all static pictures of people or things, I 
think we can decide they are all nouns and thus all identifier characters. If 
we think there are some which might be declared operators later, we can exclude 
them for now, but I'd like to at least see the bulk of them brought in.

I think addressing emoji is important not for any technical reason, but for 
nontechnical ones. Emoji are a statement about Swift's modern approach; 
modernity is important. They are fun and whimsical; whimsy is important.

And most importantly, emoji identifiers are part of Swift's culture. It's 
widely understood that you don't use them in real code, but they are very 
common in examples. Just as we worry about source compatibility and binary 
compatibility, so we should worry about culture compatibility. Removing emoji 
would cause a gratuitous cultural regression.

-- 
Brent Royal-Gordon
Architechies

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[David Sweeris via swift-evolution]

> On Oct 19, 2016, at 12:29 PM, Jonathan S. Shapiro via swift-evolution 
>  wrote:
> 
> On Wed, Oct 19, 2016 at 6:41 AM, Matthew Johnson via swift-evolution 
> > wrote:
> IMO, the best argument against using unicode symbols for operators defined by 
> mathematics is that they are currently difficult to type.
> 
> And there is no realistic hope of that changing. This issue is so compelling 
> that C and C++ introduced standardized text-ascii alternatives for the 
> punctuation operators to relieve stress on non-english keyboard users.

Wait, what? They’re only hard to type because people don’t seem to realize they 
can make their own keyboard layouts to use while they’re waiting for the USB 
Consortium to notice that it’s not the '80s anymore and update the class driver 
spec to allow keyboards to directly type unicode characters.
For macOS, I use Ukelele 
(http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi=ukelele). I don’t 
know what tools there are for Windows or Linux, but I’d be *shocked* if they 
didn’t exist.

- Dave Sweeris___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Joe Groff via swift-evolution]

> On Oct 18, 2016, at 11:34 PM, Jacob Bandes-Storch via swift-evolution 
>  wrote:
> 
> However, Swift's current identifier and operator character sets do not 
> conform to any Unicode standards, nor have they been rationalized in the 
> language or compiler documentation.

This isn't entirely true. Swift's current identifier set derives from the C 
working group WG14's proposal N1518, "Recommendations for extended identifier 
characters for C and C++":

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3146.html 


which unfortunately isn't called out anywhere in the compiler docs except this 
old language reference:

https://github.com/apple/swift/blob/master/docs/archive/LangRefNew.rst#identifier-tokens
 


-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[David Waite via swift-evolution]

The problem is that this set does not just contain mathematical operators, but 
includes among other examples  \u2205 (Empty Set) and \u221E (infinity).

-DW

> On Oct 19, 2016, at 1:49 PM, Paul Cantrell via swift-evolution 
>  wrote:

> At the very least, Swift ought to support operators using symbols from the 
> Unicode blocks called “Mathematical Operators” and “Supplemental Mathematical 
> Operators.”
> https://en.wikipedia.org/wiki/Mathematical_operators_and_symbols_in_Unicode 
> 
> 
> It’s right there in the name!™

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Paul Cantrell via swift-evolution]

> On Oct 19, 2016, at 12:21 PM, Rob Mayoff via swift-evolution 
>  wrote:
> 
> On Wed, Oct 19, 2016 at 10:47 AM, plx via swift-evolution 
> > wrote:
> In any case, I’d specifically hate to lose these:
> 
> - approximate equality: ≈
> - set operations: ∩, ∪ 
> - set relations: ⊂, ⊃, ⊄, ⊅, ⊆, ⊇, ⊈, ⊉, ⊊, ⊋
> - set membership: ∌, ∋, ∈, ∉
> - logical operators: ¬, ∧, ∨
> 
> I'd add ≤ ≥ ≠ to that set. 

I have production code that uses ± and ≈.

As with all operator overloads, there is room for abuse — but I’ve been using 
these operators to mean what it looks like they should mean, and am very 
pleased with the readability benefits.

At the very least, Swift ought to support operators using symbols from the 
Unicode blocks called “Mathematical Operators” and “Supplemental Mathematical 
Operators.”
https://en.wikipedia.org/wiki/Mathematical_operators_and_symbols_in_Unicode 


It’s right there in the name!™

Cheers, P



___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Matthew Johnson via swift-evolution]

> On Oct 19, 2016, at 12:27 PM, Erica Sadun  wrote:
> 
> 
>> On Oct 19, 2016, at 7:41 AM, Matthew Johnson via swift-evolution 
>>  wrote:
>> 
>> I very much support the proposal to rationalize our handling of identifier 
>> characters.  
>> 
>> I also support doing something similar for operator symbols.  However, I 
>> agree feedback from others that this proposal goes way to far in removing 
>> our ability to use mathematical operators.  
>> 
>> If I’m reading the proposal and discussion properly, the group has not able 
>> to reach consensus on the right criteria for operator symbols, but is 
>> hopeful that will be possible after the Unicode Consortium completes its 
>> work.  I think it would be far better to defer the changes to valid operator 
>> symbols until that time (removing only symbols which are currently treated 
>> as operators but for which the proposal suggests should be available for 
>> identifiers instead).
> 
> It's more practical to make breaking changes now and introduce the "right 
> set" (that is, a standards-based set of mathematical operators) at a future 
> date, than to justify keeping things as is and removing operators at a future 
> date.

I think that depends on who you ask.  I think I understand the argument for 
taking that approach.  I just don’t necessarily agree with it.  I haven’t seen 
a compelling enough argument that this is actually causing a problem *in 
practice* or in some way preventing the language from moving forward.  

If we can find a way to include a sizable subset of mathematical operators we 
believe will be included that goes beyond those suggested by plx I would 
support that.  I just think going all the way back to basic ascii operators is 
much to far and believe we should be able to find a better “temporary” solution 
while waiting on the Unicode Consortium.
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Wed, Oct 19, 2016 at 6:41 AM, Matthew Johnson via swift-evolution <
swift-evolution@swift.org> wrote:

> If I’m reading the proposal and discussion properly, the group has not
> able to reach consensus on the right criteria for operator symbols, but is
> hopeful that will be possible after the Unicode Consortium completes its
> work.  I think it would be far better to defer the changes to valid
> operator symbols until that time (removing only symbols which are currently
> treated as operators but for which the proposal suggests should be
> available for identifiers instead).
>

Beginning with Swift 4, there will be a major push to ensure that backwards
compatibility with existing code is not broken. It will be possible to
expand the operator character set, but very difficult to shrink it.

Given the current state of the discussion over in Unicode land, I think it
would probably be safe from a compatibility standpoint to admit code points
that fall into the following (Unicode-style) code point set:

[:S:] - [:Sc:] - [:xidcontinue:] - [:nfcqc=n:] & [:scx=Common:] -
pictographics - emoji


into operator characters. In English, this would be:

All symbols excluding currency symbols, provided they are not already in
regular identifiers, requiring that they are legal under NFC normalization
and also that they live in the Common script.

Explicitly exclude pictographics and emojis, not as a value judgment of
UAX31, but because different languages seem to be choosing to go different
ways about whether these are part of normal identifiers or operator
identifiers.

Similar rationale for currency symbols, though I personally believe those
should be operators rather than regular identifiers.


It's possible that other things will go in to UAX31, but it's very hard to
imagine that anything in the set above will end up getting excluded. In
particular, there is some inclination to add some punctuation symbols in
UAX31, but that's going to take some work to ensure that we don't make a
mess inadvertently.

As a transitional matter, I think it would be conservatively safe to add
the code points identified above. Note that it's important to exclude ASCII
code points that are currently "punctuation reserved words". In Swift this
(at least) includes:

. (period, when it does not appear [at least] two times in sequence)
; (Semicolon)
: (Full colon)
$ (Dollar sign - used in special identifiers, which I consider a flaw)
any and all brackets (for now).


IMO, the best argument against using unicode symbols for operators defined
> by mathematics is that they are currently difficult to type.
>

And there is no realistic hope of that changing. This issue is so
compelling that C and C++ introduced standardized text-ascii alternatives
for the punctuation operators to relieve stress on non-english keyboard
users.

This is an argument with a limited lifespan and should not carry more
> weight than it deserves in the design of a language positioned to be the
> language for the next 20 years.  I strongly believe that removing them,
> even temporarily, is a mistake.
>

I think it's good to be a little conservative given the fact that the issue
is more broadly "in flight". That said, I personally believe that the
current proposal has cut back too far.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Erica Sadun via swift-evolution]

> On Oct 19, 2016, at 7:41 AM, Matthew Johnson via swift-evolution 
>  wrote:
> 
> I very much support the proposal to rationalize our handling of identifier 
> characters.  
> 
> I also support doing something similar for operator symbols.  However, I 
> agree feedback from others that this proposal goes way to far in removing our 
> ability to use mathematical operators.  
> 
> If I’m reading the proposal and discussion properly, the group has not able 
> to reach consensus on the right criteria for operator symbols, but is hopeful 
> that will be possible after the Unicode Consortium completes its work.  I 
> think it would be far better to defer the changes to valid operator symbols 
> until that time (removing only symbols which are currently treated as 
> operators but for which the proposal suggests should be available for 
> identifiers instead).

It's more practical to make breaking changes now and introduce the "right set" 
(that is, a standards-based set of mathematical operators) at a future date, 
than to justify keeping things as is and removing operators at a future date.

-- E

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Wed, Oct 19, 2016 at 10:21 AM, Rob Mayoff via swift-evolution <
swift-evolution@swift.org> wrote:

> On Wed, Oct 19, 2016 at 10:47 AM, plx via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>> In any case, I’d specifically hate to lose these:
>>
>> - approximate equality: ≈
>> - set operations: ∩, ∪
>> - set relations: ⊂, ⊃, ⊄, ⊅, ⊆, ⊇, ⊈, ⊉, ⊊, ⊋
>> - set membership: ∌, ∋, ∈, ∉
>> - logical operators: ¬, ∧, ∨
>>
>
> I'd add ≤ ≥ ≠ to that set.
>

All of these seem reasonable to me. Can we please confirm that all of these
lie within

[:S:] - [:Sc]


Thanks!

Rationale will become clear in just a moment.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Wed, Oct 19, 2016 at 10:12 AM, Alex Martini via swift-evolution <
swift-evolution@swift.org> wrote:
>
>
> 
>
> I think there's a mismatch between the English and grammar.  For example,
> is +..+ allowed or not?
>

The intent is that this is allowed. Your alternative grammar captures the
intent correctly.
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Rob Mayoff via swift-evolution]

On Wed, Oct 19, 2016 at 10:47 AM, plx via swift-evolution <
swift-evolution@swift.org> wrote:

> In any case, I’d specifically hate to lose these:
>
> - approximate equality: ≈
> - set operations: ∩, ∪
> - set relations: ⊂, ⊃, ⊄, ⊅, ⊆, ⊇, ⊈, ⊉, ⊊, ⊋
> - set membership: ∌, ∋, ∈, ∉
> - logical operators: ¬, ∧, ∨
>

I'd add ≤ ≥ ≠ to that set.
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Alex Martini via swift-evolution]

> Dots in operators
> 
> The current requirements for dots in operator names are:
> 
> If an operator doesn’t begin with a dot, it can’t contain a dot elsewhere.
> This proposal changes the rule to:
> 
> Dots may only appear in operators in runs of two or more.
> Under the revised rule, ..< and ... are allowed, but <.< is not. We also 
> reserve the .. operator, permitting the compiler to use .. for a "method 
> cascade" syntax in the future, as supported by Dart 
> .
> 
> Motivations for incorporating the two-dot rule are:
> 
> It helps avoid future lexical complications arising from lone .s.
> 
> It's a conservative approach, erring towards overly restrictive. Dropping the 
> rule in future (thereby allowing single dots) may be possible.
> 
> It doesn't require special cases for existing infix dot operators in the 
> standard library, ... (closed range) and ..< (half-open range). It also 
> leaves the door open for the standard library to add analogous half-open and 
> fully-open range operators <.. and <..<.
> 
> If we fail to adopt this rule now, then future backward-compatibility 
> requirements will preclude the introduction of some potentially useful 
> language enhancements.
> 
>  
> Grammar
>  changes
> 
> operator → operator-head operator-characters[opt]
> 
> operator-head → ! % & * + - / < = > ? ^ | ~
> operator-head → operator-dot operator-dots
> operator-character → operator-head
> operator-characters → operator-character operator-character[opt]
> 
> operator-dot → .
> operator-dots → operator-dot operator-dots[opt]
>  
> I
>  think there's a mismatch between the English and grammar.  For example, is 
> +..+ allowed or not?

The English rule does allow +..+ because its dots appear in a run of two.

The grammar allows a run of one or more dots as an operator head, but never 
allows dots as characters appearing in the middle of an operator, regardless of 
how many dots appear next to each other.  The grammar wouldn't allow +..+ 
because the dots don't come at the beginning.



Here's an alternate version of the grammar that matches the "two or more" rule. 
 Because we no longer distinguish between which characters are allowed as the 
first character of an operator vs a character inside, there's no longer a need 
for a separate operator-head.

operator --> operator-character operator-OPT

operator-character --> ! % & * + - / < = > ? ^ | ~
operator-character --> operator-dots

operator-dots --> .. operator-additional-dots-OPT
operator-additional-dots --> . operator-additional-dots-OPT___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Jonathan S. Shapiro via swift-evolution]

On Wed, Oct 19, 2016 at 4:09 AM, Benjamin Spratling via swift-evolution <
swift-evolution@swift.org> wrote:

> Some extremely short-sighted points about deleting my formal operators
> that are widely recognized as operators, and that I’ve spent months adding
> into my code.  Frankly, I just couldn’t upgrade until you put them back in.
>

Benjamin:

The situation "behind the scenes" is that I've been working with Mark Davis
to add Unicode standard properties for operator start and operator continue
character sets in Unicode UAX31. That's a process whose scope needs to be
broader than just Swift, and it's something that Swift will want to be
compatible with. I think the intention would be to adopt that new part of
UAX31 as soon as practical, and I am hopeful that specification will meet
your needs and objectives. If not, I'd very much like to pick up that
conversation with you offline to see how we can improve matters in UAX31.

The UAX31 discussion seems to be converging rapidly. The proposal here is
to *temporarily* limit operator identifiers to the ASCII operator
characters. This is mainly intended to provide a bridge solution until
UAX31 changes can be published in draft form. One reason to take a
temporary step back is to ensure that we do not unintentionally specify
something now that will become incompatible as soon as the UAX31 draft
emerges.

Changes to the operator identifier space are well-localized in the compiler
implementation, and don't have any large-scale impact on later passes. They
are one of the few kinds of compiler changes that can safely be made late
in a development cycle. If this part of UAX31 converges as quickly as I
expect, I think we can get that result reflected into Swift 4, and we can
get a draft version implemented sooner.


Jonathan
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

Personally, a *very* limited cherrypicking might be OK, on the
understanding that it must be done with very stringent inclusion criteria.
However, it's specifically the set algebra operators that I have the
greatest objection to cherrypicking:

* Each of these operators has a specific meaning; it is an anti-goal to
support repurposing the union operator for any other purpose than forming a
union of two sets, for instance.

* Thus, the only rationale for inclusion of these operators is to support
aliasing the set algebra protocol members.

* Now, if it is appropriate for these set algebra operations to be
accessible through these operators, then the standard library should be
providing them.

* However, exactly such an API has been trialled on a Swift branch, and the
set algebra renaming debate of 2015 (or was it early 2016?) on this very
list resulted in a renaming that *rejected* the use of these operators.

* Given that these operators have been deemed not appropriate for the only
methods that should use them, we should *not* specifically enable these
symbols as valid operator characters.

It is of course true that a user can essentially choose to alias any method
using any other identifier. However, this particular scenario is different
because you are advocating for Swift to make express allowance in the
grammar for solely that purpose. In other words, the language design would
be saying to the user, "We don't think this is a good idea at all;
otherwise, it'd be in the standard library. We literally tried that and
decided against it. However, we're specifically going to let you act on
this not-very-good idea by explicitly leaving room for it."

Now, if we had a complete set of operator characters, which would include
set algebra operators, then it wouldn't be problematic in the same way. In
that scenario, the language would be no more responsible for your renaming
formUnion() to the union symbol than to nonsenseAsdf(), since in neither
case would the underlying characters have been included among valid
operator or identifier characters, respectively, for the express purpose of
making possible that renaming.
On Wed, Oct 19, 2016 at 23:48 plx via swift-evolution <
swift-evolution@swift.org> wrote:

> + on the emoji-related parts, +1 in general spirit, +1 for the
> identifier cleanup, -103 for being needlessly overly-restrictive for
> operators; net -1 overall.
>
> Operator abuse is a social problem, and even if a technical fix is
> possible this isn’t that…and despite the messiness of the relevant unicode
> categories, this proposal goes far too far.
>
> For operators, the reasonable thing to do at this time would be to
> hand-select a small subset of the mathematical characters to allow as
> operators—the “greatest hits” so to speak—and move on. If any grave
> oversights are discovered those characters can be included in subsequent
> major revisions; if the consortium ever finishes its recommendation it can
> be adopted at that time.
>
> There’s no need to exhaustively re-do the consortium’s work and there’s no
> need to make a correct-for-all-time decision on each character at this
> time; pick the low-hanging fruit and leave the rest for later.
>
> That not everyone will be perfectly happy with any specific subset is
> predictable and not interesting; not everyone is going to be perfectly
> happy with this proposal’s proposed subset, either.
>
> In any case, I’d specifically hate to lose these:
>
> - approximate equality: ≈
> - set operations: ∩, ∪
> - set relations: ⊂, ⊃, ⊄, ⊅, ⊆, ⊇, ⊈, ⊉, ⊊, ⊋
> - set membership: ∌, ∋, ∈, ∉
> - logical operators: ¬, ∧, ∨
>
> …although there are many more that would be nice to keep available.
>
> On Oct 19, 2016, at 1:34 AM, Jacob Bandes-Storch via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> Dear Swift-Evolution community,
>
> A few of us have been preparing a proposal to refine the definitions of
> identifiers & operators. This includes some changes to the permitted
> Unicode characters.
>
> The latest (perhaps final?) draft is available here:
>
>
> https://github.com/jtbandes/swift-evolution/blob/unicode-id-op/proposals/-refining-identifier-and-operator-symbology.md
>
> We'd welcome your initial thoughts, and will probably submit a PR soon to
> the swift-evolution repo for a formal review. Full text follows below.
>
> —Jacob Bandes-Storch, Xiaodi Wu, Erica Sadun, Jonathan Shapiro
>
>
> Refining Identifier and Operator Symbology
>
>- Proposal: SE-
>
> 
>- Authors: Jacob Bandes-Storch , Erica
>Sadun , Xiaodi Wu ,
>Jonathan Shapiro
>- Review Manager: TBD
>- Status: Awaiting review
>
>
> 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Joe Groff via swift-evolution]

I think this is a promising direction. Getting us in line with Unicode 
recommendations is an important first step, and being conservative about the 
treatment of operator characters and emoji is a good engineering approach, 
though certainly unfortunate in the short term for users who've adopted custom 
operators or found interesting uses for emoji identifiers in Swift 3 and 
earlier.

In the discussion about operators, I wonder whether it makes sense to formally 
separate "identifier" and "operator" characters at all. My hunch is that there 
isn't going to be any perfect categorization; there are so many symbols and 
scripts out there that it's going to be difficult to definitively characterize 
many symbols as "obviously" an operator or identifier. Not every developer has 
the mathematical background to even recognize common math operators beyond the 
elementary arithmetic ones. Something to consider would be to change the way 
operators work in the language so that they can use *any* symbols (subject to 
canonicalization, visibility, and confusability constraints), but require their 
use to always be explicitly declared in a source file that uses an operator 
outside of the standard library. For example, you would have to say something 
like:

import Sets
import operator Sets.∪

to make the '∪' symbol available as an operator in the import declaration's 
scope. This would provide more obvious evidence in the source code of what 
tokens are being employed as operators, and lessen the need to have formally 
distinct identifier and operator character sets.

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[plx via swift-evolution]

+ on the emoji-related parts, +1 in general spirit, +1 for the identifier 
cleanup, -103 for being needlessly overly-restrictive for operators; net -1 
overall.

Operator abuse is a social problem, and even if a technical fix is possible 
this isn’t that…and despite the messiness of the relevant unicode categories, 
this proposal goes far too far.

For operators, the reasonable thing to do at this time would be to hand-select 
a small subset of the mathematical characters to allow as operators—the 
“greatest hits” so to speak—and move on. If any grave oversights are discovered 
those characters can be included in subsequent major revisions; if the 
consortium ever finishes its recommendation it can be adopted at that time.

There’s no need to exhaustively re-do the consortium’s work and there’s no need 
to make a correct-for-all-time decision on each character at this time; pick 
the low-hanging fruit and leave the rest for later. 

That not everyone will be perfectly happy with any specific subset is 
predictable and not interesting; not everyone is going to be perfectly happy 
with this proposal’s proposed subset, either.

In any case, I’d specifically hate to lose these:

- approximate equality: ≈
- set operations: ∩, ∪ 
- set relations: ⊂, ⊃, ⊄, ⊅, ⊆, ⊇, ⊈, ⊉, ⊊, ⊋
- set membership: ∌, ∋, ∈, ∉
- logical operators: ¬, ∧, ∨

…although there are many more that would be nice to keep available. 

> On Oct 19, 2016, at 1:34 AM, Jacob Bandes-Storch via swift-evolution 
>  wrote:
> 
> Dear Swift-Evolution community,
> 
> A few of us have been preparing a proposal to refine the definitions of 
> identifiers & operators. This includes some changes to the permitted Unicode 
> characters.
> 
> The latest (perhaps final?) draft is available here:
> 
> 
> https://github.com/jtbandes/swift-evolution/blob/unicode-id-op/proposals/-refining-identifier-and-operator-symbology.md
>  
> 
> 
> We'd welcome your initial thoughts, and will probably submit a PR soon to the 
> swift-evolution repo for a formal review. Full text follows below.
> 
> —Jacob Bandes-Storch, Xiaodi Wu, Erica Sadun, Jonathan Shapiro
> 
> 
> Refining Identifier and Operator Symbology
> 
> Proposal: SE- 
> 
> Authors: Jacob Bandes-Storch , Erica Sadun 
> , Xiaodi Wu , Jonathan 
> Shapiro
> Review Manager: TBD
> Status: Awaiting review
>  
> Introduction
> 
> This proposal seeks to refine and rationalize Swift's identifier and operator 
> symbology. Specifically, this proposal:
> 
> adopts the Unicode recommendation for identifier characters, with some minor 
> exceptions;
> restricts the legal operator set to the current ASCII operator characters;
> changes where dots may appear in operators; and
> disallows Emoji from identifiers and operators.
>  
> Prior
>  discussion threads & proposals
> 
> Proposal: Normalize Unicode identifiers 
> 
> Unicode identifiers & operators 
> ,
>  with pre-proposal 
>  (a 
> precursor to this document)
> Lexical matters: identifiers and operators 
> 
> Proposal: Allow Single Dollar Sign as Valid Identifier 
> 
> Free the '$' Symbol! 
> 
> Request to add middle dot (U+00B7) as operator character? 
> 
>  
> Guiding
>  principles
> 
> Chris Lattner has written:
> 
> …our current operator space (particularly the unicode segments covered) is 
> not super well considered. It would be great for someone to take a more 
> systematic pass over them to rationalize things.
> We need a token to be unambiguously an operator or identifier - we can have 
> different rules for the leading and subsequent characters though.
> …any proposal that breaks:
> 
> let  = "moof"
> will not be tolerated. :-) :-)
>  
> 

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Benjamin Spratling via swift-evolution]

http://reference.wolfram.com/language/tutorial/Operators.html

They’ve been selling products to mathematicians and scientists for decades.  
Some of those symbols are their own, i.e. not included in unicode.

> On Oct 19, 2016, at 8:54 AM, Xiaodi Wu  wrote:
> 
> Sorry, I've not been very clear on my question. What non-ASCII operators are 
> in use in your production code? What ASCII equivalents did you consider and 
> discard for those particular operators?

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Refining Identifier and Operator Symbology

[Xiaodi Wu via swift-evolution]

Sorry, I've not been very clear on my question. What non-ASCII operators
are in use in your production code? What ASCII equivalents did you consider
and discard for those particular operators?

It's important to remember that math *symbols* are not all operators. We
cannot merely transpose Unicode characters labeled "mathematical" into our
set of operators because many (such as the null set symbol) do not operate
on anything and instead represent mathematical objects, which therefore
suggests that if anything those should be valid identifier characters and
not operator characters. This task becomes impossible once one considers
that, in mathematics, symbols such as nabla are both used as operators
_and_ may stand alone.

The bottom line is that this laborious work of classification is what the
Unicode Consortium is for. I am convinced that we, the Swift community, are
not capable of undertaking this task with any semblance of coherence. Jacob
and I spent two or three emails going back and forth about the inclusion of
"tiny" and "miny," two symbols I'll bet you've never contemplated using in
your code. We also discussed whether pentagons and hexagons were
appropriate (and yes, these are classed as mathematical symbols; on the one
hand, these are plausibly operator-looking characters, but on the other
hand, angles are not considered operators, and shapes can no more take an
operand than can angles). Extending this work throughout the disparate
ranges of mathematical symbols in Unicode 9 is untenable.

Note that while Unicode does not _yet_ have recommendations on operator
characters, and while we cannot wait until it does in order to move
forward, it is not out of the question that a future version of Swift could
incorporate that work even if we remove non-ASCII characters today.
However, if we move ahead with an ad-hoc selection of Unicode characters,
we may never be able to converge with a future Unicode recommendation
without breaking backwards source compatibility.

Thus, I think it's important to be specific about what code you can no
longer write if non-ASCII operators are removed. It is not true that we
have removed "almost every operator." Although we are proposing the removal
of the vast majority of currently valid operator characters, based on our
survey of code bases available to us, we believe that we are removing _zero
or very nearly zero operators_ in usage today. Should you have a use case
we haven't contemplated, I'd be very very keen to hear about it.


On Wed, Oct 19, 2016 at 21:06 Benjamin Spratling via swift-evolution <
swift-evolution@swift.org> wrote:


> On Oct 19, 2016, at 7:18 AM, Xiaodi Wu  wrote:
> How did you decide on those operators instead of ASCII ones?

They are the “correct” mathematical symbols.  On my machine, there is a
block of characters explicitly marked as “Math Symbols".  It’s a pretty
good start (until you get to the section where latin characters are just
drawn in different fonts,at which point, I make no assertions).  ASCII is
not the future.  We have unicode for a reason, so we can type what we mean.

>  Obviously, we would want to enable as many operators as possible to
continue functioning.
I don’t find this statement to be consistent with the others.  From my
perspective, it looks like the proposal deleted as many operators as
possible.

>   •   Operators suffer from low discoverability and difficult
readability. They use symbols, not names. This places a cognitive cost on
users with respect to both recall ("What is the operator that applies the
behavior I need?") and recognition ("What does the operator in this code
do?").

Requiring a developer to read “formUnion(with:)” is a large cognitive
burden for someone trying to use Swift to solve problems in their field.
Not all developers are only UI app makers or coding philosophers.  Many are
engineers and scientists trying to find a tool to get their job done.  For
these fields, using the correct mathematical operators significantly
reduces the difficulty in reading code, which reduces the difficulty
finding bugs.  Swift offers a major performance improvement over the tools
they may currently be using to get their jobs done, i.e. MatLab and
Mathematica.  While those tools offer other features Swift never will,
there is always a push in research fields for better performance and
independence from a desktop meta-tool, so battle tested algorithms are
frequently moved out into stand-alone libraries.  Yes, at times, that has
been my job.  Occasionally, someone working on advanced math libraries
chimes in on the forum, I know there are folks working on vector libraries
out there, I’m sure their code would be more legible using the correct
vector-oriented operators.  With the ascii-only operators, there is a lack
of distinction between scalar products, and vector products, for example.
Something which can be represented easily with extended unicode operators.

If someone really wants to