Hey, everyone. If you're like me, you're sick of the fact that
'UnsafePointer<Int>' doesn't tell you whether or not the pointer can be nil.
Why do we need to suffer this indignity when reference types—including function
pointers!—can distinguish "present" from "absent" with the standard type
'Optional'? Well, good news: here's a proposal to make pointer nullability
explicit. 'UnsafePointer<Int>?' can be null (nil), while 'UnsafePointer<Int>'
cannot. Read on for details!
https://github.com/jrose-apple/swift-evolution/blob/optional-pointers/proposals/nnnn-optional-pointers.md
Bonus good news: I've implemented this locally and updated nearly all the tests
already. Assuming this is accepting, the actual changes will go through review
as a PR on GitHub, although it's mostly going to be one big mega-patch because
the core change has a huge ripple effect.
Jordan
---
Make pointer nullability explicit using Optional
Proposal: SE-NNNN
<https://github.com/apple/swift-evolution/blob/master/proposals/NNNN-name.md>
Author(s): Jordan Rose <https://github.com/jrose-apple>
Status: Awaiting review
Review manager: TBD
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#introduction>Introduction
In Objective-C, pointers (whether to objects or to a non-object type) can be
marked as nullable or nonnull, depending on whether the pointer value can ever
be null. In Swift, however, there is no such way to make this distinction for
pointers to non-object types: an UnsafePointer<Int> might be null, or it might
never be.
We already have a way to describe this: Optionals. This proposal makes
UnsafePointer<Int> represent a non-nullable pointer, and UnsafePointer<Int>? a
nullable pointer. This also allows us to preserve information about pointer
nullability available in header files for imported C and Objective-C APIs.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#motivation>Motivation
Today, UnsafePointer and friends suffer from a problem inherited from C: every
pointer value could potentially be null, and code that works with pointers may
or may not expect this. Failing to take the null pointer case into account can
lead to assertion failures or crashes. For example, pretty much every operation
on UnsafePointer itself requires a valid pointer (reading, writing, and
initializing the pointee or performing arithmetic operations).
Fortunately, when a type has a single invalid value for which no operations are
valid, Swift already has a solution: Optionals. Applying this to pointer types
makes things very clear: if the type is non-optional, the pointer will never be
null, and if it isoptional, the developer must take the "null pointer" case
into account. This clarity has already been appreciated in Apple's Objective-C
headers, which include nullability annotations for all pointer types (not just
object pointers).
This change also allows developers working with pointers to take advantage of
the many syntactic conveniences already built around optionals. For example,
the standard library currently has a helper method on UnsafeMutablePointer
called _setIfNonNil; with "optional pointers" this can be written simply and
clearly:
ptr?.pointee = newValue
Finally, this change also reduces the number of types that conform to
NilLiteralConvertible, a source of confusion for newcomers who (reasonably)
associate nil directly with optionals. Currently the standard library includes
the following NilLiteralConvertible types:
Optional
ImplicitlyUnwrappedOptional (subject of a separate proposal by Chris Willmore)
_OptionalNilComparisonType (used for optionalValue == nil)
UnsafePointer
UnsafeMutablePointer
AutoreleasingUnsafeMutablePointer
OpaquePointer
plus these Objective-C-specific types:
Selector
NSZone (only used to pass nil in Swift)
All of the italicized types would drop their conformance to
NilLiteralConvertible; the "null pointer" would be represented by a nil
optional of a particular type.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#proposed-solution>Proposed
solution
Have the compiler assume that all values with pointer type (the italicized
types listed above) are non-null. This allows the representation of
Optional.none for a pointer type to be a null pointer value.
Drop NilLiteralConvertible conformance for all pointer types.
Teach the Clang importer to treat _Nullable pointers as Optional (and
_Null_unspecified pointers as ImplicitlyUnwrappedOptional).
Deal with the fallout, i.e. adjust the compiler and the standard library to
handle this new behavior.
Test migration and improve the migrator as necessary.
This proposal does not include the removal of the NilLiteralConvertible
protocol altogether; besides still having two distinct optional types, we've
seen people wanting to use nil for their own types (e.g. JSON values).
(Changing this in the future is not out of the question; it's just out of scope
for this proposal.)
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#detailed-design>Detailed
design
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#api-changes>API
Changes
Conformance to NilLiteralConvertible is removed from all types except Optional,
ImplicitlyUnwrappedOptional, and _OptionalNilComparisonType, along with the
implementation of init(nilLiteral:).
init(bitPattern: Int) and init(bitPattern: UInt) on all pointer types become
failable; if the bit pattern represents a null pointer, nil is returned.
Process.unsafeArgv is a pointer to a null-terminated C array of C strings, so
its type changes from UnsafeMutablePointer<UnsafeMutablePointer<Int8>> to
UnsafeMutablePointer<UnsafeMutablePointer<Int8>?>, i.e. the inner pointer type
becomes optional. It is then an error to access Process.unsafeArgv before
entering main. (Previously you would get a null pointer value.)
NSErrorPointer becomes optional:
-public typealias NSErrorPointer = AutoreleasingUnsafeMutablePointer<NSError?>
+public typealias NSErrorPointer = AutoreleasingUnsafeMutablePointer<NSError?>?
A number of methods on String that came from NSString now have optional
parameters:
public func completePathIntoString(
- outputName: UnsafeMutablePointer<String> = nil,
+ outputName: UnsafeMutablePointer<String>? = nil,
caseSensitive: Bool,
- matchesIntoArray: UnsafeMutablePointer<[String]> = nil,
+ matchesIntoArray: UnsafeMutablePointer<[String]>? = nil,
filterTypes: [String]? = nil
) -> Int {
public init(
contentsOfFile path: String,
- usedEncoding: UnsafeMutablePointer<NSStringEncoding> = nil
+ usedEncoding: UnsafeMutablePointer<NSStringEncoding>? = nil
) throws {
public init(
contentsOfURL url: NSURL,
- usedEncoding enc: UnsafeMutablePointer<NSStringEncoding> = nil
+ usedEncoding enc: UnsafeMutablePointer<NSStringEncoding>? = nil
) throws {
public func linguisticTags(
in range: Range<Index>,
scheme tagScheme: String,
options opts: NSLinguisticTaggerOptions = [],
orthography: NSOrthography? = nil,
- tokenRanges: UnsafeMutablePointer<[Range<Index>]> = nil
+ tokenRanges: UnsafeMutablePointer<[Range<Index>]>? = nil
) -> [String] {
NSZone's no-argument initializer is gone. (It probably should have been removed
already as part of the Swift 3 naming cleanup.)
A small regression: optional pointers can no longer be passed using withVaList
because it would require a conditional conformance to the CVarArg protocol. For
now, using unsafeBitCast to reinterpret the optional pointer as an Int is the
best alternative; Int has the same C variadic calling conventions as a pointer
on all supported platforms.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#conversion-between-pointers>Conversion
between pointers
Currently each pointer type has initializers of this form:
init<OtherPointee>(_ otherPointer: UnsafePointer<OtherPointee>)
This simply makes a pointer with a different type but the same address as
otherPointer. However, in making pointer nullability explicit, this now only
converts non-nil pointers to non-nil pointers. In my experiments, this has led
to this idiom becoming very common:
// Before:
let untypedPointer = UnsafePointer<Void>(ptr)
// After:
let untypedPointer = ptr.map(UnsafePointer<Void>.init)
// Usually the pointee type is actually inferred:
foo(ptr.map(UnsafePointer.init))
I consider this a bit more difficult to understand than the original code, at
least at a glance. We should therefore add new initializers of the following
form:
init?<OtherPointee>(_ otherPointer: UnsafePointer<OtherPointee>?) {
guard let nonnullPointer = otherPointer else {
return nil
}
self.init(nonnullPointer)
}
The body is for explanation purposes only; we'll make sure the actual
implementation does not require an extra comparison.
(This would need to be an overload rather than replacing the previous
initializer because the "non-null-ness" should be preserved through the type
conversion.)
The alternative is to leave this initializer out, and require the nil case to
be explicitly handled or mapped away.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#open-issue-unsafebufferpointer>Open
Issue: UnsafeBufferPointer
The type UnsafeBufferPointer represents a bounded typed memory region with no
ownership or lifetime semantics; it is logically a bare typed pointer (its
baseAddress) and a length (count). For a buffer with 0 elements, however,
there's no need to provide the address of allocated memory, since it can't be
read from. Previously this case would be represented as a nil base address and
a count of 0.
With optional pointers, this now imposes a cost on clients that want to access
the base address: they need to consider the nil case explicitly, where
previously they wouldn't have had to. There are several possibilities here,
each with their own possible implementations:
Like UnsafePointer, UnsafeBufferPointer should always have a valid base
address, even when the count is 0. An UnsafeBufferPointer with a
potentially-nil base address should be optional.
UnsafeBufferPointer's initializer accepts an optional pointer and becomes
failable, returning nil if the input pointer is nil.
UnsafeBufferPointer's initializer accepts an optional pointer and synthesizes a
non-null aligned pointer value if given nil as a base address.
UnsafeBufferPointer's initializer only accepts non-optional pointers. Clients
such as withUnsafeBufferPointermust synthesize a non-null aligned pointer value
if they do not have a valid pointer to provide.
UnsafeBufferPointer's initializer only accepts non-optional pointers. Clients
using withUnsafeBufferPointermust handle a nil buffer.
UnsafeBufferPointer should allow nil base addresses, i.e. the baseAddress
property will be optional. Clients will need to handle this case explicitly.
UnsafeBufferPointer's initializer accepts an optional pointer, but no other
changes are made.
UnsafeBufferPointer's initializer accepts an optional pointer. Additionally,
any buffers initialized with a count of 0 will be canonicalized to having a
base address of nil.
I'm currently leaning towards option (2i). Clients that expect a pointer and
length probably shouldn't require the pointer to be non-null, but if they do
then perhaps there's a reason for it. It's also the least work.
Chris (Lattner) is leaning towards option (1ii), which treats
UnsafeBufferPointer similar to UnsafePointer while not penalizing the common
case of withUnsafeBufferPointer.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#impact-on-existing-code>Impact
on existing code
Any code that uses a pointer type (including Selector or NSZone) may be
affected by this change. For the most part our existing logic to handle last
year's nullability audit should cover this, but the implementer should test
migration of several projects to see what issues might arise.
Anecdotally, in migrating the standard library to use this new logic I've been
quite happy with nullability being made explicit. There are many places where a
pointer really can't be nil.
<https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#alternatives-considered>Alternatives
considered
The primary alternative here would be to leave everything as it is today, with
UnsafePointer and friends including the null pointer as one of their normal
values. This has obviously worked just fine for nearly two years of Swift, but
it is leaving information on the table that can help avoid bugs, and is strange
in a language that makes fluent use of Optional. As a fairly major
source-breaking change, it is also something that we probably should do sooner
rather than later in the language's evolution._______________________________________________
swift-evolution mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-evolution
