Re: [swift-evolution] (core library) modern URL types

[Taylor Swift via swift-evolution]

great work! it looks like there is quite a lot of duplicated work going on
here though which is unfortunate. how do we reconcile these 2
implementations?

On Thu, Sep 21, 2017 at 12:16 PM, Aleksey Mashanov <
aleksey.masha...@gmail.com> wrote:

> I have an alternative implementation of a URI (a superset of URL). It is
> quite ready to use but still requires some polishing and adding of
> additional specific URI schemes.
> It is designed primarily for server side usage but can be useful on a
> client too. The main goals of the design is correctness (according to
> RFC3986 and RFC7230) and efficiency.
>
> You can take a look at it here: https://github.com/my-mail-ru/swift-URI
>
> 2017-08-21 7:38 GMT+03:00 Taylor Swift via swift-evolution <
> swift-evolution@swift.org>:
>
>> Okay so a few days ago there was a discussion
>> 
>> about getting pure swift file system support into Foundation or another
>> core library, and in my opinion, doing this requires a total overhaul of
>> the `URL` type (which is currently little more than a wrapper for
>> NSURL), so I’ve just started a pure Swift URL library project at <
>> https://github.com/kelvin13/url>.
>>
>> The library’s parsing and validation core (~1K loc pure swift) is already
>> in place and functional; the goal is to eventually support all of the
>> Foundation URL functionality.
>>
>> The new `URL` type is implemented as a value type with utf8 storage
>> backed by an array buffer. The URLs are just 56 bytes long each, so they
>> should be able to fit into cache lines. (NSURL by comparison is over 128
>> bytes in size; it’s only saved by the fact that the thing is passed as a
>> reference type.)
>>
>> As I said, this is still really early on and not a mature library at all
>> but everyone is invited to observe, provide feedback, or contribute!
>>
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Aleksey Mashanov via swift-evolution]

I have an alternative implementation of a URI (a superset of URL). It is
quite ready to use but still requires some polishing and adding of
additional specific URI schemes.
It is designed primarily for server side usage but can be useful on a
client too. The main goals of the design is correctness (according to
RFC3986 and RFC7230) and efficiency.

You can take a look at it here: https://github.com/my-mail-ru/swift-URI

2017-08-21 7:38 GMT+03:00 Taylor Swift via swift-evolution <
swift-evolution@swift.org>:

> Okay so a few days ago there was a discussion
> 
> about getting pure swift file system support into Foundation or another
> core library, and in my opinion, doing this requires a total overhaul of
> the `URL` type (which is currently little more than a wrapper for NSURL),
> so I’ve just started a pure Swift URL library project at <
> https://github.com/kelvin13/url>.
>
> The library’s parsing and validation core (~1K loc pure swift) is already
> in place and functional; the goal is to eventually support all of the
> Foundation URL functionality.
>
> The new `URL` type is implemented as a value type with utf8 storage backed
> by an array buffer. The URLs are just 56 bytes long each, so they should be
> able to fit into cache lines. (NSURL by comparison is over 128 bytes in
> size; it’s only saved by the fact that the thing is passed as a reference
> type.)
>
> As I said, this is still really early on and not a mature library at all
> but everyone is invited to observe, provide feedback, or contribute!
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Thorsten Seitz via swift-evolution]

Thanks!

-Thorsten

> Am 21.09.2017 um 10:07 schrieb Jonathan Hull :
> 
> Looks like the author posted a free copy of the paper here:
> http://www.hirschfeld.org/writings/media/WeiherHirschfeld_2013_PolymorphicIdentifiersUniformResourceAccessInObjectiveSmalltalk_AcmDL.pdf
> 
> Thanks,
> Jon
> 
>> On Sep 20, 2017, at 10:03 PM, Thorsten Seitz via swift-evolution 
>>  wrote:
>> 
>> 
>>> Am 24.08.2017 um 23:07 schrieb Eagle Offshore via swift-evolution 
>>> :
>>> 
>>> I liked the polymorphic identifiers paper presented at Dynamic Languages 
>>> Symposium 2013.
>>> 
>>> http://dl.acm.org/citation.cfm?doid=2508168.2508169
>>> 
>>> There's a lot of power in URI's that remains largely untapped in most 
>>> systems.
>>> 
>>> There's not a great reason for this heterogeneity other than historical 
>>> baggage.
>>> 
>>> Properly done, URI's can unify keypaths, user defaults, environment 
>>> variables, files, network stores, databases, etc.
>> 
>> Would you mind summarizing the idea as the paper is not freely available?
>> 
>> -Thorsten
>> 
>> 
 On Aug 22, 2017, at 12:52 PM, Félix Cloutier via swift-evolution 
  wrote:
 
 Just leaving it out here that the iOS/macOS app experience with paths is 
 likely to be very different from the server (or system program) 
 experience. Most of your paths are relative to a container because the 
 effective root of your data is the container. I would tend to believe that 
 a lot of your paths, not while fully static, could be expressed with 
 something like "${DOCUMENTS}/static/part", where ${DOCUMENTS} is the only 
 thing that effectively changes. A system-level program just happens to use 
 / as its prefix.
 
 Most platforms already have some notion of file domains, which are really 
 just path prefixes (the user's home folder, the file system root, the 
 network root, etc). I think that some level of support for these would be 
 desirable (would it be only a set of functions that return different path 
 prefixes). It's also my humble opinion that tilde expansion should be part 
 of a larger shell expansion feature to avoid surprises.
 
 I think that I support the conclusion.
 
 Félix
 
> Le 22 août 2017 à 12:02, Dave DeLong  a écrit :
> 
 
> I suppose, if you squint at it weirdly.
> 
> My current Path API is a “Path” protocol, with “AbsolutePath” and 
> “RelativePath” struct versions. The protocol defines a path to be an 
> array of path components. The only real difference between an 
> AbsolutePath and a RelativePath is that all file system operations would 
> only take an AbsolutePath. A URL would also only provide an AbsolutePath 
> as its “path” bit.
> 
> public enum PathComponent {
> case this // “."
> case up   // “..” 
> case item(name: String, extension: String?)
> }
> 
> public protocol Path {   
> var components: Array { get }
> init(_ components: Array) // used on protocol 
> extensions that mutate paths, such as appending components
> }
> 
> public struct AbsolutePath: Path { }
> public struct RelativePath: Path { }
> 
> By separating out the concept of an Absolute and a Relative path, I can 
> put additional functionality on each one to make semantic sense (you 
> cannot concatenate two absolute paths, but you can concat any path with a 
> relative path, for example). Or all file system operations must take an 
> AbsolutePath. 
> 
> One of the key things I realized is that a “Path” type should not be 
> ExpressibleByStringLiteral, because you cannot statically determine if a 
> Path should be absolute or relative. However, one of the initializers for 
> an AbsolutePath would handle things like expanding a tilde, and both 
> types try to reduce a set of components as much as possible (by filtering 
> out “.this” components, and handling “.up” components where possible, 
> etc). Also in my experience, it’s fairly rare to want to deal with a 
> known-at-compile-time, hard-coded path. Usually you’re dealing with paths 
> relative to known “containers” that are determined at runtime (current 
> user’s home folder, app’s sandboxed documents directory, etc).
> 
> Another thing I’ve done is that no direct file system operations exist on 
> AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those 
> are still on FileManager/FileHandle/etc in the form of extensions to 
> handle the new types. In my app, a path is just a path, and it only has 
> meaning based on the thing that is using it. An AbsolutePath for a URL is 
> used differently than an AbsolutePath on a file system, although they are 
> represented with the same “AbsolutePath” 

Re: [swift-evolution] (core library) modern URL types

[Jonathan Hull via swift-evolution]

Looks like the author posted a free copy of the paper here:
http://www.hirschfeld.org/writings/media/WeiherHirschfeld_2013_PolymorphicIdentifiersUniformResourceAccessInObjectiveSmalltalk_AcmDL.pdf

Thanks,
Jon

> On Sep 20, 2017, at 10:03 PM, Thorsten Seitz via swift-evolution 
>  wrote:
> 
> 
> Am 24.08.2017 um 23:07 schrieb Eagle Offshore via swift-evolution 
> >:
> 
>> I liked the polymorphic identifiers paper presented at Dynamic Languages 
>> Symposium 2013.
>> 
>> http://dl.acm.org/citation.cfm?doid=2508168.2508169 
>> 
>> There's a lot of power in URI's that remains largely untapped in most 
>> systems.
>> 
>> There's not a great reason for this heterogeneity other than historical 
>> baggage.
>> 
>> Properly done, URI's can unify keypaths, user defaults, environment 
>> variables, files, network stores, databases, etc.
> 
> Would you mind summarizing the idea as the paper is not freely available?
> 
> -Thorsten
> 
> 
>>> On Aug 22, 2017, at 12:52 PM, Félix Cloutier via swift-evolution 
>>> > wrote:
>>> 
>>> Just leaving it out here that the iOS/macOS app experience with paths is 
>>> likely to be very different from the server (or system program) experience. 
>>> Most of your paths are relative to a container because the effective root 
>>> of your data is the container. I would tend to believe that a lot of your 
>>> paths, not while fully static, could be expressed with something like 
>>> "${DOCUMENTS}/static/part", where ${DOCUMENTS} is the only thing that 
>>> effectively changes. A system-level program just happens to use / as its 
>>> prefix.
>>> 
>>> Most platforms already have some notion of file domains, which are really 
>>> just path prefixes (the user's home folder, the file system root, the 
>>> network root, etc). I think that some level of support for these would be 
>>> desirable (would it be only a set of functions that return different path 
>>> prefixes). It's also my humble opinion that tilde expansion should be part 
>>> of a larger shell expansion feature to avoid surprises.
>>> 
>>> I think that I support the conclusion.
>>> 
>>> Félix
>>> 
>>> Le 22 août 2017 à 12:02, Dave DeLong >> > a écrit :
>>> 
 I suppose, if you squint at it weirdly.
 
 My current Path API is a “Path” protocol, with “AbsolutePath” and 
 “RelativePath” struct versions. The protocol defines a path to be an array 
 of path components. The only real difference between an AbsolutePath and a 
 RelativePath is that all file system operations would only take an 
 AbsolutePath. A URL would also only provide an AbsolutePath as its “path” 
 bit.
 
 public enum PathComponent {
 case this // “."
 case up   // “..” 
 case item(name: String, extension: String?)
 }
 
 public protocol Path {   
 var components: Array { get }
 init(_ components: Array) // used on protocol 
 extensions that mutate paths, such as appending components
 }
 
 public struct AbsolutePath: Path { }
 public struct RelativePath: Path { }
 
 By separating out the concept of an Absolute and a Relative path, I can 
 put additional functionality on each one to make semantic sense (you 
 cannot concatenate two absolute paths, but you can concat any path with a 
 relative path, for example). Or all file system operations must take an 
 AbsolutePath. 
 
 One of the key things I realized is that a “Path” type should not be 
 ExpressibleByStringLiteral, because you cannot statically determine if a 
 Path should be absolute or relative. However, one of the initializers for 
 an AbsolutePath would handle things like expanding a tilde, and both types 
 try to reduce a set of components as much as possible (by filtering out 
 “.this” components, and handling “.up” components where possible, etc). 
 Also in my experience, it’s fairly rare to want to deal with a 
 known-at-compile-time, hard-coded path. Usually you’re dealing with paths 
 relative to known “containers” that are determined at runtime (current 
 user’s home folder, app’s sandboxed documents directory, etc).
 
 Another thing I’ve done is that no direct file system operations exist on 
 AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those 
 are still on FileManager/FileHandle/etc in the form of extensions to 
 handle the new types. In my app, a path is just a path, and it only has 
 meaning based on the thing that is using it. An AbsolutePath for a URL is 
 used differently than an AbsolutePath on a file system, although they are 
 represented with the same “AbsolutePath” type.
 
 I’m not saying this is a perfect API 

Re: [swift-evolution] (core library) modern URL types

[Thorsten Seitz via swift-evolution]

> Am 24.08.2017 um 23:07 schrieb Eagle Offshore via swift-evolution 
> :
> 
> I liked the polymorphic identifiers paper presented at Dynamic Languages 
> Symposium 2013.
> 
> http://dl.acm.org/citation.cfm?doid=2508168.2508169
> 
> There's a lot of power in URI's that remains largely untapped in most systems.
> 
> There's not a great reason for this heterogeneity other than historical 
> baggage.
> 
> Properly done, URI's can unify keypaths, user defaults, environment 
> variables, files, network stores, databases, etc.

Would you mind summarizing the idea as the paper is not freely available?

-Thorsten


>> On Aug 22, 2017, at 12:52 PM, Félix Cloutier via swift-evolution 
>>  wrote:
>> 
>> Just leaving it out here that the iOS/macOS app experience with paths is 
>> likely to be very different from the server (or system program) experience. 
>> Most of your paths are relative to a container because the effective root of 
>> your data is the container. I would tend to believe that a lot of your 
>> paths, not while fully static, could be expressed with something like 
>> "${DOCUMENTS}/static/part", where ${DOCUMENTS} is the only thing that 
>> effectively changes. A system-level program just happens to use / as its 
>> prefix.
>> 
>> Most platforms already have some notion of file domains, which are really 
>> just path prefixes (the user's home folder, the file system root, the 
>> network root, etc). I think that some level of support for these would be 
>> desirable (would it be only a set of functions that return different path 
>> prefixes). It's also my humble opinion that tilde expansion should be part 
>> of a larger shell expansion feature to avoid surprises.
>> 
>> I think that I support the conclusion.
>> 
>> Félix
>> 
>>> Le 22 août 2017 à 12:02, Dave DeLong  a écrit :
>>> 
>> 
>>> I suppose, if you squint at it weirdly.
>>> 
>>> My current Path API is a “Path” protocol, with “AbsolutePath” and 
>>> “RelativePath” struct versions. The protocol defines a path to be an array 
>>> of path components. The only real difference between an AbsolutePath and a 
>>> RelativePath is that all file system operations would only take an 
>>> AbsolutePath. A URL would also only provide an AbsolutePath as its “path” 
>>> bit.
>>> 
>>> public enum PathComponent {
>>> case this // “."
>>> case up   // “..” 
>>> case item(name: String, extension: String?)
>>> }
>>> 
>>> public protocol Path {   
>>> var components: Array { get }
>>> init(_ components: Array) // used on protocol extensions 
>>> that mutate paths, such as appending components
>>> }
>>> 
>>> public struct AbsolutePath: Path { }
>>> public struct RelativePath: Path { }
>>> 
>>> By separating out the concept of an Absolute and a Relative path, I can put 
>>> additional functionality on each one to make semantic sense (you cannot 
>>> concatenate two absolute paths, but you can concat any path with a relative 
>>> path, for example). Or all file system operations must take an 
>>> AbsolutePath. 
>>> 
>>> One of the key things I realized is that a “Path” type should not be 
>>> ExpressibleByStringLiteral, because you cannot statically determine if a 
>>> Path should be absolute or relative. However, one of the initializers for 
>>> an AbsolutePath would handle things like expanding a tilde, and both types 
>>> try to reduce a set of components as much as possible (by filtering out 
>>> “.this” components, and handling “.up” components where possible, etc). 
>>> Also in my experience, it’s fairly rare to want to deal with a 
>>> known-at-compile-time, hard-coded path. Usually you’re dealing with paths 
>>> relative to known “containers” that are determined at runtime (current 
>>> user’s home folder, app’s sandboxed documents directory, etc).
>>> 
>>> Another thing I’ve done is that no direct file system operations exist on 
>>> AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those 
>>> are still on FileManager/FileHandle/etc in the form of extensions to handle 
>>> the new types. In my app, a path is just a path, and it only has meaning 
>>> based on the thing that is using it. An AbsolutePath for a URL is used 
>>> differently than an AbsolutePath on a file system, although they are 
>>> represented with the same “AbsolutePath” type.
>>> 
>>> I’m not saying this is a perfect API of course, or even that a hypothetical 
>>> stdlib-provided Path should mimic this. I’m just saying that for my 
>>> use-case, this has vastly simplified how I deal with paths, because both 
>>> URL and String smell really bad for what I’m doing.
>>> 
>>> Dave
>>> 
 On Aug 22, 2017, at 12:37 PM, Taylor Swift  wrote:
 So are you saying we need three distinct “URI” types for local-absolute, 
 local-relative, and remote? That’s a lot of API surface to support.
 
 On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong  

Re: [swift-evolution] (core library) modern URL types

[Thorsten Seitz via swift-evolution]

Totally agree!

-Thorsten

> Am 24.08.2017 um 20:06 schrieb Jean-Daniel via swift-evolution 
> :
> 
> Yes, and a URI class that don’t provide any FS operations, but only take care 
> of proper URI parsing and building.
> 
>> Le 23 août 2017 à 12:03, Jakob Egger via swift-evolution 
>>  a écrit :
>> 
>> I would absolutely love to see an API like AbsolutePath / RelativePath for 
>> file system operations!
>> 
>>> On 22. Aug 2017, at 21:02, Dave DeLong via swift-evolution 
>>>  wrote:
>>> 
>>> I suppose, if you squint at it weirdly.
>>> 
>>> My current Path API is a “Path” protocol, with “AbsolutePath” and 
>>> “RelativePath” struct versions. The protocol defines a path to be an array 
>>> of path components. The only real difference between an AbsolutePath and a 
>>> RelativePath is that all file system operations would only take an 
>>> AbsolutePath. A URL would also only provide an AbsolutePath as its “path” 
>>> bit.
>>> 
>>> public enum PathComponent {
>>> case this // “."
>>> case up   // “..” 
>>> case item(name: String, extension: String?)
>>> }
>>> 
>>> public protocol Path {   
>>> var components: Array { get }
>>> init(_ components: Array) // used on protocol extensions 
>>> that mutate paths, such as appending components
>>> }
>>> 
>>> public struct AbsolutePath: Path { }
>>> public struct RelativePath: Path { }
>>> 
>>> By separating out the concept of an Absolute and a Relative path, I can put 
>>> additional functionality on each one to make semantic sense (you cannot 
>>> concatenate two absolute paths, but you can concat any path with a relative 
>>> path, for example). Or all file system operations must take an 
>>> AbsolutePath. 
>>> 
>>> One of the key things I realized is that a “Path” type should not be 
>>> ExpressibleByStringLiteral, because you cannot statically determine if a 
>>> Path should be absolute or relative. However, one of the initializers for 
>>> an AbsolutePath would handle things like expanding a tilde, and both types 
>>> try to reduce a set of components as much as possible (by filtering out 
>>> “.this” components, and handling “.up” components where possible, etc). 
>>> Also in my experience, it’s fairly rare to want to deal with a 
>>> known-at-compile-time, hard-coded path. Usually you’re dealing with paths 
>>> relative to known “containers” that are determined at runtime (current 
>>> user’s home folder, app’s sandboxed documents directory, etc).
>>> 
>>> Another thing I’ve done is that no direct file system operations exist on 
>>> AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those 
>>> are still on FileManager/FileHandle/etc in the form of extensions to handle 
>>> the new types. In my app, a path is just a path, and it only has meaning 
>>> based on the thing that is using it. An AbsolutePath for a URL is used 
>>> differently than an AbsolutePath on a file system, although they are 
>>> represented with the same “AbsolutePath” type.
>>> 
>>> I’m not saying this is a perfect API of course, or even that a hypothetical 
>>> stdlib-provided Path should mimic this. I’m just saying that for my 
>>> use-case, this has vastly simplified how I deal with paths, because both 
>>> URL and String smell really bad for what I’m doing.
>>> 
>>> Dave
>>> 
 On Aug 22, 2017, at 12:37 PM, Taylor Swift  wrote:
 
 So are you saying we need three distinct “URI” types for local-absolute, 
 local-relative, and remote? That’s a lot of API surface to support.
 
> On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong  wrote:
> I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
> abstraction for file system paths.
> 
> I maintain an app that deals a lot with file system paths, and using URL 
> has always felt cumbersome, but String is the absolute wrong type to use. 
> Lately as I’ve been working on it, I’ve been experimenting with a 
> concrete “Path” type, similar to PathKit 
> (https://github.com/kylef/PathKit/). Working in terms of AbsolutePath and 
> RelativePath (what I’ve been calling things) has been extremely 
> refreshing, because it allows me to better articulate the kind of data 
> I’m dealing with. URL doesn’t handle pure-relative paths very well, and 
> it’s always a bit of a mystery how resilient I need to be about checking 
> .isFileURL or whatever. All the extra properties (port, user, password, 
> host) feel hugely unnecessary as well.
> 
> Dave
> 
>> On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
>>  wrote:
>> 
>> I'm not convinced that URLs are the appropriate abstraction for a file 
>> system path. For the record, I'm not a fan of existing Foundation 
>> methods that create objects from an URL. There is a useful and 
>> 

Re: [swift-evolution] (core library) modern URL types

[Thorsten Seitz via swift-evolution]

+1

-Thorsten 

> Am 23.08.2017 um 12:03 schrieb Jakob Egger via swift-evolution 
> :
> 
> I would absolutely love to see an API like AbsolutePath / RelativePath for 
> file system operations!
> 
>> On 22. Aug 2017, at 21:02, Dave DeLong via swift-evolution 
>>  wrote:
>> 
>> I suppose, if you squint at it weirdly.
>> 
>> My current Path API is a “Path” protocol, with “AbsolutePath” and 
>> “RelativePath” struct versions. The protocol defines a path to be an array 
>> of path components. The only real difference between an AbsolutePath and a 
>> RelativePath is that all file system operations would only take an 
>> AbsolutePath. A URL would also only provide an AbsolutePath as its “path” 
>> bit.
>> 
>> public enum PathComponent {
>> case this // “."
>> case up   // “..” 
>> case item(name: String, extension: String?)
>> }
>> 
>> public protocol Path {   
>> var components: Array { get }
>> init(_ components: Array) // used on protocol extensions 
>> that mutate paths, such as appending components
>> }
>> 
>> public struct AbsolutePath: Path { }
>> public struct RelativePath: Path { }
>> 
>> By separating out the concept of an Absolute and a Relative path, I can put 
>> additional functionality on each one to make semantic sense (you cannot 
>> concatenate two absolute paths, but you can concat any path with a relative 
>> path, for example). Or all file system operations must take an AbsolutePath. 
>> 
>> One of the key things I realized is that a “Path” type should not be 
>> ExpressibleByStringLiteral, because you cannot statically determine if a 
>> Path should be absolute or relative. However, one of the initializers for an 
>> AbsolutePath would handle things like expanding a tilde, and both types try 
>> to reduce a set of components as much as possible (by filtering out “.this” 
>> components, and handling “.up” components where possible, etc). Also in my 
>> experience, it’s fairly rare to want to deal with a known-at-compile-time, 
>> hard-coded path. Usually you’re dealing with paths relative to known 
>> “containers” that are determined at runtime (current user’s home folder, 
>> app’s sandboxed documents directory, etc).
>> 
>> Another thing I’ve done is that no direct file system operations exist on 
>> AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those 
>> are still on FileManager/FileHandle/etc in the form of extensions to handle 
>> the new types. In my app, a path is just a path, and it only has meaning 
>> based on the thing that is using it. An AbsolutePath for a URL is used 
>> differently than an AbsolutePath on a file system, although they are 
>> represented with the same “AbsolutePath” type.
>> 
>> I’m not saying this is a perfect API of course, or even that a hypothetical 
>> stdlib-provided Path should mimic this. I’m just saying that for my 
>> use-case, this has vastly simplified how I deal with paths, because both URL 
>> and String smell really bad for what I’m doing.
>> 
>> Dave
>> 
>>> On Aug 22, 2017, at 12:37 PM, Taylor Swift  wrote:
>>> 
>>> So are you saying we need three distinct “URI” types for local-absolute, 
>>> local-relative, and remote? That’s a lot of API surface to support.
>>> 
 On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong  wrote:
 I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
 abstraction for file system paths.
 
 I maintain an app that deals a lot with file system paths, and using URL 
 has always felt cumbersome, but String is the absolute wrong type to use. 
 Lately as I’ve been working on it, I’ve been experimenting with a concrete 
 “Path” type, similar to PathKit (https://github.com/kylef/PathKit/). 
 Working in terms of AbsolutePath and RelativePath (what I’ve been calling 
 things) has been extremely refreshing, because it allows me to better 
 articulate the kind of data I’m dealing with. URL doesn’t handle 
 pure-relative paths very well, and it’s always a bit of a mystery how 
 resilient I need to be about checking .isFileURL or whatever. All the 
 extra properties (port, user, password, host) feel hugely unnecessary as 
 well.
 
 Dave
 
> On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
>  wrote:
> 
> I'm not convinced that URLs are the appropriate abstraction for a file 
> system path. For the record, I'm not a fan of existing Foundation methods 
> that create objects from an URL. There is a useful and fundamental 
> difference between a local path and a remote path, and conflating the two 
> has been a security pain point in many languages and frameworks that 
> allow it. Examples include remote file inclusion in PHP and malicious 
> doctypes in XML. Windows also had its share of issues with UNC paths.
> 
> Even when 

Re: [swift-evolution] (core library) modern URL types

[Eagle Offshore via swift-evolution]

I liked the polymorphic identifiers paper presented at Dynamic Languages 
Symposium 2013.

http://dl.acm.org/citation.cfm?doid=2508168.2508169 


There's a lot of power in URI's that remains largely untapped in most systems.

There's not a great reason for this heterogeneity other than historical baggage.

Properly done, URI's can unify keypaths, user defaults, environment variables, 
files, network stores, databases, etc.


> On Aug 22, 2017, at 12:52 PM, Félix Cloutier via swift-evolution 
>  wrote:
> 
> Just leaving it out here that the iOS/macOS app experience with paths is 
> likely to be very different from the server (or system program) experience. 
> Most of your paths are relative to a container because the effective root of 
> your data is the container. I would tend to believe that a lot of your paths, 
> not while fully static, could be expressed with something like 
> "${DOCUMENTS}/static/part", where ${DOCUMENTS} is the only thing that 
> effectively changes. A system-level program just happens to use / as its 
> prefix.
> 
> Most platforms already have some notion of file domains, which are really 
> just path prefixes (the user's home folder, the file system root, the network 
> root, etc). I think that some level of support for these would be desirable 
> (would it be only a set of functions that return different path prefixes). 
> It's also my humble opinion that tilde expansion should be part of a larger 
> shell expansion feature to avoid surprises.
> 
> I think that I support the conclusion.
> 
> Félix
> 
> Le 22 août 2017 à 12:02, Dave DeLong  a écrit :
> 
>> I suppose, if you squint at it weirdly.
>> 
>> My current Path API is a “Path” protocol, with “AbsolutePath” and 
>> “RelativePath” struct versions. The protocol defines a path to be an array 
>> of path components. The only real difference between an AbsolutePath and a 
>> RelativePath is that all file system operations would only take an 
>> AbsolutePath. A URL would also only provide an AbsolutePath as its “path” 
>> bit.
>> 
>> public enum PathComponent {
>> case this // “."
>> case up   // “..” 
>> case item(name: String, extension: String?)
>> }
>> 
>> public protocol Path {   
>> var components: Array { get }
>> init(_ components: Array) // used on protocol extensions 
>> that mutate paths, such as appending components
>> }
>> 
>> public struct AbsolutePath: Path { }
>> public struct RelativePath: Path { }
>> 
>> By separating out the concept of an Absolute and a Relative path, I can put 
>> additional functionality on each one to make semantic sense (you cannot 
>> concatenate two absolute paths, but you can concat any path with a relative 
>> path, for example). Or all file system operations must take an AbsolutePath. 
>> 
>> One of the key things I realized is that a “Path” type should not be 
>> ExpressibleByStringLiteral, because you cannot statically determine if a 
>> Path should be absolute or relative. However, one of the initializers for an 
>> AbsolutePath would handle things like expanding a tilde, and both types try 
>> to reduce a set of components as much as possible (by filtering out “.this” 
>> components, and handling “.up” components where possible, etc). Also in my 
>> experience, it’s fairly rare to want to deal with a known-at-compile-time, 
>> hard-coded path. Usually you’re dealing with paths relative to known 
>> “containers” that are determined at runtime (current user’s home folder, 
>> app’s sandboxed documents directory, etc).
>> 
>> Another thing I’ve done is that no direct file system operations exist on 
>> AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those 
>> are still on FileManager/FileHandle/etc in the form of extensions to handle 
>> the new types. In my app, a path is just a path, and it only has meaning 
>> based on the thing that is using it. An AbsolutePath for a URL is used 
>> differently than an AbsolutePath on a file system, although they are 
>> represented with the same “AbsolutePath” type.
>> 
>> I’m not saying this is a perfect API of course, or even that a hypothetical 
>> stdlib-provided Path should mimic this. I’m just saying that for my 
>> use-case, this has vastly simplified how I deal with paths, because both URL 
>> and String smell really bad for what I’m doing.
>> 
>> Dave
>> 
>>> On Aug 22, 2017, at 12:37 PM, Taylor Swift >> > wrote:
>>> So are you saying we need three distinct “URI” types for local-absolute, 
>>> local-relative, and remote? That’s a lot of API surface to support.
>>> 
>>> On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong >> > wrote:
>>> I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
>>> abstraction for file system paths.
>>> 
>>> I maintain an app that deals a lot with file system paths, and 

Re: [swift-evolution] (core library) modern URL types

[Jean-Daniel via swift-evolution]

Yes, and a URI class that don’t provide any FS operations, but only take care 
of proper URI parsing and building.

> Le 23 août 2017 à 12:03, Jakob Egger via swift-evolution 
>  a écrit :
> 
> I would absolutely love to see an API like AbsolutePath / RelativePath for 
> file system operations!
> 
>> On 22. Aug 2017, at 21:02, Dave DeLong via swift-evolution 
>> > wrote:
>> 
>> I suppose, if you squint at it weirdly.
>> 
>> My current Path API is a “Path” protocol, with “AbsolutePath” and 
>> “RelativePath” struct versions. The protocol defines a path to be an array 
>> of path components. The only real difference between an AbsolutePath and a 
>> RelativePath is that all file system operations would only take an 
>> AbsolutePath. A URL would also only provide an AbsolutePath as its “path” 
>> bit.
>> 
>> public enum PathComponent {
>> case this // “."
>> case up   // “..” 
>> case item(name: String, extension: String?)
>> }
>> 
>> public protocol Path {   
>> var components: Array { get }
>> init(_ components: Array) // used on protocol extensions 
>> that mutate paths, such as appending components
>> }
>> 
>> public struct AbsolutePath: Path { }
>> public struct RelativePath: Path { }
>> 
>> By separating out the concept of an Absolute and a Relative path, I can put 
>> additional functionality on each one to make semantic sense (you cannot 
>> concatenate two absolute paths, but you can concat any path with a relative 
>> path, for example). Or all file system operations must take an AbsolutePath. 
>> 
>> One of the key things I realized is that a “Path” type should not be 
>> ExpressibleByStringLiteral, because you cannot statically determine if a 
>> Path should be absolute or relative. However, one of the initializers for an 
>> AbsolutePath would handle things like expanding a tilde, and both types try 
>> to reduce a set of components as much as possible (by filtering out “.this” 
>> components, and handling “.up” components where possible, etc). Also in my 
>> experience, it’s fairly rare to want to deal with a known-at-compile-time, 
>> hard-coded path. Usually you’re dealing with paths relative to known 
>> “containers” that are determined at runtime (current user’s home folder, 
>> app’s sandboxed documents directory, etc).
>> 
>> Another thing I’ve done is that no direct file system operations exist on 
>> AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those 
>> are still on FileManager/FileHandle/etc in the form of extensions to handle 
>> the new types. In my app, a path is just a path, and it only has meaning 
>> based on the thing that is using it. An AbsolutePath for a URL is used 
>> differently than an AbsolutePath on a file system, although they are 
>> represented with the same “AbsolutePath” type.
>> 
>> I’m not saying this is a perfect API of course, or even that a hypothetical 
>> stdlib-provided Path should mimic this. I’m just saying that for my 
>> use-case, this has vastly simplified how I deal with paths, because both URL 
>> and String smell really bad for what I’m doing.
>> 
>> Dave
>> 
>>> On Aug 22, 2017, at 12:37 PM, Taylor Swift >> > wrote:
>>> 
>>> So are you saying we need three distinct “URI” types for local-absolute, 
>>> local-relative, and remote? That’s a lot of API surface to support.
>>> 
>>> On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong >> > wrote:
>>> I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
>>> abstraction for file system paths.
>>> 
>>> I maintain an app that deals a lot with file system paths, and using URL 
>>> has always felt cumbersome, but String is the absolute wrong type to use. 
>>> Lately as I’ve been working on it, I’ve been experimenting with a concrete 
>>> “Path” type, similar to PathKit (https://github.com/kylef/PathKit/ 
>>> ). Working in terms of AbsolutePath and 
>>> RelativePath (what I’ve been calling things) has been extremely refreshing, 
>>> because it allows me to better articulate the kind of data I’m dealing 
>>> with. URL doesn’t handle pure-relative paths very well, and it’s always a 
>>> bit of a mystery how resilient I need to be about checking .isFileURL or 
>>> whatever. All the extra properties (port, user, password, host) feel hugely 
>>> unnecessary as well.
>>> 
>>> Dave
>>> 
 On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
 > wrote:
 
 I'm not convinced that URLs are the appropriate abstraction for a file 
 system path. For the record, I'm not a fan of existing Foundation methods 
 that create objects from an URL. There is a useful and fundamental 
 difference between a local path and a remote path, and conflating the two 
 has 

Re: [swift-evolution] (core library) modern URL types

[Jakob Egger via swift-evolution]

I would absolutely love to see an API like AbsolutePath / RelativePath for file 
system operations!

> On 22. Aug 2017, at 21:02, Dave DeLong via swift-evolution 
>  wrote:
> 
> I suppose, if you squint at it weirdly.
> 
> My current Path API is a “Path” protocol, with “AbsolutePath” and 
> “RelativePath” struct versions. The protocol defines a path to be an array of 
> path components. The only real difference between an AbsolutePath and a 
> RelativePath is that all file system operations would only take an 
> AbsolutePath. A URL would also only provide an AbsolutePath as its “path” bit.
> 
> public enum PathComponent {
> case this // “."
> case up   // “..” 
> case item(name: String, extension: String?)
> }
> 
> public protocol Path {   
> var components: Array { get }
> init(_ components: Array) // used on protocol extensions 
> that mutate paths, such as appending components
> }
> 
> public struct AbsolutePath: Path { }
> public struct RelativePath: Path { }
> 
> By separating out the concept of an Absolute and a Relative path, I can put 
> additional functionality on each one to make semantic sense (you cannot 
> concatenate two absolute paths, but you can concat any path with a relative 
> path, for example). Or all file system operations must take an AbsolutePath. 
> 
> One of the key things I realized is that a “Path” type should not be 
> ExpressibleByStringLiteral, because you cannot statically determine if a Path 
> should be absolute or relative. However, one of the initializers for an 
> AbsolutePath would handle things like expanding a tilde, and both types try 
> to reduce a set of components as much as possible (by filtering out “.this” 
> components, and handling “.up” components where possible, etc). Also in my 
> experience, it’s fairly rare to want to deal with a known-at-compile-time, 
> hard-coded path. Usually you’re dealing with paths relative to known 
> “containers” that are determined at runtime (current user’s home folder, 
> app’s sandboxed documents directory, etc).
> 
> Another thing I’ve done is that no direct file system operations exist on 
> AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those are 
> still on FileManager/FileHandle/etc in the form of extensions to handle the 
> new types. In my app, a path is just a path, and it only has meaning based on 
> the thing that is using it. An AbsolutePath for a URL is used differently 
> than an AbsolutePath on a file system, although they are represented with the 
> same “AbsolutePath” type.
> 
> I’m not saying this is a perfect API of course, or even that a hypothetical 
> stdlib-provided Path should mimic this. I’m just saying that for my use-case, 
> this has vastly simplified how I deal with paths, because both URL and String 
> smell really bad for what I’m doing.
> 
> Dave
> 
>> On Aug 22, 2017, at 12:37 PM, Taylor Swift > > wrote:
>> 
>> So are you saying we need three distinct “URI” types for local-absolute, 
>> local-relative, and remote? That’s a lot of API surface to support.
>> 
>> On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong > > wrote:
>> I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
>> abstraction for file system paths.
>> 
>> I maintain an app that deals a lot with file system paths, and using URL has 
>> always felt cumbersome, but String is the absolute wrong type to use. Lately 
>> as I’ve been working on it, I’ve been experimenting with a concrete “Path” 
>> type, similar to PathKit (https://github.com/kylef/PathKit/ 
>> ). Working in terms of AbsolutePath and 
>> RelativePath (what I’ve been calling things) has been extremely refreshing, 
>> because it allows me to better articulate the kind of data I’m dealing with. 
>> URL doesn’t handle pure-relative paths very well, and it’s always a bit of a 
>> mystery how resilient I need to be about checking .isFileURL or whatever. 
>> All the extra properties (port, user, password, host) feel hugely 
>> unnecessary as well.
>> 
>> Dave
>> 
>>> On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
>>> > wrote:
>>> 
>>> I'm not convinced that URLs are the appropriate abstraction for a file 
>>> system path. For the record, I'm not a fan of existing Foundation methods 
>>> that create objects from an URL. There is a useful and fundamental 
>>> difference between a local path and a remote path, and conflating the two 
>>> has been a security pain point in many languages and frameworks that allow 
>>> it. Examples include remote file inclusion in PHP and malicious doctypes in 
>>> XML. Windows also had its share of issues with UNC paths.
>>> 
>>> Even when loading an arbitrary URL looks innocuous, many de-anonymizing 
>>> hacks work by causing a program to access an URL 

Re: [swift-evolution] (core library) modern URL types

[Karim Nassar via swift-evolution]

IMHO, The fact that URL has distinct interfaces for ‘scheme', ‘user', ‘host', 
‘port', ‘query', and ‘fragment’ (all of which are either irrelevant or invalid 
for addressing local files) should be an indication that it’s maybe not the 
best primary interface for directly modeling filesystem interactions.

To my mind, the fact that a ‘file’ scheme exists for URLs speaks more to the 
fact that "*when* you are operating on URLs, it’s *sometimes* useful to be able 
to talk about local files", as opposed to "*when* you talk about files, you 
should *always* use URLs".

OTOH, I really like the idea of a first-class file path abstraction. Seems like 
there are a lot of nice opportunities for pattern & traversal abstractions, 
uniformity, etc.

—Karim


> Date: Tue, 22 Aug 2017 09:24:57 -0700
> From: Dave DeLong <del...@apple.com>
> To: Félix Cloutier <felixclout...@icloud.com>
> Cc: Chris Lattner via swift-evolution <swift-evolution@swift.org>
> Subject: Re: [swift-evolution] (core library) modern URL types
> Message-ID: <5124d1ae-cb31-4298-9d52-12318bb7a...@apple.com>
> Content-Type: text/plain; charset="utf-8"
> 
> I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
> abstraction for file system paths.
> 
> I maintain an app that deals a lot with file system paths, and using URL has 
> always felt cumbersome, but String is the absolute wrong type to use. Lately 
> as I’ve been working on it, I’ve been experimenting with a concrete “Path” 
> type, similar to PathKit (https://github.com/kylef/PathKit/ 
> <https://github.com/kylef/PathKit/>). Working in terms of AbsolutePath and 
> RelativePath (what I’ve been calling things) has been extremely refreshing, 
> because it allows me to better articulate the kind of data I’m dealing with. 
> URL doesn’t handle pure-relative paths very well, and it’s always a bit of a 
> mystery how resilient I need to be about checking .isFileURL or whatever. All 
> the extra properties (port, user, password, host) feel hugely unnecessary as 
> well.
> 
> Dave
> 
>> On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
>> <swift-evolution@swift.org> wrote:
>> 
>> I'm not convinced that URLs are the appropriate abstraction for a file 
>> system path. For the record, I'm not a fan of existing Foundation methods 
>> that create objects from an URL. There is a useful and fundamental 
>> difference between a local path and a remote path, and conflating the two 
>> has been a security pain point in many languages and frameworks that allow 
>> it. Examples include remote file inclusion in PHP and malicious doctypes in 
>> XML. Windows also had its share of issues with UNC paths.
>> 
>> Even when loading an arbitrary URL looks innocuous, many de-anonymizing 
>> hacks work by causing a program to access an URL controlled by an attacker 
>> to make it disclose the user's IP address or some other identifier.
>> 
>> IMO, this justifies that there should be separate types to handle local and 
>> remote resources, so that at least developers have to be explicit about 
>> allowing remote resources. This makes a new URL type less necessary towards 
>> supporting file I/O.
>> 
>> Félix
>> 
>>> Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution 
>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :
>>> 
>>> Okay so a few days ago there was a discussion 
>>> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170814/038923.html>
>>>  about getting pure swift file system support into Foundation or another 
>>> core library, and in my opinion, doing this requires a total overhaul of 
>>> the `URL` type (which is currently little more than a wrapper for NSURL), 
>>> so I’ve just started a pure Swift URL library project at 
>>> <https://github.com/kelvin13/url <https://github.com/kelvin13/url>>.
>>> 
>>> The library’s parsing and validation core (~1K loc pure swift) is already 
>>> in place and functional; the goal is to eventually support all of the 
>>> Foundation URL functionality.
>>> 
>>> The new `URL` type is implemented as a value type with utf8 storage backed 
>>> by an array buffer. The URLs are just 56 bytes long each, so they should be 
>>> able to fit into cache lines. (NSURL by comparison is over 128 bytes in 
>>> size; it’s only saved by the fact that the thing is passed as a reference 
>>> type.)
>>> 
>>> As I said, this is still really early on and not a mature library at all 
>>> but everyone is invited t

Re: [swift-evolution] (core library) modern URL types

[Dave DeLong via swift-evolution]

Yep, I agree. My early thoughts on this included ideas around a “Container” for 
a path, but I ended up at the point that it wasn’t a useful abstraction (that I 
could see), because a container was really just a different AbsolutePath 
“prefix”.

I toyed around with the idea of something like:

protocol PathContainer: Path { }

struct ContainedPath: Path { … }

I ultimately decided against it because, for example, a “Documents” container 
doesn’t make sense, because there can be many Documents folders, and having the 
static typing for this sort of thing tended to fly in the face of the dynamic 
nature of the file system. Ultimately, the simple distinction between an 
AbsolutePath and a RelativePath felt like the right sort of balance.

My “AbsolutePath” type does have some static properties to some well-known 
“container” locations. For example:

public struct AbsolutePath: Path {

public static let root = AbsolutePath([])
public static let temporaryDirectory = AbsolutePath(fileSystemPath: 
NSTemporaryDirectory())
public static var applicationCacheDirectory: AbsolutePath { return ... }
public static var applicationSupportDirectory: AbsolutePath { return ... }

}

But, I agree that server needs are probably quite different from app needs.

Dave

> On Aug 22, 2017, at 1:52 PM, Félix Cloutier  wrote:
> 
> Just leaving it out here that the iOS/macOS app experience with paths is 
> likely to be very different from the server (or system program) experience. 
> Most of your paths are relative to a container because the effective root of 
> your data is the container. I would tend to believe that a lot of your paths, 
> not while fully static, could be expressed with something like 
> "${DOCUMENTS}/static/part", where ${DOCUMENTS} is the only thing that 
> effectively changes. A system-level program just happens to use / as its 
> prefix.
> 
> Most platforms already have some notion of file domains, which are really 
> just path prefixes (the user's home folder, the file system root, the network 
> root, etc). I think that some level of support for these would be desirable 
> (would it be only a set of functions that return different path prefixes). 
> It's also my humble opinion that tilde expansion should be part of a larger 
> shell expansion feature to avoid surprises.
> 
> I think that I support the conclusion.
> 
> Félix
> 
> Le 22 août 2017 à 12:02, Dave DeLong  a écrit :
> 
>> I suppose, if you squint at it weirdly.
>> 
>> My current Path API is a “Path” protocol, with “AbsolutePath” and 
>> “RelativePath” struct versions. The protocol defines a path to be an array 
>> of path components. The only real difference between an AbsolutePath and a 
>> RelativePath is that all file system operations would only take an 
>> AbsolutePath. A URL would also only provide an AbsolutePath as its “path” 
>> bit.
>> 
>> public enum PathComponent {
>> case this // “."
>> case up   // “..” 
>> case item(name: String, extension: String?)
>> }
>> 
>> public protocol Path {   
>> var components: Array { get }
>> init(_ components: Array) // used on protocol extensions 
>> that mutate paths, such as appending components
>> }
>> 
>> public struct AbsolutePath: Path { }
>> public struct RelativePath: Path { }
>> 
>> By separating out the concept of an Absolute and a Relative path, I can put 
>> additional functionality on each one to make semantic sense (you cannot 
>> concatenate two absolute paths, but you can concat any path with a relative 
>> path, for example). Or all file system operations must take an AbsolutePath. 
>> 
>> One of the key things I realized is that a “Path” type should not be 
>> ExpressibleByStringLiteral, because you cannot statically determine if a 
>> Path should be absolute or relative. However, one of the initializers for an 
>> AbsolutePath would handle things like expanding a tilde, and both types try 
>> to reduce a set of components as much as possible (by filtering out “.this” 
>> components, and handling “.up” components where possible, etc). Also in my 
>> experience, it’s fairly rare to want to deal with a known-at-compile-time, 
>> hard-coded path. Usually you’re dealing with paths relative to known 
>> “containers” that are determined at runtime (current user’s home folder, 
>> app’s sandboxed documents directory, etc).
>> 
>> Another thing I’ve done is that no direct file system operations exist on 
>> AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those 
>> are still on FileManager/FileHandle/etc in the form of extensions to handle 
>> the new types. In my app, a path is just a path, and it only has meaning 
>> based on the thing that is using it. An AbsolutePath for a URL is used 
>> differently than an AbsolutePath on a file system, although they are 
>> represented with the same “AbsolutePath” type.
>> 
>> I’m not saying this is a perfect API of course, or even that a hypothetical 

Re: [swift-evolution] (core library) modern URL types

[Félix Cloutier via swift-evolution]

Just leaving it out here that the iOS/macOS app experience with paths is likely to be 
very different from the server (or system program) experience. Most of your paths are 
relative to a container because the effective root of your data is the container. I would 
tend to believe that a lot of your paths, not while fully static, could be expressed with 
something like "${DOCUMENTS}/static/part", where ${DOCUMENTS} is the only thing 
that effectively changes. A system-level program just happens to use / as its prefix.

Most platforms already have some notion of file domains, which are really just 
path prefixes (the user's home folder, the file system root, the network root, 
etc). I think that some level of support for these would be desirable (would it 
be only a set of functions that return different path prefixes). It's also my 
humble opinion that tilde expansion should be part of a larger shell expansion 
feature to avoid surprises.

I think that I support the conclusion.

Félix

Le 22 août 2017 à 12:02, Dave DeLong  a écrit :

I suppose, if you squint at it weirdly.

My current Path API is a “Path” protocol, with “AbsolutePath” and 
“RelativePath” struct versions. The protocol defines a path to be an array of 
path components. The only real difference between an AbsolutePath and a 
RelativePath is that all file system operations would only take an 
AbsolutePath. A URL would also only provide an AbsolutePath as its “path” bit.

public enum PathComponent {
    case this // “."
    case up   // “..” 
    case item(name: String, extension: String?)
}

public protocol Path {   
    var components: Array { get }
    init(_ components: Array) // used on protocol extensions 
that mutate paths, such as appending components
}

public struct AbsolutePath: Path { }
public struct RelativePath: Path { }

By separating out the concept of an Absolute and a Relative path, I can put 
additional functionality on each one to make semantic sense (you cannot 
concatenate two absolute paths, but you can concat any path with a relative 
path, for example). Or all file system operations must take an AbsolutePath. 

One of the key things I realized is that a “Path” type should not be 
ExpressibleByStringLiteral, because you cannot statically determine if a Path 
should be absolute or relative. However, one of the initializers for an 
AbsolutePath would handle things like expanding a tilde, and both types try to 
reduce a set of components as much as possible (by filtering out “.this” 
components, and handling “.up” components where possible, etc). Also in my 
experience, it’s fairly rare to want to deal with a known-at-compile-time, 
hard-coded path. Usually you’re dealing with paths relative to known 
“containers” that are determined at runtime (current user’s home folder, app’s 
sandboxed documents directory, etc).

Another thing I’ve done is that no direct file system operations exist on 
AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those are 
still on FileManager/FileHandle/etc in the form of extensions to handle the new 
types. In my app, a path is just a path, and it only has meaning based on the 
thing that is using it. An AbsolutePath for a URL is used differently than an 
AbsolutePath on a file system, although they are represented with the same 
“AbsolutePath” type.

I’m not saying this is a perfect API of course, or even that a hypothetical 
stdlib-provided Path should mimic this. I’m just saying that for my use-case, 
this has vastly simplified how I deal with paths, because both URL and String 
smell really bad for what I’m doing.

Dave

On Aug 22, 2017, at 12:37 PM, Taylor Swift  wrote:
So are you saying we need three distinct “URI” types for local-absolute, 
local-relative, and remote? That’s a lot of API surface to support.

On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong  wrote:
I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
abstraction for file system paths.

I maintain an app that deals a lot with file system paths, and using URL has 
always felt cumbersome, but String is the absolute wrong type to use. Lately as 
I’ve been working on it, I’ve been experimenting with a concrete “Path” type, 
similar to PathKit (https://github.com/kylef/PathKit/). Working in terms of 
AbsolutePath and RelativePath (what I’ve been calling things) has been 
extremely refreshing, because it allows me to better articulate the kind of 
data I’m dealing with. URL doesn’t handle pure-relative paths very well, and 
it’s always a bit of a mystery how resilient I need to be about checking 
.isFileURL or whatever. All the extra properties (port, user, password, host) 
feel hugely unnecessary as well.

Dave

On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
 wrote:

I'm not convinced that URLs are the appropriate abstraction for a file system 
path. For the record, I'm not a fan of existing 

Re: [swift-evolution] (core library) modern URL types

[Dave DeLong via swift-evolution]

I suppose, if you squint at it weirdly.

My current Path API is a “Path” protocol, with “AbsolutePath” and 
“RelativePath” struct versions. The protocol defines a path to be an array of 
path components. The only real difference between an AbsolutePath and a 
RelativePath is that all file system operations would only take an 
AbsolutePath. A URL would also only provide an AbsolutePath as its “path” bit.

public enum PathComponent {
case this // “."
case up   // “..” 
case item(name: String, extension: String?)
}

public protocol Path {   
var components: Array { get }
init(_ components: Array) // used on protocol extensions 
that mutate paths, such as appending components
}

public struct AbsolutePath: Path { }
public struct RelativePath: Path { }

By separating out the concept of an Absolute and a Relative path, I can put 
additional functionality on each one to make semantic sense (you cannot 
concatenate two absolute paths, but you can concat any path with a relative 
path, for example). Or all file system operations must take an AbsolutePath. 

One of the key things I realized is that a “Path” type should not be 
ExpressibleByStringLiteral, because you cannot statically determine if a Path 
should be absolute or relative. However, one of the initializers for an 
AbsolutePath would handle things like expanding a tilde, and both types try to 
reduce a set of components as much as possible (by filtering out “.this” 
components, and handling “.up” components where possible, etc). Also in my 
experience, it’s fairly rare to want to deal with a known-at-compile-time, 
hard-coded path. Usually you’re dealing with paths relative to known 
“containers” that are determined at runtime (current user’s home folder, app’s 
sandboxed documents directory, etc).

Another thing I’ve done is that no direct file system operations exist on 
AbsolutePath (like “.exists” or “.createDirectory(…)” or whatever); those are 
still on FileManager/FileHandle/etc in the form of extensions to handle the new 
types. In my app, a path is just a path, and it only has meaning based on the 
thing that is using it. An AbsolutePath for a URL is used differently than an 
AbsolutePath on a file system, although they are represented with the same 
“AbsolutePath” type.

I’m not saying this is a perfect API of course, or even that a hypothetical 
stdlib-provided Path should mimic this. I’m just saying that for my use-case, 
this has vastly simplified how I deal with paths, because both URL and String 
smell really bad for what I’m doing.

Dave

> On Aug 22, 2017, at 12:37 PM, Taylor Swift  wrote:
> 
> So are you saying we need three distinct “URI” types for local-absolute, 
> local-relative, and remote? That’s a lot of API surface to support.
> 
> On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong  > wrote:
> I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
> abstraction for file system paths.
> 
> I maintain an app that deals a lot with file system paths, and using URL has 
> always felt cumbersome, but String is the absolute wrong type to use. Lately 
> as I’ve been working on it, I’ve been experimenting with a concrete “Path” 
> type, similar to PathKit (https://github.com/kylef/PathKit/ 
> ). Working in terms of AbsolutePath and 
> RelativePath (what I’ve been calling things) has been extremely refreshing, 
> because it allows me to better articulate the kind of data I’m dealing with. 
> URL doesn’t handle pure-relative paths very well, and it’s always a bit of a 
> mystery how resilient I need to be about checking .isFileURL or whatever. All 
> the extra properties (port, user, password, host) feel hugely unnecessary as 
> well.
> 
> Dave
> 
>> On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
>> > wrote:
>> 
>> I'm not convinced that URLs are the appropriate abstraction for a file 
>> system path. For the record, I'm not a fan of existing Foundation methods 
>> that create objects from an URL. There is a useful and fundamental 
>> difference between a local path and a remote path, and conflating the two 
>> has been a security pain point in many languages and frameworks that allow 
>> it. Examples include remote file inclusion in PHP and malicious doctypes in 
>> XML. Windows also had its share of issues with UNC paths.
>> 
>> Even when loading an arbitrary URL looks innocuous, many de-anonymizing 
>> hacks work by causing a program to access an URL controlled by an attacker 
>> to make it disclose the user's IP address or some other identifier.
>> 
>> IMO, this justifies that there should be separate types to handle local and 
>> remote resources, so that at least developers have to be explicit about 
>> allowing remote resources. This makes a new URL type less necessary towards 
>> supporting file I/O.
>> 
>> Félix
>> 
>>> Le 

Re: [swift-evolution] (core library) modern URL types

[Robert Bennett via swift-evolution]

Since an absolute URI is still relative to root, I feel like there should be a 
way to finagle relative and absolute URIs into a single type.

> On Aug 22, 2017, at 2:37 PM, Taylor Swift via swift-evolution 
>  wrote:
> 
> So are you saying we need three distinct “URI” types for local-absolute, 
> local-relative, and remote? That’s a lot of API surface to support.
> 
>> On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong  wrote:
>> I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
>> abstraction for file system paths.
>> 
>> I maintain an app that deals a lot with file system paths, and using URL has 
>> always felt cumbersome, but String is the absolute wrong type to use. Lately 
>> as I’ve been working on it, I’ve been experimenting with a concrete “Path” 
>> type, similar to PathKit (https://github.com/kylef/PathKit/). Working in 
>> terms of AbsolutePath and RelativePath (what I’ve been calling things) has 
>> been extremely refreshing, because it allows me to better articulate the 
>> kind of data I’m dealing with. URL doesn’t handle pure-relative paths very 
>> well, and it’s always a bit of a mystery how resilient I need to be about 
>> checking .isFileURL or whatever. All the extra properties (port, user, 
>> password, host) feel hugely unnecessary as well.
>> 
>> Dave
>> 
>>> On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
>>>  wrote:
>>> 
>>> I'm not convinced that URLs are the appropriate abstraction for a file 
>>> system path. For the record, I'm not a fan of existing Foundation methods 
>>> that create objects from an URL. There is a useful and fundamental 
>>> difference between a local path and a remote path, and conflating the two 
>>> has been a security pain point in many languages and frameworks that allow 
>>> it. Examples include remote file inclusion in PHP and malicious doctypes in 
>>> XML. Windows also had its share of issues with UNC paths.
>>> 
>>> Even when loading an arbitrary URL looks innocuous, many de-anonymizing 
>>> hacks work by causing a program to access an URL controlled by an attacker 
>>> to make it disclose the user's IP address or some other identifier.
>>> 
>>> IMO, this justifies that there should be separate types to handle local and 
>>> remote resources, so that at least developers have to be explicit about 
>>> allowing remote resources. This makes a new URL type less necessary towards 
>>> supporting file I/O.
>>> 
>>> Félix
>>> 
 Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution 
  a écrit :
 
 Okay so a few days ago there was a discussion about getting pure swift 
 file system support into Foundation or another core library, and in my 
 opinion, doing this requires a total overhaul of the `URL` type (which is 
 currently little more than a wrapper for NSURL), so I’ve just started a 
 pure Swift URL library project at .
 
 The library’s parsing and validation core (~1K loc pure swift) is already 
 in place and functional; the goal is to eventually support all of the 
 Foundation URL functionality.
 
 The new `URL` type is implemented as a value type with utf8 storage backed 
 by an array buffer. The URLs are just 56 bytes long each, so they should 
 be able to fit into cache lines. (NSURL by comparison is over 128 bytes in 
 size; it’s only saved by the fact that the thing is passed as a reference 
 type.)
 
 As I said, this is still really early on and not a mature library at all 
 but everyone is invited to observe, provide feedback, or contribute!
 ___
 swift-evolution mailing list
 swift-evolution@swift.org
 https://lists.swift.org/mailman/listinfo/swift-evolution
>>> 
>>> ___
>>> swift-evolution mailing list
>>> swift-evolution@swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>> 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Taylor Swift via swift-evolution]

So are you saying we need *three* distinct “URI” types for local-absolute,
local-relative, and remote? That’s a lot of API surface to support.

On Tue, Aug 22, 2017 at 12:24 PM, Dave DeLong  wrote:

> I completely agree. URL packs a lot of punch, but IMO it’s the wrong
> abstraction for file system paths.
>
> I maintain an app that deals a *lot* with file system paths, and using
> URL has always felt cumbersome, but String is the absolute wrong type to
> use. Lately as I’ve been working on it, I’ve been experimenting with a
> concrete “Path” type, similar to PathKit (https://github.com/kylef/
> PathKit/). Working in terms of AbsolutePath and RelativePath (what I’ve
> been calling things) has been *extremely* refreshing, because it allows
> me to better articulate the kind of data I’m dealing with. URL doesn’t
> handle pure-relative paths very well, and it’s always a bit of a mystery
> how resilient I need to be about checking .isFileURL or whatever. All the
> extra properties (port, user, password, host) feel hugely unnecessary as
> well.
>
> Dave
>
> On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> I'm not convinced that URLs are the appropriate abstraction for a file
> system path. For the record, I'm not a fan of existing Foundation methods
> that create objects from an URL. There is a useful and fundamental
> difference between a local path and a remote path, and conflating the two
> has been a security pain point in many languages and frameworks that allow
> it. Examples include remote file inclusion in PHP and malicious doctypes in
> XML. Windows also had its share of issues with UNC paths.
>
> Even when loading an arbitrary URL looks innocuous, many de-anonymizing
> hacks work by causing a program to access an URL controlled by an attacker
> to make it disclose the user's IP address or some other identifier.
>
> IMO, this justifies that there should be separate types to handle local
> and remote resources, so that at least developers have to be explicit about
> allowing remote resources. This makes a new URL type less necessary towards
> supporting file I/O.
>
> Félix
>
> Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution <
> swift-evolution@swift.org> a écrit :
>
> Okay so a few days ago there was a discussion
> 
> about getting pure swift file system support into Foundation or another
> core library, and in my opinion, doing this requires a total overhaul of
> the `URL` type (which is currently little more than a wrapper for NSURL),
> so I’ve just started a pure Swift URL library project at <
> https://github.com/kelvin13/url>.
>
> The library’s parsing and validation core (~1K loc pure swift) is already
> in place and functional; the goal is to eventually support all of the
> Foundation URL functionality.
>
> The new `URL` type is implemented as a value type with utf8 storage backed
> by an array buffer. The URLs are just 56 bytes long each, so they should be
> able to fit into cache lines. (NSURL by comparison is over 128 bytes in
> size; it’s only saved by the fact that the thing is passed as a reference
> type.)
>
> As I said, this is still really early on and not a mature library at all
> but everyone is invited to observe, provide feedback, or contribute!
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Jacob Williams via swift-evolution]

There’s also a library somewhat similar to PathKit called FileKit 
(https://github.com/nvzqz/FileKit ). FileKit 
has some functionality I’m not sure how I feel about, such as different types 
based on the type of data in the file. It also uses a lot of Foundation for 
NSBacked types which breaks Linux compatibility. 

I do agree that URL is not the best option for file paths, despite many of the 
similarities with URL paths, and agree that much of Foundation should be ported 
to pure Swift in order to increase linux compatibility. 

> On Aug 22, 2017, at 10:25 AM, Dave DeLong via swift-evolution 
>  wrote:
> 
> I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
> abstraction for file system paths.
> 
> I maintain an app that deals a lot with file system paths, and using URL has 
> always felt cumbersome, but String is the absolute wrong type to use. Lately 
> as I’ve been working on it, I’ve been experimenting with a concrete “Path” 
> type, similar to PathKit (https://github.com/kylef/PathKit/ 
> ). Working in terms of AbsolutePath and 
> RelativePath (what I’ve been calling things) has been extremely refreshing, 
> because it allows me to better articulate the kind of data I’m dealing with. 
> URL doesn’t handle pure-relative paths very well, and it’s always a bit of a 
> mystery how resilient I need to be about checking .isFileURL or whatever. All 
> the extra properties (port, user, password, host) feel hugely unnecessary as 
> well.
> 
> Dave
> 
>> On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
>> > wrote:
>> 
>> I'm not convinced that URLs are the appropriate abstraction for a file 
>> system path. For the record, I'm not a fan of existing Foundation methods 
>> that create objects from an URL. There is a useful and fundamental 
>> difference between a local path and a remote path, and conflating the two 
>> has been a security pain point in many languages and frameworks that allow 
>> it. Examples include remote file inclusion in PHP and malicious doctypes in 
>> XML. Windows also had its share of issues with UNC paths.
>> 
>> Even when loading an arbitrary URL looks innocuous, many de-anonymizing 
>> hacks work by causing a program to access an URL controlled by an attacker 
>> to make it disclose the user's IP address or some other identifier.
>> 
>> IMO, this justifies that there should be separate types to handle local and 
>> remote resources, so that at least developers have to be explicit about 
>> allowing remote resources. This makes a new URL type less necessary towards 
>> supporting file I/O.
>> 
>> Félix
>> 
>>> Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution 
>>> > a écrit :
>>> 
>>> Okay so a few days ago there was a discussion 
>>> 
>>>  about getting pure swift file system support into Foundation or another 
>>> core library, and in my opinion, doing this requires a total overhaul of 
>>> the `URL` type (which is currently little more than a wrapper for NSURL), 
>>> so I’ve just started a pure Swift URL library project at 
>>> >.
>>> 
>>> The library’s parsing and validation core (~1K loc pure swift) is already 
>>> in place and functional; the goal is to eventually support all of the 
>>> Foundation URL functionality.
>>> 
>>> The new `URL` type is implemented as a value type with utf8 storage backed 
>>> by an array buffer. The URLs are just 56 bytes long each, so they should be 
>>> able to fit into cache lines. (NSURL by comparison is over 128 bytes in 
>>> size; it’s only saved by the fact that the thing is passed as a reference 
>>> type.)
>>> 
>>> As I said, this is still really early on and not a mature library at all 
>>> but everyone is invited to observe, provide feedback, or contribute!
>>> ___
>>> swift-evolution mailing list
>>> swift-evolution@swift.org 
>>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>>> 
>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Dave DeLong via swift-evolution]

I completely agree. URL packs a lot of punch, but IMO it’s the wrong 
abstraction for file system paths.

I maintain an app that deals a lot with file system paths, and using URL has 
always felt cumbersome, but String is the absolute wrong type to use. Lately as 
I’ve been working on it, I’ve been experimenting with a concrete “Path” type, 
similar to PathKit (https://github.com/kylef/PathKit/ 
). Working in terms of AbsolutePath and 
RelativePath (what I’ve been calling things) has been extremely refreshing, 
because it allows me to better articulate the kind of data I’m dealing with. 
URL doesn’t handle pure-relative paths very well, and it’s always a bit of a 
mystery how resilient I need to be about checking .isFileURL or whatever. All 
the extra properties (port, user, password, host) feel hugely unnecessary as 
well.

Dave

> On Aug 20, 2017, at 11:23 PM, Félix Cloutier via swift-evolution 
>  wrote:
> 
> I'm not convinced that URLs are the appropriate abstraction for a file system 
> path. For the record, I'm not a fan of existing Foundation methods that 
> create objects from an URL. There is a useful and fundamental difference 
> between a local path and a remote path, and conflating the two has been a 
> security pain point in many languages and frameworks that allow it. Examples 
> include remote file inclusion in PHP and malicious doctypes in XML. Windows 
> also had its share of issues with UNC paths.
> 
> Even when loading an arbitrary URL looks innocuous, many de-anonymizing hacks 
> work by causing a program to access an URL controlled by an attacker to make 
> it disclose the user's IP address or some other identifier.
> 
> IMO, this justifies that there should be separate types to handle local and 
> remote resources, so that at least developers have to be explicit about 
> allowing remote resources. This makes a new URL type less necessary towards 
> supporting file I/O.
> 
> Félix
> 
>> Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution 
>> > a écrit :
>> 
>> Okay so a few days ago there was a discussion 
>> 
>>  about getting pure swift file system support into Foundation or another 
>> core library, and in my opinion, doing this requires a total overhaul of the 
>> `URL` type (which is currently little more than a wrapper for NSURL), so 
>> I’ve just started a pure Swift URL library project at 
>> >.
>> 
>> The library’s parsing and validation core (~1K loc pure swift) is already in 
>> place and functional; the goal is to eventually support all of the 
>> Foundation URL functionality.
>> 
>> The new `URL` type is implemented as a value type with utf8 storage backed 
>> by an array buffer. The URLs are just 56 bytes long each, so they should be 
>> able to fit into cache lines. (NSURL by comparison is over 128 bytes in 
>> size; it’s only saved by the fact that the thing is passed as a reference 
>> type.)
>> 
>> As I said, this is still really early on and not a mature library at all but 
>> everyone is invited to observe, provide feedback, or contribute!
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Félix Cloutier via swift-evolution]

It could be problematic to initialize, however. You would need to know Scheme 
in URL ahead of initialization time for at least two reasons: first 
because value types aren't polymorphic, and second because the size of a type 
can depend on its generic parameters.

(If we have a good solution for both, then that would be great, yes.)

Félix

> Le 21 août 2017 à 09:28, Robert Bennett  a écrit :
> 
> If value-based generics become a thing, then you’ll be able to specify a URL 
> type with URL<.file> which would pretty much solve this problem.
> 
> On Aug 21, 2017, at 11:24 AM, Félix Cloutier via swift-evolution 
> > wrote:
> 
>> There's no question that there's a lot in common between file: URLs and 
>> other URLs, but that hardly makes URLs better in the file: case.
>> 
>> I saw the enum. The problem remains that it's a common API principle to 
>> accept the broadest possible input. It's not fundamentally wrong to accept 
>> and resolve common URL types, as there's a ton of things that need to read 
>> documents from the Internet by design. However, if this is the default 
>> facility for file handling too, it invents either:
>> 
>> A responsibility for API users to check that their URL is a file: URL;
>> A responsibility for API authors to provide separate entry points for file 
>> URLs and remote URLs, and a responsibility for API users to use the right 
>> one.
>> 
>> It would also add a category of errors to common filesystem operations: 
>> "can't do this because the URL is not a file: URL", and a category of 
>> questions that we arguably shouldn't need to ask ourselves. For instance, 
>> what is the correct result of glob()ing a file: URL pattern with a hash or a 
>> query string, should each element include that hash/query string too?
>> 
>> Félix
>> 
>>> Le 20 août 2017 à 23:33, Taylor Swift >> > a écrit :
>>> 
>>> I think that’s more a problem that lies in Foundation methods that take 
>>> URLs rather than the URLs themselves. A URL is a useful abstraction because 
>>> it contains a lot of common functionality between local file paths and 
>>> internet resources. For example, relative URI reference resolution. APIs 
>>> which take URLs as arguments should be responsible for ensuring that the 
>>> URL’s schema is a `file:`. The new URL type I’m writing actually makes the 
>>> scheme an enum with cases `.file`, `.http`, `.https`, `.ftp`, and `.data` 
>>> to ease checking this.
>>> 
>>> On Mon, Aug 21, 2017 at 2:23 AM, Félix Cloutier >> > wrote:
>>> I'm not convinced that URLs are the appropriate abstraction for a file 
>>> system path. For the record, I'm not a fan of existing Foundation methods 
>>> that create objects from an URL. There is a useful and fundamental 
>>> difference between a local path and a remote path, and conflating the two 
>>> has been a security pain point in many languages and frameworks that allow 
>>> it. Examples include remote file inclusion in PHP and malicious doctypes in 
>>> XML. Windows also had its share of issues with UNC paths.
>>> 
>>> Even when loading an arbitrary URL looks innocuous, many de-anonymizing 
>>> hacks work by causing a program to access an URL controlled by an attacker 
>>> to make it disclose the user's IP address or some other identifier.
>>> 
>>> IMO, this justifies that there should be separate types to handle local and 
>>> remote resources, so that at least developers have to be explicit about 
>>> allowing remote resources. This makes a new URL type less necessary towards 
>>> supporting file I/O.
>>> 
>>> Félix
>>> 
 Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution 
 > a écrit :
 
 Okay so a few days ago there was a discussion 
 
  about getting pure swift file system support into Foundation or another 
 core library, and in my opinion, doing this requires a total overhaul of 
 the `URL` type (which is currently little more than a wrapper for NSURL), 
 so I’ve just started a pure Swift URL library project at 
 >.
 
 The library’s parsing and validation core (~1K loc pure swift) is already 
 in place and functional; the goal is to eventually support all of the 
 Foundation URL functionality.
 
 The new `URL` type is implemented as a value type with utf8 storage backed 
 by an array buffer. The URLs are just 56 bytes long each, so they should 
 be able to fit into cache lines. (NSURL by comparison is over 128 bytes in 
 size; it’s only saved by the fact that the thing is passed as a reference 
 type.)
 
 As I said, this is still really early on 

Re: [swift-evolution] (core library) modern URL types

[Taylor Swift via swift-evolution]

that might be true but I’d need to hear from more people to justify
creating two very similar URI types. Technically `file:` is just a string
that the file system interpreter gives meaning to; the scheme of a URL can
be any alphanumeric string. For example you could write a file manager
which interprets "foo:" as a local address, or ignore the scheme entirely.

On Mon, Aug 21, 2017 at 11:23 AM, Félix Cloutier 
wrote:

> There's no question that there's a lot in common between file: URLs and
> other URLs, but that hardly makes URLs better in the file: case.
>
> I saw the enum. The problem remains that it's a common API principle to
> accept the broadest possible input. It's not fundamentally wrong to accept
> and resolve common URL types, as there's a ton of things that need to read
> documents from the Internet by design. However, if this is the default
> facility for file handling too, it invents either:
>
>
>- A responsibility for API users to check that their URL is a file:
>URL;
>- A responsibility for API authors to provide separate entry points
>for file URLs and remote URLs, and a responsibility for API users to use
>the right one.
>
>
> It would also add a category of errors to common filesystem operations:
> "can't do this because the URL is not a file: URL", and a category of
> questions that we arguably shouldn't need to ask ourselves. For instance,
> what is the correct result of glob()ing a file: URL pattern with a hash or
> a query string, should each element include that hash/query string too?
>
> Félix
>
> Le 20 août 2017 à 23:33, Taylor Swift  a écrit :
>
> I think that’s more a problem that lies in Foundation methods that take
> URLs rather than the URLs themselves. A URL is a useful abstraction because
> it contains a lot of common functionality between local file paths and
> internet resources. For example, relative URI reference resolution. APIs
> which take URLs as arguments should be responsible for ensuring that the
> URL’s schema is a `file:`. The new URL type I’m writing actually makes
> the scheme an enum with cases `.file`, `.http`, `.https`, `.ftp`, and `
> .data` to ease checking this.
>
> On Mon, Aug 21, 2017 at 2:23 AM, Félix Cloutier 
> wrote:
>
>> I'm not convinced that URLs are the appropriate abstraction for a file
>> system path. For the record, I'm not a fan of existing Foundation methods
>> that create objects from an URL. There is a useful and fundamental
>> difference between a local path and a remote path, and conflating the two
>> has been a security pain point in many languages and frameworks that allow
>> it. Examples include remote file inclusion in PHP and malicious doctypes in
>> XML. Windows also had its share of issues with UNC paths.
>>
>> Even when loading an arbitrary URL looks innocuous, many de-anonymizing
>> hacks work by causing a program to access an URL controlled by an attacker
>> to make it disclose the user's IP address or some other identifier.
>>
>> IMO, this justifies that there should be separate types to handle local
>> and remote resources, so that at least developers have to be explicit about
>> allowing remote resources. This makes a new URL type less necessary towards
>> supporting file I/O.
>>
>> Félix
>>
>> Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution <
>> swift-evolution@swift.org> a écrit :
>>
>> Okay so a few days ago there was a discussion
>> 
>> about getting pure swift file system support into Foundation or another
>> core library, and in my opinion, doing this requires a total overhaul of
>> the `URL` type (which is currently little more than a wrapper for
>> NSURL), so I’ve just started a pure Swift URL library project at <
>> https://github.com/kelvin13/url>.
>>
>> The library’s parsing and validation core (~1K loc pure swift) is already
>> in place and functional; the goal is to eventually support all of the
>> Foundation URL functionality.
>>
>> The new `URL` type is implemented as a value type with utf8 storage
>> backed by an array buffer. The URLs are just 56 bytes long each, so they
>> should be able to fit into cache lines. (NSURL by comparison is over 128
>> bytes in size; it’s only saved by the fact that the thing is passed as a
>> reference type.)
>>
>> As I said, this is still really early on and not a mature library at all
>> but everyone is invited to observe, provide feedback, or contribute!
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
>>
>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Charles Srstka via swift-evolution]

> On Aug 21, 2017, at 11:29 AM, Robert Bennett via swift-evolution 
>  wrote:
> 
> If value-based generics become a thing, then you’ll be able to specify a URL 
> type with URL<.file> which would pretty much solve this problem.


And then you could make APIs that are specific to certain schemes, and maybe 
even have file references working again.

extension URL where Scheme == .file {
var path: String { … } // non-optional
}

extension URL where Scheme == .fileReference {
var path: String? { … } // optional
}

Charles

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Robert Bennett via swift-evolution]

If value-based generics become a thing, then you’ll be able to specify a URL 
type with URL<.file> which would pretty much solve this problem.

> On Aug 21, 2017, at 11:24 AM, Félix Cloutier via swift-evolution 
>  wrote:
> 
> There's no question that there's a lot in common between file: URLs and other 
> URLs, but that hardly makes URLs better in the file: case.
> 
> I saw the enum. The problem remains that it's a common API principle to 
> accept the broadest possible input. It's not fundamentally wrong to accept 
> and resolve common URL types, as there's a ton of things that need to read 
> documents from the Internet by design. However, if this is the default 
> facility for file handling too, it invents either:
> 
> A responsibility for API users to check that their URL is a file: URL;
> A responsibility for API authors to provide separate entry points for file 
> URLs and remote URLs, and a responsibility for API users to use the right one.
> 
> It would also add a category of errors to common filesystem operations: 
> "can't do this because the URL is not a file: URL", and a category of 
> questions that we arguably shouldn't need to ask ourselves. For instance, 
> what is the correct result of glob()ing a file: URL pattern with a hash or a 
> query string, should each element include that hash/query string too?
> 
> Félix
> 
>> Le 20 août 2017 à 23:33, Taylor Swift  a écrit :
>> 
>> I think that’s more a problem that lies in Foundation methods that take URLs 
>> rather than the URLs themselves. A URL is a useful abstraction because it 
>> contains a lot of common functionality between local file paths and internet 
>> resources. For example, relative URI reference resolution. APIs which take 
>> URLs as arguments should be responsible for ensuring that the URL’s schema 
>> is a `file:`. The new URL type I’m writing actually makes the scheme an enum 
>> with cases `.file`, `.http`, `.https`, `.ftp`, and `.data` to ease checking 
>> this.
>> 
>>> On Mon, Aug 21, 2017 at 2:23 AM, Félix Cloutier  
>>> wrote:
>>> I'm not convinced that URLs are the appropriate abstraction for a file 
>>> system path. For the record, I'm not a fan of existing Foundation methods 
>>> that create objects from an URL. There is a useful and fundamental 
>>> difference between a local path and a remote path, and conflating the two 
>>> has been a security pain point in many languages and frameworks that allow 
>>> it. Examples include remote file inclusion in PHP and malicious doctypes in 
>>> XML. Windows also had its share of issues with UNC paths.
>>> 
>>> Even when loading an arbitrary URL looks innocuous, many de-anonymizing 
>>> hacks work by causing a program to access an URL controlled by an attacker 
>>> to make it disclose the user's IP address or some other identifier.
>>> 
>>> IMO, this justifies that there should be separate types to handle local and 
>>> remote resources, so that at least developers have to be explicit about 
>>> allowing remote resources. This makes a new URL type less necessary towards 
>>> supporting file I/O.
>>> 
>>> Félix
>>> 
 Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution 
  a écrit :
 
 Okay so a few days ago there was a discussion about getting pure swift 
 file system support into Foundation or another core library, and in my 
 opinion, doing this requires a total overhaul of the `URL` type (which is 
 currently little more than a wrapper for NSURL), so I’ve just started a 
 pure Swift URL library project at .
 
 The library’s parsing and validation core (~1K loc pure swift) is already 
 in place and functional; the goal is to eventually support all of the 
 Foundation URL functionality.
 
 The new `URL` type is implemented as a value type with utf8 storage backed 
 by an array buffer. The URLs are just 56 bytes long each, so they should 
 be able to fit into cache lines. (NSURL by comparison is over 128 bytes in 
 size; it’s only saved by the fact that the thing is passed as a reference 
 type.)
 
 As I said, this is still really early on and not a mature library at all 
 but everyone is invited to observe, provide feedback, or contribute!
 ___
 swift-evolution mailing list
 swift-evolution@swift.org
 https://lists.swift.org/mailman/listinfo/swift-evolution
>>> 
>> 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Félix Cloutier via swift-evolution]

There's no question that there's a lot in common between file: URLs and other 
URLs, but that hardly makes URLs better in the file: case.

I saw the enum. The problem remains that it's a common API principle to accept 
the broadest possible input. It's not fundamentally wrong to accept and resolve 
common URL types, as there's a ton of things that need to read documents from 
the Internet by design. However, if this is the default facility for file 
handling too, it invents either:

A responsibility for API users to check that their URL is a file: URL;
A responsibility for API authors to provide separate entry points for file URLs 
and remote URLs, and a responsibility for API users to use the right one.

It would also add a category of errors to common filesystem operations: "can't 
do this because the URL is not a file: URL", and a category of questions that 
we arguably shouldn't need to ask ourselves. For instance, what is the correct 
result of glob()ing a file: URL pattern with a hash or a query string, should 
each element include that hash/query string too?

Félix

> Le 20 août 2017 à 23:33, Taylor Swift  a écrit :
> 
> I think that’s more a problem that lies in Foundation methods that take URLs 
> rather than the URLs themselves. A URL is a useful abstraction because it 
> contains a lot of common functionality between local file paths and internet 
> resources. For example, relative URI reference resolution. APIs which take 
> URLs as arguments should be responsible for ensuring that the URL’s schema is 
> a `file:`. The new URL type I’m writing actually makes the scheme an enum 
> with cases `.file`, `.http`, `.https`, `.ftp`, and `.data` to ease checking 
> this.
> 
> On Mon, Aug 21, 2017 at 2:23 AM, Félix Cloutier  > wrote:
> I'm not convinced that URLs are the appropriate abstraction for a file system 
> path. For the record, I'm not a fan of existing Foundation methods that 
> create objects from an URL. There is a useful and fundamental difference 
> between a local path and a remote path, and conflating the two has been a 
> security pain point in many languages and frameworks that allow it. Examples 
> include remote file inclusion in PHP and malicious doctypes in XML. Windows 
> also had its share of issues with UNC paths.
> 
> Even when loading an arbitrary URL looks innocuous, many de-anonymizing hacks 
> work by causing a program to access an URL controlled by an attacker to make 
> it disclose the user's IP address or some other identifier.
> 
> IMO, this justifies that there should be separate types to handle local and 
> remote resources, so that at least developers have to be explicit about 
> allowing remote resources. This makes a new URL type less necessary towards 
> supporting file I/O.
> 
> Félix
> 
>> Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution 
>> > a écrit :
>> 
>> Okay so a few days ago there was a discussion 
>> 
>>  about getting pure swift file system support into Foundation or another 
>> core library, and in my opinion, doing this requires a total overhaul of the 
>> `URL` type (which is currently little more than a wrapper for NSURL), so 
>> I’ve just started a pure Swift URL library project at 
>> >.
>> 
>> The library’s parsing and validation core (~1K loc pure swift) is already in 
>> place and functional; the goal is to eventually support all of the 
>> Foundation URL functionality.
>> 
>> The new `URL` type is implemented as a value type with utf8 storage backed 
>> by an array buffer. The URLs are just 56 bytes long each, so they should be 
>> able to fit into cache lines. (NSURL by comparison is over 128 bytes in 
>> size; it’s only saved by the fact that the thing is passed as a reference 
>> type.)
>> 
>> As I said, this is still really early on and not a mature library at all but 
>> everyone is invited to observe, provide feedback, or contribute!
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>> 
> 
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Taylor Swift via swift-evolution]

I think that’s more a problem that lies in Foundation methods that take
URLs rather than the URLs themselves. A URL is a useful abstraction because
it contains a lot of common functionality between local file paths and
internet resources. For example, relative URI reference resolution. APIs
which take URLs as arguments should be responsible for ensuring that the
URL’s schema is a `file:`. The new URL type I’m writing actually makes the
scheme an enum with cases `.file`, `.http`, `.https`, `.ftp`, and `.data`
to ease checking this.

On Mon, Aug 21, 2017 at 2:23 AM, Félix Cloutier 
wrote:

> I'm not convinced that URLs are the appropriate abstraction for a file
> system path. For the record, I'm not a fan of existing Foundation methods
> that create objects from an URL. There is a useful and fundamental
> difference between a local path and a remote path, and conflating the two
> has been a security pain point in many languages and frameworks that allow
> it. Examples include remote file inclusion in PHP and malicious doctypes in
> XML. Windows also had its share of issues with UNC paths.
>
> Even when loading an arbitrary URL looks innocuous, many de-anonymizing
> hacks work by causing a program to access an URL controlled by an attacker
> to make it disclose the user's IP address or some other identifier.
>
> IMO, this justifies that there should be separate types to handle local
> and remote resources, so that at least developers have to be explicit about
> allowing remote resources. This makes a new URL type less necessary towards
> supporting file I/O.
>
> Félix
>
> Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution <
> swift-evolution@swift.org> a écrit :
>
> Okay so a few days ago there was a discussion
> 
> about getting pure swift file system support into Foundation or another
> core library, and in my opinion, doing this requires a total overhaul of
> the `URL` type (which is currently little more than a wrapper for NSURL),
> so I’ve just started a pure Swift URL library project at <
> https://github.com/kelvin13/url>.
>
> The library’s parsing and validation core (~1K loc pure swift) is already
> in place and functional; the goal is to eventually support all of the
> Foundation URL functionality.
>
> The new `URL` type is implemented as a value type with utf8 storage backed
> by an array buffer. The URLs are just 56 bytes long each, so they should be
> able to fit into cache lines. (NSURL by comparison is over 128 bytes in
> size; it’s only saved by the fact that the thing is passed as a reference
> type.)
>
> As I said, this is still really early on and not a mature library at all
> but everyone is invited to observe, provide feedback, or contribute!
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] (core library) modern URL types

[Félix Cloutier via swift-evolution]

I'm not convinced that URLs are the appropriate abstraction for a file system 
path. For the record, I'm not a fan of existing Foundation methods that create 
objects from an URL. There is a useful and fundamental difference between a 
local path and a remote path, and conflating the two has been a security pain 
point in many languages and frameworks that allow it. Examples include remote 
file inclusion in PHP and malicious doctypes in XML. Windows also had its share 
of issues with UNC paths.

Even when loading an arbitrary URL looks innocuous, many de-anonymizing hacks 
work by causing a program to access an URL controlled by an attacker to make it 
disclose the user's IP address or some other identifier.

IMO, this justifies that there should be separate types to handle local and 
remote resources, so that at least developers have to be explicit about 
allowing remote resources. This makes a new URL type less necessary towards 
supporting file I/O.

Félix

> Le 20 août 2017 à 21:37, Taylor Swift via swift-evolution 
>  a écrit :
> 
> Okay so a few days ago there was a discussion 
> 
>  about getting pure swift file system support into Foundation or another core 
> library, and in my opinion, doing this requires a total overhaul of the `URL` 
> type (which is currently little more than a wrapper for NSURL), so I’ve just 
> started a pure Swift URL library project at  >.
> 
> The library’s parsing and validation core (~1K loc pure swift) is already in 
> place and functional; the goal is to eventually support all of the Foundation 
> URL functionality.
> 
> The new `URL` type is implemented as a value type with utf8 storage backed by 
> an array buffer. The URLs are just 56 bytes long each, so they should be able 
> to fit into cache lines. (NSURL by comparison is over 128 bytes in size; it’s 
> only saved by the fact that the thing is passed as a reference type.)
> 
> As I said, this is still really early on and not a mature library at all but 
> everyone is invited to observe, provide feedback, or contribute!
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[swift-evolution] (core library) modern URL types

[Taylor Swift via swift-evolution]

Okay so a few days ago there was a discussion

about getting pure swift file system support into Foundation or another
core library, and in my opinion, doing this requires a total overhaul of
the `URL` type (which is currently little more than a wrapper for NSURL),
so I’ve just started a pure Swift URL library project at <
https://github.com/kelvin13/url>.

The library’s parsing and validation core (~1K loc pure swift) is already
in place and functional; the goal is to eventually support all of the
Foundation URL functionality.

The new `URL` type is implemented as a value type with utf8 storage backed
by an array buffer. The URLs are just 56 bytes long each, so they should be
able to fit into cache lines. (NSURL by comparison is over 128 bytes in
size; it’s only saved by the fact that the thing is passed as a reference
type.)

As I said, this is still really early on and not a mature library at all
but everyone is invited to observe, provide feedback, or contribute!
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution