Re: Argumnentation against external function operator overloading is unconvincing
On 9/25/16 9:48 AM, Jonathan M Davis via Digitalmars-d wrote: On Sunday, September 25, 2016 13:10:42 ZombineDev via Digitalmars-d wrote: D's built-in dynamic arrays are hidden from you and you only get to interact with them by referring to their elements by using slices. That's a common misconception propagated by Steven's otherwise excellent article. As far as the official language spec goes, T[] _is_ the dynamic array. What backs it is irrelevant as far as that goes. It's just that if it's backed by the GC, then when you append to it, the GC might not have to allocate a new memory buffer for the array. All of the operations on a dynamic array work the same regardless of what backs it, and focusing on the memory buffer being the array risks causing you problems when you have to deal with dynamic arrays backed by something else. D can call an animal that quacks, has a flat bill, and feathers a sparrow all it wants, but it sure doesn't act like one ;) -Steve
Re: Argumnentation against external function operator overloading is unconvincing
On 9/25/2016 6:45 AM, Andrei Alexandrescu wrote: Yah, it comes as a surprise to many that static arrays are more akin to structs than to arrays. They really are records that happen to have several elements of the same type. Providing indexing for them is a convenience that somewhat adds to the confusion, and converting to dynamic arrays is unsafe because it essentially escapes the innards of the struct. Statically-sized arrays are odd, that's for sure, and that's the case in C and C++ as well. I'd like to emphasize that this is an important insight. A static array is semantically equivalent to a struct with fields of all the same type, and the fields can be accessed by index rather than field name. This insight has driven some simplifying assumptions internal to how the compiler is implemented, as well as the specification for how things work (like passing a static array as a function parameter works like passing a struct). Another simplifying insight is that a struct is a tuple of fields. I tried to generalize this by having the arguments to a function be a tuple as well, but ran afoul of the ABI for calling conventions, argghh. It would really be awesome to have tuples, structs, static arrays, and function argument lists be literally the same thing, but sadly that does not seem practical at the moment.
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, 25 September 2016 at 15:25:38 UTC, Jonathan M Davis wrote: So, if they want their code to work with anyone else's code they pretty much need to use their own set of range primitives that do not conflict with the standard ones rather than trying to redefine the standard ones. And if they don't care about interacting with anyone else's code, they can always just fork druntime and Phobos to make them do whatever they want. But trying to redefine some of the basic primitives that D's runtime and standard library use while still trying to interact with anyone else's code is a recipe for disaster. - Jonathan M Davis I don't mind writing my own code rather than interacting with someone else's, and I severely dislike many of the design decisions made regarding phobos. Which is why I've been building my own alternative to the standard library that I can use as a basis for software I develop. It depends on phobos for only a handful of things, and I'm working toward a point where I won't need it for anything. I recognize that the preference is unusual, but I insist on my and others' ability to pursue such a preference. On Sunday, 25 September 2016 at 14:50:04 UTC, Andrei Alexandrescu wrote: It seems you want to define ranges with similar syntax but subtle semantic differences, e.g. r.front and r[0] to mean different things. The entire Phobos is designed under the assumptions that ranges work a specific way, so in order to design a different mechanism you may want to use different syntactic interfaces. I have no problem with phobos being phobos, and treating ranges as it does. I don't want the core language to adopt the same way of treating ranges because while I recognize that it is far too late to change phobos' way of thinking about ranges - much less the community's - I think it was a mistake and that the quality of D as a language shouldn't suffer for its sake. The core language should define the bare minimum that it needs to for ranges to be a useful concept - as it does now - and should leave the rest up to phobos or whatever else is actually implementing the ranges. On Sunday, 25 September 2016 at 14:50:04 UTC, Andrei Alexandrescu wrote: I speculate this has to do with our community being self-selected as opinionated folks who don't do well with conventional wisdom. You have described me to a T.
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, September 25, 2016 16:50:04 Andrei Alexandrescu via Digitalmars-d wrote: > It seems you want to define ranges with similar syntax but subtle > semantic differences, e.g. r.front and r[0] to mean different things. > The entire Phobos is designed under the assumptions that ranges work a > specific way, so in order to design a different mechanism you may want > to use different syntactic interfaces. The reality of the matter is that anyone who tries to define the range primitives to work differently than how Phobos uses them (and druntime in various places even if it's not in object.d yet) is going to be screwed as soon as they interact with code written by anyone else. Anyone looking to make r[0] do something different than give you r.front might as well just redefine popFront to mean popBack and vice versa for all that it's going to work with other people's code. So, if they want their code to work with anyone else's code they pretty much need to use their own set of range primitives that do not conflict with the standard ones rather than trying to redefine the standard ones. And if they don't care about interacting with anyone else's code, they can always just fork druntime and Phobos to make them do whatever they want. But trying to redefine some of the basic primitives that D's runtime and standard library use while still trying to interact with anyone else's code is a recipe for disaster. - Jonathan M Davis
Re: Argumnentation against external function operator overloading is unconvincing
On 9/25/16 4:05 PM, pineapple wrote: On Sunday, 25 September 2016 at 13:57:04 UTC, Jonathan M Davis wrote: The way it works now is how it's always worked with dynamic arrays and ranges in D. If you're trying do anything else, you're just going to run into problems in the long run - particularly when interacting with code written by anyone else. So, while you're obviously free to do whatever you want with your own code, don't expect Phobos or D code in general to change how ranges fundamentally work. That change is exactly what I'm arguing against - that the front, popFront, etc. functions defined for dynamic arrays in phobos should not be adopted by the core language. On Thursday, 22 September 2016 at 12:51:59 UTC, Andrei Alexandrescu wrote: Would make sense to move those few primitives to object.d. I've been thinking of that a long time ago but back then there was a vague stance that object.d shouldn't contain templates. Since then that has changed. -- Andrei Please do not do this - there ways to handle ranges other than the approach phobos has taken. That's it, that's the point I'm trying to make. We learned with time that any step we're trying to take toward progress in a matter of design (i.e. no mechanical rules, reasonable people may disagree), a faction will strongly oppose it. I speculate this has to do with our community being self-selected as opinionated folks who don't do well with conventional wisdom. At the same time, we can't let this gridlock development of D. We must go with what we think is good. It seems you want to define ranges with similar syntax but subtle semantic differences, e.g. r.front and r[0] to mean different things. The entire Phobos is designed under the assumptions that ranges work a specific way, so in order to design a different mechanism you may want to use different syntactic interfaces. Andrei
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, 25 September 2016 at 13:57:04 UTC, Jonathan M Davis wrote: The way it works now is how it's always worked with dynamic arrays and ranges in D. If you're trying do anything else, you're just going to run into problems in the long run - particularly when interacting with code written by anyone else. So, while you're obviously free to do whatever you want with your own code, don't expect Phobos or D code in general to change how ranges fundamentally work. That change is exactly what I'm arguing against - that the front, popFront, etc. functions defined for dynamic arrays in phobos should not be adopted by the core language. On Thursday, 22 September 2016 at 12:51:59 UTC, Andrei Alexandrescu wrote: Would make sense to move those few primitives to object.d. I've been thinking of that a long time ago but back then there was a vague stance that object.d shouldn't contain templates. Since then that has changed. -- Andrei Please do not do this - there ways to handle ranges other than the approach phobos has taken. That's it, that's the point I'm trying to make.
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, 25 September 2016 at 13:45:01 UTC, Andrei Alexandrescu wrote: Ranges don't need necessarily an associated Iterable. This is the case in other languages, too; Lisp/Scheme/Haskell/etc lists are iterables and at the same time their own iterators. But indeed std.container is designed to have containers distinct from their associated ranges, which raises the interesting question what should happen if a range gets "orphaned", i.e. it's still active after its container ceases to exist. And an Iterator doesn't necessarily need an associated Iterable. I fully recognize how phobos handles ranges, and that it has become "the right way" to do things. But I disagree, and I do not want the core language to force me to take phobos' approach. I greatly appreciate that the only properties of ranges the core language cares about are `empty`, `front`, `back`, `popFront`, `popBack` - it does not itself imply how `opIndex` and `opSlice` are meant to behave for ranges - and I think it should stay that way. I see phobos' arrays-as-ranges functions as poor design, and most certainly not something that should become inextricably tied with the core language.
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, September 25, 2016 15:45:01 Andrei Alexandrescu via Digitalmars-d wrote: > Ranges don't need necessarily an associated Iterable. This is the case > in other languages, too; Lisp/Scheme/Haskell/etc lists are iterables and > at the same time their own iterators. But indeed std.container is > designed to have containers distinct from their associated ranges, which > raises the interesting question what should happen if a range gets > "orphaned", i.e. it's still active after its container ceases to exist. Another thing to consider here is that given how most range-based functions will create a new range from the container's range when you pass it to them, when operating on ranges over containers, you _very_ quickly end up with ranges that really have nothing to do with the container anymore (hence the fun problems with removing elements from a container by passing a range to it). And that highlights how ranges really don't act like they're backed by containers. But the safety issue that comes with ranges over containers where the container goes away is definitely a thorny one - as is the fact that removing elements from the container while you have active ranges that refer to it can do funny things to those ranges. I tend to favor C++'s approach to iterators and how they stay valid, but with D's focus on safety, I don't know that that's acceptable (though enforcing safety tends to result in additional overhead which isn't in line with the efficiency goals that go with being a system language). So, I don't know what the best approach is. - Jonathan M Davis
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, September 25, 2016 13:40:14 pineapple via Digitalmars-d wrote: > On Sunday, 25 September 2016 at 11:48:38 UTC, Jonathan M Davis > > wrote: > > It's because ranges are effectively a sliding window over > > whatever they're iterating over. > > I think this is the difference in perception - ranges do not > _have_ to be sliding windows, they can just as well be windows > that don't move. I find the latter approach to be more workable > and intuitive that the former, and in almost every case the > approach results in more concise and performant code for > performing those operations. > > When would I ever want `range[0]` to give me the same thing as > `range.front`? I want it to give me the first item in the range's > view at the time of its creation. You're going to be better off if you stop trying to think like there's some sort of original range here. What you have right now is what you have. The first element is index 0, and whether the range was just created or whether it's had a million elements popped off the front is irrelevant. If you want access to elements that aren't currently in the range, then you need to save the range and keep that around so that you can access _that_ range instead of the one that you currently have. Remember that many ranges are generative and aren't backed by any sort of container and that there is no original range that's been iterated over, and index 0 couldn't have anything to refer to except for the current front. And if you don't like how D's ranges are designed, sorry, but it's never been like what you're suggesting, and changing it now would break a ton of code even if we agreed that it were a good idea, and you're not going to get that agreement. What we have with ranges right now is by no means perfect, but the whole design is based on the sliding window concept - just like D's dynamic arrays are - and that has worked fantastically for us. > I strongly oppose any change that makes the former approach an > inherent part of D. I honestly don't think the latter approach > should be an inherent part of D, either, but I'd have a hell of a > time trying to use the language if it wasn't an option. The way it works now is how it's always worked with dynamic arrays and ranges in D. If you're trying do anything else, you're just going to run into problems in the long run - particularly when interacting with code written by anyone else. So, while you're obviously free to do whatever you want with your own code, don't expect Phobos or D code in general to change how ranges fundamentally work. - Jonathan M Davis
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, September 25, 2016 13:10:42 ZombineDev via Digitalmars-d wrote: > D's built-in dynamic arrays are hidden from you and you only get > to interact with them by referring to their elements by using > slices. That's a common misconception propagated by Steven's otherwise excellent article. As far as the official language spec goes, T[] _is_ the dynamic array. What backs it is irrelevant as far as that goes. It's just that if it's backed by the GC, then when you append to it, the GC might not have to allocate a new memory buffer for the array. All of the operations on a dynamic array work the same regardless of what backs it, and focusing on the memory buffer being the array risks causing you problems when you have to deal with dynamic arrays backed by something else. But on the whole, what you said was right. It's just the terminology that's off. - Jonathan M Davis
Re: Argumnentation against external function operator overloading is unconvincing
On 9/25/16 12:58 PM, pineapple wrote: On Sunday, 25 September 2016 at 04:06:41 UTC, Jonathan M Davis wrote: Considering that a random access range is essentially an abstraction for a dynamic array and that ranges were designed with that in mind, I don't know how you can argue that dynamic arrays shouldn't be treated as ranges. The auto-decoding is certainly an issue, but that only affects ranges of char and wchar. The vast majority of dynamic arrays are perfectly normal ranges. - Jonathan M Davis Phobos' range facilities vomit when you try to deal with static arrays, and they have to be coerced into dynamic arrays before they can be dealt with. This is silly. Yah, it comes as a surprise to many that static arrays are more akin to structs than to arrays. They really are records that happen to have several elements of the same type. Providing indexing for them is a convenience that somewhat adds to the confusion, and converting to dynamic arrays is unsafe because it essentially escapes the innards of the struct. Statically-sized arrays are odd, that's for sure, and that's the case in C and C++ as well. It gets under my skin that length and opIndex and opSlice produce different results in phobos' ranges depending on one's current position in the range. This doesn't make sense to me, and the only reason I can conceive of it having become how ranges work throughout phobos is because that's how dynamic arrays work if you force them to act as though they were ranges. Better get used to it. Ranges are a generalization of D's slices, not the other way around. In every single other language I've used, the concept of an Iterable and an Iterator are distinct and very separate. An Iterator is something that can be iterated over; an Iterable is something which can produce an Iterator for iterating over its contents. In D, arrays are Iterables, and phobos endeavors to force them to be Iterators as well. It defies years of basic design wisdom regarding how to differentiate a collection and the means by which one enumerates the items in that collection. Ranges don't need necessarily an associated Iterable. This is the case in other languages, too; Lisp/Scheme/Haskell/etc lists are iterables and at the same time their own iterators. But indeed std.container is designed to have containers distinct from their associated ranges, which raises the interesting question what should happen if a range gets "orphaned", i.e. it's still active after its container ceases to exist. Arrays are Iterables which should be able to produce an Iterator, in D's case a range. They should not themselves be Iterators. Yah, std.container.Array follows that. It's not an be-all-end-all of design though. Andrei
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, 25 September 2016 at 11:48:38 UTC, Jonathan M Davis wrote: It's because ranges are effectively a sliding window over whatever they're iterating over. I think this is the difference in perception - ranges do not _have_ to be sliding windows, they can just as well be windows that don't move. I find the latter approach to be more workable and intuitive that the former, and in almost every case the approach results in more concise and performant code for performing those operations. When would I ever want `range[0]` to give me the same thing as `range.front`? I want it to give me the first item in the range's view at the time of its creation. I strongly oppose any change that makes the former approach an inherent part of D. I honestly don't think the latter approach should be an inherent part of D, either, but I'd have a hell of a time trying to use the language if it wasn't an option.
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, 25 September 2016 at 13:10:42 UTC, ZombineDev wrote: But which opIndex and which length? Those of the container, or those of the range? It doesn't make any sense to expect container[].takeExactly(7).length to be different than 7. If range.length returns the length of the container this would break every sensible algorithm out there. How would you even implement binary search (https://rosettacode.org/wiki/Binary_search#D)? It's completely wrong to expect indexing operations to work on the original container. What you do in situations where there is no container, such as generators (e.g. iota https://github.com/dlang/phobos/blob/v2.071.2/std/range/package.d#L4722) and data coming from the network? And how would you get the number of remaining elements in the range? The argument is not that the length of a range should always evaluate to the length of some container it enumerates, but that the length of the range should not change based on your position in the range. In phobos, popFront effectively reduces the length of the range by one, and offsets all further calls to opIndex and opSlice. I expect length, opIndex, and opSlice to behave the same whether I just built the range, am halfway through the range, or have fully consumed the range. Phobos' range facilities vomit when you try to deal with static arrays, and they have to be coerced into dynamic arrays before they can be dealt with. This is silly. What's the problem here? Just slice them using the [] syntax (i.e. the analog of asrange in your library). As Jonathan pointed out there's a lot of muddying of how dynamic arrays are only sorta-kinda genuine arrays. But when using them one should be able to rely on dynamic and static arrays have the same interface, except that one can be resized and appended to and another cannot. Having to slice static arrays to pass them into phobos' HOFs is icky and completely unnecessary. And though you might say that you could just append `[]` to every array you pass in, but while it will work for static and dynamic arrays it won't work for everything that can be passed to a function accepting a range. Consistency of syntax for performing the same operation upon many different types is not to be undervalued.
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, 25 September 2016 at 10:58:24 UTC, pineapple wrote:
It gets under my skin that length and opIndex and opSlice
produce different results in phobos' ranges depending on one's
current position in the range. This doesn't make sense to me,
and the only reason I can conceive of it having become how
ranges work throughout phobos is because that's how dynamic
arrays work if you force them to act as though they were ranges.
But which opIndex and which length? Those of the container, or
those of the range? It doesn't make any sense to expect
container[].takeExactly(7).length to be different than 7. If
range.length returns the length of the container this would break
every sensible algorithm out there. How would you even implement
binary search (https://rosettacode.org/wiki/Binary_search#D)?
It's completely wrong to expect indexing operations to work on
the original container. What you do in situations where there is
no container, such as generators (e.g. iota
https://github.com/dlang/phobos/blob/v2.071.2/std/range/package.d#L4722) and data coming from the network? And how would you get the number of remaining elements in the range?
Phobos' range facilities vomit when you try to deal with static
arrays, and they have to be coerced into dynamic arrays before
they can be dealt with. This is silly.
What's the problem here? Just slice them using the [] syntax
(i.e. the analog of asrange in your library).
In every single other language I've used, the concept of an
Iterable and an Iterator are distinct and very separate. An
Iterator is something that can be iterated over; an Iterable is
something which can produce an Iterator for iterating over its
contents. In D, arrays are Iterables, and phobos endeavors to
force them to be Iterators as well. It defies years of basic
design wisdom regarding how to differentiate a collection and
the means by which one enumerates the items in that collection.
Arrays are Iterables which should be able to produce an
Iterator, in D's case a range. They should not themselves be
Iterators.
There's seems to be a misunderstanding what are D's ranges and
how they're meant to be used. Containers such as static arrays
(i.e T[n]) are Iterable-s whereas slices (i.e. T[]) are
Iterator-s by your taxonomy. You get the Iterator from the
iterable static array using the opSlice (i.e. []) method. Ranges
(and in particular slices) are just positions in an array. The
same analogy holds for the containers in std.container: you get a
range using the container[], or container.opSlice() syntax.
Regardless where the range points to and if it shrinks, the
container.length stays the same, assuming you haven't added or
removed any elements.
I guess your misunderstanding stems from the fact that you call
T[] arrays. This however is wrong. T[] is just slice of an array.
If you want to use arrays similar to those in other languages,
use std.container.array : Array
(http://dlang.org/phobos/std_container_array).
D's built-in dynamic arrays are hidden from you and you only get
to interact with them by referring to their elements by using
slices. For example, have a look at the following code:
```
import std.algorithm.comparison : equal;
int[] arr = [1, 2, 3, 4];
void append5(int[] s)
{
s ~= 5; // 1)
}
append5(arr);
assert (arr.equal([1, 2, 3, 4]));
```
As you can see in the example above, since int[] is just a
slice/range, appending to it does not modify the underlying
container. Instead a new container is created (*) and `s` is made
to point to it in 1). If you were to use a "proper" container
such as std.container.array, the results would be different:
```
import std.algorithm.comparison : equal;
import std.container.array;
auto arr = Array!int([1, 2, 3, 4]);
void append5(Array!int s)
{
s ~= 5;
}
append5(arr);
assert (arr[].equal([1, 2, 3, 4, 5]));
```
For more information, I strongly suggest reading Steven's article
: http://dlang.org/d-array-article.html (*)
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, September 25, 2016 10:58:24 pineapple via Digitalmars-d wrote: > On Sunday, 25 September 2016 at 04:06:41 UTC, Jonathan M Davis > > wrote: > > Considering that a random access range is essentially an > > abstraction for a dynamic array and that ranges were designed > > with that in mind, I don't know how you can argue that dynamic > > arrays shouldn't be treated as ranges. > > > > The auto-decoding is certainly an issue, but that only affects > > ranges of char and wchar. The vast majority of dynamic arrays > > are perfectly normal ranges. > > > > - Jonathan M Davis > > Phobos' range facilities vomit when you try to deal with static > arrays, and they have to be coerced into dynamic arrays before > they can be dealt with. This is silly. Except that static arrays are bona fide containers, and their length can't be mutated. They don't make any sense as ranges at all. > It gets under my skin that length and opIndex and opSlice produce > different results in phobos' ranges depending on one's current > position in the range. This doesn't make sense to me, and the > only reason I can conceive of it having become how ranges work > throughout phobos is because that's how dynamic arrays work if > you force them to act as though they were ranges. It's because ranges are effectively a sliding window over whatever they're iterating over. The only way that it would make sense for slicing or indexing a range to refer to anything other than what the range currently refers to is if there were some sort of container that they were a slice of, and the vast majority of ranges aren't that way. They're not designed to refer to anything other than what they currently refer to, and there is no original for the indices to refer back to. A range is simply a list of values. It may or may not be randomly accessible, and it may or may not refer to anything else like a container. It could be purely generative (e.g. a range over the fibonacci sequence or an infinite list of random numbers). It really makes no sense for indexing or slicing ranges to do anything but have the indices treat the current front of the range as index 0. I can understand that that might be confusing at first if you're thinking of them as specifically refering to elements of a container, but many ranges have nothing to do with a container at all. > In every single other language I've used, the concept of an > Iterable and an Iterator are distinct and very separate. An > Iterator is something that can be iterated over; an Iterable is > something which can produce an Iterator for iterating over its > contents. In D, arrays are Iterables, and phobos endeavors to > force them to be Iterators as well. It defies years of basic > design wisdom regarding how to differentiate a collection and the > means by which one enumerates the items in that collection. > > Arrays are Iterables which should be able to produce an Iterator, > in D's case a range. They should not themselves be Iterators. Dynamic arrays aren't really containers in D. Unlike containers, they don't own or manage their own memory. They're only slices of that memory. The fact that you can append to them, and the GC will then increase the amount of memory that the dynamic array refers to (possibly allocating a new block of memory and changing which block of memory the dynamic array is a slice of) certainly muddies things, but it's still not the case the dynamic array owns or manages its memory. What owns or manages the memory depends entirely on what allocated the memory (usually the GC, but not always). Even in the case of concatention or appending, its the GC that manages the memory, not the dynamic array (e.g. a dynamic array which is a slice of malloced memory is going to result in a memory leak if you then append to it, and nothing else was keeping track of that memory; the array itself doesn't know or care who owns or manages its memory). And multiple dynamic arrays can refer to exactly the same block of memory, which is a clear indicator that they don't own or manage their own memory. And in that respect, they're definitely more like iterators than containers. So, yes, dynamic arrays in D are weird hybrids, but aside from operations related to concatenation or appending, they don't act like containers at all. They act much more like a pair of iterators being passed around together. And both the language and Phobos are very consistent about that. The one place with dynamic arrays and ranges that is totally wacko is narrow strings, because they're auto-decoded, because then Phobos tries to stop treating them like dynamic arrays. If it didn't do that, AFAIK it would be completely consistent with how it treated dynamic arrays. I think that you'll have an easier time understanding dynamic arrays in D and ranges in general if you stopped trying to treat dynamic arrays as if they were containers, since they really aren't. And the few operations that make them sort of act like containers
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, 25 September 2016 at 04:06:41 UTC, Jonathan M Davis wrote: Considering that a random access range is essentially an abstraction for a dynamic array and that ranges were designed with that in mind, I don't know how you can argue that dynamic arrays shouldn't be treated as ranges. The auto-decoding is certainly an issue, but that only affects ranges of char and wchar. The vast majority of dynamic arrays are perfectly normal ranges. - Jonathan M Davis Phobos' range facilities vomit when you try to deal with static arrays, and they have to be coerced into dynamic arrays before they can be dealt with. This is silly. It gets under my skin that length and opIndex and opSlice produce different results in phobos' ranges depending on one's current position in the range. This doesn't make sense to me, and the only reason I can conceive of it having become how ranges work throughout phobos is because that's how dynamic arrays work if you force them to act as though they were ranges. In every single other language I've used, the concept of an Iterable and an Iterator are distinct and very separate. An Iterator is something that can be iterated over; an Iterable is something which can produce an Iterator for iterating over its contents. In D, arrays are Iterables, and phobos endeavors to force them to be Iterators as well. It defies years of basic design wisdom regarding how to differentiate a collection and the means by which one enumerates the items in that collection. Arrays are Iterables which should be able to produce an Iterator, in D's case a range. They should not themselves be Iterators. "Poisoned m are certainly an issue, but that only affects people who unwittingly eat them. You see, the vast majority of m are perfectly innocuous."
Re: Argumnentation against external function operator overloading is unconvincing
On Sunday, September 25, 2016 00:36:58 pineapple via Digitalmars-d wrote: > On Thursday, 22 September 2016 at 12:51:59 UTC, Andrei > > Alexandrescu wrote: > > On 9/22/16 6:38 AM, pineapple wrote: > >> The greatest offender I've found is how in phobos, arrays do > >> not behave > >> as ranges without importing the module defining their range > >> operations. > > > > Would make sense to move those few primitives to object.d. I've > > been thinking of that a long time ago but back then there was a > > vague stance that object.d shouldn't contain templates. Since > > then that has changed. -- Andrei > > I strongly disagree - I have been working a library that does not > treat arrays as ranges, but as a type that a range can be created > from. This design difference has proven rather elegant and solves > a lot of problems I've run into using phobos. I think the > (unlikely but ideal) solution is for phobos to adopt a similar > approach, and not to warp the core language to accommodate > phobos' strange design decisions. > > Also, auto-decoding is a plague and it need not spread any > farther than it already has done. > > https://github.com/pineapplemachine/mach.d/blob/master/readme.md#arrays-aren > t-ranges Considering that a random access range is essentially an abstraction for a dynamic array and that ranges were designed with that in mind, I don't know how you can argue that dynamic arrays shouldn't be treated as ranges. The auto-decoding is certainly an issue, but that only affects ranges of char and wchar. The vast majority of dynamic arrays are perfectly normal ranges. - Jonathan M Davis
Re: Argumnentation against external function operator overloading is unconvincing
On Thursday, 22 September 2016 at 12:51:59 UTC, Andrei Alexandrescu wrote: On 9/22/16 6:38 AM, pineapple wrote: The greatest offender I've found is how in phobos, arrays do not behave as ranges without importing the module defining their range operations. Would make sense to move those few primitives to object.d. I've been thinking of that a long time ago but back then there was a vague stance that object.d shouldn't contain templates. Since then that has changed. -- Andrei I strongly disagree - I have been working a library that does not treat arrays as ranges, but as a type that a range can be created from. This design difference has proven rather elegant and solves a lot of problems I've run into using phobos. I think the (unlikely but ideal) solution is for phobos to adopt a similar approach, and not to warp the core language to accommodate phobos' strange design decisions. Also, auto-decoding is a plague and it need not spread any farther than it already has done. https://github.com/pineapplemachine/mach.d/blob/master/readme.md#arrays-arent-ranges
Re: Argumnentation against external function operator overloading is unconvincing
On Wednesday, 21 September 2016 at 21:14:15 UTC, H. S. Teoh wrote: The problem here is that generic_code.d doesn't (and shouldn't!) import usertype.d, so usertype.opBinary is not visible in generic_code.d. So when algorithm() tries to look up the '+' operator in `t + u`, it can't find the declaration and fails. There is no way to find the correct opBinary() because it's not part of UserType, so algorithm() has no way to access that symbol. Using the operator in module main is OK, because main (rightfully) imports usertype.d, so the operator is visible. But any generic code that main imports will have a problem because they can't (and shouldn't!) know ahead of time which modules contain the declaration they need. [...] T Why wouldn't templates just pull the "context" into them? I'm sorry if this is a naïve question.
Re: Argumnentation against external function operator overloading is unconvincing
On Thursday, 22 September 2016 at 14:33:36 UTC, Andrei Alexandrescu wrote: On 9/22/16 10:20 AM, Jonathan M Davis via Digitalmars-d wrote: The main problem with moving them there is auto-decoding. front and popFront for strings require std.utf in order to do their thing. Druntime has it's own utf decoding code, necessary for foreach auto transcoding. Moving that to a template and syncing it with the better optimized Phobos implementation would be a good thing. Yah, that's a little hurdle. -- Andrei At least empty should go to object, could help to solve part of the wrong if (arr) usage.
Re: Argumnentation against external function operator overloading is unconvincing
On 23.09.2016 12:44, Stefan Koch wrote: On Friday, 23 September 2016 at 08:50:56 UTC, Timon Gehr wrote: FQN disables UFCS. Nothing specific to operators here. There is no reason why there should be any difference between a + b and a.opBinary!"+"(b). In fact, 2.opBinary!"+"(3) should work too. Currently this is tricky to implement in the compiler. This can easily be implemented in the parser and in object.d without even changing semantic. This is not the best implementation strategy, but it demonstrates that implementation can be simple. I'm curious to know what strategy you have in mind, and why it would be tricky. BTW: One case for why built-in types should have member call syntax for operators are all the places in e.g. Phobos where code special-cases built-in types in order to be able to use opCmp directly for three-way comparison on user-defined types. And it widens the scope for name-conflicts immensely! ... Operator overloading makes sense for a small minority of types and D has plenty of mechanisms to deal with name conflicts: overload sets, template constraints, private aliases, FQNs. I do not see a case where UFCS overloaded operators are worth the trouble they introduce. IMHO it's a trivial surface language feature allowing convenient syntax for a restricted set of user types. The current limitations are slightly confusing (because a.opBinary!"+"(b) is somehow not the same as a.opBinary!"+"(b)), and also annoying when you run into them. This is not the first time this is discussed.
Re: Argumnentation against external function operator overloading is unconvincing
On Friday, September 23, 2016 13:47:06 Sai via Digitalmars-d wrote: > > The greatest offender I've found is how in phobos, arrays do > > not behave as ranges without importing the module defining > > their range operations. > > Could you please tell me what module is it? is it std.array? It was, but they were moved to std.range.primitives some time in the last year or two. They're still publicly imported in std.array though, so importing std.array will work. - Jonathan M Davis
Re: Argumnentation against external function operator overloading is unconvincing
The greatest offender I've found is how in phobos, arrays do not behave as ranges without importing the module defining their range operations. Could you please tell me what module is it? is it std.array?
Re: Argumnentation against external function operator overloading is unconvincing
On Friday, 23 September 2016 at 08:50:56 UTC, Timon Gehr wrote: FQN disables UFCS. Nothing specific to operators here. There is no reason why there should be any difference between a + b and a.opBinary!"+"(b). In fact, 2.opBinary!"+"(3) should work too. Currently this is tricky to implement in the compiler. And it widens the scope for name-conflicts immensely! I do not see a case where UFCS overloaded operators are worth the trouble they introduce.
Re: Argumnentation against external function operator overloading is unconvincing
On 22.09.2016 10:44, Jonathan M Davis via Digitalmars-d wrote: On Thursday, September 22, 2016 00:14:52 H. S. Teoh via Digitalmars-d wrote: Normally this isn't a problem (D's module system will trigger an overload conflict and require explicit FQN to unambiguously select the right overload), but FQN's are not an option when the call is made from generic code. And in the case of operator overloads, FQN makes no sense at all, since it wouldn't be using the operator anymore. - Jonathan M Davis FQN disables UFCS. Nothing specific to operators here. There is no reason why there should be any difference between a + b and a.opBinary!"+"(b). In fact, 2.opBinary!"+"(3) should work too.
Re: Argumnentation against external function operator overloading is unconvincing
On Thursday, 22 September 2016 at 05:38:53 UTC, HaraldZealot wrote: So it seems to be essential point. But because we already have the same problem with UFCS, I don't see why we should prohibit external overloading of operator, it is just inequality (in political sense) for operators :) I'm not sure that it's fundamentally different from issues that can affect other functions. I seem to remember running into some issues along these lines when I tried defining math functions (e.g. `cos`) to work with an `Imaginary` type I was trying to add to `std.complex`. If you're writing generic math code and you want to take the cosine of a number whose type is not explicitly known, how do you know which `cos` to use?
Re: Argumnentation against external function operator overloading is unconvincing
On Thu, Sep 22, 2016 at 07:20:49AM -0700, Jonathan M Davis via Digitalmars-d wrote: > On Thursday, September 22, 2016 08:51:59 Andrei Alexandrescu via > Digitalmars-d > wrote: > > On 9/22/16 6:38 AM, pineapple wrote: > > > The greatest offender I've found is how in phobos, arrays do not > > > behave as ranges without importing the module defining their range > > > operations. > > > > Would make sense to move those few primitives to object.d. I've been > > thinking of that a long time ago but back then there was a vague > > stance that object.d shouldn't contain templates. Since then that > > has changed. -- Andrei > > The main problem with moving them there is auto-decoding. [...] Yet another nail in the coffin of autodecoding. But I digress. ;-) --T
Re: Argumnentation against external function operator overloading is unconvincing
On 9/22/16 10:20 AM, Jonathan M Davis via Digitalmars-d wrote: The main problem with moving them there is auto-decoding. front and popFront for strings require std.utf in order to do their thing. Yah, that's a little hurdle. -- Andrei
Re: Argumnentation against external function operator overloading is unconvincing
On Thursday, September 22, 2016 08:51:59 Andrei Alexandrescu via Digitalmars-d wrote: > On 9/22/16 6:38 AM, pineapple wrote: > > The greatest offender I've found is how in phobos, arrays do not behave > > as ranges without importing the module defining their range operations. > > Would make sense to move those few primitives to object.d. I've been > thinking of that a long time ago but back then there was a vague stance > that object.d shouldn't contain templates. Since then that has changed. > -- Andrei The main problem with moving them there is auto-decoding. front and popFront for strings require std.utf in order to do their thing. So, if we move them to druntime, then that code would need to be duplicated (though similar code already has to exist in druntime for foreach loops), and we'd have a problem with invalid unicode in that it couldn't through std.utf.UTFException like it would now (though ideally, we'd stop throwing on invalid unicode and use the replacement character). That being said, I agree that the range functions for arrays should go in object.d. It's just that the way we handle narrow strings as ranges makes it problematic. - Jonathan M Davis
Re: Argumnentation against external function operator overloading is unconvincing
On 9/22/16 6:38 AM, pineapple wrote: The greatest offender I've found is how in phobos, arrays do not behave as ranges without importing the module defining their range operations. Would make sense to move those few primitives to object.d. I've been thinking of that a long time ago but back then there was a vague stance that object.d shouldn't contain templates. Since then that has changed. -- Andrei
Re: Argumnentation against external function operator overloading is unconvincing
On Wednesday, 21 September 2016 at 19:01:40 UTC, Timon Gehr wrote: There is no technical reason that would make the implementation of this feature difficult, if that is your question. Basically, the rationale is: external operators cannot be used in generic code that does not import the module defining the operators. C++ works around this using ADL. Walter (justifiably) does not like ADL, hence the limitation. (I don't agree with that line of reasoning: obviously this is not only an issue for operators, but for any UFCS function; operators are mere syntactic sugar.) The greatest offender I've found is how in phobos, arrays do not behave as ranges without importing the module defining their range operations. Which I think is obnoxious, and maybe there's a solution to make both cases less so, but I definitely don't think it's an argument for exclusion.
Re: Argumnentation against external function operator overloading is unconvincing
On Thursday, 22 September 2016 at 08:58:54 UTC, HaraldZealot wrote: So let's vote for the following sentence: "It would be good to have an operator overloading even without support in generic function" Yes
Re: Argumnentation against external function operator overloading is unconvincing
On Thursday, 22 September 2016 at 08:53:26 UTC, HaraldZealot wrote: OK, it seems to me it's time to investigate a community opinion. So let's vote for the following sentence: "It would be good to have an operator overloading even without support in generic function"
Re: Argumnentation against external function operator overloading is unconvincing
On Thursday, 22 September 2016 at 07:14:52 UTC, H. S. Teoh wrote: It's not so simple. The UFCS operator overload could be declared in a different module from the type itself. Then there is no guarantee that it will be found. Multiple calls to the same template function with the same argument types may result in different semantics, depending on what was imported. ... T I mean (in terms of your example) that in `main` before instantiate an `algorithm` we parse all symbols visible at this point from `main`, select all possible symbols which can be called with `UserType` (including templates, meh o_O), make the set union for the such table for the rest templates params, and give this symbol table the `algorithm`. But yes this creates problem that in different module we can have different instantiation :( Probably with other radical approach (see bellow *) for generic we can solve this, but this just destroy true templates (from which we benefits now) and provide something like Java-like solution :( * radical approach for generic: each generic function creates a pseudo parameter for function pointers on the base of signature of function used in the body of generic. The caller of generic just fill this pseudo parameters with real function visible for caller at call-point.
Re: Argumnentation against external function operator overloading is unconvincing
On Thursday, September 22, 2016 00:14:52 H. S. Teoh via Digitalmars-d wrote: > Normally this isn't a problem (D's > module system will trigger an overload conflict and require explicit FQN > to unambiguously select the right overload), but FQN's are not an option > when the call is made from generic code. And in the case of operator overloads, FQN makes no sense at all, since it wouldn't be using the operator anymore. - Jonathan M Davis
Re: Argumnentation against external function operator overloading is unconvincing
On Thu, Sep 22, 2016 at 06:09:39AM +, HaraldZealot via Digitalmars-d wrote: > On Thursday, 22 September 2016 at 05:38:53 UTC, HaraldZealot wrote: > > And problem with generic code solve independently for all UFCS > > functions including operators. > > Unfortunately I don't know compiler and all related stuff internally > (time to change this ;) ), but it seems to me there is a way to solve > problem with UFCS and generic code without breakage of module > encapsulation. If understand correctly when we instantiate template > with struct/class we pass to template some kind of "static signature > of type" (if I may it call so). With the static signature of type I > mean all member function of class or struct. If we instead of this > before instantiate any type (including POD) create "dynamic signature > of type" which include all visible at the moment of call function and > templates for this type. It's not so simple. The UFCS operator overload could be declared in a different module from the type itself. Then there is no guarantee that it will be found. Multiple calls to the same template function with the same argument types may result in different semantics, depending on what was imported. Or there could be more than one module that declares the operator, possibly with conflicting semantics. Normally this isn't a problem (D's module system will trigger an overload conflict and require explicit FQN to unambiguously select the right overload), but FQN's are not an option when the call is made from generic code. T -- Ignorance is bliss... until you suffer the consequences!
Re: Argumnentation against external function operator overloading is unconvincing
On Thursday, 22 September 2016 at 05:38:53 UTC, HaraldZealot wrote: And problem with generic code solve independently for all UFCS functions including operators. Unfortunately I don't know compiler and all related stuff internally (time to change this ;) ), but it seems to me there is a way to solve problem with UFCS and generic code without breakage of module encapsulation. If understand correctly when we instantiate template with struct/class we pass to template some kind of "static signature of type" (if I may it call so). With the static signature of type I mean all member function of class or struct. If we instead of this before instantiate any type (including POD) create "dynamic signature of type" which include all visible at the moment of call function and templates for this type.
Re: Argumnentation against external function operator overloading is unconvincing
On Wednesday, 21 September 2016 at 21:14:15 UTC, H. S. Teoh wrote: Thank you both, I see now. So it seems to be essential point. But because we already have the same problem with UFCS, I don't see why we should prohibit external overloading of operator, it is just inequality (in political sense) for operators :) In any case we at least should change rationale, because current three points are have nothing with real problem and shouldn't be obstacle. But probably the best solution is implementing external operator overloading, just to omit special case. And problem with generic code solve independently for all UFCS functions including operators.
Re: Argumnentation against external function operator overloading is unconvincing
On Wed, Sep 21, 2016 at 08:53:06PM +, HaraldZealot via Digitalmars-d wrote:
> On Wednesday, 21 September 2016 at 19:01:40 UTC, Timon Gehr wrote:
> >
> > Basically, the rationale is: external operators cannot be used in
> > generic code that does not import the module defining the operators.
>
> Could you give some elaborate example, for now I can't imagine what
> your mean.
Here's a simple example:
// usertype.d
module usertype;
struct UserType { ... }
auto opBinary(string op : "+")(UserType u1, UserType u2) { ... }
// generic_code.d
module generic_code;
auto algorithm(T,U)(T t, U u) {
return t + u;
}
// main.d
module main;
import usertype;
import generic_code;
void main() {
UserType u1, u2;
auto r = u1 + u2; // OK
auto s = algorithm(u1, u2); // NO GOOD
}
The problem here is that generic_code.d doesn't (and shouldn't!) import
usertype.d, so usertype.opBinary is not visible in generic_code.d. So
when algorithm() tries to look up the '+' operator in `t + u`, it can't
find the declaration and fails. There is no way to find the correct
opBinary() because it's not part of UserType, so algorithm() has no way
to access that symbol.
Using the operator in module main is OK, because main (rightfully)
imports usertype.d, so the operator is visible. But any generic code
that main imports will have a problem because they can't (and
shouldn't!) know ahead of time which modules contain the declaration
they need.
In C++ this problem is solved using ADL, but ADL brings with it other
problems, the root of which is that it breaks module encapsulation.
There's actually some instances of this problem (w.r.t. UFCS) in Phobos,
that currently requires ugly workarounds like importing modules that the
generic code really shouldn't be depending on.
T
--
Computerese Irregular Verb Conjugation: I have preferences. You have biases.
He/She has prejudices. -- Gene Wirchenko
Re: Argumnentation against external function operator overloading is unconvincing
On 21.09.2016 22:53, HaraldZealot wrote:
On Wednesday, 21 September 2016 at 19:01:40 UTC, Timon Gehr wrote:
Basically, the rationale is: external operators cannot be used in
generic code that does not import the module defining the operators.
Could you give some elaborate example, for now I can't imagine what your
mean.
module a;
struct Foo{}
Foo opBinary(string op:"+")(Foo a, Foo b){ return Foo(); }
---
module b;
T add(T)(T a,T b){
return a + b;
}
---
module c;
import a,b;
void main(){
Foo x=add(Foo(),Foo()); // error
}
Re: Argumnentation against external function operator overloading is unconvincing
On Wednesday, 21 September 2016 at 19:01:40 UTC, Timon Gehr wrote: Basically, the rationale is: external operators cannot be used in generic code that does not import the module defining the operators. Could you give some elaborate example, for now I can't imagine what your mean.
Re: Argumnentation against external function operator overloading is unconvincing
On Wednesday, 21 September 2016 at 19:01:40 UTC, Timon Gehr wrote: Basically, the rationale is: external operators cannot be used in generic code that does not import the module defining the operators. So why not have the struct/class explicitly import external operators? You can do this currently with a separate module defining them and then the struct/class imports the whole thing. Kind of annoying, but would get the job done, no? Alternately, you could invent some new syntax. Maybe something like import this : Matrix plus(Matrix A, Matrix B);
Re: Argumnentation against external function operator overloading is unconvincing
On 21.09.2016 21:01, Timon Gehr wrote: On 21.09.2016 19:57, HaraldZealot wrote: So if someone has real rationale not to have operator overloading as external function I'm curios to arguments. [1] http://dlang.org/rationale.html There is no technical reason that would make the implementation of this feature difficult, if that is your question. Basically, the rationale is: external operators cannot be used in generic code that does not import the module defining the operators. C++ works around this using ADL. Walter (justifiably) does not like ADL, hence the limitation. (I don't agree with that line of reasoning: obviously this is not only an issue for operators, but for any UFCS function; operators are mere syntactic sugar.) BTW, another argument in favour of free function operators is opOpAssign for classes.
Re: Argumnentation against external function operator overloading is unconvincing
On 21.09.2016 19:57, HaraldZealot wrote: So if someone has real rationale not to have operator overloading as external function I'm curios to arguments. [1] http://dlang.org/rationale.html There is no technical reason that would make the implementation of this feature difficult, if that is your question. Basically, the rationale is: external operators cannot be used in generic code that does not import the module defining the operators. C++ works around this using ADL. Walter (justifiably) does not like ADL, hence the limitation. (I don't agree with that line of reasoning: obviously this is not only an issue for operators, but for any UFCS function; operators are mere syntactic sugar.)
Re: Argumnentation against external function operator overloading is unconvincing
On Wednesday, 21 September 2016 at 17:57:17 UTC, HaraldZealot wrote: In current D, overloading operator like "+" with external function is prohibited. There is the rationale [1] (see second paragraph). [...] I am completely agree. We should support external operator overloading for ndslice extension. - Ilya
