Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 05:35:20 UTC, rikki cattermole wrote: On 04/07/2022 5:30 PM, Andrej Mitrovic wrote: Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Not necessarily. It could and should most likely mean that it won't do any heap allocations. Heap allocations are expensive after all. @safe @nogc @noheap? :) Baby, you got a stew going!
Re: The D Programming Language Vision Document
On 04/07/2022 5:30 PM, Andrej Mitrovic wrote: Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Not necessarily. It could and should most likely mean that it won't do any heap allocations. Heap allocations are expensive after all.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document Under 'Memory safety': Allow the continued use of garbage collection as the default memory management strategy without impact. The GC is one of D's strengths, and we should not "throw the baby out with the bath water". Under 'Phobos and DRuntime': @nogc as much as possible. Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Then it's not part of D's strengths.
Re: The D Programming Language Vision Document
We have a perfectly good Unicode handling library already. (Okay, little out of date and doesn't handle Turkic stuff, but fixable). The standard one is called ICU. Anyway, we are straying from my original point, that limiting ourselves to the string alias and not supporting wstring or dstring in Phobos is going to bite us. Its not what people expect, its not what we have supported and code that looks like it should work won't. There better be a good reason for this that isn't just removing templates.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 20:28:18 UTC, rikki cattermole wrote: We only support UTF-16/UTF-32 for the target endian. Text input comes from many sources, stdin, files and say the windowing system are three common sources that do not make any such guarantees. Well, then the application author will use an external Unicode library anyway. If you support UTF-16 or UTF-32 there might not be a BOM mark, so you might need to use heuristics to figure out the LE/LB endian issue. For things like gzip, png, crypto and unicode there are most likely faster and better tested open source alternatives than a small community can come up with. Maybe just use out whatever Chromium or Clang uses? What I never liked about C++ is the string mess: char, signed char, unsigned char, char8_t, char16_t, char32_t, wchar_t, string, wstring, u8string, u16string, u32string, pmr::string, pmr::wstring, pmr::u8string, pmr::u16string, pmr::u32string… And this doesn't even account for endianess!! This is what happens over time as new needs pops up. One of the best things about Python3 and JavaScript is that there is one commonly used string type that is well supported. Having one common string representation is a good thing for API authors. (But make sure to have a maintained binding to a versatile C unicode library.)
Re: The D Programming Language Vision Document
On 04/07/2022 8:16 AM, Ola Fosheim Grøstad wrote: On Sunday, 3 July 2022 at 19:32:56 UTC, rikki cattermole wrote: It is required for string equivalent comparisons (which is what you should be doing in a LOT more cases! Anything user provided when compared should be normalized first. Well, I think it is reasonable for a protocol to require that the input is NFC, and just check it and reject it or call out to an external library to convert it into NFC. Anyway, UTF-8 is the only format that isn't affected by network byte order… So if you support more than UTF-8 then you have to support UTF-8, UTF16-LE, UTF16-BE, UTF-32LE, UTF-32BE… That is five formats for just a simple string… and only UTF-8 will be well tested by users. :-/ https://issues.dlang.org/show_bug.cgi?id=23186 We only support UTF-16/UTF-32 for the target endian. Text input comes from many sources, stdin, files and say the windowing system are three common sources that do not make any such guarantees.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 19:32:56 UTC, rikki cattermole wrote: It is required for string equivalent comparisons (which is what you should be doing in a LOT more cases! Anything user provided when compared should be normalized first. Well, I think it is reasonable for a protocol to require that the input is NFC, and just check it and reject it or call out to an external library to convert it into NFC. Anyway, UTF-8 is the only format that isn't affected by network byte order… So if you support more than UTF-8 then you have to support UTF-8, UTF16-LE, UTF16-BE, UTF-32LE, UTF-32BE… That is five formats for just a simple string… and only UTF-8 will be well tested by users. :-/
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 19:32:56 UTC, rikki cattermole wrote: I have just finished implementing string normalization which is based around UTF-32. There's a difference between utf-32 and unicode code points. It is required for string equivalent comparisons (which is what you should be doing in a LOT more cases! Anything user provided when compared should be normalized first. Which you can do on any translation format.
Re: The D Programming Language Vision Document
On 04/07/2022 7:18 AM, Ola Fosheim Grøstad wrote: I hardly ever use anything outside UTF-8, and if I do then I use a well tested unicode library as it has to be correct and up to date to be useful. The utility of going beyond UTF-8 seems to be limited: https://en.wikipedia.org/wiki/UTF-32#Analysis I have just finished implementing string normalization which is based around UTF-32. It is required for string equivalent comparisons (which is what you should be doing in a LOT more cases! Anything user provided when compared should be normalized first.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 18:33:29 UTC, rikki cattermole wrote: On 04/07/2022 6:10 AM, Ola Fosheim Grøstad wrote: People who are willing to use 4 bytes per code point are probably using third party C-libraries that have their own representation, so you have to convert anyway? If you use Unicode and follow their recommendations, you are going to be using dstrings at some point. I hardly ever use anything outside UTF-8, and if I do then I use a well tested unicode library as it has to be correct and up to date to be useful. The utility of going beyond UTF-8 seems to be limited: https://en.wikipedia.org/wiki/UTF-32#Analysis
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 18:33:29 UTC, rikki cattermole wrote: Its just an unnecessary goal, when most of the string algorithms we have probably don't care about the encoding and those that do probably will be using dstrings. To the contrary, I find this goal coherant with the end of autodecoding. that will probably make phobos simpler: less template overloads, less template constraints to evaluate, no more isNattowString etc.
Re: The D Programming Language Vision Document
On 04/07/2022 6:10 AM, Ola Fosheim Grøstad wrote: People who are willing to use 4 bytes per code point are probably using third party C-libraries that have their own representation, so you have to convert anyway? If you use Unicode and follow their recommendations, you are going to be using dstrings at some point. For example, string equivalence, and anything to do with case is going to use them and very likely to require multiple memory allocations to do it. Its just an unnecessary goal, when most of the string algorithms we have probably don't care about the encoding and those that do probably will be using dstrings.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 17:27:43 UTC, rikki cattermole wrote: That's going to bite us big time when it comes to Unicode handling which wants to work with dstring's. You can just use ints… It is better to do something commonly used well, than have features that not enough people use to get the quality up. People who are willing to use 4 bytes per code point are probably using third party C-libraries that have their own representation, so you have to convert anyway?
Re: The D Programming Language Vision Document
> Stronger integration with other languages One of the things I judge D's compilers by is how well they can build a shared library. This is crucial for a lot of different applications of D and can be an complete stopper in using D if it doesn't "just work". To be blunt this is embarrassing, this should have been a top priority 10+ years ago... > Phobos and DRuntime I am very worried that this is going ahead without signatures. Its a major usability issue that concepts like ranges are not written into a function signature and is a common tripping point for people new to the language. I've been meaning to talk with Walter about this, this year. Times just haven't lined up at BeerConf to sort out lining up my designs into something that could actually go in. > No wstring or dstring. Any functions in Phobos v2 that deal with strings should deal exclusively with the string type. Users can convert from and to the other string types as needed. NOPEEE That's going to bite us big time when it comes to Unicode handling which wants to work with dstring's. > Provide automatic CI for actively-maintained third-party projects. I would like a big endian system to be included if possible, if a library is actively maintained you don't want surprises to arise from that.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: Feedback is welcome. Metaprogramming This section is... light on details and does little to clear up if you share my goals. My current take on this is that I believe something with c feature ser + templates are the future, their foundation extremely shaky and awful with c++ appearing to make it powerful by accident, and that I should be wherever there is the most sugar to make it livable. This will mostly involve me wanting an endless stream of new features to play with. But this contradicts the earlier section about simplicity and avoiding complex features. Templates generally get ugly syntax, alias when you want to store first class type, static if's everywhere, ___traits; so Id suggests that's updated to be complex features that affect standard code but generally be open to experimental features in the template space; so that templates are generally going to get the short end of the stick design-wise but that but open to exploring out the foundation meta programming needs. Phobos and DRuntime You don't comment on how v2 should be organized. Currently, I believe to write functional-style code your going to import 5 different std libs and at least algorithm and range. Is that going to continue or be fixed? I find the naming conflicts of "write" to be fairly awful and makes `import std;` undoable and even if you always write out your imports, studio and file are often going to be used together. Will that be fixed? etc. etc. the goals of std v2 probably could get its own document. Stronger ecosystem Community management All D users and contributors must feel comfortable participating in the D community. There is a divide between those who believe in a "kitchen sink" standard library and those who support a minimal standard library backed up by a large ecosystem. We must find a balance that makes sense for the D programming language. Id suggest dropping std.experimental and get a std.community sort of thing going. Where given snar did sumtypes as a subtype lib, then got into std.experimental, had to follow your process and whatever; and then finally got merged into std. Instead, I'd suggest that snar makes an important lib, it gets noticed by the community, std.community.sumtype will point at snar's github either auto-downloaded or batched with compiler releases. Have whatever legal disclaimer that std.community is not your responsibility, is kinda bad style to overuse and should be verified yourself. But have a curated list of community projects that have a soft thumbs up and that are easy to use. While dropping the std.experimental take that everyone says is a bad experience.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: Feedback is welcome. Had a hope to see the "complete started but abandoned features" point.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 11:39:45 UTC, Mike Parker wrote: Language evolution doesn't really mean much until we get all of this sorted. Good point. That's right. But Walter wants to minimize its use in Phobos v2, and there's a strong desire to have a pay-as-you-go DRuntime. I'm not the person to speculate on how the GC fits into that, but I do know they don't yet want git rid of it. Yes, the Phobos issue is probably a good point, but I don't think the standard library prevents experienced developers from doing anything (People can write their own or use third party solutions). Although I have been pro GC-free in the past, I am also not so sure if GC-free is the sweet spot in 2022 (due to Rust and C++ having reduced friction significantly). To my mind the sweet spot for larger applications would be to write your own runtime/GUI-framework/libraries in @system and cover your @safe application code with a convenient «non-stop» (or at least only «local stop») GC/ARC solution. But I understand that you cannot say anything specific on this at this point in time. (On a related note, I'll soon be publishing a video of a conversation I had with Walter about origins of D, and he said something there about the GC that really surprised me.) That would be interesting to hear more about as the GC was what surprised me the most when I first tried D as a C++ «descendant».
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 11:13:42 UTC, Ola Fosheim Grøstad wrote: On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: Feedback is welcome. Thank you for putting this in clear terms. I miss an overarching «primary use scenarios» to guide further language evolution. How do you know if new language features are good or bad if you have no scenarios to measure them up against? That's not something we've discussed yet. Right now, the focus is on plugging holes in the existing language, building out the ecosystem, and overhauling Phobos. These areas are where we see some of the loudest complaints. Language evolution doesn't really mean much until we get all of this sorted. But we'll start discussing it at some point, at which time you'll see some new high-level goals appear. In the meantime, Walter and Atila will continue evaluating DIPs on a case-by-case basis. (Speaking of which, I want to look at overhauling the DIP process a bit some time next year). It is nice to see that improved move semantics is a goal, then I guess ARC could be something one could envision down the line. That said, I am a bit disappointed that there is no hint of a departure from the current STOP-the-world GC regime, but I guess that is the reflecting reality. My interpretation of the vision document is that the core team sees no need to change the current GC strategy. That's right. But Walter wants to minimize its use in Phobos v2, and there's a strong desire to have a pay-as-you-go DRuntime. I'm not the person to speculate on how the GC fits into that, but I do know they don't yet want git rid of it. (On a related note, I'll soon be publishing a video of a conversation I had with Walter about origins of D, and he said something there about the GC that really surprised me.)
Re: The D Programming Language Vision Document
On 03.07.22 10:46, Mike Parker wrote: You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document Quoting from the "Memory safety" section: The language maintainers do not see memory safety as a fad, nor is their focus on its implementation in the D programming language a form of "chasing" other languages. There's no need to be so defensive, particularly in the first sentence of the section. Also, the "chasing" part is cryptic. What other languages are you talking about? Why can their names not be uttered? I know it's Rust, but other readers might not. Just ax that sentence and start with "The language maintainers see memory safety as a critical component [...]". DIP 1000 is crucial to this because it eliminates most of the reasons why D code written as simply as possible is not @safe. Dubious claim. My experience is that people who try `scope` quickly run into its limitations. Eliminate undefined behavior in @safe code. I.e., fix bugs. That's hardly worth mentioning as a high-level goal. it's not possible to write a vector type where the following code is @safe. auto v = vector(1, 2, 3); v ~= 4; That example isn't clear at all. I suppose the point is to (1) avoid the GC and (2) still allow taking the addresses of the elements. That isn't obvious from those two lines of code, at all. As presented, `vector` can easily be implemented @safe-ly with a dynamic array. Even as we strive to increase memory safety in D, we must always ensure that programmers who need or want to eschew memory safety features can do so. And they must be able to do so with minimal friction. "Minimal friction" would mean not making @safe default, as that adds friction. Little friction is the real goal.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: Feedback is welcome. Thank you for putting this in clear terms. I miss an overarching «primary use scenarios» to guide further language evolution. How do you know if new language features are good or bad if you have no scenarios to measure them up against? It is nice to see that improved move semantics is a goal, then I guess ARC could be something one could envision down the line. That said, I am a bit disappointed that there is no hint of a departure from the current STOP-the-world GC regime, but I guess that is the reflecting reality. My interpretation of the vision document is that the core team sees no need to change the current GC strategy.
The D Programming Language Vision Document
You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document I'll fill out the remaining sections as time allows over the coming weeks. Once it's complete, I'll make sure it's linked in the Community section of the menu bar. I want to emphasize that what we have there right now are a list of high-level goals. Please read my summary of our June meeting for more info on the motivation behind it. https://forum.dlang.org/thread/buythmwbdaswuqbtk...@forum.dlang.org Some of these goals we want to actively pursue, others are things we'll take advantage of when the opportunity arises. For example, completing the memory safety story is an active project, but looking for features to remove, or to put on the "no recommended list" is something we'll evaluate when it comes up. Improving the ecosystem is a major goal. Right now, we don't have the resources in place to make that happen, but we expect to be there next year once we have an ecosystem management team in place. In the meantime, if you have an idea for a new tool or project that fits into our goals, and you don't want to start working on it until you know it's going have the foundation's approval or support, then please let me know. Hash out ideas of the project in the forums or on Discord if you need to, but we'll want you to bring a proposal to one of our monthly meetings for discussion. Then you'll know if it's worth your time. Just be aware that we do not have the resources to oversee things right now to the extent we would like, so anyone looking to make this kind of contribution should be self-motivated and independent. We'll support you as much as we can, and we may even be willing to pay out a contract in some cases, but for now you're largely going to be on your own. I also want to reiterate that this is a living document. We will modify the goals as needed as time goes by. Feedback is welcome.
