Re: The D Programming Language Vision Document
On Saturday, 9 July 2022 at 23:40:21 UTC, monkyyy wrote: Its not curated Well, limit yourself to usage of `std` or `phobos` or whatever official implementation is called - and voila, it's curated. It changes the compiler options So? Most of the people prefer to have build tool managing possible compiler options for them instead of typing the manually for each compiler, digging through all dependencies involved to find required options for them, etc, finally resulting in reinventing the same build tool. Especially when you have an option to manually override anything you dont like. and its randomly configured in js Dub has nothing common to JS. If you mean `dub.json` format - well, JSON is not a JS. And if you are allergic to human-readable formats, you can always use `dub.sdl` flavor.
Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 08:34:14 UTC, Mike Parker wrote: Walter and Atila have both talked about rethinking exceptions. Atila brought it up in his DConf Online 2021 talk here: https://youtu.be/UqW42_8kn0s?t=1429 You can see Walter's comments about it in the subsequent Q & A session here: https://youtu.be/g26eJcs2QB0?t=1809 Thank you. I hope to see something more tangible in the future as it's an interesting area to explore. I think that's a reasonable policy. The last thing any of us want is to stifle debate or censor opinions, but we feel that it's reasonable to ask people to participate in debates and express their opinions without upsetting others. So we're going to do our best to find a middle ground. Bit of shame you feel the need to make a change given the circumstances you've detailed. I think you've done a fine job thus far. Appreciate your response!
Re: The D Programming Language Vision Document
On Saturday, 9 July 2022 at 23:40:21 UTC, monkyyy wrote: ... and its randomly configured in js. json or sdl, with well documented structure, not random js.
Re: The D Programming Language Vision Document
On Saturday, 9 July 2022 at 22:59:19 UTC, Vladimir Marchevsky wrote: On Sunday, 3 July 2022 at 17:09:46 UTC, monkyyy wrote: Id suggest dropping std.experimental and get a std.community sort of thing going. Well, it's named dub :) Its not curated. Its enforces a vision. It changes the compiler options. and its randomly configured in js.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 17:09:46 UTC, monkyyy wrote: Id suggest dropping std.experimental and get a std.community sort of thing going. Well, it's named dub :)
Re: The D Programming Language Vision Document
Thanks for this. As a casual D user who is coming from a Java background, I'm of the opinion that we do indeed need to improve Phobos to either (a) include lots of stuff in the std lib itself, or (b) guide users to trusted, well-tested third-party libraries. I've tried doing things with asynchronous stuff, http, encrypted sockets, etc, for small utilities we use at my work, but a decent amount of time is spent on just looking for a good library that works. Specifically, a single unifying async library similar to JS promises or Java's futures would drastically improve the ecosystem, imo.
Re: The D Programming Language Vision Document
Mike has our full support in his moderation policy and authority.
Re: The D Programming Language Vision Document
On Wednesday, 6 July 2022 at 21:30:44 UTC, Dukc wrote: And I think there is still pretty much value in handling UTF-16 strings because that's what many other languages use. With the current vision, Phobos V2 won't handle UTF16 in place. We'll have to convert it to UTF8 before manipulation, which is probably not optimal. Oh, there is no doubt that handling UTF16 should be possible, but it can be done just as well, if not better, as a support library. But it is very much undesirable to have more than a single string format for library authors to deal with.
Re: The D Programming Language Vision Document
On Wednesday, 6 July 2022 at 15:43:25 UTC, ryuukk_ wrote: On Wednesday, 6 July 2022 at 14:30:07 UTC, Paulo Pinto wrote: On Tuesday, 5 July 2022 at 12:34:57 UTC, ryuukk_ wrote: On Monday, 4 July 2022 at 05:30:10 UTC, Andrej Mitrovic wrote: [...] GC is one of D's strength because it is optional, not making core APIs bing-your-own-memory-allocation-strategy through nogc or allocators, is making it no longer optional, which is no longer a strength imo You don't want GC when you do microcontroller development, so as a result core APIs (most of them) becomes useless, moving forward that should make the story better for everyone Which becomes a strength again Feel free to consider it a strength, when in reality it is a flaw against established market players. https://www.microej.com/ https://www.wildernesslabs.co/ https://www.ptc.com/en/products/developer-tools/perc https://www.aicas.com/wp/products-services/jamaicavm/ https://www.astrobe.com/ People also use nodejs and npm, what is your point? If you invest in the future you must take the pragmatic approach and give options Those are not Oracle's products, companies took the JVM for what it is as a foundation and built their products They haven't picked the default Oracle JDK and the default concurrent GC D should enable similar stories with what it has and can provide, read on the challenges TinyGO faced, if D provides the tools for companies to experiment with it, with a proper set of efficient and minimal Core APIs, that alone makes it a proper and more efficient alternative solution My point is that GC hate is not a fixed problem in D, and the vision does little to fix it. Meanwhile the language communities that embraced GC in embedded deployment, are at least 20 years ahead in production deployments versus D, where no GC keeps being a reason to rejoice, like the the comment I replied to. The future was when Andrei book came out.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 20:16:35 UTC, Ola Fosheim Grøstad wrote: On Sunday, 3 July 2022 at 19:32:56 UTC, rikki cattermole wrote: It is required for string equivalent comparisons (which is what you should be doing in a LOT more cases! Anything user provided when compared should be normalized first. Well, I think it is reasonable for a protocol to require that the input is NFC, and just check it and reject it or call out to an external library to convert it into NFC. Anyway, UTF-8 is the only format that isn't affected by network byte order… So if you support more than UTF-8 then you have to support UTF-8, UTF16-LE, UTF16-BE, UTF-32LE, UTF-32BE… It is pretty easy to convert those to native endian and back with functions in `std.bitmanip`. I recently did so to have a program to recognise files in all of those five. Also the Phobos functions are of high quality. They work extremely well with the range API (other than having to live with autodecoding), they are well documented and they are comprehensive enough for almost any task. I don't recall having ever considered another library for handling Unicode. And I think there is still pretty much value in handling UTF-16 strings because that's what many other languages use. With the current vision, Phobos V2 won't handle UTF16 in place. We'll have to convert it to UTF8 before manipulation, which is probably not optimal. And if the string functions have to deal with two formats anyway, also supporting UTF32 on top of them probably does not make much difference. That said, I don't feel strongly about this because if we kick UTF16 and UTF32 functions out of Phobos, they still are presumably available in Undead.
Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 08:34:14 UTC, Mike Parker wrote: [snip] I think that's a reasonable policy. The last thing any of us want is to stifle debate or censor opinions, but we feel that it's reasonable to ask people to participate in debates and express their opinions without upsetting others. So we're going to do our best to find a middle ground. Thanks for the explaination. I agree with all of this. I think we should get into habit to start a new thread for branching discussions with much lower bar than we currently do. Myself included.
Re: The D Programming Language Vision Document
On Wednesday, 6 July 2022 at 14:30:07 UTC, Paulo Pinto wrote: On Tuesday, 5 July 2022 at 12:34:57 UTC, ryuukk_ wrote: On Monday, 4 July 2022 at 05:30:10 UTC, Andrej Mitrovic wrote: On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document Under 'Memory safety': Allow the continued use of garbage collection as the default memory management strategy without impact. The GC is one of D's strengths, and we should not "throw the baby out with the bath water". Under 'Phobos and DRuntime': @nogc as much as possible. Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Then it's not part of D's strengths. GC is one of D's strength because it is optional, not making core APIs bing-your-own-memory-allocation-strategy through nogc or allocators, is making it no longer optional, which is no longer a strength imo You don't want GC when you do microcontroller development, so as a result core APIs (most of them) becomes useless, moving forward that should make the story better for everyone Which becomes a strength again Feel free to consider it a strength, when in reality it is a flaw against established market players. https://www.microej.com/ https://www.wildernesslabs.co/ https://www.ptc.com/en/products/developer-tools/perc https://www.aicas.com/wp/products-services/jamaicavm/ https://www.astrobe.com/ People also use nodejs and npm, what is your point? If you invest in the future you must take the pragmatic approach and give options Those are not Oracle's products, companies took the JVM for what it is as a foundation and built their products They haven't picked the default Oracle JDK and the default concurrent GC D should enable similar stories with what it has and can provide, read on the challenges TinyGO faced, if D provides the tools for companies to experiment with it, with a proper set of efficient and minimal Core APIs, that alone makes it a proper and more efficient alternative solution
Re: The D Programming Language Vision Document
On Tuesday, 5 July 2022 at 12:34:57 UTC, ryuukk_ wrote: On Monday, 4 July 2022 at 05:30:10 UTC, Andrej Mitrovic wrote: On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document Under 'Memory safety': Allow the continued use of garbage collection as the default memory management strategy without impact. The GC is one of D's strengths, and we should not "throw the baby out with the bath water". Under 'Phobos and DRuntime': @nogc as much as possible. Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Then it's not part of D's strengths. GC is one of D's strength because it is optional, not making core APIs bing-your-own-memory-allocation-strategy through nogc or allocators, is making it no longer optional, which is no longer a strength imo You don't want GC when you do microcontroller development, so as a result core APIs (most of them) becomes useless, moving forward that should make the story better for everyone Which becomes a strength again Feel free to consider it a strength, when in reality it is a flaw against established market players. https://www.microej.com/ https://www.wildernesslabs.co/ https://www.ptc.com/en/products/developer-tools/perc https://www.aicas.com/wp/products-services/jamaicavm/ https://www.astrobe.com/
Re: The D Programming Language Vision Document
On Tuesday, 5 July 2022 at 11:49:20 UTC, ryuukk_ wrote: I am sad that no word on the Allocator API, moving forward i personally think APIs that use memory should be required to ask for an Allocator and do their allocation using it, and only it A default GCAllocator could be used if none provided, this allows users of all kind to enjoy the APIs without having to complain about the GC or their inability to integrate the APIs in their game engine for example It should not be resolved like this. Functions that does not return memory should just be nogc. Functions that return allocated memory that are nogc should use RAII and prevent GC pointers from pointing to it. So you need a new type system. Or just overload on @system. You also want to get rid of destructors on GC objects and replace it with a finalizer that isnt sensitive to order.
Re: The D Programming Language Vision Document
On Tuesday, 5 July 2022 at 12:34:57 UTC, ryuukk_ wrote: GC is one of D's strength because it is optional, not making core APIs bing-your-own-memory-allocation-strategy through nogc or allocators, is making it no longer optional, which is no longer a strength imo You don't want GC when you do microcontroller development, so as a result core APIs (most of them) becomes useless, moving forward that should make the story better for everyone Which becomes a strength again! Sure, I agree with you. I think I was just being pedantic. :)
Re: The D Programming Language Vision Document
On 05/07/2022 11:49 PM, ryuukk_ wrote: Hopefully that includes proper built in Tagged Union, non OOP people need that otherwise we are stuck in C's era of programming C's era of programming also happens to coincide with ML which had tagged unions ;) The C family has never been very expressive.
Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 05:30:10 UTC, Andrej Mitrovic wrote: On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document Under 'Memory safety': Allow the continued use of garbage collection as the default memory management strategy without impact. The GC is one of D's strengths, and we should not "throw the baby out with the bath water". Under 'Phobos and DRuntime': @nogc as much as possible. Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Then it's not part of D's strengths. GC is one of D's strength because it is optional, not making core APIs bing-your-own-memory-allocation-strategy through nogc or allocators, is making it no longer optional, which is no longer a strength imo You don't want GC when you do microcontroller development, so as a result core APIs (most of them) becomes useless, moving forward that should make the story better for everyone Which becomes a strength again!
Re: The D Programming Language Vision Document
Pattern matching
Hopefully that includes proper built in Tagged Union, non OOP
people need that otherwise we are stuck in C's era of programming
We can, and we should do better (import std.sumtype is
embarrassing when other languages have it built in without
template soup, i still refuse to touch it)
Tuples
Awesome! hopefully they become built in and more integrated
Async/await
This one will be tricky, done well means it scale from 0 to
hearo, meaning i should be able to use it without the need of a GC
Hopefully it's not one of these import `std.concurency: async,
await, frame;`
Stronger traits
That's awesome to hear, metaprogramming is one of D's strength
On the tooling part:
A language server is long overdue..
https://pkg.go.dev/golang.org/x/tools/gopls
https://blog.rust-lang.org/2022/07/01/RLS-deprecation.html
https://rust-analyzer.github.io/
The one we have right now (serve-d) is nice, but uses the same
backend as every other projects (DCD), and this project haven't
evolved in years, the moment you step into template territory, it
falls short ``T get(T)() { return T.init } auto myInt =
get!int;`` it'll not know what myInt is.. wich is a shame when
that feature is D's signature
A better debugging story is also essential, without having to use
external (incomplete) files like:
https://github.com/Pure-D/dlang-debug
msvc, clang, gcc: they should know what D types are, and they
should be able to step properly into D syntax without the need of
the user to do weird things
Zig is a young language and yet has a better debugging story
Onto the phobos/druntime:
That's nice to hear that the goal is to reduce inter module
dependencies
Core APIs should be made independent and VERY simple, this allows
them to be easily ported to other platforms, easier to extend
I am sad that no word on the Allocator API, moving forward i
personally think APIs that use memory should be required to ask
for an Allocator and do their allocation using it, and only it
A default GCAllocator could be used if none provided, this allows
users of all kind to enjoy the APIs without having to complain
about the GC or their inability to integrate the APIs in their
game engine for example
--
Overall this document is super encouraging, thanks for making it,
D finally have a long term roadmap!
Hopefully this allows everyone to align and coordinate their
efforts
I wish i could help, i tried, but language programming is
definitely not my cup of tea, hopefully you guys are not sick of
my suggestions
A language is about allowing user to express their intent, making
it easier for them to achieve that goal is a priority, compiler
internal difficulties should not prevent language improvements,
compiler internals doesn't exist from the user point of view
Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 16:12:35 UTC, rikki cattermole wrote: https://www.unicode.org/Public/14.0.0/ucd/NormalizationTest.txt Argh, linking to large files... My implementation passes this :3 It should be complete test cases. Well, you also have to test for the cases that should not trigger any change, and also for sequencing/parsing bugs. So, not complete, but a good start.
Re: The D Programming Language Vision Document
On 04/07/2022 7:39 PM, Ola Fosheim Grøstad wrote: Yes, that is a common one that is maintained, but maybe there are BOOST licensed implementations too? One can do an exhaustive test for say two-character normalization against ICU to see if they are compliant. https://www.unicode.org/Public/14.0.0/ucd/NormalizationTest.txt My implementation passes this :3 It should be complete test cases.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document I also want to reiterate that this is a living document. We will modify the goals as needed as time goes by. Feedback is welcome. In the document it is said: `@nogc` as much as possible. I think this should be extended to clarify how conversion to `@nogc` will be tackled. IMHO adding proper stable `@nogc` allocators to the stdlib is more important. When that's done, make sure we can set the allocator for stdlib types and add `@nogc` overloads that take an allocator parameter for non-`@nogc` functions. We aren't doing no-heap programming after all. If we have a stable allocator interface, replacing GC calls with allocator calls in Phobos shouldn't be too hard.
Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 07:28:23 UTC, StarCanopy wrote: On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: [...] Given the desire to reduce GC usage in PhobosV2, would it be fair to say this implies changing error handling schemes found therein, i.e. avoiding exceptions as they are implemented now, where reasonable? And if so, have core members mentioned any alternatives? Walter and Atila have both talked about rethinking exceptions. Atila brought it up in his DConf Online 2021 talk here: https://youtu.be/UqW42_8kn0s?t=1429 You can see Walter's comments about it in the subsequent Q & A session here: https://youtu.be/g26eJcs2QB0?t=1809 With regards to moderation policy, are there specific issues; about which you feel comfortable speaking; that prompted this item's inclusion in the vision document? We've had a very, very lax moderation policy for years. Sometimes threads get derailed and posts become overly aggressive, but we've tended to let things slide as long as commenters avoid blatantly direct personal attacks, racist remarks, etc. I've always felt we have a great forum community compared to some others I've experienced, and I've found the need to delete forum posts, given the existing policy, to be relatively rare. The flip side of that is that the larger internet community has evolved over the past couple of decades. What I consider tolerable if annoying behavior, others view as toxic. What I view as sarcasm or biting wit or some such, others view as personal attacks. We've received a number of complaints in recent months, directly and indirectly, about our forums being toxic. And that's not the image we want out there. And we definitely don't want people to actively choose not to participate in the forums just to avoid what they see as toxicity. So we want to tighten the screws a little bit. In terms of directly deleting posts unprompted, the policy isn't changing. If I wouldn't have deleted a post before, I won't delete it now. What *is* changing is that we're going to be a bit more active in nudging threads back on topic when they stray too far off, and asking people to keep things toned down if their remarks become aggressive to the extent that it looks like they're going to derail the thread. We've had incidents in the past where one or two people upset a number of other forum users, and we didn't act as soon as we should have to keep things from getting out of hand. In the past, forum users have sometimes reported posts to me indicating that they felt it was a direct personal attack. Usually, if *I* didn't see it as a direct attack, I wouldn't delete it. I'd explain how I see it and explain my reason for letting it stand. Going forward, I'll instead give a warning to the author of the post in question, and ask them to refrain from such remarks in future posts. And once a warning is given (stay on topic, please tone it down, please avoid remarks like this one, etc.), any further posts in that thread that ignore the warning will be deleted. I think that's a reasonable policy. The last thing any of us want is to stifle debate or censor opinions, but we feel that it's reasonable to ask people to participate in debates and express their opinions without upsetting others. So we're going to do our best to find a middle ground.
Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 05:51:48 UTC, Andrej Mitrovic wrote: On Monday, 4 July 2022 at 05:35:20 UTC, rikki cattermole wrote: On 04/07/2022 5:30 PM, Andrej Mitrovic wrote: Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Not necessarily. It could and should most likely mean that it won't do any heap allocations. Heap allocations are expensive after all. @safe @nogc @noheap? :) @noheap is not so necessary, at least for aggregates we have `@disable new(size_t);`
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 21:06:40 UTC, rikki cattermole wrote: We have a perfectly good Unicode handling library already. (Okay, little out of date and doesn't handle Turkic stuff, but fixable). The standard one is called ICU. Yes, that is a common one that is maintained, but maybe there are BOOST licensed implementations too? One can do an exhaustive test for say two-character normalization against ICU to see if they are compliant. Anyway, normalization should not happen behind your back in a system level language. You might want to treat different encodings of the same string differently when comparing. Anyway, we are straying from my original point, that limiting ourselves to the string alias and not supporting wstring or dstring in Phobos is going to bite us. I guess some Windows programmers want 16 bit… but I don't think the conversion matters all that much in that context? There better be a good reason for this that isn't just removing templates. The good reason would be that you can focus on fast SIMD optimized algoritms that makes sense for the byte-encoding of UTF-8, and get something competitive.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: [...] Firstly, thank you for composing this. Given the desire to reduce GC usage in PhobosV2, would it be fair to say this implies changing error handling schemes found therein, i.e. avoiding exceptions as they are implemented now, where reasonable? And if so, have core members mentioned any alternatives? With regards to moderation policy, are there specific issues; about which you feel comfortable speaking; that prompted this item's inclusion in the vision document?
Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 06:01:09 UTC, Mike Parker wrote: So what the `@nogc as much as possible` goal means is that it should be a consideration for the implementation of Phobos v2 from the beginning. The design of the API should allow `@nogc` client code to make use of it as much as possible. Fair enough.
Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 05:30:10 UTC, Andrej Mitrovic wrote: On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document Under 'Memory safety': Allow the continued use of garbage collection as the default memory management strategy without impact. The GC is one of D's strengths, and we should not "throw the baby out with the bath water". Under 'Phobos and DRuntime': @nogc as much as possible. Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. `@nogc` code in Phobos still allows the use of the GC in client code. Those are not opposite goals. Then it's not part of D's strengths. No, that's not what that means. When Phobos was originally implemented, `@nogc` wasn't a thing. There was no (or little) consideration given to whether a particular function absolutely needed to use the GC, or if there was an alternative. And later on, as demand grew for more `@nogc` support, that had to be bolted on. So what the `@nogc as much as possible` goal means is that it should be a consideration for the implementation of Phobos v2 from the beginning. The design of the API should allow `@nogc` client code to make use of it as much as possible. GC is a strength, but it's not a panacea. There are specific cases where people determine it's better to avoid it, and there are some people who don't want to use it at all. As long as we continue to support `@nogc` in the language, then we ought to provide support for it in Phobos to the extent that we can. The vision document is just making it clear that this will be a goal from the beginning with v2.
Re: The D Programming Language Vision Document
On Monday, 4 July 2022 at 05:35:20 UTC, rikki cattermole wrote: On 04/07/2022 5:30 PM, Andrej Mitrovic wrote: Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Not necessarily. It could and should most likely mean that it won't do any heap allocations. Heap allocations are expensive after all. @safe @nogc @noheap? :) Baby, you got a stew going!
Re: The D Programming Language Vision Document
On 04/07/2022 5:30 PM, Andrej Mitrovic wrote: Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Not necessarily. It could and should most likely mean that it won't do any heap allocations. Heap allocations are expensive after all.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document Under 'Memory safety': Allow the continued use of garbage collection as the default memory management strategy without impact. The GC is one of D's strengths, and we should not "throw the baby out with the bath water". Under 'Phobos and DRuntime': @nogc as much as possible. Aren't these the polar opposites of each other? The GC is one of D's strengths, yet we should avoid it as much as possible in the standard library. Then it's not part of D's strengths.
Re: The D Programming Language Vision Document
We have a perfectly good Unicode handling library already. (Okay, little out of date and doesn't handle Turkic stuff, but fixable). The standard one is called ICU. Anyway, we are straying from my original point, that limiting ourselves to the string alias and not supporting wstring or dstring in Phobos is going to bite us. Its not what people expect, its not what we have supported and code that looks like it should work won't. There better be a good reason for this that isn't just removing templates.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 20:28:18 UTC, rikki cattermole wrote: We only support UTF-16/UTF-32 for the target endian. Text input comes from many sources, stdin, files and say the windowing system are three common sources that do not make any such guarantees. Well, then the application author will use an external Unicode library anyway. If you support UTF-16 or UTF-32 there might not be a BOM mark, so you might need to use heuristics to figure out the LE/LB endian issue. For things like gzip, png, crypto and unicode there are most likely faster and better tested open source alternatives than a small community can come up with. Maybe just use out whatever Chromium or Clang uses? What I never liked about C++ is the string mess: char, signed char, unsigned char, char8_t, char16_t, char32_t, wchar_t, string, wstring, u8string, u16string, u32string, pmr::string, pmr::wstring, pmr::u8string, pmr::u16string, pmr::u32string… And this doesn't even account for endianess!! This is what happens over time as new needs pops up. One of the best things about Python3 and JavaScript is that there is one commonly used string type that is well supported. Having one common string representation is a good thing for API authors. (But make sure to have a maintained binding to a versatile C unicode library.)
Re: The D Programming Language Vision Document
On 04/07/2022 8:16 AM, Ola Fosheim Grøstad wrote: On Sunday, 3 July 2022 at 19:32:56 UTC, rikki cattermole wrote: It is required for string equivalent comparisons (which is what you should be doing in a LOT more cases! Anything user provided when compared should be normalized first. Well, I think it is reasonable for a protocol to require that the input is NFC, and just check it and reject it or call out to an external library to convert it into NFC. Anyway, UTF-8 is the only format that isn't affected by network byte order… So if you support more than UTF-8 then you have to support UTF-8, UTF16-LE, UTF16-BE, UTF-32LE, UTF-32BE… That is five formats for just a simple string… and only UTF-8 will be well tested by users. :-/ https://issues.dlang.org/show_bug.cgi?id=23186 We only support UTF-16/UTF-32 for the target endian. Text input comes from many sources, stdin, files and say the windowing system are three common sources that do not make any such guarantees.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 19:32:56 UTC, rikki cattermole wrote: It is required for string equivalent comparisons (which is what you should be doing in a LOT more cases! Anything user provided when compared should be normalized first. Well, I think it is reasonable for a protocol to require that the input is NFC, and just check it and reject it or call out to an external library to convert it into NFC. Anyway, UTF-8 is the only format that isn't affected by network byte order… So if you support more than UTF-8 then you have to support UTF-8, UTF16-LE, UTF16-BE, UTF-32LE, UTF-32BE… That is five formats for just a simple string… and only UTF-8 will be well tested by users. :-/
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 19:32:56 UTC, rikki cattermole wrote: I have just finished implementing string normalization which is based around UTF-32. There's a difference between utf-32 and unicode code points. It is required for string equivalent comparisons (which is what you should be doing in a LOT more cases! Anything user provided when compared should be normalized first. Which you can do on any translation format.
Re: The D Programming Language Vision Document
On 04/07/2022 7:18 AM, Ola Fosheim Grøstad wrote: I hardly ever use anything outside UTF-8, and if I do then I use a well tested unicode library as it has to be correct and up to date to be useful. The utility of going beyond UTF-8 seems to be limited: https://en.wikipedia.org/wiki/UTF-32#Analysis I have just finished implementing string normalization which is based around UTF-32. It is required for string equivalent comparisons (which is what you should be doing in a LOT more cases! Anything user provided when compared should be normalized first.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 18:33:29 UTC, rikki cattermole wrote: On 04/07/2022 6:10 AM, Ola Fosheim Grøstad wrote: People who are willing to use 4 bytes per code point are probably using third party C-libraries that have their own representation, so you have to convert anyway? If you use Unicode and follow their recommendations, you are going to be using dstrings at some point. I hardly ever use anything outside UTF-8, and if I do then I use a well tested unicode library as it has to be correct and up to date to be useful. The utility of going beyond UTF-8 seems to be limited: https://en.wikipedia.org/wiki/UTF-32#Analysis
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 18:33:29 UTC, rikki cattermole wrote: Its just an unnecessary goal, when most of the string algorithms we have probably don't care about the encoding and those that do probably will be using dstrings. To the contrary, I find this goal coherant with the end of autodecoding. that will probably make phobos simpler: less template overloads, less template constraints to evaluate, no more isNattowString etc.
Re: The D Programming Language Vision Document
On 04/07/2022 6:10 AM, Ola Fosheim Grøstad wrote: People who are willing to use 4 bytes per code point are probably using third party C-libraries that have their own representation, so you have to convert anyway? If you use Unicode and follow their recommendations, you are going to be using dstrings at some point. For example, string equivalence, and anything to do with case is going to use them and very likely to require multiple memory allocations to do it. Its just an unnecessary goal, when most of the string algorithms we have probably don't care about the encoding and those that do probably will be using dstrings.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 17:27:43 UTC, rikki cattermole wrote: That's going to bite us big time when it comes to Unicode handling which wants to work with dstring's. You can just use ints… It is better to do something commonly used well, than have features that not enough people use to get the quality up. People who are willing to use 4 bytes per code point are probably using third party C-libraries that have their own representation, so you have to convert anyway?
Re: The D Programming Language Vision Document
> Stronger integration with other languages One of the things I judge D's compilers by is how well they can build a shared library. This is crucial for a lot of different applications of D and can be an complete stopper in using D if it doesn't "just work". To be blunt this is embarrassing, this should have been a top priority 10+ years ago... > Phobos and DRuntime I am very worried that this is going ahead without signatures. Its a major usability issue that concepts like ranges are not written into a function signature and is a common tripping point for people new to the language. I've been meaning to talk with Walter about this, this year. Times just haven't lined up at BeerConf to sort out lining up my designs into something that could actually go in. > No wstring or dstring. Any functions in Phobos v2 that deal with strings should deal exclusively with the string type. Users can convert from and to the other string types as needed. NOPEEE That's going to bite us big time when it comes to Unicode handling which wants to work with dstring's. > Provide automatic CI for actively-maintained third-party projects. I would like a big endian system to be included if possible, if a library is actively maintained you don't want surprises to arise from that.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: Feedback is welcome. Metaprogramming This section is... light on details and does little to clear up if you share my goals. My current take on this is that I believe something with c feature ser + templates are the future, their foundation extremely shaky and awful with c++ appearing to make it powerful by accident, and that I should be wherever there is the most sugar to make it livable. This will mostly involve me wanting an endless stream of new features to play with. But this contradicts the earlier section about simplicity and avoiding complex features. Templates generally get ugly syntax, alias when you want to store first class type, static if's everywhere, ___traits; so Id suggests that's updated to be complex features that affect standard code but generally be open to experimental features in the template space; so that templates are generally going to get the short end of the stick design-wise but that but open to exploring out the foundation meta programming needs. Phobos and DRuntime You don't comment on how v2 should be organized. Currently, I believe to write functional-style code your going to import 5 different std libs and at least algorithm and range. Is that going to continue or be fixed? I find the naming conflicts of "write" to be fairly awful and makes `import std;` undoable and even if you always write out your imports, studio and file are often going to be used together. Will that be fixed? etc. etc. the goals of std v2 probably could get its own document. Stronger ecosystem Community management All D users and contributors must feel comfortable participating in the D community. There is a divide between those who believe in a "kitchen sink" standard library and those who support a minimal standard library backed up by a large ecosystem. We must find a balance that makes sense for the D programming language. Id suggest dropping std.experimental and get a std.community sort of thing going. Where given snar did sumtypes as a subtype lib, then got into std.experimental, had to follow your process and whatever; and then finally got merged into std. Instead, I'd suggest that snar makes an important lib, it gets noticed by the community, std.community.sumtype will point at snar's github either auto-downloaded or batched with compiler releases. Have whatever legal disclaimer that std.community is not your responsibility, is kinda bad style to overuse and should be verified yourself. But have a curated list of community projects that have a soft thumbs up and that are easy to use. While dropping the std.experimental take that everyone says is a bad experience.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: Feedback is welcome. Had a hope to see the "complete started but abandoned features" point.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 11:39:45 UTC, Mike Parker wrote: Language evolution doesn't really mean much until we get all of this sorted. Good point. That's right. But Walter wants to minimize its use in Phobos v2, and there's a strong desire to have a pay-as-you-go DRuntime. I'm not the person to speculate on how the GC fits into that, but I do know they don't yet want git rid of it. Yes, the Phobos issue is probably a good point, but I don't think the standard library prevents experienced developers from doing anything (People can write their own or use third party solutions). Although I have been pro GC-free in the past, I am also not so sure if GC-free is the sweet spot in 2022 (due to Rust and C++ having reduced friction significantly). To my mind the sweet spot for larger applications would be to write your own runtime/GUI-framework/libraries in @system and cover your @safe application code with a convenient «non-stop» (or at least only «local stop») GC/ARC solution. But I understand that you cannot say anything specific on this at this point in time. (On a related note, I'll soon be publishing a video of a conversation I had with Walter about origins of D, and he said something there about the GC that really surprised me.) That would be interesting to hear more about as the GC was what surprised me the most when I first tried D as a C++ «descendant».
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 11:13:42 UTC, Ola Fosheim Grøstad wrote: On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: Feedback is welcome. Thank you for putting this in clear terms. I miss an overarching «primary use scenarios» to guide further language evolution. How do you know if new language features are good or bad if you have no scenarios to measure them up against? That's not something we've discussed yet. Right now, the focus is on plugging holes in the existing language, building out the ecosystem, and overhauling Phobos. These areas are where we see some of the loudest complaints. Language evolution doesn't really mean much until we get all of this sorted. But we'll start discussing it at some point, at which time you'll see some new high-level goals appear. In the meantime, Walter and Atila will continue evaluating DIPs on a case-by-case basis. (Speaking of which, I want to look at overhauling the DIP process a bit some time next year). It is nice to see that improved move semantics is a goal, then I guess ARC could be something one could envision down the line. That said, I am a bit disappointed that there is no hint of a departure from the current STOP-the-world GC regime, but I guess that is the reflecting reality. My interpretation of the vision document is that the core team sees no need to change the current GC strategy. That's right. But Walter wants to minimize its use in Phobos v2, and there's a strong desire to have a pay-as-you-go DRuntime. I'm not the person to speculate on how the GC fits into that, but I do know they don't yet want git rid of it. (On a related note, I'll soon be publishing a video of a conversation I had with Walter about origins of D, and he said something there about the GC that really surprised me.)
Re: The D Programming Language Vision Document
On 03.07.22 10:46, Mike Parker wrote: You can find the final draft of the high-level goals for the D programming language at the following link: https://github.com/dlang/vision-document Quoting from the "Memory safety" section: The language maintainers do not see memory safety as a fad, nor is their focus on its implementation in the D programming language a form of "chasing" other languages. There's no need to be so defensive, particularly in the first sentence of the section. Also, the "chasing" part is cryptic. What other languages are you talking about? Why can their names not be uttered? I know it's Rust, but other readers might not. Just ax that sentence and start with "The language maintainers see memory safety as a critical component [...]". DIP 1000 is crucial to this because it eliminates most of the reasons why D code written as simply as possible is not @safe. Dubious claim. My experience is that people who try `scope` quickly run into its limitations. Eliminate undefined behavior in @safe code. I.e., fix bugs. That's hardly worth mentioning as a high-level goal. it's not possible to write a vector type where the following code is @safe. auto v = vector(1, 2, 3); v ~= 4; That example isn't clear at all. I suppose the point is to (1) avoid the GC and (2) still allow taking the addresses of the elements. That isn't obvious from those two lines of code, at all. As presented, `vector` can easily be implemented @safe-ly with a dynamic array. Even as we strive to increase memory safety in D, we must always ensure that programmers who need or want to eschew memory safety features can do so. And they must be able to do so with minimal friction. "Minimal friction" would mean not making @safe default, as that adds friction. Little friction is the real goal.
Re: The D Programming Language Vision Document
On Sunday, 3 July 2022 at 08:46:31 UTC, Mike Parker wrote: Feedback is welcome. Thank you for putting this in clear terms. I miss an overarching «primary use scenarios» to guide further language evolution. How do you know if new language features are good or bad if you have no scenarios to measure them up against? It is nice to see that improved move semantics is a goal, then I guess ARC could be something one could envision down the line. That said, I am a bit disappointed that there is no hint of a departure from the current STOP-the-world GC regime, but I guess that is the reflecting reality. My interpretation of the vision document is that the core team sees no need to change the current GC strategy.
