Re: D Article: Memory Safety
On Thursday, 21 January 2016 at 17:42:02 UTC, Jakob Ovrum wrote: On Thursday, 21 January 2016 at 17:39:11 UTC, Andrei Alexandrescu wrote: Good work, thanks! Has this been reddited yet? -- Andrei I don't think so. Personally I don't think I have a reddit account, but people are more than welcome to post it wherever they like :) Someone has submitted it (about a half hour ago): https://www.reddit.com/r/programming/comments/420yhi/memory_safety_in_d/
Re: D Article: Memory Safety
On Thursday, 21 January 2016 at 17:56:19 UTC, Brad Anderson wrote: On Thursday, 21 January 2016 at 17:42:02 UTC, Jakob Ovrum wrote: On Thursday, 21 January 2016 at 17:39:11 UTC, Andrei Alexandrescu wrote: Good work, thanks! Has this been reddited yet? -- Andrei I don't think so. Personally I don't think I have a reddit account, but people are more than welcome to post it wherever they like :) Someone has submitted it (about a half hour ago): https://www.reddit.com/r/programming/comments/420yhi/memory_safety_in_d/ I've submitted it to Hacker News.
Re: D Article: Memory Safety
On Thursday, 21 January 2016 at 17:39:11 UTC, Andrei Alexandrescu wrote: Good work, thanks! Has this been reddited yet? -- Andrei I don't think so. Personally I don't think I have a reddit account, but people are more than welcome to post it wherever they like :)
Re: D Article: Memory Safety
On 01/20/2016 09:04 AM, Jakob Ovrum wrote: The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates. https://jakobovrum.github.io/d/2016/01/20/memory-safety.html Any and all feedback appreciated. Good work, thanks! Has this been reddited yet? -- Andrei
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote: The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates. https://jakobovrum.github.io/d/2016/01/20/memory-safety.html Any and all feedback appreciated. Nicely written, enjoyed reading it. And the visual presentation is excellent, I guess because it's based on github's code presentation libraries. Really a nice example of how you can find blog posts online that are an order of magnitude better than any book you've ever read, particularly the visuals. Is it on reddit? The dlang.org redesign, Walter's talk, and a couple other D topics did well on there this week, I think this would also.
Re: D Article: Memory Safety
On Thursday, 21 January 2016 at 13:42:13 UTC, Jakob Ovrum wrote: On Thursday, 21 January 2016 at 13:39:48 UTC, Dicebot wrote: I'd suggest at the very least to add a comment before "p.bar();" saying "Must not escape 'p' pointer or @safe-ty will be compromised". I thought about this case, but it relies on UFCS which is controlled by the callee. The caller can't inject that call if the callee is careful with its imports. For member functions, the this reference is `ref` and its address cannot be taken in @safe code. Reasonable, but the UFCS call can result from some other function defined in same module (Phobos modules are not small at all). Even small unlikely violation can completely destroy benefits of @safe so in my opinion one can't be overly cautious when documenting stuff that requires verification.
Re: D Article: Memory Safety
On Thursday, 21 January 2016 at 13:52:57 UTC, Dicebot wrote: Reasonable, but the UFCS call can result from some other function defined in same module (Phobos modules are not small at all). Even small unlikely violation can completely destroy benefits of @safe so in my opinion one can't be overly cautious when documenting stuff that requires verification. I agree. Do you think it's worth mentioning UFCS functions in the article?
Re: D Article: Memory Safety
On Thursday, 21 January 2016 at 13:39:48 UTC, Dicebot wrote: I'd suggest at the very least to add a comment before "p.bar();" saying "Must not escape 'p' pointer or @safe-ty will be compromised". I thought about this case, but it relies on UFCS which is controlled by the callee. The caller can't inject that call if the callee is careful with its imports. For member functions, the this reference is `ref` and its address cannot be taken in @safe code.
Re: D Article: Memory Safety
On Thursday, 21 January 2016 at 04:31:25 UTC, Jakob Ovrum wrote:
That was for non-templated functions where this approach makes
no sense. Indeed it is counterproductive, because @trusted on
the whole function is a better indication of what needs to be
reviewed for memory safety (the whole function!).
Thanks! I got confused because your used example actually leaves
@safe hole with this specific usage of @trusted :
void foo(T)(T t) {
auto p = () @trusted { return &t; } ();
p.bar();
}
struct S { int x; }
S* global;
void bar (S* ptr) @safe
{
global = ptr;
}
void main () @safe
{
foo(S.init);
global.x = 42; // oops, writing to some random stack memory
}
I'd suggest at the very least to add a comment before "p.bar();"
saying "Must not escape 'p' pointer or @safe-ty will be
compromised".
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote: snip Thanks for all the feedback. I've pushed a revision with further changes, most of it based on the feedback in this thread. https://github.com/JakobOvrum/jakobovrum.github.io/commit/07c270567097f6cae5d9b95c88bd4d6c8124498c (I'll try to remember not to force push over this commit and break the link, but if it is broken in the future, sorry, I probably slipped up and forgot.)
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote: The article aims to explain how to use @safe Um, no, the article doesn't explain how to use @safe, it shows patterns that can be used to write safe code. The target audience must already understand safety.
Re: D Article: Memory Safety
On Thursday, 21 January 2016 at 06:20:01 UTC, rsw0x wrote: okay, I'll just use @safe here... and nothing else in third party libraries/half of phobos is @safe friendly so I guess I'll wrap it in @trusted oh fuck it Yeah, using @trusted like that is counterproductive. Just use @system or improve the dependencies.
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote: The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates. https://jakobovrum.github.io/d/2016/01/20/memory-safety.html Any and all feedback appreciated. my experience with @safe: okay, I'll just use @safe here... and nothing else in third party libraries/half of phobos is @safe friendly so I guess I'll wrap it in @trusted oh fuck it
Re: D Article: Memory Safety
On Thu, Jan 21, 2016 at 05:09:48AM +, Basile B. via Digitalmars-d-announce wrote: [...] > I mean '@safe' at too low level is a handicap. It's like 'const'. They > are hard to use, mostly because of transitivness. These attributes are > never a noop. Transitivity also makes const really painful to use in a widespread way. I've tried writing const-correct code before too, but gave up because it quickly became too unwieldy to work with. I started spending more time hunting down missing const attributes than actually writing useful code, so I decided it was time to give up. Generally, though, const is still useful in lower-level code (i.e., near the leaf nodes of your function call tree), to prevent silly mistakes. Knowing how to use const is also helpful in utility functions that need to accept both immutable and mutable, etc.. Just like with (the current state of) @safe, though, pervasive use of const is still too onerous currently. Transitivity really makes it painful, especially when important chunks of Phobos still isn't fully const-correct (or at least const-compatible) yet. A lot of progress has been made, but, const being transitive, all it takes is for one small Phobos function to be non-const when it should be const, and your entire call tree can no longer be const. Encounter this a handful of times, and it's hard not to just throw in the towel instead of spending all of your time working around const issues rather than writing useful code. (Recently I'm slowly moving towards writing *all* my code as template functions, and letting the compiler do the tedious work of attributing my code instead of typing them out myself. My secret wish is that one day, the compiler's attribute inference will be good enough that I could just slap one or two const's (or @safe, etc.) on top of my modules and everything will Just Work.) T -- Only boring people get bored. -- JM
Re: D Article: Memory Safety
On Thu, Jan 21, 2016 at 04:59:01AM +, Basile B. via Digitalmars-d-announce wrote: [...] > Altgough one thing, attributes are not the easy part of D. I've > recently encountered a case were in the library attributes were > allright, test OK, and then suddently when I've started to use the > library in a real life context I had to remove them from the > library...@safe was unsustainable. Phobos/druntime still has some ways to go before using it from @safe code will be painless. Some pretty fundamental functionality still isn't @safe (mainly some stuff in object.di that basically interacts with too many other things that marking one thing as @safe will percolate throughout pretty much everything, breaking a whole bunch of stuff at once). I once tried writing a @safe program, and it didn't take very long before I threw that idea out the window. Once main() is @safe, you're so straitjacketed that you basically can't write anything too much more complex than Hello World. (Well, you *could* just slap @trusted on whatever it is that's holding you back, but then that breaks the promise of @safe, which defeats the purpose of the entire exercise.) There's also still a good number of @safe-related bugs on Bugzilla, several of which involve built-in language constructs that break @safe-ty outright. Things have improved a bit since I last checked, but it seems to me that @safe is still not quite ready to live up to its promise just yet. Maybe in a few more years' time... > Dealing with attributes is the hardest part of D IMO. No one is > forced to btw, there are plenty of other cool things in D but to > follow the D safety is hard... [...] I think Walter has mentioned before that attribute inference is the way to go, and I agree. Once you start writing carefully-attributed code, you'll quickly find that your declarations become painfully verbose, which is never a good sign (it encourages people not to use attributes). However, attribute inference on templates and auto functions (proposed last year, don't know if it's implemented yet) alleviates a lot of the verbosity. Hopefully the scope of attribute inference will increase until it makes attribute use more widespread in your everyday D code. T -- MS Windows: 64-bit rehash of 32-bit extensions and a graphical shell for a 16-bit patch to an 8-bit operating system originally coded for a 4-bit microprocessor, written by a 2-bit company that can't stand 1-bit of competition.
Re: D Article: Memory Safety
On Thursday, 21 January 2016 at 04:59:01 UTC, Basile B. wrote: On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote: The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates. https://jakobovrum.github.io/d/2016/01/20/memory-safety.html Any and all feedback appreciated. Good work. Someone has to re-edit-it if not yet reeddited. Altgough one thing, attributes are not the easy part of D. I've recently encountered a case were in the library attributes were allright, test OK, and then suddently when I've started to use the library in a real life context I had to remove them from the library...@safe was unsustainable. Dealing with attributes is the hardest part of D IMO. No one is forced to btw, there are plenty of other cool things in D but to follow the D safety is hard... congrats nice article. I mean '@safe' at too low level is a handicap. It's like 'const'. They are hard to use, mostly because of transitivness. These attributes are never a noop.
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote: The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates. https://jakobovrum.github.io/d/2016/01/20/memory-safety.html Any and all feedback appreciated. Good work. Someone has to re-edit-it if not yet reeddited. Altgough one thing, attributes are not the easy part of D. I've recently encountered a case were in the library attributes were allright, test OK, and then suddently when I've started to use the library in a real life context I had to remove them from the library...@safe was unsustainable. Dealing with attributes is the hardest part of D IMO. No one is forced to btw, there are plenty of other cool things in D but to follow the D safety is hard... congrats nice article.
Re: D Article: Memory Safety
On Thu, Jan 21, 2016 at 04:38:08AM +, Jakob Ovrum via Digitalmars-d-announce wrote: > On Wednesday, 20 January 2016 at 15:28:05 UTC, jmh530 wrote: > >I like the description of @trusted and template inference. Template > >inference, in particular, was not something that was obvious to me > >when first reading about D. I'm not sure how clear you make it that > >you can still mark templates @safe and what have you (you seem to > >just say don't make templates @trusted). > > Templated functions can still be explicitly annotated with attributes, > which disables inference for those attributes. This is often a good > idea even for templated functions when template arguments do not > inject code, so that every instantiation has the same, known set of > attributes. Attribute inference can handle it, but explicit > annotations provide documentation value. I might incorporate this into > the article, but I'm wary of it losing focus. A common idiom used in Phobos related to this is to use an attributed (pure, nothrow, @nogc, etc.) unittest that instantiates the template in question, to ensure that it does not accidentally become non-pure, throwing, etc. (the unittest will stop compiling if so). This way, we ensure that the template itself doesn't contain any impure / throwing / @system code, even though you can instantiate it with template parameters that may be impure, throwing, @system, etc.. > >I wasn't aware of the point that "@trusted nested functions in > >templated functions do not have to have a memory safe interface as > >long as all calls to the function are memory safe". Interesting. > > It is a necessary evil to propagate attributes correctly. Don't use it > when you don't have to. Wouldn't it be better to refactor the code to separate out the part that needs to be @trusted into a separate place, with a safe API? Or are there cases for which this is impossible? T -- The early bird gets the worm. Moral: ewww...
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 20:28:03 UTC, Jon D wrote: This is passes the @safe constraint, but 'stdout.writeln()' and 'stderr.writeln()' do not. (My program uses stderr.) stderr/stdout/stdin are __gshared and can't be referenced by safe code. The module level version of writeln, etc., access a trusted version of stdout to avoid this. Yeah, the standard library still has a ways to go even with @safe. I always imagined that the standard pipes should use shared as opposed to __gshared. I don't think the current implementation is thread-safe, but I don't know how this affects in memory safety in this case.
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 15:28:05 UTC, jmh530 wrote: I like the description of @trusted and template inference. Template inference, in particular, was not something that was obvious to me when first reading about D. I'm not sure how clear you make it that you can still mark templates @safe and what have you (you seem to just say don't make templates @trusted). Templated functions can still be explicitly annotated with attributes, which disables inference for those attributes. This is often a good idea even for templated functions when template arguments do not inject code, so that every instantiation has the same, known set of attributes. Attribute inference can handle it, but explicit annotations provide documentation value. I might incorporate this into the article, but I'm wary of it losing focus. I wasn't aware of the point that "@trusted nested functions in templated functions do not have to have a memory safe interface as long as all calls to the function are memory safe". Interesting. It is a necessary evil to propagate attributes correctly. Don't use it when you don't have to.
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 19:55:45 UTC, H. S. Teoh wrote:
On Wed, Jan 20, 2016 at 07:25:43PM +, Dicebot via
Digitalmars-d-announce wrote:
`auto p = () @trusted { return &t; } ();`
Huh, I thought Andrei was opposed to this idiom? Is it now
considered reserved for templates or something has changed?
Yeah, I thought this was exactly the case where some of us
Phobos contributors got lambasted by Andrei and Walter for
abusing @trusted.
That was for non-templated functions where this approach makes no
sense. Indeed it is counterproductive, because @trusted on the
whole function is a better indication of what needs to be
reviewed for memory safety (the whole function!).
Any exception to the strict usage of @trusted to me smells like
a time bomb waiting to explode. It may not be today or
tomorrow, but sooner or later somebody is going to slip up and
the compiler won't help you. It's bad enough that every single
change to a @trusted function must be vetted to ensure actual
safety; now we have to also vet any modification to any
function that contains @trusted anonymous functions? In a large
template function, it's too easy to miss these @trusted
sub-functions, because if the code change is far away enough,
the @trusted annotation won't even show up in the diff. So
reviewers may not even realize it's a change that may have
broken @trusted.
It is the only way to solve this problem.
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:
The article aims to explain how to use @safe, @system and
importantly, @trusted, including all the hairy details of
templates.
https://jakobovrum.github.io/d/2016/01/20/memory-safety.html
Any and all feedback appreciated.
Nice article. I got a much better understanding reading it.
A small thing - I immediately tried adding @safe to one of my
programs and hit an issue related to the first example in the
article:
void main() @safe {
import std.stdio;
writeln("hello, world");
}
This is passes the @safe constraint, but 'stdout.writeln()' and
'stderr.writeln()' do not. (My program uses stderr.)
stderr/stdout/stdin are __gshared and can't be referenced by safe
code. The module level version of writeln, etc., access a trusted
version of stdout to avoid this.
I don't have a specific suggestion for addressing this in the
article. It's nicely written and delving into this may take away
from the key points. But, I wanted to point this out in case
others hit it.
(Note: This related to a recent thread on the learn forum:
http://forum.dlang.org/thread/vkihzrwomhiwdzqel...@forum.dlang.org)
Re: D Article: Memory Safety
On Wed, Jan 20, 2016 at 07:25:43PM +, Dicebot via Digitalmars-d-announce
wrote:
> `auto p = () @trusted { return &t; } ();`
>
> Huh, I thought Andrei was opposed to this idiom? Is it now considered
> reserved for templates or something has changed?
Yeah, I thought this was exactly the case where some of us Phobos
contributors got lambasted by Andrei and Walter for abusing @trusted.
The thing is, this is too easy to abuse, and too prone to careless
mistakes with nasty consequences. Suppose you have a template function:
auto func(T)(T t)
{
auto p = () @trusted { return &t; } ();
...
p.bar();
...
}
The problem is, the separation between p and p.bar() can be very large
in a complicated function, and during maintenance, somebody accidentally
introduces unsafe operations on p in the function body without realizing
that it came from a function marked @trusted. However, the compiler
won't catch this, because of the @trusted annotation.
Any exception to the strict usage of @trusted to me smells like a time
bomb waiting to explode. It may not be today or tomorrow, but sooner or
later somebody is going to slip up and the compiler won't help you. It's
bad enough that every single change to a @trusted function must be
vetted to ensure actual safety; now we have to also vet any modification
to any function that contains @trusted anonymous functions? In a large
template function, it's too easy to miss these @trusted sub-functions,
because if the code change is far away enough, the @trusted annotation
won't even show up in the diff. So reviewers may not even realize it's a
change that may have broken @trusted.
I'm pretty sure somebody brought up a case where such a hack was
actually necessary... but I'd still tread very, very carefully before
recommending such a thing to the general D coder.
T
--
The best compiler is between your ears. -- Michael Abrash
Re: D Article: Memory Safety
`auto p = () @trusted { return &t; } ();`
Huh, I thought Andrei was opposed to this idiom? Is it now
considered reserved for templates or something has changed?
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote: The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates. https://jakobovrum.github.io/d/2016/01/20/memory-safety.html Any and all feedback appreciated. I like the description of @trusted and template inference. Template inference, in particular, was not something that was obvious to me when first reading about D. I'm not sure how clear you make it that you can still mark templates @safe and what have you (you seem to just say don't make templates @trusted). I wasn't aware of the point that "@trusted nested functions in templated functions do not have to have a memory safe interface as long as all calls to the function are memory safe". Interesting.
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote: The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates. https://jakobovrum.github.io/d/2016/01/20/memory-safety.html Any and all feedback appreciated. Nice article! Feeling like I have a much better grasp on the whole attribute system now that I read it, thanks! :-)
D Article: Memory Safety
The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates. https://jakobovrum.github.io/d/2016/01/20/memory-safety.html Any and all feedback appreciated.
