Re: D Article: Memory Safety

2016-01-21 Thread Jakob Ovrum via Digitalmars-d-announce

On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:

snip


Thanks for all the feedback. I've pushed a revision with further 
changes, most of it based on the feedback in this thread.


https://github.com/JakobOvrum/jakobovrum.github.io/commit/07c270567097f6cae5d9b95c88bd4d6c8124498c

(I'll try to remember not to force push over this commit and 
break the link, but if it is broken in the future, sorry, I 
probably slipped up and forgot.)




Re: D Article: Memory Safety

2016-01-21 Thread Jakob Ovrum via Digitalmars-d-announce

On Thursday, 21 January 2016 at 13:39:48 UTC, Dicebot wrote:
I'd suggest at the very least to add a comment before 
"p.bar();" saying "Must not escape 'p' pointer or @safe-ty will 
be compromised".


I thought about this case, but it relies on UFCS which is 
controlled by the callee. The caller can't inject that call if 
the callee is careful with its imports.


For member functions, the this reference is `ref` and its address 
cannot be taken in @safe code.




Re: D Article: Memory Safety

2016-01-21 Thread Dicebot via Digitalmars-d-announce

On Thursday, 21 January 2016 at 13:42:13 UTC, Jakob Ovrum wrote:

On Thursday, 21 January 2016 at 13:39:48 UTC, Dicebot wrote:
I'd suggest at the very least to add a comment before 
"p.bar();" saying "Must not escape 'p' pointer or @safe-ty 
will be compromised".


I thought about this case, but it relies on UFCS which is 
controlled by the callee. The caller can't inject that call if 
the callee is careful with its imports.


For member functions, the this reference is `ref` and its 
address cannot be taken in @safe code.


Reasonable, but the UFCS call can result from some other function 
defined in same module (Phobos modules are not small at all). 
Even small unlikely violation can completely destroy benefits of 
@safe so in my opinion one can't be overly cautious when 
documenting stuff that requires verification.


Re: D Article: Memory Safety

2016-01-21 Thread Dicebot via Digitalmars-d-announce

On Thursday, 21 January 2016 at 04:31:25 UTC, Jakob Ovrum wrote:
That was for non-templated functions where this approach makes 
no sense. Indeed it is counterproductive, because @trusted on 
the whole function is a better indication of what needs to be 
reviewed for memory safety (the whole function!).


Thanks! I got confused because your used example actually leaves 
@safe hole with this specific usage of @trusted :


void foo(T)(T t) {
auto p = () @trusted { return  } ();
p.bar();
}

struct S { int x; }
S* global;

void bar (S* ptr) @safe
{
global = ptr;
}

void main () @safe
{
foo(S.init);
global.x = 42; // oops, writing to some random stack memory
}

I'd suggest at the very least to add a comment before "p.bar();" 
saying "Must not escape 'p' pointer or @safe-ty will be 
compromised".




Re: D Article: Memory Safety

2016-01-21 Thread Kagamin via Digitalmars-d-announce

On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:

The article aims to explain how to use @safe


Um, no, the article doesn't explain how to use @safe, it shows 
patterns that can be used to write safe code. The target audience 
must already understand safety.


Re: D Article: Memory Safety

2016-01-21 Thread Jakob Ovrum via Digitalmars-d-announce

On Thursday, 21 January 2016 at 13:52:57 UTC, Dicebot wrote:
Reasonable, but the UFCS call can result from some other 
function defined in same module (Phobos modules are not small 
at all). Even small unlikely violation can completely destroy 
benefits of @safe so in my opinion one can't be overly cautious 
when documenting stuff that requires verification.


I agree. Do you think it's worth mentioning UFCS functions in the 
article?


Re: D Article: Memory Safety

2016-01-21 Thread Joakim via Digitalmars-d-announce

On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:
The article aims to explain how to use @safe, @system and 
importantly, @trusted, including all the hairy details of 
templates.


https://jakobovrum.github.io/d/2016/01/20/memory-safety.html

Any and all feedback appreciated.


Nicely written, enjoyed reading it.

And the visual presentation is excellent, I guess because it's 
based on github's code presentation libraries.  Really a nice 
example of how you can find blog posts online that are an order 
of magnitude better than any book you've ever read, particularly 
the visuals.


Is it on reddit?  The dlang.org redesign, Walter's talk, and a 
couple other D topics did well on there this week, I think this 
would also.


Re: D Article: Memory Safety

2016-01-21 Thread Andrei Alexandrescu via Digitalmars-d-announce

On 01/20/2016 09:04 AM, Jakob Ovrum wrote:

The article aims to explain how to use @safe, @system and importantly,
@trusted, including all the hairy details of templates.

https://jakobovrum.github.io/d/2016/01/20/memory-safety.html

Any and all feedback appreciated.


Good work, thanks! Has this been reddited yet? -- Andrei



Re: D Article: Memory Safety

2016-01-21 Thread Jakob Ovrum via Digitalmars-d-announce
On Thursday, 21 January 2016 at 17:39:11 UTC, Andrei Alexandrescu 
wrote:

Good work, thanks! Has this been reddited yet? -- Andrei


I don't think so. Personally I don't think I have a reddit 
account, but people are more than welcome to post it wherever 
they like :)


Re: D Article: Memory Safety

2016-01-21 Thread Brad Anderson via Digitalmars-d-announce

On Thursday, 21 January 2016 at 17:42:02 UTC, Jakob Ovrum wrote:
On Thursday, 21 January 2016 at 17:39:11 UTC, Andrei 
Alexandrescu wrote:

Good work, thanks! Has this been reddited yet? -- Andrei


I don't think so. Personally I don't think I have a reddit 
account, but people are more than welcome to post it wherever 
they like :)


Someone has submitted it (about a half hour ago): 
https://www.reddit.com/r/programming/comments/420yhi/memory_safety_in_d/


Re: D Article: Memory Safety

2016-01-21 Thread Brad Anderson via Digitalmars-d-announce

On Thursday, 21 January 2016 at 17:56:19 UTC, Brad Anderson wrote:

On Thursday, 21 January 2016 at 17:42:02 UTC, Jakob Ovrum wrote:
On Thursday, 21 January 2016 at 17:39:11 UTC, Andrei 
Alexandrescu wrote:

Good work, thanks! Has this been reddited yet? -- Andrei


I don't think so. Personally I don't think I have a reddit 
account, but people are more than welcome to post it wherever 
they like :)


Someone has submitted it (about a half hour ago): 
https://www.reddit.com/r/programming/comments/420yhi/memory_safety_in_d/


I've submitted it to Hacker News.


Re: D Article: Memory Safety

2016-01-20 Thread H. S. Teoh via Digitalmars-d-announce
On Thu, Jan 21, 2016 at 04:59:01AM +, Basile B. via Digitalmars-d-announce 
wrote:
[...]
> Altgough one thing, attributes are not the easy part of D. I've
> recently encountered a case were in the library attributes were
> allright, test OK, and then suddently when I've started to use the
> library in a real life context I had to remove them from the
> library...@safe was unsustainable.

Phobos/druntime still has some ways to go before using it from @safe
code will be painless.  Some pretty fundamental functionality still
isn't @safe (mainly some stuff in object.di that basically interacts
with too many other things that marking one thing as @safe will
percolate throughout pretty much everything, breaking a whole bunch of
stuff at once).

I once tried writing a @safe program, and it didn't take very long
before I threw that idea out the window.  Once main() is @safe, you're
so straitjacketed that you basically can't write anything too much more
complex than Hello World.  (Well, you *could* just slap @trusted on
whatever it is that's holding you back, but then that breaks the promise
of @safe, which defeats the purpose of the entire exercise.)

There's also still a good number of @safe-related bugs on Bugzilla,
several of which involve built-in language constructs that break
@safe-ty outright. Things have improved a bit since I last checked, but
it seems to me that @safe is still not quite ready to live up to its
promise just yet. Maybe in a few more years' time...


> Dealing with attributes is the hardest part of D IMO.  No one is
> forced to btw, there are plenty of other cool things in D but to
> follow the D safety is hard...
[...]

I think Walter has mentioned before that attribute inference is the way
to go, and I agree. Once you start writing carefully-attributed code,
you'll quickly find that your declarations become painfully verbose,
which is never a good sign (it encourages people not to use attributes).
However, attribute inference on templates and auto functions (proposed
last year, don't know if it's implemented yet) alleviates a lot of the
verbosity. Hopefully the scope of attribute inference will increase
until it makes attribute use more widespread in your everyday D code.


T

-- 
MS Windows: 64-bit rehash of 32-bit extensions and a graphical shell for a 
16-bit patch to an 8-bit operating system originally coded for a 4-bit 
microprocessor, written by a 2-bit company that can't stand 1-bit of 
competition.


Re: D Article: Memory Safety

2016-01-20 Thread Jakob Ovrum via Digitalmars-d-announce

On Wednesday, 20 January 2016 at 20:28:03 UTC, Jon D wrote:
This is passes the @safe constraint, but 'stdout.writeln()' and 
'stderr.writeln()' do not. (My program uses stderr.) 
stderr/stdout/stdin are __gshared and can't be referenced by 
safe code. The module level version of writeln, etc., access a 
trusted version of stdout to avoid this.


Yeah, the standard library still has a ways to go even with @safe.

I always imagined that the standard pipes should use shared as 
opposed to __gshared. I don't think the current implementation is 
thread-safe, but I don't know how this affects in memory safety 
in this case.


Re: D Article: Memory Safety

2016-01-20 Thread Jakob Ovrum via Digitalmars-d-announce

On Wednesday, 20 January 2016 at 19:55:45 UTC, H. S. Teoh wrote:
On Wed, Jan 20, 2016 at 07:25:43PM +, Dicebot via 
Digitalmars-d-announce wrote:

`auto p = () @trusted { return  } ();`

Huh, I thought Andrei was opposed to this idiom? Is it now 
considered reserved for templates or something has changed?


Yeah, I thought this was exactly the case where some of us 
Phobos contributors got lambasted by Andrei and Walter for 
abusing @trusted.


That was for non-templated functions where this approach makes no 
sense. Indeed it is counterproductive, because @trusted on the 
whole function is a better indication of what needs to be 
reviewed for memory safety (the whole function!).


Any exception to the strict usage of @trusted to me smells like 
a time bomb waiting to explode. It may not be today or 
tomorrow, but sooner or later somebody is going to slip up and 
the compiler won't help you. It's bad enough that every single 
change to a @trusted function must be vetted to ensure actual 
safety; now we have to also vet any modification to any 
function that contains @trusted anonymous functions? In a large 
template function, it's too easy to miss these @trusted 
sub-functions, because if the code change is far away enough, 
the @trusted annotation won't even show up in the diff. So 
reviewers may not even realize it's a change that may have 
broken @trusted.


It is the only way to solve this problem.


Re: D Article: Memory Safety

2016-01-20 Thread Basile B. via Digitalmars-d-announce

On Thursday, 21 January 2016 at 04:59:01 UTC, Basile B. wrote:
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum 
wrote:
The article aims to explain how to use @safe, @system and 
importantly, @trusted, including all the hairy details of 
templates.


https://jakobovrum.github.io/d/2016/01/20/memory-safety.html

Any and all feedback appreciated.


Good work. Someone has to re-edit-it if not yet reeddited.

Altgough one thing, attributes are not the easy part of D. I've 
recently encountered a case were in the library attributes were 
allright, test OK, and then suddently when I've started to use 
the library in a real life context I had to remove them from 
the library...@safe was unsustainable.


Dealing with attributes is the hardest part of D IMO.
No one is forced to btw, there are plenty of other cool things 
in D but to follow the D safety is hard...


congrats nice article.


I mean '@safe' at too low level is a handicap. It's like 'const'. 
They are hard to use, mostly because of transitivness. These 
attributes are never a noop.


Re: D Article: Memory Safety

2016-01-20 Thread Jakob Ovrum via Digitalmars-d-announce

On Wednesday, 20 January 2016 at 15:28:05 UTC, jmh530 wrote:
I like the description of @trusted and template inference. 
Template inference, in particular, was not something that was 
obvious to me when first reading about D. I'm not sure how 
clear you make it that you can still mark templates @safe and 
what have you (you seem to just say don't make templates 
@trusted).


Templated functions can still be explicitly annotated with 
attributes, which disables inference for those attributes. This 
is often a good idea even for templated functions when template 
arguments do not inject code, so that every instantiation has the 
same, known set of attributes. Attribute inference can handle it, 
but explicit annotations provide documentation value. I might 
incorporate this into the article, but I'm wary of it losing 
focus.


I wasn't aware of the point that "@trusted nested functions in 
templated functions do not have to have a memory safe interface 
as long as all calls to the function are memory safe". 
Interesting.


It is a necessary evil to propagate attributes correctly. Don't 
use it when you don't have to.




Re: D Article: Memory Safety

2016-01-20 Thread Basile B. via Digitalmars-d-announce

On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:
The article aims to explain how to use @safe, @system and 
importantly, @trusted, including all the hairy details of 
templates.


https://jakobovrum.github.io/d/2016/01/20/memory-safety.html

Any and all feedback appreciated.


Good work. Someone has to re-edit-it if not yet reeddited.

Altgough one thing, attributes are not the easy part of D. I've 
recently encountered a case were in the library attributes were 
allright, test OK, and then suddently when I've started to use 
the library in a real life context I had to remove them from the 
library...@safe was unsustainable.


Dealing with attributes is the hardest part of D IMO.
No one is forced to btw, there are plenty of other cool things in 
D but to follow the D safety is hard...


congrats nice article.



Re: D Article: Memory Safety

2016-01-20 Thread H. S. Teoh via Digitalmars-d-announce
On Thu, Jan 21, 2016 at 05:09:48AM +, Basile B. via Digitalmars-d-announce 
wrote:
[...]
> I mean '@safe' at too low level is a handicap. It's like 'const'. They
> are hard to use, mostly because of transitivness. These attributes are
> never a noop.

Transitivity also makes const really painful to use in a widespread way.
I've tried writing const-correct code before too, but gave up because it
quickly became too unwieldy to work with. I started spending more time
hunting down missing const attributes than actually writing useful code,
so I decided it was time to give up.

Generally, though, const is still useful in lower-level code (i.e., near
the leaf nodes of your function call tree), to prevent silly mistakes.
Knowing how to use const is also helpful in utility functions that need
to accept both immutable and mutable, etc.. Just like with (the current
state of) @safe, though, pervasive use of const is still too onerous
currently. Transitivity really makes it painful, especially when
important chunks of Phobos still isn't fully const-correct (or at least
const-compatible) yet. A lot of progress has been made, but, const being
transitive, all it takes is for one small Phobos function to be
non-const when it should be const, and your entire call tree can no
longer be const. Encounter this a handful of times, and it's hard not to
just throw in the towel instead of spending all of your time working
around const issues rather than writing useful code.

(Recently I'm slowly moving towards writing *all* my code as template
functions, and letting the compiler do the tedious work of attributing
my code instead of typing them out myself. My secret wish is that one
day, the compiler's attribute inference will be good enough that I could
just slap one or two const's (or @safe, etc.) on top of my modules and
everything will Just Work.)


T

-- 
Only boring people get bored. -- JM


Re: D Article: Memory Safety

2016-01-20 Thread rsw0x via Digitalmars-d-announce

On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:
The article aims to explain how to use @safe, @system and 
importantly, @trusted, including all the hairy details of 
templates.


https://jakobovrum.github.io/d/2016/01/20/memory-safety.html

Any and all feedback appreciated.


my experience with @safe:

okay, I'll just use @safe here... and nothing else in third party 
libraries/half of phobos is @safe friendly so I guess I'll wrap 
it in @trusted oh fuck it


Re: D Article: Memory Safety

2016-01-20 Thread H. S. Teoh via Digitalmars-d-announce
On Thu, Jan 21, 2016 at 04:38:08AM +, Jakob Ovrum via 
Digitalmars-d-announce wrote:
> On Wednesday, 20 January 2016 at 15:28:05 UTC, jmh530 wrote:
> >I like the description of @trusted and template inference. Template
> >inference, in particular, was not something that was obvious to me
> >when first reading about D. I'm not sure how clear you make it that
> >you can still mark templates @safe and what have you (you seem to
> >just say don't make templates @trusted).
> 
> Templated functions can still be explicitly annotated with attributes,
> which disables inference for those attributes. This is often a good
> idea even for templated functions when template arguments do not
> inject code, so that every instantiation has the same, known set of
> attributes. Attribute inference can handle it, but explicit
> annotations provide documentation value. I might incorporate this into
> the article, but I'm wary of it losing focus.

A common idiom used in Phobos related to this is to use an attributed
(pure, nothrow, @nogc, etc.) unittest that instantiates the template in
question, to ensure that it does not accidentally become non-pure,
throwing, etc. (the unittest will stop compiling if so). This way, we
ensure that the template itself doesn't contain any impure / throwing /
@system code, even though you can instantiate it with template
parameters that may be impure, throwing, @system, etc..


> >I wasn't aware of the point that "@trusted nested functions in
> >templated functions do not have to have a memory safe interface as
> >long as all calls to the function are memory safe". Interesting.
> 
> It is a necessary evil to propagate attributes correctly. Don't use it
> when you don't have to.

Wouldn't it be better to refactor the code to separate out the part that
needs to be @trusted into a separate place, with a safe API? Or are
there cases for which this is impossible?


T

-- 
The early bird gets the worm. Moral: ewww...


Re: D Article: Memory Safety

2016-01-20 Thread Jakob Ovrum via Digitalmars-d-announce

On Thursday, 21 January 2016 at 06:20:01 UTC, rsw0x wrote:
okay, I'll just use @safe here... and nothing else in third 
party libraries/half of phobos is @safe friendly so I guess 
I'll wrap it in @trusted oh fuck it


Yeah, using @trusted like that is counterproductive. Just use 
@system or improve the dependencies.




Re: D Article: Memory Safety

2016-01-20 Thread default0 via Digitalmars-d-announce

On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:
The article aims to explain how to use @safe, @system and 
importantly, @trusted, including all the hairy details of 
templates.


https://jakobovrum.github.io/d/2016/01/20/memory-safety.html

Any and all feedback appreciated.


Nice article! Feeling like I have a much better grasp on the 
whole attribute system now that I read it, thanks! :-)