Re: Rationale for accepting DIP 1028 as is
On Friday, 29 May 2020 at 11:33:01 UTC, Sebastiaan Koppe wrote: On Thursday, 28 May 2020 at 16:01:35 UTC, Johannes Pfau wrote: [snip] This would be another round of massively breaking user code. The breakage will be split in two rounds, but the amount of code needed to be modified would be the same as with the safe-by-default-except-for-extern-non-c. With the possibility of not needing the second round. I know, arguing for a lost cause... At the end of the day, I think people would accept "massively breaking user code" if there is a good justification and doesn't drive a hole through @safe.
Re: Rationale for accepting DIP 1028 as is
On Thursday, 28 May 2020 at 16:01:35 UTC, Johannes Pfau wrote: Am Thu, 28 May 2020 12:28:16 + schrieb Sebastiaan Koppe: If it does come back to haunt him, he can always add a DIP to make extern(!D) @system by default. It won't invalidate any work. This would be another round of massively breaking user code. The breakage will be split in two rounds, but the amount of code needed to be modified would be the same as with the safe-by-default-except-for-extern-non-c. With the possibility of not needing the second round. I know, arguing for a lost cause...
Re: Rationale for accepting DIP 1028 as is
On Friday, 29 May 2020 at 06:55:07 UTC, Robert M. Münch wrote: On 2020-05-27 06:59:28 +, Bruce Carneal said: Walter has confirmed that this is indeed the case. As you can read a few posts up his response to my "What am I missing?" query was "Nothing at all." Yes, it's really that bad. Will it be possible to see a report of these "greenwashed" call-chains, so that at least there is a chance to detect them? There was indeed activity on the "lets at least make this visible" front until Walter withdrew 1028 about 2 hours ago. The thread title in the Announce forum is: DIP 1028 "Make @safe the default" is dead
Re: Rationale for accepting DIP 1028 as is
On 2020-05-27 06:59:28 +, Bruce Carneal said: Walter has confirmed that this is indeed the case. As you can read a few posts up his response to my "What am I missing?" query was "Nothing at all." Yes, it's really that bad. Will it be possible to see a report of these "greenwashed" call-chains, so that at least there is a chance to detect them? -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Re: Rationale for accepting DIP 1028 as is
On Thursday, 28 May 2020 at 12:28:16 UTC, Sebastiaan Koppe wrote: On Thursday, 28 May 2020 at 09:21:09 UTC, Jonathan M Davis wrote: He did unfortunately manage to convince Atila, so the DIP has been accepted, but based on the discussions, I think that you may be the only person I've seen say anything positive about the DIP treating extern(C) functions as @safe. - Jonathan M Davis I think Walter had to make a tough call with many tradeoffs. The defining feature of engineering I would say. Is he wrong? Maybe, I don't know. The obvious path is far from always being a winner. A winner for what. The reasoning given doesn't make sense. Lots of people have pointed that out, and Walter has yet to expand on that. If it does come back to haunt him, he can always add a DIP to make extern(!D) @system by default. It won't invalidate any work. That's worse. It'll cause huge breakage again if you need to convert extern(C) back to @system after it was converted to @safe. If @system by default proves to be a mistake, you can just flip it to @safe and it won't break any code. Why take a gamble on something that can't be rationalized?
Re: Rationale for accepting DIP 1028 as is
Am Thu, 28 May 2020 12:28:16 + schrieb Sebastiaan Koppe: > On Thursday, 28 May 2020 at 09:21:09 UTC, Jonathan M Davis wrote: >> He did unfortunately manage to convince Atila, so the DIP has been >> accepted, but based on the discussions, I think that you may be the >> only person I've seen say anything positive about the DIP treating >> extern(C) functions as @safe. >> >> - Jonathan M Davis > > I think Walter had to make a tough call with many tradeoffs. The > defining feature of engineering I would say. > > Is he wrong? Maybe, I don't know. The obvious path is far from always > being a winner. > > If it does come back to haunt him, he can always add a DIP to make > extern(!D) @system by default. It won't invalidate any work. This would be another round of massively breaking user code. And this is going to be exactly the argument that will be used to dismiss any DIP trying to change the defaults later on. -- Johannes
Re: Rationale for accepting DIP 1028 as is
Am Thu, 28 May 2020 10:50:44 +0200 schrieb Daniel Kozak: > On Thu, May 28, 2020 at 4:56 AM Jonathan M Davis via > Digitalmars-d-announce wrote: >> >> As far as I can tell, Walter understands the issues but fundamentally >> disagrees with pretty much everyone else on the issue. > > I do not think so, the issue is, that there could be more people who > agree with Walter (like me), > but because we agree we do not participate. You can not really assume any opinion for people who did not participate, unless you can really prove why there's a bias. I did not participate either and I do not agree with Walter. So now we can say the opinions of those who did not participate in the discussion are split 50:50 ;-) We could assume there's a slight bias of those agreeing with Walter not responding because they don't have to actively convince anyone, as the DIP has been accepted. But given how much negative feedbcak there is, it's also likely people would voice their opinion to support the decision. Really the best thing we can assume is that the opionions of those not participating are split in the same way as the ones of those who are participating. The strawpoll posted recently suggests that as well: https://www.strawpoll.me/20184671/r -- Johannes
Re: Rationale for accepting DIP 1028 as is
On Thursday, 28 May 2020 at 02:47:01 UTC, Jonathan M Davis wrote: Walter has acknowledged the problem and seems to think that because it's the programmer's responsibility to deal with extern(C) functions correctly (since it's not possible for the compiler to do it), it's up to the programmer to go and fix any existing code that should be marked @system and isn't and that having the compiler incorrectly mark extern(C) declarations as @safe isn't a big problem, because programmers need to be spending the time to check them anyway. He's already created some PRs to try to fix some issues with extern(C) declarations in druntime and explicitly markingthem as @system but doesn't seem to think that it's ultimately a big deal. - Jonathan M Davis I've submitted a PR [1] to fix a whole bunch of these. It's currently blocked on what appear to be irrelevant CI failures (one of which is actually acknowledged as such in the discussion on Walter's PR). No one has given it any attention. Is there something I'm doing wrong? Should I be splitting my PR by file, to make it easier to review? I have a similar PR for Phobos in the works, and it would be nice to have some feedback on this so that one doesn't get ignored too. [1] https://github.com/dlang/druntime/pull/3117
Re: Rationale for accepting DIP 1028 as is
On Thursday, 28 May 2020 at 09:21:09 UTC, Jonathan M Davis wrote: He did unfortunately manage to convince Atila, so the DIP has been accepted, but based on the discussions, I think that you may be the only person I've seen say anything positive about the DIP treating extern(C) functions as @safe. - Jonathan M Davis I think Walter had to make a tough call with many tradeoffs. The defining feature of engineering I would say. Is he wrong? Maybe, I don't know. The obvious path is far from always being a winner. If it does come back to haunt him, he can always add a DIP to make extern(!D) @system by default. It won't invalidate any work.
Re: Rationale for accepting DIP 1028 as is
On Thu, May 28, 2020 at 03:21:09AM -0600, Jonathan M Davis via Digitalmars-d-announce wrote: [...] > With the DIP in its current state, @safe becomes a lie. The compiler > no longer guarantees that @safe code is memory safe so long as it > doesn't call any @trusted code where the programmer incorrectly marked > it as @trusted. Instead, the compiler blindly treats non-extern(D) > declarations as @safe and invisibly introduces memory safety bugs into > @safe code. Nothing about that is "OK." [...] I see it already. The next time someone wants to make a codebase @safe but the compiler complains about some violation, just add `extern(C)` to the function and move on. T -- Claiming that your operating system is the best in the world because more people use it is like saying McDonalds makes the best food in the world. -- Carl B. Constantine
Re: Rationale for accepting DIP 1028 as is
On 28.05.20 10:50, Daniel Kozak wrote: He seems to think that weakening @safe is worth doing, because it will ultimately mean that more code will be treated as @safe and mechnically checked by the compiler, And I believe he is right. No, it's a false dichotomy. Weakening @safe to allow more code to be @safe might have been sensible if there was no @trusted annotation. However, as things stand, @trusted is sufficient as a tool to introduce potentially wrong assumptions about memory safety, we don't need more, especially not implicit ones. The reason why people are not using @safe is partly that it is not the default, but it is mostly that their library dependencies _including Phobos_ are not properly annotated. This needs actual work to fix. If there is significant perceived value in performing @safety checks in @system code, we can add a new function attribute that causes non-transitive @safe checks but otherwise gets treated as @system. @safe does not have to take this role.
Re: Rationale for accepting DIP 1028 as is
On Thursday, May 28, 2020 2:50:44 AM MDT Daniel Kozak via Digitalmars-d- announce wrote: > On Thu, May 28, 2020 at 4:56 AM Jonathan M Davis via > > Digitalmars-d-announce wrote: > > As far as I can tell, Walter understands the issues but fundamentally > > disagrees with pretty much everyone else on the issue. > > I do not think so, the issue is, that there could be more people who > agree with Walter (like me), > but because we agree we do not participate. There may be some silent people who agree, but in all the discussions on this DIP, almost no one has agreed with Walter on this. It has not been a back and forth discussion with some on Walter's side and some against. It's been pretty much everyone against Walter. He did unfortunately manage to convince Atila, so the DIP has been accepted, but based on the discussions, I think that you may be the only person I've seen say anything positive about the DIP treating extern(C) functions as @safe. The fact that @safe becomes the default has garned some debate with some in favor and some against (with most seeming to be in favor), but the idea of making extern(C) declarations @safe by default has been almost universally considered a bad idea by anyone who has responded on the topic. Most DIPs do not get anywhere close to this level of negative feedback. > > But since Walter managed to convince Atila, the DIP has been accepted. > > So everything is OK right? Everything is okay? Because a bad DIP got accepted? No, most definitely not. Quite the opposite. With the DIP in its current state, @safe becomes a lie. The compiler no longer guarantees that @safe code is memory safe so long as it doesn't call any @trusted code where the programmer incorrectly marked it as @trusted. Instead, the compiler blindly treats non-extern(D) declarations as @safe and invisibly introduces memory safety bugs into @safe code. Nothing about that is "OK." From the way things currently look, we're going to have to deal with that hole in @safe in D code in the future, because the DIP has been accepted, but it adds yet another dark corner to the language of the sort that folks here tend to make fun of C++ for. Going forward, we're going to have to be explaining to people why @safe code doesn't actually guarantee that code is memory safe (in spite of the fact that that's what it claims to do) and why any and all non-extern(D) declarations have to be treated with extreme caution to avoid invisibly introducing memory safety bugs into your code. Walter is very intelligent and has made many great decisions with D, but IMHO, this is definitely not one of them. - Jonathan M Davis
Re: Rationale for accepting DIP 1028 as is
On Thu, May 28, 2020 at 4:56 AM Jonathan M Davis via Digitalmars-d-announce wrote: > > As far as I can tell, Walter understands the issues but fundamentally > disagrees with pretty much everyone else on the issue. I do not think so, the issue is, that there could be more people who agree with Walter (like me), but because we agree we do not participate. > He seems to think > that weakening @safe is worth doing, because it will ultimately mean that > more code will be treated as @safe and mechnically checked by the compiler, And I believe he is right. > whereas most everyone else thinks that weakening @safe is unacceptable. You make bad assumptions, who is "most everyone else"? I have only see few members of dlang arguing about that. But there is much bigger community around there. > But since Walter managed to convince Atila, the DIP has been accepted. > > - Jonathan M Davis > So everything is OK right?
Re: Rationale for accepting DIP 1028 as is
On Wednesday, May 27, 2020 3:30:32 AM MDT Andrei Alexandrescu via Digitalmars- d-announce wrote: > On 5/27/20 1:49 AM, Walter Bright wrote: > > On 5/26/2020 9:31 AM, Bruce Carneal wrote: > >> Currently a machine checked @safe function calling an unannotated > >> extern C routine will error out during compilation. This is great as > >> the C routine was not machine checked, and generally can not be > >> checked. Post 1028, IIUC, the compilation will go through without > >> complaint. This seems quite clear. What am I missing? > > > > Nothing at all. > > That means safe by default is effectively loosening D's notion of safety. > > This DIP must go. Which is exactly what most of us have been arguing for weeks (months?). It either needs to go or be ammended so that non-extern(D) declarations continue to be treated as @system instead of automatically becoming @safe with the DIP. The result of all of that arguing is that Walter accepted the DIP and then started this thread as his more detailed reply when there were a ton of complaints about the DIP's acceptance - and of course, you've already read and replied to his reasoning. As far as I can tell, Walter understands the issues but fundamentally disagrees with pretty much everyone else on the issue. He seems to think that weakening @safe is worth doing, because it will ultimately mean that more code will be treated as @safe and mechnically checked by the compiler, whereas most everyone else thinks that weakening @safe is unacceptable. But since Walter managed to convince Atila, the DIP has been accepted. - Jonathan M Davis
Re: Rationale for accepting DIP 1028 as is
On Tuesday, May 26, 2020 8:58:16 PM MDT Andrei Alexandrescu via Digitalmars-d- announce wrote: > On 5/26/20 12:31 PM, Bruce Carneal wrote: > > Currently a machine checked @safe function calling an unannotated extern > > C routine will error out during compilation. This is great as the C > > routine was not machine checked, and generally can not be checked. Post > > 1028, IIUC, the compilation will go through without complaint. This > > seems quite clear. What am I missing? > > If that's the case, it's the death of DIP 1028. Walter has acknowledged the problem and seems to think that because it's the programmer's responsibility to deal with extern(C) functions correctly (since it's not possible for the compiler to do it), it's up to the programmer to go and fix any existing code that should be marked @system and isn't and that having the compiler incorrectly mark extern(C) declarations as @safe isn't a big problem, because programmers need to be spending the time to check them anyway. He's already created some PRs to try to fix some issues with extern(C) declarations in druntime and explicitly markingthem as @system but doesn't seem to think that it's ultimately a big deal. - Jonathan M Davis
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 10:51:54 UTC, Walter Bright wrote: [..] I was reading Bastiaan's inspiring ideas about provable verification of @trusted and something occurred to me. If we are serious about safety, we should consider @trusted annotation on extern C harmful. It provides little value and might be misleading like a bad comment: // only call once, second call leaks void waitForClose(size_t msTimeout); The implementation was rewritten long ago, no one bothered to remove the remark. The trustworthiness of extern code can be assessed by audit. It can only be set in stone by additional tools. git can nail down the version of a verified function in the CMake list. The compiler can't know what we are linking against or what's loaded at runtime. @trusted extern C is meaningless. Pre DIP1028 we had to put it in for @safe code to compile. I'm not sure how @system fits into this. It gets outdated the same way @trusted does.
Re: Rationale for accepting DIP 1028 as is
I'm curious what the distribution looks like between whether people agree that extern(C) should be @safe, or @system. People that think it should be @trusted, vote @safe, it's pretty much the same thing. https://www.strawpoll.me/20184671
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 05:49:49 UTC, Walter Bright wrote: [snip] Nothing at all. But I doubt there is much legacy non-compiling code around. I cannot speak for all the code out there, but I know of at least one binding out there [1] that will need to get updated in light of this DIP. I also put together a wrapper [2] that sits on top of that binding (many years ago and no idea if the wrapper still works (I see no reason why the binding wouldn't), so this probably does count as legacy code) and calls the binding functions with templates and no safety annotations. So theoretically, after the DIP is accepted and without any other changes, nlopt.d becomes @safe and the binding that calls it also becomes @safe and any user code that calls it becomes @safe. *** On a separate note, the current definition of @safe is that you cannot call @system functions. This prevents calling unannotated extern(C) prototypes from @safe code. Ignoring DIP 1028 for a second, would you consider changing the rules so unannotated extern(C) prototypes are @safe instead of @system without any other change? Just this change by itself would mean that @safe functions can call unannotated extern(C) prototypes. Obviously this by itself could be rejected for special casing. However, I think the underlying question is should they be safe (ignoring @safe) or not. An argument made previously is that they should be safe and thus @safe. However, I think the overwhelming belief is that they are not safe and thus should not be @safe by default. From this perspective, you could think of the DIP as doing two things. 1) it makes unannotated extern(C) prototypes @safe instead of @system and 2) it makes all other unannotated functions @safe instead of @system. The big focus is clearly on #1. You can assume that most people agree with you on #2 and thus focus the debate on #1 So, do you think unannotated exten(C) prototypes safe? If you think they are safe, then there is no issue with making them @safe by default. However, if you do not think they are safe, then it raises serious concerns to make them @safe by default. [1] https://github.com/DlangScience/nlopt/blob/master/source/nlopt.d [2] https://github.com/jmh530/nloptd/blob/master/source/nloptd.d
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 13:50:25 UTC, Bruce Carneal wrote: On Wednesday, 27 May 2020 at 10:46:11 UTC, Walter Bright wrote: [...] You continue to miss the point. Additionally, there never was any "working legacy code". As established, the pre 1080 compiler would have rejected the code. Does-not-compile != working-code. pre 1028 compiler
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 10:46:11 UTC, Walter Bright wrote: On 5/27/2020 2:34 AM, Bastiaan Veelo wrote: On Wednesday, 27 May 2020 at 09:09:58 UTC, Walter Bright wrote: On 5/26/2020 11:20 PM, Bruce Carneal wrote: I'm not at all concerned with legacy non-compiling code of this nature. Apparently you agree it is not an actual problem. Really? I don't know if you really missed the point being made, or you're being provocative. Both seem unlikely to me. His argument was: "Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing?" I replied that it was unlikely that such legacy code existed. He replied that he was not concerned about it. I.e. working legacy code is not going break. You continue to miss the point. Additionally, there never was any "working legacy code". As established, the pre 1080 compiler would have rejected the code. Does-not-compile != working-code.
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 10:51:54 UTC, Walter Bright wrote: On 5/27/2020 3:01 AM, Timon Gehr wrote: I've addressed exactly this a dozen times or more, to you and others. Repeating myself has become pointless. It's fine to disagree with me. Argue that point. But don't say I didn't address it. I'm trying to understand the logic of "@safe by default for extern declaration". So I setup a simplified real-life example that I had in my mind... Let's say we have a project consisting of: - an ASM file containing several function definitions (assembly language scares people away, so it's a good example). - a .di file containing the extern(C) declarations of the former ASM functions. - and a .d file using those functions with @safe code. No function is annotated. Before DIP-1028, it compiles and link without annotations. If extern(C) decl are @system by default, it will not compile anymore, and a careless programmer will slap a "@trusted:" at the top of the ".di" file (greenwashing), and it'll stay there. On the other hand, a careful programmer will need to annotate @trusted ONLY on functions that he actually trusts AND make the project compile anyway (because he cannot afford to leave his project broken). So what does he do to make it compile ? Should he slap a "@trusted:" at the beginning of the file just like the careless programmer and maybe explicitly annotate individually declarations that are actually trusted ? It's weird... However, if extern(C) decl are @safe by default, he does not need to slap a "@trusted" at the top of the ".di" file, he just needs to annotate individual declarations with @trusted to advertise (to the QA, or his colleague or future self) that he reviewed successfully the function implementation, AND it will still compile. So that example would go in favor of Walter's point. So is it a good example? Does pinpoint the point? Because again, as the others said, it's still controversial, it's hard to sell, it's convoluted... I dunno...
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 05:54:32 UTC, Walter Bright wrote: On 5/26/2020 1:32 PM, Paul Backus wrote: The reason extern function declarations are particularly problematic is that changing them from @system-by-default to @safe-by-default can cause *silent* breakage in existing, correct code. Can you post an example of currently compiling and correctly working code that will break? Setting aside use of __traits(compiles, ...). I am paraphrasing an earlier post of mine in this thread: https://forum.dlang.org/post/yvcjpcxfyyobqubjr...@forum.dlang.org The breakage is that functions which are currently correctly considered @system will instead be considered @safe. As a result, future changes to @safe code are at risk of introducing memory corruption. Strictly speaking, the breakage does not occur until those future changes are made, so it is more accurate to say that changing un-annotated extern declarations to @safe-by-default will put existing code *at risk* of silent breakage. Given the prevalence of such declarations in existing code (e.g. [1]), I think it is inevitable that this risk will in fact lead to breakage in practice, so I do not consider the distinction particularly important to draw in this context. [1] https://github.com/dlang/druntime/pull/3117
Re: Rationale for accepting DIP 1028 as is
On 27.05.20 12:51, Walter Bright wrote: On 5/27/2020 3:01 AM, Timon Gehr wrote: This is clearly not possible, exactly because the old @safe rules are stronger. Thank you. We can agree on something. ... I am not sure if you noticed that I agree with most of your points, just not about their relevance to the topic at hand. But why exactly should API breakage not count? I've addressed exactly this a dozen times or more, to you No. You did not. I went over all of your responses to my posts again to make sure. Why are you making this claim? I haven't made API breakage a central point to any of my previous posts and you did not address any of my criticism in any depth. As far as I can tell, the only point you engaged with was that @trusted is not greenwashing. and others. I don't think you did, but I am not going to check. Repeating myself has become pointless. It's fine to disagree with me. Argue that point. But don't say I didn't address it. As far as I remember, you did not address this specific point, but if I had to extrapolate your response from previous points you made I would expect your opinion to be that implicitly broken APIs should be fixed by universal manual review of not explicitly annotated functions and that this is better than the compiler catching it for you because this way only people who are competent to judge which annotation should be there will notice that it is missing.
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 10:46:11 UTC, Walter Bright wrote: On 5/27/2020 2:34 AM, Bastiaan Veelo wrote: On Wednesday, 27 May 2020 at 09:09:58 UTC, Walter Bright wrote: On 5/26/2020 11:20 PM, Bruce Carneal wrote: I'm not at all concerned with legacy non-compiling code of this nature. Apparently you agree it is not an actual problem. Really? I don't know if you really missed the point being made, or you're being provocative. Both seem unlikely to me. His argument was: "Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing?" I replied that it was unlikely that such legacy code existed. He replied that he was not concerned about it. I.e. working legacy code is not going break. The legacy code is not the issue, never was. It always was about unsafe code that will become @safe with that DIP. Safe code is safe and DIP doesn't change that. It's all about UNSAFE code becoming magically labelled SAFE by the compiler but that is still UNSAFE in reality.
Re: Rationale for accepting DIP 1028 as is
On 5/27/2020 3:01 AM, Timon Gehr wrote: This is clearly not possible, exactly because the old @safe rules are stronger. Thank you. We can agree on something. But why exactly should API breakage not count? I've addressed exactly this a dozen times or more, to you and others. Repeating myself has become pointless. It's fine to disagree with me. Argue that point. But don't say I didn't address it.
Re: Rationale for accepting DIP 1028 as is
On 5/27/2020 2:34 AM, Bastiaan Veelo wrote: On Wednesday, 27 May 2020 at 09:09:58 UTC, Walter Bright wrote: On 5/26/2020 11:20 PM, Bruce Carneal wrote: I'm not at all concerned with legacy non-compiling code of this nature. Apparently you agree it is not an actual problem. Really? I don't know if you really missed the point being made, or you're being provocative. Both seem unlikely to me. His argument was: "Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing?" I replied that it was unlikely that such legacy code existed. He replied that he was not concerned about it. I.e. working legacy code is not going break.
Re: Rationale for accepting DIP 1028 as is
On 27.05.20 07:54, Walter Bright wrote: On 5/26/2020 1:32 PM, Paul Backus wrote: The reason extern function declarations are particularly problematic is that changing them from @system-by-default to @safe-by-default can cause *silent* breakage in existing, correct code. Can you post an example of currently compiling and correctly working code that will break? Setting aside use of __traits(compiles, ...). What exactly is your standard here? Are you saying we have to produce the following? - Monolithic example, API breakage does not count. - No __traits(compiles, ...) - The code has to compile under both old and new rules. - The code has to corrupt memory in @safe code under new rules. This is clearly not possible, exactly because the old @safe rules are stronger. But why exactly should API breakage not count?
Re: Rationale for accepting DIP 1028 as is
On 27.05.20 11:34, Bastiaan Veelo wrote: On Wednesday, 27 May 2020 at 09:09:58 UTC, Walter Bright wrote: On 5/26/2020 11:20 PM, Bruce Carneal wrote: I'm not at all concerned with legacy non-compiling code of this nature. Apparently you agree it is not an actual problem. Really? I don't know if you really missed the point being made, or you're being provocative. Both seem unlikely to me. -- Bastiaan. It's just selective reading and confirmation bias. Walter did not read past the quoted sentence as it successfully slaughters the straw man he set up in his previous post.
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 09:09:58 UTC, Walter Bright wrote: On 5/26/2020 11:20 PM, Bruce Carneal wrote: I'm not at all concerned with legacy non-compiling code of this nature. Apparently you agree it is not an actual problem. Really? I don't know if you really missed the point being made, or you're being provocative. Both seem unlikely to me. -- Bastiaan.
Re: Rationale for accepting DIP 1028 as is
On 5/27/20 1:49 AM, Walter Bright wrote: On 5/26/2020 9:31 AM, Bruce Carneal wrote: Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing? Nothing at all. That means safe by default is effectively loosening D's notion of safety. This DIP must go.
Re: Rationale for accepting DIP 1028 as is
On 5/26/2020 11:20 PM, Bruce Carneal wrote: I'm not at all concerned with legacy non-compiling code of this nature. Apparently you agree it is not an actual problem.
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 05:49:49 UTC, Walter Bright wrote: On 5/26/2020 9:31 AM, Bruce Carneal wrote: Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing? Nothing at all. But I doubt there is much legacy non-compiling code around. The point isn't that incorrect legacy code that didn't compile now does (although that does matter a bit), it's that newly written code will compile when it shouldn't. Existing code will be full of extern(C) declarations that are implicitly and correctly @system now and will become @safe with dip1028, which means that when I write new @safe code calling those (could be through a deep dependency chain of inferred-@safety APIs, multiple dub packages...) I could easily find my new code compiling when it shouldn't. Effectively, by silently @safe-ing things you can't infer, you will be changing APIs from @system to @trusted without any checks. Just in case there's any confusion, here's a timeline: 1. library A is written containing a dangerous but useful extern(C) declaration assuming @system by default. 2. application B is written for and compiled with dip1028, @safe: at the top of every file. 3. B adds a dependency on A. It continues to compile as @safe, calling an unsafe C function. This seems like one of those things where it's either wrong or a showstopper.
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 02:58:16 UTC, Andrei Alexandrescu wrote: On 5/26/20 12:31 PM, Bruce Carneal wrote: Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing? If that's the case, it's the death of DIP 1028. Walter has confirmed that this is indeed the case. As you can read a few posts up his response to my "What am I missing?" query was "Nothing at all." Yes, it's really that bad. Your help in undoing this mistake would be greatly appreciated by me and by the authors of, literally, hundreds of posts in this thread opposing the DIP.
Re: Rationale for accepting DIP 1028 as is
On Wednesday, 27 May 2020 at 05:49:49 UTC, Walter Bright wrote: On 5/26/2020 9:31 AM, Bruce Carneal wrote: Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing? Nothing at all. But I doubt there is much legacy non-compiling code around. The intent of my above was to illustrate the difference between pre 1028 compiler behavior and post 1028 compiler behavior. I'm not at all concerned with legacy non-compiling code of this nature. I am concerned about the change in compiler behavior. As you've confirmed, post 1028 the compiler will accept an @safe routine calling in to an un-annotated extern C function without complaint. Andre, I hope this is clear enough for you.
Re: Rationale for accepting DIP 1028 as is
On 5/26/2020 1:32 PM, Paul Backus wrote: The reason extern function declarations are particularly problematic is that changing them from @system-by-default to @safe-by-default can cause *silent* breakage in existing, correct code. Can you post an example of currently compiling and correctly working code that will break? Setting aside use of __traits(compiles, ...).
Re: Rationale for accepting DIP 1028 as is
On 5/26/2020 9:31 AM, Bruce Carneal wrote: Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing? Nothing at all. But I doubt there is much legacy non-compiling code around.
Re: Rationale for accepting DIP 1028 as is
On 5/26/20 12:31 PM, Bruce Carneal wrote: Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing? If that's the case, it's the death of DIP 1028.
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 22:47:03 UTC, Gregory wrote: If Walter believed greenwashing was actually a problem, then the best solution to prevent it would be to not make @safe by default. If it's not that serious of a problem that he will push through @safe by default, then greenwashing isn't actually a problem and extern(C) should remain @system. I understand your point now, and I agree.
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 20:32:13 UTC, Paul Backus wrote: On Tuesday, 26 May 2020 at 17:50:58 UTC, Gregory wrote: Which will just lead people to pure @trusted: at the top of their code to get it to compile again, with or without extern(C) being @safe by default. Then someone that uses it as dependency will mistaken think it is @safe. What's to stop this kind of "greenwashing" and why is greenwashing only important to prevent when talking about extern(C) but every other code that will break from this change? The reason extern function declarations are particularly problematic is that changing them from @system-by-default to @safe-by-default can cause *silent* breakage in existing, correct code. By "silent", I mean that there is no message from the compiler warning you that the memory safety of the code has been compromised. It will still compile, and you will only find out that something has gone wrong when you run into memory corruption at run-time. This undermines the entire purpose of @safe, which is to automatically catch code with potential memory errors at compile time. Exactly, that's what I think as well. What I'm trying to convey is that there's a lot of greenwashing that is going to happen. Code doesn't work, slapping @trusted everywhere til it works is what people are going to do, it doesn't matter if extern(C) is @safe or not. Greenwashing occurs no matter what. It's only natural with such a big breaking changing with a feature that you have to "trust". Consider this: Someone is writing code, it's @safe cause it is now the default (yay!). They come across unannotated code while searching for an answer. They copy paste it into their source, ops compile error! Can't use X in @safe code. Ok, so they annotate it as @system. Oh but now their @safe code can't call @system code. So what do they do? Annotate it as @trusted without checking it. If Walter believed greenwashing was actually a problem, then the best solution to prevent it would be to not make @safe by default. If it's not that serious of a problem that he will push through @safe by default, then greenwashing isn't actually a problem and extern(C) should remain @system.
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 20:38:17 UTC, Bastiaan Veelo wrote: On Tuesday, 26 May 2020 at 16:31:57 UTC, Bruce Carneal wrote: Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing? I agree that being forced to think about a trusted interface to that routine can contribute to safety. Not getting this is the price if DIP 1028 as is. Doing it the other way bears a different price. Still, it does not change the amount of code that must be vetted during an audit, either way. If you pick up 100% of the code that the post 1028 compiler lied about, then I agree. Unsafe is unsafe even if your compiler stays mum. If you pick up less than 100%, I disagree. In that situation the compiler will have "helped" you to reduce the amount of code to be audited. And finally, were the 1028 extern!(D) default changed to @system, I disagree. The amount of code in non-greenwashed @trusted wrappers should be smaller. (note that I discount the one time audit of stable @system libraries)
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 16:31:57 UTC, Bruce Carneal wrote: Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing? I agree that being forced to think about a trusted interface to that routine can contribute to safety. Not getting this is the price if DIP 1028 as is. Doing it the other way bears a different price. Still, it does not change the amount of code that must be vetted during an audit, either way. -- Bastiaan.
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 17:50:58 UTC, Gregory wrote: Which will just lead people to pure @trusted: at the top of their code to get it to compile again, with or without extern(C) being @safe by default. Then someone that uses it as dependency will mistaken think it is @safe. What's to stop this kind of "greenwashing" and why is greenwashing only important to prevent when talking about extern(C) but every other code that will break from this change? The reason extern function declarations are particularly problematic is that changing them from @system-by-default to @safe-by-default can cause *silent* breakage in existing, correct code. By "silent", I mean that there is no message from the compiler warning you that the memory safety of the code has been compromised. It will still compile, and you will only find out that something has gone wrong when you run into memory corruption at run-time. This undermines the entire purpose of @safe, which is to automatically catch code with potential memory errors at compile time.
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 18:06:00 UTC, Bastiaan Veelo wrote: On Tuesday, 26 May 2020 at 16:10:24 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:54:31 UTC, Bastiaan Veelo wrote: On Tuesday, 26 May 2020 at 15:39:11 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:01:06 UTC, Bastiaan Veelo wrote: @safe: the compiler checks The compiler does not and cannot check inside @trusted. Whether or not one requires extern(C[++]) to be behind or within @trusted does not change what the compiler can or cannot check. Completely agree but my above says nothing about @trusted. But it matters. Even if your code is @safe, that doesn't mean it is completely safe. Your @safe code can call into somebody else's @safe code that can call @trusted code that calls @system code. If you want to guarantee that your code is safe, you'll have to find and audit the @trusted bits (as well as everything that is called from there). Agree that the transitive closure is difficult, especially for humans. If extern(C) is implicitly @system, you'll have to make calls @trusted and they are subject to auditing. If extern(C) is implicitly @safe, calls are still subject to auditing. Whatever your preference, I think the compiler can be of greater help in finding the bits that require auditing than grep can, and then the difference isn't all that important anymore. Safe wrappers around C libraries are still written the same way, audited the same way, and unsafe calls into C caught the same way. Agree. @trusted doesn't mean that the code so labelled should be blindly trusted. The aim should be reduction of @trusted where practical. Silently letting all extern C functions inside the tent is not a reduction. I agree that making extern(!D) @system looks sound, like the way it should be. But it breaks compilation of existing code, and it is too easy to evade by slapping on an explicit @trusted or (God forbid) @safe. Or we could make extern(!D) to be anything other than @system an error, which would be really annoying. I think that what we all strive for is to reduce the strain of coding as well as to reduce the strain of auditing. I'm all for reducing the strain of writing correct code. As others have implied, we're dealing with, at least, two issues here: 1) any transition from opt-in to opt-out safety will break code. Some programmers will resort to greenwashing in an effort to "fix" it. Will we be better off post transition? And 2) should uncheckable code be considered @safe, ever.
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 16:10:24 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:54:31 UTC, Bastiaan Veelo wrote: On Tuesday, 26 May 2020 at 15:39:11 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:01:06 UTC, Bastiaan Veelo wrote: @safe: the compiler checks The compiler does not and cannot check inside @trusted. Whether or not one requires extern(C[++]) to be behind or within @trusted does not change what the compiler can or cannot check. Completely agree but my above says nothing about @trusted. But it matters. Even if your code is @safe, that doesn't mean it is completely safe. Your @safe code can call into somebody else's @safe code that can call @trusted code that calls @system code. If you want to guarantee that your code is safe, you'll have to find and audit the @trusted bits (as well as everything that is called from there). If extern(C) is implicitly @system, you'll have to make calls @trusted and they are subject to auditing. If extern(C) is implicitly @safe, calls are still subject to auditing. Whatever your preference, I think the compiler can be of greater help in finding the bits that require auditing than grep can, and then the difference isn't all that important anymore. Safe wrappers around C libraries are still written the same way, audited the same way, and unsafe calls into C caught the same way. I agree that making extern(!D) @system looks sound, like the way it should be. But it breaks compilation of existing code, and it is too easy to evade by slapping on an explicit @trusted or (God forbid) @safe. Or we could make extern(!D) to be anything other than @system an error, which would be really annoying. I think that what we all strive for is to reduce the strain of coding as well as to reduce the strain of auditing. -- Bastiaan.
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 16:20:23 UTC, Atila Neves wrote: On Tuesday, 26 May 2020 at 16:10:24 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:54:31 UTC, Bastiaan Veelo wrote: On Tuesday, 26 May 2020 at 15:39:11 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:01:06 UTC, Bastiaan Veelo wrote: @safe: the compiler checks The compiler does not and cannot check inside @trusted. Whether or not one requires extern(C[++]) to be behind or within @trusted does not change what the compiler can or cannot check.j Completely agree but my above says nothing about @trusted. @safe post 1028: the compiler checks, sometimes, just not in the scary parts The amount of code that requires human auditing remains the same. What matters is how to find that code, and how to maintain the validity of the audits. Another distinction: pre 1028 your compilation will error out. Post 1028 it will not. Quite the opposite. Most code out there isn't marked as @safe/@trusted/@system. If I add a dub dependency and don't bother with @safe, I can call anything, in any way. Post 1028... nope. Unmarked @system function definitions themselves won't compile. Which will just lead people to pure @trusted: at the top of their code to get it to compile again, with or without extern(C) being @safe by default. Then someone that uses it as dependency will mistaken think it is @safe. What's to stop this kind of "greenwashing" and why is greenwashing only important to prevent when talking about extern(C) but every other code that will break from this change?
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 16:20:23 UTC, Atila Neves wrote: On Tuesday, 26 May 2020 at 16:10:24 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:54:31 UTC, Bastiaan Veelo wrote: On Tuesday, 26 May 2020 at 15:39:11 UTC, Bruce Carneal wrote: [...] The compiler does not and cannot check inside @trusted. Whether or not one requires extern(C[++]) to be behind or within @trusted does not change what the compiler can or cannot check.j Completely agree but my above says nothing about @trusted. [...] The amount of code that requires human auditing remains the same. What matters is how to find that code, and how to maintain the validity of the audits. Another distinction: pre 1028 your compilation will error out. Post 1028 it will not. Quite the opposite. Most code out there isn't marked as @safe/@trusted/@system. If I add a dub dependency and don't bother with @safe, I can call anything, in any way. Post 1028... nope. Unmarked @system function definitions themselves won't compile. I think I see the difficulty. We both support 1028's elimination of the "I can call anything, in any way" default. I really like @safe by default for machine checkable code. I don't support @safe by default for extern C.
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 16:20:23 UTC, Atila Neves wrote: On Tuesday, 26 May 2020 at 16:10:24 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:54:31 UTC, Bastiaan Veelo wrote: [...] Completely agree but my above says nothing about @trusted. [...] Another distinction: pre 1028 your compilation will error out. Post 1028 it will not. Quite the opposite. Most code out there isn't marked as @safe/@trusted/@system. If I add a dub dependency and don't bother with @safe, I can call anything, in any way. Post 1028... nope. Unmarked @system function definitions themselves won't compile. Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked. Post 1028, IIUC, the compilation will go through without complaint. This seems quite clear. What am I missing?
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 16:10:24 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:54:31 UTC, Bastiaan Veelo wrote: On Tuesday, 26 May 2020 at 15:39:11 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:01:06 UTC, Bastiaan Veelo wrote: @safe: the compiler checks The compiler does not and cannot check inside @trusted. Whether or not one requires extern(C[++]) to be behind or within @trusted does not change what the compiler can or cannot check.j Completely agree but my above says nothing about @trusted. @safe post 1028: the compiler checks, sometimes, just not in the scary parts The amount of code that requires human auditing remains the same. What matters is how to find that code, and how to maintain the validity of the audits. Another distinction: pre 1028 your compilation will error out. Post 1028 it will not. Quite the opposite. Most code out there isn't marked as @safe/@trusted/@system. If I add a dub dependency and don't bother with @safe, I can call anything, in any way. Post 1028... nope. Unmarked @system function definitions themselves won't compile.
Re: Rationale for accepting DIP 1028 as is
On Tuesday, 26 May 2020 at 15:54:31 UTC, Bastiaan Veelo wrote: On Tuesday, 26 May 2020 at 15:39:11 UTC, Bruce Carneal wrote: On Tuesday, 26 May 2020 at 15:01:06 UTC, Bastiaan Veelo wrote: @safe: the compiler checks The compiler does not and cannot check inside @trusted. Whether or not one requires extern(C[++]) to be behind or within @trusted does not change what the compiler can or cannot check.j Completely agree but my above says nothing about @trusted. @safe post 1028: the compiler checks, sometimes, just not in the scary parts The amount of code that requires human auditing remains the same. What matters is how to find that code, and how to maintain the validity of the audits. Another distinction: pre 1028 your compilation will error out. Post 1028 it will not.
