Re: D Logic bug
On 11/10/18 20:16, Kagamin wrote: On Thursday, 11 October 2018 at 14:35:34 UTC, James Japherson wrote: In the ternary operator it should treat parenthesis directly to the left as the argument. I don't think parentheses are ever treated like that. They are self-contained and don't affect operators outside them. Almost. const(int) *
Farewell (of sorts)
Hello everyone, First of all, I know I've had a shorter than usual fuse of late. I'd like to apologize to everyone about this. It is the culmination of quite a few things increasing the load I'm under. One of those things is this: October 14th will be my last day working for Weka.IO. Accordingly, my involvement in D will be considerably reduced after that date, as working with D will no longer be my day job. A few of you knew that I was looking for a new job, but I postponed officially announcing this, as I wanted to see who will be taking over maintenance of Mecca. On that front, I have some good news and some bad news. The bad news is that the person taking over will not have Mecca as his sole responsibility. I am hoping he'll be able to do enough. The good news is that they put on this task my top pick for it. His name is Eyal Lotem, and is a great developer. Some of you have met him as he attended DConf three years ago. I will probably keep half an eye on the forum, and I might also be around on the Slack channels. My email address will change as a result. Feel free to find me at firstn...@lastname.biz, after applying the relevant substitutions. I think it's a better captcha than running D code :-) Shachar
LDC2 1.9.0 beta 1 bug
I got this as a report from a user, not directly running this, which is why I'm not opening a bug report. Consider the following function: void f(ARGS...)(ARGS args, bool arg1 = true, char arg2 = 'H'); Now consider the following call to it: f(true, 'S'); Theoretically, this can either be calling f!()(true, 'S') or f!(bool, char)(true, 'S', true, 'H'); Under 1.8.0, it would do the former. Under 1.9.0-beta1, the later. Why is this a bug? Two reasons. First, this is a change of behavior. More to the point, however, expanding the call to the second form means that I can *never* supply non-default values to arg1 and arg2. root@5f3623338be8:/src# ldc2 --version LDC - the LLVM D compiler (1.9.0-beta1): based on DMD v2.079.1 and LLVM 6.0.0 built with LDC - the LLVM D compiler (1.9.0-beta1) Default target: x86_64-unknown-linux-gnu Host CPU: broadwell http://dlang.org - http://wiki.dlang.org/LDC Shachar
Re: DIP 1014
On 04/10/18 13:43, Stanislav Blinov wrote: * move the data as part of the call hook rather than before * Use a different name and signature on the hook function Yes, exactly. It would have to be special if you don't want to leave room for the compiler implementors. That's not how standards work. If you don't want compiler implementors to have a choice in the matter, you put MUST in the specs. Doing anything else is, by and large, considered harmful. The calling convention for particular types (i.e. those that do have a move hook defined) would have to be enforced in some way. See the neighbor thread wrt move semantics by kinke. Two distinct things. Kinke was talking about how to pass a struct through the ABI. You are talking about special-casing a specific name. Not to mention, your special case is to transform it to something you can *already* specify in the language. Why? Which is, however, not a reason to formalize it and make it a requirement for an isolated specific case, such as this one, utilizing a syntax that is currently not used by the language. There is positively nothing in DIP 1014 that is "syntax not used by the language". Quite the contrary. As opposed to trying to fit existing language semantics to something that the language didn't seem to want to allow in the first place. Formalize it as a suggestion, and we can discuss the "as opposed to". Like I said, I think there's a lot you're glossing over here (such as backwards compatibility). Shachar
Re: DIP 1014
On 04/10/18 11:16, Paolo Invernizzi wrote: While I want to thank you both, about the quality of this thread, what kind of "consequences that go beyond what I think you understand" are you thinking of? Can you give an example? Assuming I understand Stanislav's proposal correctly (an assumption I'm reluctant to make, hence my request for something more formal), it boils down to two points: * move the data as part of the call hook rather than before * Use a different name and signature on the hook function The first one we can argue for or against. My original proposal was phrased the way it was precisely because that's the way copying works in D (copy first, patch the data later). About a week after I submitted it, Andrei came forward with requesting to move to copy constructors. The second, to me, is a non-starter. The only way you'd get a function who's signature is: void someName(Type rhs); But which actually maintains rhs's address from before the call is if the compiler treats "someName" as a special case, and emits code which would normally be emitted for the function: void someName(ref Type rhs); That's why it was important for me to clear up whether there is *ever* a case in the current language where that happens (answer: only by accident, which is the same as saying "no"). So to get that to work, you'd need to insert a special case into the ABI of the language: if the function's name is someName, treat it differently. At this point, you might as well call a spade a spade, and just give the function that signature explicitly. s/someName/opPostMove/, and you get DIP 1014 (as far as point #2 is concerned). Shachar
Re: DIP 1014
On 04/10/18 11:05, Stanislav Blinov wrote: On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh wrote: If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working. You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong. For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much. The example isn't brittle. It is simply not an example. If you want to leave it out, however, then I think you should submit an orderly proposal. The changes you seem to be suggesting have consequences that go beyond what I think you understand, and there can be no serious discussion of it while it is not clear from your posts which part of what you say is the relevant one. Shachar
Re: DIP 1014
On 03/10/18 23:25, Stanislav Blinov wrote: It *is* true when the type doesn't have a destructor. Extending that to a move hook, it will also be true because destruction will be elided. I know what you're talking about, that happens for types that have destructors. No, destructors have nothing to do with it, as well they shouldn't. The whole point of D moving structs around is that no destruction is needed. It took me a while to figure out why your program does appear to work. At first I thought it was because of inlining, but that was wrong. The reason your test case works (sometimes, if you don't breath on it too heavily) is because the object is actually moved twice. Once when returning from the function into the variable, and another when copied into opAssign's argument. This results in it returning to its original address. If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working. You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong. To verify my guess is right, I tried the following change: add to createCounter and createCounterNoNRV in your original program (no destructors) the following two lines: int a; write(a); You have added another local variable to the functions, but otherwise changed absolutely nothing. You will notice your program now has an offset. Shachar
Re: DIP 1014
On 03/10/18 20:43, Stanislav Blinov wrote: On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh wrote: I.e. - I am asserting if a move was not caught. The program fails to run on either ldc or dmd. To me, this makes perfect sense as for the way D is built. In essence, opAssign isn't guaranteed to run. Feel free to build a struct where that assert passes to convince me. That's a slightly different issue here. Well, I view this issue as a deal breaker. If you need to move the object *in order* to pass it to your move hook, then anything that requires knowing the address of the old instance will, by definition, not work. Look at the output. The operator is being run, it can't *not* run, Sure it can. Just look at the example I posted on the other thread (https://forum.dlang.org/post/pp2v16$1014$1...@digitalmars.com). The hook you mention is downright @disabled there. In fact, had that not been the case, this DIP would never have happened. Here is the flaw in your logic: void opAssign(Tracker rhs) rhs is passed by value. This means that already at the point opAssign is called, rhs *already* has a different address than the one it was passed in with. Currently that is only true if you define a destructor. No, that's not true. Try printing the instance's address in the constructor and again in your operator. In the presence of a move hook, 'rhs' would first have to pass through that hook, which will not take destructors into account at all. I'm sorry, I'm not following. What is the difference between what you're proposing and opPostMove as defined? Consider your own DIP: what you're suggesting is the ability to take the address of the original when a move is taking place. My example shows that in the simplest case even today, address of the original is already the address of the argument. This is the run I got: $ ./movetest2 Address of temporary is 'b382e390', counter points to 'b382e390' ... which is '0' bytes from the address of temporary. Address of temporary is 'b382e390', counter points to '8eb82b60' ... which is '-966984906800' bytes from the address of temporary. Address of temporary is 'b382e390', counter points to 'b382e390' ... which is '0' bytes from the address of temporary. Address of temporary is 'b382e390', counter points to '8eb82b60' ... which is '-966984906800' bytes from the address of temporary. I'm not sure what I should have seen, or what I should have concluded from it. This is your original program, unmodified. The changes are literally the same as the ones you're proposing: "When moving a struct's instance, the compiler MUST call __move_post_blt giving it both new and old instances' addresses." That is the same that would have to happen with this(typeof(this) rhs), where is the address of new instance, and is the address of old instance, but there's no need for opPostMove then. I guess what I should've said from the start is that the semantics you're proposing fit nicely within one special function, instead of two. Except, like I said, it's not working for me, and I find it hard to understand how it *can* work (inlining notwithstanding), which is why I did not propose it. this(typeof(this)), of course, would need to be special in the ABI, but again, that's one special function instead of two. No. My proposal requires one amendment to argument passing in the ABI, but no special cases at all. Changes to the ABI are not the same as changes to the run time library. Let's take a step back for a moment and look at what should actually be happening for this hook to work (which you briefly mention in the DIP): 1. The compiler constructs the value. In your case, it constructs two: It does not. It copies the bits from one to the other. This also means that a struct where some of its members have a hook is copied wholesale and patched, which is typically faster than copying in parts. the original and the new one. In my case, it constructs the original and then passes it over to the move ctor (one blit potentially avoided). 2. It calls the hook (move ctor). I'm not sure I follow you on that one. What did you mean? 3. In your case, it calls the opPostMove. In all cases, you need to call the hook, whatever it is, for those structs that have it, and do some default handling for those that don't. 4. In any case, it *doesn't* destruct the original. Ever. The alternative would be to force the programmer to put the original back into valid state, and suddenly we're back to C++ with all it's pleasantries. That last part is quite different from the current model, in which the compiler always destructs function arguments. That's why my example fails when a destructor is present. Like I said above, I don't think that's correct. The other thing to note (again something that you mention but don't expand on), and that's nodding back to my comment about making move() and emplace()
Re: DIP 1014
On 03/10/18 12:48, Corel wrote:
The fact that in D the structures to date are not moved, is known for
years ... take advantage of this fact, and move on.
I have no idea where you got this fact:
import std.stdio;
struct MoveTest {
static uint counter=1;
uint id;
@disable this(this);
@disable this(MoveTest);
this(uint dummy) {
id = counter++;
writefln("Constructed %s id %s", , id);
}
~this() {
writefln("Id %s destroyed at %s", id, );
}
}
MoveTest func1() {
return MoveTest(3);
}
void func2(MoveTest m) {
}
int main() {
func2(func1());
return 0;
}
$ rdmd movetest.d
Constructed 7FFDC7A663E0 id 1
Id 1 destroyed at 7FFDC7A66400
Our instance was constructed at one address, but destroyed at another.
In other words, it was moved.
Can we, please, put that myth to rest?
Shachar
Re: DIP 1014
On 03/10/18 18:33, Shachar Shemesh wrote:
~this() {
writefln("%s destructed", );
assert(counter is null || counter is );
}
You might also want to add @disable this(this); and remove the dead code
(i.e. - the case where the pointer is global) to reduce noise. I
verified that neither one changes anything in the outcome.
Shachar
Re: DIP 1014
On 03/10/18 17:29, Stanislav Blinov wrote:
OMG, that's so simple!!! Why didn't I think of it?
Oh wait, I did.
Now I see why sometimes your posts are greeted with hostility.
Yes. I am actually sorry about that. I was responding to your assumption
that I'm wrong. Had your post been phrased as "why didn't you", instead
of "you're wrong wrong wrong" I wouldn't have responded that way.
Like I said, I am sorry.
> Allow me to further illustrate with something that can be written in
D > today:
I am not sure what you were trying to demonstrate, so instead I wanted
to see if you succeeded. I added the following to your Tracker struct:
~this() {
writefln("%s destructed", );
assert(counter is null || counter is );
}
I.e. - I am asserting if a move was not caught. The program fails to run
on either ldc or dmd. To me, this makes perfect sense as for the way D
is built. In essence, opAssign isn't guaranteed to run. Feel free to
build a struct where that assert passes to convince me.
Here is the flaw in your logic:
void opAssign(Tracker rhs)
rhs is passed by value. This means that already at the point opAssign is
called, rhs *already* has a different address than the one it was passed
in with. I did not follow your logic on why this isn't so, but I don't
see how you can make it not so without changing the ABI quite drastically.
Shachar
Re: DIP 1014
On 03/10/18 16:56, Stanislav Blinov wrote:
struct S {
this(S rhs);
OMG, that's so simple!!! Why didn't I think of it?
Oh wait, I did.
And this simply and utterly doesn't work.
If you read the DIP, you will notice that the *address* in which the old
instance resides is quite important for performing the actual move. This
is not available with the interface you're suggesting, mainly because by
the time you have rhs, it has already moved.
In other words, for the interface above to work, the type must already
be movable, which kinda contradict what we're trying to achieve here.
in C++ would have an equivalent signature of
ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
No, it is not. You see, in C++, x is an "rvalue *reference*". x has not
moved by this point in the run, it has simply had its address passed to
foo. Please see
https://stackoverflow.com/questions/28483250/rvalue-reference-is-treated-as-an-lvalue
Shachar
Re: DIP 1014
On 03/10/18 04:10, Walter Bright wrote:
On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote:
Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler
moved structs in a number of situations - e.g. when the return value
was an rvalue. Something like
Eh, I don't think that moves it, but rather just constructs it
in-place for the next call.
The technical term for that is "copy elision".
I'm not sure I follow.
First of all, you cannot elide the copy if there is more than one
potential local variable you are returning, ala:
A someFunc() {
A a, b;
manipulate(a); manipulate(b);
if( someRandomCondition )
return a;
return b;
}
What happens then?
What happens if A has @disable this(this)?
What happens if we explicitly call std.algorithm.move?
Shachar
Re: DIP 1014
On 30/09/18 10:26, Manu wrote: Other implementations make much better use of that built-in space by not wasting 8 bytes on an interior pointer for small-strings. I will point out that a pointer that *sometimes* points to an internal member was one of the use cases I documented when I submitted the DIP. Starting a long discussion about the merits of the design is a bit off-topic. I will point out that branch prediction considerations *might* make this a wise choice, despite the loss of 8 bytes of potential storage. Either way, this is a design that is highly sensitive to precise use pattern, which, admitably, GNU's std::string probably can't know. Shachar
Re: Updating D beyond Unicode 2.0
On Saturday, 29 September 2018 at 16:19:38 UTC, ag0aep6g wrote: On 09/29/2018 04:19 PM, Shachar Shemesh wrote: On 29/09/18 16:52, Dukc wrote: [...] I know you meant Sarn, but still... can you please be a bit less aggresive with our wording? From the article (the furthest point I read in it): When I ask myself what I've found life is too short for, the word that pops into my head is "bullshit." Dukc didn't post that link. sarn did. You are 100% correct. My most sincere apologies. I am going to stop responding to this thread now. Shachar
Re: Updating D beyond Unicode 2.0
On 29/09/18 16:52, Dukc wrote: On Saturday, 29 September 2018 at 02:22:55 UTC, Shachar Shemesh wrote: I missed something he said in one of the other (as of this writing, 98) posts of this thread, and thus causing Dukc to label me a bullshitter. I know you meant Sarn, but still... can you please be a bit less aggresive with our wording? From the article (the furthest point I read in it): When I ask myself what I've found life is too short for, the word that pops into my head is "bullshit." That is the word used by the article *you* linked to, in reference to me. If it offends you enough to be accused of *calling* someone that, just imagine how I felt being *called* that very same name. Seriously, I don't make it a habit of being offended by random people on the Internet, but this is more a conscious decision than a naturally thick skin. Seeing that label hurt. Don't worry. I've been on the Internet since 1991. That's longer than the median age here (i.e. - I've been on the Internet since before most of you have been born). I've had my own fair share of flame wars, include some that, to my chagrin, I've started. In other words, I got over it. I did not reply, big though the temptation was. But the right time to be sensitive about what words are being used was *before* you linked to the article. Taking offense from being called out for calling someone something you find offensive is hypocritical. I never understood the focus on words. It's not the use of that word that offended me, it's the fact that you thought anything I did justified using it. I don't think using "cattle excrement" instead would have been any less hurtful. And it's not that the rest of your post was thoughtful, considerate and took pains to give constructive criticism, with or without hurting anyone's feelings. It's just that it doesn't seem to be that part bothers you. Shachar
Re: Updating D beyond Unicode 2.0
On 28/09/18 14:37, Dukc wrote: On Friday, 28 September 2018 at 02:23:32 UTC, sarn wrote: Shachar seems to be aiming for an internet high score by shooting down threads without reading them. You have better things to do. http://www.paulgraham.com/vb.html I believe you're being too harsh. It's easy to miss a part of a post sometimes. A minor correction: Aliak is not accusing me of missing a part of the post. He's accusing me of not taking into account something he said in a different part of the *thread*. I.e. - I missed something he said in one of the other (as of this writing, 98) posts of this thread, and thus causing Dukc to label me a bullshitter.
Re: Updating D beyond Unicode 2.0
On 27/09/18 16:38, aliak wrote: The point was that being able to use non-English in code is demonstrably both helpful and useful to people. Norwegian happens to be easily anglicize-able. I've already linked to non ascii code versions in a previous post if you want that too. If you wish to make a point about something irrelevant to the discussion, that's fine. It is, however, irrelevant, mostly because it is uncontested. This thread is about the use of non-English in *identifiers*. This thread is not about comments. It is not about literals (i.e. - strings). Only about identifiers (function names, variable names etc.). If you have real world examples of those, that would be both interesting and relevant. Shachar
Re: Updating D beyond Unicode 2.0
On 27/09/18 10:35, aliak wrote: Here's an example from this years spring semester and NTNU (norwegian uni): http://folk.ntnu.no/frh/grprog/eksempel/eks_20.cpp ... That's the basic programming course. Whether the professor would use that I guess would depend on ratio of English/non-English speakers. But it's there nonetheless. I'm sorry I keep bringing this up, but context is really important here. The program you link to has non-ASCII in the comments and in the literals, but not in the identifiers. Nobody is opposed to having those. Shachar
Re: D IDE
On 27/09/18 04:54, Nick Sabalausky (Abscissa) wrote: Man, I wish SOO much, that was true of my favorite editor (Programmer's Notepad 2). I love it, but it's a windows thing and has some issues under wine. Can you elaborate on what issues? Merely downloading and installing seem to work fine.
Re: Updating D beyond Unicode 2.0
On 26/09/18 10:26, Dukc wrote: On Wednesday, 26 September 2018 at 06:50:47 UTC, Shachar Shemesh wrote: The properties that cause city names to be poor candidates for enum values are the same as those that make them Unicode candidates. How so? City names (data, changes over time) as enums (compile time set) seem like a horrible idea. In most cases yes. But not always. You might me doing some sort of game where certain cities are a central concept, not just data with properties. Another possibility is that you're using code as data, AKA scripting. And who says anyway you can't make a program that's designed specificially for certain cities? Sure you can. It's just very poor design. I think, when asking such questions, two types of answers are relevant. One is hypotheticals where you say "this design requires this". For such answers, the design needs to be a good one. It makes no sense to design a language to support a hypothetical design which is not a good one. The other type of answer is "it's being done in the real world". If it's in active use in the real world, it might make sense to support it, even if we can agree that the design is not optimal. Since your answer is hypothetical, I think arguing this is not a good way to code is a valid one. Shachar
Re: Updating D beyond Unicode 2.0
On 25/09/18 15:35, Dukc wrote: Another reason is that something may not have a good translation to English. If there is an enum type listing city names, it is IMO better to write them as normal, using Unicode. CityName.seinäjoki, not CityName.seinaejoki. This sounded like a very compelling example, until I gave it a second thought. I now fail to see how this example translates to a real-life scenario. City names (data, changes over time) as enums (compile time set) seem like a horrible idea. That may sound like a very technical objection to an otherwise valid point, but it really think that's not the case. The properties that cause city names to be poor candidates for enum values are the same as those that make them Unicode candidates. Shachar
Re: Updating D beyond Unicode 2.0
On 23/09/18 15:38, sarn wrote: On Sunday, 23 September 2018 at 06:53:21 UTC, Shachar Shemesh wrote: On 23/09/18 04:29, sarn wrote: You can find a lot more Japanese D code on this blogging platform: https://qiita.com/tags/dlang Here's the most recent post to save you a click: https://qiita.com/ShigekiKarita/items/9b3aa8f716848278ef62 Comments in Japanese. Identifiers in English. Not advancing your point, I think. Shachar Well, I knew that when I posted, so I honestly have no idea what point you assumed I was making. I don't know what point you were trying to make. That's precisely why I posted. I don't think D currently or ever enforces what type of (legal UTF-8) text you could use in comments or strings. This thread is about what's legal to use in identifiers. The example you brought does not use Unicode in identifiers, and is, therefor, irrelevant to the discussion we're having. That was the point *I* was trying to make. Shachar
Re: Updating D beyond Unicode 2.0
On 23/09/18 04:29, sarn wrote: On Sunday, 23 September 2018 at 00:18:06 UTC, Adam D. Ruppe wrote: I have seen Japanese D code before on twitter, but cannot find it now (surely because the search engines also share this bias). You can find a lot more Japanese D code on this blogging platform: https://qiita.com/tags/dlang Here's the most recent post to save you a click: https://qiita.com/ShigekiKarita/items/9b3aa8f716848278ef62 Comments in Japanese. Identifiers in English. Not advancing your point, I think. Shachar
Re: Updating D beyond Unicode 2.0
On 22/09/18 15:13, Thomas Mader wrote: Would you suggest to remove such writing systems out of Unicode? What should a museum do which is in need of a software to somehow manage Egyptian hieroglyphs? If memory serves me right, hieroglyphs actually represent consonants (vowels are implicit), and as such, are most definitely "characters". The only language I can think of, off the top of my head, where words have distinct signs is sign language. It is a good question whether Unicode should include such a language (difficulty of representing motion in a font aside). Shachar
Re: Updating D beyond Unicode 2.0
On 22/09/18 14:28, Jonathan M Davis wrote: As I said, it's exactly the same as arguing that words should be represented in Unicode. Unfortunately, however, at least some of them are in there. :| - Jonathan M Davis To be fair to them, that word is part of the "Arabic-representation forms" section. The "Presentation forms" sections are meant as backwards compatibility toward code points that existed before, and are not meant to be generated by Unicode aware applications. Shachar
Re: Updating D beyond Unicode 2.0
On 22/09/18 11:52, Jonathan M Davis wrote: Honestly, I was horrified to find out that emojis were even in Unicode. It makes no sense whatsover. Emojis are supposed to be sequences of characters that can be interepreted as images. Treating them like Unicode symbols is like treating entire words like Unicode symbols. It's just plain stupid and a clear sign that Unicode has gone completely off the rails (if it was ever on them). Unfortunately, it's the best tool that we have for the job. - Jonathan M Davis Thank Allah that someone said it before I had to. I could not agree more. Encoding whole words as single Unicode code points makes no sense. U+FDF2 Shachar
Re: Small @nogc experience report
On 19/09/18 22:53, Walter Bright wrote: On 9/19/2018 10:13 AM, Shachar Shemesh wrote: assert(condition, string); // string is useless without actual info about what went wrong. assert(condition, format(string, arg, arg)); // No good - format is not @nogc Another method: debug assert(condition, format(string, arg, arg)); else assert(condition, string); because @nogc is ignored in debug conditionals, just like purity is ignored in debug conditionals. That doesn't cut it on so many levels... First of all, no four lines solution that requires copy/paste (or worse, retyping) as a standard will actually get employed by programmers. The disincentive is too high. If we overcome this problem, we're still left with the fact that in all of the important runs the data I need in order to debug an assert violation is not going to be there when I need it. Let me put it this way: the Weka code base has over 13,000 asserts. Every single time an assert triggered on me that either did not have a message, or had only a message but not data, I needed more data than it had. It is, unfortunately, not true to say that this only ever happened in debug builds. In fact, we do release builds with asserts as part of our CI, so a relatively rare bug has a very high chance of throwing an assert in non-debug runs. Writing asserts should be easy, and making them useful in case they trigger should also be easy. What's more, soft mandating writing asserts with assert messages and parameters all but eliminates the common anti-pattern used by many novices of asserting that the compiler does what it's supposed to, e.g: if( a>13 ) return; assert(a<13); Shachar
Re: Small @nogc experience report
On 19/09/18 21:35, Steven Schveighoffer wrote:
On 9/19/18 1:13 PM, Shachar Shemesh wrote:
There is a catch, though. Writing Mecca with @nogc required
re-implementing quite a bit of druntime. Mecca uses its own exception
allocations (mkEx, just saw it's not yet documented, it's under
mecca.lib.exception). The same module also has "enforceNGC". We also
have our own asserts. This is partially to support our internal
logging facility, that needs a static list of formats, but it also
solves a very important problem with D's @nogc:
void func() @nogc {
assert(condition, string); // string is useless without actual info
about what went wrong.
assert(condition, format(string, arg, arg)); // No good - format is
not @nogc
ASSERT!"format"(condition, arg, arg); // @nogc and convenient
}
So, yes, we do use @nogc, but it took a *lot* of work to do it.
I'm running into this coincidentally right now, when trying to debug a
PR. I found I'm getting a range error deep inside a phobos function. But
because Phobos is trying to be pure @nogc nothrow @safe, I can do almost
nothing to display what is wrong.
What I ended up doing is making an extern(C) hook that had the "right"
attributes, even though it's not @nogc (let's face it, you are about to
crash anyway).
But it got me thinking, what a useless interface to display errors we
have! Inside Throwable, there is the function toString(someDelegate
sink) which prints out the exception trace.
Near the front there is this:
if (msg.length)
{
sink(": "); sink(msg);
}
My, wouldn't it be nice to be able to override this! And forget about
the whole msg BS. When an exception trace is printed, there are almost
no restrictions as to what can be done. We should delay the generation
of the message until then as well! Not to mention that if we can output
things piecemeal through the sink, we don't even have to allocate at all.
I'm going to write up a more detailed post on this, but it's annoying to
throw exceptions without any information EXCEPT what can be converted
into a string at runtime at the time of exception. All that is missing
is this hook to generate the message.
-Steve
Then by all means, have a look at ASSERT inside mecca.lib.exception.
Re: Small @nogc experience report
On 08/09/18 11:07, Peter Alexander wrote:
I'd love to know if anyone is making good use of @nogc in a larger code
base and is happy with it. Weka.io?
No, sorry.
Actually, yes.
Well, sortof.
The main Weka codebase hardly uses any annotations of any kind. Not
@nogc nor others. This is in the process of being amended, somewhat, but
is not a high priority. We do use run-time detection of GC use. I.e. -
we've modified the druntime to invoke a callback if a GC allocation
takes place, and we then log that fact (with traceback). We then are
able to search logs for GC allocations and remove them.
So that's the "no" part.
As pointed out, one of the main motivations for running Mecca was to
clear up the strange solutions that have accumulated over the years. As
such, the Mecca code does have @nogc in much more wide use.
So, yes.
There is a catch, though. Writing Mecca with @nogc required
re-implementing quite a bit of druntime. Mecca uses its own exception
allocations (mkEx, just saw it's not yet documented, it's under
mecca.lib.exception). The same module also has "enforceNGC". We also
have our own asserts. This is partially to support our internal logging
facility, that needs a static list of formats, but it also solves a very
important problem with D's @nogc:
void func() @nogc {
assert(condition, string); // string is useless without actual info
about what went wrong.
assert(condition, format(string, arg, arg)); // No good - format is
not @nogc
ASSERT!"format"(condition, arg, arg); // @nogc and convenient
}
So, yes, we do use @nogc, but it took a *lot* of work to do it.
The good news is that mecca is available, and you can just dub it into
your project and use it, so you don't have to repeat that whole set of work.
Mecca was advertised mostly around the reactor. While it is a piece of
work I am very proud of, it is not the only part there, nor is it
necessary to use mecca's reactor if you want just the library. In fact,
nothing outside of mecca.reactor depends on the reactor running.
Hope this helps,
Shachar
Re: Small @nogc experience report
I've got plenty to say, but here is the long and the short of it: Use Mecca.
On 07/09/18 19:44, Peter Alexander wrote:
3. It was really frustrating that I had to make the compiler happy
before I was able to run anything again. Due to point #1 I had to move
code around to restructure things and wanted to make sure everything
continued working before all GC allocations were removed.
mecca.lib.reflection has "as".
https://weka-io.github.io/mecca/docs/mecca/lib/reflection/as.html
Here is how you use it:
void fun() @nogc {
as!"@nogc"( some code that is not @nogc );
}
4. I used std.algorithm.topNCopy, which is not @nogc. The error just
says "cannot call non-@nogc function [...]". I know there are efforts to
make Phobos more @nogc friendly, but seeing this error is like hitting a
brick wall. I wouldn't expect topNCopy to use GC, but as a user, what do
I do with the error? Having to dig into Phobos source is unpleasant.
Should I file a bug? What if it is intentionally not @nogc for some
subtle reason? Do I rewrite topNCopy?
5. Sometimes I wanted to add writeln to my code to debug things, but
writeln is not @nogc, so I could not. I could have used printf in
hindsight, but was too frustrated to continue.
mecca.log has logging facilities that are @nogc.
Now, to be fair, they are not actually @nogc, as by default it uses
writeln. It is, however, annotated with @nogc, for precisely the reasons
you encountered.
Shachar
Re: Java also has chained exceptions, done manually
On 07/09/18 09:42, Don wrote: A loop is not possible in ordinary operation, any more than a loop is possible in a depth-first traverse of a tree. But, what I don't know is, what happens if there is a switch to a different fiber inside a `finally` clause? I never considered that. Each fiber maintains its own EH context. In theory (and I will not claim that I actually understand how this construct works), if you switch to a different fiber, you'll be out of the EH handling code until you switch back. It caused quite a bit of headache for Mecca, as we do not switch fibers using the DRuntime mechanism. At the moment, Mecca just copies internal DRuntime implementation details while keeping our fingers tightly crossed that nothing gets borken. I have, fairly high on my to-do list, a task to submit a PR to DRuntime to properly export this interface, thus removing access to private members from Mecca. Such a change would also allow DRuntime to implement the Fiber class somewhere other than core.thread. Shachar
Re: This thread on Hacker News terrifies me
On 31/08/18 23:22, Steven Schveighoffer wrote: On 8/31/18 3:50 PM, Walter Bright wrote: https://news.ycombinator.com/item?id=17880722 Typical comments: "`assertAndContinue` crashes in dev and logs an error and keeps going in prod. Each time we want to verify a runtime assumption, we decide which type of assert to use. We prefer `assertAndContinue` (and I push for it in code review)," e.g. D's assert. Well, actually, D doesn't log an error in production. -Steve I think it's the music of the thing rather than the thing itself. Mecca has ASSERT, which is a condition always checked and that always crashes the program if it fails, and DBG_ASSERT, which, like D's built in assert, is skipped in release mode (essentially, an assert where you can log what went wrong without using the GC needing format). When you compare this to what Walter was quoting, you get the same end result, but a vastly different intention. It's one thing to say "this ASSERT is cheap enough to be tested in production, while this DBG_ASSERT one is optimized out". It's another to say "well, in production we want to keep going no matter what, so we'll just ignore the asserts". Shachar
Re: D is dead
On 25/08/18 10:56, Walter Bright wrote:
On 8/24/2018 6:34 AM, Shachar Shemesh wrote:
No, unlike what I suggest, that doesn't work without carefully
reviewing every single place you put it to see whether the constructor
actually supports destructing a partially constructed object.
All D objects are default-initialized before the constructor sees it
(unlike C++). A destructor should be able to handle a
default-initialized object.
I'm not talking about a default initialized object. I'm talking about an
object where the constructor started running, but not to completion.
With that said, this statement is, I think, representative of the Lego
problem D has. All D objects? Really? Even this one?
struct A {
int a;
@disable this();
@disable init;
this(int number);
~this();
}
If you allow a feature to be disabled, you really need to keep in mind
that feature might be well and truly disabled.
Re: D is dead
On 24/08/18 13:43, nkm1 wrote: I think Walter was talking more about "scope (failure) destroy(this)" at the top of all your structs? I don't know if it has some gotchas, though (as I don't use RAII in D...). No, unlike what I suggest, that doesn't work without carefully reviewing every single place you put it to see whether the constructor actually supports destructing a partially constructed object. Shachar
Re: D is dead
On 23/08/18 15:03, Walter Bright wrote:
So you will excuse me, but I don't think this bug is being taken as
seriously as I think it should.
It is a serious problem. (There are workarounds available, like using
scope(failure).)
I don't think you understand how unworkable this workaround is.
struct A {
this(something);
~this();
}
struct B {
A a;
this(something) {
a = A(something);
// Safeguard against 14246
scope(failure) destroy(a);
// more code
}
}
struct C {
A a;
this(something) {
a = A(something);
// Also needs a safeguard against 14246
scope(failure) destroy(a);
// more code
}
}
struct D {
this(something) {
// Do nothing
}
}
struct E {
B b;
D d;
this(something) {
b = B(something);
// B doesn't even have an explicit destructor, but we are supposed
to look at its implementation and understand that it contains A, so:
scope(failure) destroy(b);
d = D(something);
// D doesn't even contain A, but it might one day, so:
scope(failure) destroy(d);
}
}
The chances of this scheme actually working without errors are, more or
less, zero.
Re: D is dead
On 24/08/18 10:43, FeepingCreature wrote: Have you tried to use the excellent Dustmite tool? It's never failed to reduce a bug for me. Dustmite might be excellent. I wouldn't know. It cannot swallow the Weka code base. Shachar
Re: D is dead
On 24/08/18 05:33, Jonathan M Davis wrote: Yeah. I've used RAII plenty in D without problems, but the fact remains that certain uses of it are very broken right now thanks to the constructor issue. I suspect that Shachar's as negative about this as he is in part because having RAII go wrong with the kind of low-level stuff Weka does would be a serious problem Yes. I will point out that I was never bit by this bug either. We found it while trying to figure out whether we want to start relying on destructors internally. The thing is, when a destructor doesn't run, this costs you a *lot* of time in finding out why. We actually have stuff that is downright weird as a result of not trusting destructors. That stuff is so weird, that for Mecca I essentially said I'm going to rely on them. Sadly, this means that this bug has become a bigger blocker than it was. (Having throwing destructors is even worse, it's just madness. Although it is allowed in C++, it doesn't actually work.) Yeah. We probably should have required that destructors be nothrow and force destructor failures to be treated as Errors. I'm sorry, but I'm not following your logic. If you're willing to have an error raised by a destructor abort the whole program, isn't the C++ solution preferable (abort the program only on double errors, which hardly ever happens)? Shachar
Re: D is dead
On 23/08/18 23:46, Walter Bright wrote: In my experience with debugging code, if drilling down to find the cause of the problem is not done, there is no way to conclude whether whether it is a compiler bug or a user bug. (Of course, compiler internal errors are always compiler bugs.) Drilling down and finding it to be a compiler problem also usually suggests a practical workaround for it. Consider the following line from the weka code base: _trustedData.opIndex(diskIdx) &= NotBitmap(toDistrust); That's strange. Why didn't Shachar just do? _trustedData[diskIdx] &= NotBitmap(toDistrust); Answer: Because the compiler decided that it needs to call _trustedData.opIndexAssign, and then failed the compilation because it has none. There is no way that that is a bug in the code. This is a compiler bug (maybe since fixed. This is fairly old code). So, where's the issue number, I hear you ask? There is none. This problem only happens inside the code base. Once I tried to isolate it, I couldn't reproduce. At this point I can either use the work-around I already have and (try to, obviously unsuccessfully) forget about it, file a bug report that will be (justifiably) ignored because no-one else can reproduce it, or spend an unknown amount of time (two days would probably be low-balling at this point) in trying to get this to reproduce on a watered down version of the code. Which would you pick? Shachar
Re: D is dead
On 23/08/18 20:57, bachmeier wrote: On Thursday, 23 August 2018 at 17:02:12 UTC, Shachar Shemesh wrote: If you can, feel free to contact me off-list, and I'm fairly sure we can get the budget for you to work on it. The same goes for anyone else on this list. I don't think Kenji will see your message, but he may be able to help given the right financial incentive. I have no idea who that is. Can you contact me off-list (either email or the DLang slack)? Shachar
Re: D is dead
On 23/08/18 20:52, bachmeier wrote: On Thursday, 23 August 2018 at 17:19:41 UTC, Ali wrote: On Thursday, 23 August 2018 at 16:22:54 UTC, Shachar Shemesh wrote: On 23/08/18 17:01, Steven Schveighoffer wrote: My main job is to develop for Weka, not develop D itself. Weka, at some point, made the strategic decision to use a non mainstream language I dont think Weka, have a choice, they have to invest in the development of D itself I hope a startup can choose D without having to do that. Otherwise D is not really a viable option for startups because they need to focus on survival rather than language development. This! Maybe Weka can afford it, but being all smug about it is a destructive attitude to have. I know that some of Weka's leadership are uncomfortable about the fact that we, almost by definition, are facing language related issues that no-one in the community has before us. Weka is in a good place, and is going in a good direction, but don't forget that we are up against giants, and are selling a product where 0.1% failure is considered the same as utter failure. Being able to trust the compiler was supposed to be a given. Yes, Weka is, at this point, committed. The next start-up isn't. Shachar
Re: D is dead
On 23/08/18 19:58, RhyS wrote: A quick question, if Weka did not have the current 300k backlog of code, what language of choice is more likely to be picked by the team at Weka? I don't know. Like I said, while the feeling that D has completely lost its way is fairly universal, the claim that picking D was a mistake is not. There are some unique D features we are using to great effect, and people have gone used to the weaknesses. I don't think, at this point, picking a different direction would have happened even if it were technically feasible. Shachar
Re: D is dead
On 23/08/18 18:35, Joakim wrote:
So your example of a fatal flaw is that D could be 100X faster at
compilation instead of just 10X than most every other native language
out there?! C'mon.
Have you tried Stephan's example yet?
static foreach(i; 0..16384) {}
Do give it a shot, tell me what you think of its compilation time.
On Thursday, 23 August 2018 at 09:09:40 UTC, Shachar Shemesh wrote:
* Features not playing well together.
Despite what Joakim seems to think, I've actually brought up an
example in this thread.
Despite what you seem to think, perhaps because you didn't read what I
wrote very closely, I noted your bugzilla link in my post you're quoting
and asked you if you really thought it was fatal.
Each problem on its own, maybe not. All together? Most definitely.
A language needs to be coherent. A programmer needs to be able to look
at code and know what the compiler will make of that code. The less that
can happen, the less useful the language is.
This is, in fact, precisely the criticism the D community levels against
C++.
Yes, this is a known problem with D: why do you think it's fatal?
See above.
By this rationale, C++ should be dead by now. Why do you think it's
fatal to D?
C++ does not suffer from this *kind* of complexity. For the most part,
C++'s complexity is feature centric. You use a feature, you need to
really learn that feature in order to get it to work.
D's complexity is waiting to pounce you behind street corners. You use a
feature, and all's well. And then, when you're doing stuff completely
tangential to your old code, things suddenly break.
You can avoid C++'s complexity by not using features. The same is not
true of D.
With that said, who said these problems don't affect C++? Had C++ not
being plagued by these problems, D (and Rust, and Go, and so on) would
probably never had been born. These are languages written with the
explicit hope of killing C++.
They do not seem like they are going to, but D lacks quite a few things
that C++ has going for it. To name a few:
* Large community
* excellent tooling
* large use base
* Critical bugs aren't being solved
People keep advertising D as supporting RAII. I'm sorry, but "supports
RAII" means "destructors are always run when the object is destroyed".
If the community (and in this case, this includes Walter) sees a bug
where that doesn't happen as not really a bug, then there is a deep
problem, at least, over-promising. Just say you don't support RAII and
destructors are unreliable and live with the consequences.
BTW: Python's destructors are unworkable, but they advertise it and
face the consequences. The D community is still claiming that D
supports RAII.
Maybe they're not critical to everyone else?
Maybe. Just don't lie to users.
How much time or money
exactly has Weka spent on getting this issue and other "critical" bugs
fixed?
Weka is paying prominent D developers as contractors. We've had David
Nadlinger and currently employ Johan Engelen. Both said they are cannot
fix this particular bug.
If you can, feel free to contact me off-list, and I'm fairly sure we can
get the budget for you to work on it. The same goes for anyone else on
this list.
We also contribute our own workarounds for D's shortcomings for everyone
to enjoy. This include DIP-1014 and Mecca, as well as the less obvious
upstreaming of bugs our contractors fix. This is beyond the fact that
our "fork" of the compiler is, itself, public
(https://github.com/weka-io/ldc). I think claiming that Weka is leaching
off the community is simply unwarranted.
It is fairly laughable for a company that raised $42 million to
complain that a bunch of unpaid volunteers aren't fixing bugs fast
enough for them:
First of all, and this is important, I do not speak for Weka. I can pass
recommendations, and I can sometime estimate in advance what will and
what will not be approved, but that's it. As *I* don't have 42 million
dollars, I find that particular criticism irrelevant (not to mention
downright incorrect, as pointed above).
With that said, I am not complaining about anything. I am stating the
situation as I see it. I understand it is uncomfortable to hear, and
thus the aggressiveness of your response. I can only point out the
obvious: you shaming me will not make me change my mind. At best, it
will make me not say it publicly.
Shachar
Re: D is dead
On 23/08/18 17:01, Steven Schveighoffer wrote: So interestingly, you are accepting the sockaddr by VALUE. Indeed. The reason is that I cannot accept them by reference, as then you wouldn't be able to pass lvalues in. Another controversial decision by D. Had that been C++, I'd definitely get a const ref instead. Shachar
Re: D is dead
On 23/08/18 17:01, Steven Schveighoffer wrote: If they are blocking your work, complain about them loudly, every day. But not filing them doesn't help anyone. The economics don't add up. If a bug is blocking my work, there are two options: 1. I work around it, at which point it is no longer blocking my work (grep mecca for DMDBUG) 2. Work actively (in our case, get Johan to do so) until it does not. Waiting for the community to fix a bug in D that is blocking Weka will get Weka kicked out of the market. There is no value proposition. We simply have to work faster than that. The problem is that once I do work around a bug, I no longer have the resources to continue complaining about it. I need to move on. My main job is to develop for Weka, not develop D itself. So telling me to keep filing them is simply a non-starter. I've got bugs that simply don't reproduce in watered down examples. I will not spend two days just to create a test case that demonstrates the bug outside the Weka code base. If nothing else, my boss won't allow me to spend that time. Oh, and our code base over 300,000 lines. Don't say "dustmite". It is unable to process the code. I'm not saying all bugs you file will be fixed, but all bugs you *don't* file will definitely not be fixed. So far, my experience is that it has about the same chances of being fixed both ways, and not filing takes less effort. I'm reminded of a friend of mine, who kept hoping to win the lottery despite never buying a ticket. His reasoning was that the chances of winning are not much changed by buying a ticket. Shachar
Re: D is dead
On 23/08/18 15:55, Steven Schveighoffer wrote: This whole thread seems very gloomy and final, but I feel like the tone does not match in my mind how D is progressing. "Every single one of the people [at Weka] rushing to defend D at the time has since come around." Seems like you all have decided to either ditch D internally, maybe moving forward, or accepted that Weka will fail eventually due to the choice of D? It sure reads that way. I'll clarify, in order to not create the wrong impression. No, Weka is neither ditching D, nor is it banking on failing. We're doing pretty good as a company, and D will not change that. What I did mean by that is that the enthusiasm from D has *greatly* diminished. Many (but not all) developer will definitely not choose D for our next project, should the choice be ours to make. Like I said in my original post, it is not even in consensus whether picking D to begin with had been a mistake. Some think it was, some think, even in hind sight and after being more or less disillusioned, that it was still better than picking another language. As such, it is not even universally true that engineers at Weka regret going with D. I think Mecca is a great library, and I'm very proud of writing it. There are certainly aspects of it that would not be possible (or, at least, highly impractical) in any other language I know. With that said, it is also true that there are aspects of Mecca where D was holding me back from doing stuff I knew I could do much easier in C++. There were also areas where run-time bugs were discovered during integration with the main Weka code base (integration that is still ongoing) that were difficult to diagnose at best (and yes, I have it on my todo list to submit a PR to fix some aspects of some of them). Example of something I couldn't/wouldn't do in any other language: the second form of spawnFiber: https://weka-io.github.io/mecca/docs/mecca/reactor/Reactor.spawnFiber.html The fact that the arguments for spawnFiber are the arguments for F, and that's verified by the compiler, no casts, no inference, no code bloat, is huge. I am sorely going to miss it at my next project (which, like I said, will not be written in D). On the other hand, look at ConnectedSocket.connect: https://weka-io.github.io/mecca/docs/mecca/reactor/io/fd/ConnectedSocket.connect.html Why do I need two forms? What good is that? Why is the second form a template? Answer: Because in D, structs can't inherit, and I cannot define an implicit cast. What I'd really want to do is to have SockAddrIPv4 be implicitly castable to SockAddr, so that I can pass a SockAddrIPv4 to any function that expects SockAddr. Except what I'd _really_ like to do is for them to be the same thing. I'd like inheritance. Except I can't do that for structs, and if I defined SockAddr as a class, I'd mandate allocating it on the GC, violating the whole point behind writing Mecca to begin with. To summarize: Weka isn't ditching D, and people aren't even particularly angry about it. It has problems, and we've learned to live with them, and that's that. The general consensus, however, is that these problems will not be resolved (we used to file bugs in Bugzilla. We stopped doing that because we saw nothing happens with them), and as far as the future of the language goes, that's bad news. Shachar
Re: D is dead
On 23/08/18 14:02, Walter Bright wrote: On 8/23/2018 2:09 AM, Shachar Shemesh wrote: functions may be @safe, nothrow, @nogc, pure. If it's a method it might also be const/inout/immutable, static. The number of libraries that support all combinations is exactly zero (e.g. - when passing a delegate in). If, for example, a library functions allocates with the gc, then it can't work with @nogc code. But still, fair enough - if there are combinations which should work, but do not, please submit bug reports. If you have already, is there a list of them? None of that continues to work once delegates are involved. I am yet to see a library that can accept a delegate and correctly create a function around it that matches its attributes. And yes, I do include Mecca in this. I tried. It is too difficult to get right, so I gave up. Attribute inference was supposed to solve this, but attribute inference is completely broken with separate compilation. * Language complexity Raise your hand if you know how a class with both opApply and the get/next/end functions behaves when you pass it to foreach. > How about a struct? I presume you meant empty/front/popFront. Indeed. This is in the language spec: How many people know that without resorting to the specs. Does it matter if it allows copying or not? For the preference for opApply, no. But it does for empty/front/popFront, which is exactly my point. * Critical bugs aren't being solved People keep advertising D as supporting RAII. I'm sorry, but "supports RAII" means "destructors are always run when the object is destroyed". If the community (and in this case, this includes Walter) sees a bug where that doesn't happen as not really a bug, then there is a deep problem, at least, over-promising. Just say you don't support RAII and destructors are unreliable and live with the consequences. If you're referring to #14246, I posted a PR for it. I don't see how that is pretending it isn't a problem. It is. When I first reported this, about 3 and a half years ago, the forum explained to me that this is working as expected. #14246 was over 2 years old by the time you posted the PR. Once posted, however, it broke (I'm not sure why it broke, but did notice it deliberately would not work with @disable init structs. See my interoperability comment from above). Since then (over a year), no progress has been made except to revert the changelog that claims it was resolved. When I talked to you about it at the last DConf, I got a reply that could be largely summarized as "yeah, we should probably do some flow analysis, sometimes". The only time I got anyone to take this problem seriously was when someone on the forum would claim that D supports RAII, to which I tend to reply with "no, it doesn't". So you will excuse me, but I don't think this bug is being taken as seriously as I think it should. I get it, it is not a simple bug to solve. It is an unfortunate truth that some important bugs are not going to be easy. C++ went a different path with this (strictly setting when members are initialized). I get that this is a very unpopular feature of C++, and I can see why, but seeing how D struggles to resolve this issue, I can't say it is a mistake on C++'s behalf. Shachar
Re: D is dead
On 23/08/18 09:58, Joakim wrote: Because you've not listed any here, which makes you no better than some noob Here's one: the forum does not respond well to criticism. Here's an incredibly partial list: * Features not playing well together. Despite what Joakim seems to think, I've actually brought up an example in this thread. Here is another one: functions may be @safe, nothrow, @nogc, pure. If it's a method it might also be const/inout/immutable, static. The number of libraries that support all combinations is exactly zero (e.g. - when passing a delegate in). * Language complexity Raise your hand if you know how a class with both opApply and the get/next/end functions behaves when you pass it to foreach. How about a struct? Does it matter if it allows copying or not? The language was built because C++ was deemed too complex! Please see the thread about lazy [1] for a case where a question actually has an answer, but nobody seems to know it (and the person who does know it is hard pressed to explain the nuance that triggers this). * Critical bugs aren't being solved People keep advertising D as supporting RAII. I'm sorry, but "supports RAII" means "destructors are always run when the object is destroyed". If the community (and in this case, this includes Walter) sees a bug where that doesn't happen as not really a bug, then there is a deep problem, at least, over-promising. Just say you don't support RAII and destructors are unreliable and live with the consequences. BTW: Python's destructors are unworkable, but they advertise it and face the consequences. The D community is still claiming that D supports RAII. * The community Oh boy. Someone who carries weight needs to step in when the forum is trying to squash down on criticism. For Mecca, I'm able to do that [2], but for D, this simply doesn't happen. -- This is a partial list, but it should give you enough to not accusing me of making baseless accusations. The simple point of the matter is that anyone who's been following what I write should already be familiar with all of the above. The main thing for me, however, is how poorly the different D features fit together (my first point above). The language simply does not feel like it's composed of building blocks I can use to assemble whatever I want. It's like a Lego set where you're not allowed to place a red brick over a white brick if there is a blue brick somewhere in your building. Shachar 1 - https://forum.dlang.org/thread/pjp2ef$310c$1...@digitalmars.com 2 - https://forum.dlang.org/post/pctsgk$182l$1...@digitalmars.com
Re: D is dead
On 23/08/18 09:17, Jacob Carlborg wrote: On Thursday, 23 August 2018 at 05:37:12 UTC, Shachar Shemesh wrote: One that hurt me lately was a way to pass a scoped lazy argument (i.e. - to specify that the implicit delegate need not allocate its frame, because it is not used outside the function call). I don't see why we just can't add support for scoped lazy parameters. It's already in the language just with a different syntax (delegates). That would probably be an easy fix (last famous words :)). I guess it would be better if it could be inferred. -- /Jacob Carlborg Here's the interesting question, though: is this *going* to happen? We've known about this problem for ages now. No movement. Some of the other problems are considerably less easy to fix. Examples: A struct may be @disabled this(this), @disable this() and/or @disable init. Can you say that libraries.. Actually, strike that. Can you say that the *standard* libraries work with all 8 combinations? Shachar
Re: D is dead
On 23/08/18 09:04, Mike Franklin wrote: On Thursday, 23 August 2018 at 03:50:44 UTC, Shachar Shemesh wrote: And it's not just Weka. I've had a chance to talk in private to some other developers. Quite a lot have serious, fundamental issues with the language. You will notice none of them speaks up on this thread. They don't see the point. No technical project is born great. If you want a technical project to be great, the people working on it have to focus on its *flaws*. The D's community just doesn't do that. To sum it up: fatal flaws + no path to fixing + no push from the community = inevitable eventual death. The D Foundation has an Open Collective page (https://opencollective.com/dlang) with a $12,000 annual "Corporate Bronze" option that includes 3 priority bug fixes per month. Is that not a worthwhile investment for Weka or other organizations invested in D to help address some of the problems you're encountering? If not, is there an option that would be? I will definitely pass it on. Shachar
Re: D is dead
On 23/08/18 08:20, Nicholas Wilson wrote: On Thursday, 23 August 2018 at 03:50:44 UTC, Shachar Shemesh wrote: No, no and no. I was holding out on replying to this thread to see how the community would react. The vibe I'm getting, however, is that the people who are seeing D's problems have given up on affecting change. It is no secret that when I joined Weka, I was a sole D detractor among a company quite enamored with the language. I used to have quite heated water cooler debates about that point of view. Every single one of the people rushing to defend D at the time has since come around. There is still some debate on whether, points vs. counter points, choosing D was a good idea, but the overwhelming consensus inside Weka today is that D has *fatal* flaws and no path to fixing them. A list, please? Now that I actually have time to fix things, I intend to do so. Let's start with this one: https://issues.dlang.org/show_bug.cgi?id=14246#c6 The problems I'm talking about are not easily fixable. They stem from features not playing well together. One that hurt me lately was a way to pass a scoped lazy argument (i.e. - to specify that the implicit delegate need not allocate its frame, because it is not used outside the function call). Shachar
Re: D is dead
On 23/08/18 07:35, Dukc wrote: On Thursday, 23 August 2018 at 03:50:44 UTC, Shachar Shemesh wrote: Every single one of the people rushing to defend D at the time has since come around. There is still some debate on whether, points vs. counter points, choosing D was a good idea, but the overwhelming consensus inside Weka today is that D has *fatal* flaws and no path to fixing them. And by "fatal", I mean literally flaws that are likely to literally kill the language. How so? If he's right with those issues, they can definitely prevent D from becoming mainstream, but how would they kill D? I mean, will not there always be some existing users who have no need or wish to move on? Maintaining a language requires a lot of work. The "payback" for that work comes from people who actually use that work. If the D community starts to contract, it will become more and more difficult to find people willing to work on D's core features, which will lead to stagnation which is the same as death. But, again, it is interesting to see what you took from my mail. I'd be much more worried about the fact that it is working with D that caused people to recognize the problems as fundamental than about what "death" means in this context. Shachar
D is dead (was: Dicebot on leaving D: It is anarchy driven development in all its glory.)
On 22/08/18 21:34, Ali wrote: On Wednesday, 22 August 2018 at 17:42:56 UTC, Joakim wrote: Pretty positive overall, and the negatives he mentions are fairly obvious to anyone paying attention. Yea, I agree, the negatives are not really negative Walter not matter how smart he is, he is one man who can work on the so many things at the same time Its a chicken and egg situation, D needs more core contributors, and to get more contributors it needs more users, and to get more users it need more core contributors No, no and no. I was holding out on replying to this thread to see how the community would react. The vibe I'm getting, however, is that the people who are seeing D's problems have given up on affecting change. It is no secret that when I joined Weka, I was a sole D detractor among a company quite enamored with the language. I used to have quite heated water cooler debates about that point of view. Every single one of the people rushing to defend D at the time has since come around. There is still some debate on whether, points vs. counter points, choosing D was a good idea, but the overwhelming consensus inside Weka today is that D has *fatal* flaws and no path to fixing them. And by "fatal", I mean literally flaws that are likely to literally kill the language. And the thing that brought them around is not my power of persuasion. The thing that brought them around was spending a couple of years working with the language on an every-day basis. And you will notice this in the way Weka employees talk on this forum: except me, they all disappeared. You used to see Idan, Tomer and Eyal post here. Where are they? This forum is hostile to criticism, and generally tries to keep everyone using D the same way. If you're cutting edge D, the forum is almost no help at all. Consensus among former posters here is that it is generally a waste of time, so almost everyone left, and those who didn't, stopped posting. And it's not just Weka. I've had a chance to talk in private to some other developers. Quite a lot have serious, fundamental issues with the language. You will notice none of them speaks up on this thread. They don't see the point. No technical project is born great. If you want a technical project to be great, the people working on it have to focus on its *flaws*. The D's community just doesn't do that. To sum it up: fatal flaws + no path to fixing + no push from the community = inevitable eventual death. With great regrets, Shachar
Re: D, Parasail, Pascal, and Rust vs The Steelman
On 22/03/18 16:45, Radu wrote: On Thursday, 22 March 2018 at 11:58:02 UTC, Shachar Shemesh wrote: On 22/03/18 12:28, Meta wrote: On Wednesday, 21 March 2018 at 12:52:19 UTC, Paulo Pinto wrote: [...] "The central failure of the language is the myopic focus on the affine typing solution to heap allocation and thread safety. The creators do not seem to realise that other solutions already exist, and that dynamic memory allocation is not the only safety issue a programmer has to cope with." Interesting that the author's criticism of Rust lines up very closely with Andrei's. Spoken on the forum for a language that has still not managed to make sure that a destructor actually gets called every time an object is destroyed. Shachar Shaming this one? https://issues.dlang.org/show_bug.cgi?id=14246#c6 Indeed, that sucks big time! There is hope. We finally merged a PR that removes the changelog that erroneously claimed this issue was resolved. https://github.com/dlang/dlang.org/commit/45ca5e35d3de824e104c3049083eb23fa03775c5 Progress!! Shachar
Re: Is there any hope for "lazy" and @nogc?
On 01/08/18 17:13, Steven Schveighoffer wrote: The lazy variadic thing is a distinction between specifying variadic lazy parameters and a lazy variadic array. I have now read that sentence 4 times, and I still have no idea what it means. Can you give examples of both? Shachar
Re: Is there any hope for "lazy" and @nogc?
On 01/08/18 17:13, Steven Schveighoffer wrote:
On 8/1/18 3:59 AM, Shachar Shemesh wrote:
Thank you! Finally!
Let me just state, for the record, that having *yet another* syntax
special case is just appalling.
The lazy variadic thing is a distinction between specifying variadic
lazy parameters and a lazy variadic array. The distinction is so
miniscule, but necessary to have a disambiguous syntax.
But I had actually thought for a while, that you could simply specify a
delegate, and it would be treated as a lazy parameter, which would
probably solve your problem. I really think this syntax should be
available.
With that said, I was hoping that specifying it explicitly as a
delegate would allow me to scope it. Apparently, that doesn't work :-(
I guess you mean you can't scope the delegates? I'm surprised if that
doesn't work.
-Steve
import std.string;
alias Dg = string delegate() @nogc nothrow;
void myAssert(bool cond, scope Dg[1] msg_dg ...) @nogc nothrow
{
import core.stdc.stdio;
if (!cond)
{
string msg = msg_dg[0]();
printf("%*s\n", msg.length, msg.ptr);
}
}
void main() @nogc {
string msg = "Hello";
myAssert(true, msg); // <- errors on this line
}
It errors out: complains it needs to allocate main's frame on the GC,
but main is @nogc. The same happens if I move the scope to the alias.
Re: Is there any hope for "lazy" and @nogc?
Thank you! Finally!
Let me just state, for the record, that having *yet another* syntax
special case is just appalling.
With that said, I was hoping that specifying it explicitly as a delegate
would allow me to scope it. Apparently, that doesn't work :-(
Shachar
On 31/07/18 23:03, ag0aep6g wrote:
On 07/31/2018 09:17 AM, Shachar Shemesh wrote:
I'm trying to figure out what's the signature of the built-in assert.
It does not seem that I can define a similar function myself.
Looks like you can do it with a "lazy variadic function" [1], but it's
not pretty:
alias Dg = string delegate() @nogc nothrow;
void myAssert(bool cond, Dg[1] msg_dg ...) @nogc nothrow
{
import core.stdc.stdio;
if (!cond)
{
string msg = msg_dg[0]();
printf("%*s\n", msg.length, msg.ptr);
}
}
[1] https://dlang.org/spec/function.html#lazy_variadic_functions
Re: Is there any hope for "lazy" and @nogc?
On 31/07/18 10:29, Mike Franklin wrote: Please clarify if I'm missing the point. You are. I want something along the lines of: assertEQ(a, b, "a and b are not equal"); When run, it would issue an assert that says: Assertion failed: 3!=7: a and b are not equal Hooking it later is not an option. I am actually interested in a different assert like function. Whether for asserts or otherwise, the "lazy" feature is completely incompatible with function attributes, and that's not good. Shachar
Is there any hope for "lazy" and @nogc?
I'm trying to figure out what's the signature of the built-in assert. It does not seem that I can define a similar function myself. First attempt: void myAssert(bool cond, string msg) @nogc nothrow; No, because msg gets evaluated unconditionally. void myAssert(bool cond, lazy string msg) @nogc nothrow; test.d(8): Error: @nogc function test.myAssert cannot call non-@nogc delegate msg void myAssert(bool cond, lazy string msg @nogc nothrow ) @nogc nothrow; test.d(4): Error: found @ when expecting ) test.d(4): Error: semicolon expected following function declaration test.d(4): Error: no identifier for declarator nogc test.d(4): Error: declaration expected, not ) test.d(9): Error: unrecognized declaration Templates to the rescue!!! void myAssert(STR)(bool cond, lazy STR msg ); test.d(14): Error: @nogc function D main cannot call non-@nogc function test.myAssert!string.myAssert Help?? Shachar
Re: "catch" not catching the correct exception
On 26/07/18 10:31, Seb wrote:
On Thursday, 26 July 2018 at 05:52:51 UTC, Shachar Shemesh wrote:
At which point I'm stuck. I don't know how D's catch matching works,
so I don't know where to continue looking.
https://github.com/dlang/druntime/blob/master/src/rt/dwarfeh.d
You still use druntime, right?
Yes.
/
* Called when fibers switch contexts.
* Params:
* newContext = stack to switch to
* Returns:
* previous value of stack
*/
extern(C) void* _d_eh_swapContextDwarf(void* newContext) nothrow @nogc
{
auto old = ExceptionHeader.stack;
ExceptionHeader.stack = cast(ExceptionHeader*)newContext;
return old;
}
Mecca doesn't call that. Should it? Can that be the problem?
Shachar
Re: "catch" not catching the correct exception
On 26/07/18 09:22, rikki cattermole wrote: Hmm, sounds like the vtable and hence TypeInfo part of the reference is getting corrupted. Have you checked that the type matches where you throw it? Is that what "classinfo" returns? Because if so, it's printed by the logs I pasted in my question, and is correct (and, in any case, shows that Exception is the parent). Shachar
"catch" not catching the correct exception
Under mecca, we're using GC free exceptions.
I have code that uses mkEx
(https://github.com/weka-io/mecca/blob/master/src/mecca/lib/exception.d#L307).
Essentially, it constructs an exception in place inside a fiber-local
buffer. The construction code seems correct, and the parent seems set
correctly (Exception).
When thrown, however, a catch for Exception does not catch it. It is
caught if we try to catch Throwable, and also if we try to catch
FiberInterrupt (a class deriving directly from Throwable).
Thinking the buffer might have gotten corrupted by one of the scope
exits, I've added the following code:
catch (Exception ex) {
// Code that should have run goes here
}
catch(Throwable ex) {
META!"XXX NONONONONO caught %s of type %s son of
%s"(cast(void*)ex, ex.classinfo.name, ex.classinfo.base.name);
LOG_EXCEPTION(ex);
throw ex;
}
When run, I get the following log:
META XXX NONONONONO caught 0x7F1C616F39B0 of type
weka.bucket.exceptions.NotBucketLeader son of object.Exception
It seems immediately obvious that the problem is not with using a static
buffer for the exception (at least, it does not seem to be corrupted),
but something else. Somehow, the catch matching does not work.
At which point I'm stuck. I don't know how D's catch matching works, so
I don't know where to continue looking.
The problem does not happen consistently, but does happen frequently
enough for me to be able to reproduce it when needed.
I need suggestions on how to debug this.
Thank you,
Shachar
Constructing a class in-place
Forget the "why" for the moment.
T construct(T, ARGS...)(ARGS args) if( is(T==class) ) {
auto buffer = new ubyte[__traits(classInstanceSize, T)];
T cls = cast(T)buffer.ptr;
// Is this really the best way to do this?
buffer[] = cast(ubyte[])typeid(T).initializer()[];
cls.__ctor(args);
return cls;
}
My question is this: Is this the correct way to do it? There are steps
here that seem kinda arbitrary, to say the least.
I am looking for something akin to C++'s placement new.
Thank you,
Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 17/07/18 16:29, aliak00 wrote: A postblit on a class issues a compiler error. And an identity opAssign on a class also issues a compiler error. So I'm not sure how this would be different. And the page In https://dlang.org/spec/operatoroverloading.html also explicitly mentions differences between ops on classes or structs. That's actually a good argument. It should definitely be handled the same way the corresponding opAssign is handled. Thanks, Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 14/07/18 15:56, Johan Engelen wrote: First off: I am trying to wear a strict language lawyer hat. D spec is already very much ill specced which is _very_ problematic for language and compiler development. I am not attacking the proposal in order to kill it. I am merely commenting on points that I feel should be improved. Wouldn't these comments be better suited for the language spec's PR, then? I'm asking seriously. There is nothing in this DIP (or just about any other one) that can go, unmodified, into the specs. If that's the DIP's purpose, then so be it. If not, then I don't see the value. OK, so make _that_ explicit. I think there is value in prescribing the precise order of moves (like for construction of members), such that reasoning about code becomes easier. I disagree. The member variables at the point of the move are all: 1. Fully initialized. 2. Either move agnostic or know how to handle their own move 3. Oblivious to one another Obviously, you might wish to spite me by creating a case where any one of the above is not true, but the compiler has every right to assume all three points above are true. In your hypothetical spite case, your best course of action is to leave the members with no opPostMove at all, and handle their move in the containing struct's opPostMove, in which case you are fully defined. Assuming all three are correct, the order in which they are moved makes no difference. If you want the same semantic effect (as I wrote above), then the text should say that the ordering is relaxed. I have no objection to explicitly stating that exact move order of members is undefined. Why "SHOULD" and not "MUST"? Precisely for the reason you stated above. So that if you want to do something else, you may. Why is that freedom needed? Because compiler implementers might have a good reason to do something besides this. For example, a compiler writer might choose to place the actual moving code inside __move_post_blt, and I don't think we should stop her. The freedom is already provided by user-defined opPostMove? Different audiences. opPostMove serves the D programmer, __move_post_blt serves the compiler. I think the language spec doesn't say when a "move" is performed? I think Walter sees that as an advantage, but I'm not sure. Either way, the current language spec says structs must have semantics that remain correct even if the struct suddenly changes the memory address it resides in. The specs + DIP 1014 say that the above is true, or the struct must supply an opPostMove that fixes the semantics post-move. In both cases, *when* the move takes place is irrelevant. Or is it enough to define what a "move" is ? (didn't check but I guess the DIP already explains that) Only implicitly. (D's "move" is different from C++'s right? Yes. D's move after exiting a struct's constructor doesn't lead to a destructor call, but D's std.algorithm.mutation.move _does_ call the destructor of the moved source object.) Depends on which version of move you're referring to. For example, moveEmplace does not. I think the correct way to phrase this is to say that D's move *never* calls a destructor, but if the move's destination had a valid object in it, that one gets destructed. In a way, C++'s move is the same, except the actual moving of the data from the source location to the destination one is up to the programmer, and accordingly, so is destructing. Since, logically, a C++ move operator always copies, it also has to destruct the source. Technically, however, it doesn't always. A move assignment operator typically just swaps the content of the structs (i.e. - moves the source to the destination and the destination to the source), and lets the usual rvalue elimination code destruct it. Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 12/07/18 04:17, Jonathan M Davis wrote:> I'm also> not sure if going to copy constructors means that we should do something> different with this. It don't think that it's affected by it, but I could be> missing something. I actually had that very same concern myself. Andrei does not seem to share it (I talked to him about it during DConf, and he even mentioned it in his talk). He seems to think the two are completely independent. Like I said, I'm not sure I completely agree. Some of the problems postblit have will also affect us here (specifically, what happens if you want to override a member's behavior, specifically when @disabled. What tipped the scale for me was this: This DIP is relatively simple. It requires a few changes to all three (compiler, run time and phobos), but small changes to all three. Best of all, this change does not introduce what Andrei called "magic". The changes are easily lowered to existing D code. Switching to a move-constructor type implementation will make all of those advantages disappear in a puff of bits. However, I don't agree that opPostMove needs to be nothrow on the basis that it might be confusing if it's not. The problem here is that D moves *everywhere*, and it is often quite impossible to make it not move (believe me, I've tried). With that said, downgrading this to a very hearty recommendation instead of a requirement is fine by me. Also, as soon as we discuss overriding what happens when an object is moved, that opens up the question of whether it should be allowed to be @disabled. What happens when someone does @disable opPostMove(); What happens today is that if you actually try to move the struct, you will get a compilation error. IMHO, this is the behavior we actually want to keep. I can actually see some uses for such a step - if you simply cannot have that struct move, a compilation error is preferable to things breaking. With that said, the moves are so integral to D's code generation that a struct with opPostMove @disabled would not be very useful. I am of the opinion that the programmer can make up their own mind on what to do about it. However, I would argue that if we're going to take the step of allowing the programmer to hook into the move process to do custom stuff that we should also allow it to be outright disabled - the use case that comes to mind at the moment being stuff like mutexes where the object can't safely be moved, because it's not only not thread-safe, but it's probably not possible with the mutex's C API, since you handed a pointer off to it and have no control over what it does with it. Like I said above, I completely agree. Of course, that also opens the question of what should happen with a shared opPostMove, since presumably a mutex would be in a shared object, and that would then require that we finish working out shared - though I'd argue that we would have to disallow moves on a shared object in order to be thread-safe anyway. That one is actually handled by the DIP. opPostMove is called by the compiler when it has just moved the object. This means that, at least as far as the compiler is concerned, the object has no external references to it (or it couldn't move it). If a struct's pointer is shared, and there are external pointers to update, it is up to the programmer to decide how to handle it (and, yes, @disable might be the only safe option). By no means do I think that this DIP should be contingent on anything to do with disallowing moving, but I do think that it if we're going to allow mucking around with moves like this rather than simply leaving it up to the compiler that we should seriously consider allowing it to be disabled - I disagree. I think we should allow it to be disabled whether this DIP goes in or not. On the contrary: this DIP would allow cases to be handled where today they are simply not safe, no matter what you do. In other words, the cases where you'd want to disable move are only reduced by adopting this DIP. Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 29/06/18 15:35, aliak wrote: On Wednesday, 27 June 2018 at 07:24:05 UTC, Mike Parker wrote: On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote: Thanks in advance for your participation. For those of you using the NNTP or mailing list interfaces, this is the thread to respond in. Thanks! Alo! This is great! Just a clarification about the last paragraph phrasing The last line: "We can further reduce this problem by calling the function opPostMove." seemed to imply that an alternate name to opPostMove would be mentioned, but am I correct in understanding that it is just saying that "naming this second function as op* will keep code breakage to a minimum" ? This is a left over from a previous draft, where the operator was called "opMove". It should be removed. Also, what should happen if someone defines an opPostMove for a class. Compile error or? Should something about that be mentioned? I think nothing should happen. The function would be ignored, just like it is today. I am open to hear other ideas, however. I'm not sure whether it should be explicitly mentioned or not. Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 11/07/18 20:04, Johan Engelen wrote: On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote: DIP 1014, "Hooking D's struct move semantics", is now ready for final review. after quick read: (would be much easier to do inline commenting, but it appears that's not supported) ### Section "Code emitted by the compiler on move" Dangerous to talk about what "code is emitted" by the compiler. I think this DIP doesn't need that, and semantics is enough. "the compiler MUST call " should be reworded, because an _actual_ call to the function should not be mandatory, because it would limit the optimizer in eliding it or inlining it (note that it will be hard to _prevent_ the optimizer from eliding/inlining the call as currently specified by the DIP). I don't draw the same conclusions from the text as you. I'm perfectly fine with specifying that nothing here is mandatory if the compiler ensures that *the end effect* is as if it happened. AFAIK, C++ has a standing order to that effect, and it greatly simplifies documenting what you want to do. ### "__move_post_blt SHOULD be defined in a manner that is compatible" What does "compatible" mean? "Has the same effect as". It's a little as if you're complaining about something not being explicit in one section, and again about that same thing being explicit in the next. Precisely why such standing order would be a good idea. Some things should be made more explicit, such as the order of calls for compound structs. I don't think it makes sense to specify the order, except to say that member's opPostMove must be called before the instance's opPostMove (which the code already says). Why "SHOULD" and not "MUST"? Precisely for the reason you stated above. So that if you want to do something else, you may. ### "This MUST return true iff a struct or any of its members have an opPostMove defined." Doesn't cover struct A containing struct B containing struct C with opPostMove. Reword e.g. into: "hasElaborateMove!S MUST return true iff `S` has an `opPostMove` defined or if hasElaborateMove!X is true for any member of S of type X. Yes, I'm sorry. I worded the spec for humans. I can suggest a recursive definition: hasElaborateMove for a struct MUST return true iff at least one of the following is true: * The struct has opPostMove defined * hasElaborateMove returns true for at least one of the struct's members. I'm fine with this definition, and it resolves your concern, but I think it is less readable. You are free to suggest a better way of phrasing this. ### What is lacking is a clear list of exactly in which cases `opPostMove` will be called (needed for user-facing documentation of the function). I don't think I understand this point. Can you suggest what that list might contain? Thank you, Shachar
Re: Compilation is taking a ton of memory
On 28/06/18 01:46, Steven Schveighoffer wrote: This has been a thorn in many sides for a long time. I remember Weka.io's Liran talking about how they required an INSANE amount of time/memory to build their system in dconf 2015 maybe? But things have gotten a bit better since then. I think at some point there will be a reckoning where it has to be fixed. Fast compile-time is cool, but inifinte compile time (i.e. it never finishes because the OOM killer destroys your process) is not acceptable. No, we just use bigger build servers: $ cat /proc/cpuinfo ... processor : 127 vendor_id : GenuineIntel cpu family : 6 model : 63 model name : Intel(R) Xeon(R) CPU E7-8880 v3 @ 2.30GHz stepping: 4 microcode : 0x11 cpu MHz : 2300.072 cache size : 46080 KB physical id : 3 siblings: 32 core id : 15 cpu cores : 16 apicid : 223 initial apicid : 223 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq monitor est ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm ida xsaveopt fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm bogomips: 4662.97 clflush size: 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual power management: ubuntu@ip-172-31-14-32:~$ cat /proc/meminfo MemTotal: 2014743200 kB My laptop has 32GB of ram. If I do single threaded compilation and allocate enough swap, I can *barely* squeeze through a debug build compilation of the node. Shachar
Re: D code obfuscator
On 14/06/18 13:39, DigitalDesigns wrote: Wait? Are you sure you are not kidding? Do you want another shot? No, I'm fine. Thank you. I am not out here to convert anyone. If you want to believe the magic of obfuscation, go right ahead. You can probably even leverage D's CTFE to do it inside the compiler while not making your program too much uglier. Something like replacing definitions with: mixin Obfuscate!(int, "variableName"); and use with: Deobfuscate!"variableName"; Shouldn't be too difficult to create. Shachar
Re: D code obfuscator
On 14/06/18 08:21, DigitalDesigns wrote: On Thursday, 14 June 2018 at 02:13:58 UTC, Shachar Shemesh wrote: With that said, what you're trying to achieve is probably not a good idea anyways. With very few exceptions(1), reverse-engineering code to figure out what it does is not considerably more difficult than using the source, even when none of the identifiers leak at all. Certain aspects of creating attacks are even easier with good rev-eng tools than in source form. Shachar Just one question! Are you kidding me? First of all, run your program under strace. For a surprising percentage of the programs that should give you a fairly good idea of what the program is doing. ltrace goes further, but it can be easily defeated by statically linking, so probably irrelevant for our current discussion. Next, try loading your program in Ida Pro (https://www.hex-rays.com/products/ida/index.shtml). You will notice that program flow practically jumps out at you with no further work on your part. Other tricks require a little more knowledge, but are still exceedingly effective. In a demonstration I saw in 2002, Halvar Flake showed how he uses Ida to graph the branches, and then use a tool he built to place breakpoints on the branch points. Next he started feeding inputs to the program, and colored the graph where the input sent the code. He used that to find the correct input that would bring the code path to the line he thought might be vulnerable. If I had to do this trick today for *my own* programs, I'd still use Ida and the compiled code. So, no, I was not kidding. Not even close. Shachar
Re: D code obfuscator
On 14/06/18 03:01, DigitalDesigns wrote: Is there an obfuscator for D that at least renames identifiers? This is because sometimes they leak from various processes and could be potential sources of attack. It would be a tool that probably just replaces their values with, say their hash + something else and done pre release build. Ideally it would be able to compile with dmd and all in memory or use temp storage without file issues. It can't modify the code directly because then that would be permanent. I highly doubt it. You see, with introspection and run-time execution, writing such a tool is equivalent to solving the halting problem. You simply do not know what you're affecting. There are some cases where you might know at x% certainty that it's okay to rename. Someone might do a best-effort based tool. I'm not aware of one. With that said, what you're trying to achieve is probably not a good idea anyways. With very few exceptions(1), reverse-engineering code to figure out what it does is not considerably more difficult than using the source, even when none of the identifiers leak at all. Certain aspects of creating attacks are even easier with good rev-eng tools than in source form. Shachar 1- One notable exception is complex algorithmic code. I will point out that those are difficult to figure out from source code too, and it usually takes very good documentation to be able to do so, so even there I'm not sure my original statement doesn't hold.
Re: Can anyone explain this?
On 05/06/18 11:26, Nicholas Wilson wrote:
writeln("Exception"); // Why is this not caught? I've no
idea
That's the part I was referring to.
Can anyone explain this?
I set up to find out what happens if the assert string throws. I have to
admit I did not expect the result:
$ cat test.d
import std.stdio;
import core.exception;
void main() {
scope(failure) {
writeln("Not gonna happen");
}
try {
static string throwingFunc() {
throw new Exception("An exception");
}
assert(0==1, throwingFunc());
} catch(Exception ex) {
writeln("Exception");
} catch(AssertError ex) {
writeln("Assert");
}
}
$ ldc2 --version
LDC - the LLVM D compiler (1.8.0):
based on DMD v2.078.3 and LLVM 5.0.1
...
$ ./test
Not gonna happen
object.Exception@test.d(11): An exception
??:? [0x3728941e]
??:? [0x372903aa]
??:? [0x3727b15c]
??:? [0x3724991d]
??:? [0x372496c9]
??:? [0x3727aecf]
??:? [0x3727addb]
??:? [0x3724a124]
??:? __libc_start_main [0xed8b01c0]
??:? [0x372495c9]
$ dmd --version
DMD64 D Compiler v2.080.0
Copyright (C) 1999-2018 by The D Language Foundation, All Rights
Reserved written by Walter Bright
$ ./test
Not gonna happen
object.Exception@test.d(11): An exception
??:? pure @safe immutable(char)[] test.main().throwingFunc() [0xe9b1c2b3]
??:? _Dmain [0xe9b1c1ad]
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 18/05/18 22:57, kinke wrote: I checked, and the reason is that D and C++ use a different ABI wrt. by-value passing of non-POD arguments. C++ indeed passes a reference to a caller-allocated rvalue, not just on Win64; that makes it trivial, as there are no moves across call boundaries. But your proposal may imply changing the D ABI accordingly. That seems to be the case. Assuming https://dlang.org/spec/abi.html is the ABI you refer to, there is very little in way of argument calling there: https://dlang.org/spec/abi.html#function_calling_conventions " The extern (C) and extern (D) calling convention matches the C calling convention used by the supported C compiler on the host system. Except that the extern (D) calling convention for Windows x86 is described here. " So the current state is, in essence, that in C everything is PoD, and that's why that's also the case in D. Yes, something will need to change there. Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 22:29, Manu wrote:
This is great!
I've wanted this on numerous occasions when interacting with C++ code.
This will make interaction more complete.
Within self-contained D code, I have avoided self-pointers by using
self-offsets instead in the past (a bit hack-ey). But this nicely
tidies up one more little rough-edge in the language.
:+1:
Following Andrei's advice, I've actually started writing a couple of
examples to illustrate why this is needed.
The first was this:
struct S {
static int global;
int local;
Something selector; // Decides whether we want local or global.
}
Let's further assume that we have an array of S instances with random
uniform distribution between local and global.
Obviously, Something can be an enum or a boolean. If it is, however,
then we have to perform a condition to select the correct value. The
problem with conditionals is that if the CPU misses a guess about what
they are (and in our case, the CPU is going to miss about 50% of the
time), they are extremely expensive to evaluate.
Performance wise, a much saner approach is:
alias Something = int*;
Of course, this means our struct now has a self referencing pointer.
What I'm getting at is that even if there are alternatives to structs
pointing at themselves, they may not be performance wise comparable to
pointers.
Of course, the second example was a struct that has no internal
pointers, but rather maintains global pointers pointing to it. This
problem is quite a bit harder to solve.
Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 19:08, Kagamin wrote: On Thursday, 17 May 2018 at 13:50:26 UTC, Shachar Shemesh wrote: There is no such use case. Please remember that at the time opPostMove is called, both new and old memory are still allocated. That's an interesting moment too. A struct that was supposed to be moved is copied instead and exists in two places simultaneously. Can't tell it yet, but it can have a hole in type system and opPostMove can only be trusted, not safe. It is a hole (of sorts) in the type system, in that "old" is not going to have a destructor run on its code. With that said, just because the code is not safe, does not mean it is not @safe. The only inherent non @safe thing we advocate here is if you want to be able to move const/immutable structs, in which case DIP 1014 advocates casting the constness away. That will, of course, have to be either @system or @trusted. Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 18:47, kinke wrote:
On Thursday, 17 May 2018 at 15:23:50 UTC, kinke wrote:
See IR for https://run.dlang.io/is/1JIsk7.
I should probably emphasize that the LLVM `byval` attribute is strange
at first sight. Pseudo-IR `void foo(S* byval param); ... foo(S* byarg
arg);` doesn't mean that the IR callee gets the S* pointer from the IR
callsite; it means 'memcpy(param, arg, S.sizeof)', with `param` being an
*implicit* address in foo's parameters stack (calculated by LLVM and so
exposed to the callee only). That's the difficulty for LDC I mentioned
earlier.
I understand there might be difficulty, but I strongly protest the idea
that it is not possible, for one very simple reason: C++.
class Movable {
int member;
public:
Movable();
Movable( const Movable ); // Copy constructor
Movable( Movable & ); // Move constructor
}
Since clang is able to compile this struct and do everything with it,
and since the existence of the move constructor requires the precise
same type of hooking as is needed in this case, I tend to believe that
an IR representation of DIP 1014 is possible.
Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 16:42, Kagamin wrote: Looks like requirement for @nogc @safe has no consequence as the DIP suggests to infer them anyway. On ideological side safety can't be a requirement because it would contradict its purpose of providing guarantee. I think you are confusing __move_post_blt's implementation (by druntime) with opPostMove's implementation by the user. For the former, these attributes are deducted by the compiler. For the later, the user may choose to include them for all of the usual reasons for including them, not least of which is that if she does not include @nogc, then trying to move a struct's instance from @nogc code will cause compilation failure. > Especially if the suggested use case is handling of dangling > pointers. There is no such use case. Please remember that at the time opPostMove is called, both new and old memory are still allocated. Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
I'm not sure I follow all of your comments.
For the rest my comments, let's assume that the compiler may assume that
__move_post_blt is a no-op if hasElaborateMove returns false.
On 17/05/18 14:33, kinke wrote:
3. When deciding to move a struct instance, the compiler MUST emit a
call to the struct's __move_post_blt after blitting the instance and
before releasing the memory containing the old instance.
__move_post_blt MUST receive references to both the pre- and post-move
instances.
This implies that such structs must not be considered PODs, i.e., cannot
be passed in registers and must be passed on the stack.
I'm not familiar with passing structs in registers. I am familiar with
passing pointer to the structs in registers, which should not be
affected by this.
If actual (I'm assuming short) structs can be passed in registers, then,
yes, structs with elaborate move are not PoDs.
It also means
that the compiler will have to insert a __move_post_blt call right
before the call (as the callee has no idea about the old address),
Again, as far as I know, structs are not copied when passed as
arguments. They are allocated on the caller's stack and a reference is
passed to the callee. If that's the case, no move (of any kind) is done.
I might be missing something. Can you write a code snippet to which you
are referring?
after
blitting the arg to the callee params stack; this may be tricky to
implement for LDC, as that last blit is implicit in LLVM IR (LLVM byval
attribute).
And yet, C++ clang managed to do it in classes with && constructors.
As a side note, when passing a postblit-struct lvalue arg by value,
Just to be clear, are we talking about passing by reference an instance
of a struct that has postblit?
the
compiler first copies the lvalue
to a temporary on the caller's stack,
incl. postblit call, and then moves that copy to the callee. So this
requires either a postblit+postmove combo on the caller side before the
actual call, or a single postblit call for the final address (callee's
param).
Again, that does not align with my familiarity of the ABI. If I do:
struct S {
...
this(this) {
// some code
}
void opPostMove(ref S new, const ref S old) {
// some code
}
}
void func1(ref S arg) {
}
void func2(S arg) {
}
As far as I know the ABI, in func1, a pointer to S is passed. In func2,
a pointer to caller stack allocate instance is passed, and the original
is copied in. It sounds to me like you are talking about the case of:
S s;
func2(s);
in which case you need to copy s to a temporary, and then pass a pointer
to that temporary to func2. I don't see where a move enters here.
However, if a move does enter here (and one is necessary if, for
example, func2's frame needs to be dynamically allocated because an
escaping delegate references it), then, yes, an opPostMove will also
need to be called.
Again, if hasElaborateMove returns false, then no change from current
behavior is needed.
Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 11:22, rikki cattermole wrote: What is the benefit of opPostMove over copy constructors (not postblit)? The two are unrelated. A copy is a very different operation from a move. With a copy, you have to figure out how to duplicate the resources used by the object. With a move, no such duplication is needed. People are somewhat conditioned to treat a move as a "copy+destroy". One certainly may implement it that way. There is a lot of power in not having to do it, though. Think a struct with "@disable this(this)". The D compiler does moves of structs, whether they are copyable or not. This DIP is about being able to hook this move and fix external and internal references. Shachar
errno is not nothrow
At least under Linux, you cannot get or set the value of errno from a nothrow function. Is this on purpose, or is this a bug? Shachar
Re: Wait-free MPSC and MPMC implement in D
On 09/05/18 01:09, David Nadlinger wrote: The algorithm isn't wait-free (haven't thought too carefully about this, though) This mirrors a discussion I had with Maor (who originally wrote it). Let's see if I bring you around the way I was brought around. At the API level, there are two areas where the algorithm *cannot* be wait free. If a consumer tries to consume from an empty queue, or a producer tries to produce to a full one. Those two aside (which, as I stated above, are not dependent on the implementation), the algorithm actually is wait free. As far as industry-grade goes, note that many of the comments have not been updated since a previous combined implementation for SPMC/MPSC was split up into two types, so there is quite a bit of stale cruft around. Will definitely have a second look. Shachar
Re: Wait-free MPSC and MPMC implement in D
On 09/05/18 03:20, Andy Smith wrote: During Shachar's talk on the Saturday morning following the conclusion of Dconf he made it clear that the Mecca library is being used by the ~200klock Weka.io codebase ... a codebase which has very stringent latency *and* throughput requirements to satisfy customer needs in the storage space. So if any D codebase has got bragging rights on the term 'industry-grade' I think this has to be one of them. Let me start off by saying that it is great that people appreciate and enjoy Mecca. With that said, I would be wary of the direction this thread is threatening to take. I have a motto in life - if you assume you're great, you most likely aren't. I have grown out of the "my code has no bugs" phase of my life quite a while ago. The fact that code is used, to good effect and for a long period of time does not mean that it doesn't have problems, some of them might be quite fundamental. "This code is highly tested" is not a good answer to "I think I found a problem". As such, I would kindly ask everyone on this list to not take offense at criticism aimed at me or at my code. I find comments such as David's highly constructive, and the last thing I'd want is for him, or anyone else, to be wary of voicing them. I have not had the chance to parse David's critique to decide whether I agree with it or not. Regardless of my final conclusion, I think we should do everything we can to avoid suggesting it is illegitimate of him to make it. So if they've copy/pasted a few comments ... meh ... I for one happy to let it slide. These guys have got bigger fish to fry :-) Perfection is a road you take, not a place you reach. All large repositories of code tend to have areas that are less visited. Mecca is no exception. I, for one, welcome being informed of where my code can be improved. During the documentation phase, I made every effort to really understand the limitations of the code. Any code I was not sure about, I left out of the documentation (and there is a *lot* of undocumented areas in Mecca, some of which I suspect should not stay there). So, my plea is this. If you find an area worth improving in Mecca, please don't hesitate to post. If you find you do hesitate, please please feel free to send me a private message. I *want* to know of the problems. Thank you, Shachar
Re: Wait-free MPSC and MPMC implement in D
On 08/05/18 07:00, manumaster wrote: Is there some implement like this in D ? https://github.com/pramalhe/ConcurrencyFreaks/blob/master/papers/multilist-2017.pdf It's two of Mecca's containers: https://weka-io.github.io/mecca/docs/mecca/containers/otm_queue.html
Announcing Mecca
Hello everybody,
I am very happy to announce that Mecca version 0.0.1 (sorry, no more
zeros than that) is now officially available. You can get the source
code at https://github.com/weka-io/mecca. The API documentation is at
https://weka-io.github.com/mecca/docs.
Mecca is a run-time support library for fibers management, as well as a
few useful containers and libraries.
At this point in time, a getting started tutorial is still not
available. Instead, I will post this small program for an echo server.
It should give you a hint what APIs to check out with the documentation,
so you can expand your search:
import core.sys.posix.sys.socket;
import std.algorithm : move;
import std.functional : toDelegate;
import mecca.log;
import mecca.reactor;
import mecca.reactor.io.fd;
import mecca.reactor.io.signals;
enum EchoerPort = 31337;
int main()
{
theReactor.setup();
reactorSignal.registerHandler(OSSignal.SIGTERM,
toDelegate());
reactorSignal.registerHandler(OSSignal.SIGINT,
toDelegate());
theReactor.spawnFiber!listener();
return theReactor.start();
}
void termHandler(ref const(signalfd_siginfo) siginfo) {
// Signal handler, but any non-yielding operation is safe to call
from here.
theReactor.stop();
}
void listener() {
auto listenSocket = ConnectedSocket.listen( SockAddrIPv4.any(
EchoerPort ) );
listenSocket.setSockOpt( SOL_SOCKET, SO_REUSEADDR, 1 );
while( true ) {
SockAddr clientAddress;
auto clientSocket = listenSocket.accept(clientAddress);
// The next line's toString is the only reason we can't annotate
// listener as @nogc.
INFO!"Received new connection from %s"(clientAddress.toString());
theReactor.spawnFiber!echoClient( move(clientSocket) );
}
}
void echoClient(ConnectedSocket sock) @nogc {
ssize_t dataLength;
do {
char[4096] buffer;
dataLength = sock.read(buffer);
sock.write(buffer[0..dataLength]);
} while( dataLength>0 );
}
Places to hang out in Munich the day before the conference
Hello everybody, I'll be arriving in Munich on the morning of May 1st. I was wondering whether anyone has any recommendations as to how to spend that day? Thanks, Shachar
alias this and associative arrays
import std.exception;
struct AA(Key, Val) {
Val[Key] aa;
alias aa this;
void opIndexAssign(inout Val value, Key key) pure {
aa[key] = value;
}
}
void main() {
AA!(string, int) a;
//compile error -- no property 'remove' for type
'CheckedAA!(string, int)'
a.remove("aaa");
}
What's wrong here?
Re: lazy evaluation of logical operators in enum definition
On 17/04/18 13:59, Simen Kjærås wrote:
There's a kinda neat (and kinda ugly) way to implement prop in one line:
enum prop(T) = __traits(compiles, { static assert(T.member == 3); });
Now, that's not the same as short-circuiting, and only useful in some
cases, but for those cases, it's useful.
Also, extremely dangerous.
Seriously, guys and gals. __traits(compiles) (and its uglier sibling,
is(typeof())) should be used *extremely* sparingly.
The problem is that just about any use of __traits(compiles) I know of
is seeking to weed out one particular reason for the compilation
failure. There is a known bug in D, however, that the compiler
consistently fails to read the programmer's mind. The compiler only
knows that the code as provided does not compile, and that in that case
you asked for something to happen.
The usual outcome is that 80-90% of the times your code does what you
expect, but then something comes along that throws you off the beaten
track. In those cases, instead of getting an error message, you get one
of the sides of the "if", which results in random behavior by your code.
If you're lucky, you will get an error message much further down the
compilation line, and then start having a fun day of trying to figure
out what the !*#()%&!@#)!@( just happened. If you're less lucky, the
code will actually compile.
My personal rule of thumb is this: If there is *any* way of achieving
the result I want without __traits(compiles), do it that way.
Shachar
Re: isInputRange is false for non-copyable types
This is going nowhere.
Let's flesh this out face to face at dconf.
Shachar
On 16/04/18 12:01, Jonathan M Davis wrote:
On Monday, April 16, 2018 07:15:36 Shachar Shemesh via Digitalmars-d wrote:
On 16/04/18 01:28, Jonathan M Davis wrote:
The fact that foreach copies the range that it's given makes using ref
with anything other than arrays a bit of an iffy proposition. It will
compile regardless of whether front returns by ref or not (which
arguably is a bug), but it will only affect the original elements if
copying the range doesn't copy the elements
You keep referring to ranges that contain the data, but I am racking my
mind and failing to find examples.
Pretty much any generative range does. e.g. iota contains data, and each
element is then calculated from the current value. The same goes for stuff
like random number generators. It's quite common for ranges to have popFront
calculate the next front and store it as a member rather than have front do
the calculation, since then multiple calls to front don't incur the cost of
whatever has to be done to get the next element, and they can just return
the same value each time. In fact, that's generally considered to be best
practice. It's true that most ranges do not contain every element directly
in a struct, but some do (e.g. std.range.only does, which is a way to
generate a range from a list of elements without allocating anything on the
heap), and many ranges at least store the value for front. So, I'd say that
a fairly large percentage of ranges store at least one value directly in the
range object.
Plus, as I've said before, if ranges are expected to contain data,
copying them seems like an incredibly bad idea. The whole idea behind
casually copying the range about is that it's cheap to do.
Yes, it's generally assumed that it's cheap to copy a range, but it's also
generally assumed by D code in general that copying is cheap. Relatively few
functions in Phobos accept arguments by ref. Some do accept by auto ref -
either to forward the refness or to avoid a copy - but since passing by ref
doesn't work with rvalues, most code in Phobos doesn't do so unless it's
intended to mutate the argument and give the caller access to the result. I
would strongly suggest that anyone who is looking to put an object that is
expensive to copy in a range consider making it so that that object is on
the heap rather than directly in the range.
But as it stands, using ref with foreach in generic code is not going to
go well. If anything, it's actually a code smell.
You are conflating ref with owning data, which I find unsubstantiated.
My point is that if you have
foreach(ref e; range)
{
}
then unless range is a dynamic array, the odds are very high that it's a
bug, because then you risk problems like
foreach(ref e; range)
{
e = foo;
}
Outside of dynamic arrays, that assignment is usually pointless. ref implies
that it's going to actually affect the range that you passed to foreach, but
in most cases, it won't. Because the range is copied, in order for ref to
affect elements of the range that is passed to foreach, the range must be a
reference type or must act like a dynamic array in the sense that each copy
refers to exactly the same objects - which is true for some ranges (e.g.
those over a container) - but for many ranges, it isn't (e.g. generative
ranges aren't usually backed by anything). Also, the front of most ranges
does not return by ref, which would be required for the ref on foreach to
actually allow you to mutate the element. As such, I'd argue that the fact
that ref is allowed on foreach for all ranges is a bug. At minimum, it
should require that front return by ref, and even then, depending on the
range, it won't necessary work to assign to the element and have it affect
the range that was copied when it was passed to foreach. So, while it will
work in _some_ cases to pass a range to foreach and then use ref, in
general, I would consider it a code smell, because the odds are high that it
does not do what the programmer who wrote it assumed that it would do.
And if a templated function fails to compile if it's given a particular
argument, and that failure is not due to the template constraint, that
usually means that the template constraint is incomplete.
I do not accept this argument outright. In fact, our experience at Weka
has drove us to the exact opposite direction.
We deliberately set up functions to include invalid cases and then
either let them fail because of lacking constraints, or even set up a
static assert to fail the function generation rather than fail matching.
We found that the error messages produced by the alternative are simply
too difficult to parse out and figure what went wrong.
For better or for worse, it's generally considered a bug in Phobos if it's
possible to write code that gets past the template constraint and then does
not compile. There are reasons why that can be a problematic approach
lazy evaluation of logical operators in enum definition
Consider the following program:
struct S1 {
enum member = 3;
}
struct S2 {
enum member = 2;
}
struct S3 {
}
enum prop(T) = __traits(hasMember, T, "member") && T.member==3;
pragma(msg, prop!S1);
pragma(msg, prop!S2);
pragma(msg, prop!S3);
When compiled, it produces:
true
false
test.d(12): Error: no property member for type S3
test.d(16): Error: template instance `test.prop!(S3)` error instantiating
test.d(16):while evaluating pragma(msg, prop!(S3))
If I change the definition of "prop" to:
template prop(T) {
static if( __traits(hasMember, T, "member") && T.member==3 )
enum prop = true;
else
enum prop = false;
}
then everything compiles as expected.
It seems that the && evaluation does not stop when the first false is found.
Re: isInputRange is false for non-copyable types
On 16/04/18 01:28, Jonathan M Davis wrote: The fact that foreach copies the range that it's given makes using ref with anything other than arrays a bit of an iffy proposition. It will compile regardless of whether front returns by ref or not (which arguably is a bug), but it will only affect the original elements if copying the range doesn't copy the elements You keep referring to ranges that contain the data, but I am racking my mind and failing to find examples. You can have ranges over arrays,linked lists, trees (though you'd probably use opApply there), and any other container, as well as external data sources (files, sockets, pipes). In none of those cases would your range contain actual data. I need to see the use case for ranges *with* embedded data. Plus, as I've said before, if ranges are expected to contain data, copying them seems like an incredibly bad idea. The whole idea behind casually copying the range about is that it's cheap to do. which in the case of a basic input range rather than a forward range I don't see how that statement is true. is generally true, otherwise, it would be a forward range. A forward range is an input range. But as it stands, using ref with foreach in generic code is not going to go well. If anything, it's actually a code smell. You are conflating ref with owning data, which I find unsubstantiated. If non-copyable types were allowed, then functions that accept a range would fail to compile on non-copyable types if they do a copy. But the flip side is that if non-copyable would be allowed, a lot of functions that current copy would not. There are far too many unnecessary copies in the code that serve no purpose, but they are invisible, so they are not fixed. And if a templated function fails to compile if it's given a particular argument, and that failure is not due to the template constraint, that usually means that the template constraint is incomplete. I do not accept this argument outright. In fact, our experience at Weka has drove us to the exact opposite direction. We deliberately set up functions to include invalid cases and then either let them fail because of lacking constraints, or even set up a static assert to fail the function generation rather than fail matching. We found that the error messages produced by the alternative are simply too difficult to parse out and figure what went wrong. With that said, there is some merit to setting up the function constraints so that it matches what the function actually need. It allows overloading the missing combinations by another module. I don't think it is a good argument in this particular case, however. I think we should set up to document which phobos functions require copying. It will flesh out bugs and unnecessary copying in phobos, as well as force phobos developers to think about this scenario when they write code. It shouldn't even take that much time to do this run: just go over all UTs and add instantiation over a non-copyable type. The UTs that fail need work. it would mean that _every_ range-based function then has to worry about non-copyable elements No, just those in libraries. The same way they have to worry about big O complexity, type safety, @nogc and a bunch of other things. When you write a library, you have to think about your users. This doesn't change anything about that. So, allowing non-copyable types complicates range-based code across the board. Like I said above, it is more likely to find bugs than to introduce needless complications. If you code cannot be easily implemented over non-copyable types, exclude them. Don't shut me out of the entire game. Not all algorithms can call front just once without copying it Huh? Where did that come from? Phobos already assumes that calling front needs to be an O(1) operation. You can call front as many times as you like. Why would you assume otherwise? and many ranges (e.g. map) calculate front on each call, meaning that repeated calls to front can be more expensive than just calling it once and copying the result. Not by a lot, and definitely not in any way the generic code has any right to assume. I reject this argument outright. The result of all of this is that in generic code, you have no clue whatsoever what the semantics are of copying a range around, because it depends entirely on the copying semantics of whatever range that code is currently being instantiated with. As such, generic code basically has to assume that once you copy a range, you can't mutate the original (since it may or may not be affected by mutating the copy and may or may not be cleanly affected by mutating the copy), Jonathan, I'm sorry, but what you are describing is called "a bug". Input ranges provide no guarantees about copying the range itself. If you do it and expect *anything* (which it seems you do), you have a bug. If you need to copy the range itself, you absolutely
Re: isInputRange is false for non-copyable types
On 15/04/18 09:39, Jonathan M Davis wrote:
It's extremely common for range-based functions to copy front. Even foreach
does it. e.g.
Not necessarily:
foreach(ref e; [S(0), S(1), S(2)]){}
While that would work with foreach, it will not work with anything that
expects an inputRange (and since D defines a forward range as an
extension, this is also not a forward range).
If non-copyable types were allowed, then no function which accepted a range
would be allowed to copy front without first checking that front was
copyable (something that most code simply isn't going to do)
No, that's not correct.
If non-copyable types were allowed, then functions that accept a range
would fail to compile on non-copyable types if they do a copy.
But the flip side is that if non-copyable would be allowed, a lot of
functions that current copy would not. There are far too many
unnecessary copies in the code that serve no purpose, but they are
invisible, so they are not fixed.
Remember also that a large percentage of ranges that don't wrap dynamic
arrays put their elements on the stack, which means that whenever the range
is copied, the elements are copied.
If I am guaranteed to be able to copy an input range, what's the
difference between it and a forward range?
Strike that, I have a better one: If I am allowed to copy an input
range, what does the "save" function in forward range even mean?
I'm not sure why it ends up printing each of them 3 times rather than 2,
Because some code somewhere does an unnecessary copy?
the fact that foreach copies any range that it's given means that in order
to have ranges allow non-copyable types, we'd have to change how foreach
worked, which would break a lot of code. Right now,
Like I said above, foreach(ref) works with "input ranges" that define
ref return from front to uncopyable objects. Foreach without ref
doesn't, but that's mandatory from the interface: You cannot iterate
copies of a range returning uncopyable objects.
As for ranges that store the values inside them: you REALLY don't want
to copy those around. They may get quite big (not to mention that I
don't see how you can define one over a non-copyable type).
foreach(e; range)
{
}
is lowered to something like
for(auto __range = range; !__range.empty; __range.popFront())
{
auto e = __range.front;
}
And that indeed generates a lot of confusion regarding what running two
foreaches in a row does. This is a bug that already affects more than
just uncopyable objects.
That requires both copying the range and copying front. We could
theoretically change it so that everywhere that e was used, it would be
replaced with __range.front
Like I said, already solved for foreach(ref)
Now, generic range-based code really shouldn't ever use a range after it's
been copied, since whether mutating the copy affects the original depends on
the implementation of the range (and thus generic code should assume that
foreach may have consumed the range and call save if it doesn't want that to
happen), but lots of code does it anyway,
And allowing ranges with uncopyable members would turn this potential
deadly run-time bugs into easy to find compile time errors. Sound like a
great reason to allow it, to me.
and if the code isn't generic, it
can work just fine, since then the code can depend on the semantics of that
specific range type
Such as it being copyable? Non-generic code is of no relevance to this
discussion, as far as I can tell.
Also, in order for a range to work with a non-copyable type, either front
would have to return by ref or it would have to construct a new object to
return so that it could be moved rather than copied.
Correct
I expect that quite a
few ranges would have problems with such a restriction,
Then those ranges will not work with non-copyable objects. That's no
reason to make it impossible for *any* range to work with non-copyable.
In addition, it's quite possible that forcing functions to not copy front
would hurt performance.
I'm going to stop responding now, because, frankly, I think you and I
are having very different discussions.
You seem to be advocating against making *all* ranges support
non-copyable types, while I'm merely asking that *any* range be allowed
to support such types.
Current situation is that a range with uncopyable types is not
considered an input range, and cannot use any phobos range functions,
including some that it should be able to use without a problem.
There is no reason to block such ranges from using "map" or "any",
except the fact they require that the type answer "isInputRange" with true.
IIRC, when this has come up previously, Andrei ruled that supporting
non-copyable types simply wasn't worth the extra complication, though a
quick search doesn't turn up anything where he talked about it. So, I can't
verify that at the moment.
I will trust Andrei to weigh in on this point if he thinks he
isInputRange is false for non-copyable types
import std.range;
import std.stdio;
struct S {
int a;
@disable this(this);
}
void main() {
writeln(isInputRange!(S[])); // Prints false
}
The reason, as far as I can tell, is that an input range requires that
we can do:
auto a = ir.front; // Assuming ir isn't empty
That doesn't work for non-copyable types, for obvious reasons.
With that said, that seems more like a deficiency in the input range
definition than anything. There is no reason we shouldn't be able to
iterator type operations over non-copyable types.
Shachar
Re: NoCopy for overriding @disable this(this)
On 12/04/18 18:42, Uknown wrote:
On Thursday, 12 April 2018 at 12:16:53 UTC, Shachar Shemesh wrote:
[...]
test.d(19): Error: cannot convert to const(ubyte*) at compile time
[...]
Thank you,
Shachar
The problem seems to be that cast is happening at compile time, as
opposed to run time, as you might have already figured out. Do you need
to really do this cast at compile time? I tried running the snippet you
gave here: https://run.dlang.io/is/im19nL
Is this how you intend for it to be used? Then there's no need for
compile time casts. If not, could you give an example of how `NoCopy`
would be used?
struct Disabled {
int i = 17;
@disable this(this);
}
struct Container {
NoCopy!Disabled disabled;
}
Any instance you create of "Container" will have i initialized to 0 by
default.
NoCopy for overriding @disable this(this)
I'm trying to build a wrapper that will allow you to copy structs that
have members that disabled copying. The idea is that copying these
members will revert them to init.
This is what I have so far:
struct NoCopy(T) {
static assert( !hasElaborateDestructor!T, "NoCopy does not support
type " ~ T.stringof ~ " with elaborate destructor" );
private:
ubyte[T.sizeof] __bytes;
public:
this(T val) {
__bytes[] = (cast(const ubyte *))[0..T.sizeof][];
}
@property ref inout(T) value() inout nothrow @trusted @nogc {
return *cast(inout T*)__bytes.ptr;
}
this(this) {
value = T.init;
}
void opAssign(NoCopy rhs) {
value = T.init;
}
void opAssign(T rhs) {
value = move(rhs);
}
alias this value;
}
I'm having problems with setting the initial value for the byte array.
The technique I use in "value" does not work for initialization:
test.d(19): Error: cannot convert to const(ubyte*) at compile time
At first, I said "fine, the user will do it". That doesn't work,
however. It doesn't matter who tries to do it, I cannot get the byte
value of the type at compile time.
I tried using a union, but that, too, doesn't work.
I understand why the restriction is in place. What I'm wondering is
whether there is any other solution, either to the init problem or to
the overriding disable problem.
Thank you,
Shachar
Re: Mission impossible
On 11/04/18 11:51, Uknown wrote:
On Wednesday, 11 April 2018 at 08:47:12 UTC, Basile B. wrote:
On Wednesday, 11 April 2018 at 08:36:41 UTC, Shachar Shemesh wrote:
struct S {
static void func(T)(uint a, T t) {
}
static void func() {
}
}
Your mission, Jim, should you choose to accept it, is this:
Get a pointer to the version of the function that accepts no arguments.
As always, should you or any of your Force be caught or killed, the
Secretary will disavow any knowledge of your actions. This disc will
self-destruct in ten seconds.
Good luck, Jim.
I suppose __traits(getOverloads) doesn't work because of a bug ?
The template hides any other overloads that func() has:
IF it is defined before it. If it is defined after it, it does not (but
it is then possible that the non-template version hides the template one).
The problem is that S has two members called "func". One is a function,
and the other is a template. "getOverloads" is not built to distinguish
between the two.
I'm not sure if its a bug though
What else is it?
Shachar
Mission impossible
struct S {
static void func(T)(uint a, T t) {
}
static void func() {
}
}
Your mission, Jim, should you choose to accept it, is this:
Get a pointer to the version of the function that accepts no arguments.
As always, should you or any of your Force be caught or killed, the
Secretary will disavow any knowledge of your actions. This disc will
self-destruct in ten seconds.
Good luck, Jim.
Re: =void in struct definition
On 11/04/18 10:58, Jonathan M Davis wrote: All objects are initialized with their init values prior to the constructor being called. So, whether an object is simply default-initialized or whether the constructor is called, you're going to get the same behavior except for the fact that the constructor would normally do further initialization beyond the init value. As such, if there's a problem with the default-initialized value, you're almost certainly going to get the same problem when you call a constructor. - Jonathan M Davis That's horrible! That means that constructor initialized objects, regardless of size, get initialized twice. Shachar
Re: =void in struct definition
On 09/04/18 14:22, Jonathan M Davis wrote:
On Monday, April 09, 2018 14:06:50 Shachar Shemesh via Digitalmars-d wrote:
struct S {
int a;
int[5000] arr = void;
}
void func() {
S s;
}
During the s initialization, the entire "S" area is initialized,
including the member arr which we asked to be = void.
Is this a bug?
It looks like Andrei created an issue about it as an enhancement request
several years ago:
https://issues.dlang.org/show_bug.cgi?id=11331
- Jonathan M Davis
Except that issue talks about default constructed objects. My problem
happens also with objects constructed with a constructor:
extern(C) void func(ref S s);
struct S {
uint a;
int[5000] arr = void;
this(uint val) {
a = val;
}
}
void main() {
auto s = S(12);
// To prevent the optimizer from optimizing s away
func(s);
}
$ ldc2 -c -O3 -g test.d
$ objdump -S -r test.o | ddemangle > test.s
<_Dmain>:
}
}
void main() {
0: 48 81 ec 28 4e 00 00sub$0x4e28,%rsp
7: 48 8d 7c 24 04 lea0x4(%rsp),%rdi
auto s = S(12);
c: 31 f6 xor%esi,%esi
e: ba 20 4e 00 00 mov$0x4e20,%edx
13: e8 00 00 00 00 callq 18 <_Dmain+0x18>
14: R_X86_64_PLT32 memset-0x4
a = val;
18: c7 04 24 0c 00 00 00movl $0xc,(%rsp)
1f: 48 89 e7mov%rsp,%rdi
// To prevent the optimizer from optimizing s away
func(s);
22: e8 00 00 00 00 callq 27 <_Dmain+0x27>
23: R_X86_64_PLT32 func-0x4
}
27: 31 c0 xor%eax,%eax
29: 48 81 c4 28 4e 00 00add$0x4e28,%rsp
30: c3 retq
Notice the call to memset.
Shachar
