Re: A New Era for the D Community
On Wednesday, 3 May 2023 at 23:24:53 UTC, Walter Bright wrote: This initiative has my full support. While I have ceased using D because of my concerns about the project's future (I discussed my reasons in a previous message that don't need to be repeated), I have continued to check this forum occasionally, hoping to see the slope turn positive. Mike's message and your response are both the kind of thing I was hoping for. While there is no guarantee that the effort Mike describes will have the desired outcome, the mere fact that the effort has been made and endorsed by you is a significant positive step. I'm sure I needn't tell you that technical work and project management require related but different talents. I did both professionally for a very long time and I certainly was not equally good at both. You can probably guess which one was the laggard. But I have seen it done well, having worked for some really great managers. One who deserves mention is the late Frank Heart. The ARPANet (and thus the Internet) would not have existed without Frank's unique ability to herd the brilliant cats at BBN 50+ years ago. He's in the Internet Hall of Fame, deservedly. See https://en.wikipedia.org/wiki/Frank_Heart. Some of the great people in the history of computer science are in the picture on that page, including Bob Kahn, who, with Vint Cerf, won the Turing Award. Both played absolutely key roles in the development of the Internet. I really hope that this is the start of something good for this project. A lot of value has been built here, but the project has obviously foundered in an organizational way. Project management is difficult, so the trouble is not surprising or unique. The key is recognizing that the problems are happening and taking steps that have a reasonable probability of improving the situation. I will watch how this unfolds with great interest. /Don Allen
Re: More fun with toStringz and the GC
On Saturday, 6 August 2022 at 13:40:12 UTC, Don Allen wrote: On Saturday, 6 August 2022 at 02:14:24 UTC, Steven Schveighoffer wrote: On 8/5/22 8:51 PM, Don Allen wrote: And this, from Section 32.2 of the Language Reference Manual: If pointers to D garbage collector allocated memory are passed to C functions, it's critical to ensure that the memory will not be collected by the garbage collector before the C function is done with it. This is accomplished by: Making a copy of the data using core.stdc.stdlib.malloc() and passing the copy instead. -->Leaving a pointer to it on the stack (as a parameter or automatic variable), as the garbage collector will scan the stack.<-- Leaving a pointer to it in the static data segment, as the garbage collector will scan the static data segment. Registering the pointer with the garbage collector with the std.gc.addRoot() or std.gc.addRange() calls. I did what the documentation says and it does not work. I know, I felt exactly the same way in my post on it: https://forum.dlang.org/post/sial38$7v0$1...@digitalmars.com I even issued a PR to remove the problematic recommendation: https://github.com/dlang/dlang.org/pull/3102 But there was pushback to the point where it wasn't worth it. So I closed it. As I said in my previous post, the documentation issue really needs to be addressed. I do realize now that I *assumed* that what I did was going to result in a stack reference to the c-string I was trying to keep alive. At the risk of over-doing this, one more thing I want to say in the interest of clarity: the incorrect documentation led me right into this error: "This is accomplished by . Leaving a pointer to it on the stack (as a parameter or automatic variable), as the garbage collector will scan the stack." I've fixed my code using addRoot/removeRoot and so far it seems to work.
Re: More fun with toStringz and the GC
On Saturday, 6 August 2022 at 02:14:24 UTC, Steven Schveighoffer wrote: On 8/5/22 8:51 PM, Don Allen wrote: And this, from Section 32.2 of the Language Reference Manual: If pointers to D garbage collector allocated memory are passed to C functions, it's critical to ensure that the memory will not be collected by the garbage collector before the C function is done with it. This is accomplished by: Making a copy of the data using core.stdc.stdlib.malloc() and passing the copy instead. -->Leaving a pointer to it on the stack (as a parameter or automatic variable), as the garbage collector will scan the stack.<-- Leaving a pointer to it in the static data segment, as the garbage collector will scan the static data segment. Registering the pointer with the garbage collector with the std.gc.addRoot() or std.gc.addRange() calls. I did what the documentation says and it does not work. I know, I felt exactly the same way in my post on it: https://forum.dlang.org/post/sial38$7v0$1...@digitalmars.com I even issued a PR to remove the problematic recommendation: https://github.com/dlang/dlang.org/pull/3102 But there was pushback to the point where it wasn't worth it. So I closed it. As I said in my previous post, the documentation issue really needs to be addressed. I do realize now that I *assumed* that what I did was going to result in a stack reference to the c-string I was trying to keep alive. Bad assumption, obviously. But I think the point is that there is a simple, reliable mechanism -- addRoot, removeRoot -- that works and the documentation should say that and only that. Walter said this in his 9/25/21 post: "Use GC.addRoot() to keep a reference alive. That's what it's for. ". That's all that's needed. All the rest leads people like me who don't think like a compiler to make the mistake I made.
Re: More fun with toStringz and the GC
On Friday, 5 August 2022 at 23:38:22 UTC, Steven Schveighoffer wrote: On 8/5/22 7:13 PM, jfondren wrote: On Friday, 5 August 2022 at 22:51:07 UTC, Don Allen wrote: My theory: because gc_protect2 is never referenced, I'm guessing that the compiler is optimizing away the storage of the returned pointer, the supporting evidence being what I said in the previous paragraph. Anyone have a better idea? A local variable definitely isn't enough: https://forum.dlang.org/thread/xchnfzvpmxgytqprb...@forum.dlang.org This package came of it: https://code.dlang.org/packages/keepalive Yes, but I will warn you, the compilers are smart buggers. I think someone came up with a case where this still doesn't keep it alive (been a while since I made that). The only true solution is to use `GC.addRoot` on the string and `GC.removeRoot` when you are done. Steve -- Thanks for this. But this time I *did* read the documentation, specifically this: Interfacing Garbage Collected Objects With Foreign Code The garbage collector looks for roots in: the static data segment the stacks and register contents of each thread the TLS (thread-local storage) areas of each thread any roots added by core.memory.GC.addRoot() or core.memory.GC.addRange() If the only pointer to an object is held outside of these areas, then the collector will miss it and free the memory. To avoid this from happening, either maintain a pointer to the object in an area the collector does scan for pointers; add a root where a pointer to the object is stored using core.memory.GC.addRoot() or core.memory.GC.addRange(). reallocate and copy the object using the foreign code's storage allocator or using the C runtime library's malloc/free. And this, from Section 32.2 of the Language Reference Manual: If pointers to D garbage collector allocated memory are passed to C functions, it's critical to ensure that the memory will not be collected by the garbage collector before the C function is done with it. This is accomplished by: Making a copy of the data using core.stdc.stdlib.malloc() and passing the copy instead. -->Leaving a pointer to it on the stack (as a parameter or automatic variable), as the garbage collector will scan the stack.<-- Leaving a pointer to it in the static data segment, as the garbage collector will scan the static data segment. Registering the pointer with the garbage collector with the std.gc.addRoot() or std.gc.addRange() calls. I did what the documentation says and it does not work. Having a better version of C and C++ with a gc and the ability to directly call useful C/C++ libraries is a big D selling point, as far as I am concerned. It was a major motivation for the creation of Go. But getting the interaction between the GC and foreign functions properly documented is essential. Right now, there are bits and pieces of advice in the Language Reference, the Feature Overview, and the toStringz documentation and none of it tells you what you need to know. In fact, it does the opposite, telling you to do something (stick a pointer on the stack) that does not work, which leads to the "nasty bug" spoken of in the toStringz doc. When you waste a lot of a user's time with poor and inaccurate documentation, as this did mine, you are not making friends. I would advise fixing this asap. /Don
More fun with toStringz and the GC
Remember all the fun we had last year when I failed to heed the
warning in the toStringz documentation about retaining a
reference to a char * passed into C? It took a long time to find
that one, with a lot of help from Steve Schveighoffer and others.
Well, I've got another one. Consider this:
// Get number of children of the parent account
auto gc_protect = bind_text(n_children_stmt, 1,
parent.guid);
parent.children.length = one_row!(int)(n_children_stmt,
_int);
auto gc_protect2 = bind_text(account_child_stmt, 1,
parent.guid);
for (int i = 0; next_row_available_p(account_child_stmt,
_reset); i++) {
parent.children[i] = new Account;
parent.children[i].name =
fromStringz(sqlite3_column_text(account_child_stmt, 0)).idup;
parent.children[i].guid =
fromStringz(sqlite3_column_text(account_child_stmt, 1)).idup;
parent.children[i].flags =
sqlite3_column_int(account_child_stmt, 2);
parent.children[i].value =
get_account_value(parent.children[i]);
}
bind_text takes a D string, turns it into a C string with
toStringz, uses that to call sqlite3_bind_text and returns the C
string, which I store as you can see with the intention of
protecting it from the gc. The code as written above does not
work. At some point, I get an index-out-of-bounds error, because
the loop is seeing too many children. If I turn off the GC, the
code works correctly and the application completes normally.
With the GC on, if I put a debugging writeln inside the loop,
right after the 'for', that prints, among other things, the value
of gc_protect2 (I wanted to convince myself that the GC wasn't
moving what it points to; yes, I know the documentation says the
current GC won't do that), the problem goes away. A Heisenbug!
My theory: because gc_protect2 is never referenced, I'm guessing
that the compiler is optimizing away the storage of the returned
pointer, the supporting evidence being what I said in the
previous paragraph. Anyone have a better idea?
By the way, I get the same error compiling this with dmd or ldc.
/Don Allen
Re: Question re specific implicit type coercion
On Monday, 18 October 2021 at 15:58:35 UTC, Don Allen wrote: On Monday, 18 October 2021 at 15:34:45 UTC, Paul Backus wrote: On Monday, 18 October 2021 at 15:04:11 UTC, Don Allen wrote: Section 12.17 of the Language Reference does not indicate any circumstance in which a dynamic array, which the literal is, is implicitly coerced to a pointer. This is a special case for string literals, covered in [section 10.23.7][1]. [1]: https://dlang.org/spec/expression.html#string_literals Yes. Thank you. I do think it would be helpful, even essential, to have a link in Section 12.17 to the section you cite. The Language Reference is just that, a reference, not a tutorial and it should not be a requirement to read it from cover to cover. Strings and string literals are arrays and 12.17 purports to cover the implicit type conversions among the various array types, but currently leaves out this special case and should not, in my opinion. I will file an issue making this suggestion.
Re: Question re specific implicit type coercion
On Monday, 18 October 2021 at 15:34:45 UTC, Paul Backus wrote: On Monday, 18 October 2021 at 15:04:11 UTC, Don Allen wrote: Section 12.17 of the Language Reference does not indicate any circumstance in which a dynamic array, which the literal is, is implicitly coerced to a pointer. This is a special case for string literals, covered in [section 10.23.7][1]. [1]: https://dlang.org/spec/expression.html#string_literals Yes. Thank you.
Question re specific implicit type coercion
I am calling a C function, described in D as
extern (C) GtkWidget* gtk_menu_item_new_with_label
(immutable(char)*);
I call it like this
accounts_menu_item = gtk_menu_item_new_with_label("New account
(Ctrl-n)");
The type of the string literal in the call is immutable(char)[].
The type of the parameter is a pointer to immutable characters.
This setup works.
My question is about the documentation vs. what is really going
on here.
Section 12.17 of the Language Reference does not indicate any
circumstance in which a dynamic array, which the literal is, is
implicitly coerced to a pointer. I realize that this is a special
case -- the callee is a C function -- that might not be covered
by this section. So I looked at Section 32.3, Chapter 32 being
"Interfacing to C". 32.3 contains a table of correspondences
between D and C types. The entry for the D type T[] indicates no
correspondence in C.
So at least my reading of the documentation suggests that my code
should not compile. And yet it does and works, which leads me to
believe there is a documentation error that ought to be reported.
Before doing so, I decided to send this message so those of you
who know D a lot better than I do can tell me what I've missed or
gotten wrong, if that is the case.
Thanks --
/Don
Re: Metaprogramming with D
On Sunday, 7 June 2020 at 00:45:37 UTC, Ali Çehreli wrote: False. And again, even if so, that's not because of D, but because of humans. Can you imagine a CTO, say, in Silicon Valley to have guts to bring D instead of C++? With C++, the CTO will never be blamed; but D, he or she can easily be blamed upon failure. Not because of the technologies but because of politics. In other words, "Nobody ever got fired for choosing [C++]."
Re: DIP 1016--ref T accepts r-values--Formal Assessment
On Wednesday, 30 January 2019 at 13:58:38 UTC, 12345swordy wrote: I do not accept gut feeling as a valid objection here. The current workarounds is shown to be painful as shown in the dip and in the discussions that it currently link. That *the* motivation here. Like I said previously I am on the reviews side and that's it. By the way I don't like your tone when you say: "I do not accept gut feeling as a valid objection here". I don't think you would like if I say that your opinion is biased because you know the author either, so don't go that way, because it's not only me against this DIP. I am familiar with the author here, he is very involved with the C++<->D compatibility side of things. He knows the pain from first hand experience. Alright we're talking about a change that have been on hold for almost 10 years, if it was simple it would already been done. In this thread we saw some other concerns been emerged. Finally I only know the author by his postings in this forum, and I don't have anything personally against him. Donald.
Re: DIP 1016--ref T accepts r-values--Formal Assessment
On Wednesday, 30 January 2019 at 03:01:36 UTC, 12345swordy wrote: On Wednesday, 30 January 2019 at 00:25:17 UTC, Don wrote: But what I fail to see is why can't the programmer solve this themselves instead of relying on a new feature that would cause more harm? Donald. ...Did you even read the arguments in the dip? This has been discuss quite a lot in the forums, it even gives you links to them. Well, I read the DIP and the whole forum discussion back in the day, and again I think this will create more harm than benefits the way it was proposed. And starting from the beginning of this DIP - Rationale example: "void fun(int x); fun(10); // <-- this is how users expect to call a typical function But when ref is present: void fun(ref int x); fun(10); // <-- compile error; not an lvalue!! Necessitating the workaround: int temp = 10; fun(temp); This inconvenience extends broadly to every manner of rvalue passed to functions, including:" So the solution in the way I understood is pretty much a syntax sugar, creating temporary variable with destruction. But the concept is weird, because originally your function signature has a "ref parameter" and we're just creating a workaround expanding it to handle rvalues. I would prefer to handle it myself with overloading instead of being presented with new language feature creating different scenarios for something that's not the case right now. Otherwise D will be pretty much like C++ and in this case why bother with it? Donald.
Re: DIP 1016--ref T accepts r-values--Formal Assessment
I'm on the reviewers side here.
To be honest I never liked this DIP and maybe I'll sound dumb but
I think this is a case where this could bring more problem than
anything.
The way I see this would be more like a syntax sugar to create
temporary variable for ref parameters and that's it.
But what I fail to see is why can't the programmer solve this
themselves instead of relying on a new feature that would cause
more harm?
With overload some could do:
void f(int i){
f(i);
}
void f(ref int i){
++i;
writeln(i);
}
void main(){
int i = 0;
f(10);
f(i);
}
prints:
11
1
The "f" function will work with ref or literal (rvalues).
But this will be controlled by the programmer the way they want
it.
Donald.
Re: Java also has chained exceptions, done manually
On Thursday, 6 September 2018 at 14:39:12 UTC, Andrei Alexandrescu wrote: In an earlier post, Don Clugston wrote: When I originally implemented this, I discovered that the idea of "chained exceptions" was hopeless naive. The idea was that while processing one exception, if you encounter a second one, and you chain them together. Then you get a third, fourth, etc. The problem is that it's much more complicated than that. Each of the exceptions can be a chain of exceptions themselves. This means that you don't end up with a chain of exceptions, but rather a tree of exceptions. That's why there are those really nasty test cases in the test suite. The examples in the test suite are very difficult to understand if you expect it to be a simple chain! On the one hand, I was very proud that I was able to work out the barely-documented behaviour of Windows SEH, and it was really thorough. In the initial implementation, all the complexity was covered. It wasn't the bugfix-driven-development which dmd usually operates under . But on the other hand, once you can see all of the complexity, exception chaining becomes much less convincing as a concept. Sure, the full exception tree is available in the final exception which you catch. But, is it of any use? I doubt it very much. It's pretty clearly a nett loss to the language, it increases complexity with negligible benefit. Fortunately in this case, the cost isn't really high. First off, there's no tree of exceptions simply because... well it's not there. There is on field "next", not two fields "left" and "right". It's a linear list, not a tree. During construction there might be the situation whereby two lists need to be merged. But they will be merged by necessity into a singly-linked list, not a tree, because we have no structural representation of a tree. That's correct, the *end result* is not a tree of exceptions, but the intermediate state is a tree. There can be an arbitrary number of exceptions in flight at any given time. The final exception chain is the result of a depth-first traverse of the call stack. Given an exception chain A->B->C, you don't know if C was thrown while processing B, or while processing A. So the exception chain is inherently ambiguous. My point is that this reduces the appeal of the feature. Note that my message was in response to Walter being confused as to why the tests were so complicated. (As an aside, it does seem we could allow some weird cases where people rethrow some exception down the chain, thus creating loops. Hopefully that's handled properly.) A loop is not possible in ordinary operation, any more than a loop is possible in a depth-first traverse of a tree. But, what I don't know is, what happens if there is a switch to a different fiber inside a `finally` clause? I never considered that. At Sociomantic we have code to deal with exceptions being thrown across context switches. Normally it's a bad idea. However, I don't believe we considered that the exception being thrown across the context switch, might not be the only exception in flight at that moment. Maybe it's perfectly fine. I just don't know. Second, it does pay to keep abreast other languages. I had no idea (and am quite ashamed of it) that Java also has chained exceptions: https://www.geeksforgeeks.org/chained-exceptions-java/ This is fascinating. They implement them manually, i.e. the user who throws a new exception would need to pass the existing exception (or exception chain) as an argument to the new exception's constructor. Otherwise, an exception thrown from a catch/finally block obliterates the existing exception and replaces it with the new one: https://stackoverflow.com/questions/3779285/exception-thrown-in-catch-and-finally-clause So chaining exceptions in Java is a nice complementary mechanism to compensate for that loss in information: when you throw, you have the chance to chain the current exception so it doesn't get ignored. Because of that, D's chained exceptions mechanism can be seen as an automated way of doing "the right thing" in Java. We should study similarities and distinctions with Java's mechanism and discuss them in our documentation. Andrei It's implemented, it works, I was quite proud of having figured it out. Is it actually useful? Dunno. Don.
Re: D is dead
On Monday, 27 August 2018 at 07:34:37 UTC, Walter Bright wrote: On 8/23/2018 8:53 PM, David Nadlinger wrote: On Thursday, 23 August 2018 at 23:27:51 UTC, Walter Bright wrote: D deals with it via "chained exceptions", which is terrifyingly difficult to understand. If you believe it is understandable, just try to understand the various devious test cases in the test suite. I don't think that assessment is accurate. Yes, I ported EH to a few new targets and wrote the first correct implementation of exception chaining for LDC, so I'm probably a couple of standard deviations off the average D user in that regard. But said average D user doesn't care about most of the nuances of this problem, like the details of implementing exception chaining without allocating too much, I find myself unable to explain the rationale of the behavior exhibited by the current chaining system. I dared not change it, as I presumed somebody surely built a store around it. It does not simply chain exceptions. or which exceptions take precedence in various intentionally twisted test cases. The only documentation for this is the test suite itself, which does not have any documentation or rationale either, just tests. I would appreciate it if you did document what it's supposed to do and why, as likely nobody else knows. Maybe if I understood why I'd be more forgiving of it :-) What they do care about is that the fact that an error has occurred is propagated sensibly in all cases without requiring extra attention, and that information as to the origin is not lost (hence chaining rather than just replacement). Heck, the fact that we still don't have default backtrace handlers that consistently work on all platforms is probably a much bigger problem than the minutiae of exception chaining behaviour. I wish the minutiae was documented somewhere :-( as I care about the nuances of it, if only because I'm ultimately responsible for keeping it working correctly. I can explain this, since I did the original implementation. When I originally implemented this, I discovered that the idea of "chained exceptions" was hopeless naive. The idea was that while processing one exception, if you encounter a second one, and you chain them together. Then you get a third, fourth, etc. The problem is that it's much more complicated than that. Each of the exceptions can be a chain of exceptions themselves. This means that you don't end up with a chain of exceptions, but rather a tree of exceptions. That's why there are those really nasty test cases in the test suite. The examples in the test suite are very difficult to understand if you expect it to be a simple chain! On the one hand, I was very proud that I was able to work out the barely-documented behaviour of Windows SEH, and it was really thorough. In the initial implementation, all the complexity was covered. It wasn't the bugfix-driven-development which dmd usually operates under . But on the other hand, once you can see all of the complexity, exception chaining becomes much less convincing as a concept. Sure, the full exception tree is available in the final exception which you catch. But, is it of any use? I doubt it very much. It's pretty clearly a nett loss to the language, it increases complexity with negligible benefit. Fortunately in this case, the cost isn't really high. -Don.
Re: D's SwitchStatement accepts statements with ridiculous semantics
On Friday, 29 September 2017 at 10:32:02 UTC, Timon Gehr wrote:
On 29.09.2017 11:12, Don Clugston wrote:
Guess what this prints
import std.stdio;
void main()
{
int i = 0;
switch (i) for (i = 8; i < 10; ++i)
{
case 7:
writeln(i);
return;
default: ;
}
}
Why does this even compile? It's because the grammar is:
SwitchStatement:
switch ( Expression ) ScopeStatement
and ScopeStatement allows almost anything.
I think the only sane grammar is
SwitchStatement:
switch ( Expression ) BlockStatement
Initially I thought ScopeStatement was accepted in order to
enable Duff's device, but it isn't necessary for that. It
might have originally accepted ScopeStatement to support
E e; switch( e ) with (E) { }
Or it may have just been an accident.
It is very likely that this part of the grammar was
deliberately copied from C. It's also consistent with how all
other control flow constructs are parsed.
But regardless of the original motivation, it allows some
truly dreadful semantics.
I don't see what your proposed grammar change accomplishes:
switch(i){
for(i=8;i<10;++i){
case 7:
writeln(i);
return;
default:{}
}
}
I.e., you seem to have misidentified the culprit. Whether or
not to the curly braces are required by the parser has nothing
to do with switch semantics.
That case looks quite different to me.
It's rather more obvious that the `for` statement has been
skipped.
The problem I have with the original example is that it looks as
though the body of the `for` loop is the body of the switch
statement.
Can we disallow this silliness please?
Maybe this specific case can be disallowed during semantic (but
I don't really see why it helps, it mostly just makes the
language definition more complex).
I believe it makes it simpler. You cannot avoid the reference to
BlockStatement.
Note that: "A switch statement must have a default statement."
This is only possible only in two situations.
1. Silly degenerate case.
switch (i) default: ;
2. A statement which contains a BlockStatement.
It accepts unreachable code.
switch (i) if ( foo() ) {} else { default: ; }
A switch statement followed by anything other than a
BlockStatement is *always* wrong. Always.
We improved the switch statement a lot by disallowing implicit
fallthrough. But it still allows other nonsense that was
presumably inherited from the very early days of K C.
This example just struck me as exceedingly silly, and quite
misleading.
D's SwitchStatement accepts statements with ridiculous semantics
Guess what this prints
import std.stdio;
void main()
{
int i = 0;
switch (i) for (i = 8; i < 10; ++i)
{
case 7:
writeln(i);
return;
default: ;
}
}
Why does this even compile? It's because the grammar is:
SwitchStatement:
switch ( Expression ) ScopeStatement
and ScopeStatement allows almost anything.
I think the only sane grammar is
SwitchStatement:
switch ( Expression ) BlockStatement
Initially I thought ScopeStatement was accepted in order to
enable Duff's device, but it isn't necessary for that. It might
have originally accepted ScopeStatement to support
E e; switch( e ) with (E) { }
Or it may have just been an accident.
But regardless of the original motivation, it allows some truly
dreadful semantics.
Can we disallow this silliness please?
Re: Exception chaining and collectException
On Friday, 18 August 2017 at 03:31:38 UTC, Walter Bright wrote: Chained exceptions are a good idea, but are more or less a disaster: 1. No other language does chained exceptions 2. Attempting to hammer D chained exceptions into other language schemes (such as C++) makes for lots of unfun hours attempting to decode undocumented behavior in those other schemes 3. Makes D exceptions incompatible with other language exceptions and their infrastructure 4. Are incomprehensibly implemented (I defy anyone to explain how the test cases in the test suite are actually supposed to work) Well, I wrote them, so I can explain that. The problem is that the idea that you can form a "chain" of exceptions turns out to be naive. What if a chained exception needs to get chained to another chained exception? And that then needs to be chained to another exception? It forms a tree! That's why the test cases are so complicated. So to a large extent, this extremely obscure corner case destroys the elegance of the concept. Secondly, exception handling in windows is practically undocumented. Certainly it's not documented in a single place. When I began to implement it, I feared it might be impossible. There isn't any guarantee that exception chaining can actually be implemented on all platforms. 5. Are more or less incompatible with non-GC memory allocation I'd like to remove them from D. I recommend *not* designing any program that requires them. I invested quite a lot personally in implementing chained exceptions. But I agree with you. I was actually quite proud that I worked out the nasty corner cases during the initial implementation. As far as I can tell, problems with chained exceptions are not because of bugs and implementation issues, but because of problems with the concept itself. I think it's just a bit too clever.
Re: Battle-plan for CTFE
On Sunday, 15 May 2016 at 12:17:30 UTC, Daniel Murphy wrote: On 15/05/2016 9:57 PM, Martin Nowak wrote: On 05/15/2016 01:58 PM, Daniel Murphy wrote: The biggest advantage of bytecode is not the interpreter speed, it's that by lowering you can substitute VarExps etc with actual references to memory without modifying the AST. By working with something lower level than the AST, you should end up with something much less complex and with fewer special cases. Which is a bad assessment, you can stick variable indexes into VarDeclaration (we already do that) and thereby access them in O(1). Converting control flow and references into byte code is far from trivial, we're talking about another s2ir and e2ir here. -Martin For simple types that's true. For more complicated reference types... Variable indexes are not enough, you also need heap memory, but slices and pointers (and references) can refer to values either on the heap or the stack, and you can have a slice of a member static array of a class on the stack, etc. Then there are closures... Neither e2ir or s2ir are actually that complex. A lot of the mess there comes from the backend IR interface being rather difficult to work with. We can already save a big chunk of complexity by not having to translate the frontend types. E.g. implementing the logic in the interpreter to correctly unwind through destructors is unlikely to be simpler than lowering to an IR. Exactly. I think the whole idea of trying to avoid a glue layer is a mistake. CTFE is a backend. It really is. And it should be treated as one. A very simple one, of course. Once you do this, you'll find all sorts of commonalities with the existing glue layers. We should end up with at least 4 backends: DMD, GCD, LDC, and CTFE. Many people here are acting like this is something complicated, and making dangerous suggestions like using Phobos inside the compiler. (I think everyone who has fixed a compiler bug that was discovered in Phobos, will know what a nightmare that would be. The last thing compiler development needs is another level of complexity in the compiler). As I've tried to explain, the problems with CTFE historically were never with the CTFE engine itself. They were always with the interface between CTFE and the remainder of the compiler -- finding every case where CTFE can be called, finding all the bizarre cases (tuple variables, variables without a stack because they are local variables declared in comma expressions in global scope, local 'ref' variables, etc), finding all the cases where the syntax trees were invalid... There's no need for grandiose plans, as if there is some almost-insurmountable problem to be solved. THIS IS NOT DIFFICULT. With the interface cleaned up, it is the well-studied problem of creating an interpreter. Everyone knows how to do this, it's been done thousands of times. The complete test suite is there for you. Someone just needs to do it. I think I took the approach of using syntax trees about as far as it can go. It's possible, but it's really vile. Look at the code for doing assignments. Bleagh. The only thing in its favour is that originally it was the only implementation that was possible at all. Even the first, minimal step towards creating a ctfe backend -- introducing a syntax-tree-validation step -- simplified parts of the code immensely. You might imagine that it's easier to work with syntax trees than to start from scratch but I'm certain that's not true. I'm pretty sure that the simplest approach is to use the simplest possible machine-independent bytecode that you can come up with. I had got to the point of starting that, but I just couldn't face doing it in C++. TL;DR: CTFE is actually a backend, so don't be afraid of creating a glue layer for it.
Re: Battle-plan for CTFE
On Monday, 9 May 2016 at 16:57:39 UTC, Stefan Koch wrote: Hi Guys, I have been looking into the DMD now to see what I can do about CTFE. Unfortunately It is a pretty big mess to untangle. Code responsible for CTFE is in at least 3 files. [dinterpret.d, ctfeexpr.d, constfold.d] I was shocked to discover that the PowExpression actually depends on phobos! (depending on the exact codePath it may or may not compile...) Yes. This is because of lowering. Walter said in his DConf talk that lowering was a success; actually, it's a quick-and-dirty hack that inevitably leads to a disaster. Lowering always needs to be reverted. which let to me prematurely stating that it worked at ctfe [http://forum.dlang.org/thread/ukcoibejffinknrbz...@forum.dlang.org] My Plan is as follows. Add a new file for my ctfe-interpreter and update it gradually to take more and more of the cases the code in the files mentioned above was used for. Do Dataflow analysis on the code that is to be ctfe'd so we can tell beforehand if we need to store state in the ctfe stack or not. You don't need dataflow analysis. The CtfeCompile pass over the semantic tree was intended to determine how many variables are required by each function. Or baring proper data-flow analysis: RefCouting the variables on the ctfe-stack could also be a solution. I will post more details as soon as I dive deeper into the code. The current implementation stores persistent state for every ctfe incovation. While caching nothing. Not even the compiled for of a function body. Because it cannot relax purity. No. Purity is not why it doesn't save the state. It's because of history. I think I need to explain the history of CTFE. Originally, we had constant-folding. Then constant-folding was extended to do things like slicing a string at compile time. Constant folding leaks memory like the Exxon Valdez leaks oil, but that's OK because it only ever happens once. Then, the constant folding was extended to include function calls, for loops, etc. All using the existing constant-folding code. Now the crappy memory usage is a problem. But it's OK because the CTFE code was kind of proof-of-concept thing anyway. Now, everyone asks, why doesn't it use some kind of byte-code interpreter or something? Well, the reason is, it just wasn't possible. There was actually no single CTFE entry point. Instead, it was a complete mess. For example, with template arguments, the compiler would first try to run CTFE on the argument, with error messages suppressed. If that succeeded, it was a template value argument. If it generated errors, it would then see if was a type. If that failed as well, it assumed it was a template alias argument. The other big problem was that CTFE was also often called on a function which had semantic errors. So, here is what I did with CTFE: (1) Implement all the functionality, so that CTFE code can be developed. The valuable legacy of this, which I am immensely proud of, is the file "interpret3.d" in the test suite. It is very comprehensive. If an CTFE implementation passes the test suite, it's good to go. The CTFE implementation itself is practically worthless. It's value was to get the test suite developed. (2) Created a single entry point for CTFE. This involved working out rules for every place that CTFE is actually required, removing the horrid speculative execution of CTFE. It made sure that functions had actually been semantically analyzed before they were executed (there were really horrific cases where the function had its semantic tree modified while it was being executed!!) Getting to this point involved about 60 pull requests and six months of nearly full-time work. Finally it was possible to consider a byte-code interpreter or JITer. We reached this point around Nov 2012. (3) Added a 'ctfeCompile' step which runs over the semantic tree the first time the function is executed at compile time. Right now it does nothing much except that check that the semantic tree is valid. This detected many errors in the rest of the compiler. We reached this point around March 2013. My intention was to extend the cfteCompile step to a byte-code generator. But then I had to stop working on it and concentrate on looking after my family. Looking at the code without knowing the history, you'll think, the obvious way to do this would be with a byte-code generator or JITer, and wonder why the implementation is so horrible. But for most of the history, that kind of implementation was just not possible. People come up with all these elaborate schemes to speed up CTFE. It's totally not necessary. All that's needed is a very simple bytecode interpreter.
Re: Signed integer overflow undefined behavior or not?
On Friday, 13 November 2015 at 09:37:41 UTC, deadalnix wrote: On Friday, 13 November 2015 at 09:33:51 UTC, John Colvin wrote: I don't understand what you think is so complicated about it? After arithmetic operations f is applied signed: f(v) = ((v + 2^(n-1)) mod (2^n - 1)) - 2^(n-1) Complicated in the sense that: when are those semantics useful? The answer of course, is, pretty much never. They are very bizarre. It is not that it is complicated, but that signed wraparound is almost always a bug. In C/C++, that result in very questionable optimizations. But defining the thing as wraparound is also preventing it to become an error. On the other hand, detection the overflow is expensive on most machines. I think Don has a point and the spec should say something like : signed integer overflow is defined as being a runtime error. For performance reasons, the compiler may choose to not emit error checking code and use wraparound semantic instead. Or something along these lines. Oh, I like that! That does seem to be the best of both worlds. Then, as a QOI issue, the compiler can try to detect the error. If it does not detect the error, it MUST provide the two's complement result. It is not allowed to do any weird stuff.
Re: Signed integer overflow undefined behavior or not?
On Friday, 13 November 2015 at 05:47:03 UTC, deadalnix wrote: Signed overflow are defined as well, as wraparound. Can we please, please, please not have that as official policy without carefully thinking through the implications? It is undefined behaviour in C and C++, so we are not constrained by backwards compatibility with existing code. I have never seen an example where signed integer overflow happened, which was not a bug. In my opinion, making it legal is an own goal, an unforced error. Suppose we made it an error. We'd be in a much better position than C. We could easily add a check for integer overflow into CTFE. We could allow compilers and static analysis tools to implement runtime checks for integer overflow, as well. Are we certain that we want to disallow this? At the very least, we should change the terminology on that page. The word "overflow" should not be used when referring to both signed and unsigned types. On that page, it is describing two very different phenomena, and gives the impression that it was written by somebody who does not understand what they are talking about. The usage of the word "wraps" is sloppy. That page should state something like: For any unsigned integral type T, all arithmetic is performed modulo (T.max + 1). Thus, for example, uint.max + 1 == 0. There is no reason to mention the highly misleading word "overflow". For a signed integral type T, T.max + 1 is not representable in type T. Then, we have a choice of either declaring it to be an error, as C does; or stating that the low bits of the infinitely-precise result will be interpreted as a two's complement value. For example, T.max + 1 will be negative. (Note that unlike the unsigned case, there is no simple explanation of what happens). Please let's be precise about this.
Re: DConf 2016, Berlin: Call for Submissions is now open!
On Friday, 23 October 2015 at 16:37:20 UTC, Andrei Alexandrescu wrote: http://dconf.org/2016/index.html Typo: "we're grateful to benefit of their hosting" should be "we're grateful to get the benefit of their hosting" or "we're grateful to benefit from their hosting".
Re: Playing SIMD
On Sunday, 25 October 2015 at 19:37:32 UTC, Iakh wrote: Here is my implementatation of SIMD find. Function returns index of ubyte in static 16 byte array with unique values. [snip] You need to be very careful with doing benchmarks on tiny test cases, they can be very misleading. Be aware that the speed of bsf() and bsr() is very very strongly processor dependent. On some machines, it is utterly pathetic. eg AMD K7, BSR is 23 micro-operations, on original pentium is was up to 73 (!), even on AMD Bobcat it is 11 micro-ops, but on recent Intel it is one micro-op. This fact of 73 can totally screw up your performance comparisons. Just because it is a single machine instruction does not mean it is fast.
Re: opDispatch and compile time parameters
On Wednesday, 21 October 2015 at 12:32:37 UTC, Andrei
Alexandrescu wrote:
On 10/21/2015 07:40 AM, Timon Gehr wrote:
On 10/21/2015 12:55 PM, Andrei Alexandrescu wrote:
On 10/19/15 9:49 PM, Jonathan M Davis wrote:
On Monday, 19 October 2015 at 23:37:09 UTC, Timon Gehr wrote:
This is the worst part:
class C{
int[] x=[1,2,3];
}
void main(){
auto mut=new C;
auto imm=new immutable(C);
assert(imm.x[0]==1);
mut.x[0]=2;
assert(imm.x[0]==2);
}
Oooo. Ouch. Yeah, that pushes it from being a good idea to
disallow this
in order to avoid bugs to a necessity to disallow it in
order to avoid
breaking the type system.
Please file, thanks. -- Andrei
(I've filed it in 2013.
https://issues.dlang.org/show_bug.cgi?id=10376 .)
What should be the expected behaviour?
- Don't allow default initialization with mutable indirections
in
aggregates.
- Clone the mutable referenced data for each instance.
- Have different "init" for different mutability.
- ... ?
The quickest way to stop the bleeding is to disallow the code.
It's incorrect for immutable data and misleading for mutable
data. (What an user might expect is that each data comes with a
distinct array.)
I can't believe I didn't know about this huge hole.
Andrei
Fundamentally the problem is that literals of mutable reference
types do not make sense. This is why I argued (before TDPL came
out), that an explicit .dup should be required, rather than
allowing the compiler to secretly add one automatically. Implicit
conversion of an array literal to mutable is ridiculous IMHO.
This is just one manifestation of the problem. It could be quite
difficult to come up with a rule for what should be disallowed.
Re: Implement the "unum" representation in D ?
On Tuesday, 15 September 2015 at 11:13:59 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 15 September 2015 at 10:38:23 UTC, ponce wrote: On Tuesday, 15 September 2015 at 09:35:36 UTC, Ola Fosheim Grøstad wrote: http://sites.ieee.org/scv-cs/files/2013/03/Right-SizingPrecision1.pdf That's a pretty convincing case. Who does it :)? I'm not convinced. I think they are downplaying the hardware difficulties. Slide 34: Disadvantages of the Unum Format * Non-power-of-two alignment. Needs packing and unpacking, garbage collection. I think that disadvantage is so enormous that it negates most of the advantages. Note that in the x86 world, unaligned memory loads of SSE values still take longer than aligned loads. And that's a trivial case! The energy savings are achieved by using a primitive form of compression. Sure, you can reduce the memory bandwidth required by compressing the data. You could do that for *any* form of data, not just floating point. But I don't think anyone thinks that's worthwhile. The energy comparisons are plain dishonest. The power required for accessing from DRAM is the energy consumption of a *cache miss* !! What's the energy consumption of a load from cache? That would show you what the real gains are, and my guess is they are tiny. So: * I don't believe the energy savings are real. * There is no guarantee that it would be possible to implement it in hardware without a speed penalty, regardless of how many transistors you throw at it (hardware analogue of Amdahl's Law) * but the error bound stuff is cool.
Re: std.data.json formal review
On Tuesday, 28 July 2015 at 22:29:01 UTC, Walter Bright wrote: On 7/28/2015 7:07 AM, Atila Neves wrote: Start of the two week process, folks. Thank you very much, Sönke, for taking this on. Thank you, Atila, for taking on the thankless job of being review manager. Just looking at the documentation only, some general notes: 1. Not sure that 'JSON' needs to be embedded in the public names. 'parseJSONStream' should just be 'parseStream', etc. Name disambiguation, if needed, should be ably taken care of by a number of D features for that purpose. Additionally, I presume that the stdx.data package implies a number of different formats. These formats should all use the same names with as similar as possible APIs - this won't work too well if JSON is embedded in the APIs. 2. JSON is a trivial format, http://json.org/. But I count 6 files and 30 names in the public API. 3. Stepping back a bit, when I think of parsing JSON data, I think: auto ast = inputrange.toJSON(); where toJSON() accepts an input range and produces a container, the ast. The ast is just a JSON value. Then, I can just query the ast to see what kind of value it is (using overloading), and walk it as necessary. To create output: auto r = ast.toChars(); // r is an InputRange of characters writeln(r); So, we'll need: toJSON toChars JSONException The possible JSON values are: string number object (associative arrays) array true false null Since these are D builtin types, they can actually be a simple union of D builtin types. Related to this: it should not be importing std.bigint. Note that if std.bigint were fully implemented, it would be very heavyweight (optimal multiplication of enormous integers involves fast fourier transforms and all kinds of odd stuff, that's really bizarre to pull in if you're just parsing a trivial little JSON config file). Although it is possible for JSON to contain numbers which are larger than can fit into long or ulong, it's an abnormal case. Many apps (probably, almost all) will want to reject such numbers immediately. BigInt should be opt-in. And, it is also possible to have floating point numbers that are not representable in double or real. BigInt doesn't solve that case. It might be adequate to simply present it as a raw number (an unconverted string) if it isn't a built-in type. Parse it for validity, but don't actually convert it.
Re: Enhancement: issue error on all public functions that are missing ddoc sections
On Wednesday, 18 March 2015 at 22:05:18 UTC, Brian Schott wrote: On Wednesday, 18 March 2015 at 18:48:53 UTC, Walter Bright wrote: I'm fed up with this problem. It is actively hurting us every day. https://issues.dlang.org/show_bug.cgi?id=14307 Anyone want to take this on? Shouldn't be particularly difficult. D-Scanner has had this feature for a while. Here's the list for Phobos: http://dpaste.dzfl.pl/7d018aad2b10 That's a very interesting list. Things like this: std/regex/package.d(320:13)[warn]: Public declaration 'regexImpl' is undocumented. appear to be public only as an workaround (necessary for mixins or something). Perhaps such things shouldn't actually be documented. But we don't have a mechanism for that.
Re: Post increment and decrement
On Thursday, 12 March 2015 at 04:06:14 UTC, Rikki Cattermole
wrote:
On 12/03/2015 1:50 p.m., Andrei Alexandrescu wrote:
On 3/11/15 10:23 AM, welkam wrote:
Observation Nr. 1
People prefer to write var++ instead of ++var.
Observation Nr. 2
Because of observation Nr. 1 and other reasons compilers
became good at
removing code that is not needed making var++ and ++var to
produce the
same code if returned value is not used.
Observation Nr. 3
Because of observation Nr. 2 more people use var++ in place
where they
really only need ++var.
Observation Nr. 4
Because of observation Nr. 3 people learning to program may
mistakenly
learn that var++ is just incrementing. (I am included in that
list)
Observation Nr. 5
Because of observation Nr. 4 people can write slower than
necessary code
for classes with overloaded operator or even get bugs.
Because of all this why not make only one increment/decrement
operator
and have post increment/decrement to be called by template
name, because
it is a template?
template post_inc(T) {
auto tmp = T;
T++;
return tmp;
}
Observation Nr. 6
Somebody didn't Read The Fine Manual. Page 369:
=
If the result of a++ is not needed, the rewrite is ++a, which
is
subsequently rewritten to a.opUnary!++().
=
Andrei
+1
Compiler should work for you. This is one of those things it
can rewrite to preference. During optimization.
It doesn't even rely on the optimizer. This happens in the
front-end, in the semantic pass.
Re: C++ to catch up?
On Wednesday, 4 February 2015 at 03:52:26 UTC, Laeeth Isharc wrote: Excellent post. This situation is very obvious to us at Sociomantic, as we're at the forefront of a massive disruption that is happening in the advertising industry. D has far better prospects in disruptive technology, rather than trying to compete with incumbents in the rapidly disappearing traditional desktop market. Thanks, Don. I am honoured that you took the time to read through all of this, and appreciate the feedback. Every now and then I question whether I am headed in the right direction to use D (not because of anything lacking in D, but because it is less conventional, and because I have been away from the pulse of technology for a very long time). Your industry is a little different, and my needs for the time being are not even soft real-time (although that could easily change). But from listening to your talk, I am pretty sure you know what you are doing, and wanting high productivity when dealing with potentially quite respectably sized data sets is one shared aspect - so that is a source of comfort. Thanks! Yes, I think that larger data sets are not well served by existing languages. And ease of handling large data is actually more significant than raw performance. Domains like ours are at least as much I/O bound as CPU-bound, and ability to adapt rapidly is very important. Could I ask you one thing, not directly relating to D? Why did you pick Berlin to launch your startup? (You in the corporate sense, I mean). Perhaps Berlin chose the company, rather than the other way around :) The companies' founders all grew up in East Germany, I think they were just living in Berlin. But, there are a huge number of startups in Berlin. It's a place with great infrastructure, low costs, and available talent. So it's certainly an attractive place to launch a startup. First published in 1997, Christensen's book suggests that successful companies can put too much emphasis on customers' current needs, and fail to adopt new technology or business models that will meet their customers' unstated or future needs -- http://en.wikipedia.org/wiki/The_Innovator%27s_Dilemma I thought: they put too much emphasis on backwards compatibility ... Haha - I know you have been one of the proponents of breaking changes. I think that is a distinct question from the other stuff, and guess it is not easy for the language leaders to balance the different demands - impossible not to make one group unhappy. Someone cynical might say it is easier for you take that position if you are still mostly on D1, and so don't pay the same price others would. Yes, that's true, and so my opinions should be slightly weighted downwards. But even so, the reality is that bugfixes cause breakages anyway. Most code that isn't actively being maintained, is broken already. If you're an early adopter, you expect to have a lot of breakage pain. The thing that is frustrating is when decisions are made as if we were much further along the adoption/disruption cycle, than where we actually are. We don't yet have huge, inflexible users that demand stability at all costs. There was widespread agreement on this, from all of the eight companies at DConf who were using D commercially. Breaking changes aside, one can't say there isn't a sustained dynamism to the development of D. Yes. Though I wonder if we are putting too much emphasis on being a replacement for C++; I fear that the better we become at replacing it, the more we will duplicate its problems. But that's just a niggling doubt rather than a well-reasoned belief.
Re: C++ to catch up?
On Sunday, 1 February 2015 at 23:20:15 UTC, Laeeth Isharc wrote: On Monday, 5 November 2012 at 18:20:23 UTC, Jonathan M Davis wrote: The closer that C++ gets to D, the less interested that many people will be in adopting it, particularly because of the large user base and the large amount of code out there that already uses C++. Programmers have to be convinced to move to D, and for many C++ programmers, the improvements to C++11 are enough to make a move to D not worth it, even if D is a better language. (He goes on to point out that nonetheless D will always have the edge because legacy and installed base). One should be careful about superficial translation of instances from the purely commercial world to the world of languages, but it strikes me that Clayton Christensen's Innovator's Dilemma does apply somewhat to the case of D vs its peer languages. His central point is that in the beginning disruptive innovation very often tends to commence as a niche thing that may well be globally inferior - he uses the example of Honda motorbikes that allowed them to gain a foothold, and that once they dominated this niche and gained succour from it were able to use to expand their footprint to the extent that they posed a serious threat to the established dominant players. But for many years, these (and later the cars) were seen as products of clearly inferior quality that had the advantage of being cheap. The interesting thing is the emotional aspect of perception - nobody would have taken you seriously had you predicted in the early stages that Japanese auto makers would become what they subsequently became. And one could have pointed out some decades after the war ended that they had been in the business for years, and why should anything change. This is exactly what people say about D - it's been around forever and hasn't taken off, so why bother. (see recent Slashdot thread for an example of this). It is a basic insight of gestalt psychology that perception is shaped by emotion (really it's affect, which goes much deeper - emotion is the tip of the affect iceberg), and one way to know when this is occurring (my background is as an investor and speculator, so I have devoted a couple of decades to applying this in a practical way) is that on the one hand you have an emotional intensity out of proportion to the importance of the topic, and on the other the reasons people put forward to justify how they feel are observably not in accordance with the facts. See the Slashdot thread... So in any case, D is not competing on price, but has other strengths that are of very high appeal to a certain group (if you want to write native code in a productive way) even though one must honestly acknowledge its imperfections in a global sense - reading back through the forums a dozen years, this seems to occur quite regularly in waves. When is D going to be finished? even a decade back. To be upset by the imperfections is missing the point, because languages - even programming languages - have a certain innate pattern of development (that resembles Goethe's observations about the metamorphosis of plants) that can't be forced, no matter how much one grumbles or stamps one's feet. Furthermore, people tend to extrapolate superficial trends even though history tells us this is a poor guide to the future. Japanese cars really took off once crude exploded in the early 70s (and again towards the end), and auto-makers were slow to respond. Perhaps they did not organize their business on the basis of a prediction abuot energy prices, but the point is they were ready to take advantage of this shift when it occurred. I do not want to attempt to be a pundit, but it is interesting that the notable use cases of D - at Sociomantic, Adroll, and Facebook are all aligned with certain salient and very powerful underlying technological drivers and trends. It's no longer true in many applications that programmer time is expensive compared to machine time, and large data sets encountering the challenges of memory vs CPU trajectories create new challenges and require new approaches. And it is a positive for D that some of its competition does not take D seriously at this stage - one thinks for example of Guido and his insistence that execution speed ought not to be a factor given work is I/O + network bound, even though this is less true for numerical computing and some kinds of data crunching. (Not that D is mature here, but there is much that can be done within the existing framework). In any case, dissatisfaction channeled in a constructive direction is a positive thing, because it is the opposite of complacency and is the edge of the challenger. The point isn't how people feel, but how they respond to the challenges in front of them. As a newcomer, it is very satisfying to see the progress made on documentation, ecosystem, and C++
Sociomantic: We're looking for a Linux Systems Admin!
It is probably not obvious why our HR department posted this job ad to this newsgroup, particularly to anyone who doesn't know Sociomantic's relationship to the D community. Most of the apps running on our servers, are written in D. The role doesn't involve D programming, and the job ad doesn't even mention D, but it will involve working very closely with our D developers, in supporting the deployment and operation of D code. You can also review the job ad on our company website: https://www.sociomantic.com/jobs/linux-system-administrator/#.VK5_XV3ydwE - Don.
Re: CTFE pow()
On Friday, 2 January 2015 at 08:52:47 UTC, Daniel Murphy wrote: Xinok wrote in message news:pbjttgrpwhsffoovp...@forum.dlang.org... I'm wondering if there's any added benefit to implementing these as intrinsic functions instead? In this way, implementing them in CTFE should be trivial and the extra knowledge given to the compiler could enable extra optimizations that wouldn't be possible otherwise. For example, the compiler understands multiplication and division so, in certain cases, it can replace these operations with bit-shifting which is faster. I was just about to suggest this. Another easy way might be to just add get/set mantissa/exponent intrinsics and use those in the ctfe versions. Right. The problem with allowing casts between integers and floating point, is that then someone can create a pointer to some of the bits of the float. And that's a can of worms. It enables behaviour that is both complicated, and useless. BTW in the existing CTFE you can cast int - float and long - double. So you can implement pow for 64 bit floats already. It's only 80-bit reals that are a problem, because there is no integer type that is large enough. So it's really an X86-specific problem. I did suggest allowing the specific casts for getting and setting the exponent and significand, eg: ushort exponent = (cast(ushort *)cast(void*)f)[4]; ulong significand = *(cast(ulong *)cast(void *)f); which would give you everything you need. It's really an intrinsic with very very ugly syntax. But Iain was pretty unhappy with that idea, IIRC. I think treatment of real is very difficult for GDC already.
Re: Sargon component library now on Dub
On Sunday, 14 December 2014 at 03:26:56 UTC, Walter Bright wrote: http://code.dlang.org/packages/sargon These two modules failed to generate much interest in incorporating them into Phobos, but I'm still rather proud of them :-) So am I, the halffloat is much faster than any other implementation I've seen. The fast path for the conversion functions involves only a few machine instructions. I had an extra speedup for it that made it optimal, but it requires a language primitive to dump excess hidden precision. We still need this, it is a fundamental operation (C tries to do it implicitly using sequence points, but they don't actually work properly). Here they are: ◦sargon.lz77 - algorithms to compress and expand with LZ77 compression algorithm ◦sargon.halffloat - IEEE 754 half-precision binary floating point format binary16 I'll be adding more in the future.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Monday, 24 November 2014 at 21:34:19 UTC, Walter Bright wrote: On 11/24/2014 2:20 AM, Don wrote: I believe I do understand the problem. As a practical matter, overflow checks are not going to be added for performance reasons. The performance overhead would be practically zero. All we would need to do, is restrict array slices such that the length cannot exceed ssize_t.max. This can only happen in the case where the element type has a size of 1, and only in the case of slicing a pointer, concatenation, and memory allocation. (length1 + length2) / 2 That's not an issue with length, that's an issue with doing a calculation with an insufficient bit width. Unsigned doesn't actually help, it's still wrong. For unsigned values, if length1 = length2 = 0x8000_, that gives an answer of 0. In exchange, 99% of uses of unsigned would disappear from D code, and with it, a whole category of bugs. You're not proposing changing size_t, so I believe this statement is incorrect. From the D code that I've seen, almost all uses of size_t come directly from the use of .length. But I concede (see below) that many of them come from .sizeof. Also, in principle, uint-uint can generate a runtime check for underflow (i.e. the carry flag). No it cannot. The compiler does not have enough information to know if the value is intended to be positive integer, or an unsigned. That information is lost from the type system. Eg from C, wrapping of an unsigned type is not an error. It is perfectly defined behaviour. With signed types, it's undefined behaviour. I know it's not an error. It can be defined to be an error, and the compiler can insert a runtime check. (I'm not proposing this, just saying it can be done.) But it can't do that, without turning unsigned into a different type. You'd be turning unsigned into a 'non-negative' which is a completely different type. This is my whole point. unsigned has no sign, you just get the raw bit pattern with no interpretation. This can mean several things, for example: 1. extended_non_negative is where you are using it for the positive range 0.. +0x_ Then, overflow and underflow are errors. 2. a value where the highest bit is always 0. This can be safely used as int or uint. 3. Or, it can be modulo 2^^32 arithmetic, where wrapping is intended. 4. It can be part of extended precision arithmetic, where you want the carry flag. 5. It can be just a raw bit pattern. 6. The high bit can be a sign bit. This is a signed type, cast to uint. If the sign bit ever flips because of a carry, that's an error. The type system doesn't specify a meaning for the bit pattern. We've got a special type for case 6, but not for the others. The problem with unsigned is that since it can mean so many things, as if it were a union of these possibilities. So it's not strictly typed -- you need to careful, requiring some element of faith-based programming. And signed-unsigned mismatch is really where you are implicitly assuming that the unsigned value is case 2 or 6. But, if it is one of the other cases, you get nonsense. But those signed unsigned mismatch errors only catch some of the possible cases where you may forget which interpretation you are using, and act as if it were another one. To make this clear: I am not proposing that size_t should be changed. I am proposing that for .length returns a signed type, that for array slices is guaranteed to never be negative. There'll be mass confusion if .length is not the same type as .sizeof Ah, that is a good point. .sizeof is another source of unsigned. Again, quite unnecessarily, can a single type ever actually use up half of the memory space? (It was possible in the 8 and 16 bit days, but it's hard to imagine today). Even sillier, it is nearly always known at compile time! But still, .sizeof is low-level in a way that .length is not.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Monday, 24 November 2014 at 15:56:44 UTC, Andrei Alexandrescu wrote: On 11/24/14 4:54 AM, Don wrote: In D, 1u - 2u 0u. This is defined behaviour, not an overflow. I think I get what you mean, but overflow is also defined behavior (in D at least). -- Andrei Aargh! You're right. That's new, and dreadful. It didn't used to be. The offending commit is alexrp 2012-05-15 15:37:24 which only provides an unsigned example. Why are defining behaviour that is always a bug? Java makes it defined, but it has to because it doesn't have unsigned types. I think the intention probably was to improve on the C situation, where there is undefined behaviour that really should be defined. But do we really want to preclude ever having overflow checking for integers?
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Friday, 21 November 2014 at 20:17:12 UTC, Walter Bright wrote: On 11/21/2014 7:36 AM, Don wrote: On Friday, 21 November 2014 at 04:53:38 UTC, Walter Bright wrote: 0 crossing bugs tend to show up much sooner, and often immediately. You're missing the point here. The problem is that people are using 'uint' as if it were a positive integer type. Suppose D had a type 'natint', which could hold natural numbers in the range 0..uint.max. Sounds like 'uint', right? People make the mistake of thinking that is what uint is. But it is not. How would natint behave, in the type system? typeof (natint - natint) == int NOT natint !!! This would of course overflow if the result is too big to fit in an int. But the type would be correct. 1 - 2 == -1. But typeof (uint - uint ) == uint. The bit pattern is identical to the other case. But the type is wrong. It is for this reason that uint is not appropriate as a model for positive integers. Having warnings about mixing int and uint operations in relational operators is a bit misleading, because mixing signed and unsigned is not usually the real problem. Instead, those warnings a symptom of a type system mistake. You are quite right in saying that with a signed length, overflows can still occur. But, those are in principle detectable. The compiler could add runtime overflow checks for them, for example. But the situation for unsigned is not fixable, because it is a problem with the type system. By making .length unsigned, we are telling people that if .length is used in a subtraction expression, the type will be wrong. It is the incorrect use of the type system that is the underlying problem. I believe I do understand the problem. As a practical matter, overflow checks are not going to be added for performance reasons. The performance overhead would be practically zero. All we would need to do, is restrict array slices such that the length cannot exceed ssize_t.max. This can only happen in the case where the element type has a size of 1, and only in the case of slicing a pointer, concatenation, and memory allocation. Making this restriction would have been unreasonable in the 8 and 16 bit days, but D doesn't support those. For 32 bits, this is an extreme corner case. For 64 bit, this condition never happens at all. In exchange, 99% of uses of unsigned would disappear from D code, and with it, a whole category of bugs. Also, in principle, uint-uint can generate a runtime check for underflow (i.e. the carry flag). No it cannot. The compiler does not have enough information to know if the value is intended to be positive integer, or an unsigned. That information is lost from the type system. Eg from C, wrapping of an unsigned type is not an error. It is perfectly defined behaviour. With signed types, it's undefined behaviour. To make this clear: I am not proposing that size_t should be changed. I am proposing that for .length returns a signed type, that for array slices is guaranteed to never be negative.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Friday, 21 November 2014 at 17:23:51 UTC, Marco Leise wrote:
Am Thu, 20 Nov 2014 08:18:23 +
schrieb Don x...@nospam.com:
It's particularly challenging in D because of the widespread
use of 'auto':
auto x = foo();
auto y = bar();
auto z = baz();
if (x - y z) { ... }
This might be a bug, if one of these functions returns an
unsigned type. Good luck finding that. Note that if all
functions return unsigned, there isn't even any
signed-unsigned mismatch.
With those function names I cannot write code.
ℕ x = length();
ℕ y = index();
ℕ z = requiredRange();
if (x - y z) { ... }
Ah, now we're getting somewhere. Yes the code is obviously
correct. You need to be aware of the value ranges of your
variables and write subtractions in a way that the result can
only be = 0. If you realize that you cannot guarantee that
for some case, you just found a logic bug. An invalid program
state that you need to assert/if-else/throw.
Yup. And that is not captured in the type system.
I don't get why so many APIs return ints. Must be to support
Java or something where proper unsigned types aren't available.
D and C do not have suitable types either.
unsigned != ℕ.
In D, 1u - 2u 0u. This is defined behaviour, not an overflow.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Friday, 21 November 2014 at 08:46:20 UTC, Walter Bright wrote: On 11/21/2014 12:10 AM, bearophile wrote: Walter Bright: All you're doing is trading 0 crossing for 0x7FFF crossing issues, and pretending the problems have gone away. I'm not pretending anything. I am asking in practical programming what of the two solutions leads to leas problems/bugs. So far I've seen the unsigned solution and I've seen it's highly bug-prone. I'm suggesting that having a bug and detecting the bug are two different things. The 0-crossing bug is easier to detect, but that doesn't mean that shifting the problem to 0x7FFF crossing bugs is making the bug count less. BTW, granted the 0x7FFF problems exhibit the bugs less often, but paradoxically this can make the bug worse, because then it only gets found much, much later in supposedly tested robust code. Is this true? Do you have some examples of buggy code? http://googleresearch.blogspot.com/2006/06/extra-extra-read-all-about-it-nearly.html Changing signed to unsigned in that example does NOT fix the bug. It just means it fails with length = 2^^31 instead of length = 2^^30. uint a = 0x8000_u; uint b = 0x8000_0002u; assert( (a + b) /2 == 0); But actually I don't understand that article. The arrays are int, not char. Since length fits into 32 bits, the largest possible value is 2^^32-1. Therefore, for an int array, with 4 byte elements, the largest possible value is 2^^30-1. So I think the article is wrong. I don't think there is a bug in the code.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Monday, 24 November 2014 at 16:00:53 UTC, ketmar via Digitalmars-d wrote: On Mon, 24 Nov 2014 12:54:58 + Don via Digitalmars-d digitalmars-d@puremagic.com wrote: In D, 1u - 2u 0u. This is defined behaviour, not an overflow. this *is* overflow. D just has overflow result defined. No, that is not overflow. That is a carry. Overflow is when the sign bit changes.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Friday, 21 November 2014 at 04:53:38 UTC, Walter Bright wrote: On 11/20/2014 7:11 PM, Walter Bright wrote: On 11/20/2014 3:25 PM, bearophile wrote: Walter Bright: If that is changed to a signed type, then you'll have a same-only-different set of subtle bugs, This is possible. Can you show some of the bugs, we can discuss them, and see if they are actually worse than the current situation. All you're doing is trading 0 crossing for 0x7FFF crossing issues, and pretending the problems have gone away. BTW, granted the 0x7FFF problems exhibit the bugs less often, but paradoxically this can make the bug worse, because then it only gets found much, much later in supposedly tested robust code. 0 crossing bugs tend to show up much sooner, and often immediately. You're missing the point here. The problem is that people are using 'uint' as if it were a positive integer type. Suppose D had a type 'natint', which could hold natural numbers in the range 0..uint.max. Sounds like 'uint', right? People make the mistake of thinking that is what uint is. But it is not. How would natint behave, in the type system? typeof (natint - natint) == int NOT natint !!! This would of course overflow if the result is too big to fit in an int. But the type would be correct. 1 - 2 == -1. But typeof (uint - uint ) == uint. The bit pattern is identical to the other case. But the type is wrong. It is for this reason that uint is not appropriate as a model for positive integers. Having warnings about mixing int and uint operations in relational operators is a bit misleading, because mixing signed and unsigned is not usually the real problem. Instead, those warnings a symptom of a type system mistake. You are quite right in saying that with a signed length, overflows can still occur. But, those are in principle detectable. The compiler could add runtime overflow checks for them, for example. But the situation for unsigned is not fixable, because it is a problem with the type system. By making .length unsigned, we are telling people that if .length is used in a subtraction expression, the type will be wrong. It is the incorrect use of the type system that is the underlying problem.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Friday, 21 November 2014 at 15:50:05 UTC, H. S. Teoh via Digitalmars-d wrote: On Fri, Nov 21, 2014 at 03:36:01PM +, Don via Digitalmars-d wrote: [...] Suppose D had a type 'natint', which could hold natural numbers in the range 0..uint.max. Sounds like 'uint', right? People make the mistake of thinking that is what uint is. But it is not. How would natint behave, in the type system? typeof (natint - natint) == int NOT natint !!! Wrong. (uint.max - 0) == uint.max, which is of type uint. It is not uint.max. It is natint.max. And yes, that's an overflow condition. Exactly the same as when you do int.max + int.max. If you interpret it as int, you get a negative number, which is wrong. So your proposal breaks uint in even worse ways, in that now subtracting a smaller number from a larger number may overflow, whereas it wouldn't before. So that fixes nothing, you're just shifting the problem somewhere else. T This is not a proposal I am just illustrating the difference between what people *think* uint does, vs what it actually does. The type that I think would be useful, would be a number in the range 0..int.max. It has no risk of underflow. To put it another way: natural numbers are a subset of mathematical integers. (the range 0..infinity) signed types are a subset of mathematical integers (the range -int.max .. int.max). unsigned types are not a subset of mathematical integers. They do not just have a restricted range. They have different semantics. The question of what happens when a range is exceeded, is a different question.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Wednesday, 19 November 2014 at 17:55:26 UTC, Andrei
Alexandrescu wrote:
On 11/19/14 6:04 AM, Don wrote:
Almost everybody seems to think that unsigned means positive.
It does not.
That's an exaggeration. With only a bit of care one can use D's
unsigned types for positive numbers. Please let's not reduce
the matter to black and white.
Andrei
Even in the responses in this thread indicate that about half of
the people here don't understand unsigned.
unsigned means I want to use modulo 2^^n arithmetic. It does
not mean, this is an integer which cannot be negative.
Using modulo 2^^n arithmetic is *weird*. If you are using
uint/ulong to represent a non-negative integer, you are using the
incorrect type.
With only a bit of care one can use D's unsigned types for
positive numbers.
I do not believe that that statement to be true. I believe that
bugs caused by unsigned calculations are subtle and require an
extraordinary level of diligence. I showed an example at DConf,
that I had found in production code.
It's particularly challenging in D because of the widespread use
of 'auto':
auto x = foo();
auto y = bar();
auto z = baz();
if (x - y z) { ... }
This might be a bug, if one of these functions returns an
unsigned type. Good luck finding that. Note that if all
functions return unsigned, there isn't even any signed-unsigned
mismatch.
I believe the correct statement, is With only a bit of care one
can use D's unsigned types for positive numbers and believe that
one's code is correct, even though it contains subtle bugs.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Tuesday, 18 November 2014 at 18:23:52 UTC, Marco Leise wrote: Am Tue, 18 Nov 2014 15:01:25 + schrieb Frank Like 1150015...@qq.com: but now ,'int' is enough for use,not huge and not small,only enough. 'int' is easy to write,and most people are used to it. Most importantly easier to migrate code,if 'length''s return value type is 'int'. How about your idea? I get the idea of a broken record right now... Clearly size_t (which I tend to alias with ℕ in my code for brevity and coolness) can express more than 2^31-1 items, which is appropriate to reflect the increase in usable memory per application on 64-bit platforms. Yes, the 64-bit version of a program or library can handle larger data sets. Just like it was when people transitioned from 16-bit to 32-bit. I wont use `int` just because the technically correct thing is `size_t`, even it it is a little harder to type. This is difficult. Having arr.length return an unsigned type, is a dreadful language mistake. Aside from the size factor, I personally prefer unsigned types for countable stuff like array lengths. Mixed arithmetics decay to unsinged anyways and you don't need checks like `assert(idx = 0)`. It is a matter of taste though and others prefer languages with no unsigned types at all. No! No! No! This is completely wrong. Unsigned does not mean positive. It means no sign, and therefore wrapping semantics. eg length - 4 0, if length is 2. Weird consequence: using subtraction with an unsigned type is nearly always a bug. I wish D hadn't called unsigned integers 'uint'. They should have been called '__uint' or something. They should look ugly. You need a very, very good reason to use an unsigned type. We have a builtin type that is deadly but seductive.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Wednesday, 19 November 2014 at 11:43:38 UTC, John Colvin wrote: On Wednesday, 19 November 2014 at 11:04:05 UTC, Matthias Bentrup wrote: On Wednesday, 19 November 2014 at 10:03:35 UTC, Don wrote: On Tuesday, 18 November 2014 at 18:23:52 UTC, Marco Leise wrote: Am Tue, 18 Nov 2014 15:01:25 + schrieb Frank Like 1150015...@qq.com: but now ,'int' is enough for use,not huge and not small,only enough. 'int' is easy to write,and most people are used to it. Most importantly easier to migrate code,if 'length''s return value type is 'int'. How about your idea? I get the idea of a broken record right now... Clearly size_t (which I tend to alias with ℕ in my code for brevity and coolness) can express more than 2^31-1 items, which is appropriate to reflect the increase in usable memory per application on 64-bit platforms. Yes, the 64-bit version of a program or library can handle larger data sets. Just like it was when people transitioned from 16-bit to 32-bit. I wont use `int` just because the technically correct thing is `size_t`, even it it is a little harder to type. This is difficult. Having arr.length return an unsigned type, is a dreadful language mistake. Aside from the size factor, I personally prefer unsigned types for countable stuff like array lengths. Mixed arithmetics decay to unsinged anyways and you don't need checks like `assert(idx = 0)`. It is a matter of taste though and others prefer languages with no unsigned types at all. No! No! No! This is completely wrong. Unsigned does not mean positive. It means no sign, and therefore wrapping semantics. eg length - 4 0, if length is 2. Weird consequence: using subtraction with an unsigned type is nearly always a bug. I wish D hadn't called unsigned integers 'uint'. They should have been called '__uint' or something. They should look ugly. You need a very, very good reason to use an unsigned type. We have a builtin type that is deadly but seductive. int has wrapping the same semantics too, only it wraps to negative numbers instead of zero. No. Signed types do not *wrap*. They *overflow* if their range is exceeded. This is not the same thing. Overflow is always an error. And the compiler could insert checks to detect this. That's not possible for unsigned types. With an unsigned type, wrapping is part of the semantics. Moreover, hitting an overflow with a signed type is an exceptional situation. Wrapping with an unsigned type is entirely normal, and happens with things like 2u - 1u. If you insist on non-wrapping length, it should return double or long double. Which would be totally wrong for different reasons. Short of BigInts or overflow-checking, there is no perfect option. An overflow-checked type that could be reasonably well optimised would be nice, as mentioned by bearophile many times. I don't think we need to worry about the pathological cases. The problem with unsigned size_t is that it introduces inappropriate semantics everywhere for the sake of the pathological cases. IMHO the correct solution is to say that the length of a slice cannot exceed half of the memory space, otherwise a runtime error will occur. And then make size_t a positive integer. Then let typeof(size_t - size_t) == int, instead of uint. All other operations stay as size_t. Perhaps we can get most of the way, by improving range propagation.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Tuesday, 18 November 2014 at 18:03:35 UTC, ketmar via Digitalmars-d wrote: On Tue, 18 Nov 2014 17:59:04 + David Eagen via Digitalmars-d digitalmars-d@puremagic.com wrote: Isn't the purpose of size_t is to be large enough to address all available memory? A negative value is not only too small but doesn't make sense when discussing lengths. Correctness requires using size_t. yes. besides, there is no such thing as negative length, so it's somewhat... weird to use signed integer for length. A length can certainly be negative. Just as a bank balance can be negative. It's just a number. If I have two pencils of length 10 cm and 15 cm, then the first one is -5 cm longer than the other. Of course any physical pencil is always of positive length, but that doesn't mean that typeof(pencil.length) can never be negative.
Re: 'int' is enough for 'length' to migrate code from x86 to x64
On Wednesday, 19 November 2014 at 13:47:31 UTC, ketmar via Digitalmars-d wrote: On Wed, 19 Nov 2014 13:33:21 + Don via Digitalmars-d digitalmars-d@puremagic.com wrote: No. Signed types do not *wrap*. They *overflow* if their range is exceeded. same for unsigned ints. This is not the same thing. Overflow is always an error. And the compiler could insert checks to detect this. and for unsigned ints. i want compilers to has special code for this. something like `checkedInt(...)`. and this must be built-in, 'cause checking carry flag is cheap, but can be done only on machine level. I don't know what you mean. For unsigned ints, carry is not an error. That's the whole point of unsigned! That's not possible for unsigned types. With an unsigned type, wrapping is part of the semantics. see above. Moreover, hitting an overflow with a signed type is an exceptional situation. Wrapping with an unsigned type is entirely normal, and happens with things like 2u - 1u. having results of unsigned int wrapping defined doesn't mean that it's normal. it's just *defined*, so you can check for it without triggering UB. IMHO the correct solution is to say that the length of a slice cannot exceed half of the memory space, otherwise a runtime error will occur. And then make size_t a positive integer. but why? maybe 1/3 of address space fits better? or 256 bytes, to really avoid overflows and wrapping? No. The point is to get correct semantics. Unsigned types do not have the correct semantics. Signed types do. Then let typeof(size_t - size_t) == int, instead of uint. All other operations stay as size_t. check and cast. you can check length and then safely cast it to int, no probs. This is the job of the compiler, not the programmer. The compiler should do this at all possible places where a slice could exceed the int.max / long.max. That's cheap because there are hardly any places it could happen (for example, for array slices it can only happen with 1-byte types). --- Almost everybody seems to think that unsigned means positive. It does not. ---
Re: convert static arrays to dynamic arrays and return, have wrong data.
On Sunday, 9 November 2014 at 15:09:10 UTC, H. S. Teoh via Digitalmars-d wrote: On Sun, Nov 09, 2014 at 08:29:58AM +, AlanThinker via Digitalmars-d wrote: It seems that, D's array is strange, It can implicit convert static arrays to dynamic arrays no error and no warning. But when I return the converted arrays out the function. Outside function will get some wrong data. It may very easily cause some bug because no error when convert static arrays to dynamic arrays. [...] Yes, this is a known problem. There may even be an issue filed in bugzilla about it (if not, please file one!). The problem is that local static arrays are allocated on the stack, and the implicit conversion to dynamic array is simply taking a slice of the stack-allocated array. As a result, after the function returns, the slice is now pointing at stack memory that has gone out of scope. I'm not sure if the current compiler issues a warning / error if you do this in @safe code, but IMO it should do this even in @system code since the implicit conversion is almost never correct. T The problem is, that you need to be able to take a slice of a stack-allocked array (otherwise stack allocated arrays are useless). Eg you should be able to pass a slice of a stack array to writefln(). Detecting if the slice is returned, requires flow analysis. Currently the front-end doesn't do any flow analysis at all, except for a couple of special cases like closures and super() calls.
Re: UFCS in C++
On Tuesday, 14 October 2014 at 06:29:01 UTC, Jacob Carlborg wrote:
On 14/10/14 03:55, Steven Schveighoffer wrote:
The concept is in D1 for arrays since as long as I've ever
used D.
As far as I recall, it was an accidental feature of arrays and
perhaps associative arrays. Might be a bit hard to track down
that.
It wasn't accidental. It was one of the classic D easter eggs. It
was commented in the source, but wasn't documented anywhere or
mentioned when it was released.
It wasn't documented for at least a year after it was implemented.
BTW the greatest easter egg of them all was the template syntax,
class Bar(T) {..}
Previously you had to write template(T) { class Bar {} }, someone
discovered it and then Walter confessed to having done it. Those
were the days...
Re: Make const, immutable, inout, and shared illegal as function attributes on the left-hand side of a function
On Friday, 10 October 2014 at 02:38:42 UTC, Walter Bright wrote: On 10/9/2014 1:50 AM, Martin Nowak wrote: Kenji just proposed a slightly controversial pull request so I want to reach out for more people to discuss it's tradeoffs. It's about deprecating function qualifiers on the left hand side of a function. This has come up before, and has been debated at length. const is used both as a storage class and as a type constructor, and is distinguished by the grammar: const(T) v; // type constructor, it affects the type T const T v; // storage class, affects the symbol v and the type of v In particular, const T *v; does not mean: const(T)* v; For functions, const-as-storage-class applies to the function symbol. And if it is misused, the compiler will very likely complain about a mismatched type. Breaking this adds a special case inconsistency, besides breaking existing code. (I understand that there's a lot of advocacy lately about break my code, but I'm the one who bears the brunt of you guys broke my code again, even though the code was correct and worked perfectly well! D sux., besides, of course, those poor souls who have to go fix their code base, and I hear again about how D is unstable, another Reddit flame-fest about D being unsuitable because the designers can't make up their mind, etc.) None of those professional complainers matter though. They'll always find *something* to complain about. This is an excellent example of a breaking change that pays for itself within weeks. A large codebase can be converted over very quickly, without any thought required. It has the *immediate* benefit that the coding style improves. It has the longer term benefit of removing a lot of confusion. This endless search for the ideal syntax is consuming our time while we aren't working on issues that matter. (And this change will consume users' time, too, not just ours.) If we're going to break things, it needs to be for something that matters. This doesn't make the cut. No. It's a removal of one of those little friction points, that hurts everyone very slightly, all the time. One less thing to worry about, one less thing to explain, one less thing to be confused by. If you have an organisation with 50 people, every one of them benefits slightly. In aggregate, that's a big deal.
Re: What are the worst parts of D?
On Wednesday, 8 October 2014 at 21:07:24 UTC, Walter Bright wrote: On 10/6/2014 11:13 AM, Dicebot wrote: Especially because you have stated that previous proposal (range-fication) which did fix the issue _for me_ is not on the table anymore. I think it's more stalled because of the setExtension controversy. How about someone starts paying attention to what Don posts? That could be an incredible start. I don't always agree with Don, but he's almost always right and his last post was almost entirely implemented. Wow, thanks, Walter! I'm wrong pretty regularly, though. A reasonable rule of thumb is to ask Daniel Murphy, aka yebblies. If he disagrees with me, and I can't change his mind within 30 minutes, you can be certain that I'm wrong. g
Re: What are the worst parts of D?
On Monday, 6 October 2014 at 19:07:40 UTC, Andrei Alexandrescu wrote: On 10/6/14, 11:55 AM, H. S. Teoh via Digitalmars-d wrote: On Mon, Oct 06, 2014 at 06:13:41PM +, Dicebot via Digitalmars-d wrote: On Monday, 6 October 2014 at 16:06:04 UTC, Andrei Alexandrescu wrote: [...] It would be terrific if Sociomantic would improve its communication with the community about their experience with D and their needs going forward. How about someone starts paying attention to what Don posts? That could be an incredible start. I spend great deal of time both reading this NG (to be aware of what comes next) and writing (to express both personal and Sociomantic concerns) and have literally no idea what can be done to make communication more clear. I don't remember who it was, but I'm pretty sure *somebody* at Sociomantic has stated clearly their request recently: Please break our code *now*, if it helps to fix language design issues, rather than later. More particulars would be definitely welcome. I should add that Sociomantic has an interesting position: it's a 100% D shop so interoperability is not a concern for them, and they did their own GC so GC-related improvements are unlikely to make a large difference for them. So C++ and GC is likely not to be high priority for them. -- Andrei Exactly. C++ support is of no interest at all, and GC is something we contribute to, rather than something we expect from the community. Interestingly we don't even care much about libraries, we've done everything ourselves. So what do we care about? Mainly, we care about improving the core product. In general I think that in D we have always suffered from spreading ourselves too thin. We've always had a bunch of cool new features that don't actually work properly. Always, the focus shifts to something else, before the previous feature was finished. At Sociomantic, we've been successful in our industry using only the features of D1. We're restricted to using D's features from 2007!! Feature-wise, practically nothing from the last seven years has helped us! With something like C++ support, it's only going to win companies over when it is essentially complete. That means that working on it is a huge investment that doesn't start to pay for itself for a very long time. So although it's a great goal, with a huge potential payoff, I don't think that it should be consuming a whole lot of energy right now. And personally, I doubt that many companies would use D, even if with perfect C++ interop, if the toolchain stayed at the current level. As I said in my Dconf 2013 talk -- I advocate a focus on Return On Investment. I'd love to see us chasing the easy wins.
Re: What are the worst parts of D?
On Thursday, 25 September 2014 at 00:52:25 UTC, Walter Bright wrote: On 9/24/2014 7:56 AM, Don wrote: For example: We agreed *years* ago to remove the NCEG operators. Why haven't they been removed yet? They do generate a warning if compiled with -w. They should be gone completely. So should built-in sort. I think there's something important that you haven't grasped yet. It was something I didn't really appreciate before working here. ** Keeping deprecated features alive is expensive. ** As long as deprecated features still exist, they impose a cost. Not just on the language maintainers, but also on the users. On anyone writing a language parser - so for example on text editors. On anyone training new employees. And there's a little cognitive burden on all language users. What change in particular? I've got a nasty feeling that you misread what he wrote. Every time we say, breaking changes are good, you seem to hear breaking changes are bad! It would be helpful having a list of what breaking changes you had in mind. C-style declarations. Builtin sort and reverse. NCEG operators. Built-in complex types. float.min. @property. Basically, anything where it has been decided that it should be removed. Some of these things have been hanging around for six years. I'd also like to see us getting rid of those warts like assert(float.nan) being true. And adding a @ in front of pure, nothrow. Ask yourself, if D had no users other than you, so that you break *anything*, what would you remove? Make a wishlist, and then find out what's possible. Remember, when you did that before, we successfully got rid of 'bit', and there was practically no complaint. Any breaking change where it fails to compile, and where there's an essentially mechanical solution, are really not a problem. Subtle changes in semantics are the ones that are disastrous. We want to pay the one-off cost of fixing our code, so that we can get the long term return-on-investment of a cleaner language.
Re: Library Typedefs are fundamentally broken
On Tuesday, 23 September 2014 at 16:01:35 UTC, Andrei Alexandrescu wrote: On 9/23/14, 7:43 AM, Don wrote: On Monday, 22 September 2014 at 14:56:26 UTC, Andrei Alexandrescu wrote: On 9/22/14, 2:39 AM, Don wrote: Yes, but you're advocating a hack. Oh but I very much disagree. Now you are scaring me. It worries me that this kind of solution can be viewed as acceptable. It's the kind of hacky code I left C++ to escape from. Hmm... doesn't strike me as similar to C++-specific hacks, but I understand the sentiment. People in this thread said it was ugly and you dismissed that. Nononononono. I totally agree some may find it ugly! It's unusable I have a problem with. I think that unusable has been used with two meanings in this thread. That's why I've been using the word hack. But this isn't just a matter of personal aesthetics. If you want something objective, it's not DRY, and it's verbose in a non-trivial way. The hacky design leads to error-prone code. eg you can easily get a copy-paste bug because it's not DRY. alias HMENU = Typedef!(void*, __MODULE__ ~ .HMENU); alias HFONT = Typedef!(void*, __MODULE__ ~ .HMENU); // oops mixin(makeTypedef!(HMENU, void*)); mixin(makeTypedef!(HFONT, void*)); I said and I repeat: I do agree it's less convenient than a baked-in facility. Even with the proper default arguments and all. But I stand by Typedef as an adequate abstraction. You need a very, very good reason to require a string mixin in user code. I'm not seeing that here. The cure is worse than the disease. How many libraries did you use that came with no idioms for their usage? Describing this as an idiom is extremely generous. My standards are higher. Well extremely generous is not deluded so I'll take that :o). And it does seem to me, that because it isn't possible to do a proper library typedef, you've attempted to redefine what a Typedef is supposed to do. And sure, it you remove the requirement to create a unique type, Typedef isn't broken. You're two paragraphs away from library Typedefs are fundamentally broken. Now which one is it? Phobos' Typedef is fundamentally broken, and that your claim that it is not, relies on moving the goalposts. I disagree. I'm not one to dismiss good arguments. But there aren't many here. There's gotta be a point at which you'll agree the whole argument against Typedef has no legs. It's working as designed (I forgot who designed it), the design fits the purpose, the semantics never surprised me, and when people now come with the pitchforks that it's broken, all I can do is send them to the manual. IT WORKS. The argument is that if you use Typedef for real-world use cases, your code is broken unless you use an unintuitive hack. The OP was proof that this is actually happening. I think your starting point is wrong. The design does *not* fit the purpose. We got Typedef to appease objections to 'typedef' being removed from the language. And it did had the effect of silencing the critics. We all expected Typedef to be a drop-in replacement for typedef, not something with dangerously different semantics. Now, if, right from the beginning, you never expected Typedef to replace typedef, then I can see why you think that Typedef is not broken. (But in that case I have no idea what you thought Typedef would be used for). Typedef solves the wrong problem, and solves it well. But when you try to use it to solve the right problem, you have to use an unintuitive hack. But then it isn't very useful, either. You can't, for example, use it to define the various Windows HANDLEs (HMENU, etc), which was one of the most successful use cases for D1's typedef. alias HMENU = Typedef!(void*, __MODULE__ ~ .HMENU); So please s/can't/can't the same exact way built-in typedef would have done it/. No. You can hammer nails in using a rock, but I'm not prepared to accept a rock as a kind of hammer. It's not a tool that belongs in any toolbox. My assertion is, there are no use cases for Phobos's Typedef. You're always better off doing something else. But your evidence destroys your own assertion. Let me explain. You bring the typo example as the smoking gun. So I take it it's a biggie that, if fixed, would make you happy. Not really. I showed that example simply to illustrate that the complaint this is ugly is more than an personal preference. It's ugly because it's a hack. But there are a number of trivial fixes to it, such as my defineTypedef above. So it looks like (a) Typedef can be used as long as you are careful to not type the wrong name, (b) with only trivial work, Typedef can be used without even the slightest repetition. So how come Typedef is unusable when it's usable by your own testimony? I have never said it couldn't be used. I said that it's usable, in the same way that a rock is usable as a hammer. As a substitute for a built-in typedef, it's not a library solution, it's
Re: What are the worst parts of D?
On Wednesday, 24 September 2014 at 07:43:49 UTC, Walter Bright wrote: On 9/23/2014 11:28 PM, Manu via Digitalmars-d wrote: 1. Constant rejection of improvements because OMG breaking change!. Meanwhile, D has been breaking my code on practically every release for years. I don't get this, reject changes that are deliberately breaking changes which would make significant improvements, but allow breaking changes anyway because they are bug fixes? If the release breaks code, then accept that fact and make some real proper breaking changes that make D substantially better! It is my opinion that D adopters don't adopt D because it's perfect just how it is and they don't want it to improve with time, they adopt D *because they want it to improve with time*! That implies an acceptance (even a welcoming) of breaking changes. I agree completely. I would say that the #1 problem in D is the paranoid fear of breaking backwards compatibility. I said that in my 2013 talk. It is still true today. Sociomantic says, PLEASE BREAK OUR CODE! Get rid of the old design bugs while we still can. For example: We agreed *years* ago to remove the NCEG operators. Why haven't they been removed yet? As I said earlier in the year, one of the biggest ever breaking changes was the fix for array stomping, but it wasn't even recognized as a breaking change! Breaking changes happen all the time, and the ones that break noisily are really not a problem. Most D code is yet to be written. What change in particular? I've got a nasty feeling that you misread what he wrote. Every time we say, breaking changes are good, you seem to hear breaking changes are bad! The existing D corporate users are still sympathetic to breaking changes. We are giving the language an extraordinary opportunity. And it's incredibly frustrating to watch that opportunity being wasted due to paranoia. We are holding the door open. But we can't hold it open forever, the more corporate users we get, the harder it becomes. Break our code TODAY. Most D code is yet to be written.
Re: Library Typedefs are fundamentally broken
On Monday, 22 September 2014 at 14:56:26 UTC, Andrei Alexandrescu wrote: On 9/22/14, 2:39 AM, Don wrote: On Sunday, 21 September 2014 at 18:09:26 UTC, Andrei Alexandrescu wrote: On 9/21/14, 8:29 AM, ketmar via Digitalmars-d wrote: On Sun, 21 Sep 2014 08:15:29 -0700 Andrei Alexandrescu via Digitalmars-d digitalmars-d@puremagic.com wrote: alias Int1 = Typedef!(int, a.Int1); alias Int2 = Typedef!(int, b.Int2); ah, now that's cool. module system? wut? screw it, we have time-proven manual prefixing! Use __MODULE__. -- Andrei Yes, but you're advocating a hack. Oh but I very much disagree. Now you are scaring me. It worries me that this kind of solution can be viewed as acceptable. It's the kind of hacky code I left C++ to escape from. People in this thread said it was ugly and you dismissed that. But this isn't just a matter of personal aesthetics. If you want something objective, it's not DRY, and it's verbose in a non-trivial way. The hacky design leads to error-prone code. eg you can easily get a copy-paste bug because it's not DRY. alias HMENU = Typedef!(void*, __MODULE__ ~ .HMENU); alias HFONT = Typedef!(void*, __MODULE__ ~ .HMENU); // oops The original premise does seem to be correct: library Typedefs are fundamentally broken. The semantics of templates does not match what one expects from a typedef: ie, declaring a new, unique type. If you have to pass __MODULE__ in, it's not really a library solution. The user code needs to pass in a nasty implementation detail in order to get a unique type. How many libraries did you use that came with no idioms for their usage? Describing this as an idiom is extremely generous. My standards are higher. And it does seem to me, that because it isn't possible to do a proper library typedef, you've attempted to redefine what a Typedef is supposed to do. And sure, it you remove the requirement to create a unique type, Typedef isn't broken. You're two paragraphs away from library Typedefs are fundamentally broken. Now which one is it? Phobos' Typedef is fundamentally broken, and that your claim that it is not, relies on moving the goalposts. But then it isn't very useful, either. You can't, for example, use it to define the various Windows HANDLEs (HMENU, etc), which was one of the most successful use cases for D1's typedef. alias HMENU = Typedef!(void*, __MODULE__ ~ .HMENU); So please s/can't/can't the same exact way built-in typedef would have done it/. No. You can hammer nails in using a rock, but I'm not prepared to accept a rock as a kind of hammer. It's not a tool that belongs in any toolbox. My assertion is, there are no use cases for Phobos's Typedef. You're always better off doing something else.
Re: Library Typedefs are fundamentally broken
On Sunday, 21 September 2014 at 18:09:26 UTC, Andrei Alexandrescu wrote: On 9/21/14, 8:29 AM, ketmar via Digitalmars-d wrote: On Sun, 21 Sep 2014 08:15:29 -0700 Andrei Alexandrescu via Digitalmars-d digitalmars-d@puremagic.com wrote: alias Int1 = Typedef!(int, a.Int1); alias Int2 = Typedef!(int, b.Int2); ah, now that's cool. module system? wut? screw it, we have time-proven manual prefixing! Use __MODULE__. -- Andrei Yes, but you're advocating a hack. The original premise does seem to be correct: library Typedefs are fundamentally broken. The semantics of templates does not match what one expects from a typedef: ie, declaring a new, unique type. If you have to pass __MODULE__ in, it's not really a library solution. The user code needs to pass in a nasty implementation detail in order to get a unique type. And it does seem to me, that because it isn't possible to do a proper library typedef, you've attempted to redefine what a Typedef is supposed to do. And sure, it you remove the requirement to create a unique type, Typedef isn't broken. But then it isn't very useful, either. You can't, for example, use it to define the various Windows HANDLEs (HMENU, etc), which was one of the most successful use cases for D1's typedef. Having said that, though, the success of 'alias this' does raise some interesting questions about how useful the concept of a typedef is. Certainly it's much less useful than when Typedef was created. My feeling is that almost every time when you want to create a new type from an existing one, you actually want to restrict the operations which can be performed on it. (Eg if you have typedef money = double; then money*money doesn't make much sense). For most typedefs I think you're better off with 'alias this'.
Re: 438-byte Hello, world Win32 EXE in D
On Wednesday, 10 September 2014 at 13:53:32 UTC, Marco Leise wrote: Am Tue, 09 Sep 2014 10:20:43 + schrieb Don x...@nospam.com: On Monday, 8 September 2014 at 08:18:32 UTC, Ola Fosheim Grøstad wrote: On Monday, 8 September 2014 at 08:08:23 UTC, Kagamin wrote: But that downloaded file is bloatware, because it has to implement functionality, which is not provided by the system. That tiny pe file doesn't download anything, it's completely done by the system. Yeah… http://stackoverflow.com/questions/284797/hello-world-in-less-than-20-bytes My personal best -- At my first job, a customer once made a request for a very simple DOS utility. They did mention that they didn't have much disk space on their machine, so they asked me to try to make the program small. That was a once-in-a-lifetime opportunity. Naturally, I wrote it in asm. The final executable size was 15 bytes. g The customer loved it. Vladimir: Good job! Don: Nice story. What did it do? It blanked the screen in a particular way. It was purely for aesthetic reasons. During my time at a vocation school I wrote some stuff like a tiny Windows media player with some of the ASM in the DOS/PE header area. And an animated GIF player in ASM as a .com executable with the GIF included in it. (Easy since GIF algorithms are 16-bit and they use 8-bit color palettes) Nice. That was the only time I ever made a commercial release that was entirely in asm. It only took me about ten minutes to write. It would have been far more difficult in another language. On Wednesday, 10 September 2014 at 14:17:25 UTC, ketmar via Digitalmars-d-announce wrote: On Wed, 10 Sep 2014 16:02:01 +0200 Marco Leise via Digitalmars-d-announce digitalmars-d-announce@puremagic.com wrote: The final executable size was 15 bytes. g The customer loved it. and they never knows that it took at least 512 bytes anyway. or even more, depending of claster size. heh. Yeah. Plus the filename took up almost as much space as the executable code. But when they said they wanted it to be small, they actually meant less than 2 megabytes. When our sales guy saw it, he said, You got it down to 15kb? That's incredible! But I won't pollute D.announce any more. :)
Re: 438-byte Hello, world Win32 EXE in D
On Monday, 8 September 2014 at 08:18:32 UTC, Ola Fosheim Grøstad wrote: On Monday, 8 September 2014 at 08:08:23 UTC, Kagamin wrote: But that downloaded file is bloatware, because it has to implement functionality, which is not provided by the system. That tiny pe file doesn't download anything, it's completely done by the system. Yeah… http://stackoverflow.com/questions/284797/hello-world-in-less-than-20-bytes My personal best -- At my first job, a customer once made a request for a very simple DOS utility. They did mention that they didn't have much disk space on their machine, so they asked me to try to make the program small. That was a once-in-a-lifetime opportunity. Naturally, I wrote it in asm. The final executable size was 15 bytes. g The customer loved it.
Re: RFC: std.json sucessor
On Wednesday, 27 August 2014 at 23:51:54 UTC, Walter Bright wrote: On 8/26/2014 12:24 AM, Don wrote: On Monday, 25 August 2014 at 23:29:21 UTC, Walter Bright wrote: On 8/25/2014 4:15 PM, Ola Fosheim Grøstad ola.fosheim.grostad+dl...@gmail.com wrote: On Monday, 25 August 2014 at 21:24:11 UTC, Walter Bright wrote: I didn't know that. But recall I did implement it in DMC++, and it turned out to simply not be useful. I'd be surprised if the new C++ support for it does anything worthwhile. Well, one should initialize with signaling NaN. Then you get an exception if you try to compute using uninitialized values. That's the theory. The practice doesn't work out so well. To be more concrete: Processors from AMD have signalling NaN behaviour which is different from processors from Intel. And the situation is worst on most other architectures. It's a lost cause, I think. The other issues were just when the snan = qnan conversion took place. This is quite unclear given the extensive constant folding, CTFE, etc., that D does. It was also affected by how dmd generates code. Some code gen on floating point doesn't need the FPU, such as toggling the sign bit. But then what happens with snan = qnan? The whole thing is an undefined, unmanageable mess. I think the way to think of it is, to the programmer, there is *no such thing* as an snan value. It's an implementation detail that should be invisible. Semantically, a signalling nan is a qnan value with a hardware breakpoint on it. An SNAN should never enter the CPU. The CPU always converts them to QNAN if you try. You're kind of not supposed to know that SNAN exists. Because of this, I think SNAN only ever makes sense for static variables. Setting local variables to snan doesn't make sense. since the snan has to enter the CPU. Making that work without triggering the snan is very painful. Making it trigger the snan on all forms of access is even worse. If float.init exists, it cannot be an snan, since you are allowed to use float.init.
Re: RFC: std.json sucessor
On Thursday, 28 August 2014 at 12:10:58 UTC, Ola Fosheim Grøstad wrote: Or to be more explicit: If have SNAN then there is no point in trying to recompute the expression using a different algorithm. If have QNAN then you might want to recompute the expression using a different algorithm (e.g. complex numbers or analytically). ? No. Once you load an SNAN, it isn't an SNAN any more! It is a QNAN. You cannot have an SNAN in a floating-point register (unless you do a nasty hack to pass it in). It gets converted during loading. const float x = snan; x = x; // x is now a qnan.
Re: RFC: std.json sucessor
On Monday, 25 August 2014 at 23:29:21 UTC, Walter Bright wrote: On 8/25/2014 4:15 PM, Ola Fosheim Grøstad ola.fosheim.grostad+dl...@gmail.com wrote: On Monday, 25 August 2014 at 21:24:11 UTC, Walter Bright wrote: I didn't know that. But recall I did implement it in DMC++, and it turned out to simply not be useful. I'd be surprised if the new C++ support for it does anything worthwhile. Well, one should initialize with signaling NaN. Then you get an exception if you try to compute using uninitialized values. That's the theory. The practice doesn't work out so well. To be more concrete: Processors from AMD have signalling NaN behaviour which is different from processors from Intel. And the situation is worst on most other architectures. It's a lost cause, I think.
Re: RFC: std.json sucessor
On Tuesday, 26 August 2014 at 07:34:05 UTC, Ola Fosheim Gr wrote:
On Tuesday, 26 August 2014 at 07:24:19 UTC, Don wrote:
Processors from AMD have signalling NaN behaviour which is
different from processors from Intel.
And the situation is worst on most other architectures. It's a
lost cause, I think.
I disagree. AFAIK signaling NaN was standardized in IEEE
754-2008.
So it receives attention.
It was always in IEEE754. The decision in 754-2008 was simply to
not remove it from the spec (a lot of people wanted to remove
it). I don't think anything has changed.
The point is, existing hardware does not support it consistently.
It's not possible at reasonable cost.
---
real uninitialized_var = real.snan;
void foo()
{
real other_var = void;
asm {
fld uninitialized_var;
fstp other_var;
}
}
---
will signal on AMD, but not Intel. I'd love for this to work, but
the hardware is fighting against us. I think it's useful only for
debugging.
Re: RFC: std.json sucessor
On Tuesday, 26 August 2014 at 12:37:58 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 26 August 2014 at 10:55:20 UTC, Don wrote: It was always in IEEE754. The decision in 754-2008 was simply to not remove it from the spec (a lot of people wanted to remove it). I don't think anything has changed. It was implementation defined before. I think they specified the bit in 2008. fld uninitialized_var; fstp other_var; This is not SSE, but I guess MOVSS does not create exceptions either. No, it's more subtle. On the original x87, signalling NaNs are triggered for 64 bits loads, but not for 80 bit loads. You have to read the fine print to discover this. I don't think the behaviour was intentional.
Re: RFC: std.json sucessor
On Monday, 25 August 2014 at 14:04:12 UTC, Sönke Ludwig wrote:
Am 25.08.2014 15:07, schrieb Don:
On Thursday, 21 August 2014 at 22:35:18 UTC, Sönke Ludwig
wrote:
Following up on the recent std.jgrandson thread [1], I've
picked up
the work (a lot earlier than anticipated) and finished a
first version
of a loose blend of said std.jgrandson, vibe.data.json and
some
changes that I had planned for vibe.data.json for a while.
I'm quite
pleased by the results so far, although without a
serialization
framework it still misses a very important building block.
Code: https://github.com/s-ludwig/std_data_json
Docs: http://s-ludwig.github.io/std_data_json/
DUB: http://code.dlang.org/packages/std_data_json
The new code contains:
- Lazy lexer in the form of a token input range (using slices
of the
input if possible)
- Lazy streaming parser (StAX style) in the form of a node
input range
- Eager DOM style parser returning a JSONValue
- Range based JSON string generator taking either a token
range, a
node range, or a JSONValue
- Opt-out location tracking (line/column) for tokens, nodes
and values
- No opDispatch() for JSONValue - this has shown to do more
harm than
good in vibe.data.json
The DOM style JSONValue type is based on
std.variant.Algebraic. This
currently has a few usability issues that can be solved by
upgrading/fixing Algebraic:
- Operator overloading only works sporadically
- No tag enum is supported, so that switch()ing on the type
of a
value doesn't work and an if-else cascade is required
- Operations and conversions between different Algebraic
types is not
conveniently supported, which gets important when other
similar
formats get supported (e.g. BSON)
Assuming that those points are solved, I'd like to get some
early
feedback before going for an official review. One open issue
is how to
handle unescaping of string literals. Currently it always
unescapes
immediately, which is more efficient for general input ranges
when the
unescaped result is needed, but less efficient for string
inputs when
the unescaped result is not needed. Maybe a flag could be
used to
conditionally switch behavior depending on the input range
type.
Destroy away! ;)
[1]:
http://forum.dlang.org/thread/lrknjl$co7$1...@digitalmars.com
One missing feature (which is also missing from the existing
std.json)
is support for NaN and Infinity as JSON values. Although they
are not
part of the formal JSON spec (which is a ridiculous omission,
the
argument given for excluding them is fallacious), they do get
generated
if you use Javascript's toString to create the JSON. Many JSON
libraries
(eg Google's) also generate them, so they are frequently
encountered in
practice. So a JSON parser should at least be able to lex them.
ie this should be parsable:
{foo: NaN, bar: Infinity, baz: -Infinity}
This would probably best added as another (CT) optional
feature. I think the default should strictly adhere to the JSON
specification, though.
Yes, it should be optional, but not a compile-time option.
I think it should parse it, and based on a runtime flag, throw an
error (perhaps an OutOfRange error or something, and use the same
thing for values that exceed the representable range).
An app may accept these non-standard values under certain
circumstances and not others. In real-world code, you see a *lot*
of these guys.
Part of the reason these are important, is that NaN or Infinity
generally means some Javascript code just has an uninitialized
variable. Any other kind of invalid JSON typically means
something very nasty has happened. It's important to distinguish
these.
Re: RFC: std.json sucessor
On Tuesday, 26 August 2014 at 14:06:42 UTC, Sönke Ludwig wrote:
Am 26.08.2014 15:43, schrieb Don:
On Monday, 25 August 2014 at 14:04:12 UTC, Sönke Ludwig wrote:
Am 25.08.2014 15:07, schrieb Don:
ie this should be parsable:
{foo: NaN, bar: Infinity, baz: -Infinity}
This would probably best added as another (CT) optional
feature. I
think the default should strictly adhere to the JSON
specification,
though.
Yes, it should be optional, but not a compile-time option.
I think it should parse it, and based on a runtime flag, throw
an error
(perhaps an OutOfRange error or something, and use the same
thing for
values that exceed the representable range).
An app may accept these non-standard values under certain
circumstances
and not others. In real-world code, you see a *lot* of these
guys.
Why not a compile time option?
That sounds to me like such an app should simply enable parsing
those values and manually test for NaN at places where it
matters.
For all other (the majority) of applications, encountering
NaN/Infinity will simply mean that there is a bug, so it makes
sense to not accept those at all by default.
Apart from that I don't think that it's a good idea for the
lexer in general to accept non-standard input by default.
Please note, I've been talking about the lexer. I'm choosing my
words very carefully.
Part of the reason these are important, is that NaN or Infinity
generally means some Javascript code just has an uninitialized
variable.
Any other kind of invalid JSON typically means something very
nasty has
happened. It's important to distinguish these.
As far as I understood, JavaScript will output those special
values as null (at least when not using external JSON
libraries).
No. Javascript generates them directly. Naive JS code generates
these guys. That's why they're so important.
But even if not, an uninitialized variable can also be very
nasty, so it's hard to see why that kind of bug should be
silently supported (by default).
I never said it should accepted by default. I said it is a
situation which should be *lexed*. Ideally, by default it should
give a different error from simply 'invalid JSON'. I believe it
should ALWAYS be lexed, even if an error is ultimately generated.
This is the difference: if you get NaN or Infinity, there's
probably a straightforward bug in the Javascript code, but your D
code is fine. Any other kind of JSON parsing error means you've
got a garbage string that isn't JSON at all. They are very
different errors.
It's a diagnostics issue.
Re: RFC: std.json sucessor
On Thursday, 21 August 2014 at 22:35:18 UTC, Sönke Ludwig wrote:
Following up on the recent std.jgrandson thread [1], I've
picked up the work (a lot earlier than anticipated) and
finished a first version of a loose blend of said
std.jgrandson, vibe.data.json and some changes that I had
planned for vibe.data.json for a while. I'm quite pleased by
the results so far, although without a serialization framework
it still misses a very important building block.
Code: https://github.com/s-ludwig/std_data_json
Docs: http://s-ludwig.github.io/std_data_json/
DUB: http://code.dlang.org/packages/std_data_json
The new code contains:
- Lazy lexer in the form of a token input range (using slices
of the
input if possible)
- Lazy streaming parser (StAX style) in the form of a node
input range
- Eager DOM style parser returning a JSONValue
- Range based JSON string generator taking either a token
range, a
node range, or a JSONValue
- Opt-out location tracking (line/column) for tokens, nodes
and values
- No opDispatch() for JSONValue - this has shown to do more
harm than
good in vibe.data.json
The DOM style JSONValue type is based on std.variant.Algebraic.
This currently has a few usability issues that can be solved by
upgrading/fixing Algebraic:
- Operator overloading only works sporadically
- No tag enum is supported, so that switch()ing on the type
of a
value doesn't work and an if-else cascade is required
- Operations and conversions between different Algebraic types
is not
conveniently supported, which gets important when other
similar
formats get supported (e.g. BSON)
Assuming that those points are solved, I'd like to get some
early feedback before going for an official review. One open
issue is how to handle unescaping of string literals. Currently
it always unescapes immediately, which is more efficient for
general input ranges when the unescaped result is needed, but
less efficient for string inputs when the unescaped result is
not needed. Maybe a flag could be used to conditionally switch
behavior depending on the input range type.
Destroy away! ;)
[1]: http://forum.dlang.org/thread/lrknjl$co7$1...@digitalmars.com
One missing feature (which is also missing from the existing
std.json) is support for NaN and Infinity as JSON values.
Although they are not part of the formal JSON spec (which is a
ridiculous omission, the argument given for excluding them is
fallacious), they do get generated if you use Javascript's
toString to create the JSON. Many JSON libraries (eg Google's)
also generate them, so they are frequently encountered in
practice. So a JSON parser should at least be able to lex them.
ie this should be parsable:
{foo: NaN, bar: Infinity, baz: -Infinity}
You should also put tests in for what happens when you pass NaN
or infinity to toJSON. It shouldn't silently generate invalid
JSON.
Re: Bypassing std.concurrency.exec()
On Thursday, 7 August 2014 at 05:23:51 UTC, Don Viszneki wrote: So I am no closer to identifying the problem! Kapps in #d freenode suggested the problem is in the allocator not having registered http://dlang.org/phobos/core_thread.html#.thread_attachThis This worked! However, we now have problems when shutting down. It appears that during garbage collection, a call to core.thread.suspend() throws the following exception: throw new ThreadException( Unable to suspend thread ) Which results in an infinitely recursive loop, where the allocator continuously tries to throw exceptions! I think perhaps I need to give D the opportunity to clean up SDL2's audio thread before shutting it down?
Re: Bypassing std.concurrency.exec()
On Thursday, 7 August 2014 at 05:58:50 UTC, Don Viszneki wrote:
I think perhaps I need to give D the opportunity to clean up
SDL2's audio thread before shutting it down?
I added a new message that is sent first to my audio thread from
the main thread when shutting down, and then it is sent from the
audio thread to the main thread as acknowledgement.
In audio thread:
(MessageShutdownAudio msg)
{
send(mainTid, MessageShutdownAudio());
thread_detachThis();
SDL_PauseAudio(1);
}
In main thread:
send(audioTid, MessageShutdownAudio());
receive((MessageShutdownAudio bye) { /* do nothing */ });
SDL_CloseAudio();
Bypassing std.concurrency.exec()
I am using SDL2 for audio output. SDL2 creates its own thread which will periodically invoke my callback when the audio sink needs more samples. I want to use std.concurrency.receiveTimeout() with this thread, however I get a segfault when attempting to receive a message: http://pastebin.mozilla.org/5849384 Upon further investigation, it looks like this function depends on the TLS variable std.concurrency.mbox, which is marked private. (This is not strictly the same variable as std.concurrency.Tid.mbox.) I think the problem can be found by examining the private function std.concurrency._spawn(), which is the only place where std.concurrency.mbox is assigned to, except for in std.concurrency.thisTid(). My thinking on this is probably wrong, however, as adding a call to thisTid() does not fix the problem. If anyone has any ideas, I'd be happy to hear them. I'm using a really old DMD but I have compared the std.concurrency module from my GDC 4.9.0 with the latest from Github Phobos and they look substantially the same with respect to my issue. Any help would be greatly appreciated!
Re: Bypassing std.concurrency.exec()
On Thursday, 7 August 2014 at 05:16:59 UTC, Don Viszneki wrote: My thinking on this is probably wrong, however, as adding a call to thisTid() does not fix the problem. Well, I've confirmed using gdb that std.concurrency.mbox is properly assigned a new MessageBox in both my main thread and in my audio thread before the crash occurs. So I am no closer to identifying the problem!
Re: checkedint call removal
On Friday, 1 August 2014 at 09:02:36 UTC, Walter Bright wrote:
On 7/31/2014 11:24 PM, Ola Fosheim Grøstad
ola.fosheim.grostad+dl...@gmail.com wrote:
On Friday, 1 August 2014 at 02:44:51 UTC, Walter Bright wrote:
That entry makes no mention of assert being used as an
optimization hint.
Saying that a predicate is always true means it's available
to the optimizer.
An assert does not say that the predicate is always true.
Yes, it does. From Meyers' comprehensive tome on the topic
Object-Oriented Software Construction (1997) where he writes:
A run-time assertion violation is the manifestation of a bug
in the software.
-- pg. 346
In fact, Meyers calls it rule (1) of assertions.
I would rephrase it as: An assert says that either the predicate
is always true, or else the program is in an invalid state and
will not operate correctly.
But I do think this entire argument seems to me to be rather
misplaced. I think it's really it's about -release, not about
assert().
The arguments presented by Ola et al mostly seem to be arguments
against the use of the -release switch. Because it is a very
dangerous flag.
If you're removing all your asserts I'd say you're playing a
dangerous game already. If an assert would have failed, but
execution continued anyway, you're in undefined behaviour -- at
the very least, you're in a condition that the programmer
believed could never happen.
If you are disabling your asserts, but still believe that they
may fail, that means you're expecting your program to enter
undefined behaviour! That seems to be a rather illogical position.
I think very strongly that we should rename the -release
switch, especially if we do start to make use of asserts. It's
going to cause no end of confusion and passionate debate.
---
In one of the 2013 DConf talks a lint tool was discussed that
disallowed you from you writing a condition that was provably
impossible based on 'in' contracts.
eg:
void foo( int x)
in{
assert(x 6);
}
body {
if (x 2) // ERROR! This is always true!
...
}
Which is an interesting approach.
By definition, any optimisations that are performed on the basis
of an assert() that could affect control flow, are detectable
with 100% accuracy by a lint tool. And so, if you wanted to
remove all your asserts but were worried about this issue, it's
detectable at compile time.
Re: WAT: opCmp and opEquals woes
On Monday, 28 July 2014 at 06:05:03 UTC, Fool wrote: On Monday, 28 July 2014 at 00:23:36 UTC, H. S. Teoh via Digitalmars-d wrote: On Sun, Jul 27, 2014 at 07:04:08PM +, Fool via Defining opEquals only makes sense if a user wants to replace equality by some equivalence relation (different from equality). Not necessarily. The user type may be implemented in a way where member-wise binary comparison is not the correct implementation of equality. For example, it could be a tree structure implemented by integer indices into a backing array of nodes. There is no way the compiler could know, in general, how to correctly compare two instances of such a structure, since the bit-level representation of two equal objects may be completely different, yet they represent equivalent trees. You're still implementing equality, but it's equality that's not the same as binary equality. I think we agree except for a subtle difference in defining equality and equivalence. In my personal language there is a single equality but there are many equivalences. The problem with imposing these kinds of restrictions, is that they are generally not enforceable (at least, not without significantly crippling legitimate use cases). At some point, we have to stop babysitting the programmer and trust that he's competent enough to not try to subvert the language to make it do stuff it wasn't intended to do. As somebody once said: Unix was not designed to stop people from doing stupid things, because that would also stop them from doing clever things. -- Doug Gwyn We're not talking about Unix here, but the same principle applies. I agree. Please excuse my lack of creativity: in presence of opCmp I cannot see a single sensible use case for defining a.opEquals(b) different from a.opCmp(b) == 0. Floating-point numbers? ;-) Thank you for pushing me there! It's true. So D has to separate opEquals and opCmp since otherwise a user could not define floating-point 'equality' and 'comparison' himself in the same way as it is defined by the language. I'm convinced know. :-) Thanks! Be careful, though. The argument that opCmp() and opEquals() are orthogonal is not correct, though. Although they are different concepts, they are closely related. We must have: a == b implies a.opCmp(b) == 0. The converse does not apply though. Otherwise you're abusing operator overloading, like when you define + to mean reformat hard disk or something. Suppose we dealt correctly with floating point, including the = operators, etc. Then we'd require another overloaded operator. bool unordered(X other) // return true if !(this other) !(this other) Full situation is: opCmp() == 0 implies ( a==b || a.unordered(b) ) This applies to the RGB example, too. If you define opCmp() for a type, then either: (1) opEquals() is the same as opCmp()==0, OR (2) opEquals() is weird, and needs to be explicitly defined. What you're really doing is distinguishing the unordered case from the equal case. IMHO, the ideal solution would be a really smart compiler that can detect violations of (1). At least, it would be fairly simple to add a runtime assert that this.opCmp(this) == 0 for all cases where opEquals is synthesised.
Re: Integer overflow and underflow semantics?
On Monday, 21 July 2014 at 21:10:43 UTC, Artur Skawina via Digitalmars-d wrote: On 07/21/14 21:53, via Digitalmars-d wrote: On Monday, 21 July 2014 at 19:33:32 UTC, Artur Skawina via Digitalmars-d wrote: Disallowing integer overflow just at CT is not (sanely) possible in a language with D's CTFE capabilities. (Would result in code that compiles and works at runtime, but is not ctfe-able) I'd like to see compile time _constants_ be unbounded rational numbers with explicit truncation. It is when you assign it to an in-memory location that you need to worry about bounds. The same goes for calculations that doesn't do division. No need to copy the bad parts of C. Actually, C/C++ could get away with treating overflow during constant folding as an error (or at least emitting a warning) because of the lack of CTFE (and no templates in C's case). The code will either compile or it won't. For D that is not possible -- if an expression is valid at run-time then it should be valid at compile-time Why do you think that? There are many cases where that is not true. Comparing pointers to two unrelated objects will work at runtime, but causes an error in CTFE. You can read global variables at runtime, not in CTFE. Etc. The converse is true, though -- if it works at CTFE, it must work at runtime. Disallowing integer overflow in CTFE could certainly be implemented. It's not a difficult experiment to run. It would be interesting to see how many instances of overflow are bugs, and how many are intentional.
Re: Integer overflow and underflow semantics?
On Tuesday, 22 July 2014 at 15:31:22 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 22 July 2014 at 11:40:08 UTC, Artur Skawina via Digitalmars-d wrote: obey the exact same rules as RT. Would you really like to use a language in which 'enum x = (a+b)/2;' and 'immutable x = (a+b)/2;' results in different values?... With the exception of hash-functions the result will be wrong if you don't predict that the value is wrapping. If you do, I think you should make the masking explicit e.g. specifying '(a+b)0x' or something similar, which the optimizer can reduce to a single addition. That's how it is in D - the arguments are only about the /default/, and in this case about /using a different default at CT and RT/. Using a non-wrapping default would be a bad idea (perf implications, both direct and Yes, but there is a difference between saying it is ok that it wraps on addition, but it shouldn't overflow before a store takes place and it should be masked to N bits or fail on overflow even though the end-result is known to be correct. A system level language should encourage using the fastest opcode, so you shouldn't enforce 32 bit masking when the fastest register size is 64 bit etc. It should also encourage reordering so you get to use efficient SIMDy instructions. Not possible (for integers), unless you'd be ok with getting different results at CT. You don't get different results at compile time if you are explicit about wrapping. NUMBER f(NUMBER a, NUMBER b) ... Not sure what you mean here. 'f' is a perfectly fine existing function, which is used at RT. It needs to be usable at CT as is. D claims to focus generic programming. So it should also encourage pure functions that can be specified for floats, ints and other numeric types that are subtypes of (true) reals in the same clean definition. I think it's a complete fantasy to think you can write generic code that will work for both floats and ints. The algorithms are completely different. One of the simplest examples is that given float f; int i; (f + 1) and (i + 1) have totally different semantics. There are no values of i for which i + 1 == i, but if abs(f) 1/real.epsilon, then f + 1 == f. Likewise there is no value of i for which i != 0 i+1 == 1, but for any abs(f) real.epsilon, f + 1 == 1. If you express the expression in a clean way to get down to the actual (more limited type) then the optimizer sometimes can pick an efficient sequence of instructions that might be a very fast approximation if you reduce the precision sufficiently in the end-result. To get there you need to differentiate between a truncating division and a non-truncating division etc. Well, it's not a small number of differences. Almost every operation is different. Maybe all of them. I can't actually think of a single operation where the semantics are the same for integers and floating point. Negation comes close, but even then you have the special cases -0.0 and -(-int.max - 1). The philosophy behind generic programming and the requirements for efficient generic programming is quite different from the the machine-level hand optimizing philosophy of classic C, IMO. I think that unfortunately, it's a quest that is doomed to fail. Producing generic code that works for both floats and ints is a fool's errand.
Re: DConf 2014 Keynote: High Performance Code Using D by Walter Bright
On Wednesday, 16 July 2014 at 10:22:41 UTC, bearophile wrote: Andrei Alexandrescu: http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/ Despite Walter is used to pipeline programming, so the next step is to also handle failures and off-band messages in a functional way (without exceptions and global error values) with two parallel pipelines, here named Railway-Oriented Programming. This is one of the simplest introductions (and he can skip the slides 19-53) that I have found of this topic (that in the Haskell community is explained on the base of monads): http://www.slideshare.net/ScottWlaschin/railway-oriented-programming In Bugzilla there are already requests for some Railway-Oriented Programming: https://issues.dlang.org/show_bug.cgi?id=6840 https://issues.dlang.org/show_bug.cgi?id=6843 I think no language extensions are needed for such kind of programming, but of course built-in tuple syntax and basic forms of pattern matching in switch (https://d.puremagic.com/issues/show_bug.cgi?id=596 ) improve the syntax and make the code more handy, handy enough to push more D programmers in using it. For some examples of those things in a system language, this page shows some little examples of functional syntax for Rust: http://science.raphael.poss.name/rust-for-functional-programmers.html Bye, bearophile I think that approach is more convincing for functional languages than for D, especially if you are limited to a single return type. Why not just follow the use Unix stdout/stderr model, and provide an OutputRange for errors to be sent to? I don't really believe that there are two 'railway tracks' in the sense that that presentation implies. Once an error has occurred, typically not much more pipeline processing happens. As for Unix, stdout from one step is tied to stdin, but stderr is output only. There may be further processing of the stderr stream (eg, errors may be reported to a database), but the steps are completely independent from the main stdin-stdout track. I think you get a messy design if you try to combine both into a single pipeline. I think it could be quite interesting to see which algorithms can be created with an Error OutputRange model.
Re: std.math performance (SSE vs. real)
On Friday, 4 July 2014 at 17:05:16 UTC, Walter Bright wrote: On 7/4/2014 3:38 AM, Don wrote: What is the longest type supported by the native hardware? I don't know what that means, and I don't think it even makes sense. Most of the time, it is quite clear. For example, Sparc has 128-bit quads, but they only have partial support. Effectively. they are emulated. Why on earth would you want to use an emulated type on some machines, but not on others? Emulation is not native support. I think the only difference it makes is performance. But there is not very much difference in performance between double-double, and implementations using microcode. Eg PowerPC double-doubles operations require fewer clock cycles than x87 operations on 286. Perhaps the intention was the largest precision you can get for free, without sacrificing speed then that's not clearly defined. On x86-32, that was indeed 80 bits. But on other systems it doesn't have an obvious answer. On x86-64 it's not that simple. Nor on PPC or Sparc. Yes, there is some degree of subjectivity on some platforms. I don't see a good reason for hamstringing the compiler dev with legalese for Platform X with legalese that isn't quite the right thing to do for X. I agree. But I think we can achieve the same outcome while providing more semantic guarantees to the programmer. I think the intention of the spec is clear, and the compiler implementor can be relied on to exercise good judgement. The problem is that the developer cannot write code without knowing the semantics. For example, one of the original motivations for having 80-bit floats on the x87 was that for many functions, they give you correctly-rounded results for 'double' precision. If you don't have 80-bit reals, then you need to use far more complicated algorithms. If your code needs to work on a system with only 64 bit reals, then you have to do the hard work. Something I've come to realize, was that William Kahan's work was done in a world before generic programming. He had constraints that we don't have. Our metaprogramming system gives us great tools to get the highest accuracy and performance out of any processor. We can easily cope with the messy reality of real-world systems, we don't need to follow Java in pretending they are all the same. This is something we're good at! A 'real' type that has system-dependent semantics is a poor-man's generics. Take a look at std.math, and see all the instances of 'static if (real.mant_dig == '. Pretty clearly, 'real' is acting as if it were a template parameter. And my experience is that any code which doesn't treat 'real' as a template parameter, is probably incorrect. I think we should acknowledge this.
Re: std.math performance (SSE vs. real)
On Thursday, 3 July 2014 at 00:03:47 UTC, Walter Bright wrote: On 7/2/2014 3:15 PM, Sean Kelly wrote: On Wednesday, 2 July 2014 at 21:44:17 UTC, Walter Bright wrote: C long double == D real for 32 and 64 bit OSX, Linux, and FreeBSD. And it's 'double double' on PPC and 128 bit quad on SPARC. Assuming D targets those platforms, what will be the behavior there? Per the D spec, 'real' will be the longest type supported by the native hardware. This is the problem. If that is the case, it is USELESS. We *must* change that definition. What is the longest type supported by the native hardware? I don't know what that means, and I don't think it even makes sense. For example, Sparc has 128-bit quads, but they only have partial support. Effectively. they are emulated. Why on earth would you want to use an emulated type on some machines, but not on others? Perhaps the intention was the largest precision you can get for free, without sacrificing speed then that's not clearly defined. On x86-32, that was indeed 80 bits. But on other systems it doesn't have an obvious answer. On x86-64 it's not that simple. Nor on PPC or Sparc.
Re: std.math performance (SSE vs. real)
On Tuesday, 1 July 2014 at 17:00:30 UTC, Walter Bright wrote: On 7/1/2014 3:26 AM, Don wrote: Yes, it's complicated. The interesting thing is that there are no 128 bit registers. The temporaries exist only while the FMA operation is in progress. You cannot even preserve them between consecutive FMA operations. An important consequence is that allowing intermediate calculations to be performed at higher precision than the operands, is crucial, and applies outside of x86. This is something we've got right. But it's not possible to say that the intermediate calculations are done at the precision of 'real'. This is the semantics which I think we currently have wrong. Our model is too simplistic. On modern x86, calculations on float operands may have intermediate calculations done at only 32 bits (if using straight SSE), 80 bits (if using x87), or 64 bits (if using float FMA). And for double operands, they may be 64 bits, 80 bits, or 128 bits. Yet, in the FMA case, non-FMA operations will be performed at lower precision. It's entirely possible for all three intermediate precisions to be active at the same time! I'm not sure that we need to change anything WRT code generation. But I think our style recommendations aren't quite right. And we have at least one missing primitive operation (discard all excess precision). What do you recommend? It needs some thought. But some things are clear. Definitely, discarding excess precision is a crucial operation. C and C++ tried to do it implicitly with sequence points, but that kills optimisation possibilities so much that compilers don't respect it. I think it's actually quite similar to write barriers in multithreaded programming. C got it wrong, but we're currently in an even worse situation because it doesn't necessarily happen at all. We need a builtin operation -- and not in std.math, this is as crucial as addition, and it's purely a signal to the optimiser. It's very similar to a casting operation. I wonder if we can do it as an attribute? .exact_float, .restrict_float, .force_float, .spill_float or something similar? With D's current floating point semantics, it's actually impossible to write correct floating-point code. Everything that works right now, is technically only working by accident. But if we get this right, we can have very nice semantics for when things like FMA are allowed to happen -- essentially the optimiser would have free reign between these explicit discard_excess_precision sequence points. After that, I'm a bit less sure. It does seem to me that we're trying to make 'real' do double-duty as meaning both x87 80 bit floating-point number and also as something like a storage class that is specific to double: compiler, don't discard excess precision. Which are both useful concepts, but aren't identical. The two concepts did coincide on x86 32-bit, but they're different on x86-64. I think we need to distinguish the two. Ideally, I think we'd have a __real80 type. On x86 32 bit this would be the same as 'real', while on x86-64 __real80 would be available but probably 'real' would alias to double. But I'm a lot less certain about this.
Re: std.math performance (SSE vs. real)
On Monday, 30 June 2014 at 16:54:17 UTC, Walter Bright wrote: On 6/30/2014 12:20 AM, Don wrote: What I think is highly likely is that it will only have legacy support, with such awful performance that it never makes sense to use them. For example, the speed of 80-bit and 64-bit calculations in x87 used to be identical. But on recent Intel CPUs, the 80-bit operations run at half the speed of the 64 bit operations. They are already partially microcoded. For me, a stronger argument is that you can get *higher* precision using doubles, in many cases. The reason is that FMA gives you an intermediate value with 128 bits of precision; it's available in SIMD but not on x87. So, if we want to use the highest precision supported by the hardware, that does *not* mean we should always use 80 bits. I've experienced this in CTFE, where the calculations are currently done in 80 bits, I've seen cases where the 64-bit runtime results were more accurate, because of those 128 bit FMA temporaries. 80 bits are not enough!! I did not know this. It certainly adds another layer of nuance - as the higher level of precision will only apply as long as one can keep the value in a register. Yes, it's complicated. The interesting thing is that there are no 128 bit registers. The temporaries exist only while the FMA operation is in progress. You cannot even preserve them between consecutive FMA operations. An important consequence is that allowing intermediate calculations to be performed at higher precision than the operands, is crucial, and applies outside of x86. This is something we've got right. But it's not possible to say that the intermediate calculations are done at the precision of 'real'. This is the semantics which I think we currently have wrong. Our model is too simplistic. On modern x86, calculations on float operands may have intermediate calculations done at only 32 bits (if using straight SSE), 80 bits (if using x87), or 64 bits (if using float FMA). And for double operands, they may be 64 bits, 80 bits, or 128 bits. Yet, in the FMA case, non-FMA operations will be performed at lower precision. It's entirely possible for all three intermediate precisions to be active at the same time! I'm not sure that we need to change anything WRT code generation. But I think our style recommendations aren't quite right. And we have at least one missing primitive operation (discard all excess precision).
Re: std.math performance (SSE vs. real)
On Monday, 30 June 2014 at 04:15:46 UTC, Walter Bright wrote: On 6/29/2014 8:22 PM, Manu via Digitalmars-d wrote: Well, here's the thing then. Consider that 'real' is only actually supported on only a single (long deprecated!) architecture. In x64's case, it is deprecated for over a decade now, and may be removed from the hardware at some unknown time. The moment that x64 processors decide to stop supporting 32bit code, the x87 will go away, and those opcodes will likely be emulated or microcoded. Interacting real-float/double means register swapping through memory. It should be treated the same as float-simd; they are distinct (on most arch's). Since they are part of the 64 bit C ABI, that would seem to be in the category of nevah hoppen. What I think is highly likely is that it will only have legacy support, with such awful performance that it never makes sense to use them. For example, the speed of 80-bit and 64-bit calculations in x87 used to be identical. But on recent Intel CPUs, the 80-bit operations run at half the speed of the 64 bit operations. They are already partially microcoded. For me, a stronger argument is that you can get *higher* precision using doubles, in many cases. The reason is that FMA gives you an intermediate value with 128 bits of precision; it's available in SIMD but not on x87. So, if we want to use the highest precision supported by the hardware, that does *not* mean we should always use 80 bits. I've experienced this in CTFE, where the calculations are currently done in 80 bits, I've seen cases where the 64-bit runtime results were more accurate, because of those 128 bit FMA temporaries. 80 bits are not enough!!
Re: std.math performance (SSE vs. real)
On Sunday, 29 June 2014 at 18:13:59 UTC, Russel Winder via Digitalmars-d wrote: On Sun, 2014-06-29 at 07:59 -0700, Andrei Alexandrescu via Digitalmars-d wrote: […] A friend who works at a hedge fund (after making the rounds to the NYC large financial companies) told me that's a myth. Any nontrivial calculation involving money (interest, fixed income, derivatives, ...) needs floating point. He never needed more than double. Very definitely so. Fixed point or integer arithmetic for simple household finance fair enough, but for finance house calculations you generally need 22+ significant denary digits to meet with compliance requirements. Many people seem to have the bizarre idea that floating point is less accurate than integer arithmetic. As if storing a value into a double makes it instantly fuzzy, or something. In fact, providing that the the precision is large enough, every operation that is exact in integers, is exact in floating point as well. And if you perform a division using integers, you've silently lost precision. So I'm not sure what benefit you'd gain by eschewing floating point.
Re: Bounty Increase on Issue #1325927
On Thursday, 26 June 2014 at 21:20:04 UTC, Joakim wrote: On Thursday, 26 June 2014 at 17:52:13 UTC, Nick Sabalausky wrote: On 6/26/2014 7:02 AM, Shammah Chancellor wrote: I've increased the bounty on this bug. Fast CTFE is very important. https://www.bountysource.com/issues/1325927-ctfe-copy-on-write-is-slow-and-causes-huge-memory-usage This is great news, and I'm sure very much appreciated by all. I can't help being a little concerned over issue ownership, though. My understanding is that Don's already done a large amount of work towards this issue. I wonder if that could actually be holding people back from contributing to the issue, for fear of taking whole pot unfairly (ie, swooping in and just doing the last little bit, or being perceived as attempting that), or fear of stirring up disagreement over money? Don's a senior developer at a company that just got bought for $200 million. I doubt he's stressing over a $400 bounty, ;) especially if it takes some work off his plate. Yes, of course I'm not interested in bounties. But note that that issue is not really a bug, it's a project. I put hundreds of hours of work into this, to get to the point where we are now - fixing the compiler structure to the point where a JIT is possible. That work was funded by an insolvency payout :). Daniel Murphy has done some work on it, as well. I doubt bounties are effective as a motivation for this kind of thing.
Re: RFC: Value range propagation for if-else
On Wednesday, 18 June 2014 at 06:40:21 UTC, Lionello Lunesu wrote:
Hi,
https://github.com/lionello/dmd/compare/if-else-range
There, I've also added a __traits(intrange, expression) which
returns a tuple with the min and max for the given expression.
Destroy?
The compiler uses value range propagation in this {min, max}
form, but I think that's an implementation detail. It's well
suited for arithmetic operations, but less suitable for logical
operations. For example, this code can't overflow, but {min, max}
range propagation thinks it can.
ubyte foo ( uint a) {
return (a 0x8081) 0x0FFF;
}
For these types of expressions, {known_one_bits, known_zero_bits}
works better.
Now, you can track both types of range propagation
simultaneously, and I think we probably should improve our
implementation in that way. It would improve the accuracy in many
cases.
Question: If we had implemented that already, would you still
want the interface you're proposing here?
Re: Ref counting for CTFE?
On Thursday, 29 May 2014 at 18:12:59 UTC, Timon Gehr wrote: On 05/29/2014 06:53 PM, Dylan Knutson wrote: ... Is there anything so radically different in D than these other languages, that prevents the implementation of a run-of-the-mill VM to eval D code? No. (In fact, I've written a naive but mostly complete byte code interpreter in half a week or so last year, as part of an ongoing recreational D front end implementation effort.) It just seems strange to me that it's such a problem when this is basically solved by all scripting languages. And I'm really not trying to downplay the difficulty in implementing CTFE in D, but rather just figure out why it's so hard to implement in comparison. CTFE is somewhat intertwined with semantic analysis, which makes it a little harder to specify/implement than usual interpreters. However, the performance problem is mostly a structural issue of the current implementation: DMDs CTFE interpreter gradually grew out of its constant folder in some kind of best effort fashion as far as I understand. It is feasible to do everything in the usual fashion and occasionally just pause or restart interpretation at well-defined points where it needs to interface with semantic analysis. Exactly. Historically, most of the work I've done on CTFE was in fixing up the relationship between CTFE and the rest of the compiler, ironing out all of the weird semantic interactions. Almost *nothing* has ever been done on the CTFE implementation itself. The implementation is the crappiest thing you could imagine, it leaks memory like BP leaks oil. It's been hard to fix not because doing a JIT is hard, but because of the semantic interaction bugs. The good news is that most of those are fixed now. But, it's worth mentioning that at dconf, CTFE and mixins were blamed for many things they aren't responsible for. For example, Phobos takes forever to compile, but it's nothing to do with CTFE. Phobos is slow to compile because everything imports everything else, and it instantiates nearly a million templates. IE, an infinitely fast CTFE engine would make very little difference to Phobos compile times.
Re: D Language Version 3
On Wednesday, 28 May 2014 at 03:25:28 UTC, Suminda Dharmasena wrote: Hi, D2 has been out for a while. Looking to see what the roadmap is like towards D3? Suminda No, it has not been out for a while. I would even say that it's not out yet! It still doesn't exist yet in the same way as D1. D1 was a clearly defined snapshot of the language at a particular moment in time. The feature list is unchanged since D1.014, except for array operations which were finally implemented in D1.034. It reached release 1.076 just with bugfixes, ie there were 61 bugfix releases. D2 is the ongoing language development, and we still don't have a stability branch more recent than the D1 branch. Almost every release has contained a new feature, and I can't see that stopping anytime soon.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 08:14:50 UTC, luka8088 wrote: On 15.5.2014. 8:58, Jonathan M Davis via Digitalmars-d wrote: On Thu, 15 May 2014 05:51:14 + via Digitalmars-d digitalmars-d@puremagic.com wrote: Yep, purity implies memoing. No, it doesn't. _All_ that it means when a function is pure is that it cannot access global or static variables unless they can't be changed after being initialized (e.g. they're immutable, or they're const value types), and it can't call any other functions which aren't pure. It means _nothing_ else. And it _definitely_ has nothing to do with functional purity. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. But by itself, pure means nothing of the sort. So, no, purity does _not_ imply memoization. - Jonathan M Davis Um. Yes it does. http://dlang.org/function.html#pure-functions functional purity (i.e. the guarantee that the function will always return the same result for the same arguments) The fact that it should not be able to effect or be effected by the global state is not a basis for purity, but rather a consequence. Even other sources are consistent on this matter, and this is what purity by definition is. Please note: D's 'pure' annotation does *not* mean that the function is pure. It means that it is statically verified to be OK to call it from a pure function. The compiler determines if a function is pure, the programmer never does. There are two things going on here, and they are quite distinct. (1) Really the keyword should be something like '@noglobal', rather than 'pure'. It's called pure for historical reasons. To reduce confusion I'll call D's pure '@noglobal' and the functional languages pure '@memoizable'. But it turns out that @memoizable isn't actually an interesting property, whereas '@noglobal' is. No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra property which the compiler can trivially determine from @noglobal. Suppose you have function f(), which calls function g(). If f does not depend on global state, then g must not depend on global state. BUT if f() can be memoizable even if g() is not memoizable. This approach used by D enormously increases the number of functions which can be statically proven to be pure. The nomenclature can create confusion though. (2) Allowing GC activity inside a @noglobal function does indeed weaken our ability to memoize. The compiler can still perform memoizing operations on most functions that return GC-allocated memory, but it's more difficult. We don't yet have data on how much of a problem this is. An interesting side-effect of the recent addition of @nogc to the language, is that we get this ability back.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 10:31:47 UTC, luka8088 wrote: On 15.5.2014. 11:45, Don wrote: On Thursday, 15 May 2014 at 08:14:50 UTC, luka8088 wrote: On 15.5.2014. 8:58, Jonathan M Davis via Digitalmars-d wrote: On Thu, 15 May 2014 05:51:14 + via Digitalmars-d digitalmars-d@puremagic.com wrote: Yep, purity implies memoing. No, it doesn't. _All_ that it means when a function is pure is that it cannot access global or static variables unless they can't be changed after being initialized (e.g. they're immutable, or they're const value types), and it can't call any other functions which aren't pure. It means _nothing_ else. And it _definitely_ has nothing to do with functional purity. Now, combined with other information, you _can_ get functional purity out it - e.g. if all the parameters to a function are immutable, then it _is_ functionally pure, and optimizations requiring functional purity can be done with that function. But by itself, pure means nothing of the sort. So, no, purity does _not_ imply memoization. - Jonathan M Davis Um. Yes it does. http://dlang.org/function.html#pure-functions functional purity (i.e. the guarantee that the function will always return the same result for the same arguments) The fact that it should not be able to effect or be effected by the global state is not a basis for purity, but rather a consequence. Even other sources are consistent on this matter, and this is what purity by definition is. Please note: D's 'pure' annotation does *not* mean that the function is pure. It means that it is statically verified to be OK to call it from a pure function. The compiler determines if a function is pure, the programmer never does. There are two things going on here, and they are quite distinct. (1) Really the keyword should be something like '@noglobal', rather than 'pure'. It's called pure for historical reasons. To reduce confusion I'll call D's pure '@noglobal' and the functional languages pure '@memoizable'. But it turns out that @memoizable isn't actually an interesting property, whereas '@noglobal' is. No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra property which the compiler can trivially determine from @noglobal. Suppose you have function f(), which calls function g(). If f does not depend on global state, then g must not depend on global state. BUT if f() can be memoizable even if g() is not memoizable. This approach used by D enormously increases the number of functions which can be statically proven to be pure. The nomenclature can create confusion though. (2) Allowing GC activity inside a @noglobal function does indeed weaken our ability to memoize. The compiler can still perform memoizing operations on most functions that return GC-allocated memory, but it's more difficult. We don't yet have data on how much of a problem this is. An interesting side-effect of the recent addition of @nogc to the language, is that we get this ability back. Yeah, I read all about weak/string purity and I do understand the background. I was talking about strong purity, maybe I should pointed that out. So, to correct myself: As I understood strong purity implies memoization. Am I correct? Yes. 'strong pure' means pure in the way that the functional language crowd means 'pure'. 'weak pure' just means doesn't use globals. But note that strong purity isn't an official concept, it was just the terminology I used when explain to Walter what I meant. I don't like the term because it's rather misleading -- in reality you could define a whole range of purity strengths (more than just two). The stronger the purity, the more optimizations you can apply.
Re: Memory allocation purity
On Thursday, 15 May 2014 at 10:46:21 UTC, Ola Fosheim Grøstad wrote: On Thursday, 15 May 2014 at 09:45:52 UTC, Don wrote: But it turns out that @memoizable isn't actually an interesting property, whereas '@noglobal' is. No global state is a deep, transitive property of a function. Memoizable is a superficial supersetextra property which the compiler can trivially determine from @noglobal. Uhm. That is a pretty strong assumption. memoizable is very useful property when you do multihreading, transactions or anything that requires locking. It's useful, but it's not a deep property, and importantly, it isn't transient. The compiler can trivially work it out if it knows the function is @noglobal. And you can still access globals, you just need a guarantee that globals don't change until you are done. Sure, but how can the compiler statically check that? It's a tough problem. (That's not a rhetorical question, BTW. If you have a solution, that would be awesome). Considering that 90% of the functions I write don't do IO or globals I'd rather specify the opposite. io, global whatever. That is also easy to enforce, i.e. you don't get to access IO/globals if you don't annotate the function. I agree, I'd personally like to have an annotation '@global', and put 'pure:' at the top of every module.
Re: DIP60: @nogc attribute
On Thursday, 17 April 2014 at 19:51:38 UTC, Walter Bright wrote: On 4/17/2014 10:41 AM, Dicebot wrote: On Thursday, 17 April 2014 at 16:57:32 UTC, Walter Bright wrote: With current limitations @nogc is only useful to verify that embedded code which does not have GC at all does not use any GC-triggering language features before it comes to weird linker errors / rt-asserts. But that does not work good either because of next problem: Remember that @nogc will be inferred for template functions. That means that whether it is @nogc or not will depend on its arguments being @nogc, which is just what is needed. No, it looks like I have stated that very wrong because everyone understood it in completely opposite way. What I mean is that `put()` is NOT @nogc and it still should work. Same as weakly pure is kind of pure but allowed to mutate its arguments, proposed weakly @nogc can only call GC via functions directly accessible from its arguments. I don't see value for this behavior. It turns out to have enormous value. I will explain this in my DConf talk. A little preview: Almost all of our code at Sociomantic obeys this behaviour, and it's probably the most striking feature of our codebase. By almost all I mean probably 90% of our code, including all of our libraries. Not just the 5% - 10% that could marked as @nogc according to your DIP. The key property it ensures is, if you make N calls to the function, the number of GC allocations is in O(1). We don't care if makes 0 allocations or 17. We're not really interested in whether a function uses the GC or not, since most interesting functions do need to do some memory allocation. Ideally, we'd want an attribute which could applied to *all* of Phobos, except for some convenience functions. We have no interest in library code which doesn't behave in that way.
Re: It's official: Sociomantic Labs has been acquired by dunnhumby Ltd
On Friday, 4 April 2014 at 02:38:58 UTC, Andrei Alexandrescu wrote: On 4/3/14, 7:04 AM, Don wrote: https://www.sociomantic.com/dunnhumby-acquires-sociomantic/ Congratulations to all involved! How will this impact the use of D at dunnhumby? Andrei This is going to be very big for D. Our technology will be used with their data and analysis (they're not a software company). Here's what Dunnhumby said in their press release: For some time we have been watching the work of a Berlin internet start-up called Sociomantic. They are a very talented group of people who have developed ground-breaking online technology, far ahead of what anyone else is doing. We have decided to buy the company because the combination of Sociomantic’s technological capability and dunnhumby’s insight from 430m shoppers worldwide will create a new opportunity to make the online experience a lot better, because for the first time we will be able to make online content personalised for people, based on what they actually like, want and need. It is what we have been doing with loyalty programs and personalised offers for years – done with scale and speed in the digital world. http://www.dunnhumby.com/its-time-revolutionise-digital-advertising And this article gives some broader background: http://www.zdnet.com/tescos-big-data-arm-dunnhumby-buys-ad-tech-firm-sociomantic-labs-728040/ - Don.
It's official: Sociomantic Labs has been acquired by dunnhumby Ltd
https://www.sociomantic.com/dunnhumby-acquires-sociomantic/
Re: A division problem
On Monday, 24 March 2014 at 03:55:41 UTC, bearophile wrote:
This kind of code sometimes is wrong, because you forget to
cast x to double before the division and you lose precision
(but here the compiler knows that the result of the division
will go inside a double):
void main() {
int x = 15;
double y = x / 10;
}
The cause is that unfortunately in D the integer division uses
the same operator as the FP division. In Python there is the /
and // operators. In OcaML there are the / and /., in Delphi
there are the / and div operators, in Ada the two operands need
to be of the same type.
Seasoned C/C++/D programmers watch for the types every time
they perform a division, to avoid that trap. But less
experienced programmers introduce bugs with divisions. Can D
help the programmer reduce the frequency of similar bugs? And
do we want to?
Bye,
bearophile
It is indeed a common floating-point bug.
I came up with a solution for this a couple of years ago, never
got around to doing a pull request, but it's on the newsgroup
somewhere. It's a little extension to the range propagation
implementation. You add a boolean flag to the range, which
indicates 'a fractional part has been discarded'. This flag gets
set whenever you perform an integer division (or integer
exponentiation with a negative power), and is cleared whenever
there is a cast or a bitwise operation.
Then, disallow implicit casting from integer to floating point
whenever the fractional bit is set. Catches all these kinds of
bugs, doesn't require any changes to the language.
Re: How to make a global immutable associative array?
On Wednesday, 19 March 2014 at 09:12:53 UTC, Dicebot wrote: On Wednesday, 19 March 2014 at 02:52:23 UTC, bearophile wrote: Rikki Cattermole: Is an enum not appropriate? Because it can be used to push constants and available at ctfe. enum int[int] aa = [1: 2, 3: 4]; pragma(msg, aa); This is bad from an efficiency point of view. I think Don even suggested to disallow it. Bye, bearophile It still only existing way to define compile-time usable AA value, disallowing it without fixing alternatives is not an option. The only thing I've said we should disallow, is creating a runtime reference to a value which only exists at compile-time. I think it is logically nonsensical. I don't see any problem at all with creating a compile-time AA, and then looking up an index in it at compile time.
Re: Final by default?
On Thursday, 13 March 2014 at 13:47:13 UTC, Steven Schveighoffer wrote: On Thu, 13 Mar 2014 09:37:51 -0400, Dicebot pub...@dicebot.lv wrote: On Thursday, 13 March 2014 at 13:16:54 UTC, Daniel Murphy wrote: Steven Schveighoffer wrote in message news:op.xcnu55j2eav7ka@stevens-macbook-pro.local... The worst breaking change in D2, by far, is the prevention of array stomping. What is your use case(s), might I ask? Prevention of array stomping, I thought, had a net positive effect on performance, because it no longer has to lock the GC for thread-local appends. I would guess they're setting length to zero and appending to re-use the memory. Exactly. So far looks like upon transition to D2 almost all arrays used in our code will need to be replaced with some variation of Appender!T I think you might find that it will run considerably faster in that case. In the old mechanism of D1, the GC lock was used on every append, and if you had multiple threads appending simultaneously, they were contending with the single element cache to look up block info. Appender only needs to look up GC block info when it needs more memory from the GC. We don't use threads.
Re: Final by default?
On Thursday, 13 March 2014 at 19:28:59 UTC, Walter Bright wrote:
On 3/13/2014 1:43 AM, Don wrote:
The worst breaking change in D2, by far, is the prevention of
array stomping.
After that change, our code still runs, and produces exactly
the same results,
but it is so slow that it's completely unusable. This one of
the main reasons
we're still using D1.
I didn't know this. I'd like more details - perhaps I can help
with how to deal with it.
Our entire codebase assumes that stomping will happen. Simplest
example:
T[] dupArray(T)(ref T[] dest, T[] src)
{
dest.length = src.length;
if (src.length) {
dest[] = src[];
}
return dest;
}
This is equivalent to dest = src.dup, but if dest was already
long enough to contain src, no allocation occurs.
Sure, we can add a call to assumeSafeAppend() everywhere. And I
mean *everywhere*. Every single instance of array creation or
concatentation, without exception. Almost every array in our
codebase is affected by this.
Re: Possible change to array runtime?
On Friday, 14 March 2014 at 14:48:13 UTC, Steven Schveighoffer wrote: On Thu, 13 Mar 2014 11:24:01 -0400, Steven Schveighoffer schvei...@yahoo.com wrote: arr.length = 0; ... 3. Don's company uses D1 as its language, I highly recommend watching Don's Dconf13 presentation (and look forward to his Dconf14 one!) to see how effective D code can create unbelievable speed, especially where array slices are concerned. But to the above line, in D2, they must add the following code to get the same behavior: arr.assumeSafeAppend(); Just a quick note, buried in same thread that Don mentioned, he outlined a more specific case, and this does not involve setting length to 0, but to any arbitrary value. This means my approach does not help them, and although it makes sense, the idea that it would help Sociomantic move to D2 is not correct. -Steve Actually it would help a great deal. In most cases, we do set the length to 0. That example code is unusual. FYI: In D1, this was the most important idiom in the language. In the first D conference in 2007, a feature T[new] was described, specifically to support this idiom in a safe manner. Implementation was begun in the compiler. Unfortunately, it didn't happen in the end. I'm not sure if it would actually have worked or not. BTW you said somewhere that concatenation always allocates. What I actually meant was ~=, not ~. In our code it is always preceded by .length = 0 though. It's important that ~= should not allocate, when the existing capacity is large enough.
Re: Final by default?
On Thursday, 13 March 2014 at 06:02:27 UTC, Walter Bright wrote:
On 3/12/2014 9:23 PM, Manu wrote:
It's not minor, and it's not achievable by other means though.
class C { final: ... }
does it.
You and Andrei are the only resistance in this thread so far.
Why don't you ask
'temperamental client' what their opinion is? Give them a
heads up, perhaps
they'll be more reasonable than you anticipate?
I didn't even know about this client before the breakage. D has
a lot of users who we don't know about.
Both myself and Don have stated on behalf of industrial
clients that we embrace
breaking changes that move the language forward, or correct
clearly identifiable
mistakes.
Breaking changes has been a huge barrier to Don's company being
able to move from D1 to D2. I still support D1 specifically for
Don's company.
Yes, but the problem is not the changes which cause compile
errors and force you to change your code in obvious ways. The
problem is subtle changes to behaviour.
The worst breaking change in D2, by far, is the prevention of
array stomping.
After that change, our code still runs, and produces exactly the
same results, but it is so slow that it's completely unusable.
This one of the main reasons we're still using D1.
Re: Final by default?
On Thursday, 13 March 2014 at 05:15:58 UTC, Sean Kelly wrote: On Wednesday, 12 March 2014 at 22:50:00 UTC, Walter Bright wrote: The argument for final by default, as eloquently expressed by Manu, is a good one. Even Andrei agrees with it (!). The trouble, however, was illuminated most recently by the std.json regression that broke existing code. The breakage wasn't even intentional; it was a mistake. The user fix was also simple, just a tweak here and there to user code, and the compiler pointed out where each change needed to be made. But we nearly lost a major client over it. I find this a bit baffling. Given the investment this customer must have in D, I can't imagine them switching to a new language over something like this. I hate to say it, but this sounds like the instances you hear of when people call up customer service just to have someone to yell at. Not that the code breakage is okay, but I do feel like this may be somewhat of an exaggeration. And std.json is among the worst code I've ever seen. I'm a bit shocked that anyone would be using it in production code. Regarding this virtual by default issue. I entirely support Manu's argument and wholeheartedly agree with it. I even think that I'd be more likely to use D professionally if D worked this way, for many of the same reasons Manu has expressed. There may even be a window for doing this, but the communication around the change would have to be perfect. Regarding user retention... I've spent the past N months beginning the process of selling D at work. The language and library are at a point of maturity where I think it might have a chance when evaluated simply on the merits of the language itself. However, what has me really hesitant to put my shoulder behind D and really push isn't that changes occur sometimes. Even big changes. It's how they're handled. Issues come up in the newsgroup and are discussed back and forth for ages. Seriously considered. And then maybe a decision is apparently reached (as with this virtual by default thing) and so I expect that action will be taken. And then nothing happens. And other times big changes occur with seemingly little warning. Personally, I don't really require perfect compatibility between released, but I do want to see things moving decisively in a clearly communicated direction. I want to know where we're going and how we're going to get there, and if that means that I have to hold on moving to a new compiler release for a while while I sort out changes that's fine. But I want to be able to prepare for it. As things stand, I'm worried that if I got a team to move to D we'd have stuff breaking unexpectedly and I'd end up feeling like an ass for recommending it. I guess that's probably what prompted the almost lost a major client issue you mentioned above. This JSON parser change was more the proverbial straw than a major issue in itself. I agree completely. Some things that really should be fixed, don't get fixed because of a paranoid fear of breaking code. And this tends to happen with the issues that can give nice warning messages and are easy to fix... Yet there are still enough bugs that your code breaks every release anyway. We need to lose the fantasy that there is legacy code which still compiles. Anything more than a year or so old is broken already. As for the !virtual idea... I hate it. Please don't add yet more ways for people to make their code confusing.
Re: Final by default?
Some things that really should be fixed, don't get fixed because of a paranoid fear of breaking code. And this tends to happen with the issues that can give nice warning messages and are easy to fix... Yet there are still enough bugs that your code breaks every release anyway. We need to lose the fantasy that there is legacy code which still compiles. Anything more than a year or so old is broken already. Backward compatibility is more like a spectrum than a threshold. Not having it now is not an argument to cease pursuing it. Exactly, it's a spectrum. But at any given time, the whole language and library are not at a single point on the spectrum. Some things are frozen, others in practice are not. And the problem is that D has historically acted as if everything was the same, which just doesn't work. I think three levels of forwards compatibility are useful to consider: 1. Frozen. Things that you can absolutely rely on, we will NEVER change it under any circumstances. Any bugs in the design will never be fixed. 2. Stable. We will attempt to minimize changes, but we don't guarantee your code will never break. (It may break in order to prevent breakage of things in case 1, for example). We can guarantee a deprecation path in most cases. 3. We will avoid gratuitous changes, but it will almost certainly change in the future. And what we want to do, is gradually move as many things as we can from category (2) into category (1), and from (3) into (2). I'd like to see us giving a lot more guarantees, rather than trying to keep promises we never actually made.
Re: Close D1 bugs?
On Tuesday, 11 February 2014 at 19:29:14 UTC, Steven Schveighoffer wrote: I noticed Vladimir closed a D1 bug as WORKSFORME, with an explanation that it is fixed in the latest version of D2 phobos, not realizing it was a D1 bug. (https://d.puremagic.com/issues/show_bug.cgi?id=1004) However, it brought up a discussion of what to do with D1 bugs. Should they be closed? If so, what should be the reason for closing? My impression is if the bug still exists (and I'm not advocating we test for it), we should close it as WONTFIX, since D1 is deprecated. Thoughts? I closed the aforementioned bug as WONTFIX. Anyone is free to correct that if you feel it's in error :) -Steve I agree with you in this case. D1 is in heavy commercial use at Sociomantic, and we are still submitting patches to DMD, and we frequently search bugzilla for open D1 bugs. But AFAIK *nobody* is using D1 Phobos. The only remaining role of D1 Phobos, AFAIK, is to allow the D1 test suite to run. I think that we should close all D1 Phobos bugs as WONTFIX. Realistically they are never going to be fixed, and I don't think anybody cares. (Or, if the bug also applied to D2, but is already fixed in D2, I think it would be perfectly valid to mark it as FIXED).
