Re: bug in assigning to dynamic array element
On Tuesday, 4 November 2014 at 01:52:00 UTC, ketmar via Digitalmars-d wrote: sure, this decision will break some code. that's why it will never be approved, despite all hype about safety. at least leaving this feature as is should help C++ programmers to adopt D. they used to languages that has nothing common with sanity. ;-) Yes… http://en.wikipedia.org/wiki/Sequence_point http://dlang.org/expression D needs a major overhaul: 1. assigns should not be expressions, but statements 2. assign should be a sequence point, not UB. 3. re-allocation should always be explicit in non-library types. Solves all issues.
Re: bug in assigning to dynamic array element
On 11/1/14 8:04 AM, ketmar via Digitalmars-d wrote: On Sat, 01 Nov 2014 11:55:53 + anonymous via Digitalmars-d digitalmars-d@puremagic.com wrote: On Saturday, 1 November 2014 at 11:50:34 UTC, ketmar via Digitalmars-d wrote: this *IS* a bug. either compiler should error on this, or it shouldn't modify random memory. imagine the situation when old array contents not only collected by GC, but that memory was allocated to something completely different. The old array is still alive and kicking. The left-hand side still references it. It wasn't collected. You're not writing to random memory. so it's not only writes to the stale copy, but protects that copy from GC. what a great thing! and all this without even a small warning from compiler. i can expect such behavior from c or c++ with all their UB, but not from D. this is not only ugly, this is plainly wrong. No, it's not. It's exactly as you requested when you wrote that code :) Note, you can fix this by overloading opAssign in Info, and it should work as you expect. This will force a delay in the evaluation of which array is used until *after* the RHS is done. -Steve
Re: bug in assigning to dynamic array element
On 11/1/14 10:44 AM, Iain Buclaw via Digitalmars-d wrote: On 1 November 2014 14:19, ketmar via Digitalmars-d digitalmars-d@puremagic.com wrote: On Sat, 1 Nov 2014 13:56:49 + Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: if such assigns will be forbidden for any arrays... this is even worse. what? your shiny language can't do what even the simpliest C compiler can do? now try to convince me that D is not a toy. That - for sure - is where you're wrong. :-) i was talking about C, not that abomination. but if D will compete with C++ for the quirks... than C++ is a winner. ;-) besides, i clearly see 'list.ptr' there. i don't see 'dynArray.ptr' in D code. if i was using '.ptr' directly and it changes by the way... ok, it was my fault. but i never used '.ptr' in my sample! You can clearly see the 'list.length' though. You can't possibly go off the assumption that if you grow the size of a dynamic array, it's area in memory won't be relocated. I think his assumption is that an array reference should not *re-alias* into another array. This is the problem with calling them dynamic arrays -- they are slices of dynamic arrays. And altering the length of a slice can make it point at an entirely new dynamic array, with the original array (and in fact the original slice) still intact. If, for example, the array was an object, and it's *private* underlying storage moved, well then the opIndexAssign would use the new storage, and the array reference doesn't actually change. I don't think it's unreasonable to expect this, but that's based on an incorrect understanding of how D dynamic arrays work. There is a reason we get at least 1 question a week on how D arrays work. -Steve
Re: bug in assigning to dynamic array element
On Monday, 3 November 2014 at 00:16:48 UTC, ketmar via Digitalmars-d wrote: error-prone code. no, this is not another task for lint. not rejecting such code is safe in the terms of program will not segfault, but it's obviously not safe in terms of correct code. Yes, this is a common complaint. Without solid semantic analysis it would probably be better to only have dynamic vectors as a library type with fat slices that are locked to the underlying array. That's what everybody expects from a dynamic array type anyway. …it is a reaaallyyy good idea to support what most people's assumptions about dynamic arrays… D would gain more from relaxing memory safe language constructs and focus more on supporting programming constructs by semantic analysis. This is an area where the C++ crowd will be gridlocked to their backwards compatible mindset. But they are getting increasingly more powerful sanitizers…
Re: bug in assigning to dynamic array element
On Mon, 03 Nov 2014 16:39:43 + via Digitalmars-d digitalmars-d@puremagic.com wrote: On Monday, 3 November 2014 at 00:16:48 UTC, ketmar via Digitalmars-d wrote: error-prone code. no, this is not another task for lint. not rejecting such code is safe in the terms of program will not segfault, but it's obviously not safe in terms of correct code. Yes, this is a common complaint. Without solid semantic analysis it would probably be better to only have dynamic vectors as a library type with fat slices that are locked to the underlying array. That's what everybody expects from a dynamic array type anyway. …it is a reaaallyyy good idea to support what most people's assumptions about dynamic arrays… D would gain more from relaxing memory safe language constructs and focus more on supporting programming constructs by semantic analysis. This is an area where the C++ crowd will be gridlocked to their backwards compatible mindset. But they are getting increasingly more powerful sanitizers… good semantic analysis as a big task, i think we all understand it. as moving dynamic arrays out of core language is the thing that surely won't be approved, the only sane way to safely fix this issue without starting discussion about when .ptr must be loaded is just reject impure assigns to dynamic array elements. sure this will reject some cases where impure assigns are ok, but better be safe that sorry, isn't it? ah, it isn't for D. sure, this decision will break some code. that's why it will never be approved, despite all hype about safety. at least leaving this feature as is should help C++ programmers to adopt D. they used to languages that has nothing common with sanity. ;-) signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
On Mon, 03 Nov 2014 09:50:26 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: There is a reason we get at least 1 question a week on how D arrays work. and this is a clear sign that something is *very* wrong with them. i fully understand the mechanics behind dynamic arrays, slicing and so on. what i *can't* understand here is why compiler allowing me to shoot my foot even without a warning. compiler *knows* that this is dynamic array. it *knows* that assign is impure (as saveIt() obviously not pure and it can't be). yet it silently allows me do that. it's *very* error-prone code. it's easy for compiler to reject such code (or at least warn me). but it choses to carefully hide the possible bug. i'm nat talking about it must work as i expect! here. what i'm talking about is that D compiler allows potentialy unsafe and error-prone code. it's ok for pointers, i'm walking in dangerous area with pointers, but it's not ok when i using *built-in* *type*! it's like ahahaha, let's see how smart you are! i will not warn you about problematic code (despite i can do that without big problems) and we'll see how fast you'll find that gotcha! it's very funny to read discussions about safety and environment variables here. they were boring before, but now i realised the sarcasm, and they becomes very amusing. signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
Like D, Java is LTR evaluation for assignment, and I think C# too. A similar situation to OP code can be created in Java by reassigning an array reference in saveIt().
Re: bug in assigning to dynamic array element
On Sun, 02 Nov 2014 09:25:54 -0800 Dan Olson via Digitalmars-d digitalmars-d@puremagic.com wrote: Like D, Java is LTR evaluation for assignment, and I think C# too. A similar situation to OP code can be created in Java by reassigning an array reference in saveIt(). what i'm talking about is that implicit changing of .ptr is unsafe and error-prone. i *NEVER* explicitly touched '.ptr' in my code, yet somehow it affects me. good compiler should either do what programmer obviously wants it to do (is there somebody who can say that he desperately need the feature in the compiler where compiler modifies stale copy of array data in such code? i doubt), or (and this will be perfectly safe) simply forbid impure assigns for dynamic arrays. what compiler does now is using it's type system to carefully hide error-prone code. no, this is not another task for lint. not rejecting such code is safe in the terms of program will not segfault, but it's obviously not safe in terms of correct code. using built-in type system not to help me, but to cheat me is something i never expected from D. but now i at least learnt that my safety and D safety are completely different things. D safety is no segfaults, and let bad code slip in the cracks. and don't try to help programmer, he is smart enough to track it all manually. nonsense with non-uniform function attribures (@ vs plain), desire to keep prefix const as long as possible, strange requiring of utf-8 even in places that aren't compiler deal (shebang), having 0x and 0b, but ommiting 0o, not allowing 'auto' in foreach... ah, alot of small bits. now i'm starting to see a whole puzzle. it's all about my safety and D safety are completely different things. my bad, i chose a wrong language. i'll keep looking for the language that tries to help me writing programs, not cheating me and then blaming me. signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
filled bugreport for this: https://issues.dlang.org/show_bug.cgi?id=13670
Re: bug in assigning to dynamic array element
On Saturday, 1 November 2014 at 09:03:42 UTC, ketmar via
Digitalmars-d wrote:
as i can guess, the bug is in evaluating left part of '='
operation
before the right part.
I don't know how D defines this, and I couldn't find anything but
a forum discussion [1] (which I didn't read all of). But unless
it's explicitly stated that the right-hand side is evaluated
first, there is no bug.
A simpler test case:
auto list = new size_t[1];
list[0] = (){list = new size_t[1]; return 666;}();
assert(list[0] == 666);
[1] http://forum.dlang.org/thread/gu4bqu$bq$1...@digitalmars.com
Re: bug in assigning to dynamic array element
If it's indeed caused by evaluation (which seems likely), then it's not a bug. All expressions are supposed to be evaluated from left to right.
Re: bug in assigning to dynamic array element
On Sat, 01 Nov 2014 11:43:11 + via Digitalmars-d digitalmars-d@puremagic.com wrote: If it's indeed caused by evaluation (which seems likely), then it's not a bug. All expressions are supposed to be evaluated from left to right. this *IS* a bug. either compiler should error on this, or it shouldn't modify random memory. imagine the situation when old array contents not only collected by GC, but that memory was allocated to something completely different. and it's completely possible to fix this without dropping evaluation order promise: take the address of hidden dynamic array structure first and remember index, and take the address of array element just before assigning. this way it will work as expected, and will evaluate all indicies and the array address itself (not the address of the array contents, but this is irrelevant to evaluation order) first. what whay authors choose to fix this bug is irrelevant for me (but i prefer the second way), but it MUST be fixed. either error, or do what i described in the previous paragraph. signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
On 1 November 2014 11:31, anonymous via Digitalmars-d
digitalmars-d@puremagic.com wrote:
On Saturday, 1 November 2014 at 09:03:42 UTC, ketmar via
Digitalmars-d wrote:
as i can guess, the bug is in evaluating left part of '=' operation
before the right part.
I don't know how D defines this, and I couldn't find anything but
a forum discussion [1] (which I didn't read all of). But unless
it's explicitly stated that the right-hand side is evaluated
first, there is no bug.
A simpler test case:
Breakdown.
auto list = new size_t[1];
list = addressA;
list[0] = (){list = new size_t[1]; return 666;}();
list = addressB;
addressA[0] = 666;
assert(list[0] == 666);
assert(addressB[0] == 666)
Re: bug in assigning to dynamic array element
On Sat, 01 Nov 2014 11:31:51 + anonymous via Digitalmars-d digitalmars-d@puremagic.com wrote: I don't know how D defines this, and I couldn't find anything but a forum discussion [1] (which I didn't read all of). But unless it's explicitly stated that the right-hand side is evaluated first, there is no bug. there is. compiler generates code that modifies random memory addresses. this is absolutely unacceptable. and this is just illogical if we want dynamic arrays to look and work like normal arrays. besides, it's easily fixable without any changes in evaluation order. signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
On Saturday, 1 November 2014 at 11:50:34 UTC, ketmar via Digitalmars-d wrote: this *IS* a bug. either compiler should error on this, or it shouldn't modify random memory. imagine the situation when old array contents not only collected by GC, but that memory was allocated to something completely different. The old array is still alive and kicking. The left-hand side still references it. It wasn't collected. You're not writing to random memory.
Re: bug in assigning to dynamic array element
On 1 November 2014 11:56, ketmar via Digitalmars-d digitalmars-d@puremagic.com wrote: On Sat, 01 Nov 2014 11:31:51 + anonymous via Digitalmars-d digitalmars-d@puremagic.com wrote: I don't know how D defines this, and I couldn't find anything but a forum discussion [1] (which I didn't read all of). But unless it's explicitly stated that the right-hand side is evaluated first, there is no bug. there is. compiler generates code that modifies random memory addresses. this is absolutely unacceptable. and this is just illogical if we want dynamic arrays to look and work like normal arrays. besides, it's easily fixable without any changes in evaluation order. I'm not *entire* sure on that. :) If the evaluation of LHS[IDX] has a side effect, you've broken LTR. Think: Left = Index = Right vs Index = Right = Left
Re: bug in assigning to dynamic array element
On Sat, 01 Nov 2014 11:55:53 + anonymous via Digitalmars-d digitalmars-d@puremagic.com wrote: On Saturday, 1 November 2014 at 11:50:34 UTC, ketmar via Digitalmars-d wrote: this *IS* a bug. either compiler should error on this, or it shouldn't modify random memory. imagine the situation when old array contents not only collected by GC, but that memory was allocated to something completely different. The old array is still alive and kicking. The left-hand side still references it. It wasn't collected. You're not writing to random memory. so it's not only writes to the stale copy, but protects that copy from GC. what a great thing! and all this without even a small warning from compiler. i can expect such behavior from c or c++ with all their UB, but not from D. this is not only ugly, this is plainly wrong. signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
On Sat, 1 Nov 2014 12:01:04 + Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 November 2014 11:56, ketmar via Digitalmars-d digitalmars-d@puremagic.com wrote: On Sat, 01 Nov 2014 11:31:51 + anonymous via Digitalmars-d digitalmars-d@puremagic.com wrote: I don't know how D defines this, and I couldn't find anything but a forum discussion [1] (which I didn't read all of). But unless it's explicitly stated that the right-hand side is evaluated first, there is no bug. there is. compiler generates code that modifies random memory addresses. this is absolutely unacceptable. and this is just illogical if we want dynamic arrays to look and work like normal arrays. besides, it's easily fixable without any changes in evaluation order. I'm not *entire* sure on that. :) If the evaluation of LHS[IDX] has a side effect, you've broken LTR. Think: Left = Index = Right vs Index = Right = Left so forbid that. this kind of bugs are VERY hard to find, especially when dynamic arrays looks like ordinary static arrays in the source. compiler should emit error on the assign code with possible side effects for dynamic arrays, or this hole will pop again and again. a simple change in the unrelated part of code and... KABOOM! everything goes wrong. signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
On Sat, 1 Nov 2014 12:01:04 + Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 November 2014 11:56, ketmar via Digitalmars-d digitalmars-d@puremagic.com wrote: On Sat, 01 Nov 2014 11:31:51 + anonymous via Digitalmars-d digitalmars-d@puremagic.com wrote: I don't know how D defines this, and I couldn't find anything but a forum discussion [1] (which I didn't read all of). But unless it's explicitly stated that the right-hand side is evaluated first, there is no bug. there is. compiler generates code that modifies random memory addresses. this is absolutely unacceptable. and this is just illogical if we want dynamic arrays to look and work like normal arrays. besides, it's easily fixable without any changes in evaluation order. I'm not *entire* sure on that. :) If the evaluation of LHS[IDX] has a side effect, you've broken LTR. Think: Left = Index = Right vs Index = Right = Left p.s. besides, the following code LOOKS wrong: info.list[idx] = saveIt(info, count-1); assert(info.list[idx] != 0); aren't it a nonsence if we know for sure that saveIt() will never return zero? if the code *LOOKS* wrong, it is wrong. that assert() should never fire by any logical means. signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
On 1 November 2014 09:03, ketmar via Digitalmars-d digitalmars-d@puremagic.com wrote: Hello. let's run this code: info.list[idx] = saveIt(info, count-1); //!!! You could use: emplace(info.list[idx], saveIt(info, count-1));
Re: bug in assigning to dynamic array element
On Sat, 1 Nov 2014 12:34:50 + Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 November 2014 09:03, ketmar via Digitalmars-d digitalmars-d@puremagic.com wrote: Hello. let's run this code: info.list[idx] = saveIt(info, count-1); //!!! You could use: emplace(info.list[idx], saveIt(info, count-1)); i know some workarounds, i just want to know how can i explain this nonsense to newcomers. please, remember that you can't safely assign values to array elements? this is one of the best ways to turn 'em off. wut? are you kidding? that's what they call 'safe language'? thank you, but no. they can't do even array assign, what a... ;-) signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
On 1 November 2014 12:39, ketmar via Digitalmars-d digitalmars-d@puremagic.com wrote: On Sat, 1 Nov 2014 12:34:50 + Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 November 2014 09:03, ketmar via Digitalmars-d digitalmars-d@puremagic.com wrote: Hello. let's run this code: info.list[idx] = saveIt(info, count-1); //!!! You could use: emplace(info.list[idx], saveIt(info, count-1)); i know some workarounds, i just want to know how can i explain this nonsense to newcomers. please, remember that you can't safely assign values to array elements? Or how about: Every side effect is evaluated LTR. So whatever you do, don't have LHS-altering side-effects on the RHS. It may be over the top to explain simply that it is questionable and potentially wrong to assume that things happen in a given order. But you may not be too far off the mark to explain that when the need for an explicit order is required, do it yourself. Avoid surprises.
Re: bug in assigning to dynamic array element
On Sat, 1 Nov 2014 12:58:26 + Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: Or how about: Every side effect is evaluated LTR. So whatever you do, don't have LHS-altering side-effects on the RHS. It may be over the top to explain simply that it is questionable and potentially wrong to assume that things happen in a given order. But you may not be too far off the mark to explain that when the need for an explicit order is required, do it yourself. Avoid surprises. the thing is that dynamic arrays tries to disguise themselves as static arrays. so programmer have not only to know the compiler internals to explain what's going on, but he have to track all types by himself. this is bad for all means. if such assigns will be left untouched, this will inevitably lead to subtle bugs that are really hard to find. if such assigns will be forbidden only for dynamic arrays, any sane person will question that. so i can do this for char[4] and can't do this for char[]? and why do you have the same syntax for completely different things then? if such assigns will be forbidden for any arrays... this is even worse. what? your shiny language can't do what even the simpliest C compiler can do? now try to convince me that D is not a toy. signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
On 1 November 2014 13:05, ketmar via Digitalmars-d
digitalmars-d@puremagic.com wrote:
On Sat, 1 Nov 2014 12:58:26 +
Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote:
Or how about: Every side effect is evaluated LTR. So whatever you do,
don't have LHS-altering side-effects on the RHS. It may be over the
top to explain simply that it is questionable and potentially wrong to
assume that things happen in a given order. But you may not be too
far off the mark to explain that when the need for an explicit order
is required, do it yourself. Avoid surprises.
the thing is that dynamic arrays tries to disguise themselves as static
arrays. so programmer have not only to know the compiler internals to
explain what's going on, but he have to track all types by himself.
this is bad for all means.
They both infact meet each other somewhere in the middle, but that's
another matter.
This doesn't stop the fact that unless you realise that dynamic arrays
are *dynamic* - as in, the base .ptr can and will likely change if you
grow the array .length during midflight of an assignment - you will
run into these problems.
So your only solution is to pre-allocate the ptr in the GC to prevent
the .ptr from moving because it has been reallocated to another area
to accommodate growth.
if such assigns will be left untouched, this will inevitably lead to
subtle bugs that are really hard to find.
Changing the behaviour may also create new bugs that are really hard to find.
if such assigns will be forbidden only for dynamic arrays, any sane
person will question that. so i can do this for char[4] and can't do
this for char[]? and why do you have the same syntax for completely
different things then?
Some people are used to this idea. eg:
std::cout foo;
if such assigns will be forbidden for any arrays... this is even worse.
what? your shiny language can't do what even the simpliest C compiler
can do? now try to convince me that D is not a toy.
That - for sure - is where you're wrong. :-)
#include assert.h
#include stdlib.h
template typename T
struct Array
{
size_t length;
T* ptr;
Array(int size) {
this-length = 0;
this-ptr = new T[size];
}
};
int saveIt(Arraysize_t list)
{
list = Arraysize_t(1);
return 666;
}
int main()
{
Arraysize_t list = Arraysize_t(1);
list.ptr[0] = saveIt(list);
assert(list.ptr[0] == 666);
}
Looks like C++ works in the same way as D! Go figure!
Regards
Iain.
Re: bug in assigning to dynamic array element
On Sat, 1 Nov 2014 13:56:49 + Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: So your only solution is to pre-allocate the ptr in the GC to prevent the .ptr from moving because it has been reallocated to another area to accommodate growth. if this will be left as is, the only solution is to drop D. i already know the language that is filled with UBs, and there are alot of code written in that language already. if such assigns will be left untouched, this will inevitably lead to subtle bugs that are really hard to find. Changing the behaviour may also create new bugs that are really hard to find. please, what sane use can be for this misfeature? it has no sense, it's not doing anything useful and it's plainly wrong. if such assigns will be forbidden only for dynamic arrays, any sane person will question that. so i can do this for char[4] and can't do this for char[]? and why do you have the same syntax for completely different things then? Some people are used to this idea. eg: std::cout foo; that's why i dropped C++. ;-) if such assigns will be forbidden for any arrays... this is even worse. what? your shiny language can't do what even the simpliest C compiler can do? now try to convince me that D is not a toy. That - for sure - is where you're wrong. :-) i was talking about C, not that abomination. but if D will compete with C++ for the quirks... than C++ is a winner. ;-) besides, i clearly see 'list.ptr' there. i don't see 'dynArray.ptr' in D code. if i was using '.ptr' directly and it changes by the way... ok, it was my fault. but i never used '.ptr' in my sample! signature.asc Description: PGP signature
Re: bug in assigning to dynamic array element
On 1 November 2014 14:19, ketmar via Digitalmars-d digitalmars-d@puremagic.com wrote: On Sat, 1 Nov 2014 13:56:49 + Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: if such assigns will be forbidden for any arrays... this is even worse. what? your shiny language can't do what even the simpliest C compiler can do? now try to convince me that D is not a toy. That - for sure - is where you're wrong. :-) i was talking about C, not that abomination. but if D will compete with C++ for the quirks... than C++ is a winner. ;-) besides, i clearly see 'list.ptr' there. i don't see 'dynArray.ptr' in D code. if i was using '.ptr' directly and it changes by the way... ok, it was my fault. but i never used '.ptr' in my sample! You can clearly see the 'list.length' though. You can't possibly go off the assumption that if you grow the size of a dynamic array, it's area in memory won't be relocated.
Re: bug in assigning to dynamic array element
On Sat, 1 Nov 2014 14:44:54 + Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: You can clearly see the 'list.length' though. You can't possibly go off the assumption that if you grow the size of a dynamic array, it's area in memory won't be relocated. i don't even want to know if that was dynamic or static array! why should i remember that? isn't type checking is what compiler does for us? ;-) dynamic arrays either should not look like static arrays, or should work as static arrays. it's ok for class/struct opAssign() to has some side effects, but it's not ok for built-in things. compiler is perfectly able to at least emit error there. or make that code work as it should work by dereferencing .ptr before doing 'mov' and not before calling `saveIt()`. i don't even want to know about that '.ptr' thing after all. the sample should either consistently not updating the array element, or consistenly updating it, following the principle of the least surprise. signature.asc Description: PGP signature
