GC: Understanding potential sources of false pointers

[Nick Sabalausky (Abscissa) via Digitalmars-d-learn]

According to :

"Registers, the stack, and any other memory locations added through the 
GC.addRange function are always scanned conservatively."


1. Can that be safely assumed to be a canonical list of all possible 
sources of false pointers?


2. What about memory allocated through language constructs such as 
"new", append ("~"/"~="), closures, or any others I may be forgetting? 
Is this memory always/never/sometimes set to NO_SCAN? (I assume not 
"always", as that would be silly.) If "sometimes", what are the conditions?


A couple specific examples:

3. Are there any situations where a (for example) int[] or long[] that 
is stored on the GC heap could lead to false pointers?


4. Same question as #3, but if it's an array of structs, and the struct 
type contains no members that are statically-typed as pointers.

Windows X64 Calling conventions

[Tofu Ninja via Digitalmars-d-learn]

I am trying to learn the calling conventions in DMD, I am sure I 
will have more than one question about them so as I come across 
them I will ask them in this thread. I am mainly reading the MSDN 
docs on the x64 calls and looking at disassemblies to confirm 
what I learn.



While I was looking at a call of the form void foo(float), I get 
the following disassembly:


movss   xmm0,dword ptr [_TMP0]
sub rsp,20h
movdrcx,xmm0
callvoid main.foo(float)
add rsp,20h


My question is, why is it passed twice, both in xmm0 and rcx? The 
MSDN docs say floating point are passed in xmm registers, why is 
it also copied in into rcx? Is it necessary for anything? If it 
was skipped would anything break?



Thanks

Re: Forwarding calls to objects of another type

[Johan Fjeldtvedt via Digitalmars-d-learn]

On Tuesday, 11 April 2017 at 02:01:19 UTC, Nicholas Wilson wrote:

On Monday, 10 April 2017 at 21:27:34 UTC, Basile B. wrote:
2) This is about the reduce templates. As I've commented, I 
can't use a template lambda with reduce, but I can use a 
lambda taking ints as arguments. Why is this? The error 
message I get when using the template lambda is:


"template instance reduce!((a, b) => a + b) cannot use local 
'__lambda1' as parameter to non-global template reduce(alias 
fun)()"


No idea for this.


The use of the global identity template will fix this:

see 
https://blog.thecybershadow.net/2015/04/28/the-amazing-template-that-does-nothing/


Thanks, that did work. I think I understand the point about UFCS 
lookup rules, but it still seems strange (it was at least 
surprising) that I couldn't use the template (a, b) => a + b in 
place of (int a, int b) => a + b.

Re: Forwarding calls to objects of another type

[Johan Fjeldtvedt via Digitalmars-d-learn]

On Monday, 10 April 2017 at 21:27:34 UTC, Basile B. wrote:
On Monday, 10 April 2017 at 21:04:10 UTC, Johan Fjeldtvedt 
wrote:

[...]


One way:

[...]


Thanks for the reply. The traits way of doing it seems to be what 
I want. :)



[...]


[...]

Re: Can we disallow appending integer to string?

[Stanislav Blinov via Digitalmars-d-learn]

On Wednesday, 19 April 2017 at 18:40:23 UTC, H. S. Teoh wrote:

A few extra keystrokes to type cast(int) or cast(char) ain't 
gonna kill nobody. In fact, it might even save a few people by 
preventing certain kinds of bugs.


Yup. Not to mention one could have

@property
auto numeric(Flag!"unsigned" unsigned = No.unsigned, C)(C c) 
if(isSomeChar!C)

{
return cast(IntOfSize!(C.sizeof, unsigned))c;
}

auto v = 'a'.numeric;

...or even have an equivalent as a built-in property of character 
types...

Re: Can we disallow appending integer to string?

[H. S. Teoh via Digitalmars-d-learn]

On Wed, Apr 19, 2017 at 05:56:18PM +, Stanislav Blinov via 
Digitalmars-d-learn wrote:
> On Wednesday, 19 April 2017 at 17:34:01 UTC, Jonathan M Davis wrote:
> > Personally, I think that we should have taken the stricter approach
> > and not had integral types implicit convert to character types, but
> > from what I recall, Walter feels pretty strongly about the
> > conversion rules being the way that they are.
> 
> Yep, me too. Generally, I don't think that an implicit conversion (T :
> U) should be allowed if T.init is not equivalent to U.init.

Me three.

Implicit conversion of int to char/dchar/etc. is a horrible, horrible
idea that leads to hard-to-find bugs. The whole point of having a
separate type for char vs. ubyte, unlike in C where char pretty much
means byte/ubyte, is so that we can keep their distinctions straight,
not to continue to confuse them in the typical C manner by having their
distinction blurred by implicit casts.

Personally, I would rather have arithmetic on char (wchar, dchar)
produce results of the same type, so that no implicit conversions are
necessary.  It seems to totally make sense to me to have to explicitly
ask for a character's numerical value -- it documents code intent, which
often also helps the programmer clear his head and avoid mistakes that
would otherwise inevitably creep in. A few extra keystrokes to type
cast(int) or cast(char) ain't gonna kill nobody. In fact, it might even
save a few people by preventing certain kinds of bugs.


T

-- 
Gone Chopin. Bach in a minuet.

Re: Stuck with DMD, and Unit-Threaded

[Atila Neves via Digitalmars-d-learn]

On Tuesday, 18 April 2017 at 07:07:16 UTC, Russel Winder wrote:
On Mon, 2017-04-17 at 22:56 +, Atila Neves via 
Digitalmars-d-learn wrote:



[…]

https://github.com/russel/ApproxGC/pull/2

Unfortunately the auto generated integration test main file 
doesn't quite work (feel free to file a bug on unit-threaded) 
so in that PR I disabled auto-generating it and force added my 
edited version.


What I did there in dub.sdl is my current go-to solution for 
also running integration tests with unit-threaded.




Thanks for that, much appreciated. I am hesitant to commit the 
pull request for now in case get_ut_main gets fixed fairly 
quickly. For the moment I am progressing with the SCons build 
since I got it working.


I wouldn't hold my breath - the fix is annoying and non-trivial. 
Basically I special cased "source" since it's the default for dub 
packages but "test-source" gums up the works and I'd have to look 
at it properly.


The real joy is that I have Unit-Threaded working. It's 
extensions of the unittest D language feature make testing D 
codes far more fun than the basic feature. Thanks for putting 
in the effort.


I'm happy the work is appreciated :)

Atila

Re: Can we disallow appending integer to string?

[Stanislav Blinov via Digitalmars-d-learn]

On Wednesday, 19 April 2017 at 17:34:01 UTC, Jonathan M Davis 
wrote:


Personally, I think that we should have taken the stricter 
approach and not had integral types implicit convert to 
character types, but from what I recall, Walter feels pretty 
strongly about the conversion rules being the way that they are.


Yep, me too. Generally, I don't think that an implicit conversion 
(T : U) should be allowed if T.init is not equivalent to U.init.

Re: Can we disallow appending integer to string?

[Jonathan M Davis via Digitalmars-d-learn]

On Wednesday, April 19, 2017 14:50:38 Stanislav Blinov via Digitalmars-d-
learn wrote:
> On Wednesday, 19 April 2017 at 14:36:13 UTC, Nick Treleaven wrote:
> > This bug is fixed as the code no longer segfaults but throws
> > instead:
> > https://issues.dlang.org/show_bug.cgi?id=5995
> >
> > void main(){
> >
> > string ret;
> > int i = -1;
> > ret ~= i;
> >
> > }
> >
> > Why is it legal to append an integer?
>
> Because integrals implicitly convert to characters of same width
> (byte -> char, short -> wchar, int -> dchar).

Yeah, which reduces the number of casts required when doing arithmetic on
characters and thus reduces bugs there, and I believe that that's the main
reason the implicit conversion from an integral type to a character type
exists. So, having the implicit conversion fixes one set of bugs, and
disallowing it fixes another. We have similar problems with bool.

Personally, I think that we should have taken the stricter approach and not
had integral types implicit convert to character types, but from what I
recall, Walter feels pretty strongly about the conversion rules being the
way that they are.

- Jonathan M Davis

Re: The app hanging after reach 1750MB of RAM

[Stanislav Blinov via Digitalmars-d-learn]

On Wednesday, 19 April 2017 at 07:28:32 UTC, Suliman wrote:

1. You're measuring it wrong. Array length is already measured 
in terms of type size.


So should I do:
cargpspoints.length * cargpspoints[0].sizeof ?


No. .sizeof is the statically known size of a type, it can't take 
into account dynamically allocated memory.
cargpspoint.length * cargpspoints[0].sizeof will tell you the 
estimate size of the array, in bytes.
But each of the elements also has strings - dynamic arrays that 
are allocated elsewhere, their length is not included in that 
calculation.
So you could iterate over the array and sum up lengths of all 
strings to get an estimate.
Even then, that's just that: an estimate. Actual amount of memory 
allocated for a dynamic array T[] *may* be greater than length * 
T.sizeof. The only way to know that is to query the allocator 
used (in this case, GC).

Re: The app hanging after reach 1750MB of RAM

[Suliman via Digitalmars-d-learn]

On Wednesday, 19 April 2017 at 15:18:32 UTC, crimaniak wrote:

On Tuesday, 18 April 2017 at 11:43:24 UTC, Suliman wrote:
I am writing app that extract data from DB to array of 
structures.


void getSingleTrackInfo()
{

foreach(item; getTablesGPSSensorList)
{
ResultRange result = mysqlconnection.query(sqlquery);
auto MySQLPointsLonLat = result.array;
Is ResultRange closing query when exhausted? Did you try to 
call .close() on it after work?


Yes I tried:

ResultRange result = mysqlconnection.query(sqlquery);
auto MySQLPointsLonLat = result.array;
result.close();

Did not help :(

Re: The app hanging after reach 1750MB of RAM

[crimaniak via Digitalmars-d-learn]

On Tuesday, 18 April 2017 at 11:43:24 UTC, Suliman wrote:
I am writing app that extract data from DB to array of 
structures.


void getSingleTrackInfo()
{

foreach(item; getTablesGPSSensorList)
{
ResultRange result = mysqlconnection.query(sqlquery);
auto MySQLPointsLonLat = result.array;
Is ResultRange closing query when exhausted? Did you try to call 
.close() on it after work?

Re: Can we disallow appending integer to string?

[Stanislav Blinov via Digitalmars-d-learn]

On Wednesday, 19 April 2017 at 14:36:13 UTC, Nick Treleaven wrote:
This bug is fixed as the code no longer segfaults but throws 
instead:

https://issues.dlang.org/show_bug.cgi?id=5995

void main(){
string ret;
int i = -1;
ret ~= i;
}

Why is it legal to append an integer?


Because integrals implicitly convert to characters of same width 
(byte -> char, short -> wchar, int -> dchar).

Can we disallow appending integer to string?

[Nick Treleaven via Digitalmars-d-learn]

This bug is fixed as the code no longer segfaults but throws 
instead:

https://issues.dlang.org/show_bug.cgi?id=5995

void main(){
string ret;
int i = -1;
ret ~= i;
}

Why is it legal to append an integer?

Re: The app hanging after reach 1750MB of RAM

[Suliman via Digitalmars-d-learn]

I have added GC.stat 
https://dlang.org/library/core/memory/gc.stats.html here the 
result:


freeSize: 49698640  | usedSize: 170502320
freeSize: 41174592  | usedSize: 217823680
freeSize: 53868576  | usedSize: 247072736
freeSize: 86494800  | usedSize: 307769776
freeSize: 58176640  | usedSize: 499665792
freeSize: 148233232 | usedSize: 534389744
freeSize: 148141376 | usedSize: 534481600
freeSize: 108467312 | usedSize: 641264528
freeSize: 55118432  | usedSize: 694613408
freeSize: 80579472  | usedSize: 803370096
freeSize: 7880  | usedSize: 1006831680
freeSize: 291629360 | usedSize: 860755664
freeSize: 242912736 | usedSize: 976581152
freeSize: 241673232 | usedSize: 977820656
freeSize: 168092160 | usedSize: 1118510592
freeSize: 128405616 | usedSize: 1426632592
freeSize: 73146272  | usedSize: 1616109664
freeSize: 17962320  | usedSize: 1671293616
freeSize: 20342912  | usedSize: 1736021888

on the last value app is hanging.

Re: The app hanging after reach 1750MB of RAM

[Suliman via Digitalmars-d-learn]

auto mymem = cargpspoints.length * 
typeof(cargpspoints[0]).sizeof;			

writeln(mymem);

And it's print: 16963440
it's about 16MB...

What is takes all other memory?


1. You're measuring it wrong. Array length is already measured 
in terms of type size.


So should I do:
cargpspoints.length * cargpspoints[0].sizeof ?

Btw, `cargpspoints.length * typeof(cargpspoints[0]).sizeof` and 
`cargpspoints.length * cargpspoints[0].sizeof` show same result.