[Issue 24009] The garbage collector tries to allocate memory while the program is out of memory

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=24009

Vladimir Panteleev  changed:

   What|Removed |Added

   See Also||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=24329

--

[Issue 24009] New: The garbage collector tries to allocate memory while the program is out of memory

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=24009

  Issue ID: 24009
   Summary: The garbage collector tries to allocate memory while
the program is out of memory
   Product: D
   Version: D2
  Hardware: x86_64
OS: Linux
Status: NEW
  Severity: normal
  Priority: P3
 Component: druntime
  Assignee: nob...@puremagic.com
  Reporter: dlang-bugzi...@thecybershadow.net

- When the program runs out of memory, the runtime starts a GC cycle.
- While the GC is running, the program is still out of memory, so requests for
more memory from the OS will fail.
- Therefore, it should not request more memory from the OS.
- The GC requests more memory from the OS while it is running.
- Therefore, the GC may cause the program to crash when it runs out of memory.

Unfortunately I do not have a standalone reproducer, but here is a stack trace
which illustrates the problem:

#0  0x559438b8 in onOutOfMemoryErrorNoGC ()
#1  0x5593ed4b in
core.internal.gc.impl.conservative.gc.Gcx.ToScanStack!(void*).ToScanStack.grow()
()
#2  0x55939957 in
core.internal.gc.impl.conservative.gc.Gcx.collectRoots(void*, void*) ()
#3  0x559426a0 in core.thread.threadbase.thread_scanAll(scope
void(void*, void*) nothrow
delegate).__lambda2!(core.thread.threadbase.ScanType, void*,
void*).__lambda2(core.thread.threadbase.ScanType, void*, void*) ()
#4  0x559472dc in core.thread.threadbase.scanAllTypeImpl(scope
void(core.thread.threadbase.ScanType, void*, void*) nothrow delegate, void*) ()
#5  0x55947219 in core.thread.threadbase.thread_scanAllType(scope
void(core.thread.threadbase.ScanType, void*, void*) nothrow
delegate).__lambda2!(void*).__lambda2(void*) ()
#6  0x559426e0 in core.thread.osthread.callWithStackShell(scope
void(void*) nothrow delegate) ()
#7  0x559471ee in thread_scanAllType ()
#8  0x55942666 in thread_scanAll ()
#9  0x55939a46 in
core.internal.gc.impl.conservative.gc.Gcx.collectAllRoots(bool) ()
#10 0x5593bb92 in
core.internal.gc.impl.conservative.gc.Gcx.markParallel(bool) ()
#11 0x5593b1e0 in
core.internal.gc.impl.conservative.gc.Gcx.fullcollect(bool, bool, bool) ()
#12 0x55939269 in
core.internal.gc.impl.conservative.gc.Gcx.smallAlloc(ulong, ref ulong, uint,
const(TypeInfo)) ()
#13 0x5593f235 in
core.internal.gc.impl.conservative.gc.ConservativeGC.runLocked!(core.internal.gc.impl.conservative.gc.ConservativeGC.mallocNoSync(ulong,
uint, ref ulong, const(TypeInfo)),
core.internal.gc.impl.conservative.gc.mallocTime,
core.internal.gc.impl.conservative.gc.numMallocs, ulong, uint, ulong,
const(TypeInfo)).runLocked(ref ulong, ref uint, ref ulong, ref const(TypeInfo))
()
#14 0x55936cce in
core.internal.gc.impl.conservative.gc.ConservativeGC.qalloc(ulong, uint, scope
const(TypeInfo)) ()
#15 0x558c4ba3 in gc_qalloc ()
#16 0x5590e7fa in rt.lifetime.__arrayAlloc(ulong, ref
core.memory.BlkInfo_, scope const(TypeInfo), const(TypeInfo)) ()
#17 0x558cc92e in _d_arrayappendcTX ()
#18 0x558cc0e9 in _d_arrayappendT ()

--

[Issue 18763] Segfault in garbage collector

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18763

Vladimir Panteleev  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||dlang-bugzilla@thecybershad
   ||ow.net
 Resolution|--- |INVALID

--- Comment #3 from Vladimir Panteleev  ---
This bug is unreproducible as it is missing sndfile.d and the mentioned
oscillofun.flac.

--

[Issue 13801] Garbage collector fails to work after lots of small allocations

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=13801

Vladimir Panteleev  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||dlang-bugzilla@thecybershad
   ||ow.net
 Resolution|--- |WORKSFORME

--

[Issue 18763] Segfault in garbage collector

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18763

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P3  |P2
   Severity|normal  |critical

--

[Issue 10184] [GC] Port Tango's concurrent garbage collector (CDGC) to D2's druntime

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=10184

Richard Cattermole  changed:

   What|Removed |Added

 CC||alphaglosi...@gmail.com

--- Comment #5 from Richard Cattermole  ---
Do we still need this? We have a fork-based GC which from what I'm seeing in
#985 is all that is supported.

--

[Issue 10184] [GC] Port Tango's concurrent garbage collector (CDGC) to D2's druntime

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=10184

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P2  |P4

--

[Issue 13801] Garbage collector fails to work after lots of small allocations

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=13801

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P1  |P2

--

[Issue 18763] Segfault in garbage collector

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18763

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P1  |P3

--

[Issue 18441] Add comments to GC page about tradeoffs in design and impl of GC (garbage collector)

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18441

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P1  |P4

--

[Issue 20746] Change LCG in garbage collector treap to use full 64 bits of state instead of 48

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=20746

Dlang Bot  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |FIXED

--- Comment #3 from Dlang Bot  ---
dlang/druntime pull request #3042 "Change LCG in garbage collector treap to use
full 64 bits of state instead of 48" was merged into master:

- 275a33117e5266da65cb5a6393634bbdde7fd00d by Nathan Sashihara:
  Fix Issue 20746 - Change LCG in garbage collector treap to use full 64 bits
of state instead of 48

https://github.com/dlang/druntime/pull/3042

--

[Issue 20746] Change LCG in garbage collector treap to use full 64 bits of state instead of 48

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=20746

Dominikus Dittes Scherkl  changed:

   What|Removed |Added

 CC||domini...@scherkl.de

--- Comment #2 from Dominikus Dittes Scherkl  ---
Sorry, but there was no bug and this is NOT an improvement.
The generator output is 32 bit and the internal state of course will use the
whole 64 bit (even if not from the start), but it put out bits from the middle
of the state.
Output the lowest 32 bits will make it less "random" - if fact very
predictable.

Please, this is not a good pseudo-random number generateor, but at least it
worked. now its destroyed.
Revert this change!

--

[Issue 20746] Change LCG in garbage collector treap to use full 64 bits of state instead of 48

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=20746

Dlang Bot  changed:

   What|Removed |Added

   Keywords||pull

--- Comment #1 from Dlang Bot  ---
@n8sh created dlang/druntime pull request #3042 "Change LCG in garbage
collector treap to use full 64 bits of state instead of 48" fixing this issue:

- Fix Issue 20746 - Change LCG in garbage collector treap to use full 64 bits
of state instead of 48

https://github.com/dlang/druntime/pull/3042

--

[Issue 20746] New: Change LCG in garbage collector treap to use full 64 bits of state instead of 48

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=20746

  Issue ID: 20746
   Summary: Change LCG in garbage collector treap to use full 64
bits of state instead of 48
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: druntime
  Assignee: nob...@puremagic.com
  Reporter: n8sh.second...@hotmail.com

rt.util.random.Rand48 is an LCG that holds its state in a 64-bit long but only
makes use of the low 48 bits. Replacing it with an LCG that makes use of the
full 64 bits would be a strict improvement as it would take no additional space
and would not be slower.

Rand48 is used only in rt.util.container.treap.Treap which in turn is used only
by the garbage collector.

--

[Issue 18763] Segfault in garbage collector

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18763

Basile-z  changed:

   What|Removed |Added

 CC|b2.t...@gmx.com |

--

Re: § 28.3 Pointers and the Garbage Collector

[kdevel via Digitalmars-d-learn]

On Sunday, 7 April 2019 at 10:17:53 UTC, AltFunction1 wrote:

On Sunday, 7 April 2019 at 10:05:26 UTC, kdevel wrote:

In § 28.3 Pointers and the Garbage Collector [1] we read

   Do not add or subtract an offset to a pointer such that the 
result points
   outside of the bounds of the garbage collected object 
originally allocated.


[...]


No the foo() code would work in D too but in D since we have a 
true array type with ptr+length you should not write this kind 
of code, which is not @safe BTW.


I appreciate your constructive reply. What about the formal 
validity wrt. to
the documentation? Is char *e = p + 10; outside of the bounds of 
the garbage collected object? Does for (q = p; q < e; ++q) depend 
on the ordering of pointers?

Re: § 28.3 Pointers and the Garbage Collector

[AltFunction1 via Digitalmars-d-learn]

On Sunday, 7 April 2019 at 10:05:26 UTC, kdevel wrote:

In § 28.3 Pointers and the Garbage Collector [1] we read

   Do not add or subtract an offset to a pointer such that the 
result points
   outside of the bounds of the garbage collected object 
originally allocated.


[...]


No the foo() code would work in D too but in D since we have a 
true array type with ptr+length you should not write this kind of 
code, which is not @safe BTW.

§ 28.3 Pointers and the Garbage Collector

[kdevel via Digitalmars-d-learn]

In § 28.3 Pointers and the Garbage Collector [1] we read

   Do not add or subtract an offset to a pointer such that the 
result points
   outside of the bounds of the garbage collected object 
originally allocated.


  char* p = new char[10];
  char* q = p + 6; // ok
  q = p + 11;  // error: undefined behavior
  q = p - 1;   // error: undefined behavior

C and C++ allow a pointer to point to the (non-existing) element 
after the end

of the array:

   char *e = p + 10;

Does this point "outside of the bounds of the garbage collected 
object"?


In § 28.3 we also read

   Do not depend on the ordering of pointers:

  if (p1 < p2)  // error: undefined behavior
 ...

   since, again, the garbage collector can move objects around in 
memory.


In C and C++ we are used to read code like this:

   void foo ()
   {
  char *p = new char [10];
  char *e = p + 10;
  char *q;
  for (q = p; q < e; ++q)
 ...
   }

Does this for-loop "depend on the ordering of pointers"?

[1] https://dlang.org/spec/garbage.html#pointers_and_gc

Re: Java ZGC - The Z Garbage Collector

[Brian via Digitalmars-d]

The Z Garbage Collector, also known as ZGC, is a scalable low 
latency garbage collector designed to meet the following goals:


Pause times do not exceed 10ms
Pause times do not increase with the heap or live-set size
Handle heaps ranging from a few hundred megabytes to multi 
terabytes in size


At a glance, ZGC is:

Concurrent
Region-based
Compacting
NUMA-aware
Using colored pointers
Using load barriers

At its core, ZGC is a concurrent garbage collector, meaning all 
heavy lifting work is done while Java threads continue to 
execute. This greatly limits the impact garbage collection will 
have on your application's response time.

Java ZGC - The Z Garbage Collector

[Brian via Digitalmars-d]

The goal of this project is to create a scalable low latency 
garbage collector capable of handling heaps ranging from a few 
gigabytes to multi terabytes in size, with GC pause times not 
exceeding 10ms.



https://wiki.openjdk.java.net/display/zgc/Main


Be alright with D ^^

[Issue 18763] Segfault in garbage collector

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18763

|  changed:

   What|Removed |Added

 CC||dhase...@gmail.com

--- Comment #2 from |  ---
Created attachment 1685
  --> https://issues.dlang.org/attachment.cgi?id=1685=edit
Test case for segfault

Added test case (as seen on IRC).

Stacktrace (also as seen on IRC):

Program received signal SIGSEGV, Segmentation fault.
0x555d5b35 in _D2gc4impl12conservativeQw3Gcx11fullcollectMFNbbZm ()
(gdb) bt
#0 0x555d5b35 in _D2gc4impl12conservativeQw3Gcx11fullcollectMFNbbZm ()
#1 0x555d8856 in
_D2gc4impl12conservativeQw14ConservativeGC__T9runLockedS_DQCeQCeQCcQCnQBs18fullCollectNoStackMFNbZ2goFNbPSQEaQEaQDyQEj3GcxZmTQvZQDfMFNbKQBgZm
()
#2 0x555d2961 in
_D2gc4impl12conservativeQw14ConservativeGC18fullCollectNoStackMFNbZv ()
#3 0x555d2932 in
_D2gc4impl12conservativeQw14ConservativeGC14collectNoStackMFNbZv ()
#4 0x555e5cf1 in gc_term ()
#5 0x555da3cb in rt_term ()
#6 0x555c63b5 in rt.dmain2._d_run_main(int, char**, extern(C)
int(char[][]) function).runAll() ()
#7 0x555c6290 in rt.dmain2._d_run_main(int, char**, extern(C)
int(char[][]) function).tryExec(scope void() delegate) ()
#8 0x555c61fb in _d_run_main ()
#9 0x555bbede in main ()
#10 0x76bf2a87 in __libc_start_main (main=0x555bbebc ,
argc=2, argv=0x7fffea68, init=, fini=,
rtld_fini=, stack_end=0x7fffea58)
at ../csu/libc-start.c:310
#11 0x555b114a in _start ()

--

[Issue 18763] Segfault in garbage collector

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18763

Basile B.  changed:

   What|Removed |Added

 CC||b2.t...@gmx.com

--- Comment #1 from Basile B.  ---
You forgot to upload an attachment apparently.
SEGFAULT by GC can be caused by a destructor that allocates.
destructor are not allowed to.

--

[Issue 18763] New: Segfault in garbage collector

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18763

  Issue ID: 18763
   Summary: Segfault in garbage collector
   Product: D
   Version: D2
  Hardware: x86_64
OS: Linux
Status: NEW
  Severity: normal
  Priority: P1
 Component: druntime
  Assignee: nob...@puremagic.com
  Reporter: lieuw...@gmail.com

This program gives a segfault somewhere in the garbage collector when ran with
oscillofun.flac as input.

--

[Issue 18441] Add comments to GC page about tradeoffs in design and impl of GC (garbage collector)

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18441

greenify  changed:

   What|Removed |Added

 CC||greeen...@gmail.com

--- Comment #3 from greenify  ---
There is also this nice GC series: https://dlang.org/blog/the-gc-series/

We should probably link to it from the front page.

--

[Issue 18441] Add comments to GC page about tradeoffs in design and impl of GC (garbage collector)

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18441

--- Comment #2 from anonymous4  ---
https://blog.plan99.net/modern-garbage-collection-911ef4f8bd8e GC
questionnaire, may be interesting to fill.

--

[Issue 18441] Add comments to GC page about tradeoffs in design and impl of GC (garbage collector)

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18441

Michael <mich...@toohuman.io> changed:

   What|Removed |Added

 CC||mich...@toohuman.io

--- Comment #1 from Michael <mich...@toohuman.io> ---
It would be great to have a page on the dlang website tackling the stigma
associated with the garbage collector. We can demo some performance benchmarks,
perhaps, as well as discuss things to keep in mind. Most importantly, use it to
highlight all of the nice features it enables like dynamic arrays as a language
feature etc.

--

[Issue 18441] New: Add comments to GC page about tradeoffs in design and impl of GC (garbage collector)

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18441

  Issue ID: 18441
   Summary: Add comments to GC page about tradeoffs in design and
impl of GC (garbage collector)
   Product: D
   Version: D2
  Hardware: x86_64
OS: Linux
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: dlang.org
  Assignee: nob...@puremagic.com
  Reporter: jgabri...@fastmail.fm

I regularly hear complaints online about D's GC being slow or not decent, but
those complaints rarely include details.

I've twice seen Walter explain some tradeoffs that were made in the design and
implementation of the D GC. [One was on the forum]. [Another on reddit], where
he wrote:

"You can make a moving GC with D, it's called a "mostly copying" collector. The
trick is to not move things that may have a pointer to them."

and

"Certain GCs instrument the generated code with write gates which notify the GC
when memory writes are being made. GC-focused languages rely on this to make
the GC better, at the cost of lower performance in the native code.

D has a GC, but is not a GC focused language. The performance cost of write
gates is an unacceptable compromise in the context of D."

***

The [D GC page](https://dlang.org/spec/garbage.html) may need a section
describing these tradeoffs that were made in the design and implementation of
the GC so people can easily see why it is the way it is. That may help avoid
regurgitated complaints of "it's too slow" and provide a link to point those
folks to.

[One was on the forum]:
https://forum.dlang.org/post/p5i3j9$1uv6$1...@digitalmars.com

[Another on reddit]:
https://www.reddit.com/r/programming/comments/7xih66/the_expressive_c17_coding_challenge_in_d/

--

Re: For Inspiration: arena-based, quad-color incremental, generational, non-copying, high-speed, cache-optimized garbage collector

[Kagamin via Digitalmars-d]

On Sunday, 4 February 2018 at 08:41:04 UTC, rikki cattermole 
wrote:

Single threaded simplifies a lot. So no, not useful.


Useful for server applications that more often than not have 
independent threads.

Re: For Inspiration: arena-based, quad-color incremental, generational, non-copying, high-speed, cache-optimized garbage collector

[Robert M. Münch via Digitalmars-d]

On 2018-02-04 12:41:16 +, notna said:

... found this a nice read some time ago... 
https://blog.plan99.net/modern-garbage-collection-911ef4f8bd8e


Thanks, pretty interesting.

BTW: I used this stuff about 20 years ago: http://www.microquill.com/ 
and they are still around. And yes, it was damn fast and HeapAgent was 
a great tool to use.


--
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

Re: For Inspiration: arena-based, quad-color incremental, generational, non-copying, high-speed, cache-optimized garbage collector

[Dmitry Olshansky via Digitalmars-d]

On Sunday, 4 February 2018 at 09:04:41 UTC, rikki cattermole 
wrote:

On 04/02/2018 8:49 AM, Robert M. Münch wrote:

On 2018-02-04 08:41:04 +, rikki cattermole said:

"It doesn't need to be concurrent, since Lua states are 
completely independent."


Single threaded simplifies a lot. So no, not useful.


Well, I wouldn't be able to judge that fast from a single line 
in the text, if the concept can/can't be tweaked/extended. 
That's why I added "for inspiration" to the subject...


Really we ought to use Boehm, as the newer versions can be 
generational


Not really you can’t - first check the API to understand that 
without manually placed (or injected by compiler) safepoints it 
doesn’t work.





It is potentially quite a lot better than our version of it.


It might be.

Re: For Inspiration: arena-based, quad-color incremental, generational, non-copying, high-speed, cache-optimized garbage collector

[Dmitry Olshansky via Digitalmars-d]

On Sunday, 4 February 2018 at 08:33:35 UTC, Robert M. Münch wrote:

Hi, not sure if anyone is aware of this:

http://wiki.luajit.org/New-Garbage-Collector

I don't know if anythin like this would fit for D but the 
concept is described pretty extensive.


Generational -> write barriers -> no, not really.

Re: For Inspiration: arena-based, quad-color incremental, generational, non-copying, high-speed, cache-optimized garbage collector

[notna via Digitalmars-d]

On Sunday, 4 February 2018 at 09:04:41 UTC, rikki cattermole 
wrote:

On 04/02/2018 8:49 AM, Robert M. Münch wrote:

On 2018-02-04 08:41:04 +, rikki cattermole said:

"It doesn't need to be concurrent, since Lua states are 
completely independent."


Single threaded simplifies a lot. So no, not useful.


Well, I wouldn't be able to judge that fast from a single line 
in the text, if the concept can/can't be tweaked/extended. 
That's why I added "for inspiration" to the subject...


Really we ought to use Boehm, as the newer versions can be 
generational


It is potentially quite a lot better than our version of it.


... found this a nice read some time ago... 
https://blog.plan99.net/modern-garbage-collection-911ef4f8bd8e

Re: For Inspiration: arena-based, quad-color incremental, generational, non-copying, high-speed, cache-optimized garbage collector

[rikki cattermole via Digitalmars-d]

On 04/02/2018 8:49 AM, Robert M. Münch wrote:

On 2018-02-04 08:41:04 +, rikki cattermole said:

"It doesn't need to be concurrent, since Lua states are completely 
independent."


Single threaded simplifies a lot. So no, not useful.


Well, I wouldn't be able to judge that fast from a single line in the 
text, if the concept can/can't be tweaked/extended. That's why I added 
"for inspiration" to the subject...


Really we ought to use Boehm, as the newer versions can be 
generational


It is potentially quite a lot better than our version of it.

Re: For Inspiration: arena-based, quad-color incremental, generational, non-copying, high-speed, cache-optimized garbage collector

[Robert M. Münch via Digitalmars-d]

On 2018-02-04 08:41:04 +, rikki cattermole said:

"It doesn't need to be concurrent, since Lua states are completely 
independent."


Single threaded simplifies a lot. So no, not useful.


Well, I wouldn't be able to judge that fast from a single line in the 
text, if the concept can/can't be tweaked/extended. That's why I added 
"for inspiration" to the subject...


--
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

Re: For Inspiration: arena-based, quad-color incremental, generational, non-copying, high-speed, cache-optimized garbage collector

[rikki cattermole via Digitalmars-d]

On 04/02/2018 8:33 AM, Robert M. Münch wrote:

Hi, not sure if anyone is aware of this:

http://wiki.luajit.org/New-Garbage-Collector

I don't know if anythin like this would fit for D but the concept is 
described pretty extensive.




"It doesn't need to be concurrent, since Lua states are completely 
independent."


Single threaded simplifies a lot. So no, not useful.

For Inspiration: arena-based, quad-color incremental, generational, non-copying, high-speed, cache-optimized garbage collector

[Robert M. Münch via Digitalmars-d]

Hi, not sure if anyone is aware of this:

http://wiki.luajit.org/New-Garbage-Collector

I don't know if anythin like this would fit for D but the concept is 
described pretty extensive.


--
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

Re: Garbage Collector profiling and the dynamic array reserve() function

[bauss via Digitalmars-d-learn]

On Wednesday, 18 October 2017 at 15:39:43 UTC, Steven 
Schveighoffer wrote:

On 10/18/17 1:40 AM, Tony wrote:
On Tuesday, 17 October 2017 at 13:27:24 UTC, Steven 
Schveighoffer wrote:




I don't know what "allocations" represents, but reserve 
actually calls gc_malloc, and the others do not (the space is 
available to expand into the block). There should be only one 
allocation IMO.




So there should be a bug report written for this?



It all depends on what "allocations" means. I'd wait to find 
out from someone who is familiar with the GC profiling.


-Steve


I don't have a lot of clues on how the GC profiling work, but 
looking at reserve() it calls mem.xmalloc() for allocations which 
in fact calls GC.malloc().


Looking at the profiling for GC though:
https://github.com/dlang/dmd/blob/69567a32c5bffae5513b41e7691c91b50766b552/src/ddmd/e2ir.d#L5952

It doesn't look like there's anything for array reserve calls, 
unless:

 [ RTLSYM_ALLOCMEMORY, RTLSYM_TRACEALLOCMEMORY ]
are triggered from the allocations done in reserve(), but I have 
no idea about that.

Re: Garbage Collector profiling and the dynamic array reserve() function

[Steven Schveighoffer via Digitalmars-d-learn]

On 10/18/17 1:40 AM, Tony wrote:

On Tuesday, 17 October 2017 at 13:27:24 UTC, Steven Schveighoffer wrote:



I don't know what "allocations" represents, but reserve actually calls 
gc_malloc, and the others do not (the space is available to expand 
into the block). There should be only one allocation IMO.




So there should be a bug report written for this?



It all depends on what "allocations" means. I'd wait to find out from 
someone who is familiar with the GC profiling.


-Steve

Re: Garbage Collector profiling and the dynamic array reserve() function

[Tony via Digitalmars-d-learn]

On Tuesday, 17 October 2017 at 13:27:24 UTC, Steven Schveighoffer 
wrote:




I don't know what "allocations" represents, but reserve 
actually calls gc_malloc, and the others do not (the space is 
available to expand into the block). There should be only one 
allocation IMO.


-Steve


So there should be a bug report written for this?

Re: Garbage Collector profiling and the dynamic array reserve() function

[Steven Schveighoffer via Digitalmars-d-learn]

On 10/17/17 2:14 AM, Tony wrote:

Found this unanswered question on StackOverflow.

This program:

import std.stdio;

void add(ref int[] data)
{
     data ~= 1;
     data ~= 2;
}

void main()
{
     int[] a;
     writeln("capacity:",a.capacity);
     auto cap = a.reserve(1000); // allocated may be more than requested
     assert(cap >= 1000);
     assert(cap == a.capacity);
     writeln("capacity:",a.capacity);
     a.add();
     writeln(a);

}

compiled with "dmd -profile=gc"

has this output in profilegc.log

bytes allocated, allocations, type, function, file:line
   4  1    int[] profiling.add profiling.d:8
   4  1    int[] profiling.add profiling.d:7

The question is: why doesn't using reserve() cause an allocation to be 
shown?


I don't know what "allocations" represents, but reserve actually calls 
gc_malloc, and the others do not (the space is available to expand into 
the block). There should be only one allocation IMO.


-Steve

Garbage Collector profiling and the dynamic array reserve() function

[Tony via Digitalmars-d-learn]

Found this unanswered question on StackOverflow.

This program:

import std.stdio;

void add(ref int[] data)
{
data ~= 1;
data ~= 2;
}

void main()
{
int[] a;
writeln("capacity:",a.capacity);
auto cap = a.reserve(1000); // allocated may be more than 
requested

assert(cap >= 1000);
assert(cap == a.capacity);
writeln("capacity:",a.capacity);
a.add();
writeln(a);

}

compiled with "dmd -profile=gc"

has this output in profilegc.log

bytes allocated, allocations, type, function, file:line
  4   1 int[] profiling.add profiling.d:8
  4   1 int[] profiling.add profiling.d:7

The question is: why doesn't using reserve() cause an allocation 
to be shown?

[Issue 13801] Garbage collector fails to work after lots of small allocations

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=13801

calex  changed:

   What|Removed |Added

 CC||calex+bugzilla-mail@aristow
   ||eb.net

--

Re: New Garbage Collector?

[Nick Treleaven via Digitalmars-d]

On Saturday, 22 July 2017 at 10:17:49 UTC, Temtaime wrote:

The new precise GC will be never added to druntime.


2.075 has precise scanning of the DATA and the TLS segments (on 
Windows):

http://dlang.org/spec/garbage.html#precise_dataseg

This seems like significant progress towards precise scanning of 
the heap.

Re: New Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Sunday, 23 July 2017 at 09:54:36 UTC, Moritz Maxeiner wrote:


Have you read the excellent blog posts about to use the GC in D 
by Micheal Parker [2][3]?


* Have you read the excellent blog posts about how to use the GC 
in D by Michael Parker [2][3]?


Sorry about that.


[2] http://dlang.org/blog/2017/03/20/dont-fear-the-reaper/
[3] https://dlang.org/blog/2017/06/16/life-in-the-fast-lane/

Re: New Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Sunday, 23 July 2017 at 09:34:09 UTC, Temtaime wrote:

On Saturday, 22 July 2017 at 16:35:03 UTC, Igor Shirkalin wrote:

On Saturday, 22 July 2017 at 10:17:49 UTC, Temtaime wrote:

On Saturday, 22 July 2017 at 04:53:17 UTC, aedt wrote:
In the forum, I saw a thread about someone working on a new 
GC. Just wanted to know if there's any updates on that. And 
what issues is it going to fix..


Personally, I would be greatly delighted if it acknowledges 
the Stop-The-World problem[1]. Going around it is shown not 
to be impossible[2]. I do understand it's not trivial task 
but how is the community/core devs supporting this?


[1] https://dlang.org/spec/garbage.html
[2] https://hub.docker.com/r/nimlang/nim/


The new precise GC will be never added to druntime.
It is dead, man.


Are you real developer of new GC?


I am that who sees _no progress_ for many years.


Slow progress != no progess [1]


Current implementation does memory leaks


Which bug reports are you referring to specifically?


works slow


It's not the best a non-generational stop-the-world 
mark-and-sweep GC can be, but in my experience it's good enough 
for non-realtime applications as long as you don't use it for 
huge memory chunks (i.e. stick to objects without large arrays).



and has some other disadvantages.


Which are you referring to?


And we will suffer from this many years.


Have you read the excellent blog posts about to use the GC in D 
by Micheal Parker [2][3]?


[1] https://github.com/dlang/druntime/commits/master/src/gc
[2] http://dlang.org/blog/2017/03/20/dont-fear-the-reaper/
[3] https://dlang.org/blog/2017/06/16/life-in-the-fast-lane/

Re: New Garbage Collector?

[Temtaime via Digitalmars-d]

On Saturday, 22 July 2017 at 16:35:03 UTC, Igor Shirkalin wrote:

On Saturday, 22 July 2017 at 10:17:49 UTC, Temtaime wrote:

On Saturday, 22 July 2017 at 04:53:17 UTC, aedt wrote:
In the forum, I saw a thread about someone working on a new 
GC. Just wanted to know if there's any updates on that. And 
what issues is it going to fix..


Personally, I would be greatly delighted if it acknowledges 
the Stop-The-World problem[1]. Going around it is shown not 
to be impossible[2]. I do understand it's not trivial task 
but how is the community/core devs supporting this?


[1] https://dlang.org/spec/garbage.html
[2] https://hub.docker.com/r/nimlang/nim/


The new precise GC will be never added to druntime.
It is dead, man.


Are you real developer of new GC?


I am that who sees _no progress_ for many years. Current 
implementation does memory leaks, works slow and has some other 
disadvantages.
But no one of the « real » developers sees reason to rewrite the 
gc, because they want to play with theirs toys bringing new not 
so essential stuff to the d.

And we will suffer from this many years.

Re: New Garbage Collector?

[Dmitry Olshansky via Digitalmars-d]

On Saturday, 22 July 2017 at 04:53:17 UTC, aedt wrote:
In the forum, I saw a thread about someone working on a new GC. 
Just wanted to know if there's any updates on that. And what 
issues is it going to fix..


Personally, I would be greatly delighted if it acknowledges the 
Stop-The-World problem[1]. Going around it is shown not to be 
impossible[2]. I do understand it's not trivial task but how is 
the community/core devs supporting this?


[1] https://dlang.org/spec/garbage.html
[2] https://hub.docker.com/r/nimlang/nim/


I'm working on new GC. First iteration aims to be faster at 
allocation and at collection without changing the fundamental 
mark-sweep strategy. Second stage is getting concurrent 
collection. It wouldn't be a real-time miracle, but a pause time 
in the range of 10-20ms is possible.

Re: New Garbage Collector?

[Stefan Koch via Digitalmars-d]

On Saturday, 22 July 2017 at 16:35:03 UTC, Igor Shirkalin wrote:

On Saturday, 22 July 2017 at 10:17:49 UTC, Temtaime wrote:

On Saturday, 22 July 2017 at 04:53:17 UTC, aedt wrote:
In the forum, I saw a thread about someone working on a new 
GC. Just wanted to know if there's any updates on that. And 
what issues is it going to fix..


Personally, I would be greatly delighted if it acknowledges 
the Stop-The-World problem[1]. Going around it is shown not 
to be impossible[2]. I do understand it's not trivial task 
but how is the community/core devs supporting this?


[1] https://dlang.org/spec/garbage.html
[2] https://hub.docker.com/r/nimlang/nim/


The new precise GC will be never added to druntime.
It is dead, man.


Are you real developer of new GC?


He's not he's just a naysayer :)

Re: New Garbage Collector?

[Igor Shirkalin via Digitalmars-d]

On Saturday, 22 July 2017 at 10:17:49 UTC, Temtaime wrote:

On Saturday, 22 July 2017 at 04:53:17 UTC, aedt wrote:
In the forum, I saw a thread about someone working on a new 
GC. Just wanted to know if there's any updates on that. And 
what issues is it going to fix..


Personally, I would be greatly delighted if it acknowledges 
the Stop-The-World problem[1]. Going around it is shown not to 
be impossible[2]. I do understand it's not trivial task but 
how is the community/core devs supporting this?


[1] https://dlang.org/spec/garbage.html
[2] https://hub.docker.com/r/nimlang/nim/


The new precise GC will be never added to druntime.
It is dead, man.


Are you real developer of new GC?

Re: New Garbage Collector?

[tetyys via Digitalmars-d]

On Saturday, 22 July 2017 at 10:17:49 UTC, Temtaime wrote:

The new precise GC will be never added to druntime.
It is dead, man.


Why?

Re: New Garbage Collector?

[Temtaime via Digitalmars-d]

On Saturday, 22 July 2017 at 04:53:17 UTC, aedt wrote:
In the forum, I saw a thread about someone working on a new GC. 
Just wanted to know if there's any updates on that. And what 
issues is it going to fix..


Personally, I would be greatly delighted if it acknowledges the 
Stop-The-World problem[1]. Going around it is shown not to be 
impossible[2]. I do understand it's not trivial task but how is 
the community/core devs supporting this?


[1] https://dlang.org/spec/garbage.html
[2] https://hub.docker.com/r/nimlang/nim/


The new precise GC will be never added to druntime.
It is dead, man.

New Garbage Collector?

[aedt via Digitalmars-d]

In the forum, I saw a thread about someone working on a new GC. 
Just wanted to know if there's any updates on that. And what 
issues is it going to fix..


Personally, I would be greatly delighted if it acknowledges the 
Stop-The-World problem[1]. Going around it is shown not to be 
impossible[2]. I do understand it's not trivial task but how is 
the community/core devs supporting this?


[1] https://dlang.org/spec/garbage.html
[2] https://hub.docker.com/r/nimlang/nim/

Re: Current state of the Garbage Collector

[Ali Çehreli via Digitalmars-d]

Recent thread about this topic:

  http://forum.dlang.org/thread/ewdoqmvslcnypzyrb...@forum.dlang.org

Ali

Current state of the Garbage Collector

[helxi via Digitalmars-d]

First off, let me admit that I am no expert on Garbage 
Collector Technology or Language implementation. I made this 
thread only to know if there is any attempt being done to improve 
the current GC's efficiency.


As the Garbage Collection techniques are becoming more 
efficient, how is D keeping up with the competition?


As far as I understand, our GC is not the smartest one out 
there which can be a serious bottleneck to the overall 
performance. D doesn't have an incremental GC (let alone a 
generational one afaik), and the (fully conservative, 
stop-the-world) one it has just isn't very "good". So D sometimes 
would force me to revert to manual memory management, which I 
have no interest in whatsoever.


If you look at Nim, it has a nice thread local GC, it's 
claimed to be faster in theory. (I have a feeling it will pose 
problems if you want to go multi-threaded).

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Saturday, 29 April 2017 at 10:54:02 UTC, bachmeier wrote:
Many invested in Rust and C++ will look for arguments to 
support staying with their language. I've come to the 
conclusion that the D community is mostly to blame for not 
making a good case to the other group that are open to D, but 
for technical reasons or simply personal preference don't like 
GC, that D is still an option. There's no excuse for not making 
it easy to evaluate one's options for GC-less programming if we 
support that style of programming.


Of course. It is useful for D-users who are looking for 
alternatives to the GC if written for that group. Finding 
complete and up to date information is always the biggest 
challenge for newbies.


Just don't expect it to have an effect on reddit users...

Re: Garbage Collector?

[bachmeier via Digitalmars-d]

On Friday, 28 April 2017 at 19:49:35 UTC, Ola Fosheim Grøstad 
wrote:
On Friday, 28 April 2017 at 17:48:47 UTC, Ola Fosheim Grøstad 
wrote:

On Friday, 28 April 2017 at 17:42:18 UTC, bachmeier wrote:
I'm hoping to put all information in one place. Then when 
someone on Reddit or HN or here starts making claims about 
the GC, I can give them one link that shows all of their 
options.


That's nice. Just get your hopes up for it having an effect.


Typo, I meant "don't"... Sloppy of me. Documentation is nice, 
but:


1. People will complain that it isn't possible.

2. When possible people will complain that it isn't in the 
standard library.


3. When in "std" people will complain that not enough libraries 
use it.


4. When libraries use it people will complain that it doesn't 
work with older libs.


5. When older libs have been rewritten to support it they will 
complain that it is better in Rust and C++ and not compatible 
with Rust and C++.


Anyway, my main point is that programmers coming from such 
languages will most certainly complain if it isn't in the 
standard library because of interoperability between libraries, 
but that is basically just the bottom of the hill that you have 
to climb to get to a level where people stop complaining.


Many invested in Rust and C++ will look for arguments to support 
staying with their language. I've come to the conclusion that the 
D community is mostly to blame for not making a good case to the 
other group that are open to D, but for technical reasons or 
simply personal preference don't like GC, that D is still an 
option. There's no excuse for not making it easy to evaluate 
one's options for GC-less programming if we support that style of 
programming.

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Saturday, 29 April 2017 at 09:24:35 UTC, Ola Fosheim Grøstad 
wrote:
On Saturday, 29 April 2017 at 08:45:26 UTC, Moritz Maxeiner 
wrote:

On Saturday, 29 April 2017 at 07:26:45 UTC, Timon Gehr wrote:
I don't doubt that, but the implicit generalization is 
"multiple pointer types are necessarily always a royal PITA".


The "implicit generalization" is your interpretation, though.


No, it was brought up in a thread as an argument against having 
multiple pointer types.


It was brought up in that thread as an example of multiple 
pointer types having (I assume unintended) negative consequences. 
That's not the same as implying that *all* occurrences of 
multiple pointer types *will* have such negative consequences; at 
most, it implies that you have to be very careful when designing 
them, so as to avoid such consequences (and this latter part is 
*my* interpretation).


The thread was not about near/far pointers or segmented memory 
models.


I am aware; it was originally about the viability of automatic 
reference counting in D, and its potential benefits/drawbacks 
compared to garbage collection.

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Saturday, 29 April 2017 at 08:45:26 UTC, Moritz Maxeiner wrote:

On Saturday, 29 April 2017 at 07:26:45 UTC, Timon Gehr wrote:
I don't doubt that, but the implicit generalization is 
"multiple pointer types are necessarily always a royal PITA".


The "implicit generalization" is your interpretation, though.


No, it was brought up in a thread as an argument against having 
multiple pointer types. The thread was not about near/far 
pointers or segmented memory models.

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Saturday, 29 April 2017 at 07:26:45 UTC, Timon Gehr wrote:

On 28.04.2017 23:52, H. S. Teoh via Digitalmars-d wrote:
On Fri, Apr 28, 2017 at 09:50:49PM +, Atila Neves via 
Digitalmars-d wrote:
On Friday, 28 April 2017 at 19:41:15 UTC, Ola Fosheim Grøstad 
wrote:
On Friday, 28 April 2017 at 19:41:15 UTC, Ola Fosheim 
Grøstad wrote:
«Back in the old DOS days, there were multiple pointer types 
(near
and far). Programmers put up with that because it was the 
only way,

but they HATED HATED HATED it.»


It's true, we did. It was awful.

[...]

I remember working with that.  It was a royal PITA.


T



I don't doubt that, but the implicit generalization is 
"multiple pointer types are necessarily always a royal PITA".


The "implicit generalization" is your interpretation, though.

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Saturday, 29 April 2017 at 07:15:36 UTC, Timon Gehr wrote:

On 28.04.2017 17:43, Moritz Maxeiner wrote:

[...]

No. Every single thread I read in the last couple of years 
ended with
Walter pointing out issues that need to be "hashed out" and 
then nobody

doing it.


This is not the full story though. Among Walter's constraints 
for an acceptable solution always were:


1. cannot add new syntax


I've dug through what records I know of surrounding these 
discussions [1][2][3][4][5] and could not come up with a 
statement of his that supports this hard-line claim. You may be 
right, considering that adding new syntax is a non-trivial 
change, but I'd need the citation.




2. cannot add polymorphism to the type system (because it is 
"hard to understand")


Same as above.

[1] http://lists.puremagic.com/pipermail/dlang-study/
[2] 
https://forum.dlang.org/thread/kpgilxyyrrluxpepe...@forum.dlang.org
[3] 
https://forum.dlang.org/post/kluaojijixhwigouj...@forum.dlang.org

[4] http://forum.dlang.org/thread/oboaa2$17oa$1...@digitalmars.com
[5] http://forum.dlang.org/thread/nu00a6$t5i$1...@digitalmars.com

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Saturday, 29 April 2017 at 07:26:45 UTC, Timon Gehr wrote:
I don't doubt that, but the implicit generalization is 
"multiple pointer types are necessarily always a royal PITA". 
Not true. scope has worse usability than a scoped pointer type.


Yes. In this context it is was more about the GC lifetime 
management which requires:


1. The ability to tell the GC about explicit parent-child 
relationships so that the GC destruction order becomes 
deterministic.


2. Precise collection.

Re: Garbage Collector?

[Timon Gehr via Digitalmars-d]

On 28.04.2017 23:52, H. S. Teoh via Digitalmars-d wrote:

On Fri, Apr 28, 2017 at 09:50:49PM +, Atila Neves via Digitalmars-d wrote:

On Friday, 28 April 2017 at 19:41:15 UTC, Ola Fosheim Grøstad wrote:

On Friday, 28 April 2017 at 19:41:15 UTC, Ola Fosheim Grøstad wrote:
«Back in the old DOS days, there were multiple pointer types (near
and far). Programmers put up with that because it was the only way,
but they HATED HATED HATED it.»


It's true, we did. It was awful.

[...]

I remember working with that.  It was a royal PITA.


T



I don't doubt that, but the implicit generalization is "multiple pointer 
types are necessarily always a royal PITA". Not true. scope has worse 
usability than a scoped pointer type.

Re: Garbage Collector?

[Timon Gehr via Digitalmars-d]

On 28.04.2017 17:43, Moritz Maxeiner wrote:

On Friday, 28 April 2017 at 14:59:46 UTC, Ola Fosheim Grøstad wrote:

On Friday, 28 April 2017 at 09:40:07 UTC, Moritz Maxeiner wrote:

I'm sorry, but that's just plain wrong. D does not have ownership
pointers because nobody that wants them has stepped up and
1) Done the work of drafting an informal proposal that *actually
deals with _all_ of the issues involved*


Wrong. This has been discussed and hashed out to death over and over
again. The solutions are on the table.


No. Every single thread I read in the last couple of years ended with
Walter pointing out issues that need to be "hashed out" and then nobody
doing it.


This is not the full story though. Among Walter's constraints for an 
acceptable solution always were:


1. cannot add new syntax

2. cannot add polymorphism to the type system (because it is "hard to 
understand")


1. is inconvenient and I think what we get with 'scope' is approximately 
the best one can do given 2.

Re: Garbage Collector?

[Ola Fosheim Grostad via Digitalmars-d]

On Friday, 28 April 2017 at 21:21:13 UTC, jmh530 wrote:
To be fair, C++ effectively has multiple pointer types too with 
raw pointers, unique_ptr, shared_ptr, and weak_ptr. However, 
each of the extra ones has a unique purpose and are opt-in. As 
a result, people happily use them when it makes their lives 
easier.


Yes, they are not language types though, so no special effect on 
the compiler or runtime. The language types are pointers, 
 and &


By contrast, C++/CLI (I'm more familiar with that than managed 
C++) has pointer to managed heap and pointer to unmanaged heap. 
The concepts overlap more.


Yes, and I assume those are language types so that the compiler 
and runtime can take advantage of it?

Re: Garbage Collector?

[H. S. Teoh via Digitalmars-d]

On Fri, Apr 28, 2017 at 09:50:49PM +, Atila Neves via Digitalmars-d wrote:
> On Friday, 28 April 2017 at 19:41:15 UTC, Ola Fosheim Grøstad wrote:
> > On Friday, 28 April 2017 at 19:41:15 UTC, Ola Fosheim Grøstad wrote:
> > «Back in the old DOS days, there were multiple pointer types (near
> > and far). Programmers put up with that because it was the only way,
> > but they HATED HATED HATED it.»
> 
> It's true, we did. It was awful.
[...]

I remember working with that.  It was a royal PITA.


T

-- 
Those who've learned LaTeX swear by it. Those who are learning LaTeX swear at 
it. -- Pete Bleackley

Re: Garbage Collector?

[Atila Neves via Digitalmars-d]

On Friday, 28 April 2017 at 19:41:15 UTC, Ola Fosheim Grøstad 
wrote:
On Friday, 28 April 2017 at 19:41:15 UTC, Ola Fosheim Grøstad 
wrote:
«Back in the old DOS days, there were multiple pointer types 
(near and far). Programmers put up with that because it was the 
only way, but they HATED HATED HATED it.»


It's true, we did. It was awful.

Atila

Re: Garbage Collector?

[jmh530 via Digitalmars-d]

On Friday, 28 April 2017 at 20:21:34 UTC, Ola Fosheim Grøstad 
wrote:

On Friday, 28 April 2017 at 20:13:58 UTC, Moritz Maxeiner wrote:
Both of these, however, show only that he doesn't seem to 
personally like multiple pointer types (and consequently 
doesn't seem to have any interest in working on them himself); 
that's not the same as him claiming that it "is a disaster" 
(in general, which is what you were implying).


There are other threads. This is a recurring topic...



To be fair, C++ effectively has multiple pointer types too with 
raw pointers, unique_ptr, shared_ptr, and weak_ptr. However, each 
of the extra ones has a unique purpose and are opt-in. As a 
result, people happily use them when it makes their lives easier.


By contrast, C++/CLI (I'm more familiar with that than managed 
C++) has pointer to managed heap and pointer to unmanaged heap. 
The concepts overlap more.

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Friday, 28 April 2017 at 20:21:34 UTC, Ola Fosheim Grøstad 
wrote:

On Friday, 28 April 2017 at 20:13:58 UTC, Moritz Maxeiner wrote:
Both of these, however, show only that he doesn't seem to 
personally like multiple pointer types (and consequently 
doesn't seem to have any interest in working on them himself); 
that's not the same as him claiming that it "is a disaster" 
(in general, which is what you were implying).


There are other threads. This is a recurring topic...


I am aware...



What you consider not pointless is your business, again, but 
if you don't try to get it in the core language, you have no 
foundation to complain that's it's not in there.


There are no example I know of where something has gone into 
the D language that is going against the aesthetics Walter 
value... Which isn't surprising.


There are also few examples of new features going in at the 
language level based on external DIPs. Do you have any good 
examples? (Not standard lib, but language features).


I myself know of no examples where people have actually done the 
work up to and including the point where there were no technical 
issues left and were then rejected.

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Friday, 28 April 2017 at 20:13:58 UTC, Moritz Maxeiner wrote:
Both of these, however, show only that he doesn't seem to 
personally like multiple pointer types (and consequently 
doesn't seem to have any interest in working on them himself); 
that's not the same as him claiming that it "is a disaster" (in 
general, which is what you were implying).


There are other threads. This is a recurring topic...

What you consider not pointless is your business, again, but if 
you don't try to get it in the core language, you have no 
foundation to complain that's it's not in there.


There are no example I know of where something has gone into the 
D language that is going against the aesthetics Walter value... 
Which isn't surprising.


There are also few examples of new features going in at the 
language level based on external DIPs. Do you have any good 
examples? (Not standard lib, but language features).

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Friday, 28 April 2017 at 19:41:15 UTC, Ola Fosheim Grøstad 
wrote:

On Friday, 28 April 2017 at 15:43:22 UTC, Moritz Maxeiner wrote:
On Friday, 28 April 2017 at 14:59:46 UTC, Ola Fosheim Grøstad 
wrote:
Walter's position has always been that having more than a 
single pointer type is a disaster.


None of the threads I've read in the last couple of years 
regarding that support that claim.


He has restated this position many many times... Random 
snippets:


«Microsoft's Managed C++ had two pointer types, and it went 
over like a lead zeppelin.

»

http://forum.dlang.org/post/mclqt1$1e5n$1...@digitalmars.com


«Back in the old DOS days, there were multiple pointer types 
(near and far). Programmers put up with that because it was the 
only way, but they HATED HATED HATED it.»


http://forum.dlang.org/post/mcnv9u$e8p$1...@digitalmars.com



I had not read these, thank you :)
Both of these, however, show only that he doesn't seem to 
personally like multiple pointer types (and consequently doesn't 
seem to have any interest in working on them himself); that's not 
the same as him claiming that it "is a disaster" (in general, 
which is what you were implying).




You can easily find more... No point in trying to get that into 
the core language (but it is necessary to get proper 
destruction of GC managed objects in a reasonable way).


What you consider not pointless is your business, again, but if 
you don't try to get it in the core language, you have no 
foundation to complain that's it's not in there.

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Friday, 28 April 2017 at 17:48:47 UTC, Ola Fosheim Grøstad 
wrote:

On Friday, 28 April 2017 at 17:42:18 UTC, bachmeier wrote:
I'm hoping to put all information in one place. Then when 
someone on Reddit or HN or here starts making claims about the 
GC, I can give them one link that shows all of their options.


That's nice. Just get your hopes up for it having an effect.


Typo, I meant "don't"... Sloppy of me. Documentation is nice, but:

1. People will complain that it isn't possible.

2. When possible people will complain that it isn't in the 
standard library.


3. When in "std" people will complain that not enough libraries 
use it.


4. When libraries use it people will complain that it doesn't 
work with older libs.


5. When older libs have been rewritten to support it they will 
complain that it is better in Rust and C++ and not compatible 
with Rust and C++.


Anyway, my main point is that programmers coming from such 
languages will most certainly complain if it isn't in the 
standard library because of interoperability between libraries, 
but that is basically just the bottom of the hill that you have 
to climb to get to a level where people stop complaining.

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Friday, 28 April 2017 at 15:43:22 UTC, Moritz Maxeiner wrote:
On Friday, 28 April 2017 at 14:59:46 UTC, Ola Fosheim Grøstad 
wrote:
Walter's position has always been that having more than a 
single pointer type is a disaster.


None of the threads I've read in the last couple of years 
regarding that support that claim.


He has restated this position many many times... Random snippets:

«Microsoft's Managed C++ had two pointer types, and it went over 
like a lead zeppelin.

»

http://forum.dlang.org/post/mclqt1$1e5n$1...@digitalmars.com


«Back in the old DOS days, there were multiple pointer types 
(near and far). Programmers put up with that because it was the 
only way, but they HATED HATED HATED it.»


http://forum.dlang.org/post/mcnv9u$e8p$1...@digitalmars.com


You can easily find more... No point in trying to get that into 
the core language (but it is necessary to get proper destruction 
of GC managed objects in a reasonable way).

Re: Garbage Collector?

[H. S. Teoh via Digitalmars-d]

On Fri, Apr 28, 2017 at 04:03:18PM +, bachmeier via Digitalmars-d wrote:
> On Friday, 28 April 2017 at 15:23:18 UTC, H. S. Teoh wrote:
> 
> > you could save yourself the bug by writing:
> > 
> > auto x = malloc(...);
> > scope(exit) free(x);
> > // ... however many pages of stuff you want, you don't have to
> > // remember to write free() afterwards!
> > 
> > Yes, D comes with a GC... doesn't mean you have to use it if you
> > don't want to, though!
> 
> I usually use the GC, so I have limited knowledge in this area. How
> common is this pattern in D code? Is it better than using reference
> counted structs? Is there any advantage to using the GC in this
> scenario?

To be honest, in my own D code I'm not overly concerned with the GC, and
even when I am, and want to control when / how often the GC collects, I
usually just use GC.disable() and then GC.collect() explicitly.  I've
seen performance gains of about 30-40% just by carefully reducing the
frequency of GC collections (IMO the default is a bit too eager), but of
course that depends on exactly what your application is doing.  In my
case, that was in the context of batch-oriented, CPU-intensive
computations that allocate often and mostly keeps live data.  YMMV if
your program is doing something else or has other patterns of memory
access.

But my point was that, if you really wanted to, you *could* use C-style
memory management in D code. D won't stop you from doing that. The
malloc heap is distinct from the GC heap so you won't run into
conflicts.  Of course, that also means you can't use D features that
currently require the GC, such as closures and dynamic arrays. But if
you're doing C-style memory management you probably already want to
implement your own way of handling array allocations and closure
functionality anyway. You can still pass slices of things around to
things like Phobos range algorithms (excepting the few that might
allocate -- hence marking your code with @nogc for the compiler to
enforce this), so generally a decent chunk of Phobos should still be
usable.

Even Phobos itself uses malloc/free directly in several places, for
various reasons, so I wouldn't say it's exactly a foreign thing to do in
D code.


> I would like to add this info to the wiki (I don't seen it there).

That would be very nice, thanks!


T

-- 
Give a man a fish, and he eats once. Teach a man to fish, and he will sit 
forever.

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Friday, 28 April 2017 at 17:42:18 UTC, bachmeier wrote:
I'm hoping to put all information in one place. Then when 
someone on Reddit or HN or here starts making claims about the 
GC, I can give them one link that shows all of their options.


That's nice. Just get your hopes up for it having an effect. One 
of the key points of having it in the core distribution 
(compiler) is for library interoperability. Using multiple 
solutions ends up in chaos in real world projects...

Re: Garbage Collector?

[bachmeier via Digitalmars-d]

On Friday, 28 April 2017 at 17:06:51 UTC, jmh530 wrote:

On Friday, 28 April 2017 at 16:03:18 UTC, bachmeier wrote:


I usually use the GC, so I have limited knowledge in this 
area. How common is this pattern in D code? Is it better than 
using reference counted structs? Is there any advantage to 
using the GC in this scenario?


I would like to add this info to the wiki (I don't seen it 
there).


You can also use automem
https://dlang.org/blog/2017/04/28/automem-hands-free-raii-for-d/


I'm hoping to put all information in one place. Then when someone 
on Reddit or HN or here starts making claims about the GC, I can 
give them one link that shows all of their options. There's this 
page:

https://wiki.dlang.org/Memory_Management
but that isn't comprehensive or as to-the-point as it needs to be.

Re: Garbage Collector?

[jmh530 via Digitalmars-d]

On Friday, 28 April 2017 at 16:03:18 UTC, bachmeier wrote:


I usually use the GC, so I have limited knowledge in this area. 
How common is this pattern in D code? Is it better than using 
reference counted structs? Is there any advantage to using the 
GC in this scenario?


I would like to add this info to the wiki (I don't seen it 
there).


You can also use automem
https://dlang.org/blog/2017/04/28/automem-hands-free-raii-for-d/

Re: Garbage Collector?

[bachmeier via Digitalmars-d]

On Friday, 28 April 2017 at 15:23:18 UTC, H. S. Teoh wrote:


you could save yourself the bug by writing:

auto x = malloc(...);
scope(exit) free(x);
// ... however many pages of stuff you want, you don't have to
// remember to write free() afterwards!

Yes, D comes with a GC... doesn't mean you have to use it if 
you don't want to, though!


I usually use the GC, so I have limited knowledge in this area. 
How common is this pattern in D code? Is it better than using 
reference counted structs? Is there any advantage to using the GC 
in this scenario?


I would like to add this info to the wiki (I don't seen it there).

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Friday, 28 April 2017 at 14:59:46 UTC, Ola Fosheim Grøstad 
wrote:

On Friday, 28 April 2017 at 09:40:07 UTC, Moritz Maxeiner wrote:
I'm sorry, but that's just plain wrong. D does not have 
ownership pointers because nobody that wants them has stepped 
up and
1) Done the work of drafting an informal proposal that 
*actually deals with _all_ of the issues involved*


Wrong. This has been discussed and hashed out to death over and 
over again. The solutions are on the table.


No. Every single thread I read in the last couple of years ended 
with Walter pointing out issues that need to be "hashed out" and 
then nobody doing it.




Walter's position has always been that having more than a 
single pointer type is a disaster.


None of the threads I've read in the last couple of years 
regarding that support that claim.




That makes further work on it pointless.


What you consider worth working on is your business, of course :)

Re: Garbage Collector?

[H. S. Teoh via Digitalmars-d]

On Fri, Apr 28, 2017 at 09:01:03AM +, Moritz Maxeiner via Digitalmars-d 
wrote:
> On Friday, 28 April 2017 at 07:35:00 UTC, Ben wrote:
[...]
> > Is it so hard for developers when you declare a variable, to later
> > also clean it up???
> > 
> > var x = 1;
> > // Do work
> > x.free;
> 
> If you write it like that, yes, because often it's not just one such
> make/dispose pair per scope, but multiple, possibly overlapping ones
> and people make mistakes. And the more complex a piece of code gets
> the harder it becomes to decipher such pairs and/or decide if the
> "closing" dispose is missing.
> This is one of the reasons why scope guards are good:
> 
> var x = 1;
> scope (exit) x.free
> // Do work
> 
> This, as code becomes more complex, allows for much easier reading
> (and understanding) of lifetimes.
[...]

Elephant in the room: D lets you call the C library's malloc() and
free(). If you absolute insist that you don't want to use the GC, go
right ahead and import core.c.stdlib, and malloc and free away.  As
mentioned above, D's scope guards will even help you avoid mistakes by
keeping allocation and free in the same scope together, i.e., instead of
writing:

auto x = malloc(...);
// do stuff
// and more stuff
// and more stuff
// so many pages of stuff you forgot about x
free(x);// very likely you'll forget this by now

you could save yourself the bug by writing:

auto x = malloc(...);
scope(exit) free(x);
// ... however many pages of stuff you want, you don't have to
// remember to write free() afterwards!

Yes, D comes with a GC... doesn't mean you have to use it if you don't
want to, though!


T

-- 
Doubt is a self-fulfilling prophecy.

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Friday, 28 April 2017 at 09:40:07 UTC, Moritz Maxeiner wrote:
I'm sorry, but that's just plain wrong. D does not have 
ownership pointers because nobody that wants them has stepped 
up and
1) Done the work of drafting an informal proposal that 
*actually deals with _all_ of the issues involved*


Wrong. This has been discussed and hashed out to death over and 
over again. The solutions are on the table.


Walter's position has always been that having more than a single 
pointer type is a disaster.


That makes further work on it pointless.

(Although, arguably, D kinda has several pointer types already).

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Friday, 28 April 2017 at 09:12:03 UTC, Ola Fosheim Grøstad 
wrote:

On Friday, 28 April 2017 at 09:02:19 UTC, Moritz Maxeiner wrote:
On Friday, 28 April 2017 at 08:26:28 UTC, Ola Fosheim Grøstad 
wrote:

If it isn't sound then it isn't worth mentioning...


So you claim...


Actually, unsound lifetime management is worse than nothing as 
far as correctness goes... so I guess it is worth mentioning as 
a warning rather than a recommendation.


Actually, applying "sound" or "unsound" to D's GC lifetime 
management in general is worse than realizing that whether using 
it for a specific use case is sound or unsound is entirely 
dependent on that use case... so I guess it is worth mentioning 
as a tool a developer should read up on.
Also, you seem to imply that I made a recommendation to use the 
GC's lifetime management. I did not.





If D had ownership pointers then it would've been possible to 
sort this out, but D is perpetually locked to the current 
pointer/GC model for reasons that aren't particularly 
convincing, i.e. beliefs.


I'm sorry, but that's just plain wrong. D does not have ownership 
pointers because nobody that wants them has stepped up and
1) Done the work of drafting an informal proposal that *actually 
deals with _all_ of the issues involved*

2) Developed that informal proposal into a DIP
3) Wrote the implementation and sent a PR
Every time it comes up there are ideas, or sometimes even 
reasonably well formed informal proposals, but none of the 
proponents ever actually follows up and improves such proposals 
to the point where all the issues are dealt with. Claiming that D 
doesn't have ownership mechanics because of beliefs when none of 
the (vocal) proponents are willing to actually get down to it and 
*do the work* is, frankly, asinine; do you generally expect 
people to do things for you they aren't interested in without 
recompense?

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Friday, 28 April 2017 at 09:02:19 UTC, Moritz Maxeiner wrote:
On Friday, 28 April 2017 at 08:26:28 UTC, Ola Fosheim Grøstad 
wrote:

If it isn't sound then it isn't worth mentioning...


So you claim...


Actually, unsound lifetime management is worse than nothing as 
far as correctness goes... so I guess it is worth mentioning as a 
warning rather than a recommendation.


If D had ownership pointers then it would've been possible to 
sort this out, but D is perpetually locked to the current 
pointer/GC model for reasons that aren't particularly convincing, 
i.e. beliefs.

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Friday, 28 April 2017 at 08:26:28 UTC, Ola Fosheim Grøstad 
wrote:

On Friday, 28 April 2017 at 08:25:01 UTC, Moritz Maxeiner wrote:
On Friday, 28 April 2017 at 04:24:43 UTC, Ola Fosheim Grostad 
wrote:
On Thursday, 27 April 2017 at 22:43:56 UTC, Moritz Maxeiner 
wrote:

Working on the memory chunk layer is memory management.
Working on the object layer is object lifetime management.
D offers you both automatic memory management and automatic 
lifetime management via its GC.


D offers sound automatic lifetime management? Since when?


Why are you implicitly adding adjectives to my sentence that 
change its meaning?


If it isn't sound then it isn't worth mentioning...


So you claim...

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Friday, 28 April 2017 at 07:35:00 UTC, Ben wrote:
On Thursday, 27 April 2017 at 22:43:56 UTC, Moritz Maxeiner 
wrote:
D just does this transparently for you by default. If you 
already know the exact or maximum size, you can allocate 
*once* (not 6 times) using `new` and `.reserve` respectively 
*before* entering the loop, like that article explains in 
depth.


You seem to be missing the fact that i pointed this out. The 
fact that the GC might have done up to 6 collection rounds in 
that loop is "ludicrous".


You wrote:
I if did my own memory management, the variable cleanup will 
have been done in one go, right after the loop. Simply more 
efficient.


You wrote about cleaning up *after* the loop, *not* about 
allocating *before* the loop. They are semantically related (as 
allocating with the GC *might* kick off a collection cycle), but 
not the same thing.




In my experience most people's aversion to GCs can be aptly 
described by the German proverb "Was der Bauer nicht kennt, 
das frisst er nicht" (the meaning of which being that most 
people generally like living in comfort zones and rarely - if 
ever - dare to leave them of their own free will; that 
includes developers, sadly).


Unfortunately i do have plenty of experience with GC kicking in 
on the wrong moments ( or not at the right moments, people 
forget that one ).


In which language(s)? Because in D it's both easy and simple to 
avoid that (Use @nogc or compile with `-vgc` and inspect).




Maybe its my opinion only but a good language will not put 
anything in the way of the developer but will point out 
mistakes.


Safety properties may be what you're looking for. See @safe, 
that's an area of D a lot of work is currently being done on.




The Rust compiler is not a bad example but it can be taken a 
step more. Is it so hard for developers when you declare a 
variable, to later also clean it up???


var x = 1;
// Do work
x.free;


If you write it like that, yes, because often it's not just one 
such make/dispose pair per scope, but multiple, possibly 
overlapping ones and people make mistakes. And the more complex a 
piece of code gets the harder it becomes to decipher such pairs 
and/or decide if the "closing" dispose is missing.

This is one of the reasons why scope guards are good:

var x = 1;
scope (exit) x.free
// Do work

This, as code becomes more complex, allows for much easier 
reading (and understanding) of lifetimes.





Easy ... Sure, it becomes a little bit more tricky with 
ownership but that is where the compiler can help and simply 
state: "Hey, you forgot this variable, it does not seem to be 
used beyond this point".


Since the GC is memory safe by definition (sans bugs) you'd only 
need this for where you aren't using it. And for the rest I'm not 
sure it can be done sanely: How's the compiler to know, e.g., 
which functions from a C API (or C++) you call into allocate or 
deallocate memory? Introduce yet another keyword or @property to 
signal cross-language memory management? If you ignore 
cross-language calls and only remain within D, I suppose it might 
be viable to write a semantic pass for it, but from my 
perspective, the cross-language calls are the *much* bigger 
threat to memory safety.


Just calling the x.free seem to be too much work for developers 
these days.


In my experience reading about such occurences, this usually 
happens when the lifetime of the object the memory is used for 
has become unclear.




Up to now i found very few languages that did this correctly.


I don't share your assessment of what's correct, but yes, we 
digress.

Re: Garbage Collector?

[Ola Fosheim Grøstad via Digitalmars-d]

On Friday, 28 April 2017 at 08:25:01 UTC, Moritz Maxeiner wrote:
On Friday, 28 April 2017 at 04:24:43 UTC, Ola Fosheim Grostad 
wrote:
On Thursday, 27 April 2017 at 22:43:56 UTC, Moritz Maxeiner 
wrote:

Working on the memory chunk layer is memory management.
Working on the object layer is object lifetime management.
D offers you both automatic memory management and automatic 
lifetime management via its GC.


D offers sound automatic lifetime management? Since when?


Why are you implicitly adding adjectives to my sentence that 
change its meaning?


If it isn't sound then it isn't worth mentioning...

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Friday, 28 April 2017 at 04:24:43 UTC, Ola Fosheim Grostad 
wrote:
On Thursday, 27 April 2017 at 22:43:56 UTC, Moritz Maxeiner 
wrote:

Working on the memory chunk layer is memory management.
Working on the object layer is object lifetime management.
D offers you both automatic memory management and automatic 
lifetime management via its GC.


D offers sound automatic lifetime management? Since when?


Why are you implicitly adding adjectives to my sentence that 
change its meaning?

Re: Garbage Collector?

[Mike Parker via Digitalmars-d]

On Friday, 28 April 2017 at 07:35:00 UTC, Ben wrote:


so I'll reply:
Expanding the continuous memory region a dynamic array 
consists of (possibly moving it) once it overflows has 
absolutely nothing to do with the GC, or even the language, 
it's how the abstract data type "dynamic array" is defined. D 
just does this transparently for you by default. If you 
already know the exact or maximum size, you can allocate 
*once* (not 6 times) using `new` and `.reserve` respectively 
*before* entering the loop, like that article explains in 
depth.


You seem to be missing the fact that i pointed this out. The 
fact that the GC might have done up to 6 collection rounds in 
that loop is "ludicrous".


How is it ludicrous? The fact that you know the GC will only run 
during an allocation, and only if it needs to, allows you to 
control when those opportunities to collect arise. That's a much 
more palatable situation than if it were sitting in the 
background, constantly checking if it needs to collect, then 
doing so without any input from you. There, you'd have no control 
at all.


In the context of an actual program, the example would only have 
made 6 collections if you were putting putting heavy pressure on 
the GC heap, which is an extremely naive way of programming for 
anyone concerned about performance. D allows you several options 
to relieve that pressure and assert some control over when the GC 
can run. Even in a non-GC language, you wouldn't be allocating 
like that in a performance-critical loop -- you would preallocate 
before you entered the loop.

Re: Garbage Collector?

[Ben via Digitalmars-d]

On Thursday, 27 April 2017 at 22:43:56 UTC, Moritz Maxeiner wrote:
You replied to the wrong person here, seeing as I did not link 
to the article you're referring to,


Sorry...


so I'll reply:
Expanding the continuous memory region a dynamic array consists 
of (possibly moving it) once it overflows has absolutely 
nothing to do with the GC, or even the language, it's how the 
abstract data type "dynamic array" is defined. D just does this 
transparently for you by default. If you already know the exact 
or maximum size, you can allocate *once* (not 6 times) using 
`new` and `.reserve` respectively *before* entering the loop, 
like that article explains in depth.


You seem to be missing the fact that i pointed this out. The fact 
that the GC might have done up to 6 collection rounds in that 
loop is "ludicrous".




Um, what? Memory (de)allocation (in C often malloc/free) and 
object (de)contruction (in C usually functions with some naming 
conventions like `type_init`/`type_deinit`) are on two entirely 
different layers! Granted, they are often combined in C to 
functions with names like `type_new`/`type_free`, but they are 
conceptually two distinct things. Just to be very clear, here 
is a primitive diagram of how things work:

make object O of type T:
 --(allocate N bytes)--> [memory chunk M] --(call 
constructor T(args) on M)--> [O]

dispose of O:
[O] --(call destructor ~T() on O)--> [memory chunk M] 
--(deallocate M)--> 


D's garbage collector conceptually changes this to:
make object O of type T:
 --(GC allocates)--> [GC memory pool] --(allocate N 
bytes)--> [memory chunk M] --(call constructor T(args) on M)--> 
[O]

dispose of O:
[O] --(call destructor ~T() on O)--> [memory chunk M] 
--(deallocate M)--> [GC memory pool] --(GC deallocates)--> 
with the specific restriction that you have *no* control over 
'GC deallocates' and only indirect control over 'GC allocates' 
(by allocating more memory from the GC than is available its 
the pool).


Working on the memory chunk layer is memory management.
Working on the object layer is object lifetime management.
D offers you both automatic memory management and automatic 
lifetime management via its GC.
What you describe is manual object lifetime management (which 
is what std.conv.emplace and object.destroy exist for) and has 
no effect on the automatic memory management the GC performs.
You *can* do manual memory management *on top* of the GC's 
memory pool (using core.memory.GC.{alloc/free) or the newer, 
generic Alloactor interface via 
std.experimental.allocator.gc_allocator.GCAllocator.{allocate/deallocate}), but these operations will (generally) not yield any observable difference from the OS's perspective.




That is assuming the GC removes the memory reference when you 
call it. I remember seeing in some other languages ( C# 
possibly? ) that referring a variable to be freed only meant 
the GC freed the memory when it felt like it, not the exact 
spot when you told it.


Again, you seem to mix object lifetime management and memory 
management. You cannot tell the GC to free memory back to the 
operating system (which is what the free syscall does and what 
you seem to be describing). You can *only* free memory you 
allocated *from* the GC *back* to the GC. The GC decides when 
(and if) any memory is ever being freed back to the OS (which 
is kinda one major point of having a GC in the first place).


I know, i do not express myself very well in English.

In my experience most people's aversion to GCs can be aptly 
described by the German proverb "Was der Bauer nicht kennt, das 
frisst er nicht" (the meaning of which being that most people 
generally like living in comfort zones and rarely - if ever - 
dare to leave them of their own free will; that includes 
developers, sadly).


Unfortunately i do have plenty of experience with GC kicking in 
on the wrong moments ( or not at the right moments, people forget 
that one ).


I am sure that the amount of people who develop in GC languages 
is much bigger these days then manual managed languages ( what is 
more or less a rarity these days among the new languages ). Even 
Rust still has some background management going on ( like the 
byte counter ).


Maybe its my opinion only but a good language will not put 
anything in the way of the developer but will point out mistakes.


The Rust compiler is not a bad example but it can be taken a step 
more. Is it so hard for developers when you declare a variable, 
to later also clean it up???


var x = 1;
// Do work
x.free;

Easy ... Sure, it becomes a little bit more tricky with ownership 
but that is where the compiler can help and simply state: "Hey, 
you forgot this variable, it does not seem to be used beyond this 
point". Just calling the x.free seem to be too much work for 
developers these days.


Up to now i found very few languages that did this correctly. But 
this is a offtopic discussion.

Re: Garbage Collector?

[Jerry via Digitalmars-d]

On Thursday, 27 April 2017 at 17:31:42 UTC, bachmeier wrote:

On Thursday, 27 April 2017 at 15:50:56 UTC, Ben wrote:

A few days ago i was reading this topic:

https://news.ycombinator.com/item?id=14165198

And the whole GC keeps coming up as a negative ( compared to 
Rust ).


From my understanding there has been a proposal DIP1000 to 
address this issue. Is there any update on this topic?


Is it possible to run D without the GC AND the standard 
library?


There is opposition to the GC from a small but vocal group of 
mostly C++ developers. Sometimes they even make valid 
arguments. You can find many threads on the topic here.


Citation needed.

Re: Garbage Collector?

[Era Scarecrow via Digitalmars-d]

On Thursday, 27 April 2017 at 19:36:44 UTC, Ben wrote:
Frankly seeing in this example that the GC was in theory able 
to kick in 6 times in a simple 100 item loop, that is not 
efficient. I if did my own memory management, the variable 
cleanup will have been done in one go, right after the loop. 
Simply more efficient.


 I find myself trying to use the stack as much as possible for 
buffers and structs so i can control and throw them away properly 
without worrying about the GC. The downside is they have to be 
fairly small (at least under windows without forcing it using a 
new thread or fiber).

Re: Garbage Collector?

[Ola Fosheim Grostad via Digitalmars-d]

On Thursday, 27 April 2017 at 22:43:56 UTC, Moritz Maxeiner wrote:

Working on the memory chunk layer is memory management.
Working on the object layer is object lifetime management.
D offers you both automatic memory management and automatic 
lifetime management via its GC.


D offers sound automatic lifetime management? Since when?

Re: Garbage Collector?

[Mike Parker via Digitalmars-d]

On Thursday, 27 April 2017 at 19:36:44 UTC, Ben wrote:


Frankly seeing in this example that the GC was in theory able 
to kick in 6 times in a simple 100 item loop, that is not 
efficient. I if did my own memory management, the variable 
cleanup will have been done in one go, right after the loop.


Please reread that bit.

"there were six total GC allocations in the loop. That means 
there were six opportunities for the GC to collect garbage."


Keyword: opportunities. There were six allocations, not six 
collections. The GC only "kicks in" when it needs to reclaim 
memory. In this case, the six allocations are made by the array 
management in DRuntime. It's the same as you allocating with 
`new` six times. In all of those cases, as long as the GC has 
room in its pool from which to allocate, there will be no 
collections.


Preallocate as much as possible and minimize the number of GC 
allocations (allocate on the stack where possible and through 
malloc where necessary) and the GC is not going to get in the way 
for most D programs you write.

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Thursday, 27 April 2017 at 19:36:44 UTC, Ben wrote:
On Thursday, 27 April 2017 at 16:35:57 UTC, Moritz Maxeiner 
wrote:
You'll have to be more specific about what issue you're 
referring to. People not liking garbage collection? In any 
case, AFAIU DIP1000 was about more mechanically verifiable 
memory safety features when not using the GC.




Is it possible to run D without the GC AND the standard 
library?


It is possible to run a D program without the GC only if you 
don't allocate using the GC. If you want to see D code that 
does allocate using the GC, the compiler flag `-vgc` is your 
friend (compile Phobos with `-vgc` for kicks). Currently, not 
all of Phobos is free of GC allocations, most notably 
exceptions (refer to [1] and similar topics)


[1] http://forum.dlang.org/thread/occ9kk$24va$1...@digitalmars.com


Frankly seeing in this example that the GC was in theory able 
to kick in 6 times in a simple 100 item loop, that is not 
efficient. I if did my own memory management, the variable 
cleanup will have been done in one go, right after the loop. 
Simply more efficient.


You replied to the wrong person here, seeing as I did not link to 
the article you're referring to, but your statement about not 
being efficient is, honestly, ludicrous, so I'll reply:
Expanding the continuous memory region a dynamic array consists 
of (possibly moving it) once it overflows has absolutely nothing 
to do with the GC, or even the language, it's how the abstract 
data type "dynamic array" is defined. D just does this 
transparently for you by default. If you already know the exact 
or maximum size, you can allocate *once* (not 6 times) using 
`new` and `.reserve` respectively *before* entering the loop, 
like that article explains in depth.




Been thinking about this topic. Dlang has a destructor, does 
that means if you allocate on the GC and then do your own 
destructor, it technically counts as manual memory management?


Um, what? Memory (de)allocation (in C often malloc/free) and 
object (de)contruction (in C usually functions with some naming 
conventions like `type_init`/`type_deinit`) are on two entirely 
different layers! Granted, they are often combined in C to 
functions with names like `type_new`/`type_free`, but they are 
conceptually two distinct things. Just to be very clear, here is 
a primitive diagram of how things work:

make object O of type T:
 --(allocate N bytes)--> [memory chunk M] --(call constructor 
T(args) on M)--> [O]

dispose of O:
[O] --(call destructor ~T() on O)--> [memory chunk M] 
--(deallocate M)--> 


D's garbage collector conceptually changes this to:
make object O of type T:
 --(GC allocates)--> [GC memory pool] --(allocate N bytes)--> 
[memory chunk M] --(call constructor T(args) on M)--> [O]

dispose of O:
[O] --(call destructor ~T() on O)--> [memory chunk M] 
--(deallocate M)--> [GC memory pool] --(GC deallocates)--> 
with the specific restriction that you have *no* control over 'GC 
deallocates' and only indirect control over 'GC allocates' (by 
allocating more memory from the GC than is available its the 
pool).


Working on the memory chunk layer is memory management.
Working on the object layer is object lifetime management.
D offers you both automatic memory management and automatic 
lifetime management via its GC.
What you describe is manual object lifetime management (which is 
what std.conv.emplace and object.destroy exist for) and has no 
effect on the automatic memory management the GC performs.
You *can* do manual memory management *on top* of the GC's memory 
pool (using core.memory.GC.{alloc/free) or the newer, generic 
Alloactor interface via 
std.experimental.allocator.gc_allocator.GCAllocator.{allocate/deallocate}), but these operations will (generally) not yield any observable difference from the OS's perspective.




That is assuming the GC removes the memory reference when you 
call it. I remember seeing in some other languages ( C# 
possibly? ) that referring a variable to be freed only meant 
the GC freed the memory when it felt like it, not the exact 
spot when you told it.


Again, you seem to mix object lifetime management and memory 
management. You cannot tell the GC to free memory back to the 
operating system (which is what the free syscall does and what 
you seem to be describing). You can *only* free memory you 
allocated *from* the GC *back* to the GC. The GC decides when 
(and if) any memory is ever being freed back to the OS (which is 
kinda one major point of having a GC in the first place).




I personally think that people simple have a bad taste with GC 
because they kick in too much outside there control.


In my experience most people's aversion to GCs can be aptly 
described by the German proverb "Was der Bauer nicht kennt, das 
frisst er nicht" (the meaning of which being that most people 
generally like living in comfort zones and rarely - if ever - 
dare to leave them

Re: Garbage Collector?

[Patric Dexheimer via Digitalmars-d]

GC it´s not only about some performance issues that some people 
may encounter on D, but it its a marketing problem as well.


"D is a systems programming language with..." GC(!?).

Most people that are interest in D came from c/c++ or other 
backgrounds without GC or hearing their entire life that GC "may 
be bad".


Its not a question of "how bad it is" or if it is bad at all.

People know how to program without GC. So this is just something 
in the way.


Anyway D is awesome.

Re: Garbage Collector?

[ag0aep6g via Digitalmars-d]

On Thursday, 27 April 2017 at 19:36:44 UTC, Ben wrote:
Frankly seeing in this example that the GC was in theory able 
to kick in 6 times in a simple 100 item loop, that is not 
efficient. I if did my own memory management, the variable 
cleanup will have been done in one go, right after the loop. 
Simply more efficient.


You can easily disable automatic collections and trigger one 
manually:



import core.memory: GC;
GC.disable(); /* disable automatic collections */
/* ... do your loop ... */
GC.collect(); /* manually trigger collection */
GC.enable(); /* enable automatic collections */


Been thinking about this topic. Dlang has a destructor, does 
that means if you allocate on the GC and then do your own 
destructor, it technically counts as manual memory management?


I'm not sure what role the destructor supposedly plays here. A 
destructor is called automatically when the object is freed, or 
you can call it manually with `object.destroy` [1]. Freeing 
happens when the object is collected, or you can use 
`core.memory.GC.free` [2] to free without collecting. Collecting 
happens automatically, or you can trigger it with 
`core.memory.GC.collect` [3] as shown.


You can use a destructor to manage memory owned by the object. 
But the destructor being there doesn't affect how the object 
itself is managed.


That is assuming the GC removes the memory reference when you 
call it. I remember seeing in some other languages ( C# 
possibly? ) that referring a variable to be freed only meant 
the GC freed the memory when it felt like it, not the exact 
spot when you told it.


Destroying does not imply freeing. It's the other way around. If 
you want to free, call `GC.free`. That does count as manual 
memory management.


I personally think that people simple have a bad taste with GC 
because they kick in too much outside there control. For 90% 
the default behavior is good but its those 10% that leaves a 
bad taste with people ( GC on critical moments and hurting 
performance in return ).


You can:

1) Call `core.memory.GC.disable` [4] if you don't want the GC to 
collect for a while. You should make sure that you don't run out 
of memory during that time, of course. Use 
`core.memory.GC.enable` [5] to enable collections again.


2) Use the `@nogc` [6] attribute to forbid GC allocations in a 
function. GC collections are only triggered by allocations (or 
manually). So the GC won't kick in during execution of @nogc code.




[1] https://dlang.org/phobos/object.html#.destroy
[2] https://dlang.org/phobos/core_memory.html#.GC.free
[3] https://dlang.org/phobos/core_memory.html#.GC.collect
[4] https://dlang.org/phobos/core_memory.html#.GC.disable
[5] https://dlang.org/phobos/core_memory.html#.GC.enable
[6] https://dlang.org/spec/attribute.html#nogc

Re: Garbage Collector?

[Ben via Digitalmars-d]

On Thursday, 27 April 2017 at 16:35:57 UTC, Moritz Maxeiner wrote:
You'll have to be more specific about what issue you're 
referring to. People not liking garbage collection? In any 
case, AFAIU DIP1000 was about more mechanically verifiable 
memory safety features when not using the GC.




Is it possible to run D without the GC AND the standard 
library?


It is possible to run a D program without the GC only if you 
don't allocate using the GC. If you want to see D code that 
does allocate using the GC, the compiler flag `-vgc` is your 
friend (compile Phobos with `-vgc` for kicks). Currently, not 
all of Phobos is free of GC allocations, most notably 
exceptions (refer to [1] and similar topics)


[1] http://forum.dlang.org/thread/occ9kk$24va$1...@digitalmars.com


Frankly seeing in this example that the GC was in theory able to 
kick in 6 times in a simple 100 item loop, that is not efficient. 
I if did my own memory management, the variable cleanup will have 
been done in one go, right after the loop. Simply more efficient.


Been thinking about this topic. Dlang has a destructor, does that 
means if you allocate on the GC and then do your own destructor, 
it technically counts as manual memory management?


That is assuming the GC removes the memory reference when you 
call it. I remember seeing in some other languages ( C# possibly? 
) that referring a variable to be freed only meant the GC freed 
the memory when it felt like it, not the exact spot when you told 
it.


I personally think that people simple have a bad taste with GC 
because they kick in too much outside there control. For 90% the 
default behavior is good but its those 10% that leaves a bad 
taste with people ( GC on critical moments and hurting 
performance in return ).

Re: Garbage Collector?

[bachmeier via Digitalmars-d]

On Thursday, 27 April 2017 at 15:50:56 UTC, Ben wrote:

A few days ago i was reading this topic:

https://news.ycombinator.com/item?id=14165198

And the whole GC keeps coming up as a negative ( compared to 
Rust ).


From my understanding there has been a proposal DIP1000 to 
address this issue. Is there any update on this topic?


Is it possible to run D without the GC AND the standard library?


You might find this blog post relevant:
http://dlang.org/blog/2017/03/20/dont-fear-the-reaper/

There is opposition to the GC from a small but vocal group of 
mostly C++ developers. Sometimes they even make valid arguments. 
You can find many threads on the topic here.

Re: Garbage Collector?

[Moritz Maxeiner via Digitalmars-d]

On Thursday, 27 April 2017 at 15:50:56 UTC, Ben wrote:

A few days ago i was reading this topic:

https://news.ycombinator.com/item?id=14165198

And the whole GC keeps coming up as a negative ( compared to 
Rust ).


That's subjective, at best.
I see most of Rust's ownership mechanics in a negative light 
(love the lisp-inspired syntax, though), as they incur a severe 
productivity decrease while providing little safety benefit for 
/me/ over what I already have in D. But, as always, YMMV.




From my understanding there has been a proposal DIP1000 to 
address this issue.


You'll have to be more specific about what issue you're referring 
to. People not liking garbage collection? In any case, AFAIU 
DIP1000 was about more mechanically verifiable memory safety 
features when not using the GC.




Is it possible to run D without the GC AND the standard library?


It is possible to run a D program without the GC only if you 
don't allocate using the GC. If you want to see D code that does 
allocate using the GC, the compiler flag `-vgc` is your friend 
(compile Phobos with `-vgc` for kicks). Currently, not all of 
Phobos is free of GC allocations, most notably exceptions (refer 
to [1] and similar topics)


[1] http://forum.dlang.org/thread/occ9kk$24va$1...@digitalmars.com

Garbage Collector?

[Ben via Digitalmars-d]

A few days ago i was reading this topic:

https://news.ycombinator.com/item?id=14165198

And the whole GC keeps coming up as a negative ( compared to Rust 
).


From my understanding there has been a proposal DIP1000 to 
address this issue. Is there any update on this topic?


Is it possible to run D without the GC AND the standard library?

Re: Eclipse OMR project provides a reusable Garbage Collector

[Chris Wright via Digitalmars-d]

On Sat, 03 Dec 2016 15:45:00 +, Chris Wright wrote:
> Also, OMR assumes there are no unaligned pointers to GC memory.

Also no pointers-to-members, but I have a way around both these problems. 
Obvious in hindsight.

The memory overhead is O(#allocations). The time overhead is O(lg 
#allocations) per pointer reference per scan.

So we won't see numbers as impressive as Java's, unfortunately -- not 
unless we forbid pointers-to-members and rework array slicing.


The details:

When I allocate a thing, I insert an entry for the thing into an ordered 
set of allocated ranges. The entry will be a (begin, end) pointer pair 
for the allocation. I can get this info because allocations tell me how 
much to allocate, and OMR's GC gives me a pointer.

When inserting that allocated range into the set, I will have to erase 
any overlapping elements. This shouldn't be difficult -- as long as the 
no-overlapping-ranges invariant was maintained before this call, I only 
have to look one element to the left, and it's quick to determine how far 
to the right I need to look.

When scanning an object, we find the pointers in that object as we 
currently do. Then, instead of enqueueing those pointers to be scanned, 
we look them up in the allocation set and enqueue the allocation's 
`begin` pointer.

Pretty much what we do today. It's just replacing a little less of the GC 
than we would prefer.