Re: Real simple unresolved external symbols question...
On Thursday, 11 February 2021 at 00:18:23 UTC, H. S. Teoh wrote: On Wed, Feb 10, 2021 at 11:35:27PM +, WhatMeWorry via Digitalmars-d-learn wrote: [...] Okay, thanks. Then why does the README.md at https://github.com/dlang/druntime say "Runtime is typically linked together with Phobos in a release such that the compiler only has to link to a single library to provide the user with the runtime and the standard library." Probably outdated information. Somebody should submit a PR for it. ;-) It basically says there's no separate libdruntime with DMD, it's included in libphobos2 (unlike LDC). So 'runtime' in the context of the druntime repo is 'druntime', not 'C runtime and other platform libs' libphobos/druntime depend on.
Re: Real simple unresolved external symbols question...
On Wed, Feb 10, 2021 at 11:35:27PM +, WhatMeWorry via Digitalmars-d-learn wrote: [...] > Okay, thanks. Then why does the README.md at > > https://github.com/dlang/druntime > > say "Runtime is typically linked together with Phobos in a release > such that the compiler only has to link to a single library to provide > the user with the runtime and the standard library." Probably outdated information. Somebody should submit a PR for it. ;-) T -- Let X be the set not defined by this sentence...
Re: Real simple unresolved external symbols question...
On Tuesday, 9 February 2021 at 19:37:17 UTC, WhatMeWorry wrote:
I'm trying to create a super simple dynamic library consisting
of two files:
file2.d --
extern(D):
double addEight(double d) { return (d + 8.0); }
fileB.d --
extern(D)
{
string concatSuffix(string s) { return (s ~ ".ext"); }
}
dmd -m64 -c file2.d
dmd -m64 -c fileB.d
creates file2.obj and fileB.obj files
link.exe /DLL /NOENTRY file2.obj fileB.obj msvcrt.lib
If you go the easy route and use LDC, everything works:
ldc2 -shared file1.d file2.d
=> file1.dll
Or, if you really prefer separate compile + link:
ldc2 -c file1.d
ldc2 -c file2.d
ldc2 -shared file1.obj file2.obj
Use `-v` to see what the actual link.exe cmdline is, and notice
the MSVC and Windows libs that are implicitly added. - IIRC, DMD
only embeds these system libs into the object file containing
main/DllMain (and you have none), whereas LDC adds them to the
linker cmdline.
Re: Profiling
On Wednesday, 10 February 2021 at 11:52:51 UTC, JG wrote: As a follow up question I would like to know what tool people use to profile d programs? I use this one: https://code.dlang.org/packages/profdump e.g. ``` dub build --build=debug --build=profile # run your program to generate trace.log profdump -b trace.log trace.log.b profdump -f --dot --threshold 1 trace.log trace.log.dot echo 'view it with: xdot trace.log.dot' ```
Re: Real simple unresolved external symbols question...
On Wednesday, 10 February 2021 at 11:38:00 UTC, Ferhat Kurtulmuş wrote: On Tuesday, 9 February 2021 at 19:37:17 UTC, WhatMeWorry wrote: I'm trying to create a super simple dynamic library consisting of two files: [...] remove /NOENTRY, and include "mixin SimpleDllMain;" in one of the sources. And link with druntime. link /DLL file2.obj fileB.obj druntime-ldc.lib msvcrt.lib Okay, thanks. Then why does the README.md at https://github.com/dlang/druntime say "Runtime is typically linked together with Phobos in a release such that the compiler only has to link to a single library to provide the user with the runtime and the standard library."
Re: Profiling
On Wednesday, 10 February 2021 at 13:31:09 UTC, Guillaume Piolat wrote: On Wednesday, 10 February 2021 at 11:52:51 UTC, JG wrote: [...] Here is what I use for sampling profiler: (On Windows) Build with LDC, x86_64, with dub -b release-debug in order to have debug info. Run your program into: - Intel Amplifier (free with System Studio) - AMD CodeXL (more lightweight, and very good) - Very Sleepy (On Mac) Build with dub -b release-debug Run your program with Instruments.app which you can find in your Xcode.app (On Linux) I don't know. Though most of the time to validate the optimization a comparison program that runs two siilar programs and computer the speed difference can be needed. All Intel tools I'm aware of support (and are free on) Linux. Also, it's just called vTune now and it's been put under the "oneAPI" banner.
Re: GC.addRange in pure function
On Wednesday, 10 February 2021 at 16:25:44 UTC, Petar Kirov [ZombineDev] wrote: [..] A few practical examples: Here it is deemed that the only observable side-effect of `malloc` and friends is the setting of `errno` in case of failure, so these wrappers ensure that this is not observed. Surely there are low-level ways to observe it (and also the act of allocating / deallocating memory on the C heap), but this definition purity what the standard library has decided it was reasonable: https://github.com/dlang/druntime/blob/master/src/core/memory.d#L1082-L1150 These two function calls in Array.~this() can be marked as `pure`, as the Array type as a whole implements the RAII design pattern and offers at least basic exception-safety guarantees: https://github.com/dlang/phobos/blob/81a968dee68728f7ea245b6983eb7236fb3b2981/std/container/array.d#L296-L298 (The whole function is not marked pure, as the purity depends on the purity of the destructor of the template type parameter `T`.)
Re: GC.addRange in pure function
On Wednesday, 10 February 2021 at 13:44:53 UTC, vit wrote:
On Wednesday, 10 February 2021 at 12:17:43 UTC, rm wrote:
On 09/02/2021 5:05, frame wrote:
On Sunday, 7 February 2021 at 14:13:18 UTC, vitamin wrote:
Why using 'new' is allowed in pure functions but calling
GC.addRange or GC.removeRange isn't allowed?
Does 'new' violate the 'pure' paradigm? Pure functions can
only call pure functions and GC.addRange or GC.removeRange is
only 'nothrow @nogc'.
new allocates memory via the GC and the GC knows to scan this
location. Seems like implicit GC.addRange.
Yes, this is my problem, if `new` can create object in pure
function, then GC.addRange and GC.removeRange is may be pure
too.
Can I call GC.addRange and GC.removeRange from pure function
without problem? (using assumePure(...)() ).
TL;DR Yes, you can, but it depends on what "without problem"
means for you :P
# The Dark Arts of practical D code
===
According to D's general approach to purity, malloc/free/GC.* are
indeed impure as they read and write global **mutable** state,
but are still allowed in pure functions **if encapsulated
properly**. The encapsulation is done by @trusted wrappers which
must be carefully audited by humans - the compiler can't help you
with that.
The general rule that you must follow for such
*callable-from-pure* code (technically it is labeled as `pure`,
e.g.:
pragma(mangle, "malloc") pure @system @nogc nothrow
void* fakePureMalloc(size_t);
but I prefer to make the conceptual distinction) is that the
effect of calling the @trusted wrapper must not drastically leak
/ be observed.
What "drastically" means depends on what you want `pure` to mean
in your application. Which side-effects you want to protect
against by using `pure`? It is really a high-level concern that
you as a developer must decide on when writing/using @trusted
pure code in your program. For example, generally everyone will
agree that network calls are impure. But what about logging? It's
impure by definition, since it mutates a global log stream. But
is this effect worth caring about? In some specific situations it
maybe ok to ignore. This is why in D you can call `writeln` in
`pure` functions, as long as it's inside a `debug` block. But
given that you as a developer can decide whether to pass `-debug`
option to the compiler, essentially you're in control of what
`pure` means for your codebase, at least to some extent.
100% mathematical purity is impossible even in the most strict
functional programming language implementations, since our
programs run on actual hardware and not on an idealized
mathematical machine. For example, even the act of reading
immutable data can be globally observed as by measuring the
memory access times - see Spectre [1] and all other
microarchitecture side-channel [1] vulnerabilities.
[1]:
https://en.wikipedia.org/wiki/Spectre_(security_vulnerability)
[2]: https://en.wikipedia.org/wiki/Side-channel_attack
That said, function purity is not useless at all, quite the
contrary. It is about making your programs more deterministic and
easy to reason about. We all want less bugs in our code and less
time spent chasing hard to reproduce crashes, right?
`pure` is really about limiting, containing / compartmentalizing
and controlling the the (in-deterministic) global effects in your
program. Ideally you should strive to structure your programs as
a pure core, driven by an imperative, impure shell. E.g. if
you're working on an accounting application, the core is the part
that implements the main domain / business logic and should be
100% deterministic and pure. The imperative shell is the part
that reads spreadsheet files, exports to pdf, etc. (actually just
the actual file I/O needs to be impure - the actual decoding /
encoding of data structures can be perfectly pure).
Now, back to practice and the question of memory management.
Of course allocating memory is globally observable effect and
even locally one can compare pointers, as Paul Backus mentioned,
as D is a systems language. However, as a practical concession,
D's concept of pure-ity is about ensuring high-level invariants
and so such low-level concerns can be ignored, as long as the
codebase doesn't observe them. What does it mean to observe them?
Here's an example:
---
void main()
{
import std.stdio : writeln;
observingLowLevelSideEffects.writeln; // `false`, but could
be `true`
notObservingSideEffects.writeln; // always `true`
}
// BAD:
bool observingLowLevelSideEffects() pure
{
immutable a = [2];
immutable b = [2];
return a.ptr == b.ptr;
}
// OK
bool notObservingSideEffects() pure
{
immutable a = [2];
immutable b = [2];
return a == b;
}
---
`observingLowLevelSideEffects` is bad, as according to the
language rules, the compiler is free to make `a` and `b` point to
the same immutable array, the result of the function
Re: GC.addRange in pure function
On Wednesday, 10 February 2021 at 12:17:43 UTC, rm wrote: On 09/02/2021 5:05, frame wrote: On Sunday, 7 February 2021 at 14:13:18 UTC, vitamin wrote: Why using 'new' is allowed in pure functions but calling GC.addRange or GC.removeRange isn't allowed? Does 'new' violate the 'pure' paradigm? Pure functions can only call pure functions and GC.addRange or GC.removeRange is only 'nothrow @nogc'. new allocates memory via the GC and the GC knows to scan this location. Seems like implicit GC.addRange. Yes, this is my problem, if `new` can create object in pure function, then GC.addRange and GC.removeRange is may be pure too. Can I call GC.addRange and GC.removeRange from pure function without problem? (using assumePure(...)() ).
Re: Profiling
On Wednesday, 10 February 2021 at 11:52:51 UTC, JG wrote: Thanks for the suggestions. However, I would prefer not to spend time trying to debug d-profile-viewer at the moment. As a follow up question I would like to know what tool people use to profile d programs? Here is what I use for sampling profiler: (On Windows) Build with LDC, x86_64, with dub -b release-debug in order to have debug info. Run your program into: - Intel Amplifier (free with System Studio) - AMD CodeXL (more lightweight, and very good) - Very Sleepy (On Mac) Build with dub -b release-debug Run your program with Instruments.app which you can find in your Xcode.app (On Linux) I don't know. Though most of the time to validate the optimization a comparison program that runs two siilar programs and computer the speed difference can be needed.
Re: Profiling
On 2/10/21 2:52 PM, JG wrote: On Tuesday, 9 February 2021 at 18:33:16 UTC, drug wrote: On Tuesday, 9 February 2021 at 07:45:13 UTC, JG wrote: I was trying to profile a d program. So I ran: dub build --build=profile. I then ran the program and it produced trace.log and trace.def. I then ran d-profile-viewer and got the following error: std.conv.ConvException@/home/jg/dlang/ldc-1.24.0/bin/../import/std/conv.d(2382): Unexpected '-' when converting from type char[] to type ulong I'm guessing only but it looks like slurp is used to read text output but the format used does not correspond to the real data format. Thanks for the suggestions. However, I would prefer not to spend time trying to debug d-profile-viewer at the moment. As a follow up question I would like to know what tool people use to profile d programs? Could you provide trace.log that I can take a look? I have no much time but some bugs can be fixed fast. It would be ideal if you create the issue in the package repo and provides the test case. Also there is probability the author will fix this.
Re: GC.addRange in pure function
On 09/02/2021 5:05, frame wrote: On Sunday, 7 February 2021 at 14:13:18 UTC, vitamin wrote: Why using 'new' is allowed in pure functions but calling GC.addRange or GC.removeRange isn't allowed? Does 'new' violate the 'pure' paradigm? Pure functions can only call pure functions and GC.addRange or GC.removeRange is only 'nothrow @nogc'. new allocates memory via the GC and the GC knows to scan this location. Seems like implicit GC.addRange.
Re: Profiling
On Tuesday, 9 February 2021 at 18:33:16 UTC, drug wrote: On Tuesday, 9 February 2021 at 07:45:13 UTC, JG wrote: I was trying to profile a d program. So I ran: dub build --build=profile. I then ran the program and it produced trace.log and trace.def. I then ran d-profile-viewer and got the following error: std.conv.ConvException@/home/jg/dlang/ldc-1.24.0/bin/../import/std/conv.d(2382): Unexpected '-' when converting from type char[] to type ulong I'm guessing only but it looks like slurp is used to read text output but the format used does not correspond to the real data format. Thanks for the suggestions. However, I would prefer not to spend time trying to debug d-profile-viewer at the moment. As a follow up question I would like to know what tool people use to profile d programs?
Re: Real simple unresolved external symbols question...
On Tuesday, 9 February 2021 at 19:37:17 UTC, WhatMeWorry wrote: I'm trying to create a super simple dynamic library consisting of two files: [...] remove /NOENTRY, and include "mixin SimpleDllMain;" in one of the sources. And link with druntime. link /DLL file2.obj fileB.obj druntime-ldc.lib msvcrt.lib
Re: Real simple unresolved external symbols question...
On Tuesday, 9 February 2021 at 19:37:17 UTC, WhatMeWorry wrote:
I'm trying to create a super simple dynamic library consisting
of two files:
file2.d --
extern(D):
double addEight(double d) { return (d + 8.0); }
fileB.d --
extern(D)
{
string concatSuffix(string s) { return (s ~ ".ext"); }
}
dmd -m64 -c file2.d
dmd -m64 -c fileB.d
creates file2.obj and fileB.obj files
link.exe /DLL /NOENTRY file2.obj fileB.obj msvcrt.lib
fileB.obj : error LNK2019: unresolved external symbol
_D12TypeInfo_Aya6__initZ referenced in function
_D5fileB12concatPrefixFAyaZQe
fileB.obj : error LNK2019: unresolved external symbol
_d_arraycatT referenced in function
_D5fileB12concatPrefixFAyaZQe
Ok. I find _d_arraycatT in the D website at:
https://dlang.org/library/rt/lifetime/_d_arraycat_t.html
So I thought, this symbol will be in phobos64.lib. But when I
add it to the command, all hell breaks loose:
link.exe /DLL /NOENTRY file2.obj fileB.obj msvcrt.lib
phobos64.lib
phobos64.lib(bits_23fa_21c.obj) : error LNK2001: unresolved
external symbol memcpy
o o o
file2.dll : fatal error LNK1120: 57 unresolved externals
So I'm stuck and don't have a clue as to how to continue.
I thought Phobos64.lib was self-contained. Do I need to add
other libraries? And how do I know which ones?
You have 2 obj files, and you link them. Where do you want the
"self contained" to be present?
The missing symbols in your case are in druntime.lib, not in
phobos.
Re: Real simple unresolved external symbols question...
Add libraries that provide missing symbols.
Re: GC.addRange in pure function
On Tuesday, 9 February 2021 at 21:00:39 UTC, Paul Backus wrote: On Tuesday, 9 February 2021 at 19:53:27 UTC, Temtaime wrote: pure is broken. Just don't [use it] Allowing memory allocation in pure code in a language that can distinguish between pointer equality and value equality is, let's say, "unprincipled." pure in D is a very useful concept, even if it's not literally the same as pure in functional languages. Recommending not to use it is bad advice IMHO.
