Re: D-dll support: testers needed round 2
On Saturday, 10 February 2018 at 02:24:58 UTC, rikki cattermole wrote: On 09/02/2018 8:34 PM, Benjamin Thaut wrote: -import switch makes me a little concerned, what exactly is it changing that makes it required? Thank you Rikki for derailing this topic with the first post. As you might noticed, the title says "testers needed" not "concerns needed". And if you absolutly have to voice your concerns at least do it in a usefull way. Give a propper reason for your concerns, ideally a technical one. Not just "I'm concerned about x". Did you Read DIP 45 in its current state? Did you watch my DConf Talk about Dll support? Did you understand how the shared library menchanism works on windows? If you did then the details behind the -import switch should be very self explanatory. If you didn't -import is needed for the compiler to know when to dllimport a symbol. If you need more detail you will have to wait until I updated DIP 45. With the little time I have at the moment I can't re-explain the full details of the implemenation to everyone in person, that would be very bad use of my time. So if you want to help, download the binary distribution and give it a try. Try to find a corner case with dlls that my implementation doesn't cover. That would be useful. Kind Regards Benjamin Thaut
D-dll support: testers needed round 2
My work on dll support for D continues. There is another iteration I need help testing with. Getting started tutorial: http://stuff.benjamin-thaut.de/D/getting_started.html The DIP can again found be here: https://github.com/Ingrater/DIPs/blob/ReviveDIP45/DIPs/DIP45.md The DIP is not up to date. The tutorial is more recent. If the DIP and the tutorial contradict each other trust the tutorial. Its still useful to read the dip for context, especially as how to use "export" in code. This time around the shared version of phobos should have all required symbols exported. This means when you link against the shared version of phobos and find a linker error, please report that here. Ideally with a reduced repro case. A binary distribution is provided, see the tutorial for details.
Re: Debugging on Windows
On Thursday, 8 February 2018 at 21:09:33 UTC, JN wrote: Hi, is there any way to debug binaries on Windows? I'd at least like to know which line of code made it crash. If it's D code, I get a call trace usually, but if it's a call to a C library, I get a crash and that's it. I am using VSCode and I'd prefer to debug in it if possible, but using other IDEs is a possibility for me if that will help. The best option most likely is to get the Visual Studio Community Edition and then Install the VisualD extension for Visual Studio. This will give you a very good debugging experience with a bulitin D expression evaulator and the usual features the very good Visual Studio debugger comes with. Kind Regards Benjamin Thaut
Re: String Switch Lowering
Am 27.01.2018 um 08:40 schrieb Walter Bright: This clearly should be in bugzilla. https://issues.dlang.org/show_bug.cgi?id=18324 -- Kind Regards Benjamin Thaut
Re: String Switch Lowering
On Saturday, 27 January 2018 at 07:40:16 UTC, Walter Bright wrote: This clearly should be in bugzilla. As phobos or dmd bug?
Re: String Switch Lowering
Am 25.01.2018 um 19:42 schrieb Jonathan M Davis: That particular switch statement is in a function that's deprecated and scheduled to be completely removed in about six months, so I don't see much point in worrying about it unless it's causing serious problems, and while that symbol name is stupidly long, AFAIK, it's not really causing any problems. The main problem is binary size for shared library verisons of phobos. The overly long symbols names contribute significantly to binary size as for both exports and imports. The full symbol name is stored in the dll and the exe that uses the dll. -- Kind Regards Benjamin Thaut
String Switch Lowering
Re: Dll support: testers needed
Am 12.01.2018 um 04:02 schrieb Domain: On Thursday, 11 January 2018 at 18:56:23 UTC, Benjamin Thaut wrote: Am 11.01.2018 um 05:10 schrieb Domain: [...] So I cannot use phobos as a static library when I use dll? If you have a single D-dll that is used from a C program then you can use phobos as a static library. In all other cases you must use the dll version of phobos otherwise the sharing of objects between dlls/exe doesn't work correctly. -- Kind Regards Benjamin Thaut
Re: Dll support: testers needed
Am 11.01.2018 um 05:10 schrieb Domain: I restart my computer, and it can produce dll now. But when I compile exe: dmd -m64 -useshared app.d -ofapp.exe app.obj : error LNK2019: 无法解析的外部符号 _D3std12experimental6logger4core17stdThreadLocalLogFNdNfZCQCeQCdQBsQBo6Logger， 该符号在函数 _D3std12experimental6logger4core__T18defaultLogFunctionVEQCdQCcQBrQBn8LogLeveli64Z__TQByVii9VAyaa5_6170702e64VQra8_6170702e6d61696eVQBna15_766f6964206170702e6d61696e2829VQCza3_617070TQDmZQFwFNfLQDxZv 中被引用 As I said in the first post, I'm not interrested in linker errors due to missing 'export' annotation. std.experimental.logger does not have an export annotations and therefor will cause linker errors. If you absolutely must use std.experimental.logger you will have to annotate it with 'export' and recompile phobos. The idea behind my initial PR is to build in basic support for dlls, not to fully annotate all of phobos with 'export'. dmd -m64 app.d -ofapp.exe dmd -m64 -shared dll1.d -ofdll1.dll dmd -m64 -shared dll2.d -ofdll2.dll Your command line is wrong. You are missing the -useshared option and therefore your dlls / executable don't share the same version of druntime. output: 2018-01-11T12:04:51.635:app.d:main:9 Loading dll1.dll 2018-01-11T12:04:51.894:dll1.d:start:22 dll1 starting MyLogger: dll1 started 2018-01-11T12:04:51.897:app.d:main:14 Loading dll2.dll 2018-01-11T12:04:52.164:dll2.d:start:9 dll2 started How can you get the output of your program if it fails to link? -- Kind Regards Benjamin Thaut
Re: invalid or corrupt file: duplicate COMDAT / Previous Definition Different
Am 10.01.2018 um 20:32 schrieb Anonymouse: I don't have a reduced testcase yet. I figured I'd ask if it's something known before making the effort. Are you by any chance mixing debug and release builds? Or are the -version specifiers different when compiling the various parts of your program? Check your compiler flags and ansure that they are the same over your entire build process. Especiall -debug -relase -inline -O -version -- Kind Regards Benjamin Thaut
Re: Dll support: testers needed
Am 10.01.2018 um 13:39 schrieb Domain: Sorry, my mistake. But I cannot use your binary: D:\>dmd -m64 -shared dll.d -ofdll.dll Error: unrecognized file extension dll This works just fine for me. What is the output when you execute "dmd --version"? -- Kind Regards Benjamin Thaut
Re: Dll support: testers needed
Am 10.01.2018 um 05:18 schrieb Domain: On Saturday, 6 January 2018 at 19:32:51 UTC, Benjamin Thaut wrote: I'm currently back on dll support and I'm applying finishing touches to my dll support PR. Now I want to know if I missed any corner cases and it would be great if a few more people gave the dll support a try. [...] I am glad to see this happen. But my use case is not working: http://forum.dlang.org/post/giywzmvfeuddvmvkr...@forum.dlang.org Hi Domain, what your describing is a very simple use case and should definitly work. Did you try it using my binary distribution? Please post a repro-case. I don't have the time to rebuild everyones use case and saying "My use case is not working" does not help at all. -- Kind Regards Benjamin Thaut
Re: Dll support: testers needed
Am 09.01.2018 um 16:03 schrieb Andre Pany: I am building a bridge between Delphi and D. At the moment the executable is written in D and the Dll (Firemonkey UI) is written in Delphi. But I think I want to enable also the other way around. The D coding which is then located in the Dll should be able to call Delphi methods located in the executable. The whole idea idea is that all logic is written in D and you only use the Framework (ui) and libraries Delphi provides. I think this might be a valid use case for a Dll calling functionality located in the exe. Kind regards Andre First let me say that what you are describing is a very uncommon and ill-advised use case. As such there is not going to be any nice to use workflow to acieve what you are trying to do. Still this doesn't mean that it won't be possible. Why ill-advised? What you're describing is a cyclic dependency between your main executable and your dll written in delphi. If you google for "cyclic dependency dll" you will usually get the advice to break your cylic dependency by splitting your code into more dlls. My personal experience shows the same. Cyclic dependencies in dlls are usually not worth the additional effort and hassle. Also you want to export things from your executable, which is also very uncommon. What you should be doing is having 2 dlls and one executable. You have one common library written in D that is build into a dll. Then you have your delphi library which uses the common.dll. Finally you have your main executable written in D which uses both the common.dll and your delphi.dll. This should make it possibly to break the cycle and get you an easy setup. If you absolutley must do it the way you describe it, its still possible. You will have to compile all modules that export something from your executable with the "-c -shared" parameters. E.g. dmd -m64 -c -shared moduleThatExports1.d moduleThatExports2.d -ofexports.obj Now you link the resulting exports.obj file into your executable by calling dmd again dmd -m64 restOfModules.d exports.obj delphi.lib -ofmain.exe Finally you have to get the handle to your main executable by calling HMODULE handle; GetModuleHandleExA(0, null, ); in your delphi dll and then fetching the function pointer for each and every function you want to call from delphi via: GetProcAddress(handle, "mangeledFunctionSymbol") Now finally you can call these functions pointers from delphi and they will call into the D code within your executable. Instead of all that hassle you could instead just have a function in your delphi dll: void recieveFunctionPointer(const(char)* name, void* ptr); which you call for every function that you want to make available from D to delphi. Your delphi code then stores away these pointers depending on the name. That way you don't need to export anything from your executable and can build it normally. Instead of having a function call per function you could obviosuly also use a struct that is defined the same way in D and delphi and contains all relevant functions pointers. -- Kind Regards Benjamin Thaut
Bug in TypeInfo generation?
Lets say I have a library and it contains the following module:
module a;
struct SomeStruct
{
string name;
}
struct SomeOtherStruct(T)
{
~this()
{
typeid(T).initializer;
}
}
struct ThirdStruct
{
SomeOtherStruct!SomeStruct m;
}
And now I have a second module:
module b;
import a;
int main()
{
return 0;
}
I would expect that I can compile module .b into an executable like this:
dmd -m64 b.d
As I'm not using anything from module a I would expect that this
compiles and links just fine. It does not, it gives linker errors:
b.obj : error LNK2001: unresolved external symbol
"_D1a10SomeStruct9__xtoHashFNbNeKxSQBgQBhZm".
b.obj : error LNK2001: unresolved external symbol
"_D1a10SomeStruct11__xopEqualsFKxSQBfQBgKxQjZb".
Debugging dmd a bit shows that the typeid expression in the destructor
of SomeOtherStruct causes the type info of SomeStruct to be created and
then in the gen object phase the full type info for SomeStruct is
written into the object file. This is a bug in my eyes though. The
compiler should be able to figure out that the destructor of
SomeOtherStruct must already exist, as its clearly always instanciated.
As a result the compiler should not instanciate the TypeInfo for
SomeStruct but instead assume that it also already exists. By doing so
it would never write the SomeStruct type info to the object file and the
linker errors would go away.
I'm asking this because this behavior for type info generation turns out
to be a problem for my dll-work.
--
Kind Regards
Benjamin Thaut
Re: Dll support: testers needed
Am 09.01.2018 um 12:02 schrieb MrSmith: Is it possible to put common code in exe, and use that code from dlls? Or anything can be exported only by dll? You can only export from dlls. I don't know of any use case where exporting from a executable would make sense. No, you can't put common code into a executable and use that code from dlls. Thats not how dlls are designed to work on windows. If you absoutely still want to export from an executable you can still use -L/EXPORT:"mangledSymbolName" Is it possible to have circular dependencies between dlls? No. You will get linker errors. Same as in C/C++. Tips for doing dlls with dub? Well, first my PR will have to get merged. Then you'll have to wait until a release exists that actually has my changes and then maybe there is support added in dub. Before that I think trying to use dub to create dlls is wasted effort. -- Kind Regards Benjamin Thaut
Re: Dll support: testers needed
Am 09.01.2018 um 05:19 schrieb Dylan Graham: It's pretty basic but I'm glad it works. I'll try doing something more advanced. Hi Dylan, I'm glad that it works for you. I'm curios though, why are you using an import library for your plugin? Normally plugins are build without and import library (the -L/IMPLIB parameter) and then loaded dynamically through LoadLibrary by your main executable. This way you can add/remove plugins without the need to recompile your main executable. Usually such plugin systems are done like this: You have some common library that defines shared functionality and the base interfaces such as a Plugin base class or Plugin interface. This common library is build into a dll with import library. You have one ore multiple plugins that link against the common library and implement the plugin interface / plugin base class. Each of these plugins provides a function "getPluginImpl" or something along the lines which can be used to get an instance of the Plugin implementation. You have the main executable, which links against the common library. It dynamically loads the plugin.dlls and calls the getPluginImpl method on them to retrieve the implementation. No import library involved here. What your doing is also a valid use case. It is usually done when link times get to long or if you want to have clean interface between the different subsystems in your application. Your codebase is usually split into multiple dlls each with a import library which are then all used by the main executable. -- Kind Regards Benjamin Thaut
Re: Dll support: testers needed
Am 08.01.2018 um 00:06 schrieb solidstate1991: On Saturday, 6 January 2018 at 19:32:51 UTC, Benjamin Thaut wrote: I would volunteer, but I don't know how much I can do in my exam session. Maybe in February, until then not much. Just do some testing whenever you find time, keep monitoring this thread foor upates. -- Kind Regards Benjamin Thaut
Re: Memory Dump in D
Am 07.01.2018 um 16:40 schrieb H3XT3CH: I need it for windows and linux but primary for windows On windows there is the MiniDumpWriteDump function: https://msdn.microsoft.com/en-us/library/windows/desktop/ms680360(v=vs.85).aspx This might sound misleading but besides being able to write out a minidump, which essentially only contains the stacks of all threads, you can also write out a full dump which contains all memory of the process. Usually these files are named *.dmp on windows. You can then open these *.dmp files with WinDBG or the Visual Studio Debugger and inspect the state of the program at which the dump was taken just you would normally inspect a running program at a breakpoint. What I have been doing in the past is: - Set a exception handler via SetUnhandledExceptionFilter - The exception handler writes out a minidump and calls a extrnal process - The external process prompts the user for additional information on the crash and uploads the minidump into a crash database. - I inspect the crash database and see whats going wrong for the users of my programs. -- Kind Regards Benjamin Thaut
Re: Memory Dump in D
Am 07.01.2018 um 12:05 schrieb H3XT3CH: Hello i want to create a memory dump in D. The memory dump is for forensic usage so it must a dump of the complete ram. Can anyone help me ? I know that programms already exist that create correct dumps of my memory but i want to understand how it works and how to work with my dumps What operating system are you using? -- Kind Regards Benjamin Thaut
Re: Dll support: testers needed
Am 07.01.2018 um 00:42 schrieb Rubn: Looks good. If you want testers though, providing binaries would be beneficial. Compiling dmd/druntime/phobos on Windows can be a pain. I made a binary distribution. I updated http://stuff.benjamin-thaut.de/D/getting_started.html with the details. -- Kind Regards Benjamin Thaut
Re: Dll support: testers needed
Am 07.01.2018 um 15:08 schrieb MrSmith: Does the implementation support dynamically loaded dlls? Yes. There is even a test for it -- Kind Regards Benjamin Thaut
Re: Dll support: testers needed
Am 07.01.2018 um 03:57 schrieb rikki cattermole: > +infinity > > Few things I would like answered: > > - Why -useShared and not -fPIC? >- If -useShared why not use it on *nix to turn on -fPIC? Because the DIP says that -useshared and -fPIC are loosley comparable. This does not mean that they are the same or interchangeable. In fact they are not. On linux when building a executable that uses a shared library -fPIC is not neccessary, unless your distribution requires position independend code everywhere. On windows on the other hand you will need to pass -useshared when building a executable that uses shared libraries. > - Why isn't DllIsUsedFromC.no automatically detected? Because its not possible. I don't know of any way to detect wether a dll is loaded by a c-executable or d-executable. As a result the user has to specify it. The main problem here is that currently D Dlls hijack all threads. This is neccessary because currently there is no shared version of phobos, so the information about the threads can't be shared between multiple D dlls. One possibility would also be to remove the thread hijacking alltogether and require the user to manually add threads that have been created from C. This is also the current state on linux if I'm not mistaken. But doing so would break backwards compatibility. > Regarding C-runtime, yes we need to distribute every version that we support on Windows. The compiler will pick the latest (or the installer I guess) to use, but we should be able to switch between them. Either that or we ship a static lib of Phobos to be linked into a dll upon first request of a specific C-runtime. That really is a problem I'm not going to solve. It is more a problem of distributing dmd and building the installer which is beyond the scope of my current PR. -- Kind Regards Benjamin Thaut
Dll support: testers needed
I'm currently back on dll support and I'm applying finishing touches to my dll support PR. Now I want to know if I missed any corner cases and it would be great if a few more people gave the dll support a try. -To try it out you will need to build dmd, druntime and phobos yourself. I'm not going to provide a binary distribution. -You will need to target the microsoft linker. E.g. either use -m64 or -m32mscoff when compiling your programs. -be warned, the state of the dll support is not final and subject to change. There is no garantuee that there won't be any breaking changes to dll support. You can find a quick start guide here: http://stuff.benjamin-thaut.de/D/getting_started.html If you need more information and examples take a look here: https://github.com/Ingrater/DIPs/blob/ReviveDIP45/DIPs/DIP45.md Not all of phobos/druntime has correct export annotations yet as this is not the goal of my initial pull request. If you use a function from druntime/phobos that does not have a correct export annotation you will get a linker error. I'm not interrested in those. I'm interrested in cases where export is correctly used and you will still get linker errors / incorrect program behavior. So if you encounter a linker error first check if the declaration has a export annotation or not. Try adding it and see if the error goes away. If it does not go away, report it here or send me an e-mail to code at benjamin-thaut.de If you find anything, small repro cases would be appreciated as it makes my life a lot easier. -- Kind Regards Benjamin Thaut
Re: How do I set a class member value by its name in a string?
On Wednesday, 27 December 2017 at 20:04:29 UTC, Marc wrote:
I'd like to set the members of a class by its name at runtime,
I would do something like this:
__traits(getMember, myClass, name) = value;
but since name is only know at runtime, I can't use __traits().
What's a workaround for this?
You will have to use a combination of compile time and runtime
methologies.
Essentially what you want is this:
void setMemberValue(string name, int value)
{
switch(name)
{
case "member1":
member1 = value;
break;
case "member2":
member2 = value;
break:
...
}
}
As you don't want to write this for all members by hand you
should write a function which generates the source code for this
switch using static foreach and __traits(allMembers) and then
mixin the generated string into the setMemberValue method. If
your members can have different types you will also need runtime
type that can hold multiple types. For simplicity I just used
"int" in the above example. You could use "std.variant" for this.
Re: float.max + 1.0 does not overflow
On Wednesday, 27 December 2017 at 13:40:28 UTC, rumbu wrote:
Is that normal?
use std.math;
float f = float.max;
f += 1.0;
assert(IeeeFlags.overflow) //failure
assert(f == float.inf) //failure, f is in fact float.max
On the contrary, float.max + float.max will overflow. The
behavior is the same for double and real.
This is actually correct floating point behavior. Consider the
following program:
float nextReprensentableToMax = float.max;
// find next smaller representable floating point number
(*cast(int*))--;
writefln("%f", float.max - nextReprensentableToMax);
It computes the difference between float.max and the next smaller
reprensentable number in floating point. The difference printed
by the program is:
20282409603651670423947251286016.0
As you might notice this is siginificantly bigger then 1.0.
Floating point operations work like this: They perform the
operation and then round to the nearest representable number in
floating point. So adding 1.0 to float.max and then rounding to
the nearest representable number will just give you back
float.max. If you however add float.max and float.max the next
nearest reprensentable number is float.inf.
When trying to understand how floating point works I would highly
recommend that you read these articles (oldest first):
https://randomascii.wordpress.com/category/floating-point/
Kind Regards
Benjamin Thaut
Re: DMD Windows 64bit target - how to setup the environment?
On Monday, 25 December 2017 at 16:35:26 UTC, realhet wrote: Now I have my first DMD 64bit windows console app running. (And I'm already afraid of the upcoming windowed application haha) My recommendation for getting setup on Windows with D is as follows: 1) Install the latest visual studio community version: https://www.visualstudio.com/thank-you-downloading-visual-studio/?sku=Community=15 2) Install dmd through the windows installer: https://dlang.org/download.html , exe option under windows. This will correctly configure dmd to use the visual studio install from above. From here on dmd will be able to compiler with -m64 and -m32mscoff from the command line. 3) Install VisualD from http://rainers.github.io/visuald/visuald/StartPage.html 4) Create a D project in visual studio and start programming. Compile and debug as usual in Visual Studio. For you command line compilation. Why are you passing -c to the compiler? The idea behind D is that you pass all modules at the same time to the compiler. This allows the compiler to reuse template instances across modules and in general compilation with D is fast enough to be able to recompile everything each time you make changes. No need for a build system. A single batch file with one command in it is usually enough. There is no need to call the linker manually. Dmd will do this for you. Just call dmd like this: dmd -m64 source1.d source2.d source3.d -ofoutput.exe and thats it. You can add additional options like -O -inline -noboundscheck -g -debug -relase. Those are the most commoly used options. Kind Regards Benjamin Thaut
Re: Blog post: using dynamic libraries in dub
On Thursday, 21 December 2017 at 13:30:32 UTC, David Nadlinger wrote: There would probably have to be some sort of compatibility flag to avoid breaking all libraries on systems where symbols are visible externally by default (Linux, ...). — David Agree, although this currently really is a bad default. Having all symbols visible by default can lead to really long shared library load times especially with template heavy code. I've seen reports of shared libaries using boost in c++ having load times of 60 seconds and above due to the number of visible symbols. Ideally we should end up with visibility hidden being the default and only making symbols visible which use "export", because that is what it was designed for in the first place.
Re: DMD Test Suite Windows
On Wednesday, 20 December 2017 at 10:15:45 UTC, Benjamin Thaut wrote: I found that both the make that comes with msys and the make that comes with mingw work for me. I‘m currently on vacation but once I‘m back and in case you are interrested I can post the batch file I use to run the dmd tests. Here are my scripts: http://stuff.benjamin-thaut.de/D/DmdScripts.zip You need to edit import/config.cmd and put in your paths to visual studio, msys and tdm gcc. A regular mingw also works, but if you use a regular mingw you have to remove the -j4 flag from the run-tests scripts as multi job make is broken in regular mingw (Thats why I use tdm-gcc). The 5 scripts included are: run-tests-win32mscoff.cmd - Runs the dmd test suite with the -m32mscoff flag run-tests-win64.cmd - Runs the dmd test suite with the -m64 flag setup-path-32mscoff.cmd - Sets up the environment so that building of druntime / phobos using the -m32mscoff flag becomes possible. setup-path-64.cmd - Sets up the environment so that building of druntime / phobos using the -m64 flag becomes possible. make-dlls.cmd - A bit of a unlucky name, but makes sense for my local workflow. Builds both druntime and phobos in 64 and 32mscoff. Hope this helps. Kind Regards Benjamin Thaut
Re: Blog post: using dynamic libraries in dub
On Wednesday, 20 December 2017 at 08:09:53 UTC, Martin Nowak wrote: At some point `dub build` should get a `--shared` option to change the meaning of `library` target types from `staticLibrary` to `dynamicLibrary`. The shared libs could be linked with absolute paths. Would this work in all cases? Do tls variables work across Linux shared libraries? Do we expect all dub libraries to have correct export annotations once export goes live?
Re: DMD Test Suite Windows
On Monday, 18 December 2017 at 16:06:33 UTC, Jonathan Marler wrote: Trying to run the dmd test suite on windows, looks like Digital Mars "make" doesn't work with the Makefile, I tried Gnu Make 3.81 but no luck with that either. Anyone know which version of make it is supposed to work with on windows? Is it supposed to work on windows at all? I found that both the make that comes with msys and the make that comes with mingw work for me. I‘m currently on vacation but once I‘m back and in case you are interrested I can post the batch file I use to run the dmd tests.
Re: Note from a donor
On Tuesday, 24 October 2017 at 21:11:38 UTC, solidstate1991 wrote: DIP45 has the solution (make export an attribute), it needs to be updated for the new DIP format from what I heard. It needs to be pushed, as Windows still the most popular OS on the consumer side of things, then we can have Phobos and DRuntime as DLLs without using experimental versions of DMD. I have some plans with the better DLL support, such as the possibility of a D based Python (for better class interoperability with D), or even using a modified set of D for scripting (eg. SafeD only). Unfortunately I currenlty don't have a lot of spare time to spend on open source projets. I will however have some more time in December. My current plan is to revive DIP 45 and my dll implementation and give it some finishing touches as discussed with Walter at DConf 2017.
Re: Test if a class is extern(c++)
On Monday, 10 April 2017 at 18:56:42 UTC, BBasile wrote: Hello, I have a trait for this: https://github.com/BBasile/iz/blob/master/import/iz/types.d#L650 Hi BBasile, I think your trait is a good starting point for my needs. Thanks.
Test if a class is extern(c++)
In particular I want to know if the vtable of the class has the class info member. Is there any way to do this at compile time? At runtime? Kind Regards Benjamin Thaut
Re: const(Class) is mangled as Class const* const
On Sunday, 26 March 2017 at 14:30:00 UTC, deadalnix wrote: It's consistent. D's const is transitive, and D doesn't allow you to specify const on the indirection of a reference type. So there is no problem on the C++ mangling side of things, but, arguably, there is one in D's sementic, that isn't new. I disagree. When binding C++ code to D I don't care about D's const rules. I care about the C++ const rules. There are thousands of C++ libraries out there that can't be bound to D because they use const Class* instead of const Class* const. So in my eyes there is definitly something wrong with the C++ mangling of D.
const(Class) is mangled as Class const* const
Consider the following C++ and D source:
class Class
{
virtual ~Class(){}
};
// mangles as ?getClass@@YAPEAVClass@@XZ
Class * getClass() { return nullptr; }
// mangles as ?getClassConst@@YAPEBVClass@@XZ
const Class * getClassConst() { return nullptr; }
// mangles as ?getClassConstConst@@YAQEBVClass@@XZ
const Class * const getClassConstConst() { return nullptr; }
extern(C++)
{
class Class
{
void _cppDtor() {}
}
// Mangles as ?getClass@@YAPEAVClass@@XZ
Class getClass() {return null;}
// Mangles as ?getClassConst@@YAQEBVClass@@XZ
const(Class) getClassConst() {return null;}
}
As you see from the above example D mangles the getClassConst as
a "Class const * const" instead of a "Class const *" ("YAQEBV" vs
"YAPEBV"). Is this expected behavior? The core problem is that D
can not express one of the two. Either const(Class) becomes
"Class const *" or "Class const * const". I've never seen C++
code that returns const pointers to const classes so I think the
default should be "Class const *". Either way its rather bad that
we can only represent one or the other. Sooner or later someone
will hit this problem again wanting the other option or both. Any
idea how we can avoid changing C++ source code in order to bind
it to D? Using pragma(mangle,) works but is kind of ugly.
Especially because the string to pragma(mangle,) can't be
genearted using CTFE.
Should I open a bug for this?
Re: DMD 64-bit Windows
On Wednesday, 9 November 2016 at 16:34:13 UTC, Mario Silva wrote: Hey everyone, While compiling our 64-bit app under Windows, DMD is getting out of memory. I wasn't able to find a windows version of DMD64, is there even one? If not, are there any plans? This is quite a show stopper for us. Thanks in advance, Mario If you have VisualD installed just check out the dlang/dmd repository on github using the last release branch and open the visual studio solution inside the "vcbuild" directory. Then build the x64 version of ddmd and use that. It is safe to rename the resulting binary from "ddmd.exe" to "dmd.exe". You can replace dmd.exe in your install with this newly build executable and everything will work.
Bug in std.allocator?
Please consider the following program:
import std.experimental.allocator.mallocator;
import std.experimental.allocator.building_blocks.allocator_list
: AllocatorList;
import std.experimental.allocator.building_blocks.free_list;
import std.experimental.allocator;
import std.stdio;
enum uint size = 1104;
alias ScalableFreeList = AllocatorList!((n) =>
ContiguousFreeList!(Mallocator, 0, unbounded)(size * 128,
size)
);
void main(string[] args)
{
void[][20] allocs;
ScalableFreeList allocator;
for(int i=0; i < 100; i++)
{
writefln("pass %d", i);
foreach(ref alloc; allocs)
{
alloc = allocator.allocate(size);
writefln("%x", alloc.ptr);
}
foreach(alloc; allocs)
{
allocator.deallocate(alloc);
}
}
}
I would assume that this program should run forever and never run
out of memory. But instead it triggers an assert inside
alocator_list in pass 11. So I assume this is some bug in
std.allocator?
Also whats interresting. The first allocation in each new pass is
_not_ the last allocation to be freed. Instead it seems to "leak"
one allocation each pass.
From the output of the program:
229a290fd60 <- same
229a2932570 <- leaked?
pass 11
229a290fd60 <- same
Or can anyone see a bug in my program?
Kind Regards
Benjamin Thaut
Re: rt_init, rt_term and _initCount
On Saturday, 22 October 2016 at 07:12:48 UTC, Rainer Schuetze wrote: Please also consider that the main executable might not know about DLLs being written in D. In that case all termination must be triggered by the druntime DLL. The case where the main executable is not D is working fine. The problem at the moment is the case where the main executable is D. So I will be going for the dll registration approach then. As you said we can add in the thread join in case it becomes a problem.
Re: rt_init, rt_term and _initCount
On Friday, 21 October 2016 at 19:40:52 UTC, Rainer Schuetze wrote: The wrapper around main in the executable should work just as any DLLs' DllMain, i.e. it should register/unregister its own data segments with the GC and run its shared and TLS module constructors/decoontructors. Everything else should be handled by the druntime DLL. Yes, I also considered that option. The problem with that is, that druntime deinit joins alls threads before running the shared module dtors. If the main executable behaves exactly like the dlls do this joining of threads would be lost and different from the behavior of a fully statical linked executable.
Re: rt_init, rt_term and _initCount
On Thursday, 20 October 2016 at 08:44:09 UTC, Ethan Watson wrote: On Thursday, 20 October 2016 at 07:17:49 UTC, Benjamin Thaut wrote: Any suggestions how to solve this problem? Who are other platforms doing it? Would this also be a bigger problem if people use LoadLibrary and don't FreeLibrary after? No. This case actually works correctly. No worries I have your use case covered ;-)
rt_init, rt_term and _initCount
This is a topic really specific to druntime, I don't know a better place to put it though. rt_init increases the _initCount and rt_term decreases it and only terminates the runtime in case the _initCount reaches zero (see dmain2.d) The problem now is as follows. Each dynamic library that is loaded (in this case a .dll on windows) must ensure that druntime is intialized. This must be done to ensure that d dynamic libraries work with C processes that load them. So lets assume we have a exe and two dlls: DllA and DllB. The exe uses the two dlls and druntime. What will happen is the following 1) Exe starts up 2) The windows loader will load DllA. This will call rt_init _initCount will be 1. Druntime will be initialized. Module ctors of druntime will be called. Module Ctors of DllA will be called. 3) The windows loader will load DllB. _initCount will be 2. Module Ctors of DllB will be called. 4) The dmain function will be execued. It will call rt_init. _initCount is 3. Module ctors of exe will be called. 5) Dmain finishes and calls rt_term. _initCount is 2. Nothing happens 6) The windows loader will unload DllB. rt_term is called. _initCount is 1. DllB is forced to call its module dtors. 7) The windows loader will unload DllA. r_term is called. _initCount is 0. Module Dtors of exe will be called. Module Dotrs of DllA will be called. Module Dtors of druntime will be called. As might observed the issue is the order in which the module dtors are called. The order is Druntime Ctor DllA Ctor DllB Ctor ExE Ctor DllB Dtor Exe Dtor DllA Dtor Druntime Dtor Whereas it should be: Druntime Ctor DllA Ctor DllB Ctor ExE Ctor Exe Dtor DllB Dtor DllA Dtor Druntime Dtor The problem is step 5) in the above list. When the main executable leaves dmain the runtime should be deinitialized no matter if any dll is still loaded or not. If this would be the case the module Dtors would be calle din the correct order. I can't remove the rt_init calls from the dll loading code however because that would mean when loading a d-dll into a C process druntime would never be initialized. So I'm thinking of adding a force parameter to rt_term which will always deinitialize druntime disregarding the _initCount. This feels like a hack though. Any suggestions how to solve this problem? Who are other platforms doing it? Kind Regards Benjamin Thaut
std.paralellism.Task value type problems
I would like to use std.paralellism.TaskPool to schedule various tasks I create. The problem however is that these tasks don't have a lifetime which is bound to any function scope I have. So I need to create a new task object on the heap and push it into a array for bookkeeping. The problem however is that std.paralellism.Task is a struct and the only way to instanciate it is to use std.paralelism.task which returns it as a value. I have no idea at the moment how to allocate a instance of std.paralellism.Task on the heap. Any suggestions? Kind Regards Benjamin Thaut
std.experimental.allocator and GC.addRange
Lets assume I have a allocator which cains together multiple building blocks from std.experimental.allocator and at the end there is a mallocator providing the underlying memory. Now I alloacte a type which contains a pointer into GC memory. Obviously the memory of the mallocator is not scanned by the GC so the pointer might become dangling if the GC chooses to collect. I'm wondering if std.experimental.allocator has any building blocks or mechanism to automatically call GC.addRange. Kind Regards Benjamin Thaut
Re: Pointer top 16 bits use
On Saturday, 7 May 2016 at 06:08:01 UTC, Nicholas Wilson wrote: In Dicebot's DConf talk he mentioned that it is possible to use the top 16 bits for tagging pointers and the associated risks. Regarding the GC not seeing a pointer if the top 16 bits are used, whats to stop us from changing the GC to (or adding an option to) ignore those bits when checking? I imagine that it would cause a lot more false pointers, but when we get precise GC that problem would go away. The problem with this is, that its not future proof. Once the top 16-bits of a pointer get used D programs will suddenly stop working. So i think we shouldn't support that officially. If you really want to do that you can simply hack the scan routine of the gc in druntime yourself and ship with the hacked version.
Re: Strange stack variable corruption error after calling extern(C) function
On Wednesday, 4 May 2016 at 17:53:32 UTC, cc wrote: The OS is Win64 though the program is being compiled as 32-bit and I'm using the 32-bit distributed DLL. fmod.dll: PE32 executable (DLL) (GUI) Intel 80386, for MS Windows Tried int and long as the return type, same issue both ways. Tried void too just in case, same thing though. Could you please post the definition of FMOD_RESULT. Its possible that the create sound function returns it on stack and not inside a register. This is usually the case if FMOD_RESULT is defined as a struct in C/C++. But its hard to say. In your case I would actually look at the disassembly and step through it to see where its going wrong and messing up the stack.
Re: Strange stack variable corruption error after calling extern(C) function
On Tuesday, 3 May 2016 at 19:06:30 UTC, cc wrote: it fails to link with "Error 42: Symbol Undefined _FMOD_System_CreateSound@20". With extern(C) it compiles and runs but the problem from above persists. Is this on Windows x64? Try replacing FMOD_RESULT by int. When declaring the fmod create sound function and see if that helps.
Re: Stacktraces in static constructors
On Wednesday, 4 May 2016 at 06:37:28 UTC, Nordlöw wrote:
On Tuesday, 3 May 2016 at 12:31:10 UTC, Benjamin Thaut wrote:
I assume this is on windows? Yes its a known issue (I know
No, the problem occurs on my Linux aswell.
From core.runtime:
static this()
{
// NOTE: Some module ctors will run before this handler is
set, so it's
// still possible the app could exit without a stack
trace. If
// this becomes an issue, the handler could be set in C
main
// before the module ctors are run.
Runtime.traceHandler =
}
So a possible workaround would be to either import core.runtime
in each of your modules or manually run the line
Runtime.traceHandler =
Also I find it strange that the trace handler is initialized in a
regular module constructor. In my eyes it should be a shared
module ctor because the storage behind Runtime.traceHandler is
__gshared anyway.
Kind Regards
Benjamin Thaut
Re: Strange stack variable corruption error after calling extern(C) function
On Tuesday, 3 May 2016 at 11:32:31 UTC, cc wrote: Hello, I've been encountering a strange problem that seems to occur after calling some external C functions. I've been working on a program that incorporates the FMOD C API for playing sound, with a simple D binding based off the C headers, and actually everything works more or less fine, I've had working sound in my program for a few months now. However I recently started noticing some strange behavior, currently using DMD v2.070.2 (haven't tried v2.071 yet, will soon). I can't post the entire program but I'll include the relevant code, I might try to make a small working compilable sample if this isn't enough information. [...] It seems that one of the fmod functions you declared is not correct. Either the fmod api is not using the c calling convention or you made a mistake when declaring the paramters of the fmod functions. You should double check that the functions match the fmod headers. Kind Regards Benjamin Thaut
Re: Stacktraces in static constructors
On Tuesday, 3 May 2016 at 10:52:20 UTC, Nordlöw wrote: On Tuesday, 3 May 2016 at 10:48:51 UTC, Nordlöw wrote: AFAICT, stacktraces are not emitted with debug information when Should be static shared module constructors. errors occur in static module constructors. Is this a know bug? My stacktraces contain no information of the call stack so it's very very tedious work to find the reason for a failing unittest. I assume this is on windows? Yes its a known issue (I know about it). I Don't know if its filed though. As a workaround you can import "core.sys.windows.stacktrace" into each of your modules. That will force the module system to initialize the stacktracing code before the module ctors. The underlying issue is that the module system does not know about the implicit dependeny of every module on the stacktracing module. Kind Regards Benjamin Thaut
Slides bevorehand
Hi, I'm wondering if it would be a good idea to put up my slides before the talk. And when. On the day of the talk? Today? Tomorrow (Start of the Conference)? I assume if you wanted to do this you would simply do a PR to the dconf.org site on GitHub? Kind Regards Benjamin Thaut
Re: Minecraft written in D - on Android
On Tuesday, 26 April 2016 at 08:42:21 UTC, Vadim Lopatin wrote: Demo of DlangUI Scene3D engine - Minecraft-like voxel rendering - is available for Android/ARM. Post screenshots please.
Re: Directions to Ibis Hotel in Berlin from Tegel Airport
On Monday, 25 April 2016 at 09:16:11 UTC, Iain Buclaw wrote: Germany is a cash-in-hand country. Credit cards are rejected in most places that I've tried. Yes, I highly recommend having cash on you. Also ensure that its not the big bank notes (e.g. 50€ 100€) most ticket machines only take 10€ and 20€ notes, some of them might be out of change and require you to put in the amount percisely using coins.
Re: DConf 2016 offical presentation template
On Wednesday, 20 April 2016 at 20:10:56 UTC, qznc wrote: On Wednesday, 20 April 2016 at 07:53:53 UTC, Benjamin Thaut wrote: Is there a official presentation template for Dconf 2016? If not it would be great if someone could create one. Many programmers (me included) are not good with picking colors and thus presentations usually don't look as good as they could. Is there a format the majority uses? Powerpoint? LibreOffice? Reveal.js? LaTeX-Beamer? Markdown-based? I personally use LaTeX-Beamer. Mostly because it looks pretty decent without fidelling around with it too much. (e.g. compared to powerpoint). Thanks for the tips.
DConf 2016 offical presentation template
Is there a official presentation template for Dconf 2016? If not it would be greate if someone could create one. Many programmers (me included) are not good with picking colors and thus presentations usually don't look as good as they could. Kind Regards Benjamin Thaut
Re: Debugging D DLL from C# app with C linkage for native Unity 5 plugin
On Tuesday, 29 March 2016 at 23:41:28 UTC, Thalamus wrote: dmd dllmain.d dll.def -w -wi -g -map -ofLogic.dll -m64 -debug -shared Anyone know what I should try next? Am I missing something simple? :) thanks! Thalamus You should be using "-gc" instead of "-g" when building 64-bit D programs that should be debugged with visual studio. Otherwise the visual studio debugger might get confused over some of the symbol names. (Because they contain '.')
Re: Shared static constructors from C# EXE
On Thursday, 25 February 2016 at 17:46:18 UTC, Thalamus wrote: On Thursday, 25 February 2016 at 16:05:37 UTC, Benjamin Thaut wrote: [...] Thanks Benjamin. When I went to whittle this down to its barest essentials, though, the repro is pretty simple. It involves LIBs, but not Dlls, and it doesn't require anything but a single D EXE. [...] The problem is that when you build with -lib the resulting library is optimized for linker stripping. E.g. everything that is not directly used will not be pulled into the final executable. If you want everything in the final exectuable you shouldn't be using a .lib file. Instead compile with -c resutling in a .obj file and link the resulting .obj file into your executable. Then everything will end up in the final executable even if its not directly used. Kind Regards Benjamin Thaut
Re: Shared static constructors from C# EXE
On Thursday, 25 February 2016 at 14:42:14 UTC, Thalamus wrote: your entry point. Hi Guillaume, Thanks for responding so quickly! I had found that wiki page before and I'd been following the "DLLs with a C Interface" section closely. I had forgotten to add -shared when building the DLL, but the behavior didn't change when I added it. So, I added a call to Runtime.initialize() as the first line of the endpoint I'm exposing. (I also made sure that this was the only endpoint invoked and that it was only invoked once just to be cautious.) I can see Runtime.initialize() being called, but the Class A shared static constructor still is not called when run from the C# EXE. Do you have any other ideas? In the meantime, I'm working on putting together a minimal repro source, but the scenario is a bit complicated so there's a lot of details to whittle away. thanks! Gene You shouldn't be calling Runtime.initialize() manually. Just do the following in one of your source files: import core.sys.windows.dll; mixin SimpleDllMain; This will generate a DllMain that will correctly initialize and deinitialize druntime. Kind Regards Benjamin Thaut
Re: How is the TypeInfo assigned?
On Saturday, 13 February 2016 at 12:44:40 UTC, Tofu Ninja wrote: Is the TypeInfo given by typeid() guaranteed to be the same for a type regardless of where I call it? I guess my question is, is the TypeInfo a valid way to dynamically check types? I am implementing a message passing system for an entity-component system, I was planning to use structs as messages and use the TypeInfo to determine where they are supposed to go. Will this cause any problems? Also will typeid() allocate? Or is the TypeInfo statically allocated at compile time? Also how does shared libs affect this? Will a TypeInfo passed across different libs still match up? Thanks Yes, type infos are garantueed to have identity. You can use their adress to identify a type. The linux shared library mechanism ensures that they have the same adress across multiple shared libraries (if all of these libraries are loaded into the same process) With windows dlls there can be cases where a typeinfo is duplicated. But as dlls don't really work yet, I wouldn't worry about it.
Re: Octree implementation?
On Monday, 1 February 2016 at 02:56:06 UTC, Tofu Ninja wrote: Just out of curiosity, does anyone have an octree implementation for D laying around? Just looking to save some time. https://github.com/Ingrater/thBase/blob/master/src/thBase/container/octree.d Its a loose octree implementation. That means the cells overlap a bit to accomondate for the problem of objects that are on the border between to cells. I don't know though if you can rip out the implementation without some modifications. Kind Regards Benjamin Thaut
Re: `alias this` pointers and typeof(null)
On Friday, 29 January 2016 at 13:38:20 UTC, Tomer Filiba wrote: I can change all such invocations into ``func(FooPtr(null))`` but it's tedious and basically requires me to compile tens of times before I'd cover everything. Is there some workaround to make null implicitly convertible to my alias-this type? I mean, it's Foo* would accept `typeof(null)` so why can't FooPtr with an alias-this to Foo* do so too? -tomer Unfortunately D is strictly against implict conversion, so there is no way to do this. I also hit this issue a couple of times already. But implicitly calling a constructor in C++ is considered error prone and therefor not supported in D. Kind Regards Benjamin Thaut
Re: Assert failure on 2.070.0 without stack trace
On Thursday, 28 January 2016 at 18:33:19 UTC, Nordlöw wrote: Thanks, I'm aware of these tools. But it's easier to use the stacktrace...if I only get one. The function where the assert() is called is, in turn, called in hundreds of places. Which platform are you on? Are all your binaries compiled with debug symbols? If one or multiple functions on the stack are within phobos or druntime you might not have debug symbols for phobos or druntime. Using inline asm might also destroy your stack frames.
Re: Fun with extern(C++)
On Tuesday, 26 January 2016 at 16:13:55 UTC, Manu wrote: Probably, but the layout of the vtable is defined by the interface, and the interface type is always known, so I don't see why there should be any problem. Whether it's extern(C++) or extern(D), the class populating the vtable with functions knows the layout. I think it all comes down to this conversion to Object thing. If an interface must do that, then that's probably an issue without jamming an Object instance in the class somewhere. For a C++ class the first entry in the vtable is actually the first virtual function. (usually the destructor). For a D class the first entry in the vtable is the classinfo. Thus the problem if you derive a D class from a extern(C++) base class. I don't see any way to actually fix this, adjusting the this pointer won't help. Once you derive a D class from a extern(C++) base class it is no longer a fully functional D class. For example monitor (e.g. synchronized methods) won't work.
Re: C++17
On Tuesday, 26 January 2016 at 15:51:22 UTC, deadalnix wrote: Now, D can do the exact same as C++ . That is a lie. Large parts of druntime still allocate GC memory and thus will cause the GC to run. For example core.thread. We need a solution where druntime can be used without druntime ever doing a single allocation through the gc.
Re: D Dll's usefulness
On Monday, 25 January 2016 at 19:45:21 UTC, Igor wrote: Am I off target here? Dlls are currently not properly supported in D, I would strongly advice against using them. Just link everything statically and be happy for now. Kind Regards Benjamin Thaut
Re: Last revision of phobos and druntime that actually compile with cdmd
On Monday, 25 January 2016 at 03:05:36 UTC, Daniel Murphy wrote: On 25/01/2016 2:55 AM, Benjamin Thaut wrote: I tried using a commit which was the same date as the cdmd -> ddmd switch but that didn't work. That's the only way I know to do it, it should work. It seems that I was just a few revisions off. _xopEquals has been moved from module.c to object.d. Everything is working now. A git tag still would have been nice.
Last revision of phobos and druntime that actually compile with cdmd
I'm currently merging a pretty large change over the C++->D boundary within Dmd. This change includes many changes to druntime and phobos. So I need the last revision of druntime and phobos that still compile with cdmd exactly before the switch was made. Unfortunately there is no git tag for this in druntime and phobos. I tried using a commit which was the same date as the cdmd -> ddmd switch but that didn't work. Kind Regards Benjamin Thaut
Re: Shared library question
On Saturday, 23 January 2016 at 00:38:45 UTC, Dibyendu Majumdar wrote: On Friday, 22 January 2016 at 22:06:35 UTC, Dibyendu Majumdar wrote: Hi I am trying to create a simple shared library that exports a D function, but when I try to link to it I get errors such as: error LNK2001: unresolved external symbol _D7xxx12__ModuleInfoZ I have uploaded my small test here so if anyone can tell me what I am doing wrong it will hugely appreciated. Thanks! https://github.com/dibyendumajumdar/d-lab/tree/master/sharedlib Dll support on windows is basically non existant at the moment and I strongly advice against trying to use it. Trust me I've been there. I'm currently working on propper Dll support but that is most likely going to take a few more months. Your best option currently is to simply link everything statically. Kind Regards Benjamin
Re: cast fails for classes from windows dll
On Tuesday, 12 January 2016 at 19:00:26 UTC, Andre wrote:
Hi,
I am not sure, whether this is a current limitation of the
windows dll functionality of D
or I am doing s.th. which will not work.
I have developed in D a windows DLL which creates class
instances by passing the name (using object.factory method).
In another D application I am using this DLL. My issue is, that
the cast fails, although
typeid(bar).name shows the correct name .
module main;
// these classes are in a seperate module
// used for the dll & for this application
export class Foo {}
export class Bar : Foo {}
class Baz : Bar {}
void main()
{
// this method calls the dll and returns Foo
Foo c = dllCreateClass("main.Baz");
// no failure
assert( typeid(c).name == "main.Baz");
// > fails
if (auto myBar = cast(Bar) c){}
}
Kind regards
André
Thats a limitation of the current dll functionality. The type
info of the class gets duplciated into both your executable and
the dll and thus the cast fails. Until D properly supports Dlls
on windows this is going to stay this way. Currently only a C
like interface across dll boundaries is possible.
Re: cairo(D) / x64 / unresolved externals / don't know why
Am 11.01.2016 um 18:27 schrieb Robert M. Münch: Import symbols are symbols used for dll linking and start with "__imp_" Is this a DMD convention or a general one? Never heard about this. This seems to be a general convetion on windows. All c++ compilers I've seen on windows so far emit so called import symbols using the "__imp_" prefix. Basically its just another level of indirection during a function call to be able to resolve function addresses at runtime, e.g. implement dynamic linking. Its kind of a implementation detail, so in theory you don't have to know about it. But it helps when reading linker errors. In case my talk gets accepted for dconf 2016 I'm going to cover this there in more detail. -- Kind Regards Benjamin Thaut
Re: cairo(D) / x64 / unresolved externals / don't know why
On Sunday, 10 January 2016 at 22:22:03 UTC, Robert M. Münch wrote: I made to compile a bunch of libs on Win64 and got my D project compiled as well. Only problem left are some strange unresolved externals. Linking... dmd -of.dub\build\application-debug-windows-x86_64-dmd_2069-F0A1450B9B033D5CD11F3F60481557B0\webchat.exe .dub\build\application-debug-windows-x86_64-dmd_2069-F0A1450B9B033D5CD11F3F60481557B0\webchat.obj ..\vibe-d-0.7.26\lib\win-amd64\libeay32.lib ..\vibe-d-0.7.26\lib\win-amd64\ssleay32.lib ..\cairoD\cairoD.lib C:\Users\robby\AppData\Roaming\dub\packages\derelict-ft-1.0.2\lib\DerelictFT.lib C:\Users\robby\AppData\Roaming\dub\packages\derelict-util-2.0.4\lib\DerelictUtil.lib C:\Users\robby\AppData\Roaming\dub\packages\x11-1.0.9\x11.lib ..\vibe-d-0.7.26\vibe-d.lib wsock32.lib ws2_32.lib advapi32.lib user32.lib -LD:\develop\cairo\cairo\src\release\cairo-static.lib -LD:\develop\cairo\libpng\libpng.lib -Lgdi32.lib -m64 -m64 -m64 -m64 -m64 -m64 -g cairo-static.lib(cairo-image-source.obj) : error LNK2001: unresolved external "__imp__hypot" and some more missing in other .obj files: "__imp__ctime64" "__imp_ldiv" "__imp_strncpy" "__imp_rand" IMO this all looks like msvcrt standard lib stuff so wondering why it's not found. Do I explicitly have to link to the msvcrt lib? You should not need to link manually against msvcrt, dmd does this for you. You can view the linker commands that are stored inside a object file via microsoft dumpbin tool "dumpbin /DIRECTIVES your.obj". You should check the declarations of the functions that cause a unresolved external error. If they have a "export" in front of them, remove the export. I can not think of any other reason why dmd would otherwise reference a import symbol. Import symbols are symbols used for dll linking and start with "__imp_" Out of curiosity, why do you pass "-m64" 6 times to dmd? Once would be enough. For debug builds targeting windows 64 I would also highly recommend using "-gc -op" instead of "-g". This will give a much better debugging experience in Visual Studio.
Re: Linking a DLL to a DLL with packages
Am 09.01.2016 um 16:45 schrieb Thalamus: Hi Benjamin, I wouldn't say I need DLLs to work fully _really_ badly. The only non-negligible issue with single very large binaries that's crossed my mind is patching, but we're years away from having to worry about that too much. That being said, I'm definitely willing to do some testing, especially for something that helps us down the road. I'll follow up with you offline. Thanks! Great, some help with bugfixing and testing would be greatly apreciated. I didn't get any e-mail from you yet, I assume you didn't send one? Kind Regards Benjamin Thaut
Re: Linking a DLL to a DLL with packages
On Thursday, 7 January 2016 at 19:29:43 UTC, Thalamus wrote: Hi everyone, First off, I've been working with D for a couple of weeks now and I think it's the bee's knees! :) Except for DLLs. thanks! :) Dlls don't currently work on Windows. The only thing that works is giving your dlls a C-like interface. If you need any kind of D interface (classes, modules, etc) it won't work. I'm currently working on this, if you need it really badly and are willing to help bug testing send me a mail to code at benjamin-thaut.de Kind Regards Benjamin Thaut
Re: Stripping Data Symbols (Win64)
On Friday, 1 January 2016 at 13:57:01 UTC, Rainer Schuetze wrote: Please note that building with -lib puts every function/declaration into it's own object file inside the library, and unused class declarations are no longer in the linked executable. Ok, that is very good information. I should be able to build a test case out of that. Kind Regards Benjamin Thaut
Re: extern(C++, ns)
On Monday, 4 January 2016 at 15:02:05 UTC, Manu wrote: Yeah, I've used this process before. Last time I reported a raft of LDC bugs I spent a few days doing this... but it's very laborious, and I don't have time to do it. I'm doing this work on borrowed time as it is. Did you try dustmite? I usually set it up and let it run for a night on larger projects and it managed to reduce the bug every time so far. I need to feel productive and like I'm making progress, otherwise it fails to compete for timeshare with other high priority goals >_< Thats the problem with D at the moment, if it works its great, but if it doesn't, especially if you are on untested terrain like extern(C++) you loose tons of time.
Re: Stripping Data Symbols (Win64)
On Wednesday, 30 December 2015 at 09:43:32 UTC, Rainer Schuetze wrote: I noticed something similar recently when compiling a C file with /Gy, see https://github.com/D-Programming-Language/druntime/pull/1446#issuecomment-160880021 The compiler puts all functions into COMDATs, but they are all still linked in if only a single symbol is referenced, even if linked with /OPT:REF. So I suspect this is not an issue with dmd, but the Microsoft linker. I still wonder whether the approach to use "function level linking" works at all for Win64. > I noticed by looking at some object file dumps that dmd puts each > function into its own section, but data symbols, like initializers, are > all merged into the same section. Could this be the root issue? Having all data in a single section misses some possible optimizations, and it might be the reason for the behavior in your case (you can check this with "dumpbin /all objectfile"), but the issue above does not contain any data. So if I understand this correctly the microsoft linker only strips unused comdats, otherwise always the entire object file gets pulled in? For me stripping of individual data symbols not working is actually a good thing, if it doesn't work, I can't break it. ;-)
Stripping Data Symbols (Win64)
My current work on the D compiler lead me to the following test
case which I put through a unmodified version of dmd 2.069.2
import core.stdc.stdio;
struct UnusedStruct
{
int i = 3;
float f = 4.0f;
};
class UnusedClass
{
int i = 2;
float f = 5.0f;
};
void main(string[] args)
{
printf("Hello World!");
}
When compiling this on windows with dmd -m64 main.d -L/MAP
and then inspecting the map file I noticed that the following 4
data symbols end up in the final executable although they
shouldn't be used.
0003:0a90 _D4main12UnusedStruct6__initZ
000140046a90 main.obj
0003:0ad0 _D4main11UnusedClass6__initZ
000140046ad0 main.obj
0003:0af0 _D4main11UnusedClass7__ClassZ
000140046af0 main.obj
0003:0ba0 _D4main11UnusedClass6__vtblZ
000140046ba0 main.obj
For the struct this is the initializer, for the class its the
initializer, class info and vtbl.
Is this behavior correct? Shouldn't UnusedStruct and UnusedClass
be stripped completely from the binary? Is this somehow connected
to the module info / object.factory?
I noticed by looking at some object file dumps that dmd puts each
function into its own section, but data symbols, like
initializers, are all merged into the same section. Could this be
the root issue?
Re: Why my app require MSVCR120.dll?
On Friday, 6 November 2015 at 16:21:35 UTC, Suliman wrote: On Friday, 6 November 2015 at 13:50:56 UTC, Kagamin wrote: MSVCR is a C runtime. On Linux it will depend on a C runtime too. But which part of my App depend on C runtime? All of it. Phobos and druntime use the C runtime, that means the language itself depends on the C runtime.
Re: Synchronized classes have no public members
On Tuesday, 13 October 2015 at 07:17:20 UTC, Jonathan M Davis wrote: Ultimately, I think that we're better off with TDPL's definition of synchronized classes than the synchronized functions that we have now, so I do think that the change should be made. However, I also think that synchronized classes as TDPL describes are limited enough to be of questionable usefulness. Stripping off the outer layer of shared is unlikely to be sufficient in all but basic cases (and synchronized classes can't do any better than that, I don't think), meaning that you're likely going to have to cast away shared to do much with shared anyway, in which case, having a synchronized class loses at least some of its value. It can still encapsulate shared (which is good), but it doesn't necessarily make it much easier or safer to use. - Jonathan M Davis I have to agree here. I think synchronized classes are of very little use, especially because they don't "cast away" shared in a useful way. It still has to be done manually. I think we should remove them. Synchronized methods should also be removed in my eyes. Making each and every object bigger by one pointer just for the sake of a few synchronized methods doesn't seem to be a good trade off to me. The entire synchronized methods give the user the feeling that he simply slaps synchronized on his class / method and then its thread safe and he doesn't have to care about threads anymore. In the real world this is far from true however. So synchronized methods and classes just give a false sense of thread safety and should rather be removed.
Re: Synchronized classes have no public members
On Tuesday, 13 October 2015 at 12:20:17 UTC, Minas Mina wrote:
I agree that synchronized classes / functions that not that
useful.
But synchronized statements, to me, make the intention of
locking explicit.
Synchronized statements are fine and serve a good purpose, no
need to delete them in my opinion.
Maybe the internal monitor could be removed (with synchronized
classes / functions as well), and allow synchronized() {} to be
called on Lock objects, that essentially locks them at the
beginning and unlocks them at the end.
Yes, I would love that.
Re: __simd_sto confusion
On Saturday, 3 October 2015 at 14:47:02 UTC, Nachtraaf wrote:
I'm trying to create some linear algebra functions using simd
intrinsics. I watched the dconf 2013 presentation by Manu Evans
but i'm still confused about some aspects and the following
piece of code doesn't work. I'm trying to copy the result of a
dot product from the register to memory but dmd fails with an
overload resolution error, which i guess is due some implicit
conversion?
dmd error:
simd1.d(34): Error: core.simd.__simd_sto called with argument
types (XMM, float, __vector(float[4])) matches both:
/usr/include/dlang/dmd/core/simd.d(434):
core.simd.__simd_sto(XMM opcode, double op1, __vector(void[16])
op2)
and:
/usr/include/dlang/dmd/core/simd.d(435):
core.simd.__simd_sto(XMM opcode, float op1, __vector(void[16])
op2)
from the following piece of code:
float dot_simd1(float4 a, float4 b)
{
float4 result = __simd(XMM.DPPS, a, b, 0xFF);
float value;
__simd_sto(XMM.STOSS, value, result);
return value;
}
What am I doing wrong here?
core.simd is horribly broken. I recommend that you avoid it for
any serious work. If you want to do simd programming with D get
LDC or GDC and use their simd intrinsics instead of core.simd.
If you have to do simd with dmd write inline assembly.
Re: Status of Win32 C++ interop
On Tuesday, 8 September 2015 at 12:56:00 UTC, Laeeth Isharc wrote: This is really very clear and helpful, and I appreciate your taking the time. I will place it on the wiki if that's okay. Thats ok. Library support is surely one of the largest impediments to the adoption of D, and we ought to place some emphasis on updating the documentation here: http://dlang.org/cpp_interface.html How does it work when external APIs expect objects from the C++ standard library? strings, and so on? How about funny pointer types? shared_ptr etc? std::vector, std::list? For templated types like std::vector, std::list and shared_ptr you have two options: - Redo the complete implementation on the D side ensuring that the data layout is the same - Or, expose helper functions which call placement new / desturctor of your needed std::vector and do the hidden data trick I described above. The regular functions of std::vector just have to be declared and the linker will find them if and only if the c++ side instanciated the template. Here is one C++ library used by many in finance (at least as a backup). I think there might be a decent amount of value in making this usable from D. (Trying to put my own interests aside!) Paging Andy Smith ? http://quantlib.org/reference/_bermudan_swaption_8cpp-example.html I think you have to make this usable from D yourself ;-) Are there any well-known C++ libraries that you have interfaced to ? Could you give some examples of how long it takes ? I'm interfacing to a 3D engine I'm working on in my spare time with some collegues. So no well known C++ library. Would you be able to drop me an email about something else? No contact info on your blog, but my domain is kaleidicassociates.com and my user id is laeeth@ Will do. Kind Regards Benjamin Thaut
Re: Status of Win32 C++ interop
On Monday, 7 September 2015 at 19:30:44 UTC, drug wrote: 07.09.2015 21:37, Benjamin Thaut пишет: snip So far I haven't found a situation where I couldn't make it work the way I wanted. Its just some work to write the D headers for the C++ classes and vise versa, because you have to duplicate everything once more. An automated tool for this would be nice, but unfotunately there is currently none. Kind Regards Benjamin Thaut It's great. But isn't it based on your custom modifications of compiler and the others? No, the compiler modifications are only for the Windows DLL support. All modifications I did to the c++ binding are already in 2.068
Re: Status of Win32 C++ interop
On Friday, 4 September 2015 at 16:19:49 UTC, Laeeth Isharc wrote:
Hi Benjamin
Would you be able to give a little more colour on what the
limits are of interoperability for C++ with DMD master or
release ? As I understand it destructors and constructors
don't work, and obviously it will get tricky passing structures
and classes from C++ libraries not ported to D.
So, things that work really well are classes. Given that you
construct them in the their "native environment". E.g. create D
objects in D land and C++ objects in C++ land. I usually use
factory methods for this. Manually destroying objects also works
if you add some kind of "Destory" method (can be virtual, but
doesn't have to be) to your class which simply does a "delete
this" or similar. The only problems with interfacing C++ classes
to D is if they have a virtual destructor. But there is a easy
workaround like so:
C++:
class SomeBaseClass
{
public:
virtual ~SomeBaseClass();
virtual void SomeVirtualFunc();
}
D:
extern(C++) class SomeBaseClass
{
protected:
abstract void __cppDtor(); // don't call
public:
void SomeVirtualFunc(); // default virtual in D
}
Free functions, static functions, non virtual functions all work
flawlessly.
I recommend that you create your own pointer type on the C++
side, e.g. DPtr which calls the GC.addRoot / GC.removeRoot
methods of the D's gc interface in the apropriate places in case
you want to pass D objects into C++ space. If you have a
reference counting concept on the c++ side its also possible to
build a small proxy object which does reference counting in c++
and calls GC.removeRoot when the last reference count drops.
If you want to bind more complex types, e.g. c++ templates you
can either declare them extern(C++) in D and the compiler will do
the correct mangling, or what I did to make them fully functional
on both sides: I did a full implementation both in D and C++. The
implementation ensures that the data layout is exactly the same
for D and C++ but other than that the implementation is
duplicated on both sides and even different in some cases. It
still possible to pass this complex type from C++ to D because
the data layout is the same. I did this for a custom hash map
implementation and passing it around works flawlessly (by
reference)
I had another complicated case where I wanted a complex value
type that is implemented in C++ on the D side. It had a lot of
members which types don't have a equivalent in D, so I did the
following.
extern(C++)
{
void constructComplexStruct(ComplexStruct* self);
void destructComplexStruct(ComplexStruct* self);
}
struct DefaultCtor {}; // helper type
alias defaultCtor = DefaultCtor();
struct ComplexStruct
{
@disable this();
@disable this(this);
this(DefaultCtor)
{
constructComplexStruct();
}
~this()
{
destructComplexStruct(~this);
}
extern(C++) void SomeMethodOfComplexStruct();
private:
// must be sizeof(ComplexStruct) from c++, use unittest to
ensure!
void[288] m_data;
}
constructComplexStruct and destructComplexStruct just do a
placement new / call the C++ dtor.
Usage then works like this:
void someDFunc()
{
ComplexStruct myStruct(defaultCtor); // constructed using C++
default ctor
myStruct.SomeMethodOfComplexStruct(); // call C++ implemented
method
// destructed using C++ dtor
}
So far I haven't found a situation where I couldn't make it work
the way I wanted. Its just some work to write the D headers for
the C++ classes and vise versa, because you have to duplicate
everything once more. An automated tool for this would be nice,
but unfotunately there is currently none.
Kind Regards
Benjamin Thaut
Re: Status of Win32 C++ interop
On Friday, 4 September 2015 at 08:53:27 UTC, Szymon Gatner wrote: Hi, what is the current status of: - Win x86/32bit/coff32 interop with C++? - improvements for general C++ interop that were suppose to come with 2.068 If you use either the -m64 or -mscoff32 interop should be pretty good. I'm currently using it heavily and its working quite nicely.
Re: Status of Win32 C++ interop
On Friday, 4 September 2015 at 09:07:39 UTC, Szymon Gatner wrote: What about 32bit phobos? Last time I checked (2.067) only x64 was distributed. You have to compile it yourself. Use the win64 makefile and replace the arch=64 with arch=32mscoff. For more details see here: http://www.digitalmars.com/d/archives/digitalmars/D/learn/How_to_get_32mscoff_libraries_for_phobos_73980.html
Re: Status of Win32 C++ interop
On Friday, 4 September 2015 at 10:04:48 UTC, Szymon Gatner wrote: On Friday, 4 September 2015 at 09:27:14 UTC, Benjamin Thaut wrote: On Friday, 4 September 2015 at 09:07:39 UTC, Szymon Gatner wrote: What about 32bit phobos? Last time I checked (2.067) only x64 was distributed. You have to compile it yourself. Use the win64 makefile and replace the arch=64 with arch=32mscoff. For more details see here: http://www.digitalmars.com/d/archives/digitalmars/D/learn/How_to_get_32mscoff_libraries_for_phobos_73980.html Ah so that didn't change. Did you try it? Do you run hybrid C++/D apps on Win (whether 32 or 64)?. Asking because last time I tried it (Win x64 tho) writeln() call from D side crashed app because stdio wasn't properly initialized even tho rt_init() was successful. I am running hybrid D/C++ apps. I found it to work best when you give D the control over the main method, e.g. the program entry point should be in D land. Then simply call a C++ function from there to initialize your c++ stuff. I'm currently even running hybrid C++/D apps with dlls. E.g. multiple D dlls + multiple C++ dlls loaded by a c++ main program. But to do that I have heavy compiler + runtime modifications which are not ready yet to do a PR for them.
Re: Possible solution for export : `unittest export`?
On Sunday, 30 August 2015 at 14:08:15 UTC, Dicebot wrote:
Follow-up to old
http://forum.dlang.org/thread/m9lhc3$1r1v$1...@digitalmars.com
thread by Benjamin
Short reminder of the issue:
Currently unsolved issue with finishing `export` implementation
is lack of convenient semantics for its interaction with
templates. If template function is marked as `export`, its
non-template dependencies (called functions) will also need to
be marked as effectively export (even if private) to be put in
object file and avoid linker errors.
Which is impossible to do automatically because dependencies
can't be figured out without instantiaton. And to do it
manually you'd need to mark all dependencies with `export` too
which is impossible without making them also public because
currently `export` is defined as protection attribute. One of
Benjamin proposals was to split it as a separate attribute kind
but doing all manual annotation would still be hardly
convenient / maintainable.
Proposal essentials:
Define `unittest export { /* tests here */ }` which will verify
that all directly used symbols from same module/package are
marked as export and automatically mark dependencies for
placement into object files while doing semantic phase for
tested instances.
Rationale:
- fits existing "documented unittest" feature by providing
verified example of using the API
- easier change to grammar than re-defining export itself
- reasonably simple maintenance (no need to hunt each small
dependency function after internal changes, risking linker
errors if sloppy)
- if https://issues.dlang.org/show_bug.cgi?id=14825 is ever
fixed, combining this with -cov will ensure reasonable
confidence in proper API annotation
Cons:
- implies test author to be smart enough to do all necessary
instantiations (will become less of an issue with
https://issues.dlang.org/show_bug.cgi?id=14825)
- may look like a hack for those coming from more restrictive
languages
While your proposal sounds interresting to start with I don't
like some of the implications:
1) You force people to write unittest. If people don't write a
export unittest block their templates won't work across shared
library boundaries.
2) A template in D might get __very__ complex. To make sure that
each and every function needed is actually exported your
unittests would need to have 100% coverage. Looking at some of
the template code in phobos this won't be any more fun than
manually putting export in front of every required function.
3) Its going to be hard to implement. You basically need to touch
all the template instanciation code and make sure it tells you
which functions have been used after its done. Thats going to be
quite invasive.
4) Martins favorite argument. When doing C APIs you usually
ensure that you public interface (e.g. whats exported form a
shared library) stays the same between minor versions. This
automatic export thing is going to make this a lot harder for D.
Re: Possible solution for export : `unittest export`?
On Sunday, 30 August 2015 at 14:08:15 UTC, Dicebot wrote: Follow-up to old http://forum.dlang.org/thread/m9lhc3$1r1v$1...@digitalmars.com thread by Benjamin Oh, don't get me wrong. Its great that you also think about this problem. Currently you seem to be the only other person here on the NG which is actually interested to solve the problems around export, other people seam to simply ignore them or want to abuse export for other language issues. Keep up the good work ;-) Kind Regards Benjamin Thaut
Sub forum for dmd implementation details
As a occasional contributor to dmd I usually manage to get everything working the way I want by digging through the dmd source code long enough, but sometimes bad / non-existing documentation and missing knowdelge result in suboptimal solutions or unsolved problems. I would greatly appreciate a subforum which purpose it is to discuss compiler implementation details. I know that there is dmd-internals, but it seems that this forum only gets spammed by some bot with pull-request messages. Kind Regards Benjamin Thaut
Re: Object.factory() and exe file size bloat
On Saturday, 22 August 2015 at 20:22:58 UTC, David Nadlinger wrote: On Saturday, 22 August 2015 at 20:14:59 UTC, Walter Bright wrote: I'm not sure how export would help on Linux. One of the use cases for export on Linux would be to set the ELF visibility based on it. Emitting all the symbols with default visibility, like we currently do, leads to size and load time problems with large libraries. Big C++ projects are plagued regularly by this (cf. -fvisibility=hidden). — David The good news is, once I'm done with my windows DLL work the code can be trivialy reused to make export control the visibility of symbols on linux as well. Kind Regards Benjamin Thaut
Re: Object.factory() and exe file size bloat
On Saturday, 22 August 2015 at 09:44:48 UTC, Martin Nowak wrote: The export seems to be an arbitrary rule (and export is really broken currently). Let's just use every class that is linked into the binary (e.g. weakly referencing them), then it'll naturally work with all linker functionalities. This doesn't only affect Object.factory but also ModuleInfo.localClasses. I'd suggest we first add a new internal array of weakly linked classes, turn localClasses into an opApply function or range so it automatically skips null classes (weakly undefined), then change Object.factory to only load weakly linked classes. For an intermediate time we can keep the old array and print a deprecation warning in Object.factory when a class would no longer be available. https://github.com/D-Programming-Language/dmd/pull/4638 How do you implement weak linking? It would be really usefull for my DLL work as well, but I couldn't find any way to make it work with the microsoft linker.
Re: Object.factory() and exe file size bloat
On Sunday, 23 August 2015 at 13:09:46 UTC, Benjamin Thaut wrote:
The good news is, once I'm done with my windows DLL work the
code can be trivialy reused to make export control the
visibility of symbols on linux as well.
Kind Regards
Benjamin Thaut
But then you have the same problem on linux as on windows.
Export controls not only the symbol visibility across shared
library boundaries but also the module level visibility. E.g.
this is a problem
private void SomeImplementationDetail() { ... }
void SomeTemplateFunc(T)()
{
SomeImplementationDetail();
}
If you compile this into a shared library and all symbols are
hidden by default unless marked with export it will fail to
compile if someone tries to use it. Because the instanciated
template SomeTemplateFunc will call the SomeImplementationDetail
function which is not visible across the shared library boundary.
To fix this you would have to do:
export void SomeImplementationDetail() { ... }
void SomeTemplateFunc(T)()
{
SomeImplementationDetail();
}
But this means that users of the shared library would suddenly be
allowed to call SomeImplementationDetail.
The fix would be to make export an attribute instead of an
protection level resulting in:
private export void SomeImplementationDetail() { ... }
void SomeTemplateFunc(T)()
{
SomeImplementationDetail();
}
Re: Object.factory() and exe file size bloat
On Friday, 21 August 2015 at 05:06:47 UTC, Walter Bright wrote: This function: http://dlang.org/phobos/object.html#.Object.factory enables a program to instantiate any class defined in the program. To make it work, though, every class in the program has to have a TypeInfo generated for it. This leads to bloat: https://issues.dlang.org/show_bug.cgi?id=14758 and sometimes the bloat can be overwhelming. The solution seems straightforward - only have Object.factory be able to instantiate classes marked as 'export'. This only makes sense anyway. What do you think? I don't think this is a good idea. That's just abusing a already existing keyword. Export basically means Make this function or class visible across shared library boundaries. I don't see how this connects to the object factory. Maybe I want to instantiate a class via the factory that should not be visible across a shared library boundary (e.g. to keep the interface of a shared library minimal). Please also consider that as export means dllexport and dllimport at the same time (on Windows) classes marked with export will inflict additional runtime overhead due to the double indirection needed for dlls and due to the fact that you sometimes don't know if the class resides in a different binary then the one your are currently in. This runtime overhead even occurs for purely static builds, as the compiler can't know if the generated code uses some shared library or not. (unless we add a specific compiler switch for it, like -onlyStatic which many people will not be a fan of. And no the -shared flag doesn't help as executables are build without -shared and might still use shared libraries) Given how limited object.factory is I would just vote that we kill it completely. Every time I wanted to use it, it was to limited and I ended up building my own reflection / factory mechanism. Even though you rejected my proposal for making export a attribute instead of a visibility level, I think that once a broader set of contributors sees the issue behind export being a protection level the demand will be high to make export an attribute. Giving export a additional meaning now will only complicate this. Tldr: As broken as export currently is we shouldn't be using it for anything else. Kind Regards Benjamin Thaut
Re: Object.factory() and exe file size bloat
On Friday, 21 August 2015 at 08:49:37 UTC, Jacob Carlborg wrote:
How is it limiting? That it only works with default
constructors? I don't think that underlying ClassInfo.find is
limiting, which is where the interesting part happens. I'm
pretty sure that Object.factory could be extended to support
non-default constructors, if that makes it less limiting.
Yes, the usual problem was that it only works with default
constructors. Also it doesn't work with nested classes e.g.
class Outer
{
class Inner
{
}
}
I don't know if that is fixed now.
Re: Safely extend the size of a malloced memory block after realloc
On Wednesday, 19 August 2015 at 14:45:31 UTC, Steven Schveighoffer wrote: In the case where the pointer changes, you are in trouble. The original memory is now free, which means it can be overwritten by something else (either the C heap or some other thread that reallocates it). Then if your GC runs *before* you have added the new memory, it may collect the now-no-longer-referred-to data. It's no different than your original situation. I actually think the case where the pointer changes is worse. Yes I made the same observation in the meantime. Also, I note that others have said one can call GC.collect from another thread, which is true. One could call GC.enable as well. If you have concerns of this happening (i.e. you don't control all the code, and think your code may coexist with something that calls GC.collect), the likely correct mechanism is to take the GC global lock while doing your operation. I'm not sure if you can do that via the current API, you may have to add such a feature. Yes I figured as much. The entire purpose of this thraed was to point out that you can not safely forward a realloc to the GC. Unfortunately its not a option not to use realloc as I'm binding some code I don't have control over. To summarize the entire issue and a possible solutions I created the following ticket: https://issues.dlang.org/show_bug.cgi?id=14934 Kind Regards Benjamin Thaut
Re: Safely extend the size of a malloced memory block after realloc
On Tuesday, 18 August 2015 at 10:27:14 UTC, Casper Færgemand wrote: On Monday, 17 August 2015 at 19:38:21 UTC, Steven Schveighoffer wrote: // if the GC kicks in here we're f* GC.addRange(mem, 512); Can't you GC.disable around this whole thing? GC.collect can still be called from another thread. Good point, also GC.disable doesn't guarantee that the GC will never run. The documentation says that the GC may still run in out of memory or similar situations where it absolutely needs to run. So there isn't a safe way to do this after all.
Re: Safely extend the size of a malloced memory block after realloc
On Monday, 17 August 2015 at 20:33:45 UTC, welkam wrote: I might be wrong, but he should worry about GC before he removes that memory range from GC managed list not after. And his code smells to me. He gives full memory control to GC, but then wants to take it away, fiddle and give it back. I would allocate more than I need to so avoiding a need to extend the memory. If not then either allocate new chunk of memory, copy data and forget about old one or have a data structure where I could add new chunks of memory. Its best to minimise system calls such as malloc and similar because they hit OS and slow down execution. The memory in question is never controlled by the GC. But it may contain pointers into GC memory. You need to update your view on what addRange and removeRange does. Also when you bind a different language to D you don't have the option to change the code, you have to work with whats already there. Kind Regards Benjamin Thaut
Re: Safely extend the size of a malloced memory block after realloc
On Monday, 17 August 2015 at 19:38:21 UTC, Steven Schveighoffer wrote: On 8/17/15 3:27 PM, Benjamin Thaut wrote: Consider the following code void* mem = malloc(500); GC.addRange(mem, 500); mem = realloc(mem, 512); // assume the pointer didn't change GC.removeRange(mem); This is actually unsafe, you have to remove the range first, or else if it *does* change the pointer, your GC is using free'd memory. Plus, if it does change the pointer, how do you remove the original range? I specifically asked for the case where the pointer doesn't change. Obvisouly the case where it does change is easy, you first add the new range and then remove the old one. But if you do this and the pointer didn't change, the addRange doesn't do anything because its a duplicate and the removeRange then removes the range, because the pointer is still the same. You then end up with the GC not knowing anything about the range anymore. // if the GC kicks in here we're f* GC.addRange(mem, 512); Can't you GC.disable around this whole thing? -Steve Yes, this would work, but It seems kind of broken to me, that you have to make 4 API Calls to the gc to handle something as simple as a realloc. Kind Regards Benjamin Thaut
Safely extend the size of a malloced memory block after realloc
Consider the following code
void* mem = malloc(500);
GC.addRange(mem, 500);
mem = realloc(mem, 512); // assume the pointer didn't change
GC.removeRange(mem);
// if the GC kicks in here we're f*
GC.addRange(mem, 512);
I digged into GC.addRange to find out if I simply can skip the
call to GC.removeRange when the pointer doesn't change and ended
up with
void addRange(void *pbot, void *ptop, const TypeInfo ti) nothrow
@nogc
{
ranges.insert(Range(pbot, ptop));
}
where ranges is defined as: Treap!Range ranges
insert of Treap is implemented as:
Node* insert(Node* node, E element) @nogc
{
if (!node)
return allocNode(element);
else if (element node.element)
{
node.left = insert(node.left, element);
if (node.left.priority node.priority)
node = rotateR(node);
}
else if (element node.element)
{
node.right = insert(node.right, element);
if (node.right.priority node.priority)
node = rotateL(node);
}
else
{} // ignore duplicate
return node;
}
The problem is the line that says ignore duplicate. Because
that way I can not safly update a GC range.
How are you supposed to safely update a GC range that changed its
size but not its address? The documentation doesn't say anything
about it.
Either I'm missing something or this is a bug.
Kind Regards
Benjamin Thaut
Re: D for Game Development
On Monday, 10 August 2015 at 05:23:20 UTC, Walter Bright wrote: On 8/9/2015 9:26 PM, Tofu Ninja wrote: On Sunday, 9 August 2015 at 20:51:32 UTC, Walter Bright wrote: On 8/9/2015 4:38 AM, Manu via Digitalmars-d wrote: On 9 August 2015 at 15:31, Walter Bright via Digitalmars-d I agree, and now we ship a Phobos DLL, resolving that issue. Really? Where is it? (I can't see it in the distribution). Should be in the bin directory. There is no Phobos dll, only Phobos lib. There's linux/lib32/libphobos2.so, for example. But that is linux. You can't say phobos dll because everyone then expects dlls to work on windows, which still isn't the case. Its correct that we have a shared library version of phobos on linux, but a shared library version of phobos on windows is still in the future. I'm currently working on it, but there are so many issues with the language design, the export keyword, the module level visibility system the unittests etc etc that its going to take some more time. Also the front end transition from C++ to D hits me hard also.
