Re: Linux Dynamic Loading of shared libraries
Bless you, mate. You described precisely what I was looking for. Nonetheless, I am still looking for a load and unload command to select and reject plugins in runtime.
Re: D may disappoint in the presence of an alien Garbage Collector?
Registering a descriptor with moving GC is not enough, you should also fix the pointer so that it's not moved.
Re: D may disappoint in the presence of an alien Garbage Collector?
The better way would be to interact through a COM interface, which would abstract tricks of the library code. Advanced environments are usually able to generate such interface.
Re: Showing a user specified error message when no overloads match
On Tue, Jul 29, 2014 at 03:34:22AM +, Vlad Levenfeld via Digitalmars-d-learn wrote: opDispatch behaves as though it has SFINAE. When something fails in the definition (like I am having now, some of the symbols I used in it hadn't been imported) there won't ever be an error message, I just get Error: no property 'bar' for type 'Foo' In one case I had to use static ifs and pragmas and static assert(0) to get error messages out of opDispatch because static assert messages were being suppressed. Its very frustrating. You're right, opDispatch behaves like SFINAE. I've had trouble debugging it before, because when it works, it works very well, but when you accidentally make a typo, it just disappears -- you get an error that the property is missing, but the actual error inside opDispatch has been gagged and it's almost impossible to get at the actual error message. Maybe we should file an enhancement bug to improve error reporting for opDispatch. T -- Time flies like an arrow. Fruit flies like a banana.
Re: Showing a user specified error message when no overloads match
On 07/29/14 17:45, H. S. Teoh via Digitalmars-d-learn wrote: You're right, opDispatch behaves like SFINAE. I've had trouble debugging it before, because when it works, it works very well, but when you accidentally make a typo, it just disappears -- you get an error that the property is missing, but the actual error inside opDispatch has been gagged and it's almost impossible to get at the actual error message. D's overloaded operators are (usually) normal (templated) functions, you can use `a.opDispatch!blah` instead of `a.blah` to see what's wrong. What's really nasty is the way phobos handles `toString` - if that method fails to compile then you get a usually not very helpful default, and no warning that something is wrong. It's easy to break `toString` w/o noticing anything. Figuring out later what exactly broke can be interesting. Still doable via the above mentioned trick, but you'll need to create a mock `sink` etc. artur
Re: Linux Dynamic Loading of shared libraries
On Monday, 10 March 2014 at 11:59:20 UTC, Steve Teale wrote: Note that there is no call to Runtime.unloadLibrary(). The assumption her is that once the plugin has been loaded it will be there for the duration of the program. If you want to unload it you'll probably have to make sure the plugin object is purged from memory first, and I have not discovered how to do that yet ;=( A long time ago, Andrei suggested creating a GC interface for mapped memory. I think something similar might be appropriate here. Perhaps the location of instantiated classes could be determined by their vtbl pointer? Then the entire vtbl range could be treated as a dynamically allocated struct of sorts, and its dtor would queue up a job to unmap the library after collection is complete (ie. not immediately, since the order of destruction during a collection is undefined). So... (just thinking out loud) when a library is loaded, you basically perform an in-place construction of this LoadedLibrary struct on top of the vtbl range. You'd need an interface similar to the GC tracks memory mapped files idea to tell the GC to track references to this range of memory that lives outside its own pool set, and then some kind of post-collection job queue that's externally appendable so the LoadedLibrary struct could add an unload call when it's collected. Heck, we could really handle all dtors this way, so normal dtors would be inserted at the front of the list and special cases like this would be inserted at the back. It doesn't sound tremendously difficult, though we'd need the memory-mapped file support API first. Is there something I'm missing that makes this unfeasible?
Re: Showing a user specified error message when no overloads match
On Tue, Jul 29, 2014 at 06:57:00PM +0200, Artur Skawina via Digitalmars-d-learn wrote: On 07/29/14 17:45, H. S. Teoh via Digitalmars-d-learn wrote: You're right, opDispatch behaves like SFINAE. I've had trouble debugging it before, because when it works, it works very well, but when you accidentally make a typo, it just disappears -- you get an error that the property is missing, but the actual error inside opDispatch has been gagged and it's almost impossible to get at the actual error message. D's overloaded operators are (usually) normal (templated) functions, you can use `a.opDispatch!blah` instead of `a.blah` to see what's wrong. Good idea! Though it's still limited when the opDispatch call is buried under several layers of templated generic functions. The problem is, sometimes you don't even know that the problem is inside opDispatch because failure causes it to default to something else (that generates an unrelated error message), and the actual error message is gagged. Then once you do narrow it down, it isn't always so easy to get at the code that calls opDispatch, since it could be quite deep inside generic functions. Once you pinpoint it, though, your idea is pretty good -- invoke it directly so that the errors are not gagged. What's really nasty is the way phobos handles `toString` - if that method fails to compile then you get a usually not very helpful default, and no warning that something is wrong. It's easy to break `toString` w/o noticing anything. Figuring out later what exactly broke can be interesting. Still doable via the above mentioned trick, but you'll need to create a mock `sink` etc. [...] Mock sinks are very easy, because a delegate is a sink. :-) import std.range; int delegate(const(char)[]) dg; static assert(isOutputRange!(typeof(dg), const(char)[])); // passes Which makes it very handy to insert arbitrary debugging code for toString via dg.formattedWrite(%s, obj). T -- I'm running Windows '98. Yes. My computer isn't working now. Yes, you already said that. -- User-Friendly
pure delegates and opApply
I have some use cases where I am trying to iterate over a collection in a pure function. The collection doesn't change, but the foreach loop body updates a variable elsewhere in the function body. (I'm reimplementing reduce as pure nothrow, and the variable in question is the accumulator). Since opApply is implemented as a delegate, though, this prevents my function from being pure. The delegate isn't pure because it writes through its context pointer - but the context pointer shouldn't capture anything outside of the calling function's scope (I have no idea how to verify this). In other words, if I replaced the foreach with a for-loop, my function would be pure. I see 3 possible takes on this, please tell me if I'm hot or cold: 1) foreach is the idiomatic D way to iterate, but since I am insisting on purity, I have to accept that I might have to do things in a non-idiomatic way and go for the for-loop. 2) ranges are the other idiomatic D way to iterate, and if I want purity then I should be working with ranges anyway (as they have a more mathematical flavor to them than foreach or for) 3) opApply ought to be considered pure if the supplied delegate doesn't modify anything outside of the calling function's scope (bug/enhancement)
Re: Showing a user specified error message when no overloads match
On Tuesday, 29 July 2014 at 15:47:08 UTC, H. S. Teoh via Digitalmars-d-learn wrote: Maybe we should file an enhancement bug to improve error reporting for opDispatch. I found this https://issues.dlang.org/show_bug.cgi?id=8387 and cast a vote for it. I've cast a few votes in the bugtracker, don't know if that raises the visibility of the bugs or what. Maybe not since this bug had 4 votes to begin with and has been around since 2012... or maybe 4 is not that many.
Re: Showing a user specified error message when no overloads match
On Tue, Jul 29, 2014 at 11:30:01PM +, Vlad Levenfeld via Digitalmars-d-learn wrote: On Tuesday, 29 July 2014 at 15:47:08 UTC, H. S. Teoh via Digitalmars-d-learn wrote: Maybe we should file an enhancement bug to improve error reporting for opDispatch. I found this https://issues.dlang.org/show_bug.cgi?id=8387 and cast a vote for it. I've cast a few votes in the bugtracker, don't know if that raises the visibility of the bugs or what. Maybe not since this bug had 4 votes to begin with and has been around since 2012... or maybe 4 is not that many. Hmm. Looks like that bug is frequently encountered, judging by the number of duplicates it has. Maybe it's time to complain in the main D forum in hopes that one of the compiler devs will respond. ;-) T -- The number you have dialed is imaginary. Please rotate your phone 90 degrees and try again.
Re: Linux Dynamic Loading of shared libraries
Can't retrieve the archive from that URL.
britseyeview.com/plugin101.tar.bz2
Interested, so can you please fix?
On Monday, 10 March 2014 at 11:59:20 UTC, Steve Teale wrote:
On Sunday, 9 March 2014 at 12:07:22 UTC, Steve Teale wrote:
Now suppose that my D shared library contains a class, rather
that just module ctors/dtors, how do I go about creating an
instance of that class and using its methods?
After wandering down several dead-end paths, and help from
other contributors, I have finally come up with something that
looks like the basis of a plugin pattern for Linux DMD using
shared objects (.so files). This is somewhat long for a forum
post. You can download this readme and the associated files
from britseyeview.com/plugin101.tar.bz2
To get started, you need a base class that provides
declarations for all functions that the plugin will be allowed
to use externally. Why base class, and not interface? Well I
guess because interfaces don't provide any information about
data. If you create a shared library based on an interface,
then all the shared object methods that reference data in the
class that implements the interface fail miserably. I'm sure
someone will explain why - probably some obvious thing I have
overlooked.
OK, so my base class is:
module plugin;
class Plugin
{
int n;
this(int _n) { n = _n; }
int foo() { return int.min; }
void bar() {}
}
The class that implements this base in the shared library is:
module exta;
import plugin;
import std.stdio;
import std.math;
class ExtA: Plugin
{
double d;
this(int n) { super(n); d = PI; }
override int foo() { return ++n; }
override void bar() { writefln(Done my thing (%f), d); }
}
Plugin getInstance(int n)
{
return new ExtA(n);
}
shared static this() {
writeln(exta.so shared static this);
}
shared static ~this() {
writeln(exta.so shared static ~this);
}
The module ctor/dtor are included because that has become
conventional in discussions about dynamic loading. Otherwise,
the so has the class implementation - ExtA, and a shared method
to create an instance of same. It includes references to
methods in Phobos.
The test program is as follows:
module main;
import core.runtime;
import std.stdio;
import plugin;
extern(C) void* dlsym(void*, const char*);
alias Plugin function(int) pfi;
Plugin getPlugin(string name)
{
void* lib = Runtime.loadLibrary(name~.so);
if (lib is null)
{
writeln(failed to load plugin shared object);
return null;
}
void* vp = dlsym(lib,
_D4exta11getInstanceFiZC6plugin6Plugin\0.ptr);
if (vp is null)
{
writeln(plugin creator function not found);
return null;
}
pfi f = cast(pfi) vp;
Plugin x = f(42);
if (x is null)
{
writeln(creation of plugin failed);
return null;
}
return x;
}
void main()
{
Plugin x = getPlugin(exta);
int n = x.foo();
writefln(n = %d, n);
x.bar();
}
The long symbol name used in the dlsym() call is of course from
the .map file generated when the .so file is created
These can be built using the following primitive makefile,
whose main purpose is to spell out the required compiler flags:
main :
dmd -c plugin.d
dmd -c -shared -fPIC exta.d
dmd exta.o -shared -defaultlib=libphobos2.so -map
dmd -c main.d
dmd main.o plugin.o -L-ldl -defaultlib=libphobos2.so -L-rpath=.
This assumes that the plugins will be in the same directory as
the executable (rpath=.).
Note that there is no call to Runtime.unloadLibrary(). The
assumption her is that once the plugin has been loaded it will
be there for the duration of the program. If you want to unload
it you'll probably have to make sure the plugin object is
purged from memory first, and I have not discovered how to do
that yet ;=(
Steve
Re: myrange.at(i) for myrange.dropExactly(i).front
On Sun, Jul 27, 2014 at 9:20 PM, H. S. Teoh via Digitalmars-d-learn
digitalmars-d-learn@puremagic.com wrote:
On Sun, Jul 27, 2014 at 07:42:17PM -0700, Timothee Cour via
Digitalmars-d-learn wrote:
Just for clarification, I wanted 'myrange.at(i)' to be the same as
`myrange.dropExactly(i).front`
(so I don't assume it's a random access range).
myrange.dropExactly(i).front makes it much more obvious what you're
doing and that it's inefficient. It might be necessary in some cases,
but we don't want to give the impression that it's cheap, which at()
would do.
I think it's already clear that it's potentially O(n) [n=i] cost as
we're not using myrange[i]. But fine, call it atWalk/walkAt/whatever.
Point is it's a common enough operation.
[...]
You could just define your own function for it, right?
// or call it whatever you want
auto getNth(R)(R range, size_t index)
if (isInputRange!R)
{
return range.dropExactly(index).front;
}
T
--
Making non-nullable pointers is just plugging one hole in a cheese grater.
-- Walter Bright
Obviously I did that, but I thought it belonged in phobos. Anyway, closing
this.
Re: D may disappoint in the presence of an alien Garbage Collector?
On Monday, 28 July 2014 at 21:33:54 UTC, Rene Zwanenburg wrote:
If I understand you correctly, an easy way is to use RefCounted
with a simple wrapper. Something like this:
// Descriptor defined by the external library
struct DescriptorImpl
{
size_t type;
void* data;
}
// Tiny wrapper telling the alien GC of the existence of this
reference
private struct DescriptorWrapper
{
DescriptorImpl descriptor;
alias descriptor this;
@disable this();
this(DescriptorImpl desc)
{
// Make alien GC aware of this reference
}
~this()
{
// Make alien GC aware this reference is no longer valid
}
}
// This is the type you will be working with on the D side
alias Descriptor = RefCounted!DescriptorWrapper;
Just read RefCounted definition here,
http://dlang.org/phobos/std_typecons.html#.RefCounted
and it heap allocates its object, so your response above does not
stack allocate the basic type that you call DescriptorWrapper,
and is not a solution to the problem as stated.
If there was no alien GC, but everything else was the same, heap
allocation of something containing a DescriptorImpl would be
unnecessary. Now achieve the same with the alien GC present
without an extra layer of indirection and heap allocation ---
this is the essence of my question.
Re: D may disappoint in the presence of an alien Garbage Collector?
On Monday, 28 July 2014 at 20:52:01 UTC, Anton wrote: On Monday, 28 July 2014 at 19:57:38 UTC, Carl Sturtivant wrote: Suppose I want to use D as a system programming language to work with a library of functions written in another language, operating on dynamically typed data that has its own garbage collector, such as an algebra system or the virtual machine of a dynamically typed scripting language viewed as a library of operations on its own data type. For concreteness, suppose the library is written in C. (More generally, the data need not restricted to the kind above, but for concreteness, make that supposition.) Data in such a system is usually a (possibly elaborate) tagged union, that is essentially a struct consisting of (say) two words, the first indicating the type and perhaps containing some bits that indicate other attributes, and the second containing the data, which may be held directly or indirectly. Call this a Descriptor. Descriptors are small, so it's natural to want them held by value and not allocated on the heap (either D's or the library's) unless they are a part of a bigger structure that naturally resides there. And it's natural to want them to behave like values when passed as parameters or assigned. This usually fits in with the sort of heterogeneous copy semantics of such a library, where some of the dynamic types are implicitly reference types and others are not. The trouble is that the library's alien GC needs to be made aware of each Descriptor when it appears and when it disappears, so that a call of a library function that allocates storage doesn't trigger a garbage collection that vacuums up library allocated storage that a D Descriptor points to, or fails to adjust a pointer inside a D descriptor when it moves the corresponding data, or worse, follows a garbage pointer from an invalid D Descriptor that's gone out of scope. This requirement applies to local variables, parameters and temporaries, as well as to other situations, like D arrays of Descriptors that are D-heap allocated. Ignore the latter kind of occasion for now. Abstract the process of informing the GC of a Descriptor's existence as a Protect operation, and that it will be out of scope as an Unprotect operation. Protect and Unprotect naturally need the address of the storage holding the relevant Descriptor. In a nutshell, the natural requirement when interfacing to such a library is to add Descriptor as a new value type in D along the lines described above, with a definition such that Protect and Unprotect operations are compiled to be performed automatically at the appropriate junctures so that the user of the library can forget about garbage collection to the usual extent. How can this requirement be fulfilled? Suppose I want to do system programming...Would I choose the option with a GC ? Just get off. The GC is just such a fagot. People are smart enough to manage memory. It's the library to interface to that has its own GC, not my code. I just need to use D's system programming capabilities to work around the library's nasty GC so my data used by my calls to that library isn't trashed, and to do that efficiently and transparently. A system programming language should be able to efficiently interface to anything, right?
