openmethods 1.3.0
This release implements support for function attributes, except for `pure`. User-defined attributes on methods and method parameters are also supported. It is no longer necessary to call `updateMethods` explicitly, except after dynamically loading or unloading shared libraries. The internals got a major cleanup. All the mixin generating code has been moved to a separate set of modules, which I plan to contribute to the bolts library. You can read more about openmethods on githubL https://github.com/jll63/openmethods.d
Re: openmethods 1.3.0
On Sunday, 19 April 2020 at 13:13:55 UTC, Jean-Louis Leroy wrote: You can read more about openmethods on githubL https://github.com/jll63/openmethods.d Available on DUBS here: https://code.dlang.org/packages/openmethods
Re: openmethods 1.3.0
On Monday, 20 April 2020 at 08:17:24 UTC, Robert M. Münch wrote: I just read your blog post [1] and wonder if it's still up-to-date or maybe an update would make sense? The blog post is still current. I remember that, in 2017, some were annoyed by the need to call a setup function (updateMethods). As for support for attributes, it is nice to have, but it is hardly the focus of the module. I don't think the improvements deserve a new blog entry. However, in the process of implementing support for attributes and storage classes, the internals became very ugly. Also I had a to address the same problems in a new library. Eventually I found a way of factoring the mixin generation code in a package that I am going to spin off, probably as part of bolts. But I need permission from my employer to do that. I hope to get it soon. This stuff sounds like a very fundamental concept/pattern and IMO would be a good member of Phobos. Andrei is not convinced ;-) Well at least not as part of the language, but probably not as part of Phobos either. That is not a problem. If I was granted two wishes, they would be: 1/ reallocate 'ClassInfo.deallocator' to me ;-) and 2/ add a more general feature to the language, similar to Perl's 'import' function: if a module defines a 'string imported(alias Module)' or a 'mixin template imported(alias Module)', call it in the context of the importing module. That would allow me to get rid of the 'mixin(registerMethods)' after the 'import openmethods'.
Re: describe-d: an introspection library
On Wednesday, 15 April 2020 at 08:00:12 UTC, bogdan wrote: I use a similar approach in openmethods. With the added twist that I need to re-create functions from existing functions, with some modifications (e.g. change the parameter types, add a parameter), while preserving function and parameter attributes. That inevitably leads to constructing mixin code (which I call "mixtures"), and it has to work across module boundaries (i.e. use `ReturnType!F` in the mixture, not the stringified type). See here: https://github.com/jll63/openmethods.d/blob/master/source/openmethods.d#L503, here: https://github.com/jll63/openmethods.d/blob/master/source/openmethods.d#L568 and here: https://github.com/jll63/openmethods.d/blob/master/source/bolts/reflection/metafunction.d I did not shy away from using templates and indeed there is a significant increase in compilation time. On the other hand, the code is much cleaner than a previous iteration that created the mixtures directly. If these techniques get traction, it will incite compiler developers to improve template instantiation, hopefully. I wonder if templates are lazily expanded. I haven't looked at the compiler's code, my guess is: maybe not.
Re: describe-d: an introspection library
On Wednesday, 22 April 2020 at 05:20:18 UTC, Stefan Koch wrote: On Tuesday, 21 April 2020 at 14:43:04 UTC, Jean-Louis Leroy wrote: I wonder if templates are lazily expanded. I haven't looked at the compiler's code, my guess is: maybe not. If the template gets used it gets instantiated (and cached). I did not word my question precisely, I meant: is it instantiated in its *entirety*. Consider: import std.meta; template Function(Attributes...) { enum isPure = Attributes[0]; enum isNogc = Attributes[1]; alias parameters = Attributes[2]; } template Parameter(int i) { // static assert(false); } template reflectFunction(alias Fun) { alias reflectFunction = Function!( true, true, staticMap!(Parameter, 1, 2, 3)); } void foo(); pragma(msg, reflectFunction!(foo).isPure); Is `Parameter` instantiated in this example? If I uncomment the assert, it fires. This is what gives me the impression that more is instantiated than is really used. And that is probably why reflecting runtime entities as template meta-objects with properties for all aspects of them is slow. I am way of my field of competence here, but I have the impression that speeding this up would not require a change in the language (unless it is stated that templates are greedily expanded, and common idioms rely on this). you can use the -vcg-ast switch to look at how your souce code looks "expandend". I tried compiling my example with `dmd -vcg-ast -c tmt.d` and I did got neither an AST nor an error, and the option is not in `dmd -h`, where can I read about it? They already gained traction, unfortunately. Well...Two things. I ended up developing this approach because I needed to copy function attributes, UDAs, function parameter storage classes and UDAs to a generated function. If the language, or Phobos, provided me a more direct way, trust me, I would have used it. For example, storage classes are not part of a parameter's type. OK. Now you can use `Parameter!F[0]` to declare a parameter in a new function (great!), but you cannot say `__traits(getStorageClasses, F)[0] Parameter!F[0]`. Also, one of D's selling points is that it does templates and meta-programming better. It's only natural that people use these features then (note that in C++, sorry for mentioning it, in my example Parameter would *not* be instantiated).
Re: describe-d: an introspection library
On Wednesday, 22 April 2020 at 17:16:28 UTC, Stefan Koch wrote: I am working on a much more powerful and efficient meta programming system. Great! Is it going to be in a library, or part of the compiler? Can we get a preview somewhere?
Re: describe-d: an introspection library
On Wednesday, 22 April 2020 at 17:32:33 UTC, Stefan Koch wrote: It's going to be part of the compiler. You can look at the ... expression DIP. which Manu posted in General, for taste of where my stuff is going. I think I see the point - I am familiar with C++ variadic templates. Yes, speeding up `staticMap`, and often eliminating the need to use it, will hugely benefit meta-programming. However, that will not solve the problem of unnecessary instantiations that my example above demonstrates (it has a bug btw, the parameters should have been wrapped in a bolts.meta.AliasPack, but it doesn't alter the spirit). Thank you for the `-vcg-ast` switch! It looks like I am going to use it a lot. I am rewriting my system to express the accessors in terms of free functions taking meta-objects, instead of meta-objects containing properties. I have a hunch that that will curb instantiation, compile time and .o size.
Re: DIP 1030-- Named Arguments--Formal Assessment
On Thursday, 17 September 2020 at 12:58:06 UTC, Mike Parker wrote: DIP 1030, "Named Arguments", has been accepted. During the assessment, Walter and Atila had a discussion regarding this particular criticism: https://forum.dlang.org/post/mailman.1117.1581368593.31109.digitalmar...@puremagic.com "Named arguments breaks this very important pattern: auto wrapper(alias origFun)(Parameters!origFun args) { // special sauce return origFun(args); }" They say that, though it's true that `Parameters!func` will not work in a wrapper, it "doesn't really work now"---default arguments and storage classes must be accounted for. This can be done with string mixins, or using a technique referred to by Jean-Louis Leroy as "refraction", both of which are clumsy. Actually, Parameters!origFun will carry storage classes, UDAs, etc for all the parameters. And Parameters!origFun[0..1] (note the two dots) will carry everything about the first parameter. The trouble begins when, for some reason, you need to manipulate the parameter at a finer level. For example, in openmethods, I need to change the type while preserving everything else.
Re: DIP 1030-- Named Arguments--Formal Assessment
On Thursday, 17 September 2020 at 13:23:38 UTC, Jean-Louis Leroy wrote: Actually, Parameters!origFun will carry storage classes, UDAs, etc for all the parameters. And Parameters!origFun[0..1] (note the two dots) will carry everything about the first parameter. The trouble begins when, for some reason, you need to manipulate the parameter at a finer level. For example, in openmethods, I need to change the type while preserving everything else. (For the rest of this post, to make things clear, I will call Parameter declarations that appear inside the function definition, and Arguments the values that are passed to a function call). I like named arguments. However, I would be greatly disappointed if: 1/ Parameters!origFun and Parameters!origFun[i..i+1] stopped working as well as they do now. 2/ The named arguments did not come with new traits (or at least is(__parameter) magic) to allow fully analyzing the parameters. That being said, does the new feature imply any change in the *parameters* themselves? I.e. are there changes in the way the function is defined, not only in the way it is called? I have not followed the discussion and just skimmed over the DIP now. I'm going to try to find time for this. However, at this point I have some hope that the DIP is not damaging in the way Mike thinks.
Re: DIP 1030-- Named Arguments--Formal Assessment
On Thursday, 17 September 2020 at 13:45:16 UTC, Mike Parker wrote: On Thursday, 17 September 2020 at 13:42:47 UTC, Jean-Louis Leroy wrote: this point I have some hope that the DIP is not damaging in the way Mike thinks. What Mike thinks appears nowhere in my post :-) OK, s/thinks/reports/ ;-)
Re: DIP 1030-- Named Arguments--Formal Assessment
On Thursday, 17 September 2020 at 12:58:06 UTC, Mike Parker wrote: So they decided that a new `std.traits` template and a corresponding `__traits` option are needed which expand into the exact function signature of another function. I have been trying to locate that specific discussion, without success so far. Help? This is of great interest to me, and I may throw in my $.02.
Re: DIP 1030-- Named Arguments--Formal Assessment
On Friday, 18 September 2020 at 13:39:14 UTC, Mike Parker wrote: It's from a phone call they had while they were discussing whether to approve or reject the DIP. LOL no wonder I couldn't find it.
New library: open multi-methods
Hello, TL;DR: see here https://github.com/jll63/methods.d/blob/master/README.md for an explanation of what open multi-methods are, if you are not familiar with the idea.You may also want to read my article on Code Project https://www.codeproject.com/Articles/635264/Open-Multi-Methods-for-Cplusplus11-Part-1 Earlier this year I attended Ali Çehreli's talk at C++ Now. He did a good job: I walked out with the desire to learn about D and see how it measures up against C++, especially in terms of meta-programming and language extensibility. The first programming language I learned is Forth and I did some Lisp programming, so as you can imagine, my expectations are high. As an experiment, I decided to try to port parts of my yomm11 library to D. The experience turned out to be pleasant and I ended up writing a full implementation, with some friendly help from Ali and others in the Learn forum. I think that what I have now is good enough to show. The git repo is here https://github.com/jll63/methods.d and I will post a package to the registry soon. If you have the inclination, feel free to review and comment. This is my very first D project and I certainly have missed some idioms and been clumsy at times. Jean-Louis Leroy
Re: New library: open multi-methods
Thinking about it, 'openmethods' would probably be a better module/package name than just 'methods'. It emphasizes the #1 feature, i.e. polymorphism outside of classes.
Re: New library: open multi-methods
On Monday, 17 July 2017 at 21:31:20 UTC, jmh530 wrote: On Monday, 17 July 2017 at 20:41:05 UTC, Jean-Louis Leroy wrote: Thinking about it, 'openmethods' would probably be a better module/package name than just 'methods'. It emphasizes the #1 feature, i.e. polymorphism outside of classes. Googling `multimethods` brought up more programming-related topics than `openmethods`. Yeah, I know, but I can imagine someone casually browsing the registry, coming across the module and saying "multi-methods? yeah, cool, but I don't remember ever needing such a thing". Indeed "multi" is nice, but IMO "open" is much more important. It is still much more frequent to have only one virtual argument. Take the awful Visitor pattern, for example. It can be neatly replaced with an open method taking only one virtual argument (barring other considerations). 'openmultimethods' is another option but again it emphasizes 'multi'. Anyway, if I go for just 'openmethods', there are enough mentions of 'multi-methods' in the docs. I think I will rename 'methods' to 'openmethods' for the time being, but the discussion remains open. Not renaming the repo yet. Thinking about it, I should add a Visitor replacement example... J-L
Re: New library: open multi-methods
On Monday, 17 July 2017 at 22:59:03 UTC, jmh530 wrote: On Monday, 17 July 2017 at 22:46:02 UTC, Jean-Louis Leroy wrote: I think I will rename 'methods' to 'openmethods' for the time being, but the discussion remains open. Not renaming the repo yet. On the other hand, when I saw methods, my first thought was R's methods, which I imagine is similar if I'm not mistaken. I don't know R but after a trip to Wikipedia it looks like it. J-L
Re: New library: open multi-methods
On Monday, 17 July 2017 at 21:32:13 UTC, jmh530 wrote: On Monday, 17 July 2017 at 21:31:20 UTC, jmh530 wrote: On Monday, 17 July 2017 at 20:41:05 UTC, Jean-Louis Leroy wrote: Thinking about it, 'openmethods' would probably be a better module/package name than just 'methods'. It emphasizes the #1 feature, i.e. polymorphism outside of classes. Googling `multimethods` brought up more programming-related topics than `openmethods`. Or you could call it omm and then just refer to open multi-methods in the documentation. Yeah that's what the omm in yomm11 means, but I am not too fond of acronyms. In C++ it was the library name (-lyomm11) and also the project name but no the namespace. J-L
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 02:22:15 UTC, Jay Norwood wrote: An excerpt statement from this wiki page is : " dynamically dispatched based on the run-time (dynamic) type or, in the more general case some other attribute, of more than one of its arguments" Based on the 'some other attribute', I wonder if the library could conceivably be extended to dispatch based on the User Defined Attribute info https://dlang.org/spec/attribute.html @('c') string s; pragma(msg, __traits(getAttributes, s)); // prints tuple('c') For example, CLOS allows you to specialize on a value (google for "eql specialize"). IIRC Clojure allows you to specify your own dispatcher. As for specializing on D attributes, I don't think it's feasible. They are a purely compile-time mechanism. In your example, the type of "s" is "string", not "@('c') string". J-L
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 04:26:42 UTC, Ali Çehreli wrote: On 07/16/2017 10:24 AM, Jean-Louis Leroy wrote: > TL;DR: see here https://github.com/jll63/methods.d/blob/master/README.md Woot! :) I'm so happy to see this project complete. Honestly, growing up with languages without this feature (C and C++), I've not even known that I needed this feature but your example (e.g. matrix multiplication) are very convincing. Thanks :) I added another example that shows how open methods are a superior alternative to Visitor: https://github.com/jll63/methods.d/blob/master/examples/novisitor/source/app.d If there are enough differences compared to your C++ articles, perhaps you may consider following this up with a blog post. It would be nice to see some performance results as well like you have on your C++ articles. Yes I will probably write something. You mean on the D Blog? As for performance, I have a first result: https://github.com/jll63/methods.d/blob/master/benchmarks/source/benchmarks.d#L122 but I still have to implement the "first argument optimization". I am working on it. J-L Ali
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 07:06:10 UTC, Jean-Louis Leroy wrote: As for performance, I have a first result: https://github.com/jll63/methods.d/blob/master/benchmarks/source/benchmarks.d#L122 but I still have to implement the "first argument optimization". I am working on it. Now this is funny, after implementing that optimization (https://github.com/jll63/methods.d/blob/94ad5a945b3c719bd8f8402bb0aa6fda8e7a6be0/source/openmethods.d#L388, https://github.com/jll63/methods.d/blob/94ad5a945b3c719bd8f8402bb0aa6fda8e7a6be0/benchmarks/source/benchmarks.d#L139) it runs faster with ldc2 but slower with dmd. I may be testing the limits of dmd's willingness to inline my mess ;-) J-L
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 16:57:30 UTC, Ali Çehreli wrote: > As for performance, I have a first result: > https://github.com/jll63/methods.d/blob/master/benchmarks/source/benchmarks.d#L122 > but I still have to implement the "first argument optimization". I am > working on it. I could use some explanation for the results but I can for the blog article. ;) I pit a method-based call against its equivalent using virtual functions. First calling a virtual function via a base class is pitted against a method with one virtual parameter. Then the same but calling via an interface. Lastly, I compare double dispatch with a method with two virtual arguments. I use std.datetime.comparingBenchmark, which reports the result as time(base)/time(target). So open methods are a bit slower than ordinary virtual function calls but not that much. In the meantime I have applied a second optimization for unary methods and this brings them within 33% of an ordinary, compiler implemented vfunc call. Which is OK because the situation is highly artificial. If the function does anything, the difference will be imperceptible. I am more annoyed by double dispatch beating binary methods. I will have to look at the assembler, but it may be that the index pointer is too far from the object. To begin the real work, I need to fetch that pointer form an object. Currently it is stored in ClassInfo.deallocator, so I have to 1/ fetch the vptr 2/ fetch the ClassInfo* 3/ fetch 'deallocator'. What happens next depends on the arity. Any chance of Walter giving me a pointer in the vtable? Aside the ClassInfo*? Or at least a pointer in ClassInfo, or reassign the deallocator when it is eventually retired? It's not surprising that ldc (and gdc) can be much better than dmd in optimization. I would like to try gdc but it conflicts with ldc2 - you know, the "alias __va_list = __va_list_tag;" issue. I found suggestions via google but nothing worked for me so far. By the way, I'm in awe of your D skills in such a short time! Thanks :) I found out that D was much more natural, "predictable" than C++. The most cryptic error messages happened when I forgot the "!", IIRC. I'm sure there are parts of the code that can be cleaned up but it's taking advantage of many powerful features of the language. I still think the usage can be made easier but I'm not sure yet. I hope others will take a look at the code and come up with an easier interface. Perhaps they are all needed but I'm thinking about the need for forward declaration, the need for the underscore prefix, etc. (in reverse order) Regarding the prefix, it is needed to prevent overload resolution from trumping dynamic dispatch - see here: https://github.com/jll63/methods.d/blob/master/examples/whytheunderscore/source/app.d Allowing the same name would necessitate compiler support. As for the the forward declaration - I don't think it is possible to dispense with it. All open methods systems I know of have it (defgeneric in CLOS, defmulti in Clojure, Stroustrup's proposal...). Consider: class A { } class B : A { } class X : B { } class Y : B { } @method void _foo(virtual!X x) { ... } @method void _foo(virtual!Y x) { ... } What is the base method? foo(B)? foo(A)? It may well be the latter. Also don't forget that the complete specialization set is known, at the earliest, at link time. If you (arbitrarily) pick foo(B), another module may introduce a B or an A specialization. As for suggestions and advise, they are very welcome :) already got a couple of PRs. Here are the remaining questions on my mind: - the module/package name: I am pretty much set on openmethods though... - throw an exception if a method is not define for the argument set, or ambiguous: having big doubts about this. We want the possibility of @nothrow methods, don't we? So I will probably call a delegate via a pointer (a la C++) which, by default, will abort(). - the method prefix: hesitating between just _ or maybe m_ ??? - replace version(explain) with debug levels?
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 18:21:21 UTC, Ali Çehreli wrote: That reminds me: Would the following be possible and better? // From void main() { updateMethods(); // ... } // To mixin(constructMethods()); void main() { // ... } constructMethods() could return the following string: string constructMethods() { return q{ shared static this() { updateMethods(); } }; } If I'm not missing something, this is better because nothing needs to be added to main and the methods are available before main starts executing (module initialization order issues still apply.). Ah, I would love to get rid of that call in main(), but think beyond a one module program. The matrix example (https://github.com/jll63/methods.d/tree/master/examples/matrix/source) consists in three separate modules, plus an app, all defining specializations. They need the mixin, but if we put updateMehods() in there, it will be called many times. And it is a costly operation. Guarding the call with a flag will not work, because more methods may be registered afterwards. Unless you can think of a way the last mixin can detect it's the last? Incidentally, in yomm11 that function (it's called initialize()) has to be called before any method is called, after any shared object/DLL is loaded and after a DLL is unloaded. I still have to write the code to de-register the methods and the specializations in that case by the way... J-L J-L
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 18:03:30 UTC, jmh530 wrote: On Tuesday, 18 July 2017 at 16:57:30 UTC, Ali Çehreli wrote: Perhaps they are all needed but I'm thinking about the need for forward declaration, the need for the underscore prefix, etc. He might be able to at least get rid of the forward declaration (not sure on the underscore). The way it works now is that a class that inherits from an interface is not required in any way to implement the methods. Suppose he adds another attribute to an interface such that any class that inherits from it is required to have methods defined for specific functions. So for instance, the Matrix example might look something like @trait interface Matrix { @property int rows() const; @property int cols() const; @property double at(int i, int j) const; @trait void print(); } I'm not sure this would work because anything that derives from Matrix must implement print. However, if it is possible to use the attribute to allow the derived classes to ignore print, then it might work. Alternately, if there is a way to process the interface and tell the compiler to somehow ignore the @trait member functions. I don't know if it'll work, but it's an idea. Anyway, the mixin(registerMethods); could then be adjusted so that void print(virtual!Matrix m); is mixed in automatically because we now know how to construct it. There are at least problems with this. Firstly it is intrusive - something I strive to avoid (although I could be 100% orthogonal only because I hijack a deprecated pointer in ClassInfo). Also, some methods may want to treat Matrix as a virtual argument, and some not. Look at https://github.com/jll63/methods.d/blob/master/examples/matrix/source/matrix.d and https://github.com/jll63/methods.d/blob/master/examples/matrix/source/densematrix.d They know nothing about printing. They don't want to. The matrix modules do math, the app does printing. J-L
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 21:20:04 UTC, jmh530 wrote: On Tuesday, 18 July 2017 at 21:16:11 UTC, jmh530 wrote: I may not have been clear enough. My ideal solution wouldn't make any changes to that densematrix.d file, just the interface. So I don't have any issue with the matrix modules doing the math and the app doing the printing. Well, I suppose the matrix interface would be saying that it can print, so maybe not as split up as you would like. While you could define a separate interface for printing, that would require a change to densematrix. Exactly. Orthogonality is essential for good composition, that is the reason why OOP - well, the OOP that follows the Simula/Smalltalk tradition - failed so badly. CLOS got it right 40 years ago; Simula, Smalltalk, C++, Java, etc they all got it wrong.
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 22:41:13 UTC, Ali Çehreli wrote: On 07/16/2017 10:24 AM, Jean-Louis Leroy wrote: Hello, TL;DR: see here https://github.com/jll63/methods.d/blob/master/README.md Added D to the Wikipedia entry, which can be expanded. :) https://en.wikipedia.org/wiki/Multiple_dispatch Ali Haha that settles it then, openmethods it is! I'll rename the repo and upload to the registry momentarily.
Re: New library: open multi-methods
On Wednesday, 19 July 2017 at 06:27:40 UTC, James Dean wrote: Interesting. One problem I think the above approach has is adding methods after compilation. Say, a plugin adds a new derived matrix type SparseMatrix and wants to customize the addition of them. This is impossible under the current model, is it not? Why? I haven't tried it yet (putting together an example is one of the TODOs before v1.0.0) but I fully expect it to work. The dispatch tables are created at compile time. Just call updateMethods after loading or unloading the DLL and it should work. It does in the C++ version.
Re: New library: open multi-methods
openmethods is now available in the dub registry: https://code.dlang.org/packages/openmethods
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 04:26:42 UTC, Ali Çehreli wrote: It would be nice to see some performance results as well like you have on your C++ articles. Lib is in the dub registry now. Do you have a working gdc compiler? If yes, could you run the benchmark and post the results? Please make sure to build in release mode.
Re: New library: open multi-methods
On Tuesday, 18 July 2017 at 18:21:21 UTC, Ali Çehreli wrote: On 07/18/2017 11:03 AM, jmh530 wrote: > the mixin(registerMethods); could then be adjusted so that void > print(virtual!Matrix m); is mixed in automatically because we now know > how to construct it. That reminds me: Would the following be possible and better? // From void main() { updateMethods(); // ... } // To mixin(constructMethods()); void main() { // ... } constructMethods() could return the following string: string constructMethods() { return q{ shared static this() { updateMethods(); } }; } OK, I think I may have found solutions to both problems. The question is, is it too hacky? 1/ method registration Replace this: import openmethods; mixin(registerMethods); ...with: mixin(import(openmethoddecls)); ...that does the two above. Problem is, it needs -Jpath on the command line to work. Unless there is a workaround? 2/ updateMethods During static construction, I could set the dispatch tables to make all the methods call a function that does updateMethods() then re-dispatches. The cost of the first method call would be huge, but if it matters the user can still call updateMethods explicitly. Thoughts?
Re: New library: open multi-methods
On Wednesday, 19 July 2017 at 13:36:55 UTC, jmh530 wrote: On Wednesday, 19 July 2017 at 13:35:40 UTC, jmh530 wrote: On Wednesday, 19 July 2017 at 12:29:36 UTC, Jean-Louis Leroy wrote: ...that does the two above. Problem is, it needs -Jpath on the command line to work. Unless there is a workaround? I prefer the original. What if you do: shared static this(){ mixin(registerMethods); } You mean in openmethods.d?
Re: New library: open multi-methods
On Wednesday, 19 July 2017 at 15:33:28 UTC, jmh530 wrote: On Wednesday, 19 July 2017 at 13:46:24 UTC, Jean-Louis Leroy wrote: What if you do: shared static this(){ mixin(registerMethods); } You mean in openmethods.d? Yes. I haven't tried something like that, but it seems like a use case for either static this or shared static this. https://dlang.org/spec/class.html#StaticConstructor https://dlang.org/spec/class.html#SharedStaticConstructor Among other things, the mixin introduces two functions in the module's scope: the function the user actually calls (the "dispatcher"). E.g. it creates a times(double, Matrix) when it sees a times(double, virtual!Matrix). It also declares a "discriminator" function which is used to locate which method the specializations (the @method funcs) relates to (it has to deal with overloads - there are two "times" methods). This has to be done for every module that contains method declarations (virtual!) or implementations (@method). That's why it has to be a string mixin (at least until we have static foreach) and be called in the matrix etc modules, not in module openmethods.
Re: Article: Writing Julia style multiple dispatch code in D
On Thursday, 24 August 2017 at 23:50:21 UTC, data pulverizer wrote: I find OOP-polymorphic types ultimately unsatisfying, but I don't know of anyway to write, compile and load a D script with new types and methods on the fly into the same session. That is why binding membership and polymorphism together is a historical wrong turn. CLOS had it right but the world followed the Simula/Smalltalk path because of a nice metaphor (objects sending messages to each other). My openmethods library allows you to add methods "from outside" and also supports dynamic loading: you can add new methods to existing classes and new classes to hierarchies that have methods. See the blog post that just came up.
Re: Article: Writing Julia style multiple dispatch code in D
On Thursday, 24 August 2017 at 23:50:21 UTC, data pulverizer wrote: I find OOP-polymorphic types ultimately unsatisfying, but I don't know of anyway to write, compile and load a D script with new types and methods on the fly into the same session. That is why binding membership and polymorphism together is a historical wrong turn. CLOS had it right but the world followed the Simula/Smalltalk path because of a nice metaphor (objects sending messages to each other). My openmethods library allows you to add methods "from outside" and also supports dynamic loading: you can add new methods to existing classes and new classes to hierarchies that have methods. See the blog post that just came up.
Re: Open Methods: From C++ to D
On Monday, 28 August 2017 at 12:31:20 UTC, rikki cattermole wrote: On 28/08/2017 1:19 PM, Mike Parker wrote: Jean-Louis Leroy posted about his open methods library here in the forums some time ago. Now, he's written a blog post that explains what open methods are, and describes the D implementation and how it compares to his C++ library. The blog: https://dlang.org/blog/2017/08/28/open-methods-from-c-to-d/ Reddit: https://www.reddit.com/r/programming/comments/6wj0ev/open_methods_from_c_to_d/ Neat. Good to see articles in support of TypeInfo/ClassInfo! I do wish we extended it for full reflection capabilities though... Agreed. Andrei suggested using rtlInfo but AFAICT this requires building a custom druntime. It would be easy to support this method in addition to 'deallocator' and 'hash' but I suspect it would not be a very popular option. Maybe we could have a void*[] in TypeInfo and a global integer index that extensions could use to allocate entries?
Re: Open Methods: From C++ to D
On Tuesday, 29 August 2017 at 12:09:01 UTC, Mark wrote: Nice. This does seem superior to the visitor pattern. Here is another example - AST traversal: https://github.com/jll63/openmethods.d/blob/master/examples/acceptnovisitors/source/app.d
Re: Open Methods: From C++ to D
On Wednesday, 30 August 2017 at 04:48:11 UTC, Arun Chandrasekaran wrote: On Tuesday, 29 August 2017 at 12:45:50 UTC, Jean-Louis Leroy wrote: On Tuesday, 29 August 2017 at 12:09:01 UTC, Mark wrote: Nice. This does seem superior to the visitor pattern. Here is another example - AST traversal: https://github.com/jll63/openmethods.d/blob/master/examples/acceptnovisitors/source/app.d Thanks for this library. Just a suggestion. Would it possible to use `@openmethod` instead of `@method`? Ah, I think it's a little late for that. A while ago I asked if anyone had suggestions regarding that sort of things...But now that the article has been published I think it would be a very bad idea to break all the exmaples in it. What was your rationale for `openmethod` instead of just `method`? I will push a commit tonight that will make this work: import openmethods : virtual, openmethod = method, next, registerMethods, updateMethods; // ... @openmethod // implement 'kick' for dogs string _kick(Dog x) // note the underscore { return "bark"; }
Re: Open Methods: From C++ to D
On Wednesday, 30 August 2017 at 14:37:14 UTC, Arun Chandrasekaran wrote: On Wednesday, 30 August 2017 at 13:35:22 UTC, Jean-Louis Leroy wrote: On Wednesday, 30 August 2017 at 04:48:11 UTC, Arun What was your rationale for `openmethod` instead of just `method`? Just that `openmethod` precisely expresses it's intent and `method` is too generic. I sort of agree, and somewhat regret not picking 'openmethod'. I considered both. Also @specialize. If anyone had pushed for @openmethod before the article, I would almost certainly have given in. My reasoning was, I hope to promote the term 'method' as the standard name for polymorphism from outside, as opposed to vfunc. We usually say "virtual functions", rarely "virtual member functions". Membership is implicit.
Re: Open Methods: From C++ to D
On Wednesday, 30 August 2017 at 15:14:04 UTC, rikki cattermole wrote: On 30/08/2017 4:10 PM, Jean-Louis Leroy wrote: On Wednesday, 30 August 2017 at 14:37:14 UTC, Arun Chandrasekaran wrote: [...] I sort of agree, and somewhat regret not picking 'openmethod'. I considered both. Also @specialize. If anyone had pushed for @openmethod before the article, I would almost certainly have given in. My reasoning was, I hope to promote the term 'method' as the standard name for polymorphism from outside, as opposed to vfunc. We usually say "virtual functions", rarely "virtual member functions". Membership is implicit. Rename, alias to old and have it deprecated. Keep around for a couple of releases, done! Deprecated, already? :-D Hmmm maybe...Let's see if anyone speaks in favor of just @method.
Re: Open Methods: From C++ to D
On Wednesday, 30 August 2017 at 15:42:09 UTC, jmh530 wrote: One thing you didn't really cover is how seamlessly interacts with normal polymorphism. For instance, what if to your first example, I add the following function (note: without @method) and adjust main as below. I see no reason why this shouldn't work. But you wouldn't be able to create a string kick(Animal animal) function since that is created by the mixin. string kick(Dog dog) { return "ct bark"; } void main() { updateMethods(); import std.stdio : writeln; Animal snoopy = new Dog, hector = new Pitbull; writeln("snoopy.kick(): ", snoopy.kick()); // bark writeln("hector.kick(): ", hector.kick()); // bark an dbite Dog lassie = new Dog; writeln("lassie.kick(): ", lassie.kick()); // ct bark } What happens here is that kick(Animal) is shadowed by kick(Dog). kick(Animal) is a method but it appears to the user and the compiler as an ordinary function - which is generally good. As such it is eligible for UFCS. I would not recommend this sort of coding, but it's everyone's choice, methods or not. Likewise, methods can be overloaded (like here https://github.com/jll63/openmethods.d/blob/1.0.0-rc.1/examples/matrix/source/matrix.d#L12). A current limitation is that default arguments are not supported (yet), although I think it's just a matter of putting the effort in. UFCS interacts nicely with methods because you can say a.plus(b) even if 'plus' is an open method.
Re: Open Methods: From C++ to D
On Wednesday, 30 August 2017 at 16:37:20 UTC, Q. Schroll wrote: In the article it says: Finally, main calls updateMethods. This should be done before calling any method (typically first thing in main) and each time a library containing methods is dynamically loaded or unloaded. If the something has to be done at the beginning, we have a tool for that: static this (on module level). The mixin(registerMethods); at the top should therefore mix in. static this() { updateMethods(); } It's never wrong: Calling it in main, too, will at most be redundant. You can still call it manually, but for the part of main, you cannot inadvertently forget it. You can still have static this in that module as you may have multiple static this. We had a discussion about automating the call to updateMethods but I don't think that anybody thought of putting it in registerMethods. It might work. I'll look into it. Thanks for the suggestion...
Re: Article: Writing Julia style multiple dispatch code in D
On Wednesday, 30 August 2017 at 16:45:19 UTC, data pulverizer wrote: One thing that confused me was examples like this ... @method Matrix _plus(DiagonalMatrix a, DiagonalMatrix b) { // just add the elements on diagonals // return a DiagonalMatrix } Which is marked as returning a DiagonalMatrix rather than a Matrix by polymorphism however the function is marked Matrix return type. Indeed returning a DiagonalMatrix would work, and is marginally more useful (in case you want to call the specialization directly). I'll update the example. Thanks.
Re: Article: Writing Julia style multiple dispatch code in D
On Wednesday, 30 August 2017 at 17:16:59 UTC, data pulverizer wrote: p.p.s typeof(x[1]) # returns Cat so it isn't really polymorphism - the object is never converted to the "parent" type! Lol ... sorry for the confusion! Which is polymorphism Haha what I know of Julia is what wikipedia says. Confusing indeed...
Re: Open Methods: From C++ to D
On Wednesday, 30 August 2017 at 18:05:38 UTC, jmh530 wrote: On Wednesday, 30 August 2017 at 17:24:55 UTC, Jean-Louis Leroy wrote: We had a discussion about automating the call to updateMethods but I don't think that anybody thought of putting it in registerMethods. It might work. I'll look into it. Thanks for the suggestion... Ali had suggested something similar[1]. I think your concern was that it would get called multiple times, but shared static module constructors runs once a program (static module constructor runs once per thread). [1] http://forum.dlang.org/post/okljj1$ktb$1...@digitalmars.com Ah yes...So the problem is that registerMethods emits static ctors that fill data structures representing the methods and the specializations. They have to run - all of them - before updateMethods. Thus, sadly Q's suggestion won't work.
Re: Open Methods: From C++ to D
On Wednesday, 30 August 2017 at 18:20:46 UTC, Jean-Louis Leroy wrote: On Wednesday, 30 August 2017 at 18:05:38 UTC, jmh530 wrote: On Wednesday, 30 August 2017 at 17:24:55 UTC, Jean-Louis Leroy wrote: We had a discussion about automating the call to updateMethods but I don't think that anybody thought of putting it in registerMethods. It might work. I'll look into it. Thanks for the suggestion... Ali had suggested something similar[1]. I think your concern was that it would get called multiple times, but shared static module constructors runs once a program (static module constructor runs once per thread). [1] http://forum.dlang.org/post/okljj1$ktb$1...@digitalmars.com Ah yes...So the problem is that registerMethods emits static ctors that fill data structures representing the methods and the specializations. They have to run - all of them - before updateMethods. Thus, sadly Q's suggestion won't work. Yes I remember now...even if we arrange to put the call to updateMethods in its own static ctor, coming after all the other static ctors, we don't know if there will be more modules with more methods afterwards. So we would have to do the updateMethods work each time, lest this module is the last. And updateMethods is costly: not only does it figure out all the dispatch tables, it also calculates a perfect hash (if @mptr is used) using a random algorithm.
Re: Open Methods: From C++ to D
On Wednesday, 30 August 2017 at 18:16:47 UTC, jmh530 wrote: I tried installing the latest release from github. Compiling (Windows 7 on DMD with default options) the simple program below import openmethods; mixin(registerMethods); void main() { } gives me the errors: Gosh Windows I completely forgot about that...I'll take a look tonight.
Re: Article: Writing Julia style multiple dispatch code in D
On Wednesday, 30 August 2017 at 17:16:59 UTC, data pulverizer wrote: On Wednesday, 30 August 2017 at 17:14:37 UTC, data pulverizer wrote: On Wednesday, 30 August 2017 at 16:45:19 UTC, data pulverizer wrote: You mentioned Julia in your article, however for clarity I would point out that Julia doesn't have OOP-type polymorphism. There is no notion of being able to do something like: Animal snoopy = new Dog(); p.s. my bad, I was wrong about that! Turns out you can do something like this in Julia (apologies for the Julia code in a D forum): abstract type Animal end struct Dog <: Animal end struct Cat <: Animal end x = Array{Animal}(3) x[1] = Cat(); x[2] = Dog(); x[3] = Cat(); x # returns 3-element Array{Animal,1}: Cat() Dog() Cat() p.p.s typeof(x[1]) # returns Cat so it isn't really polymorphism - the object is never converted to the "parent" type! Lol ... sorry for the confusion! Which is polymorphism After mulling over this example, I don't see how this proves that Julia does *not* support run time polymorphism. On the contrary. If you translate this to D you get the same result by the way: import std.stdio; class Animal {} class Cat : Animal {} void main() { Animal[] array; array ~= new Cat(); writeln(typeid(array[0])); // typeid.Cat }
Re: Article: Writing Julia style multiple dispatch code in D
On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer wrote: In the light of this I think your package just became more interesting to me. I think that your work and mine are complementary :-)
Re: Article: Writing Julia style multiple dispatch code in D
On Wednesday, 30 August 2017 at 22:30:12 UTC, data pulverizer wrote: On Wednesday, 30 August 2017 at 22:10:38 UTC, Jean-Louis Leroy wrote: On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer wrote: In the light of this I think your package just became more interesting to me. I think that your work and mine are complementary :-) Here is one strange difference between inheriting from an interface and a class: ``` interface Animal{} class Dog: Animal{} class Cat: Animal{} void main() { Animal[] x; x ~= new Cat(); x ~= new Dog(); x ~= new Cat(); writeln(typeid(x[0])); // Gives Animal } ``` But if Animal is set to a class the typeid gives Cat, why does this happen? Does this mean that inheriting from an interface is not really polymorphism? I noticed that too. Still scratching my head.
Re: Open Methods: From C++ to D
On Wednesday, 30 August 2017 at 18:16:47 UTC, jmh530 wrote: ..\..\dubFolder\openmethods.d-1.0.0-rc.1\source\openmethods.d(970,21): Error: ca nnot implicitly convert expression h of type ulong to uint ..\..\dubFolder\openmethods.d-1.0.0-rc.1\source\openmethods.d(1076,34): Error: c annot implicitly convert expression dim of type ulong to uint ..\..\dubFolder\openmethods.d-1.0.0-rc.1\source\openmethods.d(1177,23): Error: c annot implicitly convert expression h of type ulong to uint dmd failed with exit code 1. The error at line 1076 can be fixed by changing the type of dim in the function to size_t. I couldn't fix the other errors. I tried having the hash function return size_t also, but that just causes other problems. Fixed. Committed to master and it should show up in dub soon. Gosh, all that mind bending meta polymorphic mixin reflection multi-dimensional fu and then fall prey to ints and uints and size_ts. Sobering...
Re: Open Methods: From C++ to D
On Thursday, 31 August 2017 at 11:39:30 UTC, aberba wrote: Thanks for this library. Just a suggestion. Would it possible to use `@openmethod` instead of `@method`? alias openmethod = method; Atila What happens when there is UDA name collision? if its catastrophic, then @openmethods makes it unique. After tightening a few screws, the library now supports static and selective imports.
Re: Article: Writing Julia style multiple dispatch code in D
On Thursday, 31 August 2017 at 06:58:53 UTC, Petar Kirov [ZombineDev] wrote: The workaround is to cast to Object before getting the typeid. The cause for this behavior is that if you have an interface reference to an object it points to the interface vtbl and not to the Object base class vtbl. Yeah I know. And in my openmethods lib I simply follow the pointers, without relying on typeid. It does look to me like a bug though. D has all the info it needs to implement the process you describe.
Re: Open Methods: From C++ to D
On Thursday, 31 August 2017 at 14:52:43 UTC, jmh530 wrote: On Wednesday, 30 August 2017 at 15:59:32 UTC, Jean-Louis Leroy wrote: What happens here is that kick(Animal) is shadowed by kick(Dog). kick(Animal) is a method but it appears to the user and the compiler as an ordinary function - which is generally good. As such it is eligible for UFCS. I would not recommend this sort of coding, but it's everyone's choice, methods or not. Likewise, methods can be overloaded (like here https://github.com/jll63/openmethods.d/blob/1.0.0-rc.1/examples/matrix/source/matrix.d#L12). A current limitation is that default arguments are not supported (yet), although I think it's just a matter of putting the effort in. UFCS interacts nicely with methods because you can say a.plus(b) even if 'plus' is an open method. I had a chance to try out what I had suggested above and it behaves exactly as I would have expected (i.e. it prints the line "lassie.kick(): ctbark"). You seemed to emphasize UFCS in your response, but that really wasn't what I was intending to focus on. I just as easily could have re-written Dog as below and compiled the program and it would have printed the same thing. Similarly, any Dog or Pitbull type that call kick would return "ctbark", just Animals would return the original results. class Dog : Animal { final string kick() { return "ctbark"; } } My point is one can easily mix your openmethods's dynamic dispatch and D's static dispatch. That seems like a great thing that you could emphasize. Simply stated: if you use openmethods, you're not forced to only use openmethods. If you know the type at compile-time, then you can use it. It's only if you want to dynamically dispatch it that you would need openmethods. Indeed I misunderstood. Well, I am very pleased that my stuff interacts well with the rest of the language - I strive for that. However, I found that it is difficult to get people to open their mind to the idea of open methods, initially. Unless they come from Lisp, polymorphism and membership are almost indissociable for them. I often have to jump three hurdles. 1/ They're multi-methods, and I never actually had a use for that. That is why I insist so much on openness in the article, and throw multiple dispatch in as a bonus only half way through. That's also why I call them "open methods" and not "multi-methods" or "open multi-methods". 2/ It's just function overloading. Hmmm, polymorphism? But once I get past that, it's actually a good thing. People know (more or less) how overload resolution (or partial template specialization for the more expert) works. So I don't need to explain the rules governing dispatch and ambiguities in an abstract way. Usually I just say "you already know which override will be picked - it's the same as with compile-time overload resolution". 3/ This one is specific to D - UFCS gives me the same thing. Hmmm, polymorphism again? But you see why I am very careful with anything that may obscure or confuse the message. I find the interaction of open methods and UFCS particularly cool when implementing the "call chain" idiom (e.g. a.plus(b).times(c).invert()).
Re: Open Methods: From C++ to D
On Thursday, 31 August 2017 at 20:42:36 UTC, EntangledQuanta wrote: On Wednesday, 30 August 2017 at 18:16:47 UTC, jmh530 wrote: On Wednesday, 30 August 2017 at 15:59:32 UTC, Jean-Louis Leroy wrote: What happens here is that kick(Animal) is shadowed by kick(Dog). kick(Animal) is a method but it appears to the user and the compiler as an ordinary function - which is generally good. As such it is eligible for UFCS. I would not recommend this sort of coding, but it's everyone's choice, methods or not. Likewise, methods can be overloaded (like here https://github.com/jll63/openmethods.d/blob/1.0.0-rc.1/examples/matrix/source/matrix.d#L12). A current limitation is that default arguments are not supported (yet), although I think it's just a matter of putting the effort in. UFCS interacts nicely with methods because you can say a.plus(b) even if 'plus' is an open method. I can submit this as an issue on the github page, but I figured I'd mention it here in case there was some easy fix. I tried installing the latest release from github. Compiling (Windows 7 on DMD with default options) the simple program below import openmethods; mixin(registerMethods); void main() { } gives me the errors: ..\..\dubFolder\openmethods.d-1.0.0-rc.1\source\openmethods.d(970,21): Error: ca nnot implicitly convert expression h of type ulong to uint ..\..\dubFolder\openmethods.d-1.0.0-rc.1\source\openmethods.d(1076,34): Error: c annot implicitly convert expression dim of type ulong to uint ..\..\dubFolder\openmethods.d-1.0.0-rc.1\source\openmethods.d(1177,23): Error: c annot implicitly convert expression h of type ulong to uint dmd failed with exit code 1. The error at line 1076 can be fixed by changing the type of dim in the function to size_t. I couldn't fix the other errors. I tried having the hash function return size_t also, but that just causes other problems. I was getting similar errors and simply added a cast(size_t)[used in the indexing, as he used ulongs for indexes rather than size_t] to all those you mention. After that I got more errors that I can't recall now but was much more cryptic. I did updateMethods and added the mixin but things wern't working so I gave up. Seems like a nice idea, although, the downside that I see is one doesn't get encapsulation. It's fixed now, in master and in release v1.0.0-rc.2. Actually not getting encapsulation is good. With vfuncs, if you want polymorphism you get access to private parts, need it or not. And most of the time you neither need nor want it. If you need polymorphism and privileged access, you should use a vfunc but it's usually a sign of bad design, because a vfunc is meant to be overridden. And the override won't have access to the private parts so you may end up changing access from private to protected and usually trouble follows. I can imagine legitimate cases though. Fox example, the DiagonalMatrix addition example. In that case you can write a public final member function that performs addition using privileged access and call that from the 2-method 'plus'. ANother approach is to write the fvunc - or the 1-method - in terms of the public interface. Usually it's feasible and yields a better design.
Re: Open Methods: From C++ to D
On Thursday, 31 August 2017 at 21:42:50 UTC, EntangledQuanta wrote: It's fixed now, in master and in release v1.0.0-rc.2. I'll check it out. I don't think the last errors I was getting were due to the sizing issues though, so is that all you fixed or was there some other stuff related to windows? Only size issues. Two lines in fact, see https://github.com/jll63/openmethods.d/commit/b63a88132e639bb23bb7cb305f4457450f865c6a but errors can cascade. I ran a few examples using the current dmd on Windows. Worked OK. It would be nice to have the Windows equivalent of dev/run-everything, maybe someone can PR me that? Yeah, but one should always be allowed to shoot themselves in the foot. I agree, wholeheartedly. In C++, yomm11 has macros that you can use to make a specific override or an entire method friend of a class. But alas no friendship in D.
Re: Open Methods: From C++ to D
On Thursday, 31 August 2017 at 23:37:03 UTC, EntangledQuanta wrote: [Windows] I'll try again at some point. I haven't got around to messing with it again. Did you get a chance?
Re: Open Methods: From C++ to D
On Saturday, 2 September 2017 at 20:55:13 UTC, EntangledQuanta wrote: This is when I have the mixin(registerMethods) in a module that doesn't use any open methods. It says add once per module in the help, but I think it means once per module where open methods are used? Yes I meant that. The README.md says "Every module that declares methods or define implementations must include the following line". Ah yes the ddoc. I should update it. Also I think I should allow the mixin to silently do nothing in this case. Good catch. You might look in to adding updateMethods in a static this() since it will be ran per process and do everything necessary, I think. Alas it won't work. Method registration is done via static ctors and they have to run - all of them - before updateMethods can do its work. In simple, one-module programs updateMethods in a static ctor will work, but in general it won't. Strangely enough, I had a protected member that I was using and it works, I guess because I defined the openmethod in the same module. I changed it to private and it worked too. So the issues about encapsulation I thought about before may be irrelevant as long as the openmethods are used in the same module(which is a nice feature of D). Neither the methods nor their overrides enjoy special privileges. Unless the override (i.e. the thing preceded by @method) is a static member function? But I don't think so. Currently my code just scans the direct member of the module in which mixin(registerMethods) is called. Although I could change that, thus giving privileged access to some overrides. Could be useful for 1-methods. But anyway, probably there's something you don't notice...hmmm...can you share that code? Thanks for the feedback.