Ah! Now we are getting somewhere!
Leopold Toetsch wrote:
Sam Ruby <[EMAIL PROTECTED]> wrote:
Leopold Toetsch wrote:
So *all* lookups (complete with the asterisks) does not mean *all* lookups.
How about <invoke>?
Let's first concentrate on simpler stuff like infix operators.
OK, but the point is that there will always be multiple mechanisms for dispatch.
Citing S06: "Operators are just subroutines with special names."
That statement is true for Perl. Same statement is true for Python. But the names vary based on the language.
Yes. So let's factor out the common part and have that in Parrot core, usable for Python and Perl and ...
The PyInt PMC currently duplicates almost all functionality that *should* be in the Integer PMC. We have first to fix the Integer PMC to do the Right Thing. Then we need some syntax for multiple inheritance in PMCs. The same holds for other PMCs. It was already proposed that we should have a language-neutral Hash PMC.
No question that that is the intended final goal. What you see in the current python dynclasses is not representative of that final goal.
So, why have I proceeded in this manner? Two reasons.
First, I am not about to make random, unproven changes to the Parrot core until I am confident that the change is correct. Cloning a class temporarily gives me a playground to validate my ideas.
Second, I am not going to wait around for Warnocked questions and proposals to be addressed.
Now, neither of the above are absolutes. You have seen me make changes to the core - but only when I was relatively confident. And I *have* put on hold trying to reconcile object oriented semantics as this is both more substantial and seemed to be something that was likely to be addressed.
Also, while I am not intending to make speculative changes to the core of Parrot, I don't have any objections to anybody making changes on my behalf. If you see some way of refactoring "my" code, go for it. It isn't "mine" - it is the community's.
The one thing I would like to ask is that test cases that currently pass continue to pass. The dynclass unit tests are part of the normal test. Additionally, the tests in languages/parrot have been the ones driving most of my implementation lately.
I do realize that that means checking out Pirate. Even though I don't agree with it, I do understand Michal's licensing issues. The reason I am not investing much time in resolving this issue is that Pirate is exactly one source file and could quickly be rewritten using the Perl 6 Grammar engine once that functionallity becomes sufficiently complete.
So given that we have a set of language-neutral PMCs in core that do the right thing, Python or Perl PMCs can inherit a lot of functionality from core PMCs. Language-specific behavior is of course implemented in the specific PMC.
Agreed. One area that will require a bit more thought is error cases. The behavior of integer divide by zero is likely to be different in each language. This could be approached in a number of different ways. One is by cloning such methods, like I have done. Another is to wrap such methods, catch the exception that is thrown, and handle it in a language specific manner.
A better approach would be for the core to call out to a method on such error cases. Subclasses could simply inherit the common core behavior and override this one method. It also means that the "normal" execution path length (i.e., when dividing by values other than zero) is optimal, it is only the error paths that involve extra dispatches.
That's an easy case. Overflow is a bit more subtle. Some languages might want to wrap the results (modulo 2**32). Some languages might want an exception. Other languages might want promotion to BigInt.
Even if promotion to BigInt were the default behavior, subclasses would still want to override it. In Python's case, promotion to PyLong (which ideally would inherit from and trivially specialize and extend BigIt) would be the desired effect.
Even this is only one aspect of a more general case: all morphing behavior needs to be overridable by subclasses. I believe that this can be easily handled by the current Parrot architecture by virtue of the fact that destination objects must be created before methods are called, and such destination objects can override morph methods). But it would help the cause if code were written to promote things to "Integer" instead of "PerlInt". Yes, at the moment, I'm guilty of this too.
Second: method dispatch. I've looked a bit into PyObject. It seems that you start rolling your own method dispatch. Please don't get me wrong, I'm not criticizing your implementation. It might also be needed for some reasons I'm just overlooking and it's currently needed because core functionality isn't totally finished.
I'll address your questions below, but for reference, here is the code that Pirate generates for "a=b+c":
find_type $I0, 'PyObject'
new $P0, $I0
find_lex $P1, 'b'
find_lex $P2, 'c'
$P0 = $P1 + $P2
store_lex -1, 'a', $P0What this means is that the __add__ method will not be directly used for either PyInt or PyString objects, instead Parrot mechanisms for dispatching this operation (currently MMD_ADD) will be used. Ultimately, I hope that this will help with interlanguage interoperability.
Anyway - and please correct me if my assumptions are not true - I'll try to factor out the common part again.
You have in PyObject e.g.:
METHOD PMC* __add__(PMC *value) { PMC * ret = pmc_new(INTERP, dynclass_PyObject); mmd_dispatch_v_ppp(INTERP, SELF, value, ret, MMD_ADD); return ret; }
I see six issues with that kind of approach:
* The "__add" method should be in Parrot core. That's what I've described in the MMD dispatch proposal. * the method is returning a new PMC. This doesn't follow the signature of Parrot infix MMD operations.
Here I do think you are misunderstanding. The __add__ method with precisely that signature and semantics is defined by the Python language specification. It is (somewhat rarely) used directly, and therefore must be supported exactly that way.
* well, it's dispatching twice. First the "__add__" method for PyObjects has to be searched for then the mmd_dispatch is done.
What this means is that (in the somewhat rare case) where people write Python programs that call the __add__ method directly, they will see some additional execution path length overhead. And there is a piece that I haven't written yet that will do the reverse: if MMD_ADD is called on a PyObject that has not provided such behavior, then an any __add__ method provided needs to be called.
This gets to the core of the extended discussion that we have been having. Parrot can attempt to find the "one true" behavior for all such dispatches, and then get all current and future languages to adopt it; or Parrot can simply provide the raw machinery and common reusable components, and allow each language to provide their own two way mappings.
I believe that the former is ultimately a dead end. Hence, I advocate the latter.
* it'll very likely not work together with other HLLs. It's a python-only solution.
I don't expect other languages to call __getitem__ methods, for example. Nor do I expect Python programs to be aware of other language conventions. But I do expect PyDict objects to respond to both get_pmc_keyed requests and __getitem__ requests in exactly the same manner, and for Pirate generated code to issued calls to the Parrot get_pmc_keyed version whenever possible.
That's why I like Parrot's mechanisms to different from, and orthogonal to, all languages conventions.
* rolling your own dispatch still doesn't help, if a metaclass overloads the C<+> operation * code duplication
So how would I do it:
* prelim: above mentioned core PMC cleanup is done. Inheritance works: a PyInt isa(PyObject, Integer)
Agreed. One of my first patches was to make what little of PMC inheritance works today to function. More needs to be done here. Search for SKIP in t/dynclass/pybuiltin.t for an example.
* the core PMCs define methods, like your "__add__" except that our naming conventions is "__add". The Python translator needs just a translation table for the common core methods.
In the general case, looking for "reserved" method names at "compile" time doesn't work. As everything can be overridden, this dispatch must be done at runtime.
* Method dispatch is done at the opcode level.
add Px, Py, Pz
just does the right thing. It calls the thingy that implements the "__add" method, being in core or overloaded shouldn't and doesn't matter. If inheritance changes at runtime it just works.
And the other way round:
Py."__add"(Pz, Px)
is the same. Again it doesn't matter, if it's a core PMC, a Python PMC or an overloaded PASM/PIR multi sub (or a Python metaclass). The only difference is the changed signature. But that's how Parrot core defines overloaded infix operations.
Python classes will still need an __add__ method. And ultimately, you will find that you have simply replaced the MMD_ADD integer with a string, but still be faced with the same multi-dimensional dispatch problem with inheritance, which hasn't been solved.
We have to do that anyway. It's just the correct way to go.
(And please no answers WRT efficiency ;-)
I do believe that performance and functionallity must be considered together. Clearly, "a=b+c" and "a=b.__add__(c)" should make use of common code, but I will unabashedly state that I am willing to optimize for the former at the expense of the latter. Furthermore, I'm willing to optimize for integer addition and string concatenation at the expense of generalized operator overloading. Particularly as that expense is a single dispatch, and therefore not generally significant in such cases.
IMHO, where prior discussions went off the rails were when significant architectural changes were proposed in the face of results of microbenchmarks that may or may not be indicative of real world usage.
I personally don't think that performance considerations should be out of bounds in these discussions, but that the focus needs to be on providing initial implemenations of real world languages on top of the existing infrastructure so that real tradeoffs can be made.
leo
- Sam Ruby
