Re: Python PMC's
Leopold Toetsch wrote: Sam Ruby wrote: Let me try again to move the discussion from subjective adjectives to objective code. Consider: [ example code ] If you run this, you will get 1,2,3. When called as a function, f will return the value of the second parameter. When called as a method, the same code will need to return the value of the first parameter. The calls to f() work more or less w/o problems in branches/leo-ctx5. I'm open to suggestions as to what PIR code should be emitted by Pirate to correspond to the above code. A stripped down PIR-only, pythonless translation is below. Thanks for doing this. Keeping the conversation anchored with code is very helpful. I have a number of unimportant to the topic at hand comments on that code (example: classes aren't global in Python), but for now, I'll focus on comments that are relevant to the topic at hand. The way this currently works with Pirate and the existing Py*.pmc closely follows the Python definition, which you can find here: http://users.rcn.com/python/download/Descriptor.htm#functions-and-methods Good explanation, thanks. To illustrate how this works today, lets consider the call to foo.f(2) above. This involves both a find_method and an invoke. Lets consider each in turn: * pyclass.find_method first calls getprop on f. * pyclass.find_method then calls __get__ on the pyfunc returned, passing the object on which this method was called on. * pyfunc.__get__ creates a new PyBoundMeth object, saving both the original function and the object. * This PyBoundMeth object is then returned then: * pyboundmeth.invoke shifts all registers right, inserts the original object as the first parameter, and then calls invoke on the original function. Needless to say, this doesn't necessarily show off Parrot to its best advantage from a performance perspective. I tried a number of times to argue for a combined find_and_invoke VTABLE entry as much of the above can be optimized if you know that result of the find_method will only be used a single time for an invoke. If you like, you can scan the archives to see what reception this suggestion received. Well I think that the find_and_invoke is just the callmethodcc opcode as used in my translation below. The call to __get__ and the BoundMethod shouldn't be necessary, *if* their is no userdefined descriptor. That is, when getattribute checks, if the attribute provides __get__ then call it. Take a look at the definition of callmethodcc in ops/object.ops. It is a find_method followed by an invoke. It is the VTABLE operations that I would like to see combined. Note that Python provides __get__ methods for you on every function and method, i.e., it is *not* optional, getattribute will always succeed. Observe: [EMAIL PROTECTED]:~$ python Python 2.4.1 (#2, Mar 30 2005, 21:51:10) [GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2 Type help, copyright, credits or license for more information. def f(): pass ... f.__get__ method-wrapper object at 0xb7e0404c class c: ... def m(self): pass ... c.m.__get__ method-wrapper object at 0xb7e0440c The yet unsopported thing is g = foo.g ... g(3) which really needs a BoundSub object. But this is just a special case of currying, or a special case of Perl6's .assuming(). Therefore I think that Parrot should support this natively. I agree that BoundSub is a special case of currying, or what Perl6 calls assuming. I also agree that Parrot should support this natively. - Sam Ruby [snip] Below is from the sample that Leo provided. # print foo.f(2) # emulate python find_name, which checks attributes too push_eh m_nf $P0 = foo.f(2) clear_eh goto m_f m_nf: # getattribute would also check if __get__ is there $P1 = getattribute foo, f $P0 = foo.$P1(2) m_f: print_item $P0 print_newline Note that the code above would need to be emitted for *every* method call, as there is no way of knowing at compile time what property you will get and how it is defined. In addition to increasing code size, it is error prone: for example, the above only appears to check for __get__ if the call fails, so the pie-thon b0 test will fail as hooking __get__ is integral to the way it provides a trace. Also, there are other reasons that exceptions may be thrown, and we are incurring the overhead of setting up exception blocks, etc again, this would apply to *every* method call. An alternate solution without these problems would be # print foo.f(2) $P0 = foo.f(2) print_item $P0 print_newline Ideally, callmethodcc would result in one vtable call to enable interesting optimizations for those that care to provide it. The default implementation for this vtable entry could be a call to find_method and invoke. - Sam Ruby
Re: Python PMC's
Leopold Toetsch wrote: Sam Ruby wrote: Leopold Toetsch wrote: A stripped down PIR-only, pythonless translation is below. (example: classes aren't global in Python), Yes, of course. The stripped down means essential the absence of any lexical handlings. But as you say, this doesn't matter for these sub and method calls. Agreed. Note that Python provides __get__ methods for you on every function and method, i.e., it is *not* optional, getattribute will always succeed. That's fine. But if it's not overriden by user code, you exactly know what it is doing. Therefore you can just emulate it, I think. I'm not as sure as you appear to be, but we can worry about this later. # print foo.f(2) # emulate python find_name, which checks attributes too push_eh m_nf $P0 = foo.f(2) clear_eh goto m_f m_nf: # getattribute would also check if __get__ is there $P1 = getattribute foo, f $P0 = foo.$P1(2) m_f: Note that the code above would need to be emitted for *every* method call, Above snippet should not be emitted for a method call, it just emulates it in PIR. It should demonstrate how Python's find_method() could be implemented: With that clarification, I agree in principle. - try to find a method, else - check attributes - call __get__, if it is user provided (or whatever order CPython actually uses). What's in dynclass/pyclass.pmc matches Python's semantics closer. The return value is a callable sub. More precisely: a curried function call. This is an important distinction; to see why, see below. Ideally, callmethodcc would result in one vtable call to enable interesting optimizations for those that care to provide it. The default implementation for this vtable entry could be a call to find_method and invoke. The interesting optimizations are: - cache the find_method in the global method cache. This happens already, if the method string is constant. Note that you would then be caching the results of a curried function call. This result depends not only on the method string, but also on the particular object upon which it was invoked. - use a PIC [1] (which is tied to the opcode) and cache the final resut of find_method (or any similar lookup) Again, the results will depends on the object. - run the invoked Sub inside the same run loop - specifically not within Parrot_run_meth_fromc_args I don't understand this. (Note: it may not be important to this discussion that I do understand this - all that is important to me is that it works, and somehow I doubt that Parrot_run_meth_fromc_args cares whether a given function is curried or not). [1] polymorphic inline cache The important thing is that the method lookup and the subroutine invocation happens from inside the runloop. Run cores that allow rewriting of opcodes (prederefed and JIT) can insert faster equivalent opcodes in these cases. Have a look at src/pic.c (which doesn't implement a lot - it's more a proof of concept now). Anyway here is an example that is implemented: new P0, PyInt# new_p_sc When this opcodes gets execute the first time, the type number of the class is looked up and then the opcode is replaced with the faster variant: new P0, 89 # new_p_ic - arbitrary number for PyInt The same scheme can be applied for all these opcodes that consist of any kind of a lookup (MMD, methods, attributes, ...) and some further action. The lookup is done once at runtime (and again only after cache invalidation). The above works because PyInt is a constant. It probably can be extended to handle things that seem unlikely to change very rapidly. But the combination of decisions on how to handle the passing of the self parameter to a method, keeping find_method and invoke separated at the VTABLE level, and the semantics of Python make the notion of caching the results of find_method problematic. - Sam Ruby leo - Sam Ruby
Re: Python PMC's
Leopold Toetsch wrote: Note that you would then be caching the results of a curried function call. This result depends not only on the method string, but also on the particular object upon which it was invoked. No the inner Parrot_find_method_with_cache just caches the method for a specific class (obeying C3 method resolution order as in Python). There is no curried function at that stage. Where does Parrot_find_method_with_cache get this data? Remember, in Python, there are no methods, there are only properties. Of course, there is a find_method VTABLE entry, and the implementation of this function calls __get__ which performs the curry function, per the Python specifications. Does Parrot_find_method_with_cache cache the results of the previous call to find_method? - Sam Ruby
Re: Python PMC's
Leopold Toetsch wrote: Above Parrot interface function tries to locate Sub objects in the namespace of the invocant's class and in the MRO of it. When you go back to the PIR translation of your code there is e.g. class Foo: def g(self,y): I have translated this to: .namespace [Foo] .sub g .param pmc self Therefore all the static / default / common methods inside Python code would be callable with the plain Parrot method call syntax. The classname contributes the namespace. The sub declaration creates an entry in that namespace, which is retrievable as: func = findglobal Foo, g Honestly, I don't believe that that is workable for Python. Modules are global in Python. Classes are lexically scoped. All subs emitted by Pirate are @anon. Modules are namespaces and can contain classes, functions, and variables. The way to retrieve a method from a Python class defined in module __main__ would be: $P1 = findglobal __main__, Foo getattribute $P2, $P1, 'g' - Sam Ruby
Re: [pirate] Python PMC's
Kevin Tew wrote: Problem: Python PMC's just don't work in the leo-cxt5 branch which will become head/trunk at some time in the hopefully not to distant future. What I've done up time now: I've ported pyint.pmc, pystring.pmc to pass all tests in leo-cxt5 I've written t/dynclasses/pystring.t Note that there are something like 15-20 python.pmc's or which only 4 have corresponding test in I've written t/dynclasses/ I've ripped out a lot of the explicit passing of self as the first argument of pmc methods. - We don't have to pass self around, parrot makes it available to us automatically. I added self on Leo's request. Now it is unneccessary. *shrug* Sam's implementation of Python PMC is old and outdated. He did a lot of pioneering work in the realm of the python object system on parrot. Probably the first real attempt at a full object system on top of parrot. Parrot has changed drastically since then. Sam's PMC's were also incomplete, they lacked support for a lot of the __add__ style python methods. This is quite understandable, Sam was trying to get something working. Check out parrot/t/dynclass/pyint_2.pmc. __add__ style methods were working, and tested. Proposal: I propose to gut all of PyObject, PyFunc, PyClass, PyType PMC's etc. Old New PyObject PyObject - will represent a instantiation of a python class, will inherit from ParrotObject PyClassPyClass - will represent a python class, will inherit from ParrotClass PyType PyMetaClass - will represent the meta-class of a class, this one is still open for debate. I plan on making these changes via test driven development. The plan is to reduce python object system pmc's code content and functionality to zero, and then rebuild it up Mainly because every time I try to morph Sam's code, Leo tells me the whole approach is wrong :) We will rebuild the python object system based on test cases, If functionality is missing, and you want it implemented, write and submit a test case first. No matter what you do, Leo will tell you that the whole approach is wrong. I talked this past weekend with Larry Wall, Chip Saltzberg, and chromatic. Hopefully we will get some more people involved. Leo is great at details, but has a tendency to set a new direction every week. I say that knowing that Leo follows this list. (Hi Leo!) I do agree with test-driven development. That is exactly the approach I took. I will also state that I learned a lot *ABOUT* *PYTHON* in running the pie-thon tests. I'd recommend that you don't gut until you try the following: * PirateTest.py is a good sanity test to say that you have something resembling python to work with. Until that test passes, there isn't much point. * parrot/t/dynclasses give you a similar level of confidence in the pmcs. * Once you get that far, cd to parrot/languages/python and issue make. That will run the various tests (culiminating in the pie-thon tests themselves). To run that test, you will need a small batch file in your search PATH named pirate. Mine looks like: export LD_LIBRARY_PATH=/home/rubys/runtime/parrot/dynext python /home/rubys/pirate/pirate.py $* One last design note. I had a conversation a long time ago with Dan, in which he agreed that given Python's property bag style object system. That class methods and data members should be stored in parrot properties and the vtable getattribute/setattribute methods for python pmcs should be overridden to use parrot properties instead parrot attributes. Sam's PMC appeared to follow the same idea though not fully flushed out, and I plan on building the new Python PMC in the same manner. It just might be more fully flushed out than you appreciate. Comments welcome. I'm following this list, and questions are welcome in return. Again, my suggestion is to try to get the existing tests to work again - modifying the tests as necessary to remove now unnecessary self parameters. And, again, resist the temptation to follow Leo's advice du jour. Focus on what is actually implemented - while that does change over time, running code has a tendency to change more slowly than the advice you may receive. Kevin Tew ___ pirate mailing list [EMAIL PROTECTED] http://cornerhost.com/mailman/listinfo/pirate - Sam Ruby
Re: [pirate] Python PMC's
Leopold Toetsch wrote: I do agree with test-driven development. That is exactly the approach I took. Well, I agree - until I see such tests (pyint_1): $P2 = $P1.__abs__($P1) Take a look at the difference between r7254 and r7312. The tests originally passed without a self argument. Then I was told that that was all wrong, so I changed it (both the test and the code). Now Kevin is being told that passing self is all wrong. Reverting that particular change is not all that difficult. And it can be done without gutting. - Sam Ruby
Re: [pirate] Python PMC's
Kevin Tew wrote: Point well taken, I've been leaning back between, rewrite and evolving your work. I have a current change set that is passing most of your original tests that are in t/dynclasses/py*.t It is more of an evolution than a rewrite, which I favor. Some test fail due to remaining uunimplemented features in leo-cxt5(default args), or my lack of understanding. But a lot of the tests pass. Take a look at the following: Python 2.4.1 (#2, Mar 30 2005, 21:51:10) [GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2 Type help, copyright, credits or license for more information. def f(a,b=1,c=2): ... pass ... f.func_defaults (1, 2) In December, this was fully implemented and tested. In pirate.py, search for self.append(setprop %s, 'func_defaults', %s % (ref, defaults)) and search for func_defaults in pyfunc.pmc, pyboundcall.pmc, and pybuiltin.pmc. Finally, take a look at the third test in parrot/languages/python/t/basic/func.t - Sam Ruby
defaults (was: Python PMC's)
How will Perl6 evaluate defaults? Like Python: global x x=1 def f(p1=x): return p1 x=2 print f() or like Ruby: $x=1 def f(p1=$x) return p1 end $x=2 puts f() - Sam Ruby P.S. The former prints 1, the latter, 2.
Re: Python PMC's
Leopold Toetsch wrote: On Aug 23, 2005, at 22:48, Sam Ruby wrote: From December 16, 2004: http://tinyurl.com/8smmq Sounds like a really ugly misunderstanding, the more that I've proposed not to pass the object (P2 in old parlance) out of band. I've stated several times that calling conventions need changes to properly support HLLs with minor success at these times. With the diversity of HLLs out there, I'm not certain that it is wise to declare one set of semantics proper, thereby implicitly declaring all others improper. What I've always said (and what is actually implemented now in the branch) is that methods gets the object as its first argument. A typical snippet in PIR code is now .sub __abs .param pmc self or if denoted as method: .sub __abs method # 'self' param automagically available The first argument of NCI or vtable methods is and was always SELF, which usually is the invocant and always is the object (the current only excpetion is an invocation through super()). It might help to anchor this discussion if we focus on a tangible test case. In particular, lets look at the first few lines of the first pie-thon test, which can be found here: http://svn.perl.org/parrot/trunk/languages/python/t/pie/b0.t What code should be generated for the declaration of proto___new__ or proto___repr__ functions? What code should be generated for the various assignment statements in the declaration of the method __new__ inside the MetaToken class? Particularly, the one involving a staticmethod? The conclusion I came to is that there is no way to tell in advance how a given function or method in Python is ever going to be called as there effectively is no difference between the two. - Sam Ruby
Re: Python PMC's
Chip Salzenberg wrote: On Tue, Aug 23, 2005 at 07:15:41PM -0400, Sam Ruby wrote: Leopold Toetsch wrote: I've stated several times that calling conventions need changes to properly support HLLs with minor success at these times. With the diversity of HLLs out there, I'm not certain that it is wise to declare one set of semantics proper, thereby implicitly declaring all others improper. Well, let's try more coherent and less confusing, then, or maybe just better. HLLs don't have to expose the 'self' parameter if they don't want to. Just extract it into a PMC register that's never used. Let me try again to move the discussion from subjective adjectives to objective code. Consider: def f(x,y): return y class Foo: f = f def g(self,y): return y foo = Foo() g=foo.g print f(0,1) print foo.f(2) print g(3) If you run this, you will get 1,2,3. When called as a function, f will return the value of the second parameter. When called as a method, the same code will need to return the value of the first parameter. I'm open to suggestions as to what PIR code should be emitted by Pirate to correspond to the above code. The way this currently works with Pirate and the existing Py*.pmc closely follows the Python definition, which you can find here: http://users.rcn.com/python/download/Descriptor.htm#functions-and-methods To illustrate how this works today, lets consider the call to foo.f(2) above. This involves both a find_method and an invoke. Lets consider each in turn: * pyclass.find_method first calls getprop on f. * pyclass.find_method then calls __get__ on the pyfunc returned, passing the object on which this method was called on. * pyfunc.__get__ creates a new PyBoundMeth object, saving both the original function and the object. * This PyBoundMeth object is then returned then: * pyboundmeth.invoke shifts all registers right, inserts the original object as the first parameter, and then calls invoke on the original function. Needless to say, this doesn't necessarily show off Parrot to its best advantage from a performance perspective. I tried a number of times to argue for a combined find_and_invoke VTABLE entry as much of the above can be optimized if you know that result of the find_method will only be used a single time for an invoke. If you like, you can scan the archives to see what reception this suggestion received. Precisely emulating the behavior above is an important part of distinguishing a Python-like language from a true Python language, and is necessary to pass the b0 pie-thon test (grep for __get__ to see what I mean). As to whether this is more coherent and less confusing, then, or maybe just better... well, lets just say that I'll leave the adjectives to others. - Sam Ruby
Re: Python PMC's
Chip Salzenberg wrote: I apologize to Leo for accidentally making this reply to the list. It was supposed to be private mail, but I hit 'y' just a _little_ too soon. I had no intention of embarassing anyone. Sorry. You did, however, cause me to cancel the email I was composing. If people only respond in private to emails such as the one that you responded to, the public perception is that emails such as Leo's are acceptable. In any case, here is a shorter version of what I said in the email I cancelled: From December 16, 2004: http://tinyurl.com/8smmq The commit message for December 17, 2004: r7312 | rubys | 2004-12-17 22:51:48 -0500 (Fri, 17 Dec 2004) | 2 lines Pass 'self' as the first argument on method calls - Sam Ruby
Re: [pirate] OSCON slides
Leopold Toetsch wrote: Citing 33.html: Parrot's current implementation relies on the ability to morph an object to another type. Leo has attempted to challenge this a number of times, but to my knowledge never successfully. I'm a bit bewildered that pirate folks seem not to bother following parrot dev list [1] or even summaries. For short: n_add d, l, r will create a new destination PMC and with the help of .HLL a new Python PMC. See also t/dynclass/pycomplex_3 and _4. (This is just one point of several from the slides of what I'm inclined to name FUD) leo E.g. [1] http://xrl.us/gz5b At the end of the presentation, a person in the front row asked me if I planned to continue to contribute to the project. My response was something along the lines of, I appreciate having been given the opportunity to contribute, I hope I fixed more things than I broke, but I probably won't contribute in any major way again unless I see signs of things turning around. From the back row, Chip introduced himself. At which point, I ammended my previous statement saying that Chip was in a position to make a difference. My experiences with Parrot was that everything I did was met with either direct animosity or by being ignored. If that changed, I could see myself becoming active again. In the past few weeks, I have received two pieces of feedback from the Parrot community. One was attacking me for having adopted the suggestion by Leo that the object of the method should be passed on the first argument of method call. The second was to attack me for saying something nice about morphing. I said in slide 3 that this presentation was based mostly on memory. In slide 4, I said that Parrot is evolving. I said in slide 31 that my memory was based on the work I did in December of 2004. I said in slide 33 that Leo was working to change how morphing was being used in Parrot, but my real point was made in slide 36: morphing was cheap and efficient in practice. My research prior to the presentation was to check to see if morphing was still in the VTable, and it is. For now, I'll probably limit the amount of time I devote to Parrot as there are plenty of other places where I feel more welcome. - Sam Ruby
Re: Python on parrot
Michal wrote: On Sat, 16 Apr 2005, Sam Ruby wrote: [I hope you don't mind me putting this back on the list - I would prefer that everybody who is interested can follow along and/or participate] Kevin Tew wrote: Sam Ruby wrote: Hey guys, I didn't see this until just now. Kevin Tew wrote: Sam, Just wondering what the status is on python/parrot/pirate/pyrate. These both look outdated. http://www.intertwingly.net/stories/2004/10/05/pyrate.zip http://pirate.versionhost.com/viewcvs.cgi/pirate/ I haven't looked at it for a few months now. I do plan to revisit it enough to get the Pie-thon tests completed by the time of OSCON (in August). Sam, are you still interested in this? I had better be: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6833 Is there a up to date cvs repo? http://pirate.tangentcode.com/ Can we get this code checked into the parrot svn repo? Unfortunately, no. Much of this code is copyright Michal Wallace. The good news is that the good stuff is in the parrot repo already. What is left - a simple translator - can and should, IMHO, be recoded into Perl6 once enough of that is running. - Sam Ruby So why won't Michal Wallace sign over the copyright? I talked to Michal briefly about this a while back. My impression was that he wanted to sign over the copyright to the Python foundation. Which makes a bit of sense - Pirate is based on code created by Andrew Kuchling and the copyright on that work already belongs to the Python foundation. Mostly I think pirate is just a different project from parrot and I don't want to hand over control of which patches to apply, etc. So far, I've never rejected a patch or turned down anyone for cvs access, but I'd like to reserve the right. (Also, Dan and the parrot team had other ideas about how to implement python.) the goal of having everything run on a single runtime does not necessarily imply that everything has to be owned by a single organization and put into a single repository. Yes. That was the other main idea. I was hoping that we could leverage the objects that the python team had already created. Do PMC's still have to be compiled inside the parrot source tree? To me, the problem is that the PMC's are in the parrot tree, not that the compiler isn't in there with it. Parrot (including the build system) is a moving target. Long term, the answer is no. Short term, the answer is yes. My own opinions is that Michal thinks too much. ;-) This is almost certainly true. :) My impression is that everybody here is reasonable, and if it made sense for further development to be transferred to another organization, then some reasonable arrangement would be made. I hope this is true, too. :) Also, I believe that much of the initial work is throwaway work anyway. Build one to throw away, and all that. I've never really bought that logic. :) It works, and (especially after your last batch of refactorings) the code is pretty clean. I would like to see the emitter code separated out into a builder object though. Cool I did notice all the python pmcs. By translator I assume you mean a interpreter/compiler to parrot byte code. At the moment, it is to Python to IMC, but eventually going directly to bytecode would be a good idea. That's fine with me, as long as parrots imc compiler does the work. :) Why would you do it in Perl6, why not self hosted in python? Self hosted in Python is a good idea, once it can be bootstrapped. I'm not sure what Kevin meant by self hosted but I've been thinking about this lately, and what we can do today is creating a back-end for the compiler that emits simplified python code in a way that would give us things like continuations (though with some loss of speed) without the need to duplicate the python library. Overview of the current process: 1) Leverage python's compiler class to convert source to AST 2) Pirate converts AST to IMC 3) Parrot converts IMC to bytecode I'm thinking of toying around with python and just want to leverage all the previous work. Excellent. In the meantime, if you are interested in getting commit access to the Pirate repository, I'm confident that that can be arranged. Sure. Kevin, if you're still interested let me know. My feeling, for what it is worth, the translator is known to be a solvable problem. Determining the proper mapping of Python semantics to Parrot is the research problem. The overwhelming majority of that work is in getting the PMCs right. Not having to worry about the syntax of python or the conversion to bytecodes allows one to focus on just that aspect of the problem. You're absolutely right. I finally had a chance to look into what you've done, both in the compiler and on the PMC: I'm impressed. To me the main concern is making the PMC's compatable (or at least similar to) python's C API. This is why I originally suggested wrapping the python library. I would like to be able to run things like pygame, numeric, wxpython, and extensions written with pyrex
Re: Python on parrot
Kevin Tew wrote: Sam, Just wondering what the status is on python/parrot/pirate/pyrate. These both look outdated. http://www.intertwingly.net/stories/2004/10/05/pyrate.zip http://pirate.versionhost.com/viewcvs.cgi/pirate/ I haven't looked at it for a few months now. I do plan to revisit it enough to get the Pie-thon tests completed by the time of OSCON (in August). Is there a up to date cvs repo? http://pirate.tangentcode.com/ Can we get this code checked into the parrot svn repo? Unfortunately, no. Much of this code is copyright Michal Wallace. The good news is that the good stuff is in the parrot repo already. What is left - a simple translator - can and should, IMHO, be recoded into Perl6 once enough of that is running. - Sam Ruby
Re: Python on parrot
[I hope you don't mind me putting this back on the list - I would prefer that everybody who is interested can follow along and/or participate] Kevin Tew wrote: Sam Ruby wrote: Kevin Tew wrote: Sam, Just wondering what the status is on python/parrot/pirate/pyrate. These both look outdated. http://www.intertwingly.net/stories/2004/10/05/pyrate.zip http://pirate.versionhost.com/viewcvs.cgi/pirate/ I haven't looked at it for a few months now. I do plan to revisit it enough to get the Pie-thon tests completed by the time of OSCON (in August). Is there a up to date cvs repo? http://pirate.tangentcode.com/ Can we get this code checked into the parrot svn repo? Unfortunately, no. Much of this code is copyright Michal Wallace. The good news is that the good stuff is in the parrot repo already. What is left - a simple translator - can and should, IMHO, be recoded into Perl6 once enough of that is running. - Sam Ruby So why won't Michal Wallace sign over the copyright? I talked to Michal briefly about this a while back. My impression was that he wanted to sign over the copyright to the Python foundation. Which makes a bit of sense - the goal of having everything run on a single runtime does not necessarily imply that everything has to be owned by a single organization and put into a single repository. My own opinions is that Michal thinks too much. ;-) My impression is that everybody here is reasonable, and if it made sense for further development to be transferred to another organization, then some reasonable arrangement would be made. Also, I believe that much of the initial work is throwaway work anyway. Build one to throw away, and all that. Cool I did notice all the python pmcs. By translator I assume you mean a interpreter/compiler to parrot byte code. At the moment, it is to Python to IMC, but eventually going directly to bytecode would be a good idea. Why would you do it in Perl6, why not self hosted in python? Self hosted in Python is a good idea, once it can be bootstrapped. Overview of the current process: 1) Leverage python's compiler class to convert source to AST 2) Pirate converts AST to IMC 3) Parrot converts IMC to bytecode I'm thinking of toying around with python and just want to leverage all the previous work. Excellent. In the meantime, if you are interested in getting commit access to the Pirate repository, I'm confident that that can be arranged. My feeling, for what it is worth, the translator is known to be a solvable problem. Determining the proper mapping of Python semantics to Parrot is the research problem. The overwhelming majority of that work is in getting the PMCs right. Not having to worry about the syntax of python or the conversion to bytecodes allows one to focus on just that aspect of the problem. But, as with all open source projects, feel free to scratch your own itches. Looks like I'll start with a translator and some new test cases so they can be contributed copyright free. There also are a fair amount of python test cases in the parrot repository. parrot/languages/python/*/*.t and t/dynclass/python/*.t. When I last looked, these were not complete. Undoubtably, there is likely to be some minor regressions as I have not kept up with the latest Parrot changes. If past history is any guide, this is not much of a problem. - Sam Ruby
Re: VTABLE methods and interfaces
Leopold Toetsch wrote: 2) Vtable entries should be real methods All non-defaulted, non-inherited entries of a vtable interface should be available as methods, like now with the METHOD keyword. This allows eventually code like this: Px = Py.__pop_pmc() or Px = Py.__string()# $x = ~$y string kontext This easily allows overloading of all the vtable methods. The PIC code gets even rid of the additional indirection as the function is called directly from opcodes. The vtable slots are just for calling these methods from within C source. As long as find_method itself can be overridden, this above is merely a description of the default behavior, not a hard requirement. 3) MMD functions They work basically like other methods except that there is no VTABLE slot and the method lookup is more complicated. The more detailed (and very preliminary) proposal below has some additional MMD functions that we'll need (IMHO): __i_add(Px, Py)# $x += $y Px = __n_add(Py, Pz) # x = y + z ; x is newly created The distinct slots for the inplace operations are at least necessary for Python, as these are separate methods in Python. It also simplifies the implementation as a check for self == dest isn't necessary in the plain operations. The variants with an n_ prefix create and return a new PMC. I continue to believe that it is not a good idea for Parrot to decide apriori exactly which methods are to be mmd universally for all languages and all objects. - Sam Ruby
Re: cvs commit: parrot/t/pmc object-meths.t
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
[EMAIL PROTECTED] wrote:
+else if (p-vtable-base_type == enum_class_NCI) {
It was requested[1] that I not add any Python specific methods to the
NCI method... accordingly, the majority of Python methods are morphed to
a PyNCI class which subclasses the base NCI class.
Well, I know that the above test isn't quite right. It should be:
if (VTABLE_isa(INTERP, p, CONST_STRING(INTERP, NCI)))
But that needs still more work - better PMC class inheritance in that
case. I've proposed to add an mro array to PMCs to simplify
inheritance checks.
[ for now, I've put in above line - should work ]
But then you effectively morph the resulting bound method into a NCI
instead of a PyNCI with the following line of code:
bound_meth-vtable = Parrot_base_vtables[enum_class_Bound_NCI];
Delegation is more appropriate in this instance, and would be able to
handle both NCIs and PIR methods with equal ease.
- Sam Ruby
Re: bound methods
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: The common cases I want to optimize for are common functions being called as common functions. And common methods being called as methods. Yep, that's very reasonable. The easiest way to optimize for the common methods being called as methods is if the current object is not passed as the first argument. Why so? Python methods don't have a notion of an object - it's just the first argument passed into a function. So I don't quite understand your conclusion. In the case of x.y(z) The caller is passing one argument. The recipient is expecting two. The recipient defines what is expected as the first argument (in class methods it is the class, in instance methods it is the instance). So, in the general case, the callee is responsible for inserting a parameter at the beginning of the PMC parameters. Insertion by setting a value into P2 is cheaper than insertion by shifting and then setting a value into P5. Yes, copying will be required in cases where functions are called as methods. How do you detect this case and when are arguments shifted then - note: I prefer the term shifting as copying is done anyway in all sub calls (see src/sub.c:copy_regs). I agree that shifting is a more appropriate term. At definition time, the PyFunc PMC has a number of properties (or flags) which are set defining names of the formal arguments, defaults, and the like. A similar approach can be used to define whether the first parameter is expected in P2 or P5. Note: this is not yet implemented for Python on Parrot. Note: different languages may chose different strategies as to when to shift. What is important is that we come to an agreement on what is expected of the caller. - Sam Ruby
Re: cvs commit: parrot/t/pmc object-meths.t
[EMAIL PROTECTED] wrote:
+else if (p-vtable-base_type == enum_class_NCI) {
It was requested[1] that I not add any Python specific methods to the
NCI method... accordingly, the majority of Python methods are morphed to
a PyNCI class which subclasses the base NCI class.
- Sam Ruby
[1] http://xrl.us/exsu
Re: bound methods
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
Leopold Toetsch wrote:
... But that doesn't
work fur user methods, especially if there is no indication that a user
function is used as a method in the first place.
def find(s, sub):
...
In Python, this is statically determinable. If that sequence is
directly nested inside a class, it is a method, otherwise it is a
function.
No, we had that several times. You did show examples, where this isn't
true. Here is another one:
def add(l, r):
print in add
return 42
class C(object):
__add__ = add
c = C()
print c + 1
add is just a plain function. There is no indication whatsoever that
it might be used as a method, when add is compiled. But as infix+ is
a method call l.__add__(r) this doesn't work, if the object is in P2.
I've proposed several times that arguments including the object should be
passed from P5 up. The invocant - if any - can still be reachable with
the Cinterpinfo opcode, but it'll be probably used just in calls to SUPER
or next_method or such.
Less shifting will be required if the object is passed in P2.
That's only true for NCI methods. And IIRC, your arguments were the same
some time ago. How do you compile the add above, when the object is in
P2?
Notes: at the moment, every Python method call creates a bound object,
and shifts PMC arguments.
This shouldn't be necessary for normal method calls like:
s.f(r)
But this translates into two VTABLE calls. find_method and invoke.
find_method needs to return a bound method.
Why don't you just use the callmethodcc opcode? That's exactly what is
happening here.
A find_method_and_invoke VTABLE entry (or more simply call_method) would
not need to return the intermediary bound method.
There is no need for an intermediate object, if it's a plain method
call.
If there were a call_method VTABLE entry and if P2 were passed into
methods, all of this would be unecessary in the majority of cases.
A separate vtable slot doesn't really help. Code that looks like a
function call can actually be a method call (and vv). Separating the
call into two vtables will just duplicate the call sequence. But let's
first convince Dan that all arguments are passed from P5 up, then we'll
see what we have.
Less shifting will be required if the object is passed in P2.
See above.
And why are you duplicating object arguments into P5, if the object
should go into P2?
,--[ dynclasses/pyint.pmc ]---
| METHOD PMC* __hex__(PMC *self) {
`-
You are inventing an additional self argument here just to work around
the problem that the object is passed in P2. The actual object pmc is
ignored:
,--[ dynclasses/pyint.c ]---
| PMC*
| Parrot_PyInt___hex__(Interp* interpreter, PMC* pmc, PMC *self)
`---
As said in another message, there is no problem with NCI methods. It's
just a matter of translating the O signature char.
The problem are user functions. There might be an indication that's a
method (like a definition inside a class block) - but not always. And if
the latter is the case *only once*, we just have to call all methods
with plain function call argument passing. Or prepend *all* functions
with an argument check wrapper that shift arguments around, if the call
was actually a method call.
The current CVS is in a state of transition.
I originally implemented to the current set of PDDs. Unhappy with the
amount of copying of PMC registers, I optimistically converted over to
the pass self as P5 approach that you suggested. I found that that
simply moved the problem to other places, and copying was unavoidable.
In fact, it generally made things worse.
Given that copying is unavoidable, what I would like to do is to
optimize for what I consider the most common cases, and make all the
remaining cases work properly.
The common cases I want to optimize for are common functions being
called as common functions. And common methods being called as methods.
The easiest way to optimize for the common methods being called as
methods is if the current object is not passed as the first argument.
Yes, copying will be required in cases where functions are called as
methods.
- Sam Ruby
Re: bound methods
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: WRT functionality: for a call it has to shift up PMC arguments and insert the object as P5, right? At the moment, it contains this logic. My plans are to remove the shifting and set the object into P2 / INTERP-ctx.current_object. Dan didn't answer issues WRT argument passing yet. In the current scheme your plan would be ok - for e.g. a NCI find method. But that doesn't work fur user methods, especially if there is no indication that a user function is used as a method in the first place. def find(s, sub): ... In Python, this is statically determinable. If that sequence is directly nested inside a class, it is a method, otherwise it is a function. I've proposed several times that arguments including the object should be passed from P5 up. The invocant - if any - can still be reachable with the Cinterpinfo opcode, but it'll be probably used just in calls to SUPER or next_method or such. Less shifting will be required if the object is passed in P2. Notes: at the moment, every Python method call creates a bound object, and shifts PMC arguments. This shouldn't be necessary for normal method calls like: s.f(r) But this translates into two VTABLE calls. find_method and invoke. find_method needs to return a bound method. A find_method_and_invoke VTABLE entry (or more simply call_method) would not need to return the intermediary bound method. ... Removing the shifting of PMC arguments will require the insertion of an interpinfo opcode into each method. Or fixing argument passsing like above. Which amounts to not removing the shifting. If there were a call_method VTABLE entry and if P2 were passed into methods, all of this would be unecessary in the majority of cases. A separate vtable slot doesn't really help. Code that looks like a function call can actually be a method call (and vv). Separating the call into two vtables will just duplicate the call sequence. But let's first convince Dan that all arguments are passed from P5 up, then we'll see what we have. Less shifting will be required if the object is passed in P2. - Sam Ruby
Re: bound methods
Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: At 5:04 PM -0500 1/18/05, Sam Ruby wrote: f = Parrot.find print f(r) Note that I referenced the method as an attribute, and then called it as a function. Mmm, syntax! :) Luckily it makes no difference to us at the parrot level. What that should translate to is something like: $P0 = find_method Parrot_string, find # Elided check for failed lookup and fallback to attribute fetch $P1 = make_bound_method(Parrot_string, $P0) Not quite. It's just: f = getattribute Parrot_string, find nothing more. The Cget_attr_str vtable has to do the right thing, i.e. if the attribute is a callable, it has to return a bound method object. Exactly. Furthermore, the function remembers what object it is bound to. This is accomplished by VTABLE_find_method creating a new PyBoundMeth PMC which contains two references, one to the object, and one to the method. While a good idea, I think it's not the right way to handle this. Binding objects to methods to create invokable subs is going to be something we're going to need for a lot of the languages, so I think we'd be better served providing a general facility to do it rather than leaving it to each individual language designer to do it. Should save some work all around too. Yeah. When this came up last, I've proposed two ways to handle it: 1) inside the Sub/NCI PMC 2) by a distinct Bound_Meth PMC class derived from 1) The latter is probably cleaner. Binding the object to the callable could be done e.g. by the Cset_pmc vtable. That's exactly how PyBoundMeth works today. Cset_pointer sets the pointer to the actual subroutine. Cset_pmc sets the pointer to the bound object. - Sam Ruby
Re: bound methods
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: 2) by a distinct Bound_Meth PMC class derived from 1) The latter is probably cleaner. Binding the object to the callable could be done e.g. by the Cset_pmc vtable. That's exactly how PyBoundMeth works today. Cset_pointer sets the pointer to the actual subroutine. Cset_pmc sets the pointer to the bound object. Great. I saw the checkin but didn't have a close look at it. So if even the interface is the same, we really should put it into the core classes. Cool. I'd likely still subclass it to provide a get_string. WRT implementation: I'd like to swap struct_val/pmc_val for all Sub classes. It's just cleaner but should be almost fully transparent to users of these classes. No objection. WRT functionality: for a call it has to shift up PMC arguments and insert the object as P5, right? At the moment, it contains this logic. My plans are to remove the shifting and set the object into P2 / INTERP-ctx.current_object. Notes: at the moment, every Python method call creates a bound object, and shifts PMC arguments. Removing the shifting of PMC arguments will require the insertion of an interpinfo opcode into each method. If there were a call_method VTABLE entry and if P2 were passed into methods, all of this would be unecessary in the majority of cases. - Sam Ruby
Re: Calling conventions, invocations, and suchlike things
Dan Sugalski wrote: At 5:04 PM -0500 1/18/05, Sam Ruby wrote: Dan Sugalski wrote: Hi folks. Welcome back! Parrot's got the interesting, and somewhat unfortunate, requirement of having to allow all subroutines behave as methods and all methods behave as subroutines. (This is a perl 5 thing, but we have to make it work) That is, an invokable PMC may be invoked as a method call and passed in an object, or as a plain subroutine and not have an object passed in. As far as perl 5 is concerned the object is the first parameter in the argument list, but for everyone else the object is a very distinct and separate thing. Python essentially has the same requirement, with a few twists. Specifically, methods come in be static, class, and regular flavors. But first, a simple example. Strings in python have a find method, so and can do the following: f = Parrot.find print f(r) Note that I referenced the method as an attribute, and then called it as a function. Mmm, syntax! :) Luckily it makes no difference to us at the parrot level. What that should translate to is something like: $P0 = find_method Parrot_string, find # Elided check for failed lookup and fallback to attribute fetch $P1 = make_bound_method(Parrot_string, $P0) $P1(r) This will be a recurring theme in my replies. Any thing which presumes a bit of knowledge at compile time will ultimately not work. Consider the following: class c: find = 7 def f(x): return x.find print f(c()) print f(Parrot)(r) Now, what should the code for function f look like? The only reasonable answer is something along the lines of: getattribute $P0, P5, 'find' This has to work. In both of the two calls to f(). Furthermore, the function remembers what object it is bound to. This is accomplished by VTABLE_find_method creating a new PyBoundMeth PMC which contains two references, one to the object, and one to the method. While a good idea, I think it's not the right way to handle this. Binding objects to methods to create invokable subs is going to be something we're going to need for a lot of the languages, so I think we'd be better served providing a general facility to do it rather than leaving it to each individual language designer to do it. Should save some work all around too. This would not be necessary, but the current implementation of the callmethodcc opcode unfortunately decouples the find_method (which is subject to the not at compile time restrictions alluded to above), and invoke. If these weren't decoupled (i.e., there was a VTABLE_find_method or equivalent entry), then this would not be necessary. Static methods differ in that the object is not passed. How is this different from a subroutine, then? From a callee-perspective: not at all. What is important to note is that from a caller-perspective, they will invoke such subroutines with the callmethcc opcode. Class methods differ in that the object passed is actually the class of the object in question. I'm assuming this is different from just a method on the class somehow? From a callee perspective, this appears to be a method on the instance. Note: all this is determined by the callee. It is all transparent to the caller. This is the part I'm not so sure about. It looks like, rather than having two sides (caller and calle) we have three, caller, callee, and the code that fetches the invokable in the first place. I fully agree that the caller shouldn't know about this stuff, since it may well have been handed the invokable thing as part of a function call or pulled it out of a variable or something. I don't think the callee should have to know anything special here, though -- it doesn't seem at all unreasonable to have the callee *not* have to do anything special, nor play any magic games. (And I think I'd be a bit peeved if I was writing code which passed in object A as the object being invoked on, but the method decided it wanted to use object B instead) This is especially true in a mixed-language environment when you've got a class with methods written in different languages -- setting up any conventions that'll actually be followed seems like an exercise in futility. :) Perhaps you might not find Python to your liking. That's OK. But the more general question is whether or not Parrot will implement the above as a policy, and thereby preclude langages like Python from being implemented on top of Parrot. That leaves the code that actually fetches the invokable thing in the first place, and that seems like the right place for this to happen. The language the code is written in knows what should happen based on what it gets back when querying the object, so as long as we provide a standard means to do all the binding stuff, we shoul dbe fine. I'm not sure what you mean by the code that actually fetches the invokable thing in this instance. If you mean an at compile time translation approach like you alluded above
Re: Calling conventions, invocations, and suchlike things
Luke Palmer wrote: Sam Ruby writes: Mmm, syntax! :) Luckily it makes no difference to us at the parrot level. What that should translate to is something like: $P0 = find_method Parrot_string, find # Elided check for failed lookup and fallback to attribute fetch $P1 = make_bound_method(Parrot_string, $P0) $P1(r) This will be a recurring theme in my replies. Any thing which presumes a bit of knowledge at compile time will ultimately not work. Consider the following: class c: find = 7 def f(x): return x.find print f(c()) print f(Parrot)(r) Now, what should the code for function f look like? The only reasonable answer is something along the lines of: getattribute $P0, P5, 'find' I doubt that. All languages have different semantics, and we can't implement them all, because they are conflicting. You, as a compiler designer, have the opportunity to design things so that they work. And you definitely have to be clever if you're looking for language features that Parrot doesn't natively support. I disagree. As Dan once said, as long as we have a proper protocol that everyone can conform to, we should be OK. I don't care if Perl, Python, Ruby, TCL, and others each implement different semantics. I do care that we adopt a common protocol. The current set of VTABLE entries is a excellent first order approximation for the common protocol. My first tendency here is to echo all attributes as methods (like Perl 6 does), and then always call the method when you see a dot. ParrotString has a find method, and it knows how to curry itself. Instances of c also have a find method, and that method always returns 7. The distinction between attributes and methods is a bit subtle. Any object may implement the VTABLE_invoke method (or in Python terms, a __call__ method), so all attributes may already *BE* a method. But largely, I could change the current implementation of Python on Parrot to follow such a protocol in a matter of minutes. But my original comment (Any approach 'which presumes a bit of knowledge at compile time will ultimately not work.') still stands. Although I agree that we should come up with a general, bare-bones object model that allows all of our current target languages to operate smoothly. Parrot's current model makes far too many assumptions. But that doesn't mean that anything you're trying to do is impossible; it just means it's harder. I don't believe that Parrot should impose an object model. I return to Dan's as long as we have a proper protocol that everyone can conform to, we should be OK. - Sam Ruby
Re: Calling conventions, invocations, and suchlike things
Sam Ruby wrote:
Now, what should the code for function f look like? The only
reasonable answer is something along the lines of:
getattribute $P0, P5, 'find'
I doubt that. All languages have different semantics, and we can't
implement them all, because they are conflicting. You, as a compiler
designer, have the opportunity to design things so that they work. And
you definitely have to be clever if you're looking for language features
that Parrot doesn't natively support.
I disagree. As Dan once said, as long as we have a proper protocol
that everyone can conform to, we should be OK.
I don't care if Perl, Python, Ruby, TCL, and others each implement
different semantics. I do care that we adopt a common protocol.
The current set of VTABLE entries is a excellent first order
approximation for the common protocol.
In case that isn't perfectly clear, let me illustrate this with actual
source code:
inline op getattribute(out PMC, in PMC, in STR) :object_classes {
$1 = VTABLE_get_attr_str(interpreter, $2, $3);
goto NEXT();
}
Note that the opcode doesn't actually implement any semantics. It
merely delegates the request to the $2 PMC.
Languages which share semantics for this operation can chose to inherit
a common implementation. Those with unique semantics can override this
(or can find other languages with similar requirements and pool their
implementation).
It seems rather likely to me that this was the original intent to
providing a VTABLE_get_attr_str VTABLE entry in the first place.
This seems to me to be a rather good design pattern.
- Sam Ruby
Re: Calling conventions, invocations, and suchlike things
Dan Sugalski wrote:
Hi folks.
Welcome back!
Parrot's got the interesting, and somewhat unfortunate, requirement of
having to allow all subroutines behave as methods and all methods behave
as subroutines. (This is a perl 5 thing, but we have to make it work)
That is, an invokable PMC may be invoked as a method call and passed in
an object, or as a plain subroutine and not have an object passed in. As
far as perl 5 is concerned the object is the first parameter in the
argument list, but for everyone else the object is a very distinct and
separate thing.
Python essentially has the same requirement, with a few twists.
Specifically, methods come in be static, class, and regular flavors.
But first, a simple example. Strings in python have a find method, so
and can do the following:
f = Parrot.find
print f(r)
Note that I referenced the method as an attribute, and then called it as
a function. Furthermore, the function remembers what object it is
bound to. This is accomplished by VTABLE_find_method creating a new
PyBoundMeth PMC which contains two references, one to the object, and
one to the method. The sole responsibility of a bound method is to
insert the first parameter into the call, and then to invoke the desired
method.
Static methods differ in that the object is not passed.
Class methods differ in that the object passed is actually the class of
the object in question.
Note: all this is determined by the callee. It is all transparent to
the caller.
Now, compare that to callmethod or callmethodcc in ops:
object = REG_PMC(2);
method_pmc = VTABLE_find_method(interpreter, object, REG_STR(0));
if (!method_pmc) {
real_exception(interpreter, next, METH_NOT_FOUND,
Method '%Ss' not found, REG_STR(0));
}
REG_PMC(0) = method_pmc;
interpreter-ctx.current_object = object;
dest = (opcode_t *)VTABLE_invoke(interpreter, method_pmc, next);
First, observe that I don't have any control over the exception that is
raised when a method is not found (fix: raise the exception within
find_method). Second, observe that this assumes that current object is
the object in all cases. This doesn't work well for static or class
methods (fix: create a PyBoundMeth object which overrides this, and then
call the real method).
This does make things a little tricker for the perl 5 code generator,
but not that much trickier and, let's face it, we're below the layer
where things are easy. This *also* makes building signature checking
into parrot a lot simpler (something we should do), since the signature
checking stuff doesn't have to deal with possible parameter shifting
based on whether we've a sub or method invocation.
Parameter shifting actually is made easier by placing the object in
question in a separate register... no shifting is required. (Note: in
cvs, I actually do the shifting, partially because I'm in the process of
reverting a change, and partially do to the issue raised in the next
paragraph).
My one minor request here is P2 be made available on entry to the
invoked method. This would remove some special case logic for me
requiring the use of interpinfo. I don't expect any guarantees that
this is preserved or restored across sub calls.
Not having objects handle their own method dispatch is less clear-cut,
but I do have some reasons, so here they are.
Just be aware that in order to preserve Python semantics, find_method
will need to return a bound method. This involves creating an object on
the heap, garbage collection, and a minor addition to the number of
instructions executed on invoke (including a nested C stack).
This could all be avoided if there was a VTABLE_callmethod interface as
the code would know that the intent was to only use this found method
exactly once.
*shrug*
Do you plan to choose banana cream again at OSCON 2005?
- Sam Ruby
Re: Proposed vtable changes WRT method lookup
Leopold Toetsch wrote: Below inline attached is a proposal for vtable changes, mainly WRT method lookup. First, a general question: under what circumstances is it OK and/or expected for the opcode Cgetclass and the VTABLE entry VTABLE_get_class to return different results? At the moment, these return quite different results for all of the standard PMCs. The more general observation behind this question: if Parrot is going to provide a mechanism by which a class can override get_class, then the runtime should depend on such interfaces instead of bypassing them and attempting to standardize on mechanisms. - Sam Ruby
Re: [perl #33751] [PATCH] Use INTERP in *.pmc files
Leopold Toetsch wrote: Bernhard Schmalhofer (via RT) wrote: I noticed the there is an interesting mix of 'interpreter' and 'INTERP' Thanks, applied except dynclasses. I leave that part up for Sam - dunno if he got diffs there. Applied. - Sam Ruby
Re: Key question
Leopold Toetsch wrote: But we should generalize keys eventually. Keys can provide an index for aggregates and allow chaining of indices for nested aggregates. Arrays are simple: the key is an integer. But hashes currently don't support non-string keys easily. We should be able to use arbitrary PMCs as hash keys. The way to go here is to use the VTABLE_hash method to generate an index aka hash value usable for accessing hashes. See: http://xrl.us/emnk Except for fromkeys, get_string, and __new__, the logic is not Python specific, and could easily be refactored into a common base class for others to use. - Sam Ruby
Re: [PROPOSAL] VTABLE_call_method
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: Ramblings on creating a new VTABLE_call_method slot to help with implementing Python's bound vs unbound methods: http://www.intertwingly.net/blog/2005/01/03/Bound-Methods 1) Methods are functions, where the first parameter is the object. We are currently passing the object (or the invocant) in current_object In subs denoted with method we have the magic self, which makes it available for user code. This doesn't match HLLs expectations and sematics. Proposal: P5 := object := invocant for method calls ex-P2 := current_invocant (now current_object aka self) So given: def index(self, substr) We have the common case of a method call and matching arguments: parrot.index(r) P5 ... parrot P6 ... r current_invocant ... parrot With an unbound method call: str.index(parrot, r) with this register setup prior to the call: P5 ... str P6 ... parrot P7 ... r current_invocant ... str we have to do: if (PObj_is_class_TEST(object)) shift_arguments(...); and we have again arguments matching. How should the following be handled: f = parrot.index ... print f(r) 2) WRT the proposed VTABLE_call_method() This just moves the burden of doing a method call down to all classes and doesn't really fix anything. 1) is not python-specific. Moving the code into language-specific PMCs hinders interoperbility and causes unneeded code duplication. I've learned to expect such a predictable response. The exact same number of lines of code, and the exact same external behavior somehow magically hinders interoperability and causes unneeded code duplication. Whatever. - Sam Ruby
Re: [PROPOSAL] VTABLE_call_method
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
Leopold Toetsch wrote:
How should the following be handled:
f = parrot.index
The CPython code is:
1 0 LOAD_CONST 0 ('parrot')
3 LOAD_ATTR0 (index)
6 STORE_NAME 1 (f)
The interesting thing is hidden in LOAD_ATTR. It seems that it creates a
call stub for the given (object, method) pair.
...
print f(r)
That looks like and is a plain function call (BTW it's not covered by
your call_method proposal). The f object (the method call stub) has to
shift up call arguments and place the object as the first argument.
2) WRT the proposed VTABLE_call_method()
This just moves the burden of doing a method call down to all classes
and doesn't really fix anything. 1) is not python-specific. Moving the
code into language-specific PMCs hinders interoperbility and causes
unneeded code duplication.
I've learned to expect such a predictable response. The exact same
number of lines of code, and the exact same external behavior somehow
magically hinders interoperability and causes unneeded code duplication.
What is unclear about: This just moves the burden of doing a method
call down to all classes and doesn't really fix anything.?
First you state The f object (the method call stub) has to
shift up call arguments and place the object as the first argument.
In other words, moving the burden of doing the marshalling of parameters
down to the class really *does* fix something.
Then you state that moving the burden in a similar case doesn't fix
anything.
Whatever.
Again: the problem is *not* python-specific.
Analogy: is classes/string.pmc Perl Specific?
I anticipate each language will need a language specific string class.
Hopefully each will inherit much of their functionallity from Parrot's
string class once all the multiple inheritance and MMD issues are worked
out.
In a similar manner, I expect many languages will need a language
specific class class. Again, hopefully each will inherit much of
their functionallity from a Parrot class class.
If you take a look at my recent commit, you will see that the way that
bound methods are implemented in Python leverages the way that Python
Properties are defined. This isn't merely an implementation choice -
this is the way the language is defined, and t/pie/b0.t seems carefully
crafted to ensure that the implementation faithfully implements this
part of the Python language definition.
I suspect that much of this is Python specific. That's why it is nice
that VTABLE_get_attr_str is a VTABLE entry and not hardwired into
Parrot_getattribute_p_p_sc.
In a similar manner, I expect most languages to implement call_method to
either simply inherit the default or to implement the same basic flow of
find_method followed by invoke. What is in between those two calls,
however, may vary based on language - for example, what exception is to
be thrown if the method is not found may vary from language to language.
However, I can't override this behavior if it remains hardcoded in the
op. Moving the burden down to the classes *does* allow me to fix
something - it enables me to throw the exception that Python developers
will expect, and will allow me to avoid the creation of bound method
objects in many cases.
- Sam Ruby
[PROPOSAL] VTABLE_call_method
Ramblings on creating a new VTABLE_call_method slot to help with implementing Python's bound vs unbound methods: http://www.intertwingly.net/blog/2005/01/03/Bound-Methods This is related to a previous discussion on Overloaded Operator Methods: http://xrl.us/ekh8 - Sam Ruby
Re: [perl #33603] Undef != Undef
Leopold Toetsch wrote:
Simon Glover [EMAIL PROTECTED] wrote:
This code:
new P0, .Undef
new P1, .Undef
eq P0, P1, L1
print not
L1: print ok\n
end
prints not ok. Should it?
That depends ;)
... If Parrot considers every Undef PMC to
be distinct, it's going to make tasks like comparing arrays with
large numbers of undefined elements much fiddlier than it should be.
Yep. Currently there is no chance to change that.
If the desire is that two new .Undef values are to be considered equal,
then the attached patch achieves that.
- Sam Ruby
? undef.patch
Index: undef.pmc
===
RCS file: /cvs/public/parrot/classes/undef.pmc,v
retrieving revision 1.10
diff -u -r1.10 undef.pmc
--- undef.pmc 19 Oct 2004 01:25:35 - 1.10
+++ undef.pmc 31 Dec 2004 15:16:50 -
@@ -77,6 +77,15 @@
return 0;
}
+ INTVAL is_equal (PMC* value) {
+MMD_Undef: {
+ return 1;
+}
+MMD_DEFAULT: {
+ return 0;
+}
+ }
+
}
/*
Re: [perl #32563] [BUG] missing Makefile dependencies
Will Coleda via RT wrote: Here's another one: make test before make fails a few tests: t/dynclass/pybuiltin.t5 1280 65 83.33% 1-2 4-6 t/dynclass/pyclass.t 6 1536 66 100.00% 1-6 t/dynclass/pycomplex.t1 256 11 100.00% 1 t/dynclass/pyfunc.t 4 1024 44 100.00% 1-4 t/dynclass/pyint.t 25 640025 25 100.00% 1-25 t/library/parrotlib.t 6 1536 66 100.00% 1-6 t/pmc/nci.t 1 256471 2.13% 48 These failures are all due to missing dependencies that *are* built with make, but not by make test (which is at least smart enough to build the parrot executable). I've made a fix. While I obviously have authority to update cvs, as near as I can tell, my user id is not defined to the parrot bug tracking system. - Sam Ruby
Re: dynclasses/*py* broke the build
William Coleda wrote: (Resend from the right account so it'll make it to the list.) This is on OSX. Looks very similar to an issue that I think Sam has already fixed once. I have some tcl stuff pending that I can't verify works in a clean build until I have clean build. =-) Sorry about that... I've committed what I hope to be a fix. - Sam Ruby
Re: MMD and VTABLE_find_method
Leopold Toetsch wrote:
pmclass default abstract noinit {
PMC* add (PMC* left, PMC* right) {
PRESERVE_CONTEXT;
add_func = mmd_find(__add...) // via VTABLE_find_method
and here it becomes horribly slow and weird. You are already in one
add. What do you want to find here?
I took your example code, almost word for word, and added approximately
three, count them three, machine instructions (not Parrot instructions,
not C statements, but machine instructions) to the path length, and now
adjectives like horribly slow and wierd come out.
It looks like this mailing list will not be suitable for civilized
discussion until Dan gets back.
Meanwhile, all existing Python tests pass:
http://www.intertwingly.net/blog/2004/12/25/Python-on-Parrot-test-status
Merry Christmas.
- Sam Ruby
Re: MMD and VTABLE_find_method
Leopold Toetsch wrote:
Sam Ruby wrote:
First, a few things to note: the semantics of add vary from language
to language. In particular, add is not guaranteed to be commutative
in Python (think string addition).
Yes, of course.
It seems obvious, but it leads to surprises. Example:
'1' + '2'
The result will depend on what the actual types are of the inputs
perhaps even what the context is of the caller.
As my proposal is primarily focused on where the logic is placed in
the system, not how it works, I'll like to use your proposal
http://xrl.us/egvp as a starting point. Just to make sure that I
don't mischaracterize your proposal, can you take a look at the
attached and either agree that it represents a reasonable first
approximation of what you had in mind, or modify it so that it is?
It's a reasonable respresentation, yes. Finding the right functions is
more complex, though, as described in my proposal.
I'd just outline the functionality of the add opcode like this:
inline op add (out PMC, in PMC, in PMC) :base_core {
PRESERVE_CONTEXT;
add_func = mmd_find(__add...) // via VTABLE_find_method
REG_PMC(5) = $2;
REG_PMC(6) = $3;
VTABLE_invoke(interp, add_func, 0);
res = REG_PMC(5);
RESTORE_CONTEXT;
$1 = res;
}
Cool see below.
the basics are:
* the destination PMC $1 is created by the opcode
I don't want to dwell on this point, but it would be rather handy if the
caller were able to pass in an object which was responsible for
producing the desired destination PMC on request.
It would accept requests like give me an Integer and would respond
with things like here's a PyInt.
At the moment, we have a fairly good approximation of this with
VTABLE_morph. Make yourself a BigInt...OK, (P.S. I'm really a PyLong).
This won't work for languages which have a notion of singletons, unless
the language uses a double-reference system... like Ruby and Perl 5 do
today (per Dan's previous statements).
* no other registers are changed, nor the context
* finding the __add method uses VTABLE_find_method to
find all possible __add methods and a distance
function to get the best match
* the best matching function is invoked
The word best here should be setting off alarm bells in everybody's
head. What is the chance that we can get Larry, Guido, Matz, Brenden
and others to agree on such a thing? Particularly when they can't even
agree on what + means when dealing with strings (actually, in the list
above, Larry is the lone hold out... ;-) (and apologies to all involved
for personifying this))
Once that's done, I'll sketch out all of the changes required to
enable Perl and Python to each have their own separate semantics for
this operation, and yet be able to have meaningful interop when it
comes to adding a PerlInt and a PyInt, or vice versa.
OK, here's what the proposal would look like, hypothetically assuming
that all of your proposed changes get in:
inline op add (out PMC, in PMC, in PMC) :base_core {
$1 = VTABLE_add(interp, $2, $3);
}
pmclass default abstract noinit {
PMC* add (PMC* left, PMC* right) {
PRESERVE_CONTEXT;
add_func = mmd_find(__add...) // via VTABLE_find_method
REG_PMC(5) = left;
REG_PMC(6) = right;
VTABLE_invoke(interp, add_func, 0);
res = REG_PMC(5);
RESTORE_CONTEXT;
return res;
}
}
That's it. From an external point of view, add is called in exactly the
same way. Whether MMD is involved or not, the caller shouldn't know and
shouldn't care. A similar approach can be taken in the more general
case of the foo subroutine that we were talking about previously... no
need to introduce new opcodes or a change to the Perl6 syntax.
Now, lets take a look at a hypothetical PyString implementation of add:
pmclass PyString extends PyObject dynpmc group python_group {
PMC* add (PMC* right) {
if (!VTABLE_does(right, const_string(STRING))
real_exception(INTERP, NULL, E_TypeError,
TypeError: cannot concatenate '%Ss' and '%Ss'
objects,
SELF-vtable-whoami, right-vtable-whoami);
PMC *dest = pmc_new(INTERP, dynclass_PyString);
PMC_str_val(dest) = string_concat(INTERP,
PMC_str_val(SELF), VTABLE_get_string(INTERP, value), 0);
return dest;
}
}
Note: no MMD. In fact, no need to preserve context. Either the second
parameter does string, or a Type error is thrown. This would
presumably interoperate with PerlString, but not with PerlInt. How
would this work when called from Perl instead? Well, PyInt need to be
based on from Parrot's Integer, and the MMD implementation that Perl
settles on will need to be aware of inheritance.
Now let's look at a second hypothetical example:
pmclass PyClass dynpmc group python_group {
PMC* add (PMC
Re: MMD and VTABLE_find_method
Leopold Toetsch wrote: Sam Ruby wrote: Leopold Toetsch wrote: A foo PMC could represent an entire row in a two dimensional MMD, or an entire plane in a three dimensional MMD, ... etc. What does it mean: represent a row...? What about the namespace pollution? Again: where does this hypothetical MMD PMC come from? Since you are going to snip away everything, let's start all over. First, a direct quote from http://www.perl.com/pub/a/2004/04/16/a12.html?page=10: As we mentioned, multiple dispatch is enabled by agreement of both caller and callee. From the caller's point of view, you invoke multiple dispatch simply by calling with subroutine call syntax instead of method call syntax. It's then up to the dispatcher to figure out which of the arguments are invocants and which ones are just options. (In the case where the innermost visible subroutine is declared non-multi, this degenerates to the Perl 5 semantics of subroutine calls.) This approach lets you refactor a simple subroutine into a more nuanced set of subroutines without changing how the subroutines are called at all. That makes this sort of refactoring drop-dead simple. (Or at least as simple as refactoring ever gets...) In the general case, a call to a subroutine with three arguments can have four possibilities: anywhere from zero to three arguments may be involved in the dispatch. I also read this to say that whatever code is generated by a subroutine call is independent of the number of arguments involved in the dispatch. If you read this differently, perhaps we can get a ruling from the Per6 language folks. If I am correct, this will have the nice side benefit that any such methods can be invoked transparently by all languages. None of this precludes a Polymorphic Inline Cache. Or Multidimensional Multimethod Dispatch. Or 30 to 70% performance improvements. But it does constrain where the logic needs to be placed. And it does rule out syntax changes and using a different opcode for invoking MMD subroutines than non-MMD subroutines. - Sam Ruby
Re: MMD and VTABLE_find_method
Leopold Toetsch wrote:
Sam Ruby wrote:
First, a direct quote from
http://www.perl.com/pub/a/2004/04/16/a12.html?page=10:
Please let's stay at the basics. Please describe your counter proposal
for a very elementary
add Px, Py, Pz
operation.
There's really no need to procede to Perl6 objects, if we can't even
find a common interoperating implementation that is suited for Parrot's
target languages.
And it does rule out syntax changes and using a different opcode for
invoking MMD subroutines than non-MMD subroutines.
Yep.
First, a few things to note: the semantics of add vary from language
to language. In particular, add is not guaranteed to be commutative in
Python (think string addition).
As my proposal is primarily focused on where the logic is placed in the
system, not how it works, I'll like to use your proposal
http://xrl.us/egvp as a starting point. Just to make sure that I
don't mischaracterize your proposal, can you take a look at the attached
and either agree that it represents a reasonable first approximation of
what you had in mind, or modify it so that it is?
Once that's done, I'll sketch out all of the changes required to enable
Perl and Python to each have their own separate semantics for this
operation, and yet be able to have meaningful interop when it comes to
adding a PerlInt and a PyInt, or vice versa.
- Sam Ruby
Index: math.ops
===
RCS file: /cvs/public/parrot/ops/math.ops,v
retrieving revision 1.32
diff -u -r1.32 math.ops
--- math.ops7 Dec 2004 17:24:53 - 1.32
+++ math.ops22 Dec 2004 16:08:50 -
@@ -171,7 +171,32 @@
}
inline op add (in PMC, in PMC, in PMC) :base_core {
- mmd_dispatch_v_ppp(interpreter, $2, $3, $1, MMD_ADD);
+ if (1) { /* this is here to allow declarations, even in C89 */
+ struct Parrot_Context ctx;
+ struct parrot_regs_t *bp = interpreter-ctx.bp;
+ STRING *__add = const_string(interpreter, __add);
+ INTVAL mmd_flag;
+ PMC *addsub;
+
+ /* we probably don't need to save the full context, just a few regs... */
+ save_context(interpreter, ctx);
+
+ /* let's ignore the complexities of a distance_func for now... */
+ mmd_flag = 0;
+ addsub = VTABLE_find_method(INTERP, $1, __add, 0, mdd_flag);
+ if (!addsub)
+ mmd_flag = 0;
+ addsub = VTABLE_find_method(INTERP, $2, __add, 1, mdd_flag);
+
+ /* assume NCI for now */
+ VTABLE_invoke(interpreter, sub, NULL);
+
+ /* note: this was allocated by __add */
+ $3 = BP_REG_PMC(bp,5);
+
+ /* again, probably overkill */
+ restore_context(interpreter, ctx);
+ }
goto NEXT();
}
Re: P5 is the new P2
Matt Fowles wrote: P5 is the new P2 Sam and Leo came to the conclusion that the current object should be passed in P5 as well as P2. Currently they are waiting for Dan... Nudge, Nudge... http://xrl.us/ef8r I've implemented this, but I am near the point of recanting. Examples of problematic code (both from t/pie/b3.t): return TT.__new__(cls, value) return T.__cmp__(a, b) Where the latter ends up invoking the cmp method with three arguments instead of the customary two, and confusion reigns. The way this is resolved in Python is with a distinction between bound and unbound methods. Roughly, in Parrot terms, this is a distinction between methods where the object can be found in P2 and P5 respectively. I'm resisting implementing this for now, but will likely have to cave ultimately. To implement this would require a call to pmc_new on each method call on an instance, as well as reinserting the argument as the first parameter (shifting each of the curent parameters in the process). Blech. - Sam Ruby
Re: MMD and VTABLE_find_method
Leopold Toetsch wrote: Sam Ruby wrote: Leopold Toetsch wrote: However, from http://www.perl.com/pub/a/2004/04/16/a12.html?page=10: Whenever you make a call using subroutine call syntax, it's a candidate for multiple dispatch. I read this to mean that the *caller* does nothing to distinguish between calls to single dispatch subroutines from multiple dispatch subroutines. So... how does one determine at compile time which opcode to use? This probably needs some clarification from Perl6 people. Anyway, if Parrot just gets: foo(Px, Py, Pz) there is no information, how many invocants are participating in MMD. So candidates for multiple dispatch must be declared somewhere. A few things to note: foo is a PMC. It therefore is an object. It can have state (properties, attributes, etc). It can know how many arguments are involved in multiple dispatch. All calls to foo are likely to have the same number of arguments participating in MMD. This number is known when foo is defined. If that information is captured in the PMC itself, it can be exploited at runtime. $a.foo($b, $c) := foo($a, $b, $c) Given the latter syntax, how does the compiler know when to emit a callmethodcc foo and when to emit a call_MMD foo? The compiler has to know it. E.g. foo($a: $b, $c); # 1 invocant foo($a, $b: $c); # 2 MMD invocants and as Parrot has to find a method depending on n MMD invocants, this information must be present at runtime. This leads to the assumption that we'll need an opcode call_MMD meth, n_invocants Changing the external syntax for subroutine calls in P6 may be an option. Changing the syntax for function calls in Python is not an option. Therefore, if changing the syntax is required, it is likely that Python will not be able to call arbitrary subroutines involving MMD. It would be ideal if callers did not have to know how such subroutines were defined, and could continue to emit invokecc sequences on Sub PMCs, and for this to be handled at runtime. and some changes in pdd03 as the current calling scheme doesn't have anything about argument order. I have described that in subject: MMD: more implications The only thing I am attempting to solve is the presumption that MMD calls can be detected at compile time. It's quite clear for infix operations like add. For arbitrary subroutines the compiler has to know it, as outlined above. When expressed via an infix notation, I agree that it is not a problem. When expressed via the equivalent method call notation, (as Python allows), it becomes one. And despite that it's looking like a fat method call, Parrot executes this sequence, if the __add isn't overloaded: if (cache-type == (Py-vtable-base_type 16|Pz-vtable-base_type) Px = (cache-function)(Py, Pz); which is 30% faster then the current mmd_dispatch based opcode. That code will execute just as fast if placed inside the invoke logic of a MMD PMC. - Sam Ruby
Re: cvs commit: parrot/dynclasses pybuiltin.pmc pyclass.pmc pyfunc.pmc pylist.pmc pynone.pmc
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
--- nci.pmc7 May 2004 10:33:26 - 1.27
+++ nci.pmc20 Dec 2004 22:27:11 - 1.28
+=item CPMC* get_attr_str(STRING *name)
+
+Return attribute named Cname.
+
+=cut
+
+*/
+
+PMC* get_attr_str(STRING* idx) {
+return VTABLE_getprop(INTERP, SELF, idx);
+}
+
}
What is the rational for this pythonism in Parrot core?
Good catch. I've added the following directly to nci.pmc both to
capture the rationale and to record a TODO.
--- nci.pmc 20 Dec 2004 22:27:11 - 1.28
+++ nci.pmc 21 Dec 2004 11:50:16 -
@@ -159,6 +159,16 @@
=cut
+TODO: refactor into a separate subclass, and either modify Pmc2c.pm and
+enter_nci_method to directly build this subclass, or add code to morph
+NCI instances into the correct subclass later.
+
+The line of code in test case t/pie/b3 that motivates this is:
+
+ print using, cmp.__name__
+
+Where cmp may be a NCI subroutine.
+
*/
PMC* get_attr_str(STRING* idx) {
Re: MMD and VTABLE_find_method
Leopold Toetsch wrote:
Sam Ruby wrote:
Leopold Toetsch wrote:
A few things to note: foo is a PMC. It therefore is an object. It
can have state (properties, attributes, etc). It can know how many
arguments are involved in multiple dispatch.
The MMD information can't hang off the Sub PMCs. How do you find the
correct foo Sub then?
[snip]
Well, I don't know how the Perl6 compiler will translate:
foo($a, $b, $c)
If there is more then one foo Sub PMC around, the call syntax still
must have the number of invocants attached to it, so that MMD can happen.
A foo PMC could represent an entire row in a two dimensional MMD, or
an entire plane in a three dimensional MMD, ... etc.
A Perl6 compiler could translate the above to:
87 find_lex P5, a
90 find_lex P6, b
93 find_lex P7, c
96 find_global P0, foo
108 set I0, 1
111 set I1, 0
114 set I2, 0
117 set I3, 3
120 set I4, 0
126 invokecc
The foo PMC could know how many of the arguments are relevant for MMD
dispatch. Should only the first two arguments be relevant, this code
could be as simple as:
void* invoke(void* next) {
...
if (cache-type == (Py-vtable-base_type
16|Pz-vtable-base_type)
return VTABLE_invoke( cache-function );
...
}
[snip]
That code will execute just as fast if placed inside the invoke
logic of a MMD PMC.
Where does this hypothetical MMD PMC come from? Where is it created? The
opcode is just:
add Px, Py, Pz
Py or Pz can be plain PyInts or Integers. Where does that MMD PMC come
from?
Once MMD is solved connecting it up to the PerlScalar PMC (for example)
would be rather straightforward. Syntax like the following would result
in an appropriate entry being added to the MMD dispatch table:
multi sub infix:+ (Us $us, Us $ustoo) {...}
The Parrot_PerlScalar_add function (which would revert back to being a
vanilla vtable) would know about the portion of the MMD dispatch table
that is relevant to Perl scalar add operations, and invoke it.
(Note: one thing that hasn't been discussed to date is how registers
will be handled. The current add opcode doesn't have any side effects)
- Sam Ruby
Re: cvs commit: parrot/dynclasses pybuiltin.pmc pyclass.pmc pyfunc.pmc pylist.pmc pynone.pmc
Leopold Toetsch wrote: Sam Ruby wrote: Leopold Toetsch wrote: What is the rational for this pythonism in Parrot core? +The line of code in test case t/pie/b3 that motivates this is: + + print using, cmp.__name__ + +Where cmp may be a NCI subroutine. Python's builtin cmp is basially a Sub PMC. Parrot Sub's have already a name: At the moment, Python's builtin cmp is implemented thus: enter_nci_method(interp, my_enum_class_PyBuiltin, F2DPTR(Parrot_PyBuiltin_cmp), cmp, PIOPP); .sub main .const .Sub c = cmp print c .end .sub cmp .end So I think, you could translate access to the __name__ attribute directly to VTABLE_name(). WRT NCI: The b3.py test doesn't need it, but I can imagine that the method name of Parrots cmp method is just __cmp and available with VTABE_name(). For this test to pass, the __name__ must be cmp. As this issue is obviously very important to you, I'll drop what I am doing make creating a new subclass of NCI for my purposes my top priority. - Sam Ruby
Re: MMD and VTABLE_find_method
Leopold Toetsch wrote: Sam Ruby wrote: Leopold Toetsch wrote: The caller sets: mmd_flag := NULL ... no MMD, plain method lookup mmd_flag := depth ... return the next matching method starting at the given parent search depth In the general case, how does the caller know that MMD is invoked? Perl6 multi subs are denoted with the multi keyword. We need some extensions to pdd03 that pass this information on to Parrot. It basically boils down to a new opcode: call_MMD method, n as described in subjects MMD: more implications and MMD dispatch My question was: how does the caller make such a determination? Yes, in Perl 6, the multi subs are defined with the multi keyword. However, from http://www.perl.com/pub/a/2004/04/16/a12.html?page=10: Whenever you make a call using subroutine call syntax, it's a candidate for multiple dispatch. I read this to mean that the *caller* does nothing to distinguish between calls to single dispatch subroutines from multiple dispatch subroutines. So... how does one determine at compile time which opcode to use? A cache invalidation function is called from add_method (and remove_method) which resets entries for the passed class. And, in some languages, all calls to set_attr or setprop type methods, where the value invoked may be invokable, or might obscure visibility to one that is. As calls to setting attributes/properties are frequent, my concern is that this may more than wipe out any potential benefit that such a cache may provide. You don't have operator overloading implemented in py*, do you? Anyway the code generator emits: add Px, Py, Pz Now some attribute set operations on the class, metaclass or in the __dict__ can mean an overloading of the __add__ method of CPy. To handle that correctly, you can either not emit an add opcode in the first place, or you have to track the attribute set operations so that you are able to call the user-provided __add__ method. You can of course in the current scheme install an add MMD method that does always a full method lookup, but then you got the performance problem you are worrying about. The overloading functionality has been added for a number of methods, but not yet for __add__. I've been adding methods one at a time based on the existence of test cases. Classes like PyString are primitive, and opcodes like get_iter directly access the vtable. For classes written completely in PIR, the vtable entry for get_iter causes __iter__ methods to be invoked. Note: this decision is made at runtime, not at compile time. Instead of pessimistically assuming that all such invocations will require a method lookup, this decision is deferred to the appropriate implementation of VTABLE_get_iter. Classes written in PIR but inherit from primitive classes employ a proxy, analogous to the delegate class, but in reverse. Note: proxies are only created if there is such a mix of PIR and NCI involved. All of this is taken care of by the Cinvoke methods of PyType and PyProxyType, the compiler is unaware of these details. Also, note that the Perl sub defined above is not a method. Yes. But Perl6 allows multi subs to be called as methods on the first invocant too: $a.foo($b, $c) := foo($a, $b, $c) Given the latter syntax, how does the compiler know when to emit a callmethodcc foo and when to emit a call_MMD foo? Comments welcome, Counter-proposal. I see no reason why a full multi-dimensional multi-method dispatch PMC could not commence immediately, complete with a fully-functional polymorphic inline cache. Once it is ready and tested, we can explore setting things up so that the various mmd_dispatch_* functions to exploit this functionality for the existing predefined binary operations. I don't see how this solves anything, except that you seem to be moving the burden of MMD to an additional PMC. What does this proposed MMD PMC do? How does it find the appropriate multi-method? I've described a versatile MMD scheme that is able to do n-dimensional MMD. Counter-proposals are very welcome, but the proposal has to include the mechanism how it works. A MMD PMC that does it is too thin, sorry. The only thing I am attempting to solve is the presumption that MMD calls can be detected at compile time. Unless you can describe a mechanism which enables the callers to detect at compile time whether they are invoking a MMD subroutine or not, this code needs to be either executed as a part of an VTABLE_invoke. Again, I am not suggesting that the algorithm be made any thinner, in fact, I am not suggesting any change to the algorithms that you have described. I am merely suggesting where the logic needs to be placed. - Sam Ruby
Re: MMD and VTABLE_find_method
Leopold Toetsch wrote:
2) n-dimensional MMD
Perl6 supports a more general form of MMD:
multi sub foo(ClassA $a, ClassB $b, ClassC $c : ...) { ... }
[snip]
4) Proposed changes:
a) All method lookup goes through VTABLE_find_method. To achieve MMD
functionality, two arguments are added to the find_method signature:
PMC* find_method(STRING* meth_name, INTVAL mmd_nr, INTVAL* mmd_flag)
mmd_nr is the MMD invocant number:
0 := first (or only invocant)
1 := second invocant (e.g. right argument of 2-dim MMD)
...
mmd_flag is a pointer to an integer, indicating the desired behavior:
The caller sets:
mmd_flag := NULL ... no MMD, plain method lookup
mmd_flag := depth ... return the next matching method starting
at the given parent search depth
In the general case, how does the caller know that MMD is invoked?
A PIC scheme outperforms the current static MMD table lookup by 30% up
to 70% for overloaded operations.
A cache invalidation function is called from add_method (and
remove_method) which resets entries for the passed class.
And, in some languages, all calls to set_attr or setprop type methods,
where the value invoked may be invokable, or might obscure visibility to
one that is. As calls to setting attributes/properties are frequent, my
concern is that this may more than wipe out any potential benefit that
such a cache may provide.
Also, note that the Perl sub defined above is not a method.
Comments welcome,
Counter-proposal.
With Parrot today, compiler writers need not know the difference between
a closure, a continuation, a coroutine, an nci, or a sub. Or, for that
matter, a PyFunc, a PyGen, or a PyType.
The general case is that the caller has a PMC which does sub. This
may not be a method (the Perl syntax above suggests a simple sub). It
may have been found via find_lex or find_global, been returned by a
previous subroutine call, or have been newly created. Whatever.
At this point, the caller's responsibilities are merely to set up the
registers according to the Parrot Calling Conventions and then execute
the invoke opcode, or equivalent (invokecc, tailcall, ...).
At this point, there are two paths that are possible. Either the
compiler takes full responsibility for emitting code that does the
appropriate second level dispatch. Or there can be code in the Parrot
repository in the form of a new subclass of the existing Sub class which
locates and appropriate destination. (Note: these choices are not
mutually exclusive).
I see no reason why a full multi-dimensional multi-method dispatch PMC
could not commence immediately, complete with a fully-functional
polymorphic inline cache. Once it is ready and tested, we can explore
setting things up so that the various mmd_dispatch_* functions to
exploit this functionality for the existing predefined binary operations.
- Sam Ruby
Re: runops_args vs NCI
Leopold Toetsch wrote:
Sam Ruby wrote:
However, VTABLE_invoke on NCI methods is where the real work is done
(including reading from and writing to registers), and a null dest is
returned.
One more remark:
This is classes/nci.pmc:invoke
void* invoke (void * next) {
Parrot_csub_t func = (Parrot_csub_t)D2FPTR(PMC_data(SELF));
func(INTERP, SELF);
return next;
}
It's not returning NULL.
Fixed.
- Sam Ruby
Re: runops_args vs NCI
Leopold Toetsch wrote: Sam Ruby wrote: However, VTABLE_invoke on NCI methods is where the real work is done (including reading from and writing to registers), and a null dest is returned. Ouch. Sorry, probably cut'n'paste code, relicts or whatever. runops_* isn't supposed to be called for NCI methods. It's for running PASM opcodes. A NCI methods wraps a C function to be callable by PASM. If we are calling a NCI function from C, we can run the code diretly too. I'll fix that. I have a number of cases (filter, map, reduce), where I am passed an arbirary sub and need to call it. I should not have to know how the sub is implemented. So, what you are telling me is that I need to clone runops_args and fix it right. I'll do that. - Sam Ruby
Re: runops_args vs NCI
Leopold Toetsch wrote: Sam Ruby wrote: Leopold Toetsch wrote: Sam Ruby wrote: However, VTABLE_invoke on NCI methods is where the real work is done (including reading from and writing to registers), and a null dest is returned. Ouch. Sorry, probably cut'n'paste code, relicts or whatever. runops_* isn't supposed to be called for NCI methods. It's for running PASM opcodes. A NCI methods wraps a C function to be callable by PASM. If we are calling a NCI function from C, we can run the code diretly too. I'll fix that. I have a number of cases (filter, map, reduce), where I am passed an arbirary sub and need to call it. I should not have to know how the sub is implemented. Just invoke it. So, what you are telling me is that I need to clone runops_args and fix it right. I'll do that. Argh. I didn't tell you that. The interface to run an arbitrary piece of code is VTABLE_invoke. The runops_fromc* functions run *opcodes* from inside C. Grrr. First you tell me: Crunops is a low-level function that awaits all properly setup. Around that are more function that e.g. create a return continuation and do argument or returnvalue passing. Just use one of these functions as delegate.c does. To which I respond: I *am* using Parrot_run_meth_fromc_args, just like delegate.c does. Now you tell me, Just invoke it. But I need to do more than that. I need to do what Parrot_run_meth_fromc_args is attempting to do (i.e., create a return continuation and do argument or returnvalue passing.), but correctly. - Sam Ruby
Re: runops_args vs NCI
Leopold Toetsch wrote: Sam Ruby wrote: But I need to do more than that. I need to do what Parrot_run_meth_fromc_args is attempting to do (i.e., create a return continuation and do argument or returnvalue passing.), but correctly. 1) runops_fromc and friends are running *opcodes*, nothing else 2) the functions are working, nothing needs fixing here. I've made a clone of these functions, and made the relevant fixes to my copy. Hopefully someday this will get refactored back into Parrot. For the record, in the test case I provided, I was calling Parrot_run_meth_fromc_args twice. The first time with a real Sub. One implemented with opcodes. Or, as you prefer to say it, *opcodes*. It was this call that did not restore the context interpreter-ctx.bp. As near as I can tell, this path is identical to the usage in classes/delegate.c. In other words, it is an accident waiting to happen. Take a look at Parrot_PyClass_runops_fromc. If you do, you will see that I am only restoring the value of interpreter-ctx.bp if dest is non-zero. Think about it. The second time I called Parrot_run_meth_fromc_args, it was with an NCI method. The only problem is that in this call, the parameters were set into registers after the invoke call. I believe it to be customary to set the parameters into registers prior to the invoke call. In fact, I have logic in PyFunc.pmc (again which is used to invoke Python functions which are implemented in opcodes, er, I mean *opcodes*), which depends on things happening in this order in order to handle defaults and varargs. I've already stated several times: let's adjust Parrot's method dispatch and overloaded method signatures, so that it's directly usable for target languages. When that is done, you can get rid of all the additional dispatch functions. I am not interested in discussing this any more. The current Parrot has bugs. I've notified you of a number of them. You are apparently not interested as you would prefer to rewrite Parrot instead. Undoubtably, at that time it will have a different set of bugs. At which point, undoubtably, the process will repeat. Onward. - Sam Ruby
Re: RT#31859, Plain ole Hash
Leopold Toetsch wrote: - The NCI method has an implementation in Hash and in PerlHash, this is not nice The NCI method is automatically inherited from Hash. But as it's a python method it'll be removed from Hash anyway. Any objections to the NCI methods being removed from Coroutine(next), Hash(fromkeys), Iterator(next), and PerlHash(fromkeys) now? - Sam Ruby
Re: cvs commit: parrot/ops pmc.ops
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
I don't understand this. At all. But the test case added to pyclass.t
(motivated by test 4 in pie/b3.t) only passes if this change to the
get_repr op is made.
[ ... ]
-op get_repr(out STR, in PMC) {
-$1 = $2-vtable-get_repr(interpreter, $2);
+inline op get_repr(out STR, in PMC) {
+STRING *s = $2-vtable-get_repr(interpreter, $2);
+$1 = s;
goto NEXT();
}
Strange. Stranger. Strangest. Did the test fail with JIT/i386 only?
I didn't mean to commit the inline portion of the change... that
didn't make a difference.
I've never investigated how to select another core... I've done all my
runs using the default for i386. While in gdb, I have seen mention of
slow_core and in this case, breakpoints that I set on
Parrot_get_repr_s_p were hit.
The symptoms I see are that get_repr returns the correct string, but the
appropriate REG_STR does not get updated.
- Sam Ruby
Re: overloaded operator calling conventions
Leopold Toetsch wrote: Sam Ruby wrote: A few observations, first from an Parrot Internal perspective... in general, the code for the opcodes tend to do things like the following: $1-vtable-get_string(interpreter, $1) Note that the object tends to be repeated as the first argument. It often the case that the first argument matches the object on which the dispatch is based, but it is not necessarily so. In particular, these two values will be different whenever the implementation of a method wants to invoke the equivalent of SUPER(args). Not quite. That'll be class_self.__super(args) so again the invocant is the first argument after interpreter. Believe it or not, I think we are agreeing. To invoke a method on an object using Parrot Calling Conventions, P2 needs to be the object used for dispatch purposes, and P5 needs to be the actual object. In many cases, they will be the same, but in some cases they will differ. This isn't obvious from PDD03, but a simple clarification would take care of that. - Sam Ruby
Re: cvs commit: parrot/ops pmc.ops
Leopold Toetsch wrote:
Sam Ruby wrote:
Leopold Toetsch wrote:
+inline op get_repr(out STR, in PMC) {
+STRING *s = $2-vtable-get_repr(interpreter, $2);
+$1 = s;
goto NEXT();
}
Strange. Stranger. Strangest. Did the test fail with JIT/i386 only?
I didn't mean to commit the inline portion of the change... that
didn't make a difference.
op or inline has no effect
I've never investigated how to select another core...
$ parrot --help
specifically -j -S -C -g
Here's the results *with* the above change:
[EMAIL PROTECTED]:~/parrot$ parrot test.pir
5 T(5)
[EMAIL PROTECTED]:~/parrot$ parrot -j test.pir
5 T(5)
[EMAIL PROTECTED]:~/parrot$ parrot -S test.pir
5 T(5)
[EMAIL PROTECTED]:~/parrot$ parrot -C test.pir
5 T(5)
[EMAIL PROTECTED]:~/parrot$ parrot -g test.pir
5 T(5)
Here's the results *without* the above change:
[EMAIL PROTECTED]:~/parrot$ parrot test.pir
5
[EMAIL PROTECTED]:~/parrot$ parrot -j test.pir
5
[EMAIL PROTECTED]:~/parrot$ parrot -S test.pir
5 T(5)
[EMAIL PROTECTED]:~/parrot$ parrot -C test.pir
5 T(5)
[EMAIL PROTECTED]:~/parrot$ parrot -g test.pir
5
The symptoms I see are that get_repr returns the correct string, but
the appropriate REG_STR does not get updated.
Then there is a problem elsewhere. You can inspect interpreter registers
too:
gdb p *interpreter-ctx.bp
In Parrot_PyClass_get_repr, line 197 is:
temp = Parrot_run_meth_fromc_args(INTERP, repr, SELF, REPR, P);
Before this line is executed,
(gdb) p interpreter-ctx.bp
$1 = (struct parrot_regs_t *) 0x40b6bd88
After the above line is executed:
(gdb) p interpreter-ctx.bp
$2 = (struct parrot_regs_t *) 0x40b6bae8
Two questions come to mind:
1) Should this call to run_meth change the context? In this case, repr
happens to be a Closure based on some assumptions I made early on in
this process that I need to revisit. Arguably, this is a bug, but in
the general case, can one always assume that the context does not change?
2) If there is the possibility that the context can change, into which
context should the results of get_repr be placed? Splitting the
statement in op get_repr as I did above guarantees that the macro will
expand using the values *after* the call.
Note: if the context is not supposed to change as a result of runops,
perhaps an internal exception should be thrown.
- Sam Ruby
Re: cvs commit: parrot vtable.tbl
Leopold Toetsch wrote: Sam Ruby wrote: My need is to be able to call add_method for methods defined as @ANON. This doesn't make sense to me. The @ANON pragma avoids the add_method (or store_global) call in the first place. I don't want the method added to a namespace, or stored as a global. I want it stored as a property of a class object. From the perspective of Parrot, namespaces should be viewed a perfectly reasonable mechanism for implementing classes, and there perhaps should even be special support for enabling classes to be defined that way... but: there should be no built in assumptions that all classes are defined in this way. Yes. But OTOH a different scheme should only be used if it's unavoidable. Python classes are not namespaces. Nor are they global. They are dictionaries. Take a look at the last test in parrot/languages/python/t/basic/oo_class.t to get an idea of what I am talking about. - Sam Ruby
runops_args vs NCI
pseudo-code for runops_args:
runops_args(PMC* sub, char* sig, va_list ap) {
dest = VTABLE_invoke(interpreter, sub, NULL);
REG_*[*] = ap[*], as appropriate
if (dest) runops
}
However, VTABLE_invoke on NCI methods is where the real work is done
(including reading from and writing to registers), and a null dest is
returned.
The net effect is that if you try to invoke a NCI with a signature of
PP, the NCI method will be called with whatever happens to be in
PMC_REG(5) at the time, and what it returns will be overwritten with
your first argument.
Is there a reason why the invoke couldn't be done immediately prior to
the call to runops?
- Sam Ruby
Re: Context, wrappers, and rules
Leopold Toetsch wrote:
Parrot_PyClass_get_repr is one such wrapper. How could it be done
better? If you make a specific suggestion there, I'll either adopt it
or produce a test case as a counter example.
You'd need in pyclass.pmc:
PMC* get_repr() {
return pmc_new_string( ... class ...)
}
Just to be clarify, this is from the header comment in pyclass.pmc:
These are the vtable functions for the Python Class base class (i.e.,
methods you would expect to see on python objects).
For reference, here is the current definition of get_repr.
%%%
/*
=item CSTRING *get_repr()
Return the representation of this object.
=cut
*/
STRING* get_repr() {
PMC *repr = VTABLE_find_method(INTERP, SELF, REPR);
if (repr) {
PMC *temp = Parrot_run_meth_fromc_args(INTERP, repr, SELF,
REPR,
PP, SELF);
return VTABLE_get_string(INTERP, temp);
}
else {
STRING *res;
res = string_from_cstring(INTERP, , 0);
res = string_append(INTERP, res,
VTABLE_name(INTERP, VTABLE_get_class(INTERP, SELF)), 0);
res = string_append(INTERP, res,
const_string(INTERP, instance at ), 0);
res = string_append(INTERP, res,
Parrot_sprintf_c(INTERP, %#x, (INTVAL) SELF), 0);
res = string_append(INTERP, res,
const_string(INTERP, ), 0);
return res;
}
}
%%%
In this case, there is a Python specific default. In other cases, there
are other Python specific behaviors like mapping not found to an
exception or to None. In short, I don't see wrappers going away.
A method that returns the string representation of a PyClass object.
That's not a wrapper but an implementation for one specific class. No
wrapper for other classes or a dispatch through run_meth_fromc_args is
needed.
Nothing more will be needed, *when* all is fixed:
* we return PMCs normally (other types are optimizations)
* the methods create new PMCs - no LHS PMC is passed in
* all overloadable MMDs and vtables are registered as NCI methods
like the current METHOD syntax in .pmc files.
get_repr is overloadable. Being overloadable is the norm rather than
the exception in Python.
* Parrot knows common type names (Integer, Float, String, Bool, ...) of
the supported languages
The alternative (which is supported today) is that callers pass in a PMC
which defines a morph method which maps common type names to language
specific alternatives. Take a look at PyObject.pmc where I substitute
PyLong for BigInt.
The advantage of what exists today is that adding a new language does
not require any changes to Parrot. The caller defines the mapping.
* all dispatch is based on VTABLE_find_method
Just to be clear, in the cases where a wrapper is needed (which I would
argue is the majority of cases), this works out to be: a call to
VTABLE_find_method for the wrapper which in turn uses VTABLE_find_method
to access the real logic.
That's a simple scheme and should be well suited for all current target
languages - IMHO.
Again, just to be clear: this hinges on dual assumptions: (1) that
wrappers aren't needed in the majority of cases, and (2) every time
someone gets or sets a method, a mapping can be done from language
defined names to Parrot conventions.
Note that in Python, all attributes may potentially be a method.
- Sam Ruby
Re: Context, wrappers, and rules
Leopold Toetsch wrote: On the other hand, if runops can't change the context, then why is runops_args careful to capture the context prior to calling runops? Crunops is a low-level function that awaits all properly setup. Around that are more function that e.g. create a return continuation and do argument or returnvalue passing. Just use one of these functions as delegate.c does. I *am* using Parrot_run_meth_fromc_args, just like delegate.c does. Single stepping through the debugger, I find that interpreter-ctx.bp is changed by Parrot_Sub_invoke, line 299. The value is not changed after that point, and still retains this new value when control returns to Parrot_get_repr_s_p. - Sam Ruby
Re: overloaded operator calling conventions
Leopold Toetsch wrote: Luke Palmer [EMAIL PROTECTED] wrote: Leopold Toetsch writes: Why do we have the special notion of current_object in the first place? Why not just pass all in as P5, P6, ...? I agree that this is the way to go. Especially if we have some marker somewhere that tells us that we were called as a method. Does the Perl6 compiler know function signatures at compile time? S06 states: Passing two many or too few invocants is a fatal error. For a runtime check we'd need additionally the MMD object count. A few observations, first from an Parrot Internal perspective... in general, the code for the opcodes tend to do things like the following: $1-vtable-get_string(interpreter, $1) Note that the object tends to be repeated as the first argument. It often the case that the first argument matches the object on which the dispatch is based, but it is not necessarily so. In particular, these two values will be different whenever the implementation of a method wants to invoke the equivalent of SUPER(args). As to compile time/runtime checks Python will need complete function signature (not just the number, but the actual names of each of the paramters) at runtime. At the moment, I'm storing this as a property. - Sam Ruby
Re: [CVS ci] class autoload
Leopold Toetsch wrote: 1) There is a new opcode new Px, Sc: new P0, PyInt 2) If the given class doesn't yet exist, Parrot_autoload_class is called which has hardcode mapping for Py = python_group, Tcl = tcl_group. A bit hackish but working ... Perhaps longer term, a change to the pbc format may be in order... the idea would be for each binary to have a list of dependencies which need to be loaded previously. Developers could declare such a dependency with a IMCC macro, something like this: .lib python_group This would enable JITs to convert the above new opcode into the equivalent new Px, Ic variety in many cases. 3) The pybuiltin.__load__() function is automatically called by the library _init loading hook now. There is one more test in t/dynclases/pybuiltins that has an example. Cool, thanks! - Sam Ruby
Re: cvs commit: parrot vtable.tbl
Leopold Toetsch wrote: Sam Ruby wrote: Leopold Toetsch wrote: cvsuser 04/12/15 02:36:29 Modified:.vtable.tbl Log: stub in object vtables [snip] +void add_parent(PMC* parent) +void become_parent(PMC* class) +INTVAL class_type() +void add_method(STRING* method) +void remove_method(STRING* method) +STRING* namespace_name() +PMC* new_singleton() +PMC* get_anonymous_subclass() Cool. Are there plans for opcodes? I could make use of add_parent and add_method... add_parent has already an opcode Caddparent. add_method should probably get one, yes. But add_method is a bit more complicated. It's called automatically during bytecode loading, *if* the class is already existing. E.g. .namespace [Integer] .sub hex method but the problem is code like this: newclass cl, Foo ... .namespace [Foo] .sub foo method now the class isn't constructed, when the bytecode is loaded and the old Parrot_store_global is used. But if you load the method code with load_bytecode again add_method is called. My need is to be able to call add_method for methods defined as @ANON. From the perspective of Parrot, namespaces should be viewed a perfectly reasonable mechanism for implementing classes, and there perhaps should even be special support for enabling classes to be defined that way... but: there should be no built in assumptions that all classes are defined in this way. - Sam Ruby
Context, wrappers, and rules (was: cvs commit: parrot/ops pmc.ops)
Leopold Toetsch wrote: Sam Ruby wrote: Before this line is executed, (gdb) p interpreter-ctx.bp $1 = (struct parrot_regs_t *) 0x40b6bd88 After the above line is executed: (gdb) p interpreter-ctx.bp $2 = (struct parrot_regs_t *) 0x40b6bae8 Then is obviously your implementation of get_repr broken. Two questions come to mind: 1) Should this call to run_meth change the context? Yes because a subroutine is run. But returning from it via a (return-) continuation restores the context and the register frame pointer or it ought to. ... In this case, repr happens to be a Closure based on some assumptions I made early on in this process that I need to revisit. Arguably, this is a bug, but in the general case, can one always assume that the context does not change? Only function calls / returns may change the context. A simple opcode like get_repr isn't allowed to do that. The question isn't about what opcodes are or are not allowed to do. The question is what VTABLE_* functions are or are not allowed to do. Can VTABLE_invoke change the context? How about VTABLE_get_string? If you look at the current delegate PMC, every VTABLE call is translated to a call of a method. Are such methods allowed to change the context? What are the rules? Note: if the context is not supposed to change as a result of runops, perhaps an internal exception should be thrown. We can't check that on each opcode. That just must not happen :) I mustn't have been clear. If things invoked by runops are not allowed to change the context, then runops should throw an internal exception if the context changes. On the other hand, if runops can't change the context, then why is runops_args careful to capture the context prior to calling runops? Again, what are the rules? But before you implement more and more methods I'd rather have inheritance and method calling conventions fixed. Parrot should call the approriate method if it exists in the class namespace (if find_method did return something). How I chose to invest my time is up to me. If all that comes of it is the few minor contributions I have made to the core of Parrot... then I will be OK with that. In most cases, I think I could adapt to a change in a matter of minutes. You've seen me clone classes like BigInt and Complex wholesale, only to gut them back once I had some partial ability to inherit MMD methods. In the worst case, adapting to a major change would be a matter of days. Again, my time to burn. Also, if you note, I am only writing the minimum necessary to pass the given set of tests. That way there is less for me to change when things change. The reason why I am proceeding this way is that I found the previous discussions on topics like object, classes, and metaclasses to be frustrating. I want the next round to be based on specifics. In particular, I don't want to have to respond to statements like It doesn't buy us anything, if we force all languages to create wrappers.. I want to be able to point at a specific wrapper and ask the question: how could it be done better? Parrot_PyClass_get_repr is one such wrapper. How could it be done better? If you make a specific suggestion there, I'll either adopt it or produce a test case as a counter example. - Sam Ruby
[patch] runops
Below is a rather straightforward patch, but as it represents an
interface change (albeit a fully backwards compatible one), I thought I
would post it for discussion.
Background on the proposed change: there apparently are two sets of
runops functions, I'd characterize Parrot_runops_fromc as a do it
yourself function in which you are responsible for all registers, and a
set of convenience functions which take care of marshalling arguments.
At the moment, there is one important way in which the convenience
functions are more functional: set_retval has access to the register set
which were active when the Sub was invoked.
This patch brings Parrot_runops_fromc to parity by providing access to
those registers.
- Sam Ruby
Index: include/parrot/interpreter.h
===
RCS file: /cvs/public/parrot/include/parrot/interpreter.h,v
retrieving revision 1.164
diff -u -r1.164 interpreter.h
--- include/parrot/interpreter.h24 Nov 2004 05:56:55 - 1.164
+++ include/parrot/interpreter.h15 Dec 2004 03:12:20 -
@@ -400,7 +400,7 @@
void runops(Interp *, size_t offset);
void runops_int(Interp *, size_t offset);
-void Parrot_runops_fromc(Interp *, PMC *sub);
+struct parrot_regs_t* Parrot_runops_fromc(Interp *, PMC *sub);
void* Parrot_runops_fromc_args(Interp *, PMC *sub, const char *sig, ...);
INTVAL Parrot_runops_fromc_args_reti(Interp *, PMC *, const char *, ...);
FLOATVAL Parrot_runops_fromc_args_retf(Interp *, PMC *, const char *, ...);
Index: src/inter_run.c
===
RCS file: /cvs/public/parrot/src/inter_run.c,v
retrieving revision 1.24
diff -u -r1.24 inter_run.c
--- src/inter_run.c 13 Dec 2004 21:51:00 - 1.24
+++ src/inter_run.c 15 Dec 2004 03:12:20 -
@@ -102,7 +102,7 @@
/*
-=item Cvoid
+=item Cstruct parrot_regs_t *
Parrot_runops_fromc(Parrot_Interp interpreter, PMC *sub)
Runs the Parrot ops, called from C code. The function arguments are
@@ -113,11 +113,12 @@
*/
-void
+struct parrot_regs_t *
Parrot_runops_fromc(Parrot_Interp interpreter, PMC *sub)
{
PMC *ret_c, *p1;
opcode_t offset, *dest;
+struct parrot_regs_t *bp;
/* we need one return continuation with a NULL offset */
p1 = REG_PMC(1);
@@ -130,11 +131,13 @@
* Passing a dummy true destination copies registers
*/
dest = VTABLE_invoke(interpreter, sub, (void*) 1);
+bp = interpreter-ctx.bp;
if (dest) {
offset = dest - interpreter-code-byte_code;
runops(interpreter, offset);
}
REG_PMC(1) = p1;
+return bp;
}
/*
Re: [perl #33036] [BUG] python dynclasses build failure
Will Coleda via RT wrote: Sam's latest patch seems to have resolved this issue - dynclasses now build, and: perl t/harness t/dynclass/py* skips 1 test, passes everything else. What test is skipped? [EMAIL PROTECTED]:~/parrot/dynclasses$ make test cd .. ; perl -Ilib t/harness t/dynclass/*.t t/dynclass/pybuiltinok t/dynclass/pyclass..ok t/dynclass/pyfunc...ok t/dynclass/pyintok All tests successful. Files=4, Tests=37, 8 wallclock secs ( 7.59 cusr + 0.62 csys = 8.21 CPU) - Sam Ruby
Re: cvs commit: parrot/t/pmc delegate.t object-meths.t objects.t sub.t
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: I'll go back and all the necessary interpinfo calls, but I feel compelled to ask: why isn't P2 passed? P2 is a non-preserved register and as such rather improper to hold the object throughout a whole method. I wasn't assuming is lasted throughout the whole method - only on entry. ... It seems to me quite likely that most methods will want to know what object they are working on. Yes. Just use self which does all the magic you need inside a .sub that is labeled method. Thanks! That works. - Sam Ruby
Re: overloaded operator calling conventions
Leopold Toetsch wrote: Comments? leo As enjoyable as this discussion has been, I'd like to ask that it be put on hold for a few days. I've nearly got all the previously defined languages/python/t/basic tests running, and once they are running, I'd like to do a bit of refactoring and documentation of a complete proposal on how this can work. - Sam Ruby P.S. Peeking at the current implementation may not be wise, as a number of shortcuts were taken (example: int objects behave as their own class, etc), which confuses the picture.
Re: cvs commit: parrot/t/pmc delegate.t object-meths.t objects.t sub.t
Leopold Toetsch wrote: cvsuser 04/12/11 04:08:13 Modified:classes sub.pmc imcc/t/syn op.t runtime/parrot/library parrotlib.imc src inter_run.c t/op gc.t t/pmcdelegate.t object-meths.t objects.t sub.t Log: more pdd03 cleanup * P0, P2 aren't visible in caller aynmore This caused a number of Python tests to fail... specifically any tests which were improperly depending on P2 as PDD03 states: Note particularly that P0-P2 are not passed. I'll go back and all the necessary interpinfo calls, but I feel compelled to ask: why isn't P2 passed? It seems to me quite likely that most methods will want to know what object they are working on. - Sam Ruby .sub main @MAIN newclass $P0, 'c' find_type $I0, 'c' new $P1, $I0 new $P2, .PerlInt $P2 = 1234 setprop $P1, 'p', $P2 $P3 = $P1.m() print $P3 print \n .end .namespace ['c'] .sub m method $P4 = P2 getprop $P5, 'p', $P4 .return ($P5) .end
Re: Dynamic libs don't seem to work
Klaas-Jan Stol wrote: Hello, I just got a fresh cvs checkout, compiled it, compiled pge and tried to make tcl. This is what I get: (cd ../../ ./parrot --output=languages/tcl/lib/tcllib.pbc languages/tcl/lib/tcllib.imc) Couldn't load 'tcl_group': tcl_group: cannot open shared object file: No such file or directory Try rebuilding after you issue the following from your top level parrot directory: perl Configure.pl --prefix=`pwd` - Sam Ruby
Re: Python method overloading
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
I continue to disagree with the part of this conclusion where you insert
find_method into the discussion. To give a concrete example: at the
moment the lookup involved in abs_p_p does not involve the use of
find_method.
$ cat abs.imc
.sub main
.local pmc cl, o
cl = newclass A
$I0 = typeof cl
o = new $I0
$P0 = new Undef
$P0 = abs o
print $P0
.end
.namespace [A]
.sub __absolute
.param pmc d
d = ok\n
.end
$ parrot abs.imc
ok
$ parrot -t abs.imc
...
# find_method class 'A' method '__absolute': Sub
# Calling sub '__absolute'
...
But only for classes that inherit from delegate.
If Perl programmers need to know about Parrot method names, then
effectively this beomes part of the Perl language specification. I do
not have the luxury of changing the Python language specification.
If you are targeting Parrot you have to know the opcode names *and* the
reserved method names, sorry.
People writing Python to Parrot translators need to know Parrot
internals. People who merely write in Python should not.
... (BTW, a combined instantiate method
does not map well to Python which has separate __new__ and __init__
methods.
We have the __init hook too. This is separate.
Ultimately, I believe that this will need to be revisited.
$ parrot -t abs.imc
...
6 new P17, I30 - P17=PMCNULL, I30=72
# find_method class 'A' method '__init': no
... My recommendation
is to stick to primitives, and simply provide a new_p_p).
What is the second _p for?
What I am thinking of is something like:
inline op new(out PMC) {
$1 = VTABLE_instantiate(interpreter, $2);
goto NEXT();
}
Where instantiate does not support a variable number of arguments
(Parrot calling conventions). If you then want to init the PMC you have
obtained, you can use the separate init hook (with a variable number of
arguments).
That is only because the design you have in mind conflates Parrot and
language operations. There is no reason that __abs__ couldn't call
VTABLE_abs, or that __add__ can't make use of MMD_ADD.
And if the class implements it's own __absolute or __add, we do a
separate redispatch? And dynclasses/py* does it differently to
dynclasses/perl*. Why don't you just believe me that that's error prone
and slow ;-)
Only in the sense that CoRoutine and RetContinuation provide
incompatible (i.e., different) implementations of invoke. However,
they are very compatible in the only sense that matters: they both
implement the common protocol named invoke.
Arguably, the very reason that a find_method VTABLE entry was provided
was to enable different PMCs to provide different implementations of
this protocol.
The code that backs perl classes can have implementations find_method
that looks for __add methods with a given parameter signature, and the
code that backs python classes can have implementations of find_metho
that looks for __add__ methods with a different parameter signature.
- Sam Ruby
Re: [CVS ci] class refactoring 1 - Integer
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: I took a look into this. Apparently, in Perl5, the result of xor'ing undef with anything is undef. I'm not suggesting that this is either right or wrong (it actually was surprising to me), Yep. It doesn't really follow the definition of xor, nor does it match the implementation of other types. I'd leave that to language lawyers. Undef (like null in SQL) could be interpreted to mean I don't know, and a case could be made that I don't know xor'ed with anything is I don't know. Furthmore, the concept of a logical xor seems to be a Perl innovation. Languages like C and Python don't have such a concept. Finally, Perl doesn't seem to be consistent. Anything xor undef always seems to return the leftmost argument. *shrug* Unfortunately, this does not work as inheritance seems to take precedence over defaults. Partly. Specific functions are set last. The precedence seems to be specific functions, inheritance, and then defaults. ... I think that this should be the other way around: a subclass should be able to easily override all MMD operations for a given method. The problem is that the passed in mmd init list has no indication, if a default function is installed or a inherited or a specific one. The passed in mmd init list only has specific and defaults. Finally, I took a look at Parrot_mmd_register_parents. Don't ;-) Oops. ;-) ... This seems more complicated than it needs to be, Well, yes. But I'm not even sure if it's worth the effort to improve it. The static MMD table doesn't handle dynamic inheritance. While you can install a function with mmdvtregister, this is just for a pair of types, which is rather useless for classes that inherit from one of the types. But which other functions for which types should be registered too? This is the reason for the proposal of going fully dynamic with MMD too. OK, I'll leave this alone unless I have a specific problem that I need to fix. Hopefully by then, all this will go away. - Sam Ruby
Re: Dynamic libs don't seem to work
Klaas-Jan Stol wrote: Sam Ruby wrote: Klaas-Jan Stol wrote: Hello, I just got a fresh cvs checkout, compiled it, compiled pge and tried to make tcl. This is what I get: (cd ../../ ./parrot --output=languages/tcl/lib/tcllib.pbc languages/tcl/lib/tcllib.imc) Couldn't load 'tcl_group': tcl_group: cannot open shared object file: No such file or directory Try rebuilding after you issue the following from your top level parrot directory: perl Configure.pl --prefix=`pwd` - Sam Ruby thanks, that worked! (what does this --prefix='pwd' do?) dynoplibs still doesn't work, however. is this correct? Ultimately, when Parrot is released and someone installs it, shared libraries will go into a place like /usr/local/lib. So, by default, that's where Parrot is looking for tcl_group. However, you aren't installing it, or copying these shared libraries. Instead you want Parrot to look *here*. That's where prefix comes in. Configure.pl saves your prefix in src/parrot_config.h where it is used to construct paths used at runtime. `pwd` simply is your current working directory. Sorry, I don't know anything about dynopslibs. - Sam Ruby
Re: [CVS ci] class refactoring 1 - Integer
Leopold Toetsch wrote:
Leopold Toetsch wrote:
Currently one test (t/pmc/pmc_43.pasm) is failing due to wrong
inheritance.
Actually not because of inheritance. The implementation of
PerlUndef.logical_xor was bogus. I've fixed this and the test.
I took a look into this. Apparently, in Perl5, the result of xor'ing
undef with anything is undef. I'm not suggesting that this is either
right or wrong (it actually was surprising to me), but if Parrot wanted
to provide this behavior, the seemingly obvious way for this to be coded
would be for PerlUndef to have the following:
void logical_xor (PMC* value, PMC* dest) {
MMD_DEFAULT: {
VTABLE_set_pmc(INTERP, dest, SELF);
}
Unfortunately, this does not work as inheritance seems to take
precedence over defaults. I think that this should be the other way
around: a subclass should be able to easily override all MMD operations
for a given method.
Finally, I took a look at Parrot_mmd_register_parents. This seems more
complicated than it needs to be, and that is because it is only looking
at the mmd table (the final results after overriding) instead of at the
specific overrides (the various _temp_mmd_init arrays).
An alternate approach would be for the make process to generate a
function that registers a given class into a given type slot, and code
to call each parent in order (top down) to override their slots as required.
A concrete example might help:
Parrot_PerlInt_mmd_init(interp, entry) {
const MMD_init _temp_mmd_init[] = {...};
Parrot_mmd_register(interp, entry, _temp_mmd_init, N_MMD_INIT);
}
followed later by the following calls:
Parrot_scalar_mdd_init(interp, entry);
Parrot_Integer_mmd_init(interp, entry);
Parrot_PerlInt_mmd_init(interp, entry);
- Sam Ruby
Re: [CVS ci] class refactoring 1 - Integer
Eirik Berg Hanssen wrote: Leopold Toetsch [EMAIL PROTECTED] writes: Sam Ruby [EMAIL PROTECTED] wrote: I took a look into this. Apparently, in Perl5, the result of xor'ing undef with anything is undef. I'm not suggesting that this is either right or wrong (it actually was surprising to me), Yep. It doesn't really follow the definition of xor, nor does it match the implementation of other types. Which Perl5 (xor, undef) would this be? It does not look like the result is undef around here: My bad. I misinterpreted an empty string as an undef. Here's the test I ran: my $a = undef; $c = $a xor $a; print $c; $c = $a xor 0; print $c; $c = $a xor 1; print $c; $c = 2 xor $a; print $c; $c = 3 xor 4; print $c; $c = 0 xor $a; print $c; $c = 5 xor 0; print $c; $c = 0 xor 6; print $c; print \n; perl -v returns: This is perl, v5.8.4 built for i386-linux-thread-multi - Sam Ruby
Re: [CVS ci] class refactoring 1 - Integer
Mike Guy wrote: Perl5 Cxor always returns a standard boolean value, i.e. dualvar(0, '') or dualvar(1, '1').Perl6/Parrot should do the same thing. Try: perl -le print 'day' xor 'night' On the version of Perl I have installed, I get day as the result. - Sam Ruby
Re: Python method overloading
Leopold Toetsch wrote: Leo - at one point you indicated that you might be interested in helping to factor out the common code again. Sure, and I'm not stopping that. The overall conclusion of (even infix operator) method lookup was that it has to be done at runtime. It is up to the PMCs to provide a proper and language-specific find_method to locate the involved operation, MMD or not. I continue to disagree with the part of this conclusion where you insert find_method into the discussion. To give a concrete example: at the moment the lookup involved in abs_p_p does not involve the use of find_method. Nor does the lookup involved in find_method_p_p_s for that matter. If Perl programmers need to know about Parrot method names, then effectively this beomes part of the Perl language specification. I do not have the luxury of changing the Python language specification. I continue to push for the namespace for Parrot internal dispatching and language level method names to be disjoint. If Python metaclasses get surprising results if they happen to define a method named instantiate, expect a bug report. (BTW, a combined instantiate method does not map well to Python which has separate __new__ and __init__ methods. You are going to find issues like these every time you try to put object oriented semantics into the Parrot core. My recommendation is to stick to primitives, and simply provide a new_p_p). There needs to be separate find_method operations for external (i.e., visible at the language level) names and Parrot internal names. And the runcore is responsible for calling the method. While you seem to admit that it has to be done at runtime you are doubting that it can be done fast and you are rolling your own incompatible dispatch scheme (how should that be faster then?). This made me utter above sentence. as long as we have a proper protocol that everyone can conform to, we should be OK. - Dan Sugalski, 2004/11/29 Further: The function signature of overloaded infix operations (and maybe others) is currently not appropriate for Python (and likely other languages). That is only because the design you have in mind conflates Parrot and language operations. There is no reason that __abs__ couldn't call VTABLE_abs, or that __add__ can't make use of MMD_ADD. - Sam Ruby
Re: Premature pessimization
Ah! Now we are getting somewhere! Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: So *all* lookups (complete with the asterisks) does not mean *all* lookups. How about invoke? Let's first concentrate on simpler stuff like infix operators. OK, but the point is that there will always be multiple mechanisms for dispatch. Citing S06: Operators are just subroutines with special names. That statement is true for Perl. Same statement is true for Python. But the names vary based on the language. Yes. So let's factor out the common part and have that in Parrot core, usable for Python and Perl and ... The PyInt PMC currently duplicates almost all functionality that *should* be in the Integer PMC. We have first to fix the Integer PMC to do the Right Thing. Then we need some syntax for multiple inheritance in PMCs. The same holds for other PMCs. It was already proposed that we should have a language-neutral Hash PMC. No question that that is the intended final goal. What you see in the current python dynclasses is not representative of that final goal. So, why have I proceeded in this manner? Two reasons. First, I am not about to make random, unproven changes to the Parrot core until I am confident that the change is correct. Cloning a class temporarily gives me a playground to validate my ideas. Second, I am not going to wait around for Warnocked questions and proposals to be addressed. Now, neither of the above are absolutes. You have seen me make changes to the core - but only when I was relatively confident. And I *have* put on hold trying to reconcile object oriented semantics as this is both more substantial and seemed to be something that was likely to be addressed. Also, while I am not intending to make speculative changes to the core of Parrot, I don't have any objections to anybody making changes on my behalf. If you see some way of refactoring my code, go for it. It isn't mine - it is the community's. The one thing I would like to ask is that test cases that currently pass continue to pass. The dynclass unit tests are part of the normal test. Additionally, the tests in languages/parrot have been the ones driving most of my implementation lately. I do realize that that means checking out Pirate. Even though I don't agree with it, I do understand Michal's licensing issues. The reason I am not investing much time in resolving this issue is that Pirate is exactly one source file and could quickly be rewritten using the Perl 6 Grammar engine once that functionallity becomes sufficiently complete. So given that we have a set of language-neutral PMCs in core that do the right thing, Python or Perl PMCs can inherit a lot of functionality from core PMCs. Language-specific behavior is of course implemented in the specific PMC. Agreed. One area that will require a bit more thought is error cases. The behavior of integer divide by zero is likely to be different in each language. This could be approached in a number of different ways. One is by cloning such methods, like I have done. Another is to wrap such methods, catch the exception that is thrown, and handle it in a language specific manner. A better approach would be for the core to call out to a method on such error cases. Subclasses could simply inherit the common core behavior and override this one method. It also means that the normal execution path length (i.e., when dividing by values other than zero) is optimal, it is only the error paths that involve extra dispatches. That's an easy case. Overflow is a bit more subtle. Some languages might want to wrap the results (modulo 2**32). Some languages might want an exception. Other languages might want promotion to BigInt. Even if promotion to BigInt were the default behavior, subclasses would still want to override it. In Python's case, promotion to PyLong (which ideally would inherit from and trivially specialize and extend BigIt) would be the desired effect. Even this is only one aspect of a more general case: all morphing behavior needs to be overridable by subclasses. I believe that this can be easily handled by the current Parrot architecture by virtue of the fact that destination objects must be created before methods are called, and such destination objects can override morph methods). But it would help the cause if code were written to promote things to Integer instead of PerlInt. Yes, at the moment, I'm guilty of this too. Second: method dispatch. I've looked a bit into PyObject. It seems that you start rolling your own method dispatch. Please don't get me wrong, I'm not criticizing your implementation. It might also be needed for some reasons I'm just overlooking and it's currently needed because core functionality isn't totally finished. I'll address your questions below, but for reference, here is the code that Pirate generates for a=b+c: find_type $I0, 'PyObject' new $P0, $I0
Re: Premature pessimization
Leopold Toetsch wrote:
Good. Now Evil Leo (who can't program in Python ;) writes some piece of
code like this:
$ cat m.py
class M(type):
def __new__(meta, name, base, vars):
cls = type.__new__(meta, name, base, vars)
cls.__add__ = myadd
return cls
def myadd(self, r):
return 44 - r
I = M('Int', (int,), {})
i = I(5)
print i
print i + 2
$ python m.py
5
42
What this means is that the __add__ method will not be directly used for
either PyInt or PyString objects
Well, and that's not true, IMHO. See above. It has to be part of
Parrot's method dispatch. What if your translator just sees the last 3
lines of the code and M is in some lib? That implies that you either
can't translate to $P0 = $P1 + $P2, or that you just translate or
alias __add__ to Parrot's __add and let Parrot fiddle around to find
the correct method.
Here's the part that you snipped that addresses that question:
And there is a piece that I haven't written yet that will do the
reverse: if MMD_ADD is called on a PyObject that has not provided
such behavior, then an any __add__ method provided needs to be
called.
* the method is returning a new PMC. This doesn't follow the signature
of Parrot infix MMD operations.
Here I do think you are misunderstanding. The __add__ method with
precisely that signature and semantics is defined by the Python language
specification. It is (somewhat rarely) used directly, and therefore
must be supported exactly that way.
| __add__(...)
| x.__add__(y) == x+y
Parrot semantics are that the destination exists. But having a look at
above myadd, we probably have to adjust the calling conventions for
overloaded infix operators, i.e. return the destination value. Or
provide both schemes ... dunno.
Since you provided an Evil Leo sample, let me provide an Evil Sam sample:
d = {
__init__: lambda self,x: setattr(self, value, x),
__add__: lambda self,x: str(self.value) + str(x.value)
}
def dict2class(d):
class c: pass
c.__dict__.update(d)
return c
c = dict2class(d)
a=c(2)
b=c(3)
print a+b
Things to note:
1) classes which are created every time a function is called
2) classes are thin wrappers over a dictionary object
Now, given the above sample, let's revisit the statement that The
Python translator needs just a translation table for the common core
methods.
How, exactly, would that be done? Given that the method name is simply
a string... used as a key in dictionary... with a different parameter
signature than the hypothetical Parrot __add method.
That's why I say:
In the general case, looking for reserved method names at compile
time doesn't work.
__add__ is reserved in Python and corresponds directly to __add in
Parrot. I don't think that doesn't work.
__add__ is *not* reserved in Python. There just is some syntatic sugar
that provide a shorthand for certain signatures. I am free to define
__add__ methods that have zero or sixteen arguments. I won't be able to
call such methods with the convenient shorthand, but other than that,
they should work.
I personally don't think that performance considerations should be out
of bounds in these discussions
I've already shown that it's possible to go with fully dynamic dispatch
*and* 30% faster for MMD and 70% faster for overloaded operations. First
correct and complete, then speed considerations.
Neither of which match Python semantics. We are going to need a system
where classes are anonymous, not global. Where methods are properties
that can be added simply by calling the equivalent of set_pmc_keyed.
- Sam Ruby
Re: Python method overloading
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
Leopold Toetsch wrote:
Here's the part that you snipped that addresses that question:
And there is a piece that I haven't written yet that will do the
reverse: if MMD_ADD is called on a PyObject that has not provided
such behavior, then an any __add__ method provided needs to be
called.
Ok. But that would imply that HLL interoperbility isn't really possible. Or
just at a minimal surface level. But see below.
I don't believe that to be the case. If a Perl subroutine were to call
a Python function and pass a PerlInt as a parameter, the receiving
function should expect to be able to do addition via tha + operator,
but should not be expect to find an __add__ method on such objects.
Instead, and if they cared to, they could explicitly call the __add
method provided.
The reverse should also be true, if Python function were to call a Perl
subroutine and pass a PyInt as a parameter, the receiving subroutine
should expect to be able to do addition via the + operator, but not
expect to find an __add method on such objects. Instead, and if they
cared to, they could explicitly call the __add__ method provided.
I would consider that significant interoperability with only minimal
restrictions.
While the Python people aren't stopping to talk about the clearness of
their language, nothing is clear and explicit, when it comes to overloading
or metaclasses.
Please don't do that. I am not trying to extoll the virtues of Python,
merely trying to implement it.
Anyway, IMHO, class.__add__ = foo or your example manipulating
class.__dict__ (another special attribute name!) is the point, where
you can install Parrot semantics WRT method overloading.
Hold that thought. I'll answer this below.
Now, given the above sample, let's revisit the statement that The
Python translator needs just a translation table for the common core
methods.
We both know that's a simplification :) You've to install the methods of
course ...
Again, it can't be done exclusively at translation time. It needs to be
done at runtime. And if it is done at runtime, it need not be done at
translation time at all. More below.
How, exactly, would that be done? Given that the method name is simply
a string... used as a key in dictionary... with a different parameter
signature than the hypothetical Parrot __add method.
The class.__dict__ dictionary is special. Setting an __add__ key too.
The combined meaning is overloading. The different signature is a
problem, yes - I've already mentioned that. And Parrot's __add method
is not hypothetical :-)
$ grep __add t/pmc/object*.t
Here I'll apologize for being unclear. Yes, there is code in the
existing object class in support of Perl's S06. What's hypothetical is
the presumption that all languages will adopt Perl 6's naming convention
for methods.
That's why I say:
In the general case, looking for reserved method names at compile
time doesn't work.
__add__ is reserved in Python and corresponds directly to __add in
Parrot. I don't think that doesn't work.
__add__ is *not* reserved in Python.
Does it matter if the name is actually reserved? The meaning is
important.
It does matter. Python classes are dictionaries of objects, some of
which may be functions. You may extract objects from that dictionary
and access them later. The meaning in such a scenario is not apparent
until well after all interaction with the compile and runtime
dictionaries is over.
... There just is some syntatic sugar
that provide a shorthand for certain signatures. I am free to define
__add__ methods that have zero or sixteen arguments. I won't be able to
call such methods with the convenient shorthand, but other than that,
they should work.
I'd say, if you define an '__add__' method with 16 arguments, Python
will throw an exception, if you try to use C+ with an object of that
class:
If I define an __add__ method with 16 arguments, Python will not throw
an exception.
I've already shown that it's possible to go with fully dynamic dispatch
*and* 30% faster for MMD and 70% faster for overloaded operations. First
correct and complete, then speed considerations.
Neither of which match Python semantics. We are going to need a system
where classes are anonymous, not global.
Why? And how do you find your class then:
c = C()
...
3 22 LOAD_NAME1 (C)
25 CALL_FUNCTION0
$pirate -d c.py
...
find_lex $P0, 'C'
$P1=$P0()
store_lex -1, 'c', $P1
The important part isn't simply in which hash a given class name is
looked up in, but that classes themselves in Python are transient
objects subject to garbage collection.
... Where methods are properties
that can be added simply by calling the equivalent of set_pmc_keyed.
Nah. Methods aren't properties, but ...
No? Try the following:
x = abcdef.find
print x('c')
The set_pmc_keyed on __dict__ (or an equivalent setattribute
Re: Python method overloading
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: [ snipped - all ok } If I define an __add__ method with 16 arguments, Python will not throw an exception. I didn't write that. I've said: *if* you call it via a + b, Python throws an exception - that one I've shown. Anyway... What you wrote (and snipped) was I'd say, if you define an '__add__' method with 16 arguments, Python will throw an exception,... To which I responded with the above. You are still not getting the principal of the scheme, IMHO. It has nothing to do with Perl6 or any other language, nor with Python. Either that, or I *am* getting the principle of the scheme. I guess that this is the point where I need to return back to writing code and test cases. Leo - at one point you indicated that you might be interested in helping to factor out the common code again. Please feel free to do so whenever you are ready. All I ask is that you don't break the test cases. - Sam Ruby P.S. No fair changing the test cases either. ;-)
Re: Premature pessimization
Leopold Toetsch wrote: The current delegation internals are not likely a good match for languages like Python or Ruby. I see such languages as implementing their own semantics for classes, and Parrot limiting its knowledge to methods like findmeth and invoke. Another good reason to use pmc-vtable-fine_method in *all* lookups. So the PMC has full control over the dispatch. How does one lookup the Cfind_method method? That will remain a VTABLE, entry right? All *external* names should be looked up using find_method. All names needed by the Parrot runtime need to be orthogonal to the set of names available for use by languages. Ideally this would be by virtue of using a separate mechanism (e.g., VTABLES), but acceptably it could be done by naming convention (leading double underscores with no trailing double underscores). We also need to be aware that in the general case, we are talking about two lookups. One to find the language specific wrapper. And one to find the common code which backs it. The second dispatch may be spelled string_str_index, in which case the loader does all the necessary fixup. The second dispatch may be spelled VTABLE_get_pmc_keyed in which case we are looking at a couple of pointer hops and some integer arithmetic. The second dispatch may be spelled SUPER() and use some combination of these. Or it could be using find_method, or perhaps even another technique entirely. But in all cases, we are looking at two different things. = = = Why is this issue so important to me? I see a lot of things I like about Parrot, but at the moment, the notions that are baked into the core as to how classes should work is not one of them. I see a change that went in today to invalidate a method cache if a store_global call is made. That presumes a lot about how find_method works internally. An assumption that will not be valid for a number of languages. ... What do we benefit from the premature pessimisation of mapping this to a string? Well, method names happen to be strings ;) Using the table below, at the moment, there are exactly zero strings materialized and/or compared against during the execution of vtable/PMC methods. I don't get that sentence. VTABLE_xxx depends only on pointer arithmetic and dereferencing. Yes, 20% of the operations may need more, and that should be provided for, but if 80% of the calls can be done this way, the performance benefits are significant. As you tirelessly point out, this is all internal. The way the ops are coded in the IMCC source need not change. However, prematurely mapping *all* method names to strings so that a second method dispatch can be done in 100% of the cases is a premature pessimization. = = = I plan to spend the next day or so looking into catching named exceptions, after which point I plan to implement Python classes on Parrot. I do not anticipate using globals or ParrotObject or ParrotClass in that implementation. I expect that there will be tests that replace methods, and I hope that optimizations that attempt to cache the results of find_method don't prevent these updates from being seen. I don't believe that it is a good idea for Python specific functionallity to be present in the core Parrot runtime. For that reason, I will implement this functionality inside dynclasses. If the runtime provides reasonable hooks for the functionally, I will use them. If not, I will temporarily clone and modify what I need, with the intent of getting it working first, and then refactoring the appropriate parts back into the Parrot runtime, this time with the necessary intercept points. In rare cases - when I feel confident enough - I may modify the Parrot runtime to provide the hooks I need. And all of this will be backed by test cases so that such refactoring can be done confidently and correctly. If anybody has any issues with this plan, please let me know now. - Sam Ruby
Re: [CVS ci] pow, hash, instantiate
Leopold Toetsch wrote: I've added a few ops and methods: * pow MMD function and opcodes * hash vtable and opcode Both aren't yet implemented in classes/* The new_extended opcode and vtable got renamed to the more appropriate instantiate. classes/complex.pmc has an implementation of this method, the default is a fallback to pmc_new(). Excellent! Question: what is the difference (conceptually) between new and instantiate? If they are different, these differences should be highlighted. If not, the opcode should be named the same in both cases. Also, I would prefer not to have to deal with implicit registers. new_p_p would be the method signature I would prefer. The name of the vtable entry could remain instantiate. - Sam Ruby
Re: Premature pessimization
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
Leopold Toetsch wrote:
Another good reason to use pmc-vtable-fine_method in *all* lookups. So
the PMC has full control over the dispatch.
How does one lookup the Cfind_method method? That will remain a
VTABLE, entry right?
Yes of course.
So *all* lookups (complete with the asterisks) does not mean *all* lookups.
How about invoke?
All *external* names should be looked up using find_method.
Yes. But what's an external name? What happens if a Perl6 guru installs
a global overloaded add function:
multi sub *infix:+(Int $l, Int $r) { ... }
for example to count all add operations or whatever. This can
currently be done with the Cmmdvtregister opcode. But that installs only
one MMD function for one type pair. What if some classes inherit the
__add method from CInt. Or:
multi sub *infix:+(Any $l, Any $r) { ... }
which presumably would influence all Perl types.
Citing S06: Operators are just subroutines with special names.
That statement is true for Perl. Same statement is true for Python.
But the names vary based on the language.
IMCC should be independent of Perl semantics.
Doing the same overloading with prefix or postfix operators OTOH would
change vtable methods. That's currently not possible because the vtable
dispatch is static, except for objects where it's dynamic.
We also need to be aware that in the general case, we are talking about
two lookups. One to find the language specific wrapper. And one to
find the common code which backs it.
Yes, but when the HLL method is found (e.g. PyStr.index) you just call
the implementation statically and adjust e.g. return values.
The second dispatch may be spelled string_str_index,
... the second dispatch is basically not dispatch, it's a function call.
So far, I have utterly failed to get my point across.
Either all languages are going to need to adopt a common set of special
names (not going to happen), or Parrot is a single language runtime, or
the mechanism provided to do a language level dispatch needs to be
orthogonal to the way that Parrot internal dispatches are done.
Currently, Parrot is the latter. It is the suggestion that it moves
towards the previous alternative that I object to.
Why is this issue so important to me? I see a lot of things I like
about Parrot, but at the moment, the notions that are baked into the
core as to how classes should work is not one of them.
You can straightforwardly use Parrot's object system for attributes and
methods with all stuff known at compile time, AFAIK. You basically just
extend the find_method and get_attr_str vtables.
straighforwardly... known at compile time?
Python's classes are glorified hashes, populated at runtime. The are
essentially anonymous and lexically scoped.
WRT find_method: S13 mentions multi subs per package or even lexically
scoped. So we'll probably have to extend the method lookup anyway.
I see a change that went in today to invalidate a method cache if a
store_global call is made. That presumes a lot about how find_method
works internally. An assumption that will not be valid for a number of
languages.
Well, the builtin method lookup system with the default find_method
implementation is using it. But the cache is fully transparent.
VTABLE_xxx depends only on pointer arithmetic and dereferencing. Yes,
20% of the operations may need more, and that should be provided for,
but if 80% of the calls can be done this way, the performance benefits
are significant.
Please don't start arguing with estimated performance benefits. I've
already shown that we can do the current MMD operations 30% faster -
but with dynamic dispatch too.
We need at the opcode level a dynamic dispatch. Internally the vtable
call remains. I've written that already a few times.
I think that all we can agree upon here is the MMD support is both
incomplete and unoptimized. Making decisions based on how this
implementation performs is premature.
As you tirelessly point out, this is all internal. The way the ops are
coded in the IMCC source need not change. However, prematurely mapping
*all* method names to strings so that a second method dispatch can be
done in 100% of the cases is a premature pessimization.
Well better schemes for method overloading (e.g. in Python metaclass)
are always welcome. I've proposed a scheme. And yes, it's an
implementation detail.
I watched the CLR folks try to come to consensus on as simple a matter
as multiple inheritance. Each language will aproach this differently.
The best Parrot can hope to do is to standardize on a protocol, and to
provide enough primitives to enable languages to build proper support.
[ dynclasses ]
If anybody has any issues with this plan, please let me know now.
Python specific stuff should of course move out from core. For the rest:
It depends. If you end with a duplicate of CPython's Objects directory
interoperbility will be difficult.
If interoperability is defined as implementing
Re: Premature pessimization
Leopold Toetsch wrote: correct. I've discovered and analysed the problem with continuations. I've made a proposal to fix that. No one has said that it's technically wrong or couldn't work. It seems you are liking the idea, but Dan doesn't. Now what? I would suggest focusing on one issue at a time and starting with writing a test case. But, overall, it is clear that you are frustrated. It show when you give responses like Doesn't really matter when people like me ask questions about your proposals. That, in turn, makes people like me frustrated. Re: classoffset... I don't currently use this opcode, nor do I plan to. This opcode has a builtin assumption that attributes are stored as attributes. They may in fact be methods or properties in the Parrot sense, or Property's in the Python sense. Re: VTABLES... I disagree with you on this one. Prematurely mapping an operation to a string is a premature pessimization. Add to that the fact that such names can conflict with usages by languages of this runtime. Overall, the design of the opcodes and PMC layout should focus on trying to retain as much information as possible. Re: continuations... frankly, I'm hoping that a solution emerges that doesn't involve significant reduction in functionallity. I might be misunderstanding you, but it sounds to me like you are proposing ditching lexical pads. - Sam Ruby
Re: Premature pessimization
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
Leopold Toetsch wrote:
correct. I've discovered and analysed the problem with continuations.
I've made a proposal to fix that. No one has said that it's technically
wrong or couldn't work. It seems you are liking the idea, but Dan
doesn't. Now what?
I would suggest focusing on one issue at a time and starting with
writing a test case.
But, overall, it is clear that you are frustrated.
... a bit by the lack of progress that can be achieved to e.g. fix
broken continuation behavior. But not much ;)
My philosophy is simple: things without test cases tend not not get
fixed, and when fixed, tend not to stay fixed.
... It show when you
give responses like Doesn't really matter when people like me ask
questions about your proposals. That, in turn, makes people like me
frustrated.
Well, you worried about different classes having identical method names,
but the methods do different things. I asked why this is a problem. I
gave an example. Here is one more:
obj.__set_integer_native(10)
does 2 totally different things for arrays (set array size) or for
scalars (assign value 10).
I am worried about Parrot trying to establish common public names for
common functions. Many of the examples you gave did not include the
prerequisite double underscores. For example: sub, imag, eof.
These names are not likely to have a commmon behavior across all languages.
Re: VTABLES... I disagree with you on this one. Prematurely mapping an
operation to a string is a premature pessimization.
Except that we are doing that already. Please read through
classes/delegate.c. The problem is that this scheme doesn't work for
MMD of real objects.
I don't know enough to have an informed opinion yet on MMD. But
__set_integer_native is not a MMD operation. It already is efficiently
dispatched to the correct method. What do we benefit from the premature
pessimisation of mapping this to a string?
... Add to that the
fact that such names can conflict with usages by languages of this
runtime.
Parrots method names (two leading underscores + vtable name) are
reserved. And again, when two distinct clases use the same method name
this is no conflict at all. Issues with classes in an inheritance chain
are easily avoided by following that convention.
Again, my point is that the examples need to follow that convention.
Without exception.
... Overall, the design of the opcodes and PMC layout should focus
on trying to retain as much information as possible.
Nothing get lost. Visible opcodes don't change. An example
.sub m
$P0 = newclass Foo
$I0 = find_type Foo
.local pmc obj
obj = new $I0
obj = 5
.end
running that snippet gives:
Can't find method '__set_integer_native' for object 'Foo'.
The vtable method set_integer_native clearly maps to a real method that
can be either inherited or be provided by the user. This doesn't work
for MMD functions.
Again, set_integer_native is not an MMD function?
Re: continuations... frankly, I'm hoping that a solution emerges that
doesn't involve significant reduction in functionallity. I might be
misunderstanding you, but it sounds to me like you are proposing
ditching lexical pads.
No. I'm not ditching lexical pads at all. They are of course needed. The
top pad is in the registers. Outer pads are looked up as usual.
I guess I don't understand. I guess it is time for a test case. ;-)
How would the following work if the top pad is in registers?
var = 1
def g():
global var
var = 2
print before, var
g()
print after, var
Such behavior is the default for perl5:
my $var = 1;
sub g {
$var = 2;
}
print before $var\n;
g();
print after $var\n;
- Sam Ruby
Re: [perl #32877] parrot build broken in Tru64, cc/ld confusion
Andrew Dougherty wrote: The offending line in config/gen/makefiles/dynclasses_pl.in is probably this one: $LD $CFLAGS $LDFLAGS $LD_LOAD_FLAGS $LIBPARROT That CFLAGS doesn't belong there. CFLAGS are intended to be sent to $CC, not to $LD. The command being called here is $LD, which is defined in config/init/data.pl as the Tool used to build shared libraries and dynamically loadable modules. I can't find anything that fails if this is removed, so I committed the change. - Sam Ruby
Re: Premature pessimization
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
Leopold Toetsch wrote:
My philosophy is simple: things without test cases tend not not get
fixed, and when fixed, tend not to stay fixed.
There is of course a test case. I have mentioned it at least 20 times ;)
t/op/gc_13.imc - currently using lexicals.
$ parrot t/op/gc_13.imc
3 * 5 == 15!
What's broken?
I am worried about Parrot trying to establish common public names for
common functions. Many of the examples you gave did not include the
prerequisite double underscores. For example: sub, imag, eof.
Ah ok. Sorry. __subtract. OTOH some might be that common and the usage
is the same in all languages that we might use that common name. But
that's probably not worth the effort. So yes, we should establish the
notion that all methods follow that convention.
Cool.
Re: VTABLES... I disagree with you on this one. Prematurely mapping an
operation to a string is a premature pessimization.
Except that we are doing that already. Please read through
classes/delegate.c. The problem is that this scheme doesn't work for
MMD of real objects.
I don't know enough to have an informed opinion yet on MMD. But
__set_integer_native is not a MMD operation. It already is efficiently
dispatched to the correct method.
For PMCs yes. For objects not efficiently and only with a separate
delegate meta-class.
The current delegation internals are not likely a good match for
languages like Python or Ruby. I see such languages as implementing
their own semantics for classes, and Parrot limiting its knowledge to
methods like findmeth and invoke.
... What do we benefit from the premature
pessimisation of mapping this to a string?
Well, method names happen to be strings ;)
Using the table below, at the moment, there are exactly zero strings
materialized and/or compared against during the execution of vtable/PMC
methods. To the extent that these are represent common operations, this
can be significant from a performance perspective.
Anyway, I'll try to summarize our current method dispatch scheme:
vtableMMD NCI method
-
opcodeset P0, 4 add P0, P1, P2 io.__eof()
-
PMC pmc-vtable-.. mmd_dispatch_* callmethod
objectdelegate.c 1) callmethod
object(PMC) deleg_pmc.c 1) callmethod
NCI methods are working, the method lookup is dynamic, inheritance
works.
MMD inheritance is totally static (set up during PMC compilation).
PMCs dispatch with the mmd_dispatch_* functions in ops/*.ops.
1) overloading a MMD infix operation needs the mmdvtregister opcode, but
this does not effect any class inheritance and it's just for the given
two types.
Multi-dimensional MD is not implemented.
For vtables objects and objects derived from PMCs use two helper classes
that bridge the dynamic inheritance of objects to the static inheritance
in PMCs. This doesn't support runtime overloading of methods that
defaulted to the PMC method.
Looking at 5 different schemes that work for ~50% of the cases can -
well - make one pessimistic ;)
I'm proposing that *internally* (implementation detail ...) one scheme
should be enough. Again: the opcodes don't change.
I don't think that there can be a single dynamic mechanism that works
for all languages, let alone one that works for Parrot internals too.
For starters, findmeth is a method. One that merits its own VTABLE
entry (otherwise, how would you find it?). Others that map cleanly onto
existing language syntax also can benefit from such optimizations.
MMD might help out with mutiple language interop, but Python as
currently designed doesn't need any such facility.
The vtable method set_integer_native clearly maps to a real method that
can be either inherited or be provided by the user. This doesn't work
for MMD functions.
Again, set_integer_native is not an MMD function?
__set_integer_native is a vtable method.
Cool.
No. I'm not ditching lexical pads at all. They are of course needed. The
top pad is in the registers. Outer pads are looked up as usual.
I guess I don't understand. I guess it is time for a test case. ;-)
How would the following work if the top pad is in registers?
The top pad is always the currently executing one.
var = 1
def g():
global var
var = 2
print before, var
g()
print after, var
It's obviously up to the global to do the right thing. My impression
(I don't have more Python knowledge ;) is, that global refers to the
outermost lexical pad in the main module.
Such behavior is the default for perl5:
my $var = 1;
sub g {
$var = 2;
}
As there is no my $var in the inner pad, that's just a Cfind_lex
opcode. It follows the Cprev pointer to the outer pad and finds in the
lexical hash the $var which maps to a position
Re: Perl 6 Summary for 2004-11-29 through 2004-12-06
Matt Fowles wrote: keyword arguments Sam Ruby wondered how he out to handle keyword arguments to functions. Dan admitted that this is complex and outlined the cheat he has been contemplating. No one has either commented on or implemented it yet. http://xrl.us/d955 Oh, yes, I did. ;-) Here's a test case that now passes: http://pirate.versionhost.com/viewcvs.cgi/pirate/test/python/varargs.py?rev=HEADcontent-type=text/vnd.viewcvs-markup At the moment, there is no Parrot conventions for such things, so this can't be expected to iteroperate across languages; but if such conventions ever do materialize, I'll adjust. - Sam Ruby
Re: MMD: more implications
General thoughts on the proposal to replace VTABLES with methods: 1) Anything that is unary (i.e., contains exactly or one argument) is not affected by MMD. There is no need to replace any such methods. 2) While I don't see Python translators using a sin opcode, I can see the implementation of Math.sin doing a VTABLE_sin(INTERP,value). 3) My biggest problem with the runtime introducing methods is that language semantics vary. Here's a concrete example: both Ruby and Python have methods named index on string. Ruby returns Cnil if not found. Python throws Cvalue_error if not found. String.replace may be an even better example. Ruby and Python's methods by this name don't mean the same thing or even have the same signature. - - - Overall, for any non-trivial method, I think we are looking at a double dispatch: first to the language specific wrapper, and then to the common code which backs it. There are advantages and disadvantages to making the dispatch methods the same. Ultimately, if they are the same, the names should be picked in a way that minimizes the possibility of collisions. If they differ, no such possibility exists. - Sam Ruby
perlhash iter busted
Test case attached. - Sam Ruby Index: t/pmc/perlhash.t === RCS file: /cvs/public/parrot/t/pmc/perlhash.t,v retrieving revision 1.48 diff -u -r1.48 perlhash.t --- t/pmc/perlhash.t1 Oct 2004 21:16:52 - 1.48 +++ t/pmc/perlhash.t3 Dec 2004 19:18:51 - @@ -19,7 +19,7 @@ =cut -use Parrot::Test tests = 36; +use Parrot::Test tests = 37; use Test::More; output_is(CODE, OUTPUT, Initial PerlHash tests); @@ -1215,4 +1215,19 @@ 0 OUTPUT +output_is( 'CODE', 'OUTPUT', iter); +##PIR## +.sub __main__ @MAIN +new P0, .PerlHash +set P0['a'], 'x' +iter P1, P0 +shift P2, P1 +print P2 +print \n +end +.end +CODE +a +OUTPUT + 1;
Re: Objects, classes, metaclasses, and other things that go bump in the night
Leopold Toetsch wrote:
Dan Sugalski [EMAIL PROTECTED] wrote:
Anyway, so much for the 'outside world' view of objects as black box
things that have properties and methods.
[ ... ]
Almost everything we do here is going to be with method calls.
There's very little that I can see that requires any faster access
than that, so as long as we have a proper protocol that everyone can
conform to, we should be OK.
The distinction object vs class is still in the way a bit, when it comes
to method calls.
$ python
...
i = 4
i.__sub__(1)
3
help(int)
...
| __sub__(...)
| x.__sub__(y) == x-y
Given that a PyInt is just a PMC it currently needs a lot of ugly hacks
to implement the full set of operator methods. dynclasses/py*.pmc is
mostly just duplicating existing code from standard PMCs.
Iff the assembler just emits a method call for the opcode:
sub P0, P1, P2
we have that equivalence too, with proper inheritance and static and dynamic
operator overloading.
I've duplicated FixedPMCArray.get_pmc_keyed_int() as a method, starting
with this line:
METHOD PMC* __getitem__(INTVAL key) {
Point of order: discussions would be a lot easier if they didnt *start
out* with perjorative terms like ugly hacks. One of my goals is to
eliminate the need for if (!Interp_flags_TEST(INTERP,
PARROT_PYTHON_MODE)) sprinkled throughout the various standard (i.e.
Perl) PMCs.
Yes, this all needs to be radically refactored - and all perlisms and
pythonisms removed from the standard classes.
Meanwhile, feel free to ask questions or make suggestions.
With that out of the way, I will assert that in all the Python code I
have seen, use of infix operators and square brackets for indexing
overwhelmingly dominate direct use of the methods. Furthermore, it is
clear that operators like sub_p_p_p and set_p_p_kic will always
perform better than callmethodcc_sc __sub__ and callmethodcc_sc
__getitem__ respectively.
Therefore, Pirate emits the opcodes whenever direct use is made of
things like infix operators and square brackets. And implemented
methods like __sub__ which, while rarely used, will operate correctly by
invoking to the opcodes.
I *don't* see a need to heavily optimize for rarely used mechanisms.
I encourage you to check out Pirate. The IMCC output of pirate.py is
now remarkably close to the output of pie-thon.pl.
- Sam Ruby
PDD 03 Issue: keyword arguments
Python provides the ability for any function to be called with either
positional or keyword [1] arguments. Here is a particularly brutal example:
args={'a':1,'b':2,'c':3}
def f(a,b,c): return (a,b,c)
def g(b,c,a): return (a,b,c)
for j in [f,g]: print j(1,2,3)
for j in [f,g]: print j(a=1,b=2,c=3)
for j in [f,g]: print j(*args.values())
for j in [f,g]: print j(**args)
I see nothing in pdd 03 that provides any guidance as to how to handle
this. What makes this issue so critical is that any solution will
potentially affect *every* function.
- Sam Ruby
P.S. Jython handles the above test correctly. Iron-Python handles all
but the **args properly.
[1] http://docs.python.org/ref/calls.html
Re: EcmaScript
Jeff Clites wrote: On Nov 27, 2004, at 5:58 PM, liorean wrote: On Sat, 27 Nov 2004 19:30:20 -0500, Sam Ruby [EMAIL PROTECTED] wrote: Overall, JavaScript would be a good match for Parrot. One place where it would significantly diverge at the moment is in the concept of a class. Objects in JavaScript are little more than bundles of properites, some of which may be functions. And classes are essentially templates for such objects. I don't really think it's that strange. Essentially, all objects contain a reference to their prototype. When getting a member of an object, the object will first check it's own members for the corresponding identifier, then ask it's prototype, and so on until the prototype chain is depleted. Setting is always done on the object itself. It's really not so much inheritance as it is conditional runtime delegation. Functions are of course first class and shouldn't differ from any other member - there is no native method/property distinction in JavaScript, even though host object may have such a distinction. This seems to me to be very much the way Python works as well, though the emphasis is different. (That is, the common case in Python is to define methods per-class rather than per-instance, and in JavaScript it's the opposite. But that's not a technological difference, just a cultural one.) I would think that the implementations would share a lot. I agree with both of you (and with Luke's observation on the existance of a find_method vtable entry, which I've used to good advantage already inside PyClass.pmc). What I am finding is that a very different approach is embedded inside object.c and needs to be factored out into what is currently (mis-)named ParrotObject and ParrotClass. I have ideas on how this should be handled, but my needs are not (yet) urgent, and Dan has expressed an interest in making the change, so I'm willing to wait for a little bit. - Sam Ruby
Re: deprecated transcendental ops with I arguments
Leopold Toetsch wrote: These opcodes are currently still in the ops files, with a function body DEPRECATED and a name prefix 'deprecated_' but will be removed soon. I'm actively following this list, I am confident that this and other change will be in the Perl 6 summaries, and I agree with this deprecation as well as introduction of the new push_eh opcode. All that being said, would it make sense to collect these type of changes to a single place in CVS or on the web? A simple CHANGES file in CVS, with a list of interfaces deprecated and removed since 0.1.1 would sufficient. - Sam Ruby
Re: Inconsistent opcode names
Leopold Toetsch wrote: There are likely some more inconsistencies, which should be fixed rather sooner then later. One that I noticed: =item Bgetattribute(out PMC, in PMC, in STR) =item Bgetprop(out PMC, in STR, in PMC) - Sam Ruby
Re: cvs commit: parrot/imcc/t/syn bsr.t
Leopold Toetsch wrote: cvsuser 04/11/19 08:28:06 Modified:imcc/t/imcpasm opt0.t optc.t pcc.t imcc/t/syn bsr.t Log: replace invoke P1 in imcc tests Revision ChangesPath 1.7 +2 -2 parrot/imcc/t/imcpasm/opt0.t [snip] - invoke P1/ + returncc/ Ever since this commit, I've been seeing the following: imcc/t/imcpasm/opt0.t2 512 62 33.33% 3 6 imcc/t/imcpasm/optc.t1 256 61 16.67% 1 Can this either be fixed or skipped? More details below: $ perl -Ilib -Iimcc imcc/t/imcpasm/opt0.t 1..6 ok 1 - add_n_i_n ok 2 - sub_n_ic_n not ok 3 - sub_n_i_n # Failed test (imcc/t/imcpasm/opt0.t at line 31) # '_test: # set N30, I0 # sub N0, N30, N1 # set N30, I0 # div N0, N30, N1 # null I0 # null I3 # invoke P1 # ' # doesn't match '/_test: # set N(\d+), I0 # sub N0, N\1, N1 # set N\1, I0 # div N0, N\1, N1 # null I0 # null I3 # returncc/ # ' ok 4 - added return - end ok 5 - added return - exit not ok 6 - added return - nil # Failed test (imcc/t/imcpasm/opt0.t at line 72) # got: '_test: # noop # null I0 # null I3 # invoke P1 # ' # expected: '_test: # noop # null I0 # null I3 # returncc # ' # Looks like you failed 2 tests of 6. $ perl -Ilib -Iimcc imcc/t/imcpasm/optc.t 21 | more 1..6 not ok 1 - in P param # Failed test (imcc/t/imcpasm/optc.t at line 11) # '_main: # set P17, P1 # new P30, 35 # .PerlUndef # set P30, 42 # set_p_pc P16, foo # @pcc_sub_call_0: # set P5, P30 # set I0, 1 # null I1 # null I2 # set I3, 1 # null I4 # set P0, P16 # invokecc # set P1, P17 # noop # end # foo: # set P30, P5 # print P30 # null I0 # null I3 # invoke P1 # ' # doesn't match '/_main: # set P\d+, P1 # new (P\d+), \d+ # \.PerlUndef # set \1, 42 # set_p_pc (P\d+), foo # @pcc_sub_call_\d: # set P5, \1 # set I0, 1 # null I1 # null I2 # set I3, 1 # null I4 # set P0, \2 # invokecc # set P1, P\d+ # noop # end # foo: # set (P\d+), P5 # print \3 # null I0 # null I3 # returncc/ # ' ok 2 # skip PCC changes ok 3 # skip PCC changes ok 4 # skip PCC changes ok 5 # skip PCC changes ok 6 # skip PCC changes # Looks like you failed 1 tests of 6.
Re: [PATCH] allow find-method to be inherited
Sam Ruby wrote: A patch is attached, but it bears a little discussion. Well, that didn't exactly work. I've since commmitted these patches, and more. A the moment, all the python and pirate unit tests pass. (Woot!) In the absense of other direction, I plan to write more tests and use them to drive the evolution of object.c and related sources. A description of my current thinking can be found at[1], but I am open to any implementation which continues to pass the tests that I define. In fact, the unit tests can change, as long as alternatives are provided that I can build upon to get the desired functionallity as defined by the pirate tests. Now that dynclasses build as a part of the 'all' target, I'm planning on adding the dynclass tests to the 'test' target. I've noticed that the test target is not successful unless the all target had been built previously, so this should not be a problem. If anybody has an issue with this then perhaps a 'testall' target can be defined. - Sam Ruby [1] http://www.intertwingly.net/blog/2004/11/14/Parrot-classes
[PATCH] allow find-method to be inherited
A patch is attached, but it bears a little discussion. Methods in pie-thon are stored as attributes. Methods in pirate are stored as properties. I'm not clear on what the difference is, and which should be used. In neither case are methods stored as methods, and this would prevent other languages which expect to invoke methods on Python objects from doing so - at least without this patch. This patch allows properties to be transparently accessed as methods. I'm also of the belief that getprop and/or getattribute should also be inheritable. My initial testing indicates that getprop is not currently. Before providing a patch, I'd like some reassurance that this assumption is correct. I have a set of classes and changes I'd like to see get into the Parrot cvs sooner or later. In fact, I'd prefer if it were sooner as maintaining what effectively is a separate branch is getting tiresome. You can see all the changes at [1] or [2]. It is mostly new files, but for the files that have changed, both the full source and current diffs are provided. With these changes, and with the corresponding changes to Pirate [3], all non-skipped Parrot and Pirate tests will pass, with the exception of the getprop test mentioned above. I've also posted more thoughts and background at [4]. - Sam Ruby [1] http://intertwingly.net/stories/2004/11/09/parrot/ [2] http://intertwingly.net/stories/2004/11/09/parrot.tgz [3] http://intertwingly.net/stories/2004/11/09/pirate/ [4] http://www.intertwingly.net/blog/2004/11/09/Python-vs-Parrot Index: src/objects.c === RCS file: /cvs/public/parrot/src/objects.c,v retrieving revision 1.123 diff -u -u -r1.123 objects.c --- src/objects.c 3 Nov 2004 14:29:58 - 1.123 +++ src/objects.c 9 Nov 2004 15:41:07 - @@ -1212,7 +1212,6 @@ STRING *isa; UINTVAL start; INTVAL pos; -find_in_pmc: class_name = class-vtable-whoami; method = Parrot_find_global(interpreter, @@ -1287,7 +1286,7 @@ class = curclass; if (class-vtable-base_type == enum_class_delegate) break; -goto find_in_pmc; +return VTABLE_find_method(interpreter, curclass, method_name); } method = Parrot_find_global(interpreter, VTABLE_get_string(interpreter,
Re: Basic operations for MMD
Dan Sugalski wrote: Sam made reference to the CLS last week, and it's a point well-taken. We need a list of standard activities that parrot provides so that people writing compilers can use the provided facilities and people implementing data types can make sure they do the right things to work with the standards. Before I go rattling this off and starting a PDD that'll likely languish a bit, does anyone want to step up and volunteer to put together a skeleton PDD with the current facilities, at which point we can start moving forward to fill in any holes and getting things detailed enough that it can be reasonably used? To start it'd include the vtable and MMD functions, the ops that call 'em, and the methods we look for by default. My goal at the moment is to get up to speed to the point where I *could* write such a document. I'm actively working on Pirate, have been in concact with the authors of Pint, and also plan to look into a Parrot binding to libxml2 (which would explain my interest in utf-8). Note: while I was in no way the author of the CLA, I was the conveener (EMCA-speak for chair) of the working group which produced it, so I am familiar with both its content and how it was produced. With that in mind, it is my feeling that such a document is premature. Unless somebody is much further along than I am aware of, we don't know where all the pitfalls are just yet; let alone knowing which of the pitfalls we can pave over, and which we simply need to post a big bold lighthouse at. If you can wait a few weeks, I'm willing to take a stab at it, but just not yet. If somebody else takes the lead, I will certainly contribute. - Sam Ruby
Re: No Cpow op with PMC arguments?
Jeff Clites wrote:
a) As Sam says, in Python y**z is just shorthand for
y.__pow__(z)--they will compile down to exactly the same thing
(required for Python to behave correctly). Since __pow__ isn't
special, we don't need anything to support it that we wouldn't need
for any other arbitrary method, say y.elbowCanOpener(z).
[snip]
So I don't think an op really gives us what we want. (Specifically, we
could define a pow_p_p_p op, but then Python wouldn't use it for the
case Sam brought up.) I'd apply the same argument to many of the other
p_p_p ops that we have--they don't gives us what we need at the HLL
level (though they may still be necessary for other uses).
It is my intent that Python *would* use this method. What is important
isn't that y**z actually call y.__pow__(z), but that the two have the
same effect.
Let's take a related example: y+z. I could make sure that each are
PMCs, and then find the __add__ attribute, retrieve it, and then use it
as a basis for a subroutine call, as the semantics of Python would seem
to require. Or I could simply emit the add op.
How do I make these the same? I have a common base class for all python
objects which defines an __add__ method thus:
METHOD PMC* __add__(PMC *value) {
PMC * ret = pmc_new(INTERP, dynclass_PyObject);
mmd_dispatch_v_ppp(INTERP, SELF, value, ret, MMD_ADD);
return ret;
}
... so, people who invoke the __add__ method explicitly get the same
function done, albeit at a marginally higher cost.
Now to complete this, what I plan to do is to also implement all
MMD/vtable operations at the PyObject level and have them call the
corresponding Python methods. This will enable user level __add__
methods to be called.
Prior to calling the method, there will need to be a check to ensure
that the method to be called was, in fact, overridden. If not, a
type_error exception will be thrown.
- Sam Ruby
