Re: A tentative list of vtable functions
Ken Fox [EMAIL PROTECTED] wrote: use complex; my $c = 2__3; # 2 + 3i That's really gross. 2 + 3i is just add(integer(2), complex(0,3)) with compile-time constant folding eliminating the add(). I would even go so far as to say that 3i is just syntactic sugar for multiply(integer(3),sqrt(-1)) with constant folding doing the simplification. The only rules we need are the standard ones we must have for constant folding and *one* additional macro that says bareword "i" used after a scalar should be re-written as "* sqrt(-1)". Yes I agree that it was a quite gross suggestion! However, this was just a throw-away comment embedded in part of a more general discussion about how to handle literals in user-defined numeric types. Now of course if we have the luxury of deciding that core perl 'knows' about complex numbers, then of the parser can be made to recognise a pretty syntax like 3i; however, I was discussing how to handle the more general case of someone adding a 3rd party scalar numeric type (be it bigint, bigrat, complex or whatever), not known to the core perl parser, and how to sensibly handle numeric literals in that case. It seemed to me that there were two main options. Firstly, the parser could attempt it's own conversions, which would (IMHO) be a *bad* thing: use bigreal; $x = 1e99; might cause '1e99' to be initally converted to a double, and thus $x probably gets assigned the value Infinity. On the other hand, the tokeniser/lexer/parser/optimiser or whatever could be made *not* responsible for literal converations, but instead just extracts the string of legal numerical constant characters, and passes it it to some newliteral() class method associated with the current numeric type in lexical scope (as specificed by "use x;"), then it would be up to that numeric type to make what it will of that type, including possible bodged sematics like 2__3. ie the perl code above gets evaluated as assign(x, bigreal-newliteral("1e99")) or thereabouts. In summary: Perl shouldn't do interpetation of numeric literals, but should instead delegate it to the numeric class currently in scope. A similar agument might apply to string literals, meaning that people can add their own UFTX/UNICODE-XX string types, and still handle literals sensibly. An extension of this idea would be for numeric (and string) classes to provide their own tokeniser routine for literals - assuming appropriate hooks are available in the main tokeniser. Then within the scope of 'use complex', the definition of a numeric literal could be locally extended to allow a trailing 'i' or whatever, without the Perl authors ever needing to know about it. On the other hand, some people might think that this way lies madness Dave. * Dave Mitchell, Operations Manager, * Fretwell-Downing Facilities Ltd, UK. [EMAIL PROTECTED] * Tel: +44 114 281 6113.The usual disclaimers * * Standards (n). Battle insignia or tribal totems
Re: A tentative list of vtable functions
"ye, wei" wrote: One C++ problem I just found out is memory management. It seems that it's impossible to 'new' an object from an specified memory block. So it's impossible to put free'd objects in memory pool and re-allocate them next time. Stuff like that isn't the problem with using C++. (In fact a class can provide its own allocator. You can even provide a global allocator if you like to live dangerously.) The trouble is that the object model for C++ isn't exactly the object model (I mean at the internals level like "SV") for Perl. That means we still have to do a lot of work to get Perl to work right (and even more work to defeat some C++ compiler assumptions to get Perl to work fast). Another big problem with C++ is lack of internal documentation and object code standards. Some of Perl's dynamic module loading capability would be complex using C++ -- and possibly impossible with code built by different compilers. I think the general idea is that the advantages of C++ don't move us far enough out of our comfortable local minimum to make it worthwhile. - Ken
Re: A tentative list of vtable functions
At 01:34 PM 10/17/00 -0400, Ken Fox wrote: I think the general idea is that the advantages of C++ don't move us far enough out of our comfortable local minimum to make it worthwhile. Yup, that pretty much covers it. C++ also has an awful lot of stuff in it that, while interesting, is too likely to be misused, or be really, really unfamiliar to too many people. (The perl guts are going to present enough problems for folks without adding all of C++'s potential idiosyncracies to the mix) Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
Further to my earlier ramblings and worries about binary operators and overloading etc Here is a proposal for the numerical part of the SV API that provides a framework for arbitrary precision arithmetic, while still allowing standard ints and floats to be handled efficiently. Some of the ideas here could be borrowed to handle the string part of the API, especially regarding unicode and its various representations. It is also based on some of my early muddled thinking on how operator overloading should in general be handled at this level and may provide futher discussion on that topic. If nothing else, since this document shows examples both for the body of a binary op, and for the code that calls it, it may add some focus to discussions of the meaningĀ of the functions in the API, even if it is only to show how it shouldnt be done! The summary of this mini API looks like: p = sv-precision() i = sv-get_int(sv) sv-get_intn(sv,buf,n) r = sv-get_real(sv) sv-get_realn(sv,buf,plen,elen) sv-set_int(sv,i) sv-set_intn(sv,buf,n) sv-set_real(sv,r) sv-set_realn(sv,buf,plen,elen) --or possibly-- sv = vtable-newSViv(7) sv = vtable-newSVnv(7.0) [ There is also a need for sv1-overwrite(sv1,sv2) or similar, which is needed for assignment ] Some guiding principles: 1) binary operators should in general always return an SV rather rather than just an int or char* say. Otherwise ops won't have any way of propagating the 'specialness' of their operands (eg the sum of two complex numbers should also be complex). 2) in general we want the result of a binop to be of the same type as the 'biggest' of its operands, eg $real + $intis a real $int + $realis a real $bigint + $int is a bigint $real + $complexis a complex $complex + $bigcomplex is a bigcomplex [the last example above wont calculate a sensible result since the API in this proposal only provides limited support for multi-dimensional scalars such as complex numbers. In fact $complex + $bigcomplex is probably equivalent to $bigcomplex + modulus($complex) ] 3) an operation on 2 numerical SVs of the same type should be efficient 4) an operation on 2 numerical SVs of different types should in general work, but need not in general be efficient. How this all works 1) Precision The 'bigness' of an operand is determined by a constant integer property per SV class, which for lack of a better name, I call precision. sv-precision() returns the precision associated with a particular class (and all variables of the same class have the same precision value). We need some formula to calculate precision per class. One approach would be the following: Let d be the dimension of the variable (eg complex numbers would be 2) Let t be: 0 for integer 1 for floating point let b be the bits of storage used (eg 32 for an INT32, 64 for a double) then define the precision by (d24 + t23 + b) or similar. Classes which allow arbirtary dimension or precision should set d or t to its maximum value. This means that complex numbers always win out over plain numbers, floats are better than ints, and if they are otherwise the same, the bits of storage used wins out. With this, the operator to subtract 2 elements on the stack might look something like this: pp_subtract { SV *sv1 = POP; SV *sv2 = POP; SV *result; if ( // if they're the same type (common case), // avoid the overhead of calling precision() twice sv1-vtable == sv2-vtable || sv1-precision() = sv2-precison() ) result = sv1-subtract(sv1,sv2,0); else result = sv2-subtract(sv2,sv1,1); // operands swapped PUSH(result); } Note that the action of this code is to call the subtact function associated with the 'biggest' arg. The implementation of numeric ops for a particular class is based on the idea that if both ops are of the same class, the op is carried out efficiently by directly accessing the internal representations of both ops; otherwise the 2nd op is asked to extract its value in a standard but not necessarily efficient form that is portable between all classes. With this in mind, a scalar class must provide the following methods: i = sv-get_int(sv) The scalar returns its value as a 'standard integer' (eg whatever integer type is used when 'use integer' is in effect). If its internal value is too big, an exception is thrown. This function is typically used in places like array subscripts, the mode value in chmod $mode, etc etc. sv-get_intn(sv,buf,n) here, the sv is passed a buf of size N bytes, and is asked to fill it with an n*8 bit integer represenation of its internal value - again thowing an exception if it wont fit. In practice this will just be a case of filling the top part of the buffer with a 0 or 1 sign-extension. In principle the API could have a
Re: A tentative list of vtable functions
Here is a proposal for the numerical part of the SV API that provides a framework for arbitrary precision arithmetic, while still allowing standard ints and floats to be handled efficiently. [some quick very high-level comments] Don't forget bigrats. 1) binary operators should in general always return an SV rather rather than just an int or char* say. Hear ye, hear ye. 2) in general we want the result of a binop to be of the same type as the 'biggest' of its operands, eg $real + $int is a real $int + $real is a real $bigint + $intis a bigint $real + $complex is a complex $complex + $bigcomplexis a bigcomplex Well, kinda. But sometimes it is perfectly fine or even desirable to 'collapse' back to the 'smaller' kind: when computing with bigints and bigrats. In other words, I would like 87238382456823758823554867875243 - 87238382456823758823554867875242 to be int (an IV) 1, not a bigint 1. -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
Re: A tentative list of vtable functions
Jarkko Hietaniemi [EMAIL PROTECTED] wrote: [some quick very high-level comments] Don't forget bigrats. I'm not too familiar with the concept of rational numbers in a computing complex. What's your definition of a (big)rat? Fixed point?? 2) in general we want the result of a binop to be of the same type as the 'biggest' of its operands, eg $real + $intis a real $int + $realis a real $bigint + $int is a bigint $real + $complexis a complex $complex + $bigcomplex is a bigcomplex Well, kinda. But sometimes it is perfectly fine or even desirable to 'collapse' back to the 'smaller' kind: when computing with bigints and bigrats. In other words, I would like 87238382456823758823554867875243 - 87238382456823758823554867875242 to be int (an IV) 1, not a bigint 1. Perhaps 2) should have been better phrased as 2) in general a binop should cause the op method associated with the 'biggest' of its operands to be called. Having said that, should an operator ever return an SV of something other than its own type - especially when it has no context to know what might be useful to the caller? IE should bigint_subtract(bigint1,bigint2) 1) always return a new bigint SV 2) a bigint normally, but an int if it happens to know that its result will fit in a standard int, or 3) a bigint or an int depending on some as yet undefined external context? My own feeling is that it should just stick with (1) - if someone has some code that uses bigints, the chances are that the results of bigint expressions are most likely to fed into further bigint expressions, so demoting to int then promoting again would probably less efficient; also if you're working with bigints in the first place, then I'd expect cases where the result of an op fits in an IV would be the minority. But never having worked with bigints myself, I could be speaking from my derierre ;-) I guess ops should be allowed to do (2) if they want; I just personally think its a bit messy. * Dave Mitchell, Operations Manager, * Fretwell-Downing Facilities Ltd, UK. [EMAIL PROTECTED] * Tel: +44 114 281 6113.The usual disclaimers * * Standards (n). Battle insignia or tribal totems
Re: A tentative list of vtable functions
On Mon, Oct 02, 2000 at 12:47:18PM +0100, David Mitchell wrote: Jarkko Hietaniemi [EMAIL PROTECTED] wrote: [some quick very high-level comments] Don't forget bigrats. I'm not too familiar with the concept of rational numbers in a computing complex. What's your definition of a (big)rat? Fixed point?? bigint1 / bigint2. Possibly represented as a triad of bigints, bigint1 + bigint2 / bigint3. Remember also abnormalities like NaN. IE should bigint_subtract(bigint1,bigint2) 1) always return a new bigint SV 2) a bigint normally, but an int if it happens to know that its result will fit in a standard int, or 3) a bigint or an int depending on some as yet undefined external context? My own feeling is that it should just stick with (1) - if someone has some code that uses bigints, the chances are that the results of bigint expressions are most likely to fed into further bigint expressions, so demoting to int then promoting again would probably less efficient; also if you're working with bigints in the first place, then I'd expect cases where the result of an op fits in an IV would be the minority. But never having worked with bigints myself, I could be speaking from my derierre ;-) Likewise. But having such a reduction/collapsion code is very natural because we need to have the 'inverse' of that logic to know when to stop using ints and start using bigints; 2**1000. -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
Re: A tentative list of vtable functions
Don't forget bigrats. I'm not too familiar with the concept of rational numbers in a computing complex. What's your definition of a (big)rat? Fixed point?? bigint1 / bigint2. Possibly represented as a triad of bigints, bigint1 + bigint2 / bigint3. I'm tempted to suggest that bigrats, like complex numbers, fall into the category of 'only mostly compatible'. ie 1 ($bigrat1 op $bigrat2) accurately gives $bigrat3 2 ($bigrat1 op [any numeric scalar that can be accurately extracted using get_realn()]) accurately gives $bigrat3 3 ($bigrat1 op $daves_different_but_smaller_bigrat_implementation2) gives a bigrat, but its accuracry depends on (2) Remember also abnormalities like NaN. I think this depends largely on how we standardise the arbitrary real represenation as retrieved by get_realn - if our format allows NaN et al, then get_realn() on a Nan should get a NaN, otherwise an exception should be thrown. Of course, get_intn() on a NaN should always throw an exception. IE should bigint_subtract(bigint1,bigint2) 1) always return a new bigint SV 2) a bigint normally, but an int if it happens to know that its result will fit in a standard int, or 3) a bigint or an int depending on some as yet undefined external context? My own feeling is that it should just stick with (1) - if someone has some code that uses bigints, the chances are that the results of bigint expressions are most likely to fed into further bigint expressions, so demoting to int then promoting again would probably less efficient; also if you're working with bigints in the first place, then I'd expect cases where the result of an op fits in an IV would be the minority. But never having worked with bigints myself, I could be speaking from my derierre ;-) Likewise. But having such a reduction/collapsion code is very natural because we need to have the 'inverse' of that logic to know when to stop using ints and start using bigints; Hmmm, 2**1000. raises some interesting issues. Cureently the perl language itself has no builtin support for big numbers, and all I have been proposing so far is an scheme for the numerical part of the vtable API that in principle allows other people to write their own large types which (mostly) interoperate with standard perl numeric types. Also, the current Perl language has no general mechanism for telling an op what type it should return, and I've sort of come to the conclusion that it would hard/messy to do so. Perl currently has the syntax my Bigint $b = ...; but that is a compiler hint that $b is a reference to an object of type Bigint - not that $b is a scalar with vtable type bigint. I had vaguely assumed that if someone wrote a Bigint scalar type, they would also provide a perl-level constructor. So, $x = 2^1000; would evaluate 2^1000 at compile time, and if it didnt fit into an NV (or IV if 'use integer' is in effect), create a compile-time error. The compiler has no way of knowing that you want a compile-type bigint constant. To do proper bigint arithmetic, you might do use Bigint; $x = bigint(2)^1000; where bigint is a function imported from Bigint that returns a bigint scalar (as opposed to a Bigint object ref). This code would be evaluated as follows: a standard SV with value 1000 is pushed on the stack. a standard SV with value 2 is pushed on the stack bigint is called, and returns a new SV of type bigint, with value 2. This is pushed on the stack. pp_exp is called. It examines its 2 args; then as defined by precison(), the bigint is 'bigger' than the std scalar, so bigint_power() is called. This evaluates 2^1000 and returns the result to pp_exp as a new bigint SV. pp_exp then pushes this value on the stack. pp_assign is then called, whihc blows away the current contents of $x, and relaces them with the return value from pp_exp. Without language extensions, perl XS functions are essentially the only way to create values of a user-defined type. As long as this remains the case, it might be wise for ops not to 'degrade' their results. For example, $b1 = bigint(99); $b2 = bigint(00); $b3 = $b2 - $b1; # happens to be 99, but could just as easily be 10^100 $b4 = $b3 ^ 10; In this case, $b3 *sometimes* gets downgraded a std int, which means that *sometimes* the last line will die. On the other hand, if ops never downgrade, there needs to be a manual way of converting a bigint to a standard one for the odd occasion when the programmer needs it. We could argue that it up to the author of Bigint to provide a perl-level function that does this, eg big2int(), whihc might be used a context like this: if ($b = 0 and $b 2^32) { freeze(big2int($b); } else { freeze($b); } (Of course a lot of the time, extracting the integer value from a bigint would be automatic, eg eg $array[$b4] - the get_int method of $b4 would
Re: A tentative list of vtable functions
For the record: I hate the current policy of defaulting to NVs for arithmetic ops. If I say '2' I do mean an IV of 2, not an NV of 2.000. Currently if I say $a = 2; $b = 3; $c = $a + $3; the $c will be an NV of of 5.000, or thereabouts, een while $a and $b are IVs. I think one should stay within one type/class of numbers for as long as possible. Assuming that the perl parser generated IV SVs rather than NVs for the 2 constants 2,3, then my scheme would handle this fine; the IV version of add() would be called, and an IV SB would result. However, my current scheme crashes and burns for the following: $a = 2; $b = 33; $c = $a ** $b; The way perl5 handles these cases is that $a + $b, $a * $b etc just silengly ignore any overlow and return an int, while $a ** $b appears to automatically upgrade to a float. My system currently says nothing about upgrades, although it would be faily simple to specifiy that certain standard int ops should automatically pass their args to the std real type if theres any change of overflow. Similarly for literals: if I say $x = 10715086071862673209484250490600018105614048117055336074437503883703510511249361 22493198378815695858127594672917553146825187145285692314043598457757469857480393 45677748242309854210746050623711418779541821530464749835819412673987675591655439 46077062914571196477686542167660429831652624386837205668069376; I would prefer getting a bigint, not an NV. I am happy with requiring a use bigint; for this to happen, though. I presume that with bigint in force, even $x = 2 would make $x a bigint? One way to implenment this is for there to be a vtable entry for each class called say from_string(), which would return an SV of its own type from a numeric literal. [ NB - I'm not clear whether conversion from a string like '2' to a constant SV value is performed - during parsing or execution. If necessary, replace the word 'parse' as appropriate below...] The parsing process would have a concept of 'who is responsible for parsing numeric literals in the current block' - ie a current pointer to someone's vtable. On startup this would point to vtable_standard_nv; 'use integer' would change it to vtable_standard_iv; 'use bigint' would change it to vtable_bigint etc. Then when a numeric literal needs converting to an SV, the current from_string() method is called.
Re: A tentative list of vtable functions
Assuming that the perl parser generated IV SVs rather than NVs for the 2 constants 2,3, then my scheme would handle this fine; the IV It currently does so. version of add() would be called, and an IV SB would result. "The IV version of add()"? Beware of combinatorial explosion: addII, addIU, addUI, addUU, addIN, addNI, addNN, addblahbah However, my current scheme crashes and burns for the following: $a = 2; $b = 33; $c = $a ** $b; Would work just fine in my 64-bit Alpha box :-) The way perl5 handles these cases is that $a + $b, $a * $b etc just silengly ignore any overlow and return an int, while $a ** $b appears to automatically upgrade to a float. My system currently says nothing Yup. Welcome to the wonderful world of unsolicited and surprising type conversions. I presume that with bigint in force, even $x = 2 would make $x a bigint? Not necessarily. The tokenizer is able to tell the difference It currently does this by using UVs if a positive IV wouldn't be enough, and NVs if even a UV wouldn't be. 'use bigint' could be a hint saying "don't go NV unless absolutely unavoidable", such as "sin(2)". For division such as 2/3, NV could still be be default unless a "use bigrat" is in effect. -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
Re: A tentative list of vtable functions
would need to know a fair bit about particular user-defined types that have been loaded in, on order to make clever interpretations of literals. Precisely. Assume I want $a = 2 + 3i; to work... -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
Re: A tentative list of vtable functions
would need to know a fair bit about particular user-defined types that have been loaded in, on order to make clever interpretations of literals. Precisely. Assume I want $a = 2 + 3i; to work... Which I what I suggest we abandon attempts to make the parser do intellignet decisons on numeric liternal, and instead just grab all the characters that appear to make up thye string constant, and pass the whole lot to the from_string() method of the current defualt numeric type (as defined by use integer et al) to deal with. NB - there's no reason why the from_string method of complex number class couldnt have a private convention that 2 _'s separate out real and imaginary parts, eg use complex; my $c = 2__3; # 2 + 3i although this wouldnt work if both compnents were floats. So, I strongly suggest that when use xtype is in force, all numeric literals are created as type xtype, to avoid madness!
Re: A tentative list of vtable functions
Dan Sugalski wrote: At 02:29 PM 9/29/00 +0100, David Mitchell wrote: Regarding the tentative list of vtable functions: I'm rather worried about binary operators, eg 'is_equal', 'add' etc. The danger with these is that they may impose a single implementation of scalars upon us. As an example, suppose I wrote an alternative scalar implementation that was optimised for many small string appends followed by a single get of the wholpe string at the end. (I might internally do this with a linked list of blocks, say; then allocate a single large block at the end to return the whole string.) If sv1 is a standard SV and sv2 is an append-optimised one, then (sv1-vtable[IS_EQUAL])(sv1,sv2) would crash and burn, because sv1's 'is_equal' function doent know that sv2 has a different internal representation. Your worries are well-founded, but ultimately not a problem. (I do like those kinds... :) Since the generic internal representation of a variable is hidden from anything but that variable's vtable, the scenario you propose can't happen. What'd happen instead for a plain ASCII string compare would look something like: bool is_equal(pmc1, pmc2) { if (strcmp(pmc1-string, to_string(ASCII, pmc2)) { return Perl_FALSE; } else { return Perl_TRUE; } } Overloading does present an ordering issue, and one I am worried about. It either requires all the vtable functions to know about all the variable types (which is impossible, unfortunately) or it means perl needs to dynamically reorder the operation so the 'bigger' variable's function gets called. Reordering's also not a 100% panacea--it's fine for ops involving a base type and an extended type, but it can potentially break down with two extended types. (What happens when you add a complex number with a complex bigint, if the two types have no idea about each other?) I have been wondering why not write Perl in C++ from ground? Performance? Portability? In C, it takes alot of time to design vtable ,however still couldn't work completely as OO. If write in C++, the vtable will be generated during the compile phrase and perfectly. Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk -- Sincerely, Ye, Wei
Re: A tentative list of vtable functions
Nathan Torkington [EMAIL PROTECTED] writes: Dan Sugalski writes: It's possible, for example, for a tied/overloaded/really-darned-strange variable to look true but still be false. If you do: $foo = $bar || $baz; and both $bar and $baz are objects, the 'naive' way is to make $foo be $bar. But it's distinctly possible that $bar really should be treated as a false value and $baz be used instead. Why? Dunno. Serious hand-waving here. (And yes, I know that's a danger sign... :) But I don't see any reason to preclude the possibility. You can do that right now in perl5, by using overload.pm and supplying a 'bool' method. In practice both Damian and I have been bitten by inability to overload || and - you can indeed pick which side is kept but you cannot make it keep both. So "defered" action is not possible. I can make $a + $b return bless ['+',$a,$b],'OperatorNode' but you cannot get $a $b to produce bless ['',$a,$b],'OperatorNode' whatever you do. -- Nick Ing-Simmons
Re: A tentative list of vtable functions
Nick Ing-Simmons wrote: Ken Fox [EMAIL PROTECTED] writes: Dan Sugalski wrote: For something like: @foo = @bar || @baz; I have no problem with the call sequence looking like (pseudo-codish here): set_context(ARRAY, ASSIGN); foo-store(bar-log_or(bar, baz)); But log_or must short circuit -- And what above suggests it does not? It is up to bar's log_or not to evaluate baz if bar is considered true. Dan suggested that log_or didn't have to short circuit. I guess I was also reading in how code like @foo = @bar || some_big_hairy_function(); would work. Does log_or get a lazy second argument? Are lazy arguments going to be fast enough to implement conditionals? I think we have to preserve that behavior for all types or the (hypothetical future) optimizer might break things. It might translate "if" statements into ||, or vice versa. It might do dead code elimination. It might do liveness analysis. It already does and it is a pain when you are trying to give meaning to and || for overloaded objects. I happend to have a 'need' for || / which "short circuit later" i.e. I absolutely agree that || and need to be over-loadable -- even for changing the always-short-circuit behavior to DWIM behavior. My only point is that we shouldn't do this in the vtable. The vtable IMHO needs consistent and well defined semantics so that other parts of the system can safely make assumptions about what transformations can be legally done to the code. I'd like to see over-loading || done with a macro system. Maybe something like: BEGIN { syntax $expr : $expr || $expr { my $t = $1; (has_method($t, 'op_logical_or')) ? $t-op_logical_or($2) : ($t) ? $t : $2 } } (That should be read as a tree substitution macro with $1 and $2 bound to the parse trees for the operands of the || operator.) There could be several macros like this defined by "use overload". IMHO syntax changes (like creating non-short circuiting logicals) Semantic not syntax. Maybe you're right. IMHO it's one of those fuzzy boundaries between the syntax and semantics. An over-loaded || operator is a regular method call. The non-over-loaded || operator is a special form of an "if" statement. - Ken
Re: A tentative list of vtable functions
Dan Sugalski writes: It's possible, for example, for a tied/overloaded/really-darned-strange variable to look true but still be false. If you do: $foo = $bar || $baz; and both $bar and $baz are objects, the 'naive' way is to make $foo be $bar. But it's distinctly possible that $bar really should be treated as a false value and $baz be used instead. Why? Dunno. Serious hand-waving here. (And yes, I know that's a danger sign... :) But I don't see any reason to preclude the possibility. You can do that right now in perl5, by using overload.pm and supplying a 'bool' method. An application might be when your object represents an array and you want the boolean context of your object to report whether it has data in it or not, to facilitate: if ($pdl) { # do something if the $pdl has data in it } Nat
Re: A tentative list of vtable functions
At 03:56 PM 9/13/00 -0400, Ken Fox wrote: Nick Ing-Simmons wrote: Ken Fox [EMAIL PROTECTED] writes: Dan Sugalski wrote: For something like: @foo = @bar || @baz; I have no problem with the call sequence looking like (pseudo-codish here): set_context(ARRAY, ASSIGN); foo-store(bar-log_or(bar, baz)); But log_or must short circuit -- And what above suggests it does not? It is up to bar's log_or not to evaluate baz if bar is considered true. Dan suggested that log_or didn't have to short circuit. I guess I was also reading in how code like @foo = @bar || some_big_hairy_function(); would work. Does log_or get a lazy second argument? Are lazy arguments going to be fast enough to implement conditionals? The second argument would have to be taken as a list, and I hadn't much considered that case. I've been thinking that lists should be considered perl primitives under the hood, and this is another good argument for that. I think we have to preserve that behavior for all types or the (hypothetical future) optimizer might break things. It might translate "if" statements into ||, or vice versa. It might do dead code elimination. It might do liveness analysis. It already does and it is a pain when you are trying to give meaning to and || for overloaded objects. I happend to have a 'need' for || / which "short circuit later" i.e. I absolutely agree that || and need to be over-loadable -- even for changing the always-short-circuit behavior to DWIM behavior. My only point is that we shouldn't do this in the vtable. The vtable IMHO needs consistent and well defined semantics so that other parts of the system can safely make assumptions about what transformations can be legally done to the code. Speed, consistency, and ease of overloading are the reasons I'd like it in the vtable, though I may well be completely missing the mark here. (Not that it'd be the first time... :) I'd like to see over-loading || done with a macro system. Maybe something like: BEGIN { syntax $expr : $expr || $expr { my $t = $1; (has_method($t, 'op_logical_or')) ? $t-op_logical_or($2) : ($t) ? $t : $2 } } (That should be read as a tree substitution macro with $1 and $2 bound to the parse trees for the operands of the || operator.) Yech. (And I say that with deep conviction... :) Though it may be a reasonable way to do it. IMHO syntax changes (like creating non-short circuiting logicals) Semantic not syntax. Maybe you're right. IMHO it's one of those fuzzy boundaries between the syntax and semantics. An over-loaded || operator is a regular method call. The non-over-loaded || operator is a special form of an "if" statement. I'd rather they be treated the same, otherwise it means more work at runtime to know which way to handle it. (Since we can't reliably tell at compile time whether something's overloaded or not) Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
At 07:58 PM 9/9/00 +, Nick Ing-Simmons wrote: Ken Fox [EMAIL PROTECTED] writes: Short circuiting should not be customizable by each type for example. We are already having that argument^Wdiscussion elsewhere ;-) But I agree variable vtables are not the place for that. As do I, up to a point. For something like: @foo = @bar || @baz; I have no problem with the call sequence looking like (pseudo-codish here): set_context(ARRAY, ASSIGN); foo-store(bar-log_or(bar, baz)); and having the log_or function do whatever it deems appropriate with its arguments, up to and including a stringified version of Larry's kids names in kanji... Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
Ken Fox [EMAIL PROTECTED] writes: Short circuiting should not be customizable by each type for example. We are already having that argument^Wdiscussion elsewhere ;-) But I agree variable vtables are not the place for that. -- Nick Ing-Simmons
Re: A tentative list of vtable functions
At 10:46 PM 8/31/00 +, David L. Nicol wrote: Dan Sugalski wrote: Okay, here's a list of functions I think should go into variable vtables. All the math functions are in here. Can the entries that my type does not use be replaced with other functions that my type does use? That's what overloading does. Functions marked with a * will take an optional type offset so we can handle asking for various permutations of the basic type. These aren't going to be that huge then, with each one taking *void() thismuch[30] or so; why don't we skip the "optional offset" and make subclasses keep their whole own copy? Will 240 bytes per type blow out a modern cache? Nope. When I said optional offset, I meant that the version of perl you were using might or might not have the capability to do this, depending on compile-time options. clarify (where does this type go to resolve uncached method names?) or is that better kept in a global clarifier that keeps it's own mapping. You're confusing perl-level objects with these low-level internals things. This stuff isn't really visible except to people writing new variants on base perl types, so you won't see it unless you're writing overload code. I think the list is too long, for the base type. Could the base type be something that just knows how to return its type name, in order to build types that do not have defined STRING methods? We're shooting for speed here. Any common operation that could be affected by the type of the variable should be represented so a custom function can be called that does exactly what needs to be done. Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
"DS" == Dan Sugalski [EMAIL PROTECTED] writes: DS Okay, here's a list of functions I think should go into variable vtables. DS Functions marked with a * will take an optional type offset so we can DS handle asking for various permutations of the basic type. DS type DS name What are this used for? DS get_bool Is this allowed to return a non-true/false result? Or is everything true or false? DS get_string * DS get_int * DS get_float * What does the optional type argument do? What about collection types? Why not simply collapse these into a single one with an option argument? What should a get_* do if inappropriate for its type? DS get_value What does this do that the above three do not? DS set_string * DS set_int * DS set_float * DS set_value DS add * DS subtract * DS multiply * DS divide * DS modulus * Where is the argument to be added/subtracted/etc. ? On the stack? DS clone (returns a new copy of the thing in question) Isn't this a deep problem? (Should be near the top of the vtbl DS new (creates a new thing) Why does the thing have to do a new? (move earlier) DS concatenate DS is_equal (true if this thing is equal to the parameter thing) DS is_same (True if this thing is the same thing as the parameter thing) How does THIS figure out how to get THAT to give a usable value? DS logical_or DS logical_and DS logical_not DS bind (For =~) DS repeat (For x) Are these so that operators can be overriden? DS Anyone got anything to add before I throw together the base vtable RFC? Are you going to fully specify the expected input and results in the RFC? chaim -- Chaim FrenkelNonlinear Knowledge, Inc. [EMAIL PROTECTED] +1-718-236-0183
Re: A tentative list of vtable functions
DS get_bool Is this allowed to return a non-true/false result? Or is everything true or false? Dunno yet. I'm thinking just a true/false value, but... A tri-state bool would be really cool (i.e, true/false/undef). Although I understand that this probably isn't where Perl 6 is going, most embedded systems these days have I/O that can exist in three states (1, 0, or Hi-z). Grant M.
Re: A tentative list of vtable functions
Dan Sugalski wrote: We're shooting for speed here. Any common operation that could be affected by the type of the variable should be represented so a custom function can be called that does exactly what needs to be done. Dan so if I want to make up a type that is strictly a 16-bit integer, I overload everything except the math operations with pointers to errors? That's the direction I'm going (off by myself), to merging C in, and that's why a even more limited base type appeals to me. But of course, with a more limited base type, every call to plus would have to check to see if plus was there before resolving, instead of just hopping over the edge and trusting the top end of the rope to be tied.
Re: A tentative list of vtable functions
Dan Sugalski writes: : Type returns a magic cookie value of some sort (Not sure what sort yet), : name returns a string with the name of the type of the variable. Why can't the type object just stringify to the name of the type? From a language level, I'm inclined to say that any bare identifier that is known to be a type name should be compiled to a type object that stringifies to the name of its type. Then class methods don't have to do an extra symbol table lookup. Larry
Re: A tentative list of vtable functions
Dan Sugalski [EMAIL PROTECTED] writes: is_equal (true if this thing is equal to the parameter thing) is_same (True if this thing is the same thing as the parameter thing) is_equal in what sense? (String, Number, ...) and how is is_same different from just comparing addresses of the things? -- Nick Ing-Simmons
Re: A tentative list of vtable functions
"NI" == Nick Ing-Simmons [EMAIL PROTECTED] writes: NI Dan Sugalski [EMAIL PROTECTED] writes: is_equal (true if this thing is equal to the parameter thing) is_same (True if this thing is the same thing as the parameter thing) NI is_equal in what sense? (String, Number, ...) NI and how is is_same different from just comparing addresses of the things? Proxies? Wrappers? The proxy might want to answer on behalf of the proxied. chaim -- Chaim FrenkelNonlinear Knowledge, Inc. [EMAIL PROTECTED] +1-718-236-0183
Re: A tentative list of vtable functions
At 06:07 PM 9/1/00 +, Nick Ing-Simmons wrote: Dan Sugalski [EMAIL PROTECTED] writes: is_equal (true if this thing is equal to the parameter thing) is_same (True if this thing is the same thing as the parameter thing) is_equal in what sense? (String, Number, ...) I was thinking if you did: $a = $b; then the equal method for $a would return true if passed $b to compare with, regardless of the type. Equality is a rather nebulous thing without context, though. Perhaps this should get a context passed in as a parameter. and how is is_same different from just comparing addresses of the things? We might have aliased variables or something, where two variable pointers which point to two different variable structures are actually the same thing. Threads do this too in the current implementation. (And in the pthreads interface, though neither are a great argument for it...) Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
At 10:23 AM 9/1/00 -0700, Larry Wall wrote: Dan Sugalski writes: : Type returns a magic cookie value of some sort (Not sure what sort yet), : name returns a string with the name of the type of the variable. Why can't the type object just stringify to the name of the type? I'd figured that the type would be an integer, rather than an object, and one that wouldn't necessarily map to just one 'real' variable type. A tied integer, for example, would return an integer type though the name would likely be the package the variable was tied into. From a language level, I'm inclined to say that any bare identifier that is known to be a type name should be compiled to a type object that stringifies to the name of its type. Then class methods don't have to do an extra symbol table lookup. None of this should really be visible from a language level except indirectly. (Through the overload mechanism, perhaps) The name function for most variables would likely just return a constant string and, while there's some speed hit from that instead of just storing it in the variable somewhere, it's not likely to be called often enough to make a difference. For real perl language-level method calls, I'd figure we'd cache a pointer to the package stash in the variable, store the hash value of the method name (if it's a bareword method call) in the opcode and, if the call's on a typed variable, a pointer to the CV for the method. (Ignored if the real variable's not of the type declared in the code) Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
At 11:49 AM 9/1/00 -0500, David L. Nicol wrote: Dan Sugalski wrote: We're shooting for speed here. Any common operation that could be affected by the type of the variable should be represented so a custom function can be called that does exactly what needs to be done. Dan so if I want to make up a type that is strictly a 16-bit integer, I overload everything except the math operations with pointers to errors? That's the direction I'm going (off by myself), to merging C in, and that's why a even more limited base type appeals to me. No, you provide functions that do the right thing. If you have a 16-bit integer scalar, you should still return a floating point value or string representation, or a 32-bit (or native sized, at least) integer if asked. But of course, with a more limited base type, every call to plus would have to check to see if plus was there before resolving, instead of just hopping over the edge and trusting the top end of the rope to be tied. No, we hop over the edge. It's not unreasonable for perl to expect internal consistency. Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
On Fri, 01 Sep 2000, Dan Sugalski wrote: I'm not sure. They're there mainly for guaranteed unfiltered access to the variable's guts, and I'm not sure what things will need that. I use direct, raw access in prototyping when dipping back into C for "heavy" data manipulations - bit stream analysis, signals processing, etc. Since I'm usually doing a lot of buffer manipulations, I don't want all the scalar overhead that comes into play. Of course, one could argue that that's a completely different subject - we're talking about calls within perl itself. Extraction of the raw data could just as well happen underneath without an interface of any kind. But I'd like to bring a lot of that work back up into perl proper, which, unfortunately, often results in the code running two to three hundred times slower than native C. (It's only prototyping, keep in mind, but I'd like run-times to at least be the same day. :-) I've been trying (off and on) for a couple of months to hack an interface to a lightweight buffer from within Perl, to at least allow some of the cruder constructs without *too* much overlap. In the end, though, it still comes down to pointers, arrays of pointers, or pointer offsets. All of which were still scalars, so it didn't buy too much. Furthermore, what is "guaranteed unfiltered access", and how can it be used? For a Perl 5 string, for instance, would it be the entire allocated buffer, or the space from the beginning pointer to the end of the string? Are you going to be able to manipulate this raw data in the raw, or are you still going to have to go through perl to work on it? My reason for asking is that the more of the perl core that relies on specific representations of data, the more complexity there is in porting to other architectures. Fair enough. It may turn out to be unnecessary, in which case we'll toss it, or if not we'll just make sure to force some reasonable requirements on it so porting's easier. I think, through line disciplines or string tags or what-have-you, that some data should be allowed to be tagged as raw - perhaps to prevent any automatic promotion, for instance - but actually needing that raw data underneath is rather incestuous. If something relies on that data so much, it knows where to find it. -- Bryan C. Warnock ([EMAIL PROTECTED])
Re: A tentative list of vtable functions
At 04:43 PM 8/31/00 -0400, Dan Sugalski wrote: Okay, here's a list of functions I think should go into variable vtables. Functions marked with a * will take an optional type offset so we can handle asking for various permutations of the basic type. Perhaps I'm missing something... Is this for scalars alone? I see no arrays/hashes here. type name get_bool get_string * get_int * get_float * get_value set_string * set_int * set_float * set_value add * subtract * multiply * divide * modulus * clone (returns a new copy of the thing in question) new (creates a new thing) concatenate is_equal (true if this thing is equal to the parameter thing) is_same (True if this thing is the same thing as the parameter thing) logical_or logical_and logical_not bind (For =~) repeat (For x) Anyone got anything to add before I throw together the base vtable RFC? Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
get_int * get_float * Could you elaborate on these a lot? What's an 'int'? What's a 'float'? Having lately been battling a lot with quad ints and doubles vs long doubles I seriously want this interface not to suck. I was a tad concerned there, too. I'm hoping one can painlessly transition to "infinite" precision for those two at will. --tom
Re: A tentative list of vtable functions
Dan Sugalski wrote: get_value set_value Wouldn't these go on the SV and not on the inner type? Maybe I'm thinking value when you're saying variable? The following seem useful on variables too: before_get_value after_get_value before_set_value after_set_value There ought to be specializations of get_value and set_value that call these hooks if they're defined -- no sense in making the normal case slow. We also need GC functions: allocation_size -- size of required (or actual) allocation children -- nested SV *s (for live object traversals) move -- move object to new memory location resize_granted -- object's resize request was granted finalize -- release private resources before destruction The clone function should take a depth parameter so that lazy deep copies can be type specific. Is is_same an identify function? Or does it check the type? Either way it seems like that could be implemented on the SV and not on every type. What purpose do the logical_* functions serve? Shouldn't there just be one is_true function that the logical_* ops call? Persistence functions would be really useful too: freeze thaw I'm not too worried about getting the vtbl right at the first because it will be pretty obvious how it should go once the code starts to form. - Ken
Re: A tentative list of vtable functions
Wouldn't these go on the SV and not on the inner type? Maybe I'm thinking value when you're saying variable? The following seem useful on variables too: before_get_value after_get_value before_set_value after_set_value There ought to be specializations of get_value and set_value that call these hooks if they're defined -- no sense in making the I hope *lists* of hooks. I'm not too worried about getting the vtbl right at the first because it will be pretty obvious how it should go once the code starts to form. Some planning isn't that painful :-) -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
Re: A tentative list of vtable functions
Jarkko Hietaniemi writes: I'm not too worried about getting the vtbl right at the first because it will be pretty obvious how it should go once the code starts to form. Some planning isn't that painful :-) Yes. Especially given that vtables are an unbenchmarked change. It'd be good to see someone prototype the vtables and then benchmark the code against similarly fleshed out SV structures. We need to know just how much of a hit this is going to be in terms of memory and in terms of speed. Everyone's saying "this is the way to go", but I'm a little afraid that nobody has said why. Nat
Re: A tentative list of vtable functions
At 04:59 PM 8/31/00 -0400, Buddha Buck wrote: At 04:43 PM 8/31/00 -0400, Dan Sugalski wrote: Okay, here's a list of functions I think should go into variable vtables. Functions marked with a * will take an optional type offset so we can handle asking for various permutations of the basic type. Perhaps I'm missing something... Is this for scalars alone? I see no arrays/hashes here. Gah. It's supposed to be for all types. I'll dredge together the bits for arrays and hashes and wedge them in too. Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
At 03:12 PM 8/31/00 -0600, Tom Christiansen wrote: get_int * get_float * Could you elaborate on these a lot? What's an 'int'? What's a 'float'? Having lately been battling a lot with quad ints and doubles vs long doubles I seriously want this interface not to suck. I was a tad concerned there, too. I'm hoping one can painlessly transition to "infinite" precision for those two at will. They're shorthand for integer and floating point (or fixed point, I suppose--something with a decimal, at least), not any particular type of int or float. Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
At 05:30 PM 8/31/00 -0400, Ken Fox wrote: Dan Sugalski wrote: get_value set_value Wouldn't these go on the SV and not on the inner type? Maybe I'm thinking value when you're saying variable? Nope. The get/set value functions are for when something knows what the SV (or whatever we call it) really is and can handle the raw data. For example, if my code knew a SV held a complex number (which doesn't map well to the int/float/char translation) and I could handle it, then I'd call get_value and get the raw data. The following seem useful on variables too: before_get_value after_get_value before_set_value after_set_value There ought to be specializations of get_value and set_value that call these hooks if they're defined -- no sense in making the normal case slow. You could override the vtable function in that case, I think. We also need GC functions: allocation_size -- size of required (or actual) allocation I can see that. children -- nested SV *s (for live object traversals) I'm not sure. Maybe. move -- move object to new memory location Don't think this is needed. The base SV structures won't move, I think, and we should be able to move the variable's contents (hanging off the sv_any analog) at will. resize_granted -- object's resize request was granted I don't think this is needed--the guts of the set functions would handle this as needed. finalize -- release private resources before destruction Ah, yep. Destroy. Forgot that one. The clone function should take a depth parameter so that lazy deep copies can be type specific. Hadn't gotten that far, but it's a good idea. Is is_same an identify function? Or does it check the type? Identity. If two SVs pointed to the identical same data, this'd return true. Either way it seems like that could be implemented on the SV and not on every type. What purpose do the logical_* functions serve? Shouldn't there just be one is_true function that the logical_* ops call? In case one wants to overload @foo || @bar to do a piecewise logical or of the two arrays. Persistence functions would be really useful too: freeze thaw Cool, makes sense. Might be the wrong place for it, but it can't hurt to think about it and remove it later if need be. Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
At 03:45 PM 8/31/00 -0600, Nathan Torkington wrote: Jarkko Hietaniemi writes: I'm not too worried about getting the vtbl right at the first because it will be pretty obvious how it should go once the code starts to form. Some planning isn't that painful :-) Yes. Especially given that vtables are an unbenchmarked change. It'd be good to see someone prototype the vtables and then benchmark the code against similarly fleshed out SV structures. We need to know just how much of a hit this is going to be in terms of memory and in terms of speed. Everyone's saying "this is the way to go", but I'm a little afraid that nobody has said why. That's one of the reasons I'm doing this--I'd like to start in on the implementation of various scalar, array, and hash types to see how the performance stands up. I figure that having a general shape of the target is good before we start throwing code together. Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: A tentative list of vtable functions
Dan Sugalski wrote: Okay, here's a list of functions I think should go into variable vtables. All the math functions are in here. Can the entries that my type does not use be replaced with other functions that my type does use? Functions marked with a * will take an optional type offset so we can handle asking for various permutations of the basic type. These aren't going to be that huge then, with each one taking *void() thismuch[30] or so; why don't we skip the "optional offset" and make subclasses keep their whole own copy? Will 240 bytes per type blow out a modern cache? type name get_bool get_string * get_int * get_float * get_value set_string * set_int * set_float * set_value add * subtract * multiply * divide * modulus * clone (returns a new copy of the thing in question) new (creates a new thing) concatenate is_equal (true if this thing is equal to the parameter thing) is_same (True if this thing is the same thing as the parameter thing) logical_or logical_and logical_not bind (For =~) repeat (For x) Anyone got anything to add before I throw together the base vtable RFC? clarify (where does this type go to resolve uncached method names?) or is that better kept in a global clarifier that keeps it's own mapping. I think the list is too long, for the base type. Could the base type be something that just knows how to return its type name, in order to build types that do not have defined STRING methods? Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk -- David Nicol 816.235.1187 [EMAIL PROTECTED] Kansas City Perl Mongers will meet Sept. 20th at 7:00 in Westport Flea Market Bar Grill http://tipjar.com/kcpm
Re: A tentative list of vtable functions
How about to_string * from_string * as generalizations of formatted/pretty input/output and freeze/thaw (cf printf/Data::Dumper/Storable)? -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
Re: A tentative list of vtable functions
Dan Sugalski wrote: get_value set_value The get/set value functions are for when something knows what the SV (or whatever we call it) really is and can handle the raw data. For example, if my code knew a SV held a complex number (which doesn't map well to the int/float/char translation) and I could handle it, then I'd call get_value and get the raw data. Are you envisioning that a lot of the perl core will use these functions, or are they for special things like: my complex_number $ii; ? My reason for asking is that the more of the perl core that relies on specific representations of data, the more complexity there is in porting to other architectures. -- Bradley M. Kuhn - http://www.ebb.org/bkuhn PGP signature