Re: optimization of rule-based model on discrete variables

2021-06-16 Thread Greg Ewing
On 16/06/21 10:51 pm, Elena wrote: sorry I wrote it wrongly, my bad, I will use f just to predict yi from new coming Xi. Then what do you do with the new yi? -- Greg -- https://mail.python.org/mailman/listinfo/python-list

Re: optimization of rule-based model on discrete variables

2021-06-16 Thread Elena via Python-list
Il Wed, 16 Jun 2021 11:37:42 +1200, Greg Ewing ha scritto: > On 15/06/21 10:07 pm, Elena wrote: >> After the optimization, I will use f just to predict new Xi. > > So you're going to use f backwards? > > I don't see how that will work. Where are you going to f

Re: optimization of rule-based model on discrete variables

2021-06-15 Thread Greg Ewing
On 15/06/21 10:07 pm, Elena wrote: After the optimization, I will use f just to predict new Xi. So you're going to use f backwards? I don't see how that will work. Where are you going to find a new yi to feed into the inverse of f? I think I don't understand what role g plays

Re: optimization of rule-based model on discrete variables

2021-06-15 Thread Elena via Python-list
Il Tue, 15 Jun 2021 01:53:09 +, Martin Di Paola ha scritto: > From what I'm understanding it is an "optimization problem" like the > ones that you find in "linear programming". > > But in your case the variables are not Real (they are Integers) and

Re: optimization of rule-based model on discrete variables

2021-06-15 Thread Elena via Python-list
n+1] you can use f to predict how well it will work. > But that will just give you a y[n+1], and it's not clear what to do with > that. Do you append it to Y and feed an n+1 component vector into g? > > I think I still need more information about the underlying problem > befo

Re: optimization of rule-based model on discrete variables

2021-06-14 Thread Martin Di Paola
From what I'm understanding it is an "optimization problem" like the ones that you find in "linear programming". But in your case the variables are not Real (they are Integers) and the function to minimize g() is not linear. You could try/explore CVXPY (https://www

Re: optimization of rule-based model on discrete variables

2021-06-14 Thread Greg Ewing
On 15/06/21 12:51 am, Elena wrote: I see what you mean, so I try to explain it better: Y is a vector say [y1, y2, ... yn], with large (n>>10), where yi = f(Xi) with Xi = [x1i, x2i, ... x10i] 1<=i<=n. All yi and xji assume discrete values. I already have a dataset of X={Xi} and would like to find

Re: optimization of rule-based model on discrete variables

2021-06-14 Thread Richard Damon
On 6/13/21 12:15 PM, Elena via Python-list wrote: > Hi, I have, say 10 variables (x1 ... x10) which can assume discrete finite > values, for instance [0,1 or 2]. > I need to build a set of rules, such as: > > 1) if x1==0 and x2==1 and x10==2 then y = 1 > 2) if x2==1 and x3==1 and x4==2 and x6==0 t

Re: optimization of rule-based model on discrete variables

2021-06-14 Thread Elena via Python-list
Il Mon, 14 Jun 2021 19:39:17 +1200, Greg Ewing ha scritto: > On 14/06/21 4:15 am, Elena wrote: >> Given a dataset of X={(x1... x10)} I can calculate Y=f(X) where f is >> this rule-based function. >> >> I know an operator g that can calculate a real value from Y: e = g(Y) >> g is too complex to be

optimization of rule-based model on discrete variables

2021-06-14 Thread Elena via Python-list
Hi, I have, say 10 variables (x1 ... x10) which can assume discrete finite values, for instance [0,1 or 2]. I need to build a set of rules, such as: 1) if x1==0 and x2==1 and x10==2 then y = 1 2) if x2==1 and x3==1 and x4==2 and x6==0 then y = 0 3) if x2==0 and x3==1 then y = 2 4) if x6==0 and x7

Re: optimization of rule-based model on discrete variables

2021-06-14 Thread Greg Ewing
On 14/06/21 4:15 am, Elena wrote: Given a dataset of X={(x1... x10)} I can calculate Y=f(X) where f is this rule-based function. I know an operator g that can calculate a real value from Y: e = g(Y) g is too complex to be written analytically. I would like to find a set of rules f able to minim

profile guided optimization of loadable python modules?

2018-07-04 Thread Neal Becker
Has anyone tried to optimize shared libraries (for loadable python modules) using gcc with profile guided optimization? Is it possible? Thanks, Neal -- https://mail.python.org/mailman/listinfo/python-list

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-18 Thread Gregory Ewing
Antoon Pardon wrote: It seems a bit strange that with the immense value that is given to stack frames, that these wouldn't be available somehow. There was discussion recently about the idea of providing access to the frame of a suspended generator, so that debuggers can inspect the stack of an

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Terry Reedy
On 7/17/2015 7:43 AM, Antoon Pardon wrote: On 07/17/2015 01:05 PM, Chris Angelico wrote: def gen(): yield stuff yield more stuff for stuff in gen(): bomb with exception The error didn't happen in the generator, so I wouldn't expect to see it in the traceback. Yes something l

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Chris Angelico
On Fri, Jul 17, 2015 at 10:54 PM, Antoon Pardon wrote: > On 07/17/2015 01:49 PM, Chris Angelico wrote: >> On Fri, Jul 17, 2015 at 9:43 PM, Antoon Pardon >> wrote: >> >> >>> Sure, but in this case, the generator is still active. The Runtime >>> would be able to jump to and somehow activates it's s

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Antoon Pardon
On 07/17/2015 01:49 PM, Chris Angelico wrote: > On Fri, Jul 17, 2015 at 9:43 PM, Antoon Pardon > wrote: > > >> Sure, but in this case, the generator is still active. The Runtime >> would be able to jump to and somehow activates it's stack record >> for the next value. So why would we expect it to

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Chris Angelico
On Fri, Jul 17, 2015 at 9:43 PM, Antoon Pardon wrote: > On 07/17/2015 01:05 PM, Chris Angelico wrote: >> On Fri, Jul 17, 2015 at 8:48 PM, Antoon Pardon >> wrote: >>> Just wondering, are traceback records of generators available? They are >>> if an exception is raised in the generator itself, but

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Antoon Pardon
On 07/17/2015 01:05 PM, Chris Angelico wrote: > On Fri, Jul 17, 2015 at 8:48 PM, Antoon Pardon > wrote: >> Just wondering, are traceback records of generators available? They are >> if an exception is raised in the generator itself, but what if an exception >> is raised in the loop that is driven

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Chris Angelico
On Fri, Jul 17, 2015 at 8:48 PM, Antoon Pardon wrote: > Just wondering, are traceback records of generators available? They are > if an exception is raised in the generator itself, but what if an exception > is raised in the loop that is driven by a generator. They don't appear in > the standard s

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Antoon Pardon
On 07/16/2015 06:43 PM, Chris Angelico wrote: > On Fri, Jul 17, 2015 at 12:32 AM, Antoon Pardon > wrote: > >> What is unclear about "as it is generally produced on stderr"? That you >> can do a whole lot of stuff, doesn't mean that this whole lot of stuff is >> part of what generally happens. When

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Rustom Mody
On Wednesday, July 15, 2015 at 4:54:51 PM UTC+5:30, Marko Rauhamaa wrote: > Ned Batchelder: > > > On Wednesday, July 15, 2015 at 6:56:10 AM UTC-4, Marko Rauhamaa wrote: > >> Ned Batchelder : > >> > I don't understand this, can you explain more? Are you saying that the > >> > Python specification s

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Gregory Ewing
Joonas Liik wrote: https://docs.python.org/2/library/sys.html#sys.setrecursionlimit so as per the docs the programmer has no real control over how much stack his program can have. all you can say is "let me ignore the safeguard a little longer, i hope i wont crash the program" that is not the sam

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Gregory Ewing
Antoon Pardon wrote: On 07/16/2015 12:43 AM, Gregory Ewing wrote: Whatever value you choose for N, keeping only the first/last N traceback frames will lead to someone tearing their hair out. I would say, that someone should get over himself. Traceback are not the only or even the most useful

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Terry Reedy
On 7/16/2015 7:45 AM, Chris Angelico wrote: On Thu, Jul 16, 2015 at 5:31 PM, Antoon Pardon wrote: Traceback are not the only or even the most useful tool for debugging code. The current stack trace doesn't even contain the value's of the variables on the stack. So in case of Terry Reedy's ex

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Ethan Furman
On 07/16/2015 12:14 PM, Joonas Liik wrote: You are giving the programmer a choice between "run out of stack and crash" and "mutilate interpreter internals and crash or zero out the hard drive" this is not a real choice.. +1 -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Joonas Liik
On 16 July 2015 at 21:58, Steven D'Aprano wrote: > On Fri, 17 Jul 2015 03:34 am, Joonas Liik wrote: > >> Now i admit that it is possible to have infinite recursion but it is >> also possiblew to have infinite loops. and we don't kill your code >> after 1000 iterations of a while loop so why should

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Steven D'Aprano
On Fri, 17 Jul 2015 03:34 am, Joonas Liik wrote: > Now i admit that it is possible to have infinite recursion but it is > also possiblew to have infinite loops. and we don't kill your code > after 1000 iterations of a while loop so why should we treat recursion > any differently? Because a while

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Fri, Jul 17, 2015 at 4:23 AM, Joonas Liik wrote: > On 16 July 2015 at 20:49, Chris Angelico wrote: >> >> This sounds like a denial-of-service attack. If you can state that no >> reasonable document will ever have more than 100 levels of nesting, >> then you can equally state that cutting the p

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Joonas Liik
On 16 July 2015 at 20:49, Chris Angelico wrote: > > This sounds like a denial-of-service attack. If you can state that no > reasonable document will ever have more than 100 levels of nesting, > then you can equally state that cutting the parser off with a tidy > exception if it exceeds 100 levels

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Fri, Jul 17, 2015 at 3:34 AM, Joonas Liik wrote: > That all sounds reasonable. However that can be looked another way. > Soppose you have some code that traverses some tree, a strange > imbalanced tree (say from some xml) > > It is, semantically at least, a reasonable aproach to process such a

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Joonas Liik
On 16 July 2015 at 20:03, Chris Angelico wrote: > > The trouble with that is that it can quickly run you out memory when > you accidentally trigger infinite recursion. A classic example is a > simple wrapper function... > > def print(msg): > print(ctime()+" "+msg) > > With the recursion limit

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Ethan Furman
On 07/16/2015 09:43 AM, Chris Angelico wrote: True. That said, though, it's not a justification for dropping stack frames; even in the form that's printed to stderr, there is immense value in them. It may be possible to explicitly drop frames that a programmer believes won't be useful, but a gen

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Fri, Jul 17, 2015 at 2:50 AM, Joonas Liik wrote: > Wouldn't it be possible to have like a dynamically > sized stack so that you can grow it endlessly > with some acceptable overhead.. > > That would pretty much take care of the stack-overflow > argument without many painful side effects on > th

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Joonas Liik
Wouldn't it be possible to have like a dynamically sized stack so that you can grow it endlessly with some acceptable overhead.. That would pretty much take care of the stack-overflow argument without many painful side effects on the semantics at least.. -- https://mail.python.org/mailman/listinf

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Fri, Jul 17, 2015 at 12:32 AM, Antoon Pardon wrote: > On 07/16/2015 04:00 PM, Chris Angelico wrote: >> On Thu, Jul 16, 2015 at 11:56 PM, Antoon Pardon >> wrote: >>> Fine, I should have been more clear. >>> >>> The stack trace as it is generally produced on stderr after an uncought >>> exceptio

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Antoon Pardon
On 07/16/2015 04:00 PM, Chris Angelico wrote: > On Thu, Jul 16, 2015 at 11:56 PM, Antoon Pardon > wrote: >> Fine, I should have been more clear. >> >> The stack trace as it is generally produced on stderr after an uncought >> exception, doesn't contain the values of the variables on the stack. > S

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
on stderr after an uncought > exception, doesn't contain the values of the variables on the stack. Sure. So you catch it at top level and grab whatever info you need. In some frameworks, this is already done for you - an uncaught exception from application code drops you into a debugger that

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Antoon Pardon
On 07/16/2015 01:45 PM, Chris Angelico wrote: > On Thu, Jul 16, 2015 at 5:31 PM, Antoon Pardon > wrote: >> >> I would say, that someone should get over himself. >> Traceback are not the only or even the most useful >> tool for debugging code. The current stack trace >> doesn't even contain the val

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Thu, Jul 16, 2015 at 5:31 PM, Antoon Pardon wrote: > On 07/16/2015 12:43 AM, Gregory Ewing wrote: > >> Antoon Pardon wrote: >>> But it doesn't need to be all or nothing. How about the following >>> possibility. >>> When the runtime detects a serie of tail calls, it will keep the bottom >>> th

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Antoon Pardon
On 07/16/2015 12:43 AM, Gregory Ewing wrote: > Antoon Pardon wrote: >> But it doesn't need to be all or nothing. How about the following >> possibility. >> When the runtime detects a serie of tail calls, it will keep the bottom three >> and the top three backtrace records of the serie. > Whatever

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Mark Lawrence
On 15/07/2015 23:34, Gregory Ewing wrote: Mark Lawrence wrote: IIRC the realms of the C setjmp and longjmp. Not really the same thing. A longjmp chops the stack back, whereas a tail call avoids putting something on the stack to begin with. Thanks for that :) -- My fellow Pythonistas, ask n

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Gregory Ewing
Antoon Pardon wrote: But it doesn't need to be all or nothing. How about the following possibility. When the runtime detects a serie of tail calls, it will keep the bottom three and the top three backtrace records of the serie. Whatever value you choose for N, keeping only the first/last N trac

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Gregory Ewing
Mark Lawrence wrote: IIRC the realms of the C setjmp and longjmp. Not really the same thing. A longjmp chops the stack back, whereas a tail call avoids putting something on the stack to begin with. -- Greg -- https://mail.python.org/mailman/listinfo/python-list

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Gregory Ewing
Chris Angelico wrote: Which really says that TCO is impossible if you have any sort of clean-up or deallocation to be done after the call begins. The only way to truly turn your tail call into a GOTO is to do all your cleanup first. Indeed. In compilers that implement TCO, there's quite a lot m

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Marko Rauhamaa
Ian Kelly : > On Tue, Jul 14, 2015 at 11:27 AM, Marko Rauhamaa wrote: >> You'll see that the generated code is tail-call-optimized. > > This can't be done generally, though. What if, instead of a local > variable, the assignment were to a dict item? Even if the dict itself > is a local variable,

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Ian Kelly
(const char *str, int n) > { > if (str && *str) > n = atoi(str + 1, n * 10 + *str - '0'); > return n; > } > > (Example modified from http://stackoverflow.com/questions/3412

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Chris Angelico
On Wed, Jul 15, 2015 at 7:13 PM, Gregory Ewing wrote: > Chris Angelico wrote: >> >> I'm still interested in the explicit "replace current stack frame with >> this call" operation. Calling it "goto" seems wrong, as most languages >> with goto restrict it to _within_ a function, > > > This just sugg

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Chris Angelico
On Wed, Jul 15, 2015 at 10:00 PM, MRAB wrote: > On 2015-07-15 12:22, Mark Lawrence wrote: >> >> On 15/07/2015 10:13, Gregory Ewing wrote: >>> >>> Chris Angelico wrote: I'm still interested in the explicit "replace current stack frame with this call" operation. Calling it "goto" seem

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread MRAB
On 2015-07-15 12:22, Mark Lawrence wrote: On 15/07/2015 10:13, Gregory Ewing wrote: Chris Angelico wrote: I'm still interested in the explicit "replace current stack frame with this call" operation. Calling it "goto" seems wrong, as most languages with goto restrict it to _within_ a function,

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Marko Rauhamaa
Ned Batchelder : > On Wednesday, July 15, 2015 at 6:56:10 AM UTC-4, Marko Rauhamaa wrote: >> Ned Batchelder : >> > I don't understand this, can you explain more? Are you saying that the >> > Python specification shouldn't specify what x becomes?: >> > >> > def who_knows(): >> > pass >>

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Mark Lawrence
On 15/07/2015 10:13, Gregory Ewing wrote: Chris Angelico wrote: I'm still interested in the explicit "replace current stack frame with this call" operation. Calling it "goto" seems wrong, as most languages with goto restrict it to _within_ a function, This just suggests to me is that most lang

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Ned Batchelder
On Wednesday, July 15, 2015 at 6:56:10 AM UTC-4, Marko Rauhamaa wrote: > Ned Batchelder : > > > On Wednesday, July 15, 2015 at 2:44:55 AM UTC-4, Marko Rauhamaa wrote: > >> The other problem for tail call elimination is the requirement that > >> functions return None by default. Smooth tail call el

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Antoon Pardon
On 07/15/2015 12:55 PM, Marko Rauhamaa wrote: > Ned Batchelder : > >> On Wednesday, July 15, 2015 at 2:44:55 AM UTC-4, Marko Rauhamaa wrote: >>> The other problem for tail call elimination is the requirement that >>> functions return None by default. Smooth tail call elimination would >>> require t

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Marko Rauhamaa
Ned Batchelder : > On Wednesday, July 15, 2015 at 2:44:55 AM UTC-4, Marko Rauhamaa wrote: >> The other problem for tail call elimination is the requirement that >> functions return None by default. Smooth tail call elimination would >> require that Python leave the default return value unspecified

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Ned Batchelder
On Wednesday, July 15, 2015 at 2:44:55 AM UTC-4, Marko Rauhamaa wrote: > Gregory Ewing : > > > Marko Rauhamaa wrote: > > >> It might even be tail-call optimized by Python. Only you can't count > >> on it because the language spec doesn't guarantee it. > > > > The language spec might permit it, bu

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Marko Rauhamaa
Gregory Ewing : > A tail call *is* a goto. That's how you implement one in assembly > language -- you write a jump instruction instead of a call > instruction. The jump doesn't have to be to the same function. In functional programming languages you might not even have a call stack. Instead, you

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Gregory Ewing
Chris Angelico wrote: I'm still interested in the explicit "replace current stack frame with this call" operation. Calling it "goto" seems wrong, as most languages with goto restrict it to _within_ a function, This just suggests to me is that most language designers are not very imaginative. :-

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Antoon Pardon
On 07/15/2015 02:41 AM, Terry Reedy wrote: > On 7/14/2015 10:02 AM, Antoon Pardon wrote: >> On 07/14/2015 03:43 PM, Chris Angelico wrote: >>> On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa >>> wrote: > >>>> No, tail call optimization doesn't c

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Antoon Pardon
On 07/14/2015 07:48 PM, Steven D'Aprano wrote: > On Wed, 15 Jul 2015 03:29 am, Marko Rauhamaa wrote: > >> Chris Angelico : >> >>> On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa >>> wrote: >>>> No, tail call optimization doesn't cha

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Gregory Ewing : > Marko Rauhamaa wrote: >> It might even be tail-call optimized by Python. Only you can't count >> on it because the language spec doesn't guarantee it. > > The language spec might permit it, but the BDFL has explicitly > expressed a dislike for the idea of implicit tail call remo

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Gregory Ewing
Marko Rauhamaa wrote: It might even be tail-call optimized by Python. Only you can't count on it because the language spec doesn't guarantee it. The language spec might permit it, but the BDFL has explicitly expressed a dislike for the idea of implicit tail call removal, so it's unlikely to ev

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Wed, Jul 15, 2015 at 11:01 AM, Terry Reedy wrote: > On 7/14/2015 1:52 PM, Chris Angelico wrote: >> >> On Wed, Jul 15, 2015 at 3:43 AM, Marko Rauhamaa wrote: >>> >>> I don't like the way integer overflows are explicitly undefined in >>> modern C. >>> >>> Similarly, I don't like the way tail cal

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Terry Reedy
I think I'm beginning to understand. Tail call elimination is necessary to cope with a system of programming that has deliberately excluded certain other options (eg mutables, in pure functional languages), Tail recursion is the functional way to write a while loop. To put is another way, a

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Terry Reedy
On 7/14/2015 1:52 PM, Chris Angelico wrote: On Wed, Jul 15, 2015 at 3:43 AM, Marko Rauhamaa wrote: I don't like the way integer overflows are explicitly undefined in modern C. Similarly, I don't like the way tail call behavior is undefined in Python. Where in the Python spec is it stated tha

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Terry Reedy
On 7/14/2015 10:02 AM, Antoon Pardon wrote: On 07/14/2015 03:43 PM, Chris Angelico wrote: On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: No, tail call optimization doesn't change the behavior of the program, for the worse anyway. It does, because you lose traceback records. T

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Terry Reedy
On 7/14/2015 1:15 AM, Paul Rubin wrote: def quicksort(array, start, end): midp = partition(array, start, end) if midp <= (start+end)//2: quicksort(array, start, midp) quicksort(array, midp+1, end) else: quicksort(array, midp+1, end) quicks

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Mark Lawrence
has a tco decorator. A direct link https://github.com/lihaoyi/macropy#tail-call-optimization for anybody who is interested. And I've just stumbled across this https://github.com/baruchel/tco which was only put up two days ago. -- My fellow Pythonistas, ask not what our language can do for yo

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Mark Lawrence
github.com/lihaoyi/macropy#tail-call-optimization for anybody who is interested. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > Tail call elimination is necessary to cope with a system of > programming that has deliberately excluded certain other options (eg > mutables, in pure functional languages), Tail call elimination is necessary for the functional programming style. While Scheme strongly promotes

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Wed, Jul 15, 2015 at 4:37 AM, Marko Rauhamaa wrote: > Steven D'Aprano : > >> Tail call behaviour is not undefined in Python. There is a strong >> convention (followed by all implementations I know of) to follow the >> reference implementation's (CPython) behaviour, which is not to >> optimize t

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Steven D'Aprano : > Tail call behaviour is not undefined in Python. There is a strong > convention (followed by all implementations I know of) to follow the > reference implementation's (CPython) behaviour, which is not to > optimize tail calls. But even if an implementation chooses to not > follo

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Steven D'Aprano
On Wed, 15 Jul 2015 03:43 am, Marko Rauhamaa wrote: > The problem there is in the C standard, not the compiler that complies > with the standard. > > I don't like the way integer overflows are explicitly undefined in > modern C. > > Similarly, I don't like the way tail call behavior is undefined

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Paul Rubin
Steven D'Aprano writes: > Here's an example where an overzealous but standards-conforming optimization > introduced a security bug: > http://code.google.com/p/nativeclient/issues/detail?id=245 In that particular example, the refactored code simply looks wrong. -- https://mail

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Wed, Jul 15, 2015 at 3:43 AM, Marko Rauhamaa wrote: > I don't like the way integer overflows are explicitly undefined in > modern C. > > Similarly, I don't like the way tail call behavior is undefined in > Python. Where in the Python spec is it stated that tail call behaviour is undefined? The

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Steven D'Aprano
On Wed, 15 Jul 2015 03:29 am, Marko Rauhamaa wrote: > Chris Angelico : > >> On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa >> wrote: >>> No, tail call optimization doesn't change the behavior of the >>> program, for the worse anyway. >> >>

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Steven D'Aprano : > C, in my opinion, is the poster child for how a programming language > should NOT be designed, and I don't want Python to go down the same > path. John Regehr writes: > > "... there are compilers (like GCC) where integer overflow behaved a > certain way for many years and then

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Steven D'Aprano
s that this would put responsibility on the programmer to decide whether to TCO or not. Well, potential downside. Using a TCO decorator or switch to enable it also puts responsibility on the programmer, but that's usually considered a good thing. >> This is what led to the confusion respon

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > You're proposing making _every_ instance of "return func(...)" into > this kind of thing. Yes. > That's not always recursion, Correct. > and it's certainly not always clear what called what to get you there. Correct. Marko -- https://mail.python.org/mailman/listinfo/pyth

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: >> No, tail call optimization doesn't change the behavior of the >> program, for the worse anyway. > > It does, because you lose traceback records. That's pretty significant > when

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
n = atoi(str + 1, n * 10 + *str - '0'); return n; } (Example modified from http://stackoverflow.com/questions/34125/wh ich-if-any-c-compilers-do-tail-recursion-optimization#220660>.) You'll see that the generated code is tail-call-optimized. Marko -- https://mail.python.org/mailman/listinfo/python-list

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
gt;>>> I would rather optimize by default and disable optimizations with >>>>> --debug or equivalent. >>>> That assumes that it's simply an optimization. This is a distinct >>>> semantic change. >>> No, tail call optimization doesn

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Grant Edwards
On 2015-07-14, Ian Kelly wrote: > And yet, Python somehow manages to gain new features with each release. > > The reason why most proposals get rejected is because most proposals > are bad. If every idea that came along just got accepted, we'd have a > disastrous hodge-podge of a language. And P

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Antoon Pardon
>>>> --debug or equivalent. >>> That assumes that it's simply an optimization. This is a distinct >>> semantic change. >> No, tail call optimization doesn't change the behavior of the program, >> for the worse anyway. > It does, because you lose

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: >>> I would rather optimize by default and disable optimizations with >>> --debug or equivalent. >> >> That assumes t

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: >> I would rather optimize by default and disable optimizations with >> --debug or equivalent. > > That assumes that it's simply an optimization. This is a distinct > semantic change. No, t

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Ian Kelly
; optimize; > > Why not? > >> it just means to return the result. > > Same difference. > >> This is what led to the confusion responsible for the bug that Chris >> pointed out in the first place. With a keyword that explicitly means >> "perform tail-call o

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
it would be to hack this >> into CPython... Anyone feel like tackling it? > > I would rather optimize by default and disable optimizations with > --debug or equivalent. That assumes that it's simply an optimization. This is a distinct semantic change. Would you, for instance,

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > Done explicitly like this, it avoids the problem of bad tracebacks, > because by definition you would use it only when you want the current > stack frame to be dropped. I wonder how hard it would be to hack this > into CPython... Anyone feel like tackling it? I would rather opt

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > On Tue, Jul 14, 2015 at 3:41 PM, Marko Rauhamaa wrote: >> Tail-call optimization has nothing to do with converting algorithms into >> iterations. It's a prosaic trick of dropping an unneeded stack frame >> before making a function call. >>

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Tue, Jul 14, 2015 at 3:41 PM, Paul Rubin wrote: > Chris Angelico writes: >> That's a prime example of recursion... but not of recursion that can >> be tail-call optimized into iteration. It's an example of forking >> recursion, where one call can result in multiple calls (same with tree >> tra

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Tue, Jul 14, 2015 at 3:41 PM, Marko Rauhamaa wrote: > Tail-call optimization has nothing to do with converting algorithms into > iterations. It's a prosaic trick of dropping an unneeded stack frame > before making a function call. > >> The claim that TCO means you don

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Antoon Pardon
On 07/14/2015 10:34 AM, Ian Kelly wrote: > On Tue, Jul 14, 2015 at 1:26 AM, Antoon Pardon > wrote: >> I expect any example given, to be used as a contest by those reading, >> for finding >> the least warped alternative and then using that to dismiss the example. >> >> So I really don't feel compel

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
This is what led to the confusion responsible for the bug that Chris > pointed out in the first place. With a keyword that explicitly means > "perform tail-call optimization *and* return", That could well be the explicit definition of the "return" statement in Pytho

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Ian Kelly
On Tue, Jul 14, 2015 at 1:26 AM, Antoon Pardon wrote: > On 07/14/2015 05:20 AM, Steven D'Aprano wrote: >> On Tue, 14 Jul 2015 12:05 am, Chris Angelico wrote: >> >>> If you want to make an assertion that iterative >>> code requires equivalent warping to tail-recursive code, I want to see >>> an exa

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Ian Kelly
vor of an explicit tail-call keyword. Then one >> couldn't make that mistake. > > How about "return"? I think you miss my point entirely. "return" doesn't mean tail-call optimize; it just means to return the result. This is what led to the confusion respons

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Mark Lawrence
On 14/07/2015 07:29, Gregory Ewing wrote: Marko Rauhamaa wrote: Ian Kelly : Another point in favor of an explicit tail-call keyword. Then one couldn't make that mistake. How about "return"? How about "goto"? :-) Why not "goto"? It's use is scattered throughout the cpython code, and to

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Antoon Pardon
On 07/14/2015 05:20 AM, Steven D'Aprano wrote: > On Tue, 14 Jul 2015 12:05 am, Chris Angelico wrote: > >> If you want to make an assertion that iterative >> code requires equivalent warping to tail-recursive code, I want to see >> an example of it. Is that difficult? > Of course it is. If it wasn't

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Gregory Ewing : > Marko Rauhamaa wrote: >> How about "return"? > > How about "goto"? :-) > > That's not entirely an unserious suggestion -- if you > really believe that a "goto with arguments" is a good > feature for a language to have, you shouldn't be > afraid to spell it as such. > > def quic

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Gregory Ewing
Marko Rauhamaa wrote: Ian Kelly : Another point in favor of an explicit tail-call keyword. Then one couldn't make that mistake. How about "return"? How about "goto"? :-) That's not entirely an unserious suggestion -- if you really believe that a "goto with arguments" is a good feature for

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Marko Rauhamaa
Ian Kelly : > On Mon, Jul 13, 2015 at 11:25 PM, Chris Angelico wrote: >> (Also, side point: Python can't actually optimize the above function, >> because it actually means "call quicksort, then discard its return >> value and return None". A true tail call has to return the result of >> the recur

  1   2   3   4   5   >