Dear Oyibo, first of all, I should say that I'm not convinced that the example contained in the matecdev site is a good example. Whilst I haven't checked it seems that using `interpolate.LSQUnivariateSpline` may achieve what that poster was trying to accomplish, and would be much faster than the python code presented.
Most of the answer to your question is pretty much contained in https://github.com/scipy/scipy/issues/11783. In terms of speed the most important thing is to ensure that the objective function calculates quickly. This typically involves moving calculations from Python to C/Cython, etc. Only once this has been investigated is it worth looking into optimiser overhead. This overhead is often (but not always) much smaller than the time required for objective function calculation. Such overhead includes objective function call time, or overhead from the optimiser itself. There is a wide heterogeneity of code design in scipy.optimise. Some work: A) in pure python (e.g. BFGS, nelder-mead) B) in C/Fortran where you supply a callback function for the native code to call (e.g. tnc, leastsq) C) in C/Fortran but where iteration is controlled by python and the native code expects a objective/gradient function updates each time it's called (L-BFGS-B). Using LowlevelCallable would not furnish any gains for case A. Rewriting of those codes to cython would be a lot of work for an uncertain amount of gain. Using LowlevelCallable for case B would likely furnish the most gain. Using LowlevelCallable for case C would probably involve rewriting so that all iteration occurred from Cython/C, but the level of gain is still uncertain. There remain further issues. Let's take `tnc` as an example (case B). For that minimiser it's necessary to supply both objective AND gradient evaluations. The user must supply the first, but if the user cannot supply the gradient then `minimize` is asked to estimate it via numerical differentiation. Estimating the gradient with numerical differentiation occurs in Python. This kind of setup is taken care of by using the `optimize._differentiable_functions.ScalarFunction` object. Thus, a LowlevelCallable that only furnishes objective evaluations is not much use, because the numerical differentiation (in Python) is still required to evaluate gradients, which requires at least N objective calls (from Python to whatever the objective is contained in). In order for this to be fully streamlined the numerical differentiation code would also have to be written in cython/c. Note that the user can either supply the objective function, separate objective and gradient functions, or a combined objective and gradient function. All those different cases have to be catered for. Moreover, for each of those cases one would either be supplying a `LowlevelCallable` or Python function. Last, but not least, most minimisers often ask numpy to perform operations, so porting minimisers to cython will still have python calls. Thus, it's a large task just to amend a *single* minimiser. There are lots of minimisers and one would want to do this in a uniform manner across them all. Given the width of such a project, one would want to conclusively know that it would be worth it, and that's definitely not clear. Andrew.
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-le...@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: arch...@mail-archive.com
