Re: Functions Of Functions Returning Functions

[Lawrence D’Oliveiro]

On Monday, September 19, 2016 at 6:54:31 PM UTC+12, dieter wrote:
> Some time ago, we had a (quite heated) discussion here ...

I have noticed that people get very defensive about things they don’t 
understand.

> Often, functions returning functions are more difficult to understand
> than "first order" functions (those returning simple values only) --
> at least for people not familiar with higher abstraction levels.

Maybe if the alternative meant writing 200 extra lines of code (as in this 
case), they might feel differently...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Functions Of Functions Returning Functions

[dieter]

Lawrence D’Oliveiro <lawrenced...@gmail.com> writes:
> The less code you have to write, the better. Less code means less
> maintenance, and fewer opportunities for bugs.

While I agree with you in general, sometimes less code can be harder
to maintain (when it is more difficult to understand).

Some time ago, we had a (quite heated) discussion here about
a one line signature transform involving a triply nested lambda construction.
It was not difficult for me to understand what the construction did;
however, the original poster found it very difficult to grasp.

Often, functions returning functions are more difficult to understand
than "first order" functions (those returning simple values only) --
at least for people not familiar with higher abstraction levels.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Functions Of Functions Returning Functions

[Steve D'Aprano]

On Sun, 18 Sep 2016 08:28 pm, Lawrence D’Oliveiro wrote:

> This shows the power of functions as first-class objects. The concept
> is older than object orientation, and is often left out of
> object-oriented languages. I think Python benefits from the fact that
> it had functions before it had classes.

You're right about Python having functions first:

steve@runes:~$ python0.9.1
>>> def f():
... pass
...
>>> class A:
Parsing error: file , line 1:
class A:
^
Unhandled exception: run-time error: syntax error



However it only gained closures and nested scopes in Python 2.2, or with a
__future__ directive in 2.1.


https://www.python.org/dev/peps/pep-0227/




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Functions Of Functions Returning Functions

[Lawrence D’Oliveiro]

The less code you have to write, the better. Less code means less
maintenance, and fewer opportunities for bugs. Here is an example of
how I was able to knock a few hundred lines off the size of a Python
module.

When I was writing my Python wrapper for HarfBuzz
, there were a lot of places where you
could define routines for HarfBuzz to make calls to, to customize the
type-shaping process in various ways.

Of course, HarfBuzz  is a library
written in C++, and it doesn’t know that the callbacks you pass are
actually written in Python. But ctypes
 provides an answer to
this: its CFUNCTYPE function lets you wrap Python functions so that
they become callable from C/C++ code.

When I said there were a lot of places for callbacks, I meant a lot of
places. For example, there is a HarfBuzz object called “hb_font_funcs”
, which
consists of nothing more than a container for 14 different action
callback routines. Not only that, but each one lets you pass a
separate “user data” pointer to the action callback, along with an
optional “destroy” callback which can do any necessary cleanup of this
data, which will be called when the hb_font_funcs object is disposed.

Imagine having to set all of these up by hand. Each API call to
install a callback would look something like this:

def set_xxx_callback(self, callback_func, user_data, destroy) :
"sets the xxx callback, along with an optional destroy callback" \
" for the user_data. The callback_func should be declared as 
follows:\n" \
"\n" \
"def callback_func(self, ... xxx-specific args ..., user_data)\n" \
"\n" \
"where self is the FontFuncs instance ... description of xxx-specific 
args."

def def_wrap_xxx_callback(self, callback_func, user_data)
# generates ctypes wrapper for caller-specified Python function.

@HB.font_get_xxx_func_t
def wrap_xxx_callback(... xxx-specific args ..., c_user_data) :
... convert xxx-specific args from ctypes representation to ...
... higher-level Python representation, pass to callback_func, 
...
... along with user_data, then convert any result to ctypes ...
... representation and return as my result ...
#end wrap_xxx_callback

#begin def_wrap_xxx_callback
return \
wrap_xxx_callback
#end def_wrap_xxx_callback

#begin set_xxx_callback
wrap_callback_func = def_wrap_xxx_callback(self, callback_func, 
user_data)
if destroy != None :
@HB.destroy_func_t
def wrap_destroy(c_user_data) :
destroy(user_data)
#end wrap_destroy
else :
wrap_destroy = None
#end if
# save references to wrapper objects to prevent them prematurely
# disappearing (common ctypes gotcha)
self._wrap_xxx_func = wrap_callback_func
self._wrap_xxx_destroy_func = wrap_destroy
hb.hb_font_funcs_set_xxx_func(self._hbobj, wrap_callback_func, None, 
wrap_destroy)
#end set_callback

Just think if you had to do this 14 times. And then there are a couple
of other HarfBuzz objects with their own similar collections of
callbacks as well...

Luckily, I figured out a way to cut the amount of code needed for this
by about half. It’s the recognition that the only part that is
different between all these callback-setting calls is the
“def_wrap_xxx_callback” function, together with a few different
attribute names elsewhere. So I encapsulated the common part of all
this setup into the following routine:

def def_callback_wrapper(celf, method_name, docstring, callback_field_name, 
destroy_field_name, def_wrap_callback_func, hb_proc) :
# Common routine for defining a set-callback method. These all have the 
same form,
# where the caller specifies
#  * the callback function
#  * an additional user_data pointer (meaning is up the caller)
#  * an optional destroy callback which is passed the user_data pointer
#when the containing object is destroyed.
# The only variation is in the arguments and result type of the 
callback.

def set_callback(self, callback_func, user_data, destroy) :
# This becomes the actual set-callback method.
wrap_callback_func = def_wrap_callback_func(self, callback_func, 
user_data)
if destroy != None :
@HB.destroy_func_t
def wrap_destroy(c_user_data) :
destroy(user_data)
#end wrap_destroy
else :
wrap_destroy = None
#end if
setattr(self, callback_field_name, wrap_callback_func)
setattr(self, destroy_field_name,