Quick One-Positional-Argument Function Currying In Pythonpermalink categories: programming originally posted: 2011-11-01 14:48:12
Function "currying" in Python means pre-adding arguments to a function. If you have a function that takes two arguments, you can create a new function from it that only takes one, if you can somehow automatically set the second parameter. (My understanding is that the term "curry" isn't totally correct here; the correct mathematical term would be "partial application". Calling this "currying" is but one of the Python community's many bad habits.)
The classic way to achieve this in Python is with a closure:
def make_closure(fn, argument): def closure(*a, **kw): return fn(argument, *a, **kw) return closure def fn(a, b): print(a, b) curried = make_closure(fn, 33) curried(44)
In the above example, curried is function that only takes one argument, but it calls fn with two—curried passes in 33 to fn for you, each time you call it. If you run the code above, Python will print 33 and 44.
The Python standard library has a relatively-new function to make closures for you, functools.partial. It's just as flexible as using a closure but much faster. That's because it's implemented in C. (Thus proving the eternal rule: if there's code in the Python standard library to do what you want, you should always use it.)
However! There's yet a third technique for closures that, while somewhat restrictive, can be even faster. And it works with older versions of Python—I imagine all the way back to Python 2.2! It abuses the mechanism behind how Python implements methods.
Consider the following code:
class C(object): def foo(self): pass c = C() x = c.foo
When that finishes execution, x will be a callable object, and when you call it you'll call the foo method of class C, passing in c as its first argument. In short, a method call. And that's what x is! It's a method object:
>>> type(x) <class 'method'>
A method object contains two things: a reference to the callable, and a reference to the object you wish passed in as the first positional parameter. Method objects are also callable. When you execute the method object, it takes the arguments passed in, adds its internal argument to the front of the list of positional arguments, and calls its internal callable with those arguments.
(And, in case you were wondering, yes: every time you call a method in Python, it creates one of these objects, through something called the "descriptor protocol". Since this happens so frequently, CPython is heavily optimized to make this fast.)
As it happens, Python will happily let you construct your own method objects. All you need is the method type object, which Python publishes in the types module as MethodType, though I'm going to refer to it as method for the rest of this article. method takes two parameters: the callable, and the argument you want used as the first function. It returns the callable method object.
Obviously this is not as fully-featured as functools.partial. But it is a hair faster. My microbenchmarks suggest that a method object is 20% faster than functools.partial, and 3x faster than a closure! Creating them is faster too.
However: you shouldn't use this for more than one argument at a time. It's possible to compose a function with two curried arguments with method, just use it twice:
def double_curry(fn, a, b): return MethodType(MethodType(fn, a), b) # or as many arguments as you like def method(fn, *a): if not a: return fn return method(MethodType(fn, a), a[1:])
But now this approach is slower than functools.partial. The reason why is obvious: with two method objects, we unpack and repack the arguments twice, adding one argument each time. But functools.partial adds both arguments in one pass. And it wasn't that much slower in the first place. With two arguments it races ahead!
I'm hard-pressed to suggest that you'd ever actually use this approach. I suppose, if you know you're only going to be using CPython, and you need to curry exactly one positional argument, and it's performance-critical code, you should use it. Otherwise, just stick with functools.partial.
If you're curious, you can see my microbenchmark here.