Decorating bound-methods in Python — a general and scalable solution

9 min readMar 16, 2021

Decorators in Python can quickly become a complicated matter when dealing with bound-methods (i.e. class-instance bound function calls). In some cases a basic decorator is required. In other cases, we would like to supply parameters to the decorator itself. We also would like to extend/modify decorator definitions and not have to implement the complexities each time. Lastly we would like to control execution before, during and after a wrapped method call and having each such phase be configurable.

After experimenting with a variety of approaches, I have come up with a generalized solution that enables highly configurable and extensible bound-method decorators which use the elegance of object-oriented (OO) mechanics and the Python descriptor protocol (optional reading and not for the feint of heart).

In this article I describe the solution and the features it provides. It will also become quite clear why this is in fact a somewhat tricky problem that requires a deep understanding of the Python interpreter data-model dynamics. Despite these complexities, the solution presented is very easy to use. Besides providing a re-usable solution, I hope this article also goes some way to deepen your understanding of the Python language.

Decorator Semantics

In order to properly appreciate the challenges involved and to better define the problem statement, it’s important to elaborate the underlying semantics of function decorators and how things quickly get complicated when we require variability in the declarations and thus dynamics.

Consider the following decorator usage styles:

Let’s compare and contrast these styles to comprehend what actually happens:

The statement Decorator(method1) must return a callable so that invocations of method1 follow the semantics of a bound method-call. This syntax seems consistent with initializing a Decorator object with the wrapped function object reference as the initializer argument. This suggests that the Decorator object (let’s call it decorator) is in fact itself a callable, but even more-so, it must be callable in a way that captures the bound object of the method-call (more on this complication later)
The statement Decorator(param1="foo")(method2) must also return a callable, but this syntax looks more compounded, in the sense that the Decorator object initialization takes some parameter(s) other than the wrapped function and this is followed by a call on this object with the wrapped function object reference, i.e. the call occurs when this line in the class definition is interpreted. It suffices to say however, that this call only happens once, and subsequent calls are on the returned callable. Such subsequent calls must also be on a bound object

The point here is that despite the syntactic sugar of the decorator syntax, the underlying dynamics of the effective assignments could not be more different and this is what creates trouble in trying to support both formats.

Usage goals

Now that we’ve shown how a subtle difference in high-level syntax creates completely different mechanisms under the hood, lets formalize what makes for a complete and general solution for bound-method decorators. Consider a class with methods we may wish to decorate. We wish to do the following:

Decorate specific bound-methods with a decorator
In some cases, the decorator will have no parameters and can use a typical syntax without parentheses, e.g. @Decorator
In other cases, we would like parameters to be passed to the decorator, e.g. @Decorator(*args,**kwargs)

Each decorator should have the option to do the following (let’s call this the decorator functionality):

Optionally perform actions before calling the wrapped method with the ability to operate on the bound object instance
Optionally override the default calling of the wrapped method in a particular way (e.g. fixing some argument) as a bound method call
Optionally perform actions after calling the wrapped method with the ability to operate on the bound object instance. This should also allow the return value of the wrapped method to be modified for example

We would also like to create specializations of the decorator because we may want variations in the behaviors of the decorator functionality. The wrapped calling sub-phases are shown in the figure below. Any such sequence must be defined in a base class so that it can be overridden.

Solution Overview

The solution requires handling two underlying mechanisms with some common structures. It turns out that the second case (decorator with parameters) is in fact the simpler case. First we will discuss the essence of the solution for each case.

Using a dedicated class

It perhaps goes without saying that using a dedicated class is the best approach to solving this problem. The main reasons are the following:

Classes are great for problems that require stateful behaviors
Classes provide construction semantics (via __init__())
Classes provide callability semantics (via __call__())
Classes provide the ability to extend and override
The descriptor protocol of the Python interpreter is only available via a dedicated class

Decorators with parameters

This case (case #2 in the previous code example), can be implemented with the simplest approach. The reason is that it can be done completely without the descriptor protocol. Let’s look again at the following line which is the effective representation of the decorator assignment:

method2=Decorator(param1="foo")(method2)

Here method2 is a bound method (i.e. one that takes the bound object reference typically denoted as self as the first argument). Three things happen when this line is executed by the interpreter during what I would refer to as the decoration phase (i.e., the point at which the interpreter is executing this line as it reads the class definition):

Reading from left to right, we first initialize a new Decorator object which calls the Decorator.__init__(param1="foo") method and returns a reference to a new Decorator object. Let’s refer to this anonymous object as decorator. It’s anonymous because we never assign its reference directly to any variable
Continuing to read the statement, we now have an object reference decorator which we subsequently call, which under the hood will look like decorator.__call__(method2). Again, this call occurs during the decoration phase. We’re not referring to any bound method calls here
The decorator object call method returns a callable which represents the wrapped method call matching the previous figure showing the decorated call broken out in sub-phases

There is nothing especially complicated here, except that we must make sure to understand that when the now wrapped method2 is invoked as a bound-method call (e.g., some_object.method2()), that the first argument in the callable is a reference to the bound instance. It turns out that the interpreter takes care of providing this reference which is passed into the callable call because this callable is local to each object of the class which defines method2. This is important to understand because the other case acts differently.

Decorators without parameters

This is where things get a a bit more tricky. Let’s dissect the mechanics of what such a decorator assignment actually does:

method1 = Decorator(method1) # equivalent to @Decorator

This seems harmless enough. But what is it actually doing? Clearly it is just a an initializer call of the Decorator class which passes the method reference (method1) in this case as the only argument. This means that method1 is assigned a reference to a Decorator object (let’s call it decorator). There is no subsequent call to get a callable. So, once the interpreter has processed this line, we effectively have:

method1 = decorator # A class-variable reference to a Decorator object

Bound-method calls of method1 thus will result in calling the decorator class-level object (i.e. this object is bound to the class which defines method1).

What makes this different is that now all bound calls are actually bound to the decorator object

The difference is that in the previous case (parameterized-decorator), a function object (callable) is provided to method2 during the decoration phase, which is always called as a bound-method call, such that calls to this callable follow regular bound-method call semantics.

In the parameter-less case, the mechanics resulted in a Decorator initializer method being called. There is no way to return a function object from a class initializer call. An initializer always returns a reference to the initialized object. In this case, the returned decorator object is just an associated object (via composition), so when method1() is called (during a bound-method call), decorator.__call__() is called. The call is bound, but it’s bound to the decorator object, not the client object. This may appear to be an inconvenience, but any other behavior would in fact be nonsense.

How do we provide the bound object instance reference to this __call__()method?

The solution to this problem requires leveraging the Python “descriptor protocol”

To get into all the background and mechanics of the descriptor protocol would fill several posts, so let’s not go there. Instead, we can focus on the relevant dynamics which allow us to intercept and capture the caller’s instance reference before the decorator.__call__() method is invoked.

To make this work, we add the __get__() method in the Decorator class, which, via Python’s wonderful data-model type-emulation system (affectionately known as duck-typing), turns a Decorator class into a descriptor.

A descriptor allows attributes to be implemented via a class-bound object which has access to the instance reference of any object of that client class.

You may have already used descriptors, albeit indirectly if you have leveraged properties. Properties are just a very common pattern implemented upon the descriptor protocol framework

Once the Decorator class becomes a descriptor, it turns the decorator object into an attribute of the client class . Descriptors are always class-bound (because attribute definitions and method definitions are in fact class-bound). When a descriptor is accessed as a callable via an object attribute,

some wonderful things happen under the hood:

The __get__() method is called in the first phase. This is called when the attribute is read. It is during this time that the caller’s instance reference is passed to our descriptor object. We save this information for the subsequent call phase (this is the crux of the magic)
Following that, there is a function call attempt which will target the __call__() method of the descriptor object. At this point in time, we have the bound object(instance) reference (this was saved in the last step)

So one can see that it is via the __get__() method, that we can capture the calling object reference and then we use this captured reference during the call phase to implement the decorator call-phases (with full access to the calling object reference) to implement the earlier diagram. Just to be clear, both __get__() and __call__() are called by the interpreter in sequence when performing a calling action on an attribute implemented by a descriptor. I hope you’re still with me because that was the heavy part.

Example Code and Source

The source-code for the bound-method decorator base-class BMDecorate and an associated test are available in my personal GitHub project here:

https://github.com/sebastian-ahmed/python-etc/tree/main/bound_method_decorators

It is best to demonstrate the solution via an example. The example below creates a specialization of BMDecorate called my_decorator which overrides all three phases (this is equivalent to the test.py code in the above GitHub link:

This example demonstrates that specializations of BMDecorate can be used with our without parameters. Running the above example generates the following output:

Output of running https://github.com/sebastian-ahmed/python-etc/blob/main/bound_method_decorators/test.py

The output above shows in detail all the phases and steps that occur, including the actual decoration phase which occurs when the interpreter is processing the A and B class definitions (which declare the decorators).

Conclusion

In this article I presented a general solution for bound-method decorators via a specialized class which can be extended for custom use. Providing the flexibility for both bare and parametrized decorators along with multi-phased execution required getting into the guts of the descriptor protocol which is the fundamental mechanism in Python to implement properties and other intelligent attributes.

I mostly wrote this article so that anyone who is wishing to decorate bound-methods can find and re-use this solution. Lastly, I hope that it provided you some increased depth in your Python OO knowledge.