Injecting attributes into Python modules

dan — Fri, 27 Nov 2009 08:13:49 +0000

One of the things I don’t like about Django is the amount of boilerplate imports you end up having at the beginning of modules, especially those modules that are more like configuration files. It spoils the DSL-like nature of them, and I wanted a way to be able to import a module with certain attributes already defined. The __import__ function doesn’t allow you to do this, because the locals argument is ignored, so I needed another way. Before I describe the method I came up with, here’s how you might use it:

import elixir
from inject import module_inject
module_inject('myapp.models', elixir)
import myapp.models

Easy!

PEP 302 describes the import hooks that have been available since Python 2.3, and defines an import protocol. By adding an object with find_module and load_module methods to sys.meta_path, you can get hooked into the import process. find_module is called with the module name to see if an object knows how to load it. load_module is then called to do the actual loading. The class below implements both of those methods.

class InjectionLoader(object):
    def __init__(self, name, dicts):
        self.name = name
        self.dicts = dicts
 
    def find_module(self, fullname, path=None):
        if fullname == self.name:
            return self
 
    def load_module(self, fullname):
        # Get the leaf module name and the directory it should be found in
        if '.' in fullname:
            package, leaf = fullname.rsplit('.', 1)
            path = sys.modules[package].__path__   
        else:
            leaf = fullname
            path = None
 
        # Open the module file
        file, filename, _ = imp.find_module(leaf, path)
 
        # Get the existing module or create a new one (for reload to work)
        module = sys.modules.setdefault(fullname, imp.new_module(fullname))
        module.__file__ = filename
        module.__loader__ = self  
 
        code = compile(file.read(), filename, 'exec')
 
        # Populate the module namespace with the injected attributes
        for d in self.dicts:
            module.__dict__.update(d)
 
        # Finally execute the module with its injected attributes
        eval(code, module.__dict__)
        return module

It’s instantiated with the module name it’s injecting to, and the dicts it is injecting. To make it easier to use, I wrote a helper function, module_inject. It takes a module name, and one or more dicts or modules. Dicts are injected as-is. Modules have their __dict__s injected, but only those attributes listed in the module’s __all__ attribute, or if that isn’t present then only those that don’t begin with a double underscore, are used. This is like doing a from module import * at the beginning of the imported module. Here is its implementation:

def module_inject(name, *args):
    """Set a hook so that when module 'name' is imported, it is executed with
    the attributes in 'args' already in module scope. The arguments can be
    dictionaries or modules (see 'normalize_dict')."""
    args = map(normalize_dict, args)
    sys.meta_path.append(InjectionLoader(name, args))
 
def normalize_dict(d):
    """If the argument is a module, return the module's dictionary filtered
    by the module's __all__ attribute, otherwise return the argument as-is.
    If the module doesn't have an __all__ attribute, use all the attributes
    that don't begin with a double underscore."""
    if isinstance(d, types.ModuleType):
        keys = getattr(
            d,
            '__all__',
            filter(lambda k: not k.startswith('__'), d.__dict__.keys())
        )
        d = dict([(key, d.__dict__[key]) for key in keys])
    return d

Initializing attributes from init arguments

dan — Sat, 03 Oct 2009 19:44:27 +0000

Every once in a while, I get fed up of having to do lots of self.foo = foo in Python __init__ methods, and wonder if it couldn’t be done automatically. I came up with the following function to do just that, but I doubt I’ll ever use it myself, because it goes against the explicit is better than implicit philosophy of Python.

#!/usr/bin/env python
import inspect
 
def init_from_args():
    frame = inspect.stack()[1][0]
    code = frame.f_code
    var_names = code.co_varnames # __init__'s parameters and locals
    init_locals = frame.f_locals # __init__'s dict of locals
    num_args = code.co_argcount # Number of arguments
    arg_names = var_names[1:num_args] # Positional argument names
 
    # If there's a **kwargs parameter, get the name of it
    kw_name = None
    if code.co_flags | 12:
        kw_name = var_names[num_args + 1]
    elif code.co_flags | 8:
        kw_name = var_names[num_args]
 
    # Copy the positional arguments
    for name in arg_names:
        setattr(init_locals[var_names[0]], name, init_locals[name])
 
    # If there was a **kwargs parameter, copy the keywork arguments.
    if kw_name:
        for name, value in init_locals[kw_name].items():
            setattr(init_locals[var_names[0]], name, value)
 
class Foo:
    def __init__(self, a, b, *args, **kwargs):
        init_from_args()
        bar = 123
        baz = "hello"
        quux = "foo"
 
if __name__ == "__main__":
    foo = Foo(1, 2, 3, something="something else")
    print foo.__dict__

Electric Words » hacks

Injecting attributes into Python modules

Initializing attributes from __init__ arguments

Initializing attributes from init arguments