Skip to content

Electric Words

Dan Ellis’s technical blog

Archive

Tag: Python

Injecting attributes into Python modules

Nov 27
Python

One of the things I don’t like about frameworks like Django and Pyramid is the amount of boilerplate imports you end up having at the beginning of modules, especially those modules that are more like configuration files. It spoils the DSL-like nature of them, and I was interested in finding a way to be able to import a module with certain attributes already defined. The __import__ function doesn’t allow you to do this, because the locals argument is ignored, so I looked for another way. Before I describe the method I came up with, here’s how you might use it:

import elixir
from inject import module_inject
module_inject('myapp.models', elixir)
import myapp.models

Easy!

PEP 302 describes the import hooks that have been available since Python 2.3, and defines an import protocol. By adding an object with find_module and load_module methods to sys.meta_path, you can get hooked into the import process. find_module is called with the module name to see if an object knows how to load it. load_module is then called to do the actual loading. The class below implements both of those methods.

class InjectionLoader(object):
    def __init__(self, name, dicts):
        self.name = name
        self.dicts = dicts
 
    def find_module(self, fullname, path=None):
        if fullname == self.name:
            return self
 
    def load_module(self, fullname):
        # Get the leaf module name and the directory it should be found in
        if '.' in fullname:
            package, leaf = fullname.rsplit('.', 1)
            path = sys.modules[package].__path__   
        else:
            leaf = fullname
            path = None
 
        # Open the module file
        file, filename, description = imp.find_module(leaf, path)
 
        # Get the existing module or create a new one (for reload to work)
        module = sys.modules.setdefault(fullname, imp.new_module(fullname))
        module.__file__ = filename
        module.__loader__ = self  
 
        code = compile(file.read(), filename, 'exec')
 
        # Populate the module namespace with the injected attributes
        for d in self.dicts:
            module.__dict__.update(d)
 
        # Finally execute the module with its injected attributes
        eval(code, module.__dict__)
        return module

It’s instantiated with the module name it’s injecting to, and the dicts it is injecting. To make it easier to use, I wrote a helper function, module_inject. It takes a module name, and one or more dicts or modules. Dicts are injected as-is. Modules have their __dict__s injected, but only those attributes listed in the module’s __all__ attribute, or if that isn’t present then only those that don’t begin with a double underscore, are used. This is like doing a from module import * at the beginning of the imported module. Here is its implementation:

def module_inject(name, *args):
    """Set a hook so that when module 'name' is imported, it is executed with
    the attributes in 'args' already in module scope. The arguments can be
    dictionaries or modules (see 'normalize_dict')."""
    args = map(normalize_dict, args)
    sys.meta_path.append(InjectionLoader(name, args))
 
def normalize_dict(d):
    """If the argument is a module, return the module's dictionary filtered
    by the module's __all__ attribute, otherwise return the argument as-is.
    If the module doesn't have an __all__ attribute, use all the attributes
    that don't begin with a double underscore."""
    if isinstance(d, types.ModuleType):
        keys = getattr(
            d,
            '__all__',
            filter(lambda k: not k.startswith('__'), d.__dict__.keys())
        )
        d = dict([(key, d.__dict__[key]) for key in keys])
    return d

It’s something to be used with caution, though. In general, the Python mantra of explicit is better than implicit is a good guideline to follow.

Update: somebody asked me about the use of file as a local variable. I’m actually torn on the issue. Yes, it does shadow the built-in file function, but on the other hand it’s concise, and it’s the same name used in the Python documentation.

Comments Off , Read more

Initializing attributes from __init__ arguments

Oct 3
Python

Every once in a while, I get fed up of having to do lots of self.foo = foo in Python __init__ methods, and wonder if it couldn’t be done automatically. I came up with the following function to do just that, but I doubt I’ll ever use it myself, because it goes against the explicit is better than implicit philosophy of Python.

#!/usr/bin/env python
import inspect
 
def init_from_args():
    frame = inspect.stack()[1][0]
    code = frame.f_code
    var_names = code.co_varnames # __init__'s parameters and locals
    init_locals = frame.f_locals # __init__'s dict of locals
    num_args = code.co_argcount # Number of arguments
    arg_names = var_names[1:num_args] # Positional argument names
 
    # If there's a **kwargs parameter, get the name of it
    kw_name = None
    if code.co_flags | 12:
        kw_name = var_names[num_args + 1]
    elif code.co_flags | 8:
        kw_name = var_names[num_args]
 
    # Copy the positional arguments
    for name in arg_names:
        setattr(init_locals[var_names[0]], name, init_locals[name])
 
    # If there was a **kwargs parameter, copy the keywork arguments.
    if kw_name:
        for name, value in init_locals[kw_name].items():
            setattr(init_locals[var_names[0]], name, value)
 
class Foo:
    def __init__(self, a, b, *args, **kwargs):
        init_from_args()
        bar = 123
        baz = "hello"
        quux = "foo"
 
if __name__ == "__main__":
    foo = Foo(1, 2, 3, something="something else")
    print foo.__dict__
12,138 Comments , Read more

Parameterised classes in Python

Aug 7
Python

Recently, when working with Django, I wanted to create a class that would automatically render instances of a model as an editable HTML table. I used a Django-esque approach to defining the table columns, such that it was similar to Model and Form classes:

class ProductTable(Table):
    _model = Product
    name = TextWidget()
    description = TextWidget()
    size = DropDownWidget()

The underscore in _model is there to prevent a name collision with the column names, but it’s a bit ugly. If I was strictly following the Django approach, I probably would have added an inner class:

class ProductTable(Table):
    class Meta:
        _model = Product
 
    name = TextWidget()
    description = TextWidget()
    size = DropDownWidget()

But that’s a lot of extra noise. What I really wanted was something more like template classes in C++.

template <class Model>
class Table : public TableBase {
  // ...
}
 
class ProductTable : public Table<Product> {
  // ...
}

If Python supported class decorators, I could have just put @model(Product) before the class definition. The approach I chose is to inherit from a class dynamically generated by a function that takes the parameter I need, so my ProductTable class now becomes:

class ProductTable(Table(Product)):
    name = TextWidget()
    description = TextWidget()
    size = DropDownWidget()

Much nicer! Now it pretty much reads as ‘a ProductTable is a kind of Table (a Product one)’, which makes sense. To support this, I have a Table function that looks like this:

def Table(model):
    class TableDynamicBase(TableBase):
        _model = model
    return TableDynamicBase

TableBase contains whatever was in the original Table base class (which in my case was some unrelated metaclass magic).

Comments Off Read more

Introduction to Django

Aug 6
Python

I haven’t had any kind of personal web site for ages, but I recently wrote an article, Introduction to Django, for Digital Web magazine, so I thought I’d put a quick something up here as a point of contact.

12,941 Comments , , Read more
Electric Words
Copyright © 2009 Electric Words Theme by SRS Solutions top