Sunday, 22 February 2009

New way to organise Python test code

py.test just grew a new way to provide test state for a test function. First the problem: those of us dealing with writing tests in the hundreds or thousands usually setup test state at class, method or module level. Then we access it indirectly, through self, local or global helper functions. For larger applications, this usually leads to scattered, complex and boilerplatisch test code. This then stands in the way of refactoring the original code base ... but wait, weren't tests meant to ease refactoring, not hinder it?

Here is the idea: Python Test functions use their function definition to state their needs and the test tools calls a function that provides the value. For example, consider this test function:

def test_ospath(self, tempdir):
# needs tempdir to create files etc.

py.test provides the value for tempdir by calling a matching method that looks like this:

def pytest_pyfuncarg_tempdir(pyfuncitem):

This matching provider function returns a value for tempdir that is
then supplied to the test function. For more complex purposes, the
pyfuncitem argument provides full access to the test collection
process including cmdline options, test options, project specific
configuration. You can write down this provider method in the test
module, in configuration files or in a plugin.

Once i started using this new paradigm, i couldn't resist and refactored
pytest's own tests to use the new method everywhere. Here are my findings so far:

  • self contained test functions: i don't need to wade through unneccessary layers and indirection of test setup.

  • fewer imports: my test modules don't need to import modules that are only needed for setting up application state.

  • easy test state setup: I can place test support code
    in one place and i can grep for pytest_pyfuncarg_NAME. I can reuse this setup code easily across modules, directories or even projects. Think about providing test database object or mocking objects.

  • more flexible test grouping: I can logically group tests however i like, independently from test setup requirements. I found it very easy to shuffle test functions between classes or modules because they are rather self-contained.


  1. Is it available in py-trunk on svn or in some obscure branch?

  2. Hi Jakub - it is available on and soon on py/trunk

    btw, please note that i just moved the blog to because i like the interface better as it turns out.