Usage¶
As a quick-start, here’s how you would set up a conftest.py
in your
Sphinx source directory such that running pytest would check
doctest and
code-block
examples in your documentation source files, taking
into account the different representation of bytes
and
unicode
between Python 2 and 3, and also prefixing all
code-block
examples with a from __future__ import print_function
:
from doctest import ELLIPSIS
from sybil import Sybil
from sybil.parsers.codeblock import CodeBlockParser
from sybil.parsers.doctest import DocTestParser, FIX_BYTE_UNICODE_REPR
from sybil.parsers.skip import skip
pytest_collect_file = Sybil(
parsers=[
DocTestParser(optionflags=ELLIPSIS|FIX_BYTE_UNICODE_REPR),
CodeBlockParser(future_imports=['print_function']),
skip,
],
pattern='*.rst',
).pytest()
An example of a documentation source file that could be checked using the above configuration is shown below:
Sample Documentation
====================
Let's put something in the Sybil document's namespace:
.. invisible-code-block: python
remember_me = b'see how namespaces work?'
Suppose we define a function, convoluted and pointless but shows stuff nicely:
.. code-block:: python
import sys
def prefix_and_print(message):
print('prefix:', message.decode('ascii'))
Now we can use a doctest REPL to show it in action:
>>> prefix_and_print(remember_me)
prefix: see how namespaces work?
The namespace carries across from example to example, no matter what parser:
>>> remember_me
b'see how namespaces work?'
Method of operation¶
Sybil works by discovering a series of
documents
as part of the
test runner integration. These documents are then
parsed into a set of non-overlapping
regions
. When the tests are run, the
test runner integration turns each Region
into an Example
before evaluating each
Example
in the document’s
namespace
. The examples are evaluated
in the order in which they appear in the document.
If an example does not evaluate as expected, a test failure occurs and Sybil
continues on to evaluate the remaining
examples
in the
Document
.
Test runner integration¶
Sybil aims to integrate with all major Python test runners. Those currently catered for explicitly are listed below, but you may find that one of these integration methods may work as required with other test runners. If not, please file an issue on GitHub.
To show how the integration options work, the following documentation examples
will be tested. They use doctests,
code blocks
and require a temporary directory:
Another fairly pointless function:
.. code-block:: python
import sys
def write_message(filename, message):
with open(filename, 'w') as target:
target.write(message)
Now we can use a doctest REPL to show it in action:
>>> write_message('test.txt', 'is this thing on?')
>>> with open('test.txt') as source:
... print(source.read())
is this thing on?
pytest¶
To have pytest check the examples, Sybil makes use of the
pytest_collect_file
hook. To use this, configuration is placed in
a confest.py
in your documentation source directory, as shown below.
pytest
should be invoked from a location that has the opportunity to
recurse into that directory:
from os import chdir, getcwd
from shutil import rmtree
from tempfile import mkdtemp
import pytest
from sybil import Sybil
from sybil.parsers.codeblock import CodeBlockParser
from sybil.parsers.doctest import DocTestParser
@pytest.fixture(scope="module")
def tempdir():
# there are better ways to do temp directories, but it's a simple example:
path = mkdtemp()
cwd = getcwd()
try:
chdir(path)
yield path
finally:
chdir(cwd)
rmtree(path)
pytest_collect_file = Sybil(
parsers=[
DocTestParser(),
CodeBlockParser(future_imports=['print_function']),
],
pattern='*.rst',
fixtures=['tempdir']
).pytest()
The file glob passed as pattern
should match any documentation source
files that contain examples which you would like to be checked.
As you can see, if your examples require any fixtures, these can be requested
by passing their names to the fixtures
argument of the
Sybil
class.
These will be available in the Document
namespace
in a way that should feel natural
to pytest
users.
The setup
and teardown
parameters can still be used to pass
Document
setup and teardown callables.
The path
parameter, however, is ignored.
unittest¶
To have Test Discovery check the example, Sybil makes use of
the load_tests protocol. As such, the following should be placed in a test
module, say test_docs.py
, where the unit test discovery process can find it:
from os import chdir, getcwd
from shutil import rmtree
from tempfile import mkdtemp
from sybil import Sybil
from sybil.parsers.codeblock import CodeBlockParser
from sybil.parsers.doctest import DocTestParser
def sybil_setup(namespace):
# there are better ways to do temp directories, but it's a simple example:
namespace['path'] = path = mkdtemp()
namespace['cwd'] = getcwd()
chdir(path)
def sybil_teardown(namespace):
chdir(namespace['cwd'])
rmtree(namespace['path'])
load_tests = Sybil(
parsers=[
DocTestParser(),
CodeBlockParser(future_imports=['print_function']),
],
path='../docs', pattern='*.rst',
setup=sybil_setup, teardown=sybil_teardown
).unittest()
The path
parameter gives the path, relative to the file containing this
code, that contains the documentation source files.
The file glob passed as pattern
should match any documentation source
files that contain examples which you would like to be checked.
Any setup or teardown necessary for your tests can be carried out in
callables passed to the setup
and teardown
parameters,
which are both called with the Document
namespace
.
The fixtures
parameter, is ignored.
nose¶
Sybil acts as a test loader plugin for nose that provides a
--test-suite-func
ooption which defaults to load_tests
, so making
nose respect the load_tests protocol.
Provided Sybil is activated as a nose plugin, the following code, when placed in a test module where nose can find it, will result in your documentation examples being checked:
from os import chdir, getcwd
from shutil import rmtree
from tempfile import mkdtemp
from sybil import Sybil
from sybil.parsers.codeblock import CodeBlockParser
from sybil.parsers.doctest import DocTestParser
def sybil_setup(namespace):
# there are better ways to do temp directories, but it's a simple example:
namespace['path'] = path = mkdtemp()
namespace['cwd'] = getcwd()
chdir(path)
def sybil_teardown(namespace):
chdir(namespace['cwd'])
rmtree(namespace['path'])
load_tests = Sybil(
parsers=[
DocTestParser(),
CodeBlockParser(future_imports=['print_function']),
],
path='../docs', pattern='*.rst',
setup=sybil_setup, teardown=sybil_teardown
).nose()
The path
parameter gives the path, relative to the file containing this
code, that contains the documentation source files.
The file glob passed as pattern
should match any documentation source
files that contain examples which you would like to be checked.
Any setup or teardown necessary for your tests can be carried out in
callables passed to the setup
and teardown
parameters,
which are both called with the Document
namespace
.
The fixtures
parameter is ignored.