Pavilion Test Run Objects

pavilion.pav_test module

Contains the TestRun class, as well as functions for getting the list of all known test runs.

class pavilion.test_run.TestRun(pav_cfg, config, build_tracker=None, var_man=None, _id=None, rebuild=False, build_only=False)

The central pavilion test object. Handle saving, monitoring and running tests.

Test LifeCycle 1. Test Object is Created – TestRun.__init__

  1. Test id and directory (working_dir/test_runs/0000001) are created.
  2. Most test information files (config, status, etc) are created.
  3. Build script is created.
  4. Build hash is generated.
  5. Run script dry run generation is performed.
  1. Test is built. – test.build()
  2. Test is finalized. – test.finalize()
    1. Variables and config go through final resolution.
    2. Final run script is generated.
  3. Test is run. – test.run()
  4. Results are gathered. – test.gather_results()
Variables:
  • id (int) – The test id.
  • config (dict) – The test’s configuration.
  • test.path (Path) – The path to the test’s test_run directory.
  • suite_path (Path) – The path to the test suite file that this test came from. May be None for artificially generated tests.
  • results (dict) – The test results. Set None if results haven’t been gathered.
  • builder (TestBuilder) – The test builder object, with information on the test’s build.
  • build_origin_path (Path) – The path to the symlink to the original build directory. For bookkeeping.
  • status (StatusFile) – The status object for this test.
  • opt (TestRunOptions) – Test run options defined by OPTIONS_DEFAULTS
__init__(pav_cfg, config, build_tracker=None, var_man=None, _id=None, rebuild=False, build_only=False)

Create an new TestRun object. If loading an existing test instance, use the TestRun.from_id() method.

Parameters:
  • pav_cfg – The pavilion configuration.
  • config (dict) – The test configuration dictionary.
  • build_tracker (pavilion.build_tracker.MultiBuildTracker) – Tracker for watching and managing the status of multiple builds.
  • var_man (variables.VariableSetManager) – The variable set manager for this test.
  • build_only (bool) – Only build this test run, do not run it.
  • rebuild (bool) – After determining the build name, deprecate it and select a new, non-deprecated build.
  • _id (int) – The test id of an existing test. (You should be using TestRun.load).
build(cancel_event=None)

Build the test using its builder object and symlink copy it to it’s final location. The build tracker will have the latest information on any encountered errors.

Parameters:cancel_event (threading.Event) – Event to tell builds when to die.
Returns:True if build successful
gather_results(run_result, regather=False, log_file=None)

Process and log the results of the test, including the default set of result keys.

Parameters:
  • run_result (int) – The return code of the test run.
  • regather (bool) – Gather results without performing any changes to the test itself.
  • log_file (IO[str]) – The file to save result logs to.
is_built

Whether the build for this test exists.

Returns:True if the build exists (or the test doesn’t have a build), False otherwise.
Return type:bool
job_id

The job id of this test (saved to a jobid file). This should be set by the scheduler plugin as soon as it’s known.

classmethod load(pav_cfg, test_id)

Load an old TestRun object given a test id.

Parameters:
  • pav_cfg – The pavilion config
  • test_id (int) – The test’s id number.
Return type:

TestRun
load_results()

Load results from the results file.

:returns A dict of results, or None if the results file doesn’t exist. :rtype: dict

run()

Run the test.

Return type:

bool
Returns:

The return code of the test command.
Raises:
  • TimeoutError – When the run times out.
  • TestRunError – We don’t actually raise this, but might in the future.
save_results(results)

Save the results to the test specific results file and the general pavilion results file.

Parameters:results (dict) – The results dictionary.
set_run_complete()

Write a file in the test directory that indicates that the test has completed a run, one way or another. This should only be called when we’re sure their won’t be any more status changes.

wait(timeout=None)

Wait for the test run to be complete. This works across hosts, as it simply checks for files in the run directory.

Parameters:timeout (Union(None,float)) – How long to wait in seconds. If this is None, wait forever.
Raises:TimeoutError – if the timeout expires.
class pavilion.test_run.TestAttributes(path: pathlib.Path, group: str = None, umask: int = None)

Bases: object

A object for accessing test attributes. TestRuns inherit from this, but it can be used by itself to access test information with less overhead.

All attributes should be defined as getters and (optionally) setters. Getters should return None if no value is available. Getters should all have a docstring.

WARNING

This object is not thread or any sort of multi-processing safe. It relies on the expectation that the test lifecycle should generally mean there’s only one instance of a test around that might change these values at any given time. The upside is that it’s so unsafe, problems reveal themselves quickly in the unit tests.

It’s safe to set these values in the TestRun __init__, finalize, and gather_results.

ATTR_FILE_NAME = 'attributes'
classmethod attr_deserializer(attr) → Callable[[Any], Any]

Get the deserializer for this attribute, if any.

attr_dict(include_empty=True, serialize=False)

Return the attributes as a dictionary.

classmethod attr_doc(attr)

Return the documentation string for the given attribute.

classmethod attr_serializer(attr) → Callable[[Any], Any]

Get the deserializer for this attribute, if any.

build_name

The name of the test run’s build.

build_only

Only build this test, never run it.

complete

Whether the test run considers itself done (regardless of whether it ran).

created

When the test was created.

deserializers = {'created': <function deserialize_datetime>, 'finished': <function deserialize_datetime>, 'started': <function deserialize_datetime>, 'suite_path': <function TestAttributes.<lambda>>}
finished

The end time for this test run.

id

The test run id (unique per working_dir at any given time).

static list_attrs()

List the available attributes. This always operates on the base RunAttributes class, so it won’t contain anything from child classes.

load_attributes()

Load the attributes from file.

load_legacy_attributes(initial_attrs=None)

Try to load attributes in older Pavilion formats, primarily before the attributes file existed.

name

The full name of the test.

rebuild

Whether or not this test will rebuild it’s build.

result

The PASS/FAIL/ERROR result for this test. This is kept here forfast retrieval.

save_attributes()

Save the attributes to file in the test directory.

serializers = {'suite_path': <function TestAttributes.<lambda>>}
skipped

Did this test’s skip conditions evaluate as ‘skipped’?

started

The start time for this test run.

suite_path

Path to the suite_file that defined this test run.

sys_name

The system this test was started on.

user

The user who created this test run.

uuid

A completely unique id for this test run (test id’s can rotate).

exception pavilion.test_run.TestRunError

Bases: RuntimeError

For general test errors. Whatever was being attempted has failed in a non-recoverable way.

exception pavilion.test_run.TestRunNotFoundError

Bases: RuntimeError

For when we try to find an existing test, but it doesn’t exist.

pavilion.test_run.basic_attr(name, doc)

Create a basic attribute for the TestAttribute class. This will produce a property object with a getter and setter that simply retrieve or set the value in the self._attrs dictionary.

pavilion.test_run.get_latest_tests(pav_cfg, limit)

Returns ID’s of latest test given a limit

Parameters:
  • pav_cfg – Pavilion config file
  • limit (int) – maximum size of list of test ID’s
Returns:

list of test ID’s
Return type:

list(int)
pavilion.test_run.test_run_attr_transform(path)

A dir_db transformer to convert a test_run path into a dict of test attributes.