Pavilion Test Run Objects

pavilion.pav_test module

Contains the TestRun class, as well as functions for getting the list of all known test runs.

class pavilion.test_run.TestRun(pav_cfg, config, build_tracker=None, var_man=None, _id=None, **options)

The central pavilion test object. Handle saving, monitoring and running tests.

Test LifeCycle 1. Test Object is Created – TestRun.__init__

  1. Test id and directory (working_dir/test_runs/0000001) are created.
  2. Most test information files (config, status, etc) are created.
  3. Build script is created.
  4. Build hash is generated.
  5. Run script dry run generation is performed.
  1. Test is built. – test.build()
  2. Test is finalized. – test.finalize()
    1. Variables and config go through final resolution.
    2. Final run script is generated.
  3. Test is run. – test.run()
  4. Results are gathered. – test.gather_results()
Variables:
  • id (int) – The test id.
  • config (dict) – The test’s configuration.
  • test.path (Path) – The path to the test’s test_run directory.
  • results (dict) – The test results. Set None if results haven’t been gathered.
  • builder (TestBuilder) – The test builder object, with information on the test’s build.
  • build_origin_path (Path) – The path to the symlink to the original build directory. For bookkeeping.
  • status (StatusFile) – The status object for this test.
  • opt (TestRunOptions) – Test run options defined by OPTIONS_DEFAULTS
  • OPTIONS_DEFAULTS – A dictionary of defaults for additional options for the test run. Values given to Pavilion are expected to be the same type as the default value.
__init__(pav_cfg, config, build_tracker=None, var_man=None, _id=None, **options)
Create an new TestRun object. If loading an existing test
instance, use the TestRun.from_id() method.
Parameters:
  • pav_cfg – The pavilion configuration.
  • config (dict) – The test configuration dictionary.
  • build_tracker (builder.MultiBuildTracker) – Tracker for watching and managing the status of multiple builds.
  • var_man (variables.VariableSetManager) – The variable set manager for this test.
  • build_only (bool) – Only build this test run, do not run it.
  • rebuild (bool) – After determining the build name, deprecate it and select a new, non-deprecated build.
  • _id (int) – The test id of an existing test. (You should be using TestRun.load).
build(cancel_event=None)

Build the test using its builder object and symlink copy it to it’s final location. The build tracker will have the latest information on any encountered errors.

Parameters:cancel_event (threading.Event) – Event to tell builds when to die.
Returns:True if build successful
gather_results(run_result)

Process and log the results of the test, including the default set of result keys.

Default Result Keys:

name
The name of the test
id
The test id
created
When the test was created.
started
When the test was started.
finished
When the test finished running (or failed).
duration
Length of the test run.
user
The user who ran the test.
sys_name
The system (cluster) on which the test ran.
job_id
The job id set by the scheduler.
result
Defaults to PASS if the test completed (with a zero exit status). Is generally expected to be overridden by other result parsers.
Parameters:run_result (bool) – The result of the run.
is_built

Whether the build for this test exists.

Returns:True if the build exists (or the test doesn’t have a build), False otherwise.
Return type:bool
job_id

The job id of this test (saved to a jobid file). This should be set by the scheduler plugin as soon as it’s known.

classmethod load(pav_cfg, test_id)

Load an old TestRun object given a test id.

Parameters:
  • pav_cfg – The pavilion config
  • test_id (int) – The test’s id number.
Return type:

TestRun
load_results()

Load results from the results file.

:returns A dict of results, or None if the results file doesn’t exist. :rtype: dict

run()

Run the test.

Return type:

bool
Returns:

True if the test completed and returned zero, false otherwise.
Raises:
  • TimeoutError – When the run times out.
  • TestRunError – We don’t actually raise this, but might in the future.
save_results(results)

Save the results to the results file.

Parameters:results (dict) – The results dictionary.
set_run_complete()

Write a file in the test directory that indicates that the test has completed a run, one way or another. This should only be called when we’re sure their won’t be any more status changes.

timestamp

Return the unix timestamp for this test, based on the last modified date for the test directory.

wait(timeout=None)

Wait for the test run to be complete. This works across hosts, as it simply checks for files in the run directory.

Parameters:timeout (Union(None,float)) – How long to wait in seconds. If this is None, wait forever.
Raises:TimeoutError – if the timeout expires.
exception pavilion.test_run.TestRunError

Bases: RuntimeError

For general test errors. Whatever was being attempted has failed in a non-recoverable way.

exception pavilion.test_run.TestRunNotFoundError

Bases: RuntimeError

For when we try to find an existing test, but it doesn’t exist.

class pavilion.test_run.TestRunOptions(build_only=False, rebuild=False, **_)

Bases: object

A ‘struct’ of options for test runs, to keep minor options fairly well contained.

KNOWN_OPTIONS = ['build_only', 'rebuild']
OPTIONS_FN = 'options'
as_dict()

Returns the options as a dictionary.

classmethod load(test)

Load the options from file.

Return type:TestRunOptions
save(test)

Create a file that records the options for this test run.

Parameters:test (TestRun) – The TestRun to save this under.
pavilion.test_run.get_latest_tests(pav_cfg, limit)

Returns ID’s of latest test given a limit

Parameters:
  • pav_cfg – Pavilion config file
  • limit (int) – maximum size of list of test ID’s
Returns:

list of test ID’s
Return type:

list(int)