Status Files

Status Files

Every test run has a status file that tracks it’s progress. It’s one of the first things created (after the test directory itself) when creating a test object.

A test run will transition through several states (all available as part of the status_file.STATES object). These, along with a description and timestamp, are saved as a ‘state’ in the status file. Each state is a single line of the file with a max size of 4096 bytes to ensure atomic writes.

The state of a test run represents where that run is in its lifecycle. It does not represent whether a test passed or failed. States are ephemeral and asynchronous, and should generally not be used to decide to do something with a test run. (The only exception is the ‘SCHEDULED’ state, which tells Pavilion to ask the scheduler about its current state).

Usage:

status_file = StatusFile('/tmp/mystatus')

status.set(STATES.RUNNING, "I'm running!")

state = status.current()

state.note
class pavilion.status_file.SeriesStatesStruct

Bases: StatesStruct

States for series objects.

ABORTED = "Aborted, through no fault of it's own."
ALL_STARTED = 'All tests that will be started, have been.'
BUILD_ERROR = 'Error building tests for a test set.'
CANCELED = 'Series was canceled by the user.'
COMPLETE = 'For when the test is completely complete.'
CREATED = 'The series object/directory is being created.'
CREATION_ERROR = 'The test object/directory could not be created.'
ERROR = 'General (fatal) error status.'
KICKOFF_ERROR = 'Error kicking off tests.'
RUN = 'Running the series.'
SET_BUILD = 'For when test sets are building.'
SET_CANCELED = 'Canceling a test set.'
SET_CREATED = 'For when test sets are created.'
SET_KICKOFF = 'For when test sets are being kicked off.'
SET_MAKE = 'For when test runs are created in the test set.'
SET_SKIPPED = 'Note that a whole test set was skipped.'
TESTS_SKIPPED = 'For logging when tests are skipped.'
class pavilion.status_file.SeriesStatusFile(path: Optional[Path])

Bases: TestStatusFile

A status file for series.

info_class

alias of SeriesStatusInfo

states = <pavilion.status_file.SeriesStatesStruct object>
class pavilion.status_file.SeriesStatusInfo(state: str, note: str, when: Optional[float] = None)

Bases: TestStatusInfo

A status info object, except for series.

states_obj = <pavilion.status_file.SeriesStatesStruct object>
class pavilion.status_file.StatesStruct

Bases: object

A class containing the valid state constants. This is meant to be inherited from and the states added as FOO=”bar” class variables.

Rules:

  • The value should be an ascii string of the constant name.

  • The constants have a max length of 15 characters.

  • The constants are in all caps.

  • The constants must be a valid python identifier that starts with a letter.

  • Error states should end in ‘_ERROR’. They should be the result of an OS level problem (like missing directories), or problems with Pavilion itself.

  • Failure states should end in ‘_FAILED’. They should be the result of trying something, and it just not succeeding.

Note: The states are written in the class as <state_name> = <help_text>, however, on class init the help text is stored separately, and the state value is set to the name of the state itself. So STATES.ENV_FAILED will have a value of ‘ENV_FAILED’ when used.

Known States:

INVALID = 'The status given to set was invalid.'
STATUS_CREATED = 'The status file has been created.'
STATUS_ERROR = 'An error with the status file itself.'
UNKNOWN = "We can't determine the status."
WARNING = 'General warning (non-fatal error) status information.'
get(key)

Get the value of the status key.

help(state)

Get the help string for a state.

list()

List all the known state names.

max_length = 15
validate(key)

Make sure the key conforms to the above rules.

exception pavilion.status_file.StatusError

Bases: RuntimeError

Error raised by any status file related problems.

class pavilion.status_file.TestStatesStruct

Bases: StatesStruct

Struct containing pre-defined test states.

ABORTED = "Aborted, through no fault of it's own."
BUILDING = 'The test is currently being built.'
BUILD_COPIED = 'The build has completed the copy step.'
BUILD_CREATED = 'The builder for this build was created.'
BUILD_DEFERRED = 'The build will occur on nodes.'
BUILD_DONE = 'The build is complete and in place.'
BUILD_ERROR = 'An unexpected error occurred while setting up the build.'
BUILD_FAILED = 'The build has failed.'
BUILD_REUSED = 'The build was reused from a prior step.'
BUILD_SUCCESS = 'The build script was run successfully.'
BUILD_TIMEOUT = 'The build has timed out.'
BUILD_WAIT = 'Waiting for the build lock.'
CANCELLED = 'The test was cancelled.'
COMPLETE = 'For when the test is completely complete.'
CREATED = 'The test object/directory is being created.'
CREATION_ERROR = 'The test object/directory could not be created.'
ENV_FAILED = 'Unable to load the environment requested by the test.'
INFO = 'This is for logging information about a test, and can occurwithin any state.'
PREPPING_RUN = 'Performing final (on node) steps before the test run.'
RESULTS = "For when we're getting the results."
RESULTS_ERROR = 'A result parser raised an error.'
RUNNING = "For when we're currently running the test."
RUN_DONE = 'For when the run step is complete.'
RUN_ERROR = 'An unexpected error has occurred when setting up the test run.'
RUN_TIMEOUT = 'The test run went long without any output.'
RUN_USER = 'Jobs can report extra status using pav set_status and this status value.'
SCHEDULED = 'The test has been scheduled with a scheduler.'
SCHED_CANCELLED = 'Cancelled by the scheduler or external to Pavilion.'
SCHED_ERROR = 'There was a scheduler related error.'
SCHED_RUNNING = 'DEPRECATED - Should be converted to SCHED_STARTUP'
SCHED_STARTUP = 'The scheduler job has started, but not the test.'
SKIPPED = 'The test has been skipped due to an invalid condition.'
class pavilion.status_file.TestStatusFile(path: Optional[Path])

Bases: object

The wraps the status file that is used in each test, and manages the creation, reading, and modification of that file.

NOTE: The status file does not perform any locking to ensure that it’s created in an atomic manner. It does, however, limit it’s writes to appends of a size such that those writes should be atomic.

add_status(status: TestStatusInfo) TestStatusInfo

Add the status object as a status for the test.

current() TestStatusInfo

Return the most recent status object.

has_state(state) bool

Check if the given state is somewhere in the history of this status file.

history() List[TestStatusInfo]

Return a list of all statuses recorded.

info_class

alias of TestStatusInfo

set(state: str, note: str) TestStatusInfo
Set the status and return the StatusInfo object. Well return a

‘STATUS_ERROR’ status on write failures.

Parameters
  • state – The current state.

  • note – A note about this particular instance of the state.

states = <pavilion.status_file.TestStatesStruct object>
class pavilion.status_file.TestStatusInfo(state: str, note: str, when: Optional[float] = None)

Bases: object

Represents a single status.

Variables
  • state (str) – A state string (from STATES or SERIES_STATES).

  • note (str) – The note for this status update.

  • when (datetime) – A datetime object representing when this state was saved.

LINE_MAX = 4096
MAX_TS = 253402218061.12344
NOTE_MAX = 4052
TIME_FORMAT = '%Y-%m-%dT%H:%M:%S.%f'
TS_FORMAT = '{:0.6f}'
TS_LEN = 26
as_dict()

Convert to a dictionary.

Return type

dict

states_obj = <pavilion.status_file.TestStatesStruct object>
status_line()

Convert this to a line of text as it would be written to the status file.