Test Configuration

Test File Format

Pavilion Test configurations, like the base Pavilion configuration, utilize the YamlConfig library to define the config structure. Because of the dynamic nature of test configs, there are a few extra complications this module handles that are documented below.

class pavilion.test_config.file_format.TestConfigLoader

Bases: YamlConfigLoader

This class describes a test section in a Pavilion config file. It is expected to be added to by various plugins.

ELEMENTS = [<yaml_config StrElem name>, <yaml_config StrElem cfg_label>, <yaml_config StrElem suite>, <yaml_config StrElem suite_path>, <yaml_config StrElem working_dir>, <yaml_config StrElem os>, <yaml_config StrElem host>, <yaml_config ListElem modes>, <yaml_config ListElem overrides>, <yaml_config RegexElem inherits_from>, <yaml_config StrElem subtitle>, <yaml_config StrElem group>, <yaml_config StrElem umask>, <yaml_config KeyedElem maintainer>, <yaml_config StrElem summary>, <yaml_config StrElem doc>, <yaml_config ListElem permute_on>, <yaml_config StrElem permute_base>, <yaml_config VarCatElem variables>, <yaml_config RegexElem scheduler>, <yaml_config StrElem sched_data_id>, <yaml_config StrElem chunk>, <yaml_config CondCategoryElem only_if>, <yaml_config CondCategoryElem not_if>, <yaml_config StrElem compatible_pav_versions>, <yaml_config StrElem test_version>, <yaml_config KeyedElem spack>, <yaml_config KeyedElem build>, <yaml_config KeyedElem run>, <yaml_config EvalCategoryElem result_evaluate>, <yaml_config ModuleWrapperCatElem module_wrappers>, <yaml_config KeyedElem result_parse>]

Each YamlConfig instance in this list defines a key for the test config.

  • Each element must result in a string (which is why you see a lot of StrElem below), or a structure that contains only strings at the lowest layer.

    • So lists of dicts of strings are fine, etc.

    • yc.RegexElem also produces a string.

  • Everything should have a sensible default.

    • An empty config should be a valid test.

  • For bool values, accept [‘true’, ‘false’, ‘True’, ‘False’].

    • They should be checked with val.lower() == ‘true’, etc.

  • Every element must have a useful ‘help_text’.

SCHEDULE_CLASS

alias of ScheduleConfig

classmethod add_result_parser_config(name, config_items)

Add the given list of config items as a result parser configuration named ‘name’. Throws errors for invalid configurations.

classmethod check_leaves(elem)

Make sure all of the config elements have a string element or equivalent as the final node.

Parameters

elem (yc.ConfigElement) –

classmethod remove_result_parser_config(name)

Remove the given result parser from the result parser configuration section.

Parameters

name (str) – The name of the parser to remove.

class pavilion.test_config.file_format.CondCategoryElem(name=None, sub_elem=None, defaults=None, allow_empty_keys=False, key_case='mixed', **kwargs)

Bases: CategoryElem

Allow any key. They’ll be validated later.

class pavilion.test_config.file_format.EnvCatElem(name=None, sub_elem=None, defaults=None, allow_empty_keys=False, key_case='mixed', **kwargs)

Bases: CategoryElem

A category element that ensures environment variables retain their order.

type

alias of OrderedDict

class pavilion.test_config.file_format.EvalCategoryElem(name=None, sub_elem=None, defaults=None, allow_empty_keys=False, key_case='mixed', **kwargs)

Bases: CategoryElem

Allow keys that start with underscore. Lowercase only.

class pavilion.test_config.file_format.ModuleWrapperCatElem(name=None, sub_elem=None, defaults=None, allow_empty_keys=False, key_case='mixed', **kwargs)

Bases: CategoryElem

Allow glob wildcards in key names.

type

alias of OrderedDict

class pavilion.test_config.file_format.PathCategoryElem(name=None, sub_elem=None, defaults=None, allow_empty_keys=False, key_case='mixed', **kwargs)

Bases: CategoryElem

This is for category elements that need a valid unix path regex.

class pavilion.test_config.file_format.ResultParserCatElem(name=None, sub_elem=None, defaults=None, allow_empty_keys=False, key_case='mixed', **kwargs)

Bases: CategoryElem

Result parser category element.

class pavilion.test_config.file_format.SubVarDict(*args, defaults=None, **kwargs)

Bases: dict

Subclass for variable sub-items, mainly to track defaults properly.

flatten() → dict

Return as a regular dictionary with all available values set.

is_defaults_only() → bool

Returns true if there isn’t any regular data, just defaults.

items()

Return all values from both self and defaults.

keys()

Return all keys from both ourselves and the defaults as keys.

super_keys()
values()

Return all values across both self and the defaults.

class pavilion.test_config.file_format.TestCatElem(name=None, sub_elem=None, defaults=None, allow_empty_keys=False, key_case='mixed', **kwargs)

Bases: CategoryElem

A category element that ensures order of Keyed Elems retain order.

type

alias of OrderedDict

pavilion.test_config.file_format.TestSuiteLoader()

Create a new test suite loader instance. This is a function masquerading as a constructor because the class has to be defined dynamically after plugins have modified the test config.

class pavilion.test_config.file_format.VarCatElem(name=None, sub_elem=None, defaults=None, allow_empty_keys=False, key_case='mixed', **kwargs)

Bases: CategoryElem

For describing how the variables’ section itself works.

Just like a regular category elem (any conforming key, but values must be the same type), but with some special magic when merging values.

Variables

_NAME_RE – Unlike normal categoryElem keys, these can have dashes.

merge(old, new)

Merge, but allow for special keys that change our merge behavior.

‘key?: value’

Allows values from lower levels in the config stack to override this one. The value is only used if no other value is given.

‘key+: value/s’

The values are appended to the list of whatever is given by lower levels of the config stack.

validate(value, partial=False)

Make sure default value propagate through validation.

class pavilion.test_config.file_format.VarKeyCategoryElem(name=None, sub_elem=None, defaults=None, allow_empty_keys=False, key_case='mixed', **kwargs)

Bases: CategoryElem

Allow Pavilion variable name like keys.

class pavilion.test_config.file_format.VariableElem(name=None, **kwargs)

Bases: CategoryElem

This is for values in the ‘variables’ section of a test config.

A variable entry can be either a single string value or an arbitrary dictionary of strings. If given a single string value, a dictionary of ‘{None: value}’ will be returned.

normalize(value, root_name='root')

Normalize to either a dict of strings or just a string.

type

alias of SubVarDict

validate(value, partial=False)

Check for a single item and return it, otherwise return a dict.

Test Setup