Variable Components

Variable Inner Workings (test_config.variables)

This module contains functions and classes for building variable sets for string insertion.

There are three layers to every variable:

  • A list of variable values
  • A dictionary of sub-keys
  • The values of those sub keys. ie: [{key:value},…]

From the user perspective, however, all but the value itself is optional.

While variables are stored in this manner, these layers are automatically resolved in the trivial cases such as when there is only one element, or a single value instead of a set of key/value pairs.

There are expected to be multiple variable sets: plain variables (var), plugin provided via sys_vars (sys), core pavilion provided (pav), and scheduler provided (sched).

class pavilion.test_config.variables.DeferredVariable

The value for some variables may not be available until a test is actually running. Deferred variables act as a placeholder in such circumstances, and output an escape sequence when converted to a str.

__init__

Initialize self. See help(type(self)) for accurate signature.

get(index, sub_var)

Deferred variables should never have their value retrieved.

class pavilion.test_config.variables.VariableSetManager

Bases: object

This class manages the various sets of variables, provides complex key based lookups, manages conflict resolution, and so on. Anything that works with pavilion variables should do so through an instance of this class.

Usage:

var_man = VariableSetManager()
# pav_vars and sys_vars should be dictionary like objects
var_man.add_var_set('sys', sys_vars)
var_man.add_var_set('pav', pav_vars)

var_man['sys.sys_name']
var_man['sys_name']
VAR_SETS = ('var', 'sys', 'pav', 'sched')
add_var_set(name, value_dict)

Add a new variable set to this variable set manager. Variables in the set can then be retrieved by complex key.

Parameters:
  • name (str) – The name of the var set. Must be one of the reserved keys.
  • value_dict (Union(dict,collections.UserDict)) – A dictionary of values to populate the var set.
Returns:

None
Raises:

VariableError – On problems with the name or data.
any_deferred(key: Union[str, tuple]) → bool

Return whether any members of the given variable are deferred.

as_dict()

Return the all variable sets as a single dictionary. This will be structured as the config data is expected to be received. (A dict of variables where the values are lists of either structs or string values).

Return type:dict
get(key: str, default=None) → str

Get an item, or the provided default, as per dict.get.

get_permutations(used_per_vars)

For every combination of permutation variables (that were used), return a new var_set manager that contains only a single value (possibly a complex one) for each permutation var, in every possible permutation.

Parameters:str)] used_per_vars (list[(str,) – A set of permutation variable names that were used, as a tuple of (var_set, var_name).
Returns:A list of permuted variable managers.
Return type:VariableSetManager
is_deferred(key)

Return whether the given variable is deferred. Fully specified variables (with idx and sub_var set) may be deferred specifically or in general at various levels.

Return type:bool
static key_as_dotted(key)

Turn a tuple based key reference back into a dotted string.

len(var_set, var)

Get the length of the given key.

Parameters:
  • var_set (str) – The var set to fetch from.
  • var (str) – The variable to fetch.
Return type:

int
Returns:

The number of items in the found ‘var_set.var’.
Raises:

KeyError – When the key has problems, or can’t be found.
classmethod load(path)

Load a saved variable set.

Parameters:path (pathlib.Path) – The variable file to load.
classmethod parse_key(key)

Parse the given complex key, and return a reasonable (var_set, var, index, sub_var) tuple.

Parameters:

key (Union(str,list,tuple)) – A 1-4 part key. These may either be given as a list/tuple of strings, or dot separated in a string. The components are var_set, var, index, and sub_var. Var is required, the rest are optional. Index is expected to be an integer, and var_set is expected to be a key category.
Raises:
  • KeyError – For bad keys.
  • TypeError – When the key isn’t a string.
Returns:

A (var_set, var, index, sub_var) tuple. Any component except var may be None.
resolve_key(key)

Resolve the given key using this known var sets. Unlike parse_key, the var_set returned will never be None, as the key must correspond to a found variable in a var_set. In case of conflicts, the var_set will be resolved in order.

Parameters:key (Union(str,list,tuple)) – A 1-4 part key. These may either be given as a list/tuple of strings, or dot separated in a string. The components are var_set, var, index, and sub_var. Var is required, the rest are optional. Index is expected to be an integer, and var_set is expected to be a key category.
Raises:KeyError – For bad keys, and when the var_set can’t be found.
Returns:A tuple of (var_set, var, index, sub_var), index and sub_var may be None.
resolve_references()

Resolve all variable references that are within variable values defined in the ‘variables’ section of the test config.

Raises:
  • TestConfigError – When reference loops are found.
  • KeyError – When an unknown variable is referenced.
save(path)

Save the variable set to the given stream as JSON.

Parameters:path (pathlib.Path) – The file path to write to.
set_deferred(var_set, var, idx=None, sub_var=None)

Set the given variable as deferred. Variables may be deferred as a whole, or as individual list or sub_var items.

Parameters:
  • var_set (str) – The var_set of the deferred var.
  • var (str) – The variable name.
  • None) idx (Union(int,) – The idx of the deferred var. If set to None, the variable is deferred for all indexes. Note that single valued variables have an index of zero.
  • None) sub_var (Union(str,) – The sub_variable name that is deferred.
undefer(new_vars)

Get non-deferred values for all the deferred values in this variable set, leaving the non-deferred values intact.

Parameters:new_vars (VariableSetManager) – A completely non-deferred variable set manager.
class pavilion.test_config.variables.VariableSet(name, value_dict=None)

Bases: object

A set of of variables. Essentially a wrapper around a mapping of var names to VariableList objects.

get(var, index, sub_var)

Return the value of the var given the var name, index, and sub_var name.

set_value(var, index, sub_var, value)

Set the value at the given location to value.

class pavilion.test_config.variables.VariableList(values=None)

Bases: object

Wraps a list of SubVariable objects. Even variables with a single value end up as a list (of one).

get(index, sub_var)

Return the variable value at the given index and sub_var.

set_value(index, sub_var, value)

Set the value at the given location to value.

class pavilion.test_config.variables.SubVariable(value_pairs=None)

Bases: object

The final variable tier. Variables with no sub-var end up with a dict with a single None: value pair.

get(sub_var)

Gets the actual variable value.

set_value(sub_var, value)

Set the value at the given location to value.

exception pavilion.test_config.variables.VariableError(message, var_set=None, var=None, index=None, sub_var=None)

Bases: ValueError

This error should be thrown when processing variable data, and something goes wrong.

Variable Dict Class (var_dict)

This provides a dictionary class that can register functions to dynamically provide <function_name>:<return value> key:value pairs. The functions are lazily executed, and the results are cached.

class pavilion.var_dict.VarDict(name)

Bases: collections.UserDict

A dictionary for defining dynamic variables in Pavilion.

Usage: To add a variable, create a method and decorate it with either @var_method or @dfr_var_method(). The method name will be the variable name, and the method will be called to resolve the variable value. Methods that start with ‘_’ are ignored.

get(key, default=None)

As per the dict class.

info(key)

Get an info dictionary about the given key.

items()

As per the dict class.

keys()

As per the dict class.

values()

As per the dict class.

pavilion.var_dict.normalize_value(value, level=0)

Normalize a value to one compatible with Pavilion variables. This means it must be a dict of strings, a list of strings, a list of dicts of strings, or just a string. Returns None on failure. :param value: The value to normalize. :param level: Controls what structures are allowed as this is called recursively.

pavilion.var_dict.var_method(func)

This decorator marks the given function as a scheduler variable. The function must take no arguments (other than self).

Pavilion Defined Variables (pav_vars)

This module provides the ‘pav’ variables, hardcoded into a VarDict object.

class pavilion.pavilion_variables.PavVars

Bases: pavilion.var_dict.VarDict

The pavilion provided variables. Note that these values are generated once, then reused.

day()

The current day of the month.

month()

The current month.

time()

An ‘HH:MM:SS.usec’ timestamp.

timestamp()

The current unix timestamp.

user()

The current user’s login name.

version()

The current version of Pavilion.

weekday()

The current weekday.

year()

The current year.

System Variable Plugins

System Variables provide a way for pavilion users to add additional variables for Pavilion tests to use. In particular, these are useful for gathering site-specific information for your tests.

class pavilion.system_variables.SysVarDict(defer=False, unique=False)

Bases: collections.UserDict

This dictionary based object provides lazy, cached lookups of all system variable values according to what system variable plugins are actually loaded. The values, once retrieved, are thus static for a given run of the pavilion command.

classmethod get_obj(name)

Return the corresponding object without invoking the .get method.

static help(key)

Return help information for the given key.

items()

As per dict.items()

keys()

As per dict.keys() (except we’re really listing the loaded plugins.)

values()

As per dict.values()

class pavilion.system_variables.SystemPlugin(name, description, priority=10, is_deferable=False, sub_keys=None)

Bases: yapsy.IPlugin.IPlugin

Each system variable plugin provides a key and value for the system variables dictionary. These are only evaluated if asked for, and generally only once.

NAME_VERS_RE = re.compile('^[a-zA-Z0-9_.-]+$')
PRIO_COMMON = 10
PRIO_CORE = 0
PRIO_USER = 20
activate()

Add this plugin to the system plugin list.

deactivate()

Remove this plugin from the system plugin list.

get(defer)

Get the value for this system variable.

Params bool defer:
 If the variable is deferable, return a DeferredVariable object instead.
exception pavilion.system_variables.SystemPluginError

Bases: RuntimeError

Error thrown when a system plugin encounters an error.

pavilion.system_variables.get_vars(defer)

Get the dictionary of system plugins.

Parameters:defer (bool) – Whether the deferable plugins should be deferred.
Return type:SysVarDict