Expression Function Plugins
Expression Function plugins are what provide the implementation for all functions in Pavilion string expressions.
mytest:
run:
env:
TASKS: "{{ max([5, sched.min_ppn]) }}"
The max()
function takes a list of numbers, and returns the largest. That
value will be assigned to the TASKS
environment variable.
In this tutorial, we’ll show you how to add a new function to Pavilion for use in your tests’ expressions.
Contents
Needed Files
Expression Function plugins are just like every other Pavilion plugin. See Plugin Basics for instructions on setting up the basic files.
The Plugin Base Class
Your plugin module file will need to contain the class definition for your plugin.
from pavilion import expression_functions
# The 'num' function will accept any numerical looking type generically.
from pavilion.expression_functions import num
# All function plugins inherit from the 'FunctionPlugin' class
class Max(expression_functions.FunctionPlugin):
# As with other plugin types, we override __init__ to provide the
# basic information on our plugin.
def __init__(self):
super().__init__(
# The name of our plugin and function
name="max",
# The short description shown when listing these plugins.
description="Get the max of a list of numbers",
# The arg_specs define how to auto-convert arguments to the
# appropriate types. More on that below.
# Note: (foo,) is a single item tuple containing foo.
arg_specs=([num],)
)
# This method is the 'function' this plugin defines. It should take
# arguments as defined by the arg_spec. It should also return one of the
# types understood by Pavilion expressions (int, float, bool, string, or
# lists/dicts containing only those types).
@staticmethod
def max(nums):
"""The docstring of the function will serve as its long
documentation."""
# We don't need to do any type checking, those conversions
# will already be done for us (and will raise the appropriate
# errors).
return max(nums)
Arg Specs
The arg_specs
tell Pavilion what types to expect for each argument of the
function, and how to handle type autoconversion. This is necessary because we
will often have numbers as strings in variable values that will need to be
converted to a numerical type.
Basic Types
Items in the arg_specs
can be a type conversion function to auto-convert
a value into the type needed by the function. This typically means using one of
float()
, str()
, int()
:
# This list of arg_specs denote that the function should expect
# three arguments: a float, a string, and an int.
arg_specs=(float, str, int)
The Num Type
If your function can take any numerical value, use
the num
function as we did above in our Max
. This will convert the
given value to an int, float or bool, according to what the input (or input
string) most closely resembles. It also handles ‘True’ and ‘False’ strings as
boolean values:
>>> from pavilion.expression_functions import num
>>> num("7")
7
>>> num("7.0")
7.0
>>> num("False")
False
Depending on the function, you may also want to take care to maintain and return the original type.
Lists and Dicts
Function plugin arguments can also be any structure of lists and dicts as long as the final contained values are one of the basic types listed above.
For lists, simply give a list with the expected type as the only item.:
# The function expects two arguments, a list of ints and a list of strings.
arg_specs=([int], [str])
Similarly for dicts, include the expected keys in a dictionary and the expected type functions as the values. Only keys listed will be visible to the function.:
# The function expects a dict with 'host' (str) and 'speed' (float) keys.
arg_specs=({'host': str, 'speed': float}, )
More usefully, you can combine lists and dicts.:
# The function expects a list of host/speed dicts
arg_specs([{'host': str, 'speed': float}],)
Overriding Arg Specs
Not all functions fit the mold of what we can do with arg specs. When this
happens you may want to override the arg specs entirely. To do this,
set arg_specs
to None
. You then have to override the _validate_args
method of your plugin class, to provide your own validation and type
conversion.:
class LenPlugin(CoreFunctionPlugin):
"""Return the length of the given item, where item can be a string,
list, or dict."""
def __init__(self):
"""Setup plugin"""
super().__init__(
name='len',
description='Return the integer length of the given str, int or '
'mapping/dict.',
arg_specs=None,
)
def _validate_arg(self, arg, spec):
if not isinstance(arg, (list, str, dict)):
raise FunctionPluginError(
"The list_len function only accepts lists, dicts, and "
"strings. Got {} of type {}.".format(arg, type(arg).__name__)
)
return arg
@staticmethod
def len(arg):
"""Just return the length of the argument."""
return len(arg)
The Plugin Function
As mentioned above, the plugin must define a method that takes the expected
arguments. In our example, we used a @static_method
, but that isn’t
necessary. You may also use a regular or class method, or even assign a function
to the class directly.:
class Min(expression_functions.FunctionPlugin):
def __init__(self):
super().__init__(
name='min',
description='Minimum value of a list',
arg_spec=([num],)
)
# Just use the built-in min function. Note that the function doc string
# will be the long form documentation for the plugin, so make sure
# it is appropriate.
min = min
Core Plugins
Pavilion provides several built-in ‘core’ expression functions, but not using
the normal plugin mechanism. They’re located in
expression_functions/core.py
. If you would like to add your function to
Pavilion’s core list, simply place the plugin class in that module, and make
sure it inherits from CoreFunctionPlugin
. A .yapsy-plugin
file isn’t
needed in this case.:
class log(expression_plugins.CoreFunctionPlugin):
def __init__(self):
super().__init__(
name=log,
description="Take the log given the number and base."
arg_specs=(num, num))
log = math.log