Miscellaneous Modules¶
Various Pavilion support code modules.
Table of Contents
LockFiles¶
Pavilion uses lock files to handle concurrency across multiple nodes and systems. It has to assume the file-system that these are written to has atomic, O_EXCL file creation.
-
class
pavilion.lockfile.
LockFile
(lockfile_path, group=None, timeout=None, expires_after=18000)¶ Bases:
object
An NFS friendly way to create a lock file. Locks contain information on what host and user created the lock, and have a built in expiration date. To be used in a ‘with’ context.
Variables: - DEFAULT_EXPIRE (int) – Time till file is considered stale, in seconds. (5 minute default)
- SLEEP_PERIOD (int) – How long to sleep between lock attempts. This shouldn’t be any less than 0.01 or so on a regular filesystem. 0.2 is pretty reasonable for an nfs filesystem and sporadically used locks.
- LOCK_PERMS (int) – Default lock permissions
-
DEFAULT_EXPIRE
= 18000¶
-
LOCK_PERMS
= 508¶
-
SLEEP_PERIOD
= 0.2¶
-
lock
()¶ Try to create and lock the lockfile.
-
read_lockfile
()¶ Returns the components of the lockfile content, or None for each of these values if there was an error..
Returns: host, user, expiration (as a float), id
-
unlock
()¶ Delete the lockfile, thereby releasing the lock.
Raises: RuntimeError – When we can’t delete our own lockfile for some reason.
Script Composition¶
The script composer makes it easy to build up a script with a prescribed environ in a programmatic way.
It also handles translating our module specifications into specific actions to add to the script.
-
class
pavilion.scriptcomposer.
ScriptComposer
(header=None, details=None)¶ Bases:
object
Manages the building of bash scripts for Pavilion.
-
command
(command)¶ Add a line unadulterated to the script lines.
Parameters: command (str) – String representing the whole command to add.
-
comment
(comment)¶ Function for adding a comment to the script.
Parameters: comment (str) – Text to be put in comment without the leading ‘# ‘.
-
env_change
(env_dict)¶ Function to take the environment variable change requested by the user and add the appropriate line to the script.
Parameters: env_dict (dict) – A dictionary (preferably an OrderedDict) of environment keys and values to set. A value of None will unset the variable.
-
module_change
(module, sys_vars)¶ Take the module changes specified in the user config and add the appropriate lines to the script. This will parse the module name into various actions, find the appropriate module_wrapper plugin, and use that to get the lines to add to the script.
Parameters: - module (str) – Name of a module or a list thereof in the format used in the user config.
- sys_vars (dict) – The pavilion system variable dictionary.
-
newline
()¶ Function that just adds a newline to the script lines.
-
static
parse_module
(mod_line)¶ Parse a module specification into it’s components. These can come in one of three formats:
- ‘mod-name[/version]’ - Load the given module name and version
- ‘-mod-name[/version]’ - Unload the given module/version.
- ‘old_name[/old_vers]->mod-name[/version]’ - Swap the given old module for the new one.
Parameters: mod_line (str) – String provided by the user in the config. Return type: (str, (str, str), (str, str)) Returns: action, (name, vers), (old_name, old_vers)
-
write
()¶ Function to write the script out to file.
Return bool result: Returns either True for successfully writing the file or False otherwise.
-
-
exception
pavilion.scriptcomposer.
ScriptComposerError
¶ Bases:
RuntimeError
Class level exception during script composition.
-
class
pavilion.scriptcomposer.
ScriptDetails
(path=None, group=None, perms=None)¶ Bases:
object
Class to contain the final details of the script.
-
group
¶
-
path
¶
-
perms
¶
-
-
class
pavilion.scriptcomposer.
ScriptHeader
(shell_path=None, scheduler_headers=None)¶ Bases:
object
Class to serve as a struct for the script header.
-
get_lines
()¶ Function to retrieve a list of lines for the script header.
-
reset
()¶ Function to reset the values of the internal variables back to None.
-
scheduler_headers
¶ Function to return the list of scheduler header lines.
-
shell_path
¶ Function to return the value of the internal shell path variable.
-
Wget¶
-
exception
pavilion.wget.
WGetError
¶ Bases:
RuntimeError
-
pavilion.wget.
get
(pav_cfg, url, dest)¶ Download the file at the given url and store it at dest. If a file already exists at dest it will be overwritten (assuming we have the permissions to do so). Proxies are handled automatically based on pav_cfg settings. This is done atomically; the download is saved to an intermediate location and then moved. :param pav_cfg: The pavilion configuration object. :param str url: The url for the file to download. :param Path dest: The path to where the file will be stored.
-
pavilion.wget.
head
(pav_cfg, url)¶ Get the header information for the given url. :param pav_cfg: The pavilion configuration object :param str url: The url we need information on. :returns: The http headers for the given url. :rtype dict:
-
pavilion.wget.
missing_libs
()¶ You should call this before using the wget module functions, to ensure all the dependencies are available. :returns: A list of one or more missing libraries. It won’t necessarily catch them all in one pass. An empty list is good.
-
pavilion.wget.
update
(pav_cfg, url, dest)¶ Check if the file needs to be re-downloaded, and do so if necessary. This will careate a ‘{dest}.info’ file in the same directory that will be used to check if updates are necessary. :param pav_cfg: The pavilion configuration object. :param str url: The url for the file to download. :param Path dest: The path to where we want to store the file.