Installing Pavilion

Installing Pavilion is mostly a matter of placing it’s source somewhere, providing its (few) dependencies, and creating a pavilion.yaml config file.

Requirements

Pavilion has very few dependencies and requirements:

  • Python 3.6 or newer

  • A writeable space on a filesystem shared across all hosts (to be tested) in each cluster. (Assuming you’re scheduling jobs across a cluster).

    • The path to this directory must be consistent across all cluster hosts.

    • It must support atomic file creation and appends of < 4kb.

  • Lmod or ‘environment modules’ is recommended.

Filesystems

Pavilion works by recursively running itself in different modes at different points in the testing process. This means certain paths, like the Pavilion source directory, working directory, and used config directories must have paths that are consistent across the nodes and front-ends of any given system.

Pavilion places all builds, test working spaces, and lockfiles in a working directory specified in the pavilion configuration (defaults to ~/.pavilion/).

  • Atomic (O_EXCL) file creation is needed here for the creation of lock files.

  • Atomic small appends are needed for writing to the status file. Not having this has a small chance of resulting in corrupted test status files.

  • Both of these requirements are probably already satisfied by one or more of your cluster NFS partitions. Lustre filesystems are not recommended, mostly due to the type of load Pavilion presents to these.

Testing Filesystems

If you’re unsure if your shared filesystem is reliable, there’s a test for that in test/utils.

$ python3 lock_test.py --help

Result Log

The result log can be configured to write to an arbitrary filesystem. That filesystem should be shared and have consistent paths as well, as the log is written as the final on-node step in tests.

Install

Pavilion installs are meant to be dropped into place as a complete directory that contains the source and any dependencies missing on your system. This generally starts with a git pull of the latest release of Pavilion.

git clone https://github.com/hpc/pavilion2.git
cd pavilion2
# You can also, optionally, check out a specific release version.
git checkout tags/v2.4

You can also simply download and extract the source.

Releases

You should probably pick the latest Pavilion release when installing Pavilion for a couple reasons.

  1. While we try to maintain backwards compatibility as much as possible, the reality is that every release contains several major compatibility breaks both for test configurations and plugins. These are documented per-release in the RELEASE.txt file.

  2. We run a large bevy of unit tests against every change in Pavilion, but each release is used in production before it is actually tagged. This often reveals bugs, regressions, and practical usage issues. We fix those issues, then tag the release. Bugfix releases are provided as needed.

Dependencies

Pavilion has a few dependencies, and most aren’t required. Pavilion was designed and tested against fairly recent (as of 2022-01) versions of these, but it’s likely that older, system provided versions may work just as well. Conversely, the latest version should be fine as well. The supported and tests versions for each are recorded in requirements.txt.

  • yaml_config (required) - Used to define the test and pavilion configurations.

  • yapsy (required) - The basis for Pavilion’s plugin architecture.

  • lark (required) - Used for Pavilion string value and expression parsing.

  • matplotlib - Needed for the graph command.

  • requests - Used for automatic downloads of test source files. This feature is disabled in the absence of this library, and tests that use it will fail with an error. The remaining dependencies are needed by requests. The following are sub-dependencies of requests.

Installing Dependencies

Dependencies auto-install when you run the pav command for the first time. If you’ve checked out Pavilion as a git repo, it will even auto-update dependencies if they change.

Environment Modules

Pavilion uses the module command to load modules for tests. It will work with either lmod or the tcl based ‘environment modules’ systems. This is generally only needed if your cluster/s have a complex software environment that supports multiple compilers and conflicting builds of libraries.

It is assumed that the module environment is set up before you run Pavilion. If you need to set up this environment separately on allocations, use the ‘env_setup’ option in the Configuring Pavilion to add the commands to do so.