Pavilion2

Pavilion is a Python 3 (3.4+) based framework for running and analyzing tests targeting HPC systems. It provides a rich YAML-based configuration system for wrapping test codes and running them against various systems. The vast majority of the system is defined via plugins, giving users the ability to extend and modify Pavilion’s operation to suit their needs. Plugin components include those for gathering system data, adding additional schedulers, parsing test results, and more.

Project goals:

• Robust testing in postDST, automated, and acceptance testing (system validation) scenarios.
• End-to-end status tracking for all tests.
• Simple, powerful test configuration language.
• System agnostic test configs.
• Hide common platform and environment idiosyncrasies from tests.
• System specific defaults.
• Eliminate unnecessary build repetition.
• Extreme extensibility (plugins everywhere).

Contents:

• Getting Started
  • Setup
  • Configure Tests
  • Running tests
  • Test Series and ID’s
  • Test Results
• Installing Pavilion
  • Requirements
  • Recommended Directory Structure
  • Install
  • Dependencies
• Environment Modules
• Pavilion Advanced Usage
  • Mode Configs
  • Advanced Test Configs
  • Environment
  • Schedulers
• Configuring Pavilion
  • Config Directories
  • Pavilion.yaml
• Test Format
  • Tests and Suites
  • Test Formatting and Structure
  • Hidden Tests
  • Host Configs
  • Mode Configs
  • Order of Resolution
  • Top Level Test Config Keys
• Pavilion Test Variables
  • Variable Sets
  • Variable Types
  • Test Variables
  • Deferred Variables
  • Substrings
  • Default Values
  • Permutations
• Running Tests
  • Run Configuration
  • Test Run LifeCycle
• Building Tests
  • Build Config Keys
  • Building
  • Create a Build Script
• Build and Run Environments
  • Assumptions
  • Environment Variables
  • Modules
  • Module Wrappers
• Test Results
  • Command Result Parser (cmd)
  • Regex Result Parser (regex)
  • Constant Result Parser (const)
  • Table Result Parser
  • Basic Result Keys
  • Using Result Parsers
  • Keys
  • Errors
  • Included result parsers
• Pavilion Plugins
  • Plugin Basics
  • Pavilion Command Plugins
  • Module Wrapper Plugins
  • Pavilion Result Parser Plugins
  • System Variables

Tutorials

• Writing Your First Test
  • Preliminary Steps
  • Where to Write Your Tests
  • Writing a Test
  • Getting the Test to Run
  • Test Results
  • Conclusion
• Writing a Unit Test
  • What You Need To Test
  • Unit Test Files
  • Using the PavTestCase class
  • Using Pavilion Components in Tests
  • Testing Commands

API Documentation

• API Documentation
  • Command Arguments and Base Configuration
  • Pavilion Builder Objects
  • Command Plugins
  • Miscellaneous Modules
  • Module Wrapper Plugins
  • Utilities
  • Result Parsers
  • Schedulers
  • Test Series
  • Status Files
  • Test Configuration
  • Pavilion Test Run Objects
  • UnitTests
  • Utilities
  • Variable Components
  • Index by Module

For Developers:

• Pavilion Development Guidelines
  • Style
  • Publishing
  • Site Specific Code