Writing a Series File

A test series is a group of interrelated test runs. While tests started together via pav run are automatically grouped together as a simple series, more complex relationships are possible by explicitly defining a series with a test series config file. A series file allows the user to specify test hierarchies, assign modes to specific tests, and dictate other settings for the group of tests. Creating a test series in this way also gives the user the option of limiting the number of tests running or scheduled at any given time and running the tests in a continuous manner.

Where to Write Series Files

Series configs are yaml files and are placed in the <pav config dir>/series directory. Like all the other config files in Pavilion, the config file name must follow the format <series name>.yaml.

Series Configuration

Here’s an example series file. Let’s call it sanity_tests.yaml and include tests that are normally run just to make sure the machine works.

# This is the body of the series file. This is where we define test sets,
# which are groups of tests that we want to apply the same configurations to.
test_sets:

  # We'll call this test set 'front_end_tests' and run all the tests that
  # don't run on compute nodes with this set.
  front_end_tests:

    # Names of the tests that will run as part of this test set.
    # The ordering of these tests (relative to each other) is generally up to
    # the scheduler.
    tests:
      - check_mounts
      - ping_compute_nodes
      - check_commands.front_end

    # List modes that need to be applied to this set of tests
    modes:
      - fe_settings
      - maintenance_testing

    # Define conditions. These will be added to the only_if/not_if conditions
    # that already exist in each test, if any.

    # Only run this test if it's being run on the machine 'blue_cluster'
    only_if:
      "{{sys.sys_name}}": "blue_cluster"

    # Don't run this set if the week day is Friday
    not_if:
      "{{pav.weekday}}": "Friday"

  individual_node_tests:

    # This group of tests depends on the previous set ('front_end_tests').
    # Because 'depends_pass' is set to 'True', this set requires that all the
    # tests in the set this set depends on must PASS in order for this set
    # to be run. The default is that 'depends_pass' is 'False', which means
    # the tests in the set this set depends on need only complete (regardless
    # of whether or not they PASS) in order for this set to run.
    depends_on: ['front_end_tests']
    depends_pass: True

    tests:
      - stream.per_node

  mp_tests:

    # The 'mpi8.yaml' mode file will be applied to each of the tests in
    # this group.
    modes: ['mpi8']

    depends_on: ['front_end_tests']
    depends_pass: True

    tests:
      - stream.mpi
      - supermagic.basic

# if 'ordered' is set to 'True', then Pavilion will automatically run this
# series such that each test set depends on the set before that.
ordered: False

# The total number of tests that will be run or scheduled at any given time.
# default: no limit
simultaneous: 3

# series-level modes: These modes will be applied to all sets and therefore
# all tests
modes: ['all_partitions', 'all_nodes']

# Run this entire series just once.
repeat: 1

Running, Monitoring, and Cancelling the Series

Run pav show series to show the list of possible series to run.

To run a series file, the user must use the command pav series run <series_name>. To run the series above, we would run pav series run sanity_tests. This will output instructions on how to check the status of and kill the series.

$ pav series run sanity_tests
Started series s19. Run `pav status s19` to view status. PGID is 9102.
To kill, use `pav cancel s19` or `kill -15 -9102`.