psimulate CLI

Command line interface for psimulate.

psimulate

A command line utility for running many simulations in parallel.

You may initiate a new run with the run sub-command or restart a run from where it was stopped by using the restart sub-command.

Usage

psimulate [OPTIONS] COMMAND [ARGS]...

expand

Expand a previous run.

This expands a previous run at RESULTS_ROOT by adding input draws and/or random seeds. Expanding will not erase existing results, but will start workers to perform the additional simulations determined by the added draws/seeds. RESULTS_ROOT is expected to be an output directory from a previous psimulate run invocation.

Usage

psimulate expand [OPTIONS] [RESULTS_ROOT_POS]

Options

-c, --run-config <run_config>

Path to a YAML configuration file. Keys use the same snake_case names as CLI parameters (e.g., peak_memory, max_runtime, result_directory). Values in this file serve as defaults and are overridden by any argument provided on the command line.

-R, --results-root <results_root_opt>

Path to results directory from a previous psimulate run.

--add-draws <add_draws>

The number of input draws to add to a previous run.

Default:

0

--add-seeds <add_seeds>

The number of random seeds to add to a previous run.

Default:

0

-s, --sim-verbosity <sim_verbosity>

Logging verbosity level of each individual simulation.

Default:

'0'

Options:

0 | 1 | 2

--pdb

Drop into python debugger if an error occurs.

-v

Configure logging verbosity of main runner for a parallel simulation.

--backup-freq <backup_freq>

Interval in minutes between saving backups. Set to ‘None’ or ‘none’ to disable backups.

Default:

'30'

--max-attempts <max_attempts>

Maximum number of attempts Jobmon will make for each task before marking it as failed.

Default:

3

-w, --max-workers <max_workers>

The maximum number of workers (and therefore jobs) to run concurrently.

Default:

8000

-h, --hardware <hardware>

The (comma-separated) specific hardware to run the jobs on. This can be useful to request specifically fast nodes (‘-h r650xs’) vs high capacity nodes (‘-h r630,r650,r650v2’). Note that the hardware changes on a roughly annual schedule. The currently-supported options are: [‘c6320’, ‘r630’, ‘c6420v1’, ‘c6420v2’, ‘r650’, ‘r650v2’, ‘r650xs’]. For details, refer to: https://docs.cluster.ihme.washington.edu/#hpc-execution-host-hardware-specifications

-m, --peak-memory <peak_memory>

The memory request in GB of each individual simulation job. The simulations will be killed if they exceed this limit.

Default:

3

-r, --max-runtime <max_runtime>

The runtime request (hh:mm:ss) of each individual simulation job. The maximum supported runtime is 3 days. Keep in mind that the session you are launching from must be able to live at least as long as the simulation jobs and that runtimes by node vary wildly.

Default:

'24:00:00'

-q, --queue <queue>

The cluster queue to assign psimulate jobs to. Queue defaults to the appropriate queue based on max-runtime. long.q allows for much longer runtimes but there may be reasons to send jobs to that queue even if they don’t have runtime constraints, such as node availability.

Options:

all.q | long.q

-P, --project <project>

Required The cluster project under which to run the simulation.

Options:

proj_simscience | proj_simscience_prod | proj_csu

Arguments

RESULTS_ROOT_POS

Optional argument

restart

Restart a parallel simulation.

This restarts a parallel simulation from a previous run at RESULTS_ROOT. Restarting will not erase existing results, but will start workers to perform the remaining simulations. RESULTS_ROOT is expected to be an output directory from a previous psimulate run invocation.

Usage

psimulate restart [OPTIONS] [RESULTS_ROOT_POS]

Options

-c, --run-config <run_config>

Path to a YAML configuration file. Keys use the same snake_case names as CLI parameters (e.g., peak_memory, max_runtime, result_directory). Values in this file serve as defaults and are overridden by any argument provided on the command line.

-R, --results-root <results_root_opt>

Path to results directory from a previous psimulate run.

-s, --sim-verbosity <sim_verbosity>

Logging verbosity level of each individual simulation.

Default:

'0'

Options:

0 | 1 | 2

--pdb

Drop into python debugger if an error occurs.

-v

Configure logging verbosity of main runner for a parallel simulation.

--backup-freq <backup_freq>

Interval in minutes between saving backups. Set to ‘None’ or ‘none’ to disable backups.

Default:

'30'

--max-attempts <max_attempts>

Maximum number of attempts Jobmon will make for each task before marking it as failed.

Default:

3

-w, --max-workers <max_workers>

The maximum number of workers (and therefore jobs) to run concurrently.

Default:

8000

-h, --hardware <hardware>

The (comma-separated) specific hardware to run the jobs on. This can be useful to request specifically fast nodes (‘-h r650xs’) vs high capacity nodes (‘-h r630,r650,r650v2’). Note that the hardware changes on a roughly annual schedule. The currently-supported options are: [‘c6320’, ‘r630’, ‘c6420v1’, ‘c6420v2’, ‘r650’, ‘r650v2’, ‘r650xs’]. For details, refer to: https://docs.cluster.ihme.washington.edu/#hpc-execution-host-hardware-specifications

-m, --peak-memory <peak_memory>

The memory request in GB of each individual simulation job. The simulations will be killed if they exceed this limit.

Default:

3

-r, --max-runtime <max_runtime>

The runtime request (hh:mm:ss) of each individual simulation job. The maximum supported runtime is 3 days. Keep in mind that the session you are launching from must be able to live at least as long as the simulation jobs and that runtimes by node vary wildly.

Default:

'24:00:00'

-q, --queue <queue>

The cluster queue to assign psimulate jobs to. Queue defaults to the appropriate queue based on max-runtime. long.q allows for much longer runtimes but there may be reasons to send jobs to that queue even if they don’t have runtime constraints, such as node availability.

Options:

all.q | long.q

-P, --project <project>

Required The cluster project under which to run the simulation.

Options:

proj_simscience | proj_simscience_prod | proj_csu

Arguments

RESULTS_ROOT_POS

Optional argument

run

Run a parallel simulation.

The simulation itself is defined by a MODEL_SPECIFICATION yaml file and the parameter changes across runs are defined by a BRANCH_CONFIGURATION yaml file.

The path to the data artifact can be provided as an argument here, in the branch configuration, or in the model specification file. Values provided as a command line argument or in the branch specification file will override a value specified in the model specifications file. If an artifact path is provided both as a command line argument and to the branch configuration file a ConfigurationError will be thrown.

Within the provided or default results directory, a subdirectory will be created with the same name as the MODEL_SPECIFICATION if one does not exist. Results will be written to a further subdirectory named after the start time of the simulation run.

Usage

psimulate run [OPTIONS] [MODEL_SPECIFICATION_POS] [BRANCH_CONFIGURATION_POS]

Options

-c, --run-config <run_config>

Path to a YAML configuration file. Keys use the same snake_case names as CLI parameters (e.g., peak_memory, max_runtime, result_directory). Values in this file serve as defaults and are overridden by any argument provided on the command line.

-M, --model-specification <model_specification_opt>

Path to the model specification yaml file.

-B, --branch-configuration <branch_configuration_opt>

Path to the branch configuration yaml file.

-i, --artifact_path <artifact_path>

The path to the artifact data file.

-o, --result-directory <result_directory>

Required The directory to write results to. A folder will be created in this directory with the same name as the configuration file.

-s, --sim-verbosity <sim_verbosity>

Logging verbosity level of each individual simulation.

Default:

'0'

Options:

0 | 1 | 2

--pdb

Drop into python debugger if an error occurs.

-v

Configure logging verbosity of main runner for a parallel simulation.

--backup-freq <backup_freq>

Interval in minutes between saving backups. Set to ‘None’ or ‘none’ to disable backups.

Default:

'30'

--max-attempts <max_attempts>

Maximum number of attempts Jobmon will make for each task before marking it as failed.

Default:

3

-w, --max-workers <max_workers>

The maximum number of workers (and therefore jobs) to run concurrently.

Default:

8000

-h, --hardware <hardware>

The (comma-separated) specific hardware to run the jobs on. This can be useful to request specifically fast nodes (‘-h r650xs’) vs high capacity nodes (‘-h r630,r650,r650v2’). Note that the hardware changes on a roughly annual schedule. The currently-supported options are: [‘c6320’, ‘r630’, ‘c6420v1’, ‘c6420v2’, ‘r650’, ‘r650v2’, ‘r650xs’]. For details, refer to: https://docs.cluster.ihme.washington.edu/#hpc-execution-host-hardware-specifications

-m, --peak-memory <peak_memory>

The memory request in GB of each individual simulation job. The simulations will be killed if they exceed this limit.

Default:

3

-r, --max-runtime <max_runtime>

The runtime request (hh:mm:ss) of each individual simulation job. The maximum supported runtime is 3 days. Keep in mind that the session you are launching from must be able to live at least as long as the simulation jobs and that runtimes by node vary wildly.

Default:

'24:00:00'

-q, --queue <queue>

The cluster queue to assign psimulate jobs to. Queue defaults to the appropriate queue based on max-runtime. long.q allows for much longer runtimes but there may be reasons to send jobs to that queue even if they don’t have runtime constraints, such as node availability.

Options:

all.q | long.q

-P, --project <project>

Required The cluster project under which to run the simulation.

Options:

proj_simscience | proj_simscience_prod | proj_csu

Arguments

MODEL_SPECIFICATION_POS

Optional argument

BRANCH_CONFIGURATION_POS

Optional argument

test

Usage

psimulate test [OPTIONS] {sleep|large_results}

Options

-n, --num-workers <num_workers>

Default:

1000

-o, --result-directory <result_directory>

Default:

'/mnt/team/simulation_science/priv/engineering/load_tests'

-s, --sim-verbosity <sim_verbosity>

Logging verbosity level of each individual simulation.

Default:

'0'

Options:

0 | 1 | 2

--pdb

Drop into python debugger if an error occurs.

-v

Configure logging verbosity of main runner for a parallel simulation.

--backup-freq <backup_freq>

Interval in minutes between saving backups. Set to ‘None’ or ‘none’ to disable backups.

Default:

'30'

--max-attempts <max_attempts>

Maximum number of attempts Jobmon will make for each task before marking it as failed.

Default:

3

-w, --max-workers <max_workers>

The maximum number of workers (and therefore jobs) to run concurrently.

Default:

8000

-h, --hardware <hardware>

The (comma-separated) specific hardware to run the jobs on. This can be useful to request specifically fast nodes (‘-h r650xs’) vs high capacity nodes (‘-h r630,r650,r650v2’). Note that the hardware changes on a roughly annual schedule. The currently-supported options are: [‘c6320’, ‘r630’, ‘c6420v1’, ‘c6420v2’, ‘r650’, ‘r650v2’, ‘r650xs’]. For details, refer to: https://docs.cluster.ihme.washington.edu/#hpc-execution-host-hardware-specifications

-m, --peak-memory <peak_memory>

The memory request in GB of each individual simulation job. The simulations will be killed if they exceed this limit.

Default:

3

-r, --max-runtime <max_runtime>

The runtime request (hh:mm:ss) of each individual simulation job. The maximum supported runtime is 3 days. Keep in mind that the session you are launching from must be able to live at least as long as the simulation jobs and that runtimes by node vary wildly.

Default:

'24:00:00'

-q, --queue <queue>

The cluster queue to assign psimulate jobs to. Queue defaults to the appropriate queue based on max-runtime. long.q allows for much longer runtimes but there may be reasons to send jobs to that queue even if they don’t have runtime constraints, such as node availability.

Options:

all.q | long.q

-P, --project <project>

Required The cluster project under which to run the simulation.

Options:

proj_simscience | proj_simscience_prod | proj_csu

Arguments

TEST_TYPE

Required argument