Workflow Builder

Build Jobmon workflows from workflow configuration.

vivarium.cluster_tools.dagger.config.builder.STEP_TYPE_API_FNS: dict[str, Callable[..., list['Task']]] = {'bash': <function get_bash_step_tasks>, 'notebook': <function get_notebook_step_tasks>, 'pytest': <function get_pytest_step_tasks>, 'python': <function get_python_step_tasks>, 'simulation': <function get_simulation_step_tasks>}

Maps each YAML step_type to the API function that builds its tasks. Paired with the dispatch tables in vivarium.cluster_tools.dagger.config.parsing.

vivarium.cluster_tools.dagger.config.builder.build_workflow_from_config(config, workflow_args, *, resume=False)[source]

Build a complete Jobmon workflow from a workflow configuration.

For each step in the workflow, dispatches to the matching interface API function (one of get_*_step_tasks) and wires dependencies so that steps execute in sequential order (all tasks from step N must complete before any task in step N+1 starts).

Return type:

Workflow

Parameters:

  • config (WorkflowConfig) – The validated workflow configuration to build.

  • workflow_args (str) – Deterministic string that Jobmon uses to identify the workflow. Must be identical across runs for resume to work.

  • resume (bool) – Whether this build is a resume (dagger restart). Forwarded only to the simulation step, the one step type whose output layout varies on resume.