Skip to content

Backend Acceptance Tutorial

This guide documents the manual backend acceptance lane for COFOLDER maintainers.

Use it when you need fresh-environment validation of the real CLI workflows against a real backend, especially before promoting backend-, runner-, or CLI-adjacent changes toward main.

This lane is intentionally expensive:

  • it is not part of routine pytest
  • it is not part of CI-default checks
  • it should be run deliberately when backend behavior or runner integration has changed

When To Run This Lane

Run the backend acceptance notebooks before promoting changes that affect:

  • runner implementations under src/cofolder/modules/runners/
  • shared CLI behavior in src/cofolder/cli.py
  • workflow behavior in src/cofolder/recipes/validate.py, screen.py, or oracle.py
  • packaging or dependency behavior for boltz1, boltz2, boltz-community, or the acceptance extra
  • OpenFold3 installation guidance, runner wiring, or confidence-only workflow behavior

If you are adding or integrating a new runner, pair this guide with Adding New Runners.

Python Version

Use a fresh conda environment with:

  • Python >=3.11
  • Python 3.11 or 3.12 strongly recommended for this acceptance lane
  • do not use Python 3.13 for boltz1 or boltz2 acceptance environments in this project, because those extras are currently constrained to <3.13 in pyproject.toml

Clean Install Rules

Treat each backend as a separate environment:

  • create one new conda environment per backend
  • install exactly one backend extra per environment
  • do not reuse an existing development environment
  • do not install boltz1, boltz2, and boltz-community into the same environment
  • if you accidentally mixed backend installs, remove that conda environment and start over with a new one

The commands below assume you are in the repository root and want the current checkout installed in editable mode.

Full Clean Installs

boltz1

conda create -n cofolder-acceptance-boltz1 python=3.12
conda activate cofolder-acceptance-boltz1
python -m pip install -e ".[acceptance,analysis,boltz1]"
python --version
cofolder --help

If Python 3.12 is not available in your conda setup, use Python 3.11 instead.

boltz2

conda create -n cofolder-acceptance-boltz2 python=3.12
conda activate cofolder-acceptance-boltz2
python -m pip install -e ".[acceptance,analysis,boltz2]"
python --version
cofolder --help

boltz-community

conda create -n cofolder-acceptance-boltz-community python=3.12
conda activate cofolder-acceptance-boltz-community
python -m pip install -e ".[acceptance,analysis,boltz-community]"
python --version
cofolder --help

openfold3

OpenFold3 now installs through a COFOLDER optional extra. Use a fresh environment, install the acceptance tooling plus the backend extra, then run the repo setup script before launching the expensive acceptance cells:

conda create -n cofolder-acceptance-openfold3 python=3.12
python -m pip install -e ".[acceptance,analysis,openfold3]"
export OPENFOLD_CACHE="$PWD/cache/.openfold3-cache"
cofolder-tools setup-openfold3
python --version
cofolder --help

Notes:

  • OpenFold3 upstream currently recommends pixi for reproducible environments, but the upstream openfold3 pip package is also documented and is what the COFOLDER openfold3 extra installs.
  • cofolder-tools setup-openfold3 wraps upstream setup_openfold and prepares the cache, model parameters, and CCD before you run the acceptance workflows. The command above places those downloads in the repository's gitignored cache/.openfold3-cache directory; without an explicit OPENFOLD_CACHE, the tool retains the upstream ~/.openfold3 default.
  • The tool also removes the current upstream checkpoint-choice ambiguity by answering the setup prompts explicitly: it uses OPENFOLD_CACHE for both path questions and selects parameter download choice 1 by default. It answers the force-redownload prompt with no; use OPENFOLD3_FORCE_DOWNLOAD_PARAMETERS=yes cofolder-tools setup-openfold3 only when you intentionally want to replace an existing checkpoint. Use OPENFOLD3_PARAMETER_CHOICE=2 cofolder-tools setup-openfold3 if you want all published checkpoints instead.
  • The tool answers the upstream integration-test prompt with no by default. Use OPENFOLD3_RUN_INTEGRATION_TESTS=yes cofolder-tools setup-openfold3 only when you intentionally want those upstream tests to run during setup.
  • Upstream docs note that first inference can also download default model parameters to $HOME/.openfold3, but for manual acceptance this project prefers the explicit setup script so environment readiness is checked before the expensive lane starts.
  • This first-pass acceptance lane is confidence-only. It does not claim affinity support for OpenFold3.

Input-contract acceptance lane

The packaged acceptance inputs also include system_nucleic_acid.yaml plus runner-specific constrained systems. In each clean backend environment, run validate against the nucleic-acid fixture and the matching constrained fixture from src/cofolder/acceptance/data/. Confirm that the long-form metrics.csv contains chain_id values A, D, R, and L in that order. The Boltz-1 fixture uses its single 6 Å pocket form, Boltz-2/community exercise pocket and contact inputs, and the OpenFold3 fixture exercises pocket translation. These runs are opt-in because they execute the real prediction backends.

Launch the shared lane with:

marimo run src/cofolder/acceptance/input_contract_backend_acceptance.py

If you prefer a non-editable install from the current checkout, replace:

  • python -m pip install -e ".[acceptance,analysis,boltz1]"
  • python -m pip install -e ".[acceptance,analysis,boltz2]"
  • python -m pip install -e ".[acceptance,analysis,boltz-community]"
  • python -m pip install -e ".[acceptance,analysis,openfold3]"

with:

  • python -m pip install ".[acceptance,analysis,boltz1]"
  • python -m pip install ".[acceptance,analysis,boltz2]"
  • python -m pip install ".[acceptance,analysis,boltz-community]"
  • python -m pip install ".[acceptance,analysis,openfold3]"

To discard an acceptance environment completely:

conda deactivate
conda env remove -n cofolder-acceptance-boltz1

Swap the environment name as needed for boltz2, boltz-community, or openfold3.

Launching The Notebooks

Recommended launch pattern from inside the matching clean environment:

marimo edit "$(python -c 'import cofolder.acceptance.boltz2_backend_acceptance as nb; print(nb.__file__)')"

Swap the module name for:

  • cofolder.acceptance.boltz1_backend_acceptance
  • cofolder.acceptance.boltz_community_backend_acceptance
  • cofolder.acceptance.openfold3_backend_acceptance

Each notebook opens safely by default:

  • the expensive validate, screen, and oracle steps are gated behind explicit checkboxes
  • each subprocess run is bound to the notebook workspace so relative cache paths stay isolated
  • the packaged options file is rewritten per run so the backend cache points into that run's temporary workspace
  • command output streams live to the notebook console while the command runs

For openfold3, the notebook uses a dedicated OpenFold3 options fixture and now requires an explicit cache path. Set OPENFOLD_CACHE before launching marimo, or enter the cache path in the notebook setup field before enabling any expensive workflow cell. The notebook will block validate, screen, and oracle until the selected cache path is setup-ready.

Changing OPENFOLD_CACHE in your shell after marimo has already started does not update the running notebook session; restart marimo or use the notebook cache-path override field.

What The Notebooks Exercise

Each backend notebook runs the command-line workflows that matter for shared behavior:

  • cofolder validate
  • cofolder screen
  • cofolder oracle

The notebooks call the real CLI in a clean environment instead of importing recipes directly, which makes them suitable as a pre-merge acceptance lane for runner and workflow changes.

For openfold3, the positive path is:

  • validate with confidence_metrics
  • screen with confidence_metrics
  • oracle with a real OpenFold3-produced confidence metric such as sample_ranking_score

Expected Pass/Fail Semantics

  • boltz2 and boltz-community: validate, screen, and oracle should produce populated affinity-related outputs.
  • boltz1: validate and screen should warn that affinity scoring groups are unsupported while leaving downstream affinity-related columns present-but-empty; oracle should use confidence_score as the supported positive-path metric.
  • openfold3: validate, screen, and oracle should produce populated confidence outputs, including OpenFold3-native confidence fields. This lane does not request affinity groups, and it must not imply that structural-confidence outputs are affinity proxies.

The Boltz2 screen cell additionally requests SASA and distance IFPs, evaluates a custom pocket reference, enables non-destructive filtering and IFP clustering, and runs bias diagnostics against packaged acceptance references. It checks populated manuscript-facing confidence, affinity, SASA, IFP, protein/ligand proximity, pocket coverage, and filter columns and requires ifp_cluster_summary.csv. It also verifies that exactly one fixed-protein MSA generation attempt populated the shared manifest and that every ligand row YAML uses that same staged artifact.

Keep scientific wording disciplined when reviewing these outputs:

  • validation metrics, confidence metrics, and affinity metrics are distinct concepts
  • structural confidence should not be described as a proxy for binding affinity

Optional Bias Setup

Bias-related setup is outside the first-pass lanes except for Boltz2, whose Screen acceptance cell uses small packaged protein and ligand references to verify populated proximity outputs.

If you need --assess_bias, install mmseqs2 in the same environment and follow the bias setup documented in Installation. Keep that work separate from the base backend acceptance pass unless the change under review specifically affects bias behavior.