Methods, validation and citation
MaveScape reimplements published methods for scoring deep mutational scans. This page shows how its results are checked against the tools those methods come from and against experiments whose answers are known, how to cite it, and whose work it builds on.
Validation
Unit tests check each function against hand-computed and published values. The validation suite runs the complete pipelines, the way they run in the app, and reports how closely MaveScape agrees with the reference, against a stated tolerance. It runs in continuous integration on every change, with the public data it needs fetched and checked against their SHA-256 checksums.
| Area | Checked against | Result |
|---|---|---|
| Variant names | mavehgvs 0.8.1 on 16,959 strings: its own tests, the feasibility data, generated variants and single-character edits | The same decision, reason and canonical form for every string |
| Import | The three feasibility tables from MaveDB and DiMSum's demo, shuffled, split and part-read, and a table with one problem of each kind | Every name valid against its target; missing never read as 0; each problem reported by line |
| Designs | Hand-written designs of three public experiments of different kinds, sample sheets and DiMSum's design file | Every column accounted for; the editor and the sample sheets rebuild each design |
| Scores | Enrich2 2.0.2 on GRB2, BRCA1 and a fixture with every edge case of the product requirements planted, all three normalizations | Within 5 × 10⁻¹³ |
| Scores | dms_variants 1.6.0, func_scores, on the fixture | Within 4.6 × 10⁻¹³ |
| Time series | Enrich2 2.0.2's WLS and OLS on BRCA1's E2 (six rounds) and Y2H (four unevenly spaced times) assays and a time-series fixture with its edge cases planted | Within 5 × 10⁻¹³ (32,000 replicate scores, each with its SE) |
| Time series | statsmodels 0.15 WLS and OLS on the fixture: slopes, residual-scaled SEs, the SE from counting alone (numpy) and the departure from a line | Within 5 × 10⁻¹⁴ |
| Time series per generation | The time-series fixture scored per unit of time, by WLS, OLS and the ratio of the ends | The whole-course scores and SEs over the time span, within 5 × 10⁻¹⁶ |
| Time-series SEs | The fixture's true effects | 95% intervals hold them 90% of the time combined by REML, against 81% with Enrich2's residual-scaled SE; 93% by the moderated combination |
| Sorted bins | Factor IX's published MultiSTEP scores (MaveDB urn:mavedb:00001200-a-1): 29,325 replicate scores in 9 replicates of 3 tiles, and the combined scores | Every replicate score reproduced from the counts within 1.3 × 10⁻¹⁵ (on the variants the authors kept); combined scores and SEs exactly |
| Sorted bins | fitdistrplus 1.2.6 fitdistcens on a simulated sort: the censored log-normal, σ free and σ the wild type's | μ and σ within 3 × 10⁻⁷ over 4,800 fits; SE of μ within 0.07% |
| Sorted bins | The simulated sort's true shifts | Maximum likelihood r = 0.996, weighted average 0.989; bootstrap SEs equal the analytic ones (median ratio 1.003) |
| Barcodes | dms_variants 1.6.0, func_scores by barcode and by aa_substitutions, on a simulated library of 4,608 barcodes in two libraries, read with MaveScape's map, in dms_variants' own variant_counts and in Enrich2's layout | Every barcode's score and variance, and every variant's summed counts, score and variance, within 4.8 × 10⁻¹³; the same scores all three ways |
| Barcodes | The simulated library's true effects, and the barcodes planted 1.5–3 off their variant | Scored by barcode and combined by REML r = 0.990 (summed 0.977), 95% intervals hold 94%; 76% of the planted outliers found, 0.05% of the others called |
| DiMSum's model | DiMSum 1.4's own R functions on GRB2, DiMSum's demo (TDP-43, four replicates) and a fixture with zero counts and dropouts: the input threshold, the variants fitted, the scales and shifts (nlm), the error model (nls) and its 100 bootstrap fits, and every fitness, σ and merged score | Fitness, σ and merged scores within 8 × 10⁻¹³ given DiMSum's parameters; the same threshold and variants; scales and shifts within 4.5 × 10⁻⁷; the error model, fitted exactly, within 4.3 × 10⁻⁶ of DiMSum's fit on every variant and inside its percentiles |
| DiMSum's model | GRB2's published scores, and the true effects of simulated experiments with a bottleneck (25 cells per variant, three seeds) | r = 0.994; 95% intervals hold the truth 93–96% of the time (counting alone 67–69%, REML 87–88%); the moderated combination, from counting alone, 95–97% |
| Differential scores | mutscan 1.2.0 calculateRelativeFC (limma 3.68.5) on a two-condition fixture (shared inputs, an unpaired replicate, missing and zero counts planted) and on CBS at two vitamin B6 levels (MaveDB urn:mavedb:00000005-a-5 and -a-6: four shared inputs, 9,409 variants, relative to the synonymous variants) | Log₂ fold changes and t within 2 × 10⁻¹⁰, SEs, p and adjusted p within 10⁻¹²; the same rows and prior |
| Differential scores | Enrich2 2.0.2's z between conditions (calc_pvalues_pairwise) and each condition's combined scores, on the fixture | Within 5 × 10⁻¹³ |
| Differential scores | Simulated shared inputs and a bottleneck, three seeds; and with noise between replicates | 95% intervals: paired 93–97%, limma 98–99%, independent 100% (too wide). With noise: limma 95–99% and at most 0.4% of unchanged variants called; paired, combined by the moderated combination, 88–99% (mean 95%) and at most 2.7% (by REML: 82–94% and 1–11%) |
| Replicates | metafor 5.2-1 REML on 2,865 variants and six synthetic sets | Within 3.4 × 10⁻¹² (required 10⁻⁶) |
| Coverage of intervals | 40 simulated experiments of each of 26 kinds with known effects: 30 to 2,000 reads per variant, two to six replicates, bottlenecks, selection noise, shared inputs, overdispersed reads, time series, sorted bins, barcodes, DiMSum's fitness, rescaled and differential scores | With MaveScape's defaults, 95% intervals hold the truth 93.6–96.5% of the time in every kind (required 93–97%); REML's 81.8–95.5%. A time course that is not a line, scored by its slope, 79%: QC flags it in 39 of 40 experiments, and its ratio of ends holds 95% |
| Coverage of intervals | dms_variants 1.6.0's simulated barcoded libraries (an independent simulator): three selections sharing one input, with counting noise alone, a bottleneck, and noise | 94.1%, 97.6% and 96.2% for variants with 5 reads or more in every sample (required 92–98%); REML 83–86% |
| Calibration | Each replicate of GRB2, BRCA1's E2 assay, factor IX and CBS held out and predicted from the others | Beyond ±1.96 predicted SDs (5% if calibrated): GRB2 6.1%, CBS 5.6%, factor IX 10.0%, BRCA1 12.3% (its libraries differ in selection strength); REML 22.6%, 3.5%, 26.3%, 17.7% |
| Determinism | The same inputs in another row or column order, and in Chrome and Node | The same bits |
| Scale | A simulated table of a million barcodes and its million-line map, read in 16 MB parts as the window reads them; 105,000 variants × 6 samples | Imported in 3.5 s, the process at most 830 MB, the table then holding 102 MB (in the window, in Chrome: 3–4 s, the browser up by about 600 MB); scored in 1.3 s (barcodes) and 0.5 s (variants). Budgets 15 s, 1 GB and 10 s, checked on every change |
| Quality control | Simulated experiments with one problem each (poor agreement, a failing replicate, a bottleneck, a low-count tail, a missing sample, a time series with a bottleneck at every passage; barcodes that disagree, outlier barcodes, a map missing barcodes, one barcode per variant), three seeds | Exactly their findings, and none for a clean experiment; the variance ratio follows a simulated bottleneck within 30%, and DiMSum's terms place it before or after selection |
| The cells recorded | Simulated bottlenecks with the cells recorded as made (20 and 100 cells per variant into selection, and 50 recovered after it), with noise between replicates, and with a tenth of the cells recorded; three seeds | The recorded cells account for the excess in every pair; with noise, the replicates show more in 8 of 9 pairs; with a tenth recorded, less in every pair |
| The map | A golden SVG; each state where the fixture plants it; BRCA1's RING domain | Byte for byte; missing, filtered and low-confidence cells at least ΔE 10 from neutral; the zinc ligands least tolerant |
| The record | Workspaces with runs, selections and changed thresholds, saved as archives, reopened and saved again | The same bytes; every export identical; tampered and hostile archives caught |
| Findings in context | Every finding the planted problems raise; a selection that enriches loss of function (inputs and outputs swapped), three seeds; a library of exactly the single-base substitutions; BRCA1's yeast two-hybrid assay, whose late stops keep binding | Each says what could cause it and what to do; with the direction stated the controls separate, without it separation fails and says why; coverage judged against what the library can make; the position where stops stop losing the function found (93) and, the controls limited to it, separation passes |
| What each analysis can do | Every example and fixture (12), whole and with one part taken away (the wild-type row, a replicate, the gates, the cells, the times in generations, time points, what the assay measures, the nonsense or synonymous controls): 74 workspaces | The readiness model names exactly what was taken away; its verdict on every analysis agrees with what the engine does (scored or refused; each QC finding assessed or not); the same in mavescape validate --json as in Node |
| The analysis package | A run's package (counts, target, design, sample sheet, parameters) for the fixture, GRB2 and two conditions, scored by Node and, written by the window, by mavescape run; every validation design written as a sample sheet | The run's output hash from its own files; the same bytes written twice; every sample sheet reads back as its design |
| Archives that keep opening | Workspaces saved by MaveScape 0.1.0 (two runs, a selection, a changed threshold), 0.2.0 (six runs: barcodes with their map, sorted bins, two conditions, Hsp90's codon variants per generation, DiMSum's model, the moderated combination; an acknowledged finding) and 0.3.0 (three opened from MaveDB, seven runs: GRB2 with its record, mapped variants, UniProt check and seven annotations; DHFR's two score sets together; BRCA1's nucleotide variants on a RefSeq transcript with MaveDB's annotations) 0.4.0 (two, three runs: TP53 with two score sets kept beside, GRB2 with its record's scores, AlphaMissense and UniProt's features beside it; selections by query) 0.5.0 (two: GRB2 with three saved views of 2VWF and its AlphaFold DB model, their files in the archive; PTEN with two views of 1D5R, named by checksum only), 0.6.0 (GRB2 with a figure of six panels, a comparison and a robustness check kept in its report and a checkpoint, and its certificate; Hsp90 with a checkpoint before its first three samples are fitted) 0.7.0 (GRB2 with an AI agent's proposals accepted and rejected, and its certificate; Hsp90 saved with an agent's proposal open) and 0.8.0 (factor IX with two ClinVar control sets, three calibrations, a figure of a calibration's plots and a checkpoint, and its certificate; the simulated calibration saved with an agent's control set and calibration waiting), opened by this one | Every file's checksum and the history's chain hold; each run recomputed to its recorded output hash, or (scored with 0.1.0's browser logarithms elsewhere) to its last digits, saying so; every annotation's tracks read again from the archive; every score set kept beside read again; every saved query selecting its variants again; every saved view's structure read again (or fetched again by its ID), the file recorded, its target mapped as saved; each checkpoint as kept and every number reported computed again; each certificate verified bit for bit; an agent's changes under its name, and an open proposal accepted or rejected as 0.7 decides it; each ClinVar control set read again from the archive's file the same labels, and each calibration computed again with the thresholds it keeps |
| Examples | All eighteen, opened as the window opens them; the published ones' counts against MaveDB's files; their published comparisons as their guides state them; GRB2's figure, Hsp90's checkpoint and BRCA1 RING's certificate made as their guides make them | Each scored, raising exactly the QC findings it teaches; its readout stated and its readiness naming what its guide points to; GRB2's, Hsp90's and factor IX's counts MaveDB's, byte for byte; DHFR's conditions as the paper found them; BRCA1 RING's published replicate scores reproduced and its deposit read back exactly; TP53's two screens, r −0.78 as published and Lin's concordance 0.78 on one scale; Hsp90's rescoring, r 0.95 but Lin's concordance 0.48; GRB2's figure read back as drawn; Hsp90's first three samples moving 148 of 188 variants, its pseudocount none; BRCA1 RING's 110,975 numbers verified bit for bit, with its tables or with MaveDB's files beside it |
| Codon variants | Hsp90's 568 rows (MaveDB urn:mavedb:00000011-a-1), each codon on its own row and the wild type once per position | 188 protein variants, the wild type's nine identical copies read once; MaveDB's protein names agree with the translation on every row but those nine, which it names as synonymous changes; mavescape run reads the codons for the example's design and gives Node's scores |
| Per generation | Hsp90's published fitness (the slope per generation, null-like variants on the first three time points) | r 0.95, a median 0.005 apart, where the authors fit every time point, in natural logarithms (the record says log₂); stops less negative, as a slope over 21 generations of a course at the floor must be |
| One replicate | Hsp90 combined by the moderated combination, which has no pairs of replicates to fit | Each score keeps its own slope's residual-scaled SE, within 2 × 10⁻¹⁶ |
| Problems found | The simulated problems example: a bottleneck recorded with the inputs, and a failing third replicate | The cells account for replicates 1 and 2, not 3; with replicate 3 the intervals hold 99.5% (too wide) and DiMSum's terms misplace the bottleneck; without it 95.9%, closer scores, the bottleneck before selection |
| Headless runs | mavescape run on the GRB2 example, two conditions and a table of barcodes; again with the same time, from its own archive (--from-workspace), in Node from that archive, and through remote control in a window | Every file the same bytes with a fixed time, the results without one; Node's and the window's files the same; a rerun exact, a tampered one refused; each failure with its exit status |
| Remote control | Every action, in the program and a browser, against the same analysis in Node | Identical scores; every export identical to Node's |
| AI agents | Every tool through mavescape mcp on stdio, as an agent's client calls it, in a browser; the agent's proposals of each kind accepted and rejected in the window; the guide's session on Hsp90 | Results the remote actions' and Node's; tools that only read change nothing; every change under the agent's name; each kind of change waiting, applied or removed as it should; every number the guide quotes |
| R and Python | Both packages against a running MaveScape (R CMD check too); the GRB2 notebook executed and the DHFR script run | GRB2's scores table every number of Node's; results the HTTP API's; each example's checks |
| Protein changes | biocommons hgvs 1.5.7 on BRCA1's, TP53's and BAP1's transcripts: every name of their three MaveDB score sets (TP53's 5,609 deletions and insertions among them) and generated substitutions, deletions, insertions, duplications and deletion-insertions, 16,188 in all | The same protein change for every one, in MAVE-HGVS's forms; MaveScape names nothing after a stop a variant makes, where biocommons names the rest of an insertion (12 variants) |
| Published protein names | The protein names five MaveDB records give beside their nucleotide names (CBS at two vitamin B6 levels, BRCA1's RING domain, Hsp90, TP53 base editing): 44,037 rows | The same protein on every row; 22,280 of CBS's written alike |
| Nucleotide to protein | DiMSum 1.4's own dimsum__aggregate_AA_variants_fitness on CBS's 11,478 codon variants | 7,561 amino-acid variants merged within 5 × 10⁻¹³; synonymous codons where DiMSum keeps them |
| GenBank | Biopython 1.88 on two RefSeq mRNAs, a plasmid of three coding features (one on the reverse strand) and HBB joined from three exons | Every field, feature location, coding sequence and translation |
| MaveDB deposits | MaveDB's own validators (its mavedb package 2026.2.7.4, with mavehgvs 0.8.1 and biocommons hgvs 1.5.7 given each transcript) on 21 deposits: protein variants, c. names with their protein changes, n. names, names on a RefSeq or Ensembl transcript; runs scored from counts of every design and published scores passed through; factor IX calibrated, its calibration as MaveDB's score calibration. Each read back as MaveScape opens a package and written again | Every table accepted (111,617 rows, 152,537 names strict MAVE-HGVS); the experiment, score set and score calibration accepted, or refused for exactly what MaveScape says is missing; every deposited score in the class MaveScape gives it by MaveDB's own test of a range; written again, the same bytes |
| Records with a transcript | BRCA1's and BAP1's saturation genome editing and TP53's DNA-binding domain, opened from MaveDB, their transcripts fetched from NCBI and Ensembl; MaveDB's mapping of GRB2's variants to the human reference | Every variant valid on its transcript; each mapped protein at the target's position plus the UniProt offset |
| Robustness | Ten simulated experiments of four replicates with two kinds of fragile result planted among wild type-like variants (one replicate driving an effect, its output cut to a quarter; no reads after selection, an effect whose size appears at one pseudocount), checked against 47 other analyses each, declared by fixed effects, and three under MaveScape's defaults; the declared analysis against the run; the classes against an independent computation; GRB2 and DHFR checked | The variants reported fragile exactly the replicate-driven ones every time (92 of 93 naming that replicate left out), those whose effect's size depends on the pseudocount exactly the zero-count ones; the clearly lost variants robust; under the defaults the replicate-driven effect not called, nothing fragile; the declared analysis the run to the last bit; DHFR's 377 differences between −Lon and +Lon: 54 in every analysis, 172 in 90% |
| Selections | Fourteen queries on GRB2's Run 1 against an independent account of each of its map's 1,176 cells, read from the run's own arrays; set operations by query against the sets' own; the robustness verdicts as queries on GRB2's check and three simulated experiments with planted fragile results; factor IX's disulfide bonds from UniProt P00740; freeform shapes against the rectangle tool and a circle's cells | The same variants in the same order for every query; union, intersection, difference, De Morgan's laws and a saved selection on another run as the sets give them; each verdict's variants exactly the check's, and the planted ones exactly; the entry's 22 bonded cysteines; the same cells |
| Comparisons | Seven pairs (six simulated, and DHFR's published scores without and with Lon), a pair of 600 variants with planted differences, two assays of 361 variants on two scales rescaled by their anchors, and GRB2's published scores against AlphaMissense by its classes, against R (cor.test, DescTools' CCC, BlandAltmanLeh, lm, pnorm, p.adjust, pROC's DeLong interval); 2,000 simulated pairs of known agreement of each of five kinds, 30 to 1,000 variants; 20 pairs with planted discordant variants and positions, 10 with none; 600 pairs of two assays differing only in scale; 6,000 simulated pairs of classes of known AUC; the compatibility report's rules on GRB2, DHFR and BRCA1; a MaveDB score set kept beside the workspace against the same record opened; the examples compared as their guides say | Every statistic, z, p and q equal to R's (to 10⁻¹⁰); the 95% intervals holding the truth 94–96% of the time (Spearman's near r = 1, and Lin's with 30 variants on two scales, less: reported as they are); 96% of planted variants found with 4% of calls false, 57 of 60 planted positions, none called with nothing planted; rescaled by their anchors, false calls in 4% of null pairs with the anchors' uncertainty in the test, 10–55% without; DeLong's interval holding the AUC 94–95% of the time; a score set not opened giving the same comparison as opened; DHFR's discordance its run's own differential z; BRCA1's counts scored against its published scores r 0.76, correlations only |
| Structures | Each example protein's PDB entry and AlphaFold DB model, read as the page reads them; 2VWF in three formats; biological assemblies; Proteoscope 0.10.0's tests of the copied viewer | Each chain holds its target on the UniProt residues its entry gives; the same atoms to 0.001 Å in every format; each model's pLDDT within 0.05 of AlphaFold DB's mean; the tests pass here |
| The target on a structure | PDBe's residue-level SIFTS mapping of each example's entry, read from its updated mmCIF with gemmi; each AlphaFold DB model's numbering | Every residue placed where SIFTS places it, the unobserved called so; the one place they part an alignment that scores the same both ways, called ambiguous |
| Scores on a structure | Each example's run on its entry, by every coloring, against its results read without the map's model; PTEN's fraction abnormal-like by its record's published calibrations | Every residue the value and color of its position; nothing scored drawn gray, at least 100 (RGB) from the wild type's color; the calibrations' shares as their own JSON gives the ranges |
| Structure as evidence | DSSP 4.6.1's own output for each entry (the DSSP databank); SciPy 1.18.1's Mann–Whitney test and Benjamini–Hochberg; the cluster test on simulated maps of 1D5R and 1TUP | Every secondary structure code but five π-helix residues; accessibility within 1 Ų on average, no bias; the tests to 10⁻¹²; with scores that depend on burial alone, p ≤ 0.05 in 3–6% of maps, a planted patch found in 62–71% |
| Saved views | A saved view of each example protein's entry and model (14) through the workspace archive, included and named by checksum; a view of 2VWF saved, changed and opened again in the page | The same files, the mapping made again the one saved; the same figure panel, pixel for pixel, reopened, from the archive and fetched again |
| Open in Proteoscope | Each example's session on its entry and model; Proteoscope 0.10.0 itself opening the links of 2VWF and GRB2's model and the session file of a PDB-format file | Every value the position's median, on a residue Proteoscope's own reader reads; the residues and camera as MaveScape had them |
| Figures | Nine examples' maps as figure panels with their tracks, and every view's plots as panels (96 QC findings of eight examples, scores by class, a volcano, variants' evidence, a comparison), written as SVG and PDF, against the map's own SVG and the results themselves; the PDF's content read back; figures exported by the window against Node's; each file's manifest read back, edited, compared after a rescoring and rebuilt | Every cell the map's own color (9,681 of factor IX's), every bar and point the results'; the PDF the SVG's rectangles and text, mark for mark; the same bytes written again and through the archive; the window's SVG and manifest Node's, byte for byte; a structure panel the view's own pixels; an edited manifest caught, exactly the panels a rescoring touches flagged, a rebuilt figure the same bytes |
| Report | GRB2's workspace built one operation at a time (runs, a UniProt check, thresholds, an acknowledgement, a selection by query, a comparison and a robustness check kept, annotations, a structure view, a figure); the 18 examples' runs; every cited DOI against Crossref's record of it (recorded); the window's report against Node's | Every operation its sentence, nothing said that was not done; each run's paragraphs its methods word for word; 50 of 50 references agree; the window's report Node's text |
| Checkpoints and changes | Five planted changes from a checkpoint of GRB2's run (a pseudocount, a filter, a replicate left out, a QC threshold, the readout); the variants each rescoring moved, computed apart from the scores and SEs; Hsp90 scored per generation, then over its whole time course; the window's diff against Node's, from the workspace's archive and between two archives | Each change described exactly and nothing else; the variants moved, crossed and filtered the same; a change of units moving nothing; an analysis against itself unchanged; the window's changes Node's |
| Certificates | GRB2's workspace of every operation certified, each kind of number computed apart from the certificate (scores, SEs and classes; QC; the analyses kept in the report; each figure panel; the report); six kinds of tampering, from a count in the table to a checkpoint's state, with the archive made to list each; a certificate without its tables; the 18 examples and every release's archives; the window's certificates against Node's, and mavescape verify | Every number as computed apart, verified bit for bit, the same bytes again; every tampering caught and said; without its tables not checkable, then verified with them; 224,792 numbers of the examples verified; the window's certificate Node's, byte for byte, its figure panels the drawings their manifests record |
| Review reports | Every traced number of the review report of GRB2's workspace of every operation and of the 18 examples, against where it says it comes from; the file opened in Chrome (its requests; axe-core in the light and dark themes); the window's report, mavescape verify --review's and export --format review's against Node's | Every number the certificate's, the workspace's or a count of the certificate's classes, as it says; nothing loaded from outside the file (planted references caught); no axe-core violation in either theme; the same bytes everywhere |
| Control sets | Factor IX's ClinVar records under four rules and those last evaluated before 2025, against an account read straight from NCBI's cached answers by a reader of its own; a table of invented labels with every case a reader meets planted, and one dated as ClinVar's tabular download dates it; each kind of circularity planted | Every control, side, review stars, records, latest evaluation and kind the same (169 P and 22 B by default); each planted case as it must become; every date read; each circularity where planted and none elsewhere |
| OddsPath | Brnich et al. 2019's own spreadsheet (Additional file 1: Table S1, a perfect readout; Table S2, one control indeterminate); their Table 3 | Every row of both tables to 15 digits but one whose counts cannot be; each threshold of Table 3; 11 controls the fewest with which both readouts reach moderate, as the paper says |
| Likelihood ratios | acmgscaler 1.0.0 on its own example (BRCA1 and TP53), at Pejaver et al.'s prior, on negated scores and on factor IX's Run 1 with 8,827 unlabeled variants; R 4.6.1's numerics as a black box; Pejaver et al. 2022's Table 1 | Every likelihood ratio, interval, evidence and score threshold to 10⁻¹²; R's random numbers bit for bit, its densities and bandwidths within 10⁻¹¹; Table 1 as printed |
| Published calibrations | MaveDB's calibrations of six records (BRCA1, PTEN ×2, TP53 ×2, factor IX); Fayer et al. 2021's BRCA1 data | 105 classes counted as MaveDB counts them, every stated strength Table 3's; Fayer et al.'s OddsPath, 52.4 and 0.02, reproduced |
| Calibration in the record | Factor IX's Run 1 calibrated against its ClinVar controls: the report and methods, a figure of its four plots, its certificate and review report, the diff after a rescoring, its MaveDB deposit read back and calibrated again; the window's against Node's | Every number in the report as computed; the figure rebuilt from its manifest, every drawing the same; certified and verified bit for bit (one OddsPath changed in its last bit caught); the classes changed counted directly; the deposit's thresholds and OddsPath given back exactly; the window's certificate Node's, byte for byte |
| Calibration examples | BRCA1's, TP53's and PTEN's published scores against ClinVar's records of the gene as the examples bring them (13 October 2026), against NCBI's answers they were made from and an account read straight from them; a simulated calibration whose labels are half made from its own scores; the window's against Node's | Every number each guide states, as computed; the controls by three rules the account's; each calibration against the simulated truth as the guide says; the window's every number Node's |
| Calibration's metrics | pROC 1.19, PRROC 1.4 and binom 1.1 on seven cases (synthetic, ties, sides well and fully apart, few controls, factor IX's controls in Run 1); a stratified bootstrap of 2,000 samples | The ROC curve's points, the AUC and DeLong's interval, the precision–recall curve and its area, each rule's thresholds and classes, Wilson's intervals, to 13 digits; the bootstrap's 1,092 quantiles within their Monte Carlo error |
The data are MaveDB records under CC0, chosen to differ: GRB2's SH3 domain from the Human Domainome (two populations, three replicates), BRCA1's RING domain (a time series with shared inputs and a second assay in the same table), factor IX (sorted bins over overlapping tiles), and CBS at two vitamin B6 levels (two conditions selected from the same inputs). validation/README.md lists every check and how the reference outputs were made.
The same bits in every browser
JavaScript's Math.log and Math.exp may differ in their last bit between engines, and they do: Chrome 154 and Node 22 disagree on about 2% of logarithms. MaveScape computes its logarithms and exponentials with fdlibm's algorithms written in plain arithmetic, which every engine performs identically, so a run's output hash does not depend on the browser that scored it and a saved run reproduces after a browser update.
Methods it reimplements
MaveScape's scores are reimplementations of published methods. Their authors are cited in the code and in the methods paragraph MaveScape writes for each run:
- Scores: log ratios of frequencies before and after selection, with normalization to the wild type, and for time series the slope of a weighted regression on time, as in Enrich2 (Rubin et al., Genome Biology 2017).
- Replicates: by default, a shared error model of multiplicative and additive terms fitted to the differences between replicates, as DiMSum's (Faure et al., Genome Biology 2020), with each variant's dispersion moderated by empirical Bayes as limma moderates variances (Smyth, Stat Appl Genet Mol Biol 2004), degrees of freedom combined by Satterthwaite's approximation (Biometrics Bulletin 1946), replicates sharing an input combined by generalized least squares, and the rescaling anchors' uncertainty by the delta method; REML random effects (Viechtbauer, J Educ Behav Stat 2005), computed as metafor does (Viechtbauer, J Stat Softw 2010); Enrich2's own estimator; heterogeneity as Q and I² (Higgins and Thompson, Stat Med 2002).
- Barcodes: a variant's barcodes summed before scoring, as Enrich2 (Rubin et al., Genome Biology 2017), or each scored against the summed wild type, as the Bloom lab's dms_variants
func_scoresby barcode, and combined. - Sorted bins: the weighted average of the bins' values and its scaling (Matreyek et al., Nature Genetics 2018), and the censored log-normal maximum-likelihood estimate (Peterman and Levine, BMC Genomics 2016).
- DiMSum's model: fitness, the scaling of replicates and the error model of multiplicative and additive terms, as DiMSum 1.4 computes them (Faure et al., Genome Biology 2020), the error model fitted exactly by bounded least squares (Lawson and Hanson 1974).
- Differential scores: limma's moderated t-statistics (Smyth, Stat Appl Genet Mol Biol 2004) on voom log counts with precision weights (Law et al., Genome Biology 2014), as mutscan computes them (Soneson et al., Genome Biology 2023), written from the publications (limma is GPL, and serves only as the reference in validation); the shared input cancelling from paired replicates' differences; q-values by Benjamini and Hochberg (J R Stat Soc B 1995).
- Quality control: variance beyond counting read as DiMSum's multiplicative and additive error terms, and DiMSum's terms fitted to place a bottleneck (Faure et al., Genome Biology 2020).
- Variant names and exports: MAVE-HGVS and MaveDB's columns (Esposito et al., Genome Biology 2019); a deposit's keywords in the MAVE minimum information's controlled vocabulary, as MaveDB keeps it (Claussnitzer et al., Genome Biology 2024).
- Structures: PDB entries (Berman et al., Nucleic Acids Research 2000) and AlphaFold DB models (Jumper et al., Nature 2021; Varadi et al., Nucleic Acids Research 2024) read with Proteoscope's readers; the target aligned to each chain with BLOSUM62 (Henikoff and Henikoff, PNAS 1992), checked against PDBe's SIFTS (Dana et al., Nucleic Acids Research 2019); secondary structure by DSSP (Kabsch and Sander, Biopolymers 1983); accessibility by Shrake and Rupley's method (J Mol Biol 1973) with DSSP's radii, relative to Tien et al.'s maxima (PLoS ONE 2013); exposure and disorder by StructureMap's part-sphere exposure (Bludau et al., PLoS Biology 2022); contacts typed by PLIP's rules (Adasme et al., Nucleic Acids Research 2021); the scores by class by Mann and Whitney's test (Ann Math Stat 1947), adjusted by Benjamini and Hochberg's.
- Calibration, for research: the ROC curve's points and the AUC with DeLong's interval as pROC computes them (Robin et al., BMC Bioinformatics 2011; DeLong et al., Biometrics 1988), the precision–recall curve's interpolation and area as PRROC's (Keilwagen et al., PLoS ONE 2014; Grau et al., Bioinformatics 2015), Wilson's interval (Wilson, JASA 1927) and Youden's index (Youden, Cancer 1950); OddsPath and its evidence strengths as the ClinGen SVI Working Group define them (Brnich et al., Genome Medicine 2020); score-level likelihood ratios as acmgscaler computes them (Badonyi, Bioinformatics 2025), with the evidence thresholds of Tavtigian et al. (Genetics in Medicine 2018) or Pejaver et al. (AJHG 2022), and the numerics they rest on: Matsumoto and Nishimura's Mersenne Twister, kernel density by the FFT (Silverman 1982), biased cross-validation (Scott and Terrell 1987), Brent's minimizer.
- Protein changes: a nucleotide variant's protein change as HGVS describes it, computed as biocommons hgvs computes it (Hart et al., Bioinformatics 2015); nucleotide variants merged at the protein level as DiMSum merges them (Faure et al., Genome Biology 2020); MaveDB's mapped variants as GA4GH VRS alleles (Wagner et al., Cell Genomics 2021; Arbesfeld et al., Genome Biology 2025).
Cite MaveScape
If you use MaveScape in published work, please cite it together with the methods its methods paragraph lists for your run. The repository's CITATION.cff holds the citation, and GitHub's Cite this repository button formats it.
McDermott R. MaveScape, version 0.8.0. https://github.com/robert-mcdermott/mavescapePrivacy and security
MaveScape listens on 127.0.0.1 only, refuses requests whose Host header is not this computer and API calls from other web pages, and serves its page with a strict Content-Security-Policy. It makes no network requests of its own: public data are fetched when you ask for them (Public data), and the one other request it makes stays on this computer, asking whether Proteoscope is running, for the Structure view (--proteoscope off stops it). Remote control is off unless MaveScape is started with --remote-control; even then only programs on this computer can use it, and reading or writing files needs a token that only you can read. mavescape run and validate drive a private copy of MaveScape of their own, on a port of its own with a temporary library, which ends with them.
What MaveScape is not
MaveScape reports experimental functional effects for research. It does not classify variants as pathogenic or benign, and its scores are not clinical evidence on their own.
Credits
- The GRB2 example is MaveDB urn:mavedb:00000835-a-1, from Beltran A, et al. Site-saturation mutagenesis of 500 human protein domains. Nature 2025, published under CC0 1.0.
- The validation data are from MaveDB (CC0 1.0) and DiMSum's demonstration data (MIT).
- The Structure view's readers, viewer, DSSP, exposure and interactions are Proteoscope 0.10.0's (Apache 2.0, the same author), copied with their tests and noted in each file. The structures the examples and validation use are from the PDB (CC0) and AlphaFold DB (CC BY 4.0).
web/lib/dmath.jsis fdlibm 5.3's logarithm and exponential (Sun Microsystems, 1993), whose notice it keeps.- MaveScape's program, window and site come from its sibling, CytoWeave, noted in each file.