Data & ML
Verify the methodology, not just the syntax.
Spearman
Spearman reviews Data + ML code for silent methodological failures that ordinary tests and general-purpose code review often miss — including leakage, invalid metrics, incorrect joins, flawed evaluation logic, and reproducibility risks.
What it evaluates
- Temporal, label, and feature leakage
- Evaluation integrity and metric validity
- Feature logic, joins, grain, and data transformations
- Reproducibility and non-determinism
- Experiments, causal logic, forecasting, and performance claims
Where to use it
- Python, SQL
- Pull requests before merge
- API and agent workflows
- Wyrd requirement-aware verification
The work
A SQL training-table query executes and looks plausible, but joins multiple one-to-many sources before computing labels and metrics.
FROM orders LEFT JOIN order_items ON ... LEFT JOIN refunds ON ... LEFT JOIN sessions ON ... AVG(target) AS refund_rate SUM(total_amount) AS gross_sales
Spearman finding
Duplicated, future-leaking rows make the training table methodologically invalid.
One logical order can become many rows. Revenue and refund metrics are then calculated over duplicated observations, while unconstrained customer activity can leak future information into features.
Proposed remediation: restore one-row-per-order grain, pre-aggregate one-to-many sources, apply point-in-time predicates, then compute metrics and labels.
Create a Bohmian account for Spearman, API credentials, usage, and supported integrations.