A whiteboard mapping the model risk taxonomy and out-of-sample backtesting findings.
Home/ Practice areas/Model validation
04 / 08 · Led by Edgar
Practice area · 04/08

Model validation.

Credit, market, AML, and AI/ML models — independently validated, conformant with SR 11-7, and written for examiners to read.

01

The practice

Independent validation, by someone who has built models.

A validation report should hold up to an examiner — and to the modeler whose model it covers.

The SR 11-7 framework asks for independent validation across three dimensions: conceptual soundness, ongoing monitoring, and outcomes analysis. Done well, a validation gives the model risk committee real comfort. Done poorly, it gives them a 60-page document that says 'no material issues identified' and an examiner who finds three.

Our practice covers credit risk models (PD/LGD/EAD, CECL, CCAR loss forecasting), market risk models (VaR, sensitivity, stress), AML transaction monitoring models (rule sets and machine-learning hybrids), and the increasing class of AI/ML models used in underwriting, fraud detection, and customer-facing applications. For each, we validate against the data, the assumptions, the implementation, and the use.

The validator has to understand the model. That sounds obvious; it is what most validation reports fail at. Edgar built his career as a quantitative practitioner before he became a validator. The reports our practice produces show the math; they do not paper over it.

02

What we do

The work in this practice, named.

01 · 06 Credit risk models

PD/LGD/EAD, CECL allowance models, CCAR/DFAST loss forecasting, scorecard models.

02 · 06 Market risk models

VaR, expected shortfall, sensitivity, scenario and reverse-stress models.

03 · 06 AML / TM models

Rule-set calibration, threshold tuning, hybrid ML-based detection systems.

04 · 06 AI / ML models

Underwriting, fraud, churn, and customer-facing models — including fairness, explainability, and drift monitoring.

05 · 06 Conceptual soundness

Theory, assumptions, choice of methodology, alternatives considered, data appropriateness.

06 · 06 Ongoing monitoring & outcomes

Backtesting, benchmarking, sensitivity, monitoring plan, threshold setting.

03

A typical engagement

A model validation, beginning to end.

01
Weeks 1–2 Intake

Model documentation reviewed, data dictionary received, scope confirmed with model risk management.

02
Weeks 3–6 Replication

Independent replication on the same data; alternative specifications considered.

03
Weeks 7–8 Testing

Sensitivity, stability, fairness (where applicable), backtesting, benchmarking.

04
Weeks 9–10 Reporting

Findings rated, validation report drafted, MRMC presentation prepared.

04

Who leads it

Validated by someone who has built models.

Edgar Osuna, Ph.D.
Practice lead · Model validation

Edgar Osuna, Ph.D.

Quantitative Risk · Analytics · MIT

Edgar holds a Ph.D. in Operations Research from MIT and has spent twenty-five years in quantitative risk and analytics — co-leading Mercantil's Global Risk Management and serving as CDAO at iuvity. He validates models the way he would want his own validated.

What this practice is not

We do not build or operate models that we also validate. Independence here is not a preference — it is the regulatory baseline. If we have built a model for an institution, we will not be the firm that validates it.

05

Related practice areas

What often runs alongside this.

Start an engagement

Bring the partners to the table.