# soilKey v0.9.18 vs KSSL / NCSS Lab Data Mart -- 2026-05-01

**First US-context external validation** of soilKey. Runs on the
NCSS Lab Data Mart `ncss_labdata.gpkg` GeoPackage (5.1 GB), reading
the joined `lab_combine_nasis_ncss` (taxonomy + pedon_key) /
`lab_site` (lat/lon) / `lab_layer` (horizon depth + designation) /
`lab_chemical_properties` (CEC / BS / OC / pH / Al / Ca / Mg / K /
Na) / `lab_physical_properties` (clay / silt / sand / bulk density)
layers via the new \code{\link{load_kssl_pedons_gpkg}} loader.

## Headline number

**USDA top-1 = 21.4 %** on n = 3 213 quality-filtered pedons,
CI 95 % = [19.9 %, 22.7 %] (200 bootstrap replicates).

The benchmark was sourced from the first 5 000 of 36 090 KSSL
pedons that have a populated `samp_taxorder`. Quality filter:
clay measured + at least one of CEC / BS / pH measured + at least
one B horizon designation (`grepl("^B", designation)`). 4 817 of
the 5 000 loaded successfully (97 %); 3 213 of those passed the
quality filter (66 %).

## Per-Order breakdown

| Order | n | correct | accuracy |
|---|---:|---:|---:|
| Histosols   | 3    | 2   | **66.7 %** |
| Entisols    | 108  | 72  | **66.7 %** |
| Oxisols     | 49   | 24  | 49.0 %     |
| Aridisols   | 446  | 161 | 36.1 %     |
| Mollisols   | 727  | 177 | 24.3 %     |
| Alfisols    | 663  | 158 | 24.0 %     |
| Ultisols    | 411  | 94  | 22.9 %     |
| **Spodosols**   | 276  | **0** | **0 %** |
| **Inceptisols** | 463  | **0** | **0 %** |
| **Vertisols**   | 63   | **0** | **0 %** |
| **Andisols**    | 4    | **0** | **0 %** |

## What the numbers say

The KSSL benchmark exposes **four distinct calibration regimes**:

1. **Saturated -- Histosols / Entisols / Oxisols / Aridisols**
   (>= 36 % each). The diagnostic gates for these Orders are
   well-calibrated against the actual KSSL pedon data.

2. **Mid-range -- Mollisols / Alfisols / Ultisols** (22-25 %
   each, n = 727 / 663 / 411). The Mollic / argillic gates fire
   correctly on roughly a quarter of the reference set; the
   remaining 3/4 mostly fail silently because of missing
   chemistry / Munsell columns. Same v0.9.17-v0.9.18 inference
   fallbacks that lifted Embrapa Mollisols from 0 to 26.5 % are
   doing the work here too.

3. **Failed -- Spodosols (0/276) + Inceptisols (0/463)**. These
   are the two biggest single-Order misses by reference count.

   - **Spodosols** require a spodic horizon (Bs / Bh) defined by
     Al-humus / Fe-humus accumulation indicators (specific Al_ox
     + Fe_ox thresholds, specific colour signature). The KSSL
     pedons have most of these fields measured, so the failure is
     in `spodic()` / `spodosol_usda()` itself rather than in
     missing data. v0.9.19 work item.
   - **Inceptisols** are the catch-all "cambic horizon, no
     stronger Order qualifies" Order in the canonical key. Our
     `inceptisol_usda()` may be too strict on the cambic test or
     may be excluded by an upstream Order that should have failed.

4. **Insufficient sample -- Vertisols (n=63) + Andisols (n=4)**.
   Both at 0 %. The Vertisol gate requires slickensides /
   shrink-swell evidence that KSSL records but our fixtures may
   not extract; the Andisol n is too small to draw conclusions.

## Cross-dataset consistency

The same v0.9.18 software produced 47.6 % USDA accuracy on Embrapa
FEBR (n=614) and 21.4 % on KSSL (n=3 213). The 26 pp gap reflects
where soilKey was originally calibrated:

* **Embrapa FEBR** is dominated by tropical Latossolos (Oxisols)
  and Argissolos (Ultisols), the two Orders where soilKey's
  ferralic / kandic / argillic gates are mature. Top-1 accuracy
  on Oxisols = 81.3 % on Embrapa.
* **KSSL** is dominated by temperate Mollisols / Alfisols /
  Inceptisols / Spodosols, where the missing-data fallbacks are
  v0.9.17-v0.9.18 work and Spodosol detection is still pending.

Both numbers are real, externally-validated, and reproducible
from the bundled benchmark scripts. They also bracket the
"reasonable" expectation for a deterministic key on out-of-sample
real data: the high end (Embrapa Latossolos) at 80-100 % when the
gates are calibrated, the low end (KSSL Spodosols / Inceptisols /
Vertisols) at 0 % when the gates haven't been pressure-tested.

## Reproducing this run

```r
Sys.setenv(SOILKEY_SKIP_NETWORK = "true")
library(soilKey)

peds <- load_kssl_pedons_gpkg(
  "soil_data/KSSL/ncss_labdata.gpkg",
  head = 5000)

quality <- vapply(peds, function(p) {
  any(!is.na(p$horizons$clay_pct)) &&
    (any(!is.na(p$horizons$cec_cmol)) ||
       any(!is.na(p$horizons$bs_pct))   ||
       any(!is.na(p$horizons$ph_h2o))) &&
    any(grepl("^B", p$horizons$designation))
}, logical(1))
peds_q <- peds[quality]

res <- benchmark_run_classification(peds_q, system = "usda",
                                          level = "order",
                                          boot_n = 200L)
```

Saved RDS:
[`kssl_results_5000_2026-05-01.rds`](kssl_results_5000_2026-05-01.rds).
