# soilKey v0.9.20 KSSL + NASIS integration -- 2026-05-01

The "field morphology unlocks the lab" release. Integrates the
NASIS Morphological export
(`NASIS_Morphological_09142021.sqlite`, 562 MB, 431 415 phorizon
rows) with the existing NCSS Lab Data Mart GeoPackage. The lab
gpkg has chemistry + physics; NASIS has Munsell colour, structure
grade, clay films, slickensides, cracks, and surveyor-identified
diagnostic horizons. Joining them on `peiid` (Pedon Element ID)
unlocks every diagnostic gate that needed field morphology to fire.

## Headline numbers

| Run         | n   | top-1   | 95 % CI    |
|-------------|----:|--------:|------------|
| v0.9.19 (lab-only, 1 500-head)         | 998   | 24.3 % | [21.9, 26.7] |
| **v0.9.20 (lab + NASIS, 1 500-head)**  | 998   | **32.7 %** | [29.7, 35.5] |
| v0.9.19 (lab-only, 3 000-head)         | 1 997 | 22.6 % | [21.0, 24.1] |
| **v0.9.20 (lab + NASIS, 3 000-head)**  | 2 002 | **29.8 %** | [27.5, 32.0] |
| v0.9.19 (lab-only, **5 000-head**)     | 3 213 | 26.0 % | [24.6, 27.3] |
| **v0.9.20 (lab + NASIS, 5 000-head)**  | 3 218 | **32.2 %** | [30.7, 33.6] |

**+ 6.2 pp** at the definitive 5 000-head sample. The improvement
is driven entirely by Orders that **need field morphology** to
disambiguate from the catch-all Entisol path.

## Per-Order accuracy (1 500-head sample)

| Order        | v0.9.19 lab | v0.9.20 lab+NASIS | Δ |
|--------------|------------:|------------------:|---:|
| **Inceptisols**  | 5.4 % (7/129)   | **53.8 % (70/130)** | **+ 48.4 pp** |
| **Spodosols**    | 10.0 % (15/150) | **29.9 % (26/87)**  | **+ 19.9 pp** |
| **Vertisols**    | 35.7 % (5/14)   | **56.3 % (9/16)**   | **+ 20.6 pp** |
| Mollisols    | 20.3 % (54/266) | 21.8 % (58/266)   | +1.5  |
| Alfisols     | 21.6 % (50/232) | 20.9 % (49/234)   | -0.6  |
| Aridisols    | 50.0 % (65/130) | 53.1 % (69/130)   | +3.1  |
| Ultisols     | 27.4 % (26/95)  | 26.3 % (25/95)    | -1.0  |
| Entisols     | 68.8 % (22/32)  | 53.1 % (17/32)    | -15.6 |
| Oxisols      | 60.0 % (3/5)    | 60.0 % (3/5)      | 0     |

The Entisol drop is expected: KSSL pedons that were previously
falling into the Entisol catch-all because Inceptisol /
Spodosol / Vertisol gates couldn't fire (no morphology) now
correctly route to those Orders.

## Why it works scientifically

The lab gpkg lacks every field morphology variable that KST 13ed
Ch 3 lists as "the diagnostic feature that disambiguates Order
membership when chemistry alone is ambiguous":

| Diagnostic | KST 13ed | Lab gpkg | NASIS sqlite |
|---|---|---|---|
| Mollic epipedon (Ch 3 p 15) | value moist <= 3 + chroma <= 3 | none | `phcolor` (528 421 rows) |
| Argillic horizon (Ch 3 p 4) | clay films / lamellae / clay bridges | none | `phpvsf` clay films (109 793 rows) |
| Cambic horizon (Ch 3 p 13) | structure or designation evidence | none | `phstructure` (421 881 rows) |
| Vertic horizon (Ch 3 p 23) | slickensides + cracks (or LE >= 6 cm) | only COLE | `phpvsf` slickensides (4 275 rows) + `phcracks` (170 rows) |
| Spodic horizon (Ch 3 p 23) | spodic morphology + B[hs] | designation only | enriched designation via `phorizon.hzname` |

NASIS provides 99 % Munsell coverage, 93 % structure-grade
coverage, 36 % clay films, 3 % slickensides directly + another
~ 5 % via `pediagfeatures.featkind`.

## Code architecture

`load_kssl_pedons_with_nasis(gpkg, sqlite, head, ...)`:

1. Calls the existing `load_kssl_pedons_gpkg()` for lab data.
2. Reads the NASIS tables (phorizon / phcolor / phstructure /
   phpvsf / phcracks / pediagfeatures) **fully into memory** and
   filters with data.table keyed lookups -- empirically more
   reliable than SQL `IN (...)` with thousands of pedon IDs.
3. For each lab layer, finds the NASIS phorizon with the largest
   `(hzdept, hzdepb)` overlap (depth-based join).
4. Pulls dominant Munsell (largest `colorpct`), structure grade /
   size / type (lowercase-normalised), clay films
   (`pvsfpct` mapped to qualitative tiers), slickensides, cracks.
5. Replaces NA designations with NASIS `hzname` (richer than the
   lab gpkg's `hzn_desgn`).

The function adds `Suggests: DBI, RSQLite` to DESCRIPTION;
package-level loaders that don't touch the NASIS sqlite remain
dependency-free.

## Reproducing this run

```r
library(soilKey)

peds <- load_kssl_pedons_with_nasis(
  gpkg   = "soil_data/KSSL/ncss_labdata.gpkg",
  sqlite = "soil_data/KSSL/NASIS_Morphological_09142021.sqlite",
  head   = 5000)

quality <- vapply(peds, function(p) {
  any(!is.na(p$horizons$clay_pct)) &&
    (any(!is.na(p$horizons$cec_cmol)) ||
       any(!is.na(p$horizons$bs_pct))   ||
       any(!is.na(p$horizons$ph_h2o))) &&
    any(grepl("^B", p$horizons$designation))
}, logical(1))
peds_q <- peds[quality]

res <- benchmark_run_classification(peds_q, system = "usda",
                                          level = "order",
                                          boot_n = 200L)
```
