# BDsolos v0.9.60 benchmark report

**Date**: 2026-05-06
**soilKey version**: 0.9.60
**Scope**: RJ.csv (single UF, 722 perfis, 100% Munsell, 99.7% SiBCS labels)
**Benchmark function**: `benchmark_bdsolos()` (new in v0.9.60)
**Generator**: `inst/benchmarks/run_bdsolos_v0960_focused.R`

## What this report closes

The first publication of an empirical accuracy number for the soilKey
classifiers against the Embrapa BDsolos national reference labels.
v0.9.55 shipped the loader; v0.9.58 shipped the full export schema;
v0.9.59 destravou 7 UFs com encoding malformado; v0.9.60 ships
**(a)** `benchmark_bdsolos()` -- a triple-system validator that runs
WRB / SiBCS / USDA classifiers per pedon and tallies accuracy per
system; and **(b)** the quote-aware `.bdsolos_find_header_line()`
fix without which RJ.csv silently returned 0% taxon / 0% Munsell
(the v0.9.58 sintetico tests didn't catch it).

## Coverage on RJ.csv

| Field            | n   | %     |
|------------------|-----|-------|
| Total pedons     | 722 | 100.0 |
| `reference_sibcs` | 720 |  99.7 |
| `reference_wrb`  |   5 |   0.7 |
| `reference_st`   |  14 |   1.9 |
| Coords (DMS)     | 561 |  77.7 |
| Munsell (any hz) | 722 | 100.0 |

WRB/USDA labels are sparse on this UF; they're surveyor-optional in
BDsolos. SiBCS is essentially universal.

## Per-system result (post-fix)

| System     | Label coverage | Accuracy           | n_compared | n_correct | errors |
|------------|---------------:|-------------------:|-----------:|----------:|-------:|
| WRB 2022   |    0.7% (5)    | 20.0% (1/5)        |          5 |         1 |      0 |
| **SiBCS 5**| **99.7% (720)**| **35.8%** (254/710)|        710 |       254 |      0 |
| USDA-ST 13 |    1.9% (14)   | 33.3% (4/12)       |         12 |         4 |      0 |

The **35.8% SiBCS Order accuracy on 710 perfis** (post legacy-label
fix; pre-fix 27.9% on 720) is the headline empirical baseline for
soilKey v0.9.60 on the BDsolos national reference set. WRB and
USDA accuracies are shown for completeness but the label coverage
on RJ.csv alone (5 / 14 pedons respectively) is too sparse to
interpret -- see the project-wide at-scale benchmarks for those:

| Existing benchmark               | System | n     | Order acc | CI |
|----------------------------------|--------|-------|----------:|----|
| FEBR superconjunto (v0.9.27)     | SiBCS  |   554 | **56.7%** | 52.7-60.6 |
| KSSL+NASIS (v0.9.27, w/ filter)  | USDA   |   865 | **37.0%** | 33.9-40.2 |
| KSSL+NASIS+Tiebreaker (v0.9.22)  | USDA   | 2,002 |   31.3% | 29.0-33.5 |
| LUCAS topsoil-only (v0.9.49)     | WRB    |   200 |    3.0% | (artifact) |

## SiBCS per-class recall (post legacy-fix)

After the `.SIBCS_LEGACY_ORDER_MAP` fix in `normalise_febr_sibcs()`:

| Reference Order  | n_ref | n_correct | Recall |
|------------------|------:|----------:|-------:|
| Argissolos*      |   240 |       174 | 72.5%  |
| Latossolos       |   114 |         9 |  7.9%  |
| Gleissolos**     |    98 |         0 |  0.0%  |
| Cambissolos      |    90 |        16 | 17.8%  |
| Neossolos***     |    57 |        46 | 80.7%  |
| Planossolos      |    36 |         1 |  2.8%  |
| Aterros          |    24 |         0 |  0.0%  |
| `<unmapped>`     |    10 |         0 |  0.0%  |
| Espodossolos     |    10 |         3 | 30.0%  |
| Organossolos     |     5 |         1 | 20.0%  |
| Chernossolos     |     4 |         1 | 25.0%  |
| Nitossolos       |     4 |         0 |  0.0%  |
| Plintossolos     |     4 |         0 |  0.0%  |
| Podzois          |     4 |         0 |  0.0%  |
| Areias           |     3 |         0 |  0.0%  |

(*) Includes 54 pre-2018 "Podzolicos" labels mapped to Argissolos
via `.SIBCS_LEGACY_ORDER_MAP`. (**) Includes 44 pre-2018 "Gleis"
labels mapped to Gleissolos -- but recall remains 0%, indicating
the `glei_horizon` diagnostic is the actual gap, not the label.
(***) Includes 13 pre-2018 "Aluviais" labels mapped to Neossolos.

## Diagnostic findings (v0.9.61 / v0.9.62 priorities)

Sorted by impact on this benchmark:

1. **Gleissolos diagnostic (n_ref = 98, recall 0%)**. With the
   legacy fix in place, both "Gleissolos" (54) AND "Gleis" (44)
   references map to a single class. The classifier still
   misclassifies all of them, predominantly as Neossolos (53)
   and Argissolos (34). The `glei` test likely requires
   saturation regime conditions that the BDsolos chemistry
   export doesn't fully capture (e.g. n_d / r_d / a_d -- redoximorphic
   features in NASIS speak -- aren't ported).

2. **Latossolos diagnostic (n_ref = 114, recall 7.9%)**. Predominantly
   confused with Argissolos (24), Cambissolos (43), Neossolos (38).
   The `latossolico` B-horizon test in SiBCS Cap 18 has thresholds
   that may be too conservative -- need to revisit against
   published Embrapa manuals.

3. **Aterros / `<unmapped>` -> 0 recall**. These are non-SiBCS-5
   categories ("Aterros" = anthropogenic fill, "Solos
   Halomorficos" = orphan UI fragments). Could be filtered upstream
   rather than treated as ground truth.

## Diagnostic findings (v0.9.61 / v0.9.62 priorities)

Sorted by impact on this benchmark:

1. **Legacy-label translation in `normalise_febr_sibcs()`**.
   Adding `Podzolicos -> Argissolos`, `Gleis -> Gleissolos`,
   `Aluviais -> Neossolos Fluvicos` would lift SiBCS Order accuracy
   from 27.9% to ~33% mechanically, without touching any
   classifier. This is the obvious v0.9.61 lift.

2. **Latossolos confusion**. 114 reference Latossolos, only 9
   classified correctly. The confusion matrix shows
   24 -> Argissolos, 43 -> Cambissolos, 38 -> Neossolos, 0 -> Podzolicos,
   0 -> Latossolos. This suggests the classifier is too conservative
   on the Latossolos diagnostic horizon (`latossolico` per SiBCS Cap
   18) -- need to revisit the threshold lookup against published
   manuals.

3. **Gleissolos reform**. 54 reference Gleissolos all mis-classified,
   majority -> Neossolos. Likely the `glei` test is requiring
   conditions that the BDsolos chemistry cannot satisfy
   (saturation regime not fully captured in the export).

4. **Aterros / Solos -> 0 recall**. These are non-standard /
   pre-SiBCS-5 categories ("Aterros" = anthropogenic fill,
   "Solos" = orphan label fragments from the BDsolos UI). Could be
   filtered upstream rather than treated as ground truth.

## Run it

```bash
# Single UF (RJ, 722 perfis, ~30s):
Rscript inst/benchmarks/run_bdsolos_v0960_focused.R

# All 27 UFs (~10-20 min, uses preallocated list-of-lists):
Rscript inst/benchmarks/run_bdsolos_v0960.R
```

## Output paths

- `inst/benchmarks/reports/bdsolos_v0960_RJ_<DATE>.rds` - full result
- `inst/benchmarks/reports/bdsolos_v0960_RJ_<DATE>.txt` - text summary
