High-Performance Open-Source Archive
Softplus reparameterization replaces hard-clamp
positivity enforcement. Parameters in
parts$positive (formerly parts$T and
parts$P) are now handled internally via
theta_i = softplus(z_i), guaranteeing strict positivity
analytically. theta0[positive] must now be strictly
> 0; v1 silently clamped zero or negative starting
values, v2 raises an error.
parts API simplified. The preferred
form is now list(positive = ..., euclidean = ...). The
three-way {T, P, E} form from v1 remains accepted as a
backwards-compatibility alias (T and P indices both map to
positive; E maps to euclidean).
Relative function-stall detection
(FUNC_STALL_REL). In addition to the existing absolute
stall criterion (FUNC_STALL_ABS), convergence is now also
declared when |ΔV| / (|V| + ε) < tol_f_rel over five
consecutive iterations. New control parameter tol_f_rel
(default 1e-9).
rho-based trust-region adaptation. The trust
radius now adapts using the ratio of actual to predicted objective
reduction (rho), with configurable rho_shrink
(default 0.25), rho_grow (default 0.75),
shrink_factor (default 0.5), and grow_factor
(default 1.5). The v1 binary accept/reject heuristic is used as a
fallback when rho is not finite.
Richer per-iteration history.
fit$history now contains ten columns (was eight):
iter, f, g_inf,
step_norm, df, eta,
method, armijo_iters (new),
pred_red (new), rho
(new). The Lyapunov-specific columns
(delta_V_rel, lyapunov_ok) from v1 are
removed.
galahad_numgrad() — exported
finite-difference gradient helper for use when analytical gradients are
unavailable.
galahad_parts() — exported
constructor that validates and assembles the parts argument
with an optional total-parameter-count check.
rho_accept, use_rho_accept control
options enable an alternative acceptance mode based on trust-region
quality ratio instead of Armijo decrease.
| Parameter | v1 default | v2 default | Notes |
|---|---|---|---|
delta_min |
0.1 | 1e-6 | Allows finer convergence near zero |
delta_max |
100.0 | 1e3 | Slightly wider upper bound |
tol_f_rel |
(absent) | 1e-9 | New relative stall criterion |
armijo_max |
20 (hard) | 20 (in control) |
Now user-configurable |
Halpern averaging removed. The per-iteration
Halpern candidate
alpha_k * theta0 + (1 - alpha_k) * theta_proj is no longer
evaluated. This saves approximately one function evaluation per
iteration; the function-stall detection provides equivalent convergence
assurance on the smooth objectives for which GALAHAD is
designed.
Lyapunov violation tracking removed
(lyapunov_violations, monotone_descent from
diagnostics). The rho history in
fit$history$rho provides equivalent information about step
quality.
make_safe_function() wrappers
removed. Errors in V or gradV now surface
immediately via galahad_assert_finite(). Callers are
expected to supply well-defined objectives; the previous silent-fallback
behaviour could mask model specification bugs.
grad_clip_norm control option
removed.
eps_safe control option removed (no
longer applicable with softplus).
st$z bare expression in the internal
state-update helper that had no effect but generated lint warnings.Polyak (1969) corrected. Version 1 incorrectly cited the paper “Minimization of unsmooth functionals”, Vol. 9(3), pp. 14–29. The correct paper implemented in GALAHAD is “The conjugate gradient method in extremal problems”, Vol. 9(4), pp. 94–112. doi:10.1016/0041-5553(69)90035-4
Xu and An (2024) added. Citation for the regularized Barzilai-Borwein trust-region extension was absent from v1 despite the method being used. doi:10.48550/arXiv.2409.14383
Dugas et al. (2009) added. Citation for the softplus activation function was absent from v1 despite softplus being central to the positivity geometry. Journal of Machine Learning Research, 10(42), 1239–1262. https://www.jmlr.org/papers/v10/dugas09a.html
Need mirroring services?
Contact our team at info@vpspulse.com.
Mirror powered by VPSpulse
Infrastructure sponsored by VPSPulse & Secure Payments by ArionPay.