High-Performance Open-Source Archive
workflow() now supports convey package
functions (svygini, svyatk,
svyarpr, svyfgt, etc.) for poverty and
inequality estimation with proper standard errors and CVs.validate_weight() now shows available columns when
weight is not found.validate_replicate() now lists missing replicate ID
columns in the error.set_use_copy() and set_lazy_processing()
reject non-scalar logical input.step_join, step_remove,
step_rename, step_validate,
step_filter) now return invisible(out) per R
convention for side-effect functions.@rdname provenance to S3 method
documentation.stats::median and
stats::quantile imports.stats::confint() in
cat_estimation.svyratio.provenance_diff print output.survey_to_data.table() is deprecated in favor of
survey_to_datatable() (avoids non-standard dots in function
names). The old function still works with a deprecation warning.metasurvey_msg() internal helper wraps
message() behind
options(metasurvey.verbose = FALSE) for silent operation in
scripts.metasurvey/<version> User-Agent header.read_file() now gives an actionable install hint when
the required reader package is missing (e.g., haven for
.dta files).call. = FALSE to all stop() and
warning() calls for cleaner error tracebacks without
internal call stack noise.& vs && in
compute() and cat_estimation() — single
& would evaluate both sides even when the first was
FALSE.get_metadata() now uses cat() instead of
message(), making output testable with
expect_output() and suppressible with
capture.output()..onAttach() startup message (package loads
silently).api-database vignette in build (was excluded
via .Rbuildignore, causing broken cross-vignette file URI
references).visNetwork widget rendering
(eval = FALSE).\dontrun{} examples in
get_implantation() and get_follow_up() with
runnable examples using test data.provenance_to_json() and
provenance_diff().@family transpiler cross-references to all
transpiler functions.step_*
and tidy API functions.print.metasurvey_provenance_diff man page.digest from Suggests.step_filter(): new step type for row-level filtering
within the pipeline. Multiple conditions are AND-combined. Supports
.by for within-group filtering, lazy evaluation, and
RotativePanelSurvey dispatch.provenance(): lightweight data lineage tracking.
Surveys now record source file, load timestamp, initial row count, and
per-step history (type, N before/after, duration). Workflow results
carry provenance as an attribute. Includes
provenance_to_json() for audit trails and
provenance_diff() for cross-edition comparison.workflow_table(): publication-quality formatted tables
from workflow() results using gt. Features
confidence intervals, CV quality classification with color coding,
compare_by for side-by-side edition comparison,
locale-aware formatting (“en”/“es”), and provenance-based source notes.
Falls back to knitr::kable() if gt is not
installed.if (!is.null(auth_err)) to
check authentication, but require_auth() always returns
non-NULL (user object on success, error list on failure). The endpoints
returned the JWT payload immediately without ever reaching the database
write. Fixed all 5 affected endpoints (recipe star, recipe comment,
workflow star, workflow comment, delete comment) and added
result$ok checks in Shiny modules so errors are reported
instead of silently swallowed.gt (>= 0.10.0) to Suggests.step_validate(): declarative data validation step that
checks invariants without mutating data. Supports row-level logical
expressions, named checks, .min_n, and
.action = "stop" or "warn".api_star_recipe(),
api_get_recipe_stars(), api_comment_recipe(),
api_get_recipe_comments(),
api_delete_comment(),
api_get_recipe_dependents(), and workflow equivalents.GET /recipes/:id/dependents returns recipes
that depend on a given recipe via depends_on_recipes.optimize_steps() now collapses consecutive independent
step_compute() calls into single calls with dependency
analysis, reducing transpiled step counts./indicators REST endpoints
for publishing estimation results with full traceability (indicator
-> workflow -> recipe). Feature-gated via
METASURVEY_ENABLE_INDICATORS env var.POST /indicators/compute. The worker loads private
microdata, applies recipes, and runs workflow(). Deployed
as a separate Docker container, never exposes raw survey data.crayon to cli for terminal output
coloring. All crayon::bold(),
crayon::silver(), crayon::green(), etc.
replaced with cli::style_bold(),
cli::col_silver(), cli::col_green(), etc.
across 8 files.httr to httr2 for HTTP requests.
All API client, ANDA catalog, and Shiny helper functions now use the
pipe-based httr2 API
(request() |> req_headers() |> req_perform()).glue::glue_col() calls (which depended on
crayon color functions) with glue::glue() +
cli::col_*() wrappers.bit64 and jose (already unused
since 0.0.14) from Dockerfile.casen from Suggests (GitHub-only package, not
available on CRAN).pak from Suggests (no longer used in
vignettes).options(survey.lonely.psu = "adjust") for strata
with single PSUs.pak; replaced with
requireNamespace() guards.*-es.Rmd) vignettes from CRAN
tarball via .Rbuildignore (remain available on pkgdown
site).strata parameter to Survey$new(),
survey_empty(), load_survey(), and
load_panel_survey() for stratified sampling designs. Passed
to survey::svydesign(strata = ...).eval=FALSE from all vignettes — chunks now
use conditional eval or markdown code blocks for non-runnable
examples.\dontrun{} with
\donttest{} and runnable local-data examples for
cat_design, bake_recipes, recipe,
steps_to_recipe, transpile_stata,
parse_do_file, and others.Step$comments field renamed to
Step$comment. The old name still works as a legacy alias
but will be removed in a future version.workflow() first parameter renamed from
survey to svy for consistency. Positional
usage is unchanged.Survey$set_weight() no longer emits
message("Setting weight").step_recode() .name_step parameter is
deprecated; use comment instead.use_copy parameter in all step functions
(step_compute, step_recode,
step_join, step_remove,
step_rename) is deprecated in favour of .copy,
following dplyr’s dot-prefix convention for secondary arguments.has_steps(),
has_recipes(), is_baked(),
has_design().step_recode(), step_join(),
step_remove(), step_rename() no longer default
svy to survey_empty(); the survey argument is
now required.extract_time_pattern() now recognises quarter/trimester
editions ("2023-T3", "2023-Q1"), parsed as
Quarterly periodicity.validate_time_pattern() crash when
svy_edition is NA.shallow_clone() re-running
validate_time_pattern() on already-parsed editions, causing
cascading NULL → NA → crash.get_metadata() crash when edition is NULL or
NA.cat_estimation.svyby() producing wrong output:
margins index was integer (not column name), SE column "se"
was missed by grep, and cv() data.frame was not handled.
Output now shows group labels in the stat column
(e.g. svyby: x [sexo=1]).svymean + svyby)
no longer fail on rbindlist due to mismatched columns.set_engine() logic bug where
identical(.engine, show_engines()) compared string vs
vector, silently failing.stop("Error in step") in
step_compute() validation — now reports which variables
failed.step_remove() and
step_rename().bit64 import (CRAN compliance).jose dependency with
jsonlite::base64url_dec() for JWT decoding — one less hard
dependency.design_active active binding now delegates to
ensure_design(), removing duplicated design-building
logic.step_* functions now document lazy evaluation
behavior in @details.step_remove() and step_rename()
roxygen with full @details sections matching
step_compute() depth.workflow() @param svy now prominently
documents the list() requirement.Step$comment instead of
Step$comments.e31 →
e30).svy_before retention chain that prevented GC from freeing N
copies of the data.table.bake_step re-recording
step_join/step_remove/step_rename during bake, which caused duplicate
steps.bake_steps() uses shallow_clone() instead
of clone(deep=TRUE), avoiding duplication of the entire
step chain on every bake.add_step() invalidates design lazily instead of
rebuilding svydesign on every step addition.copy() calls in
step_compute and step_recode.\dontrun{} instead of
\donttest{}.RecipeWorkflow examples: corrected
recipe_ids parameter name, removed non-existent
variables parameter.add_weight/add_replicate examples:
removed double-escaped regex patterns, removed
load_survey() calls that depend on external files.set_workflow_backend/get_workflow_backend
examples runnable without \dontrun{} wrapper.main to R-CMD-check PR
triggers, replaced master with main in
test-coverage and pkgdown, removed broken
setup-r-test.yml.docker-compose.yml for local development (API + Shiny
stack).api_url() no longer falls back to a hardcoded
production URL; returns NULL when no URL is configured via option or
environment variable.\dontrun{} with
\donttest{} across all exported examples for CRAN
compliance.RecipeWorkflow,
save_workflow, read_workflow,
search_workflows, rank_workflows,
RecipeBackend.stata-transpiler-es,
ech-demographics-recipe-es.METASURVEY_REF
build arg, added jose and git
dependencies.use_copy →
metasurvey.use_copy, lazy_processing →
metasurvey.lazy_processing.set_use_copy() and set_lazy_processing()
return the previous value invisibly, matching set_engine()
behavior.set_backend() and set_workflow_backend()
return the previous value invisibly.\dontrun{} examples with runnable or
\donttest{} where appropriate (set_engine(),
view_graph(), survey_empty(),
set_workflow_backend(),
get_workflow_backend()).print.RecipeWorkflow example: removed
non-existent recipe_name and results
arguments.extract_time_pattern() roxygen block: was
accidentally attached to internal validate_monthly(),
causing the export to be dropped.@keywords tags: removed comma-separated
values in PoolSurvey and RotativePanelSurvey,
removed duplicate description block in Survey that was
parsed as keyword entries..onAttach() to a single version
message..onLoad().metasurvey-es.Rmd..Rbuildignore entries for deployment artifacts
(inst/scripts, inst/seed-data, Dockerfiles,
inst/shiny-auth.R).inst/extdata/ directory.add_weight() regex example.load_survey() example.set_lazy() → set_lazy_processing()
in vignettes.eval=TRUE on in-memory vignette chunks
(bake, get-steps).load_survey_example(): removed dead
file.exists() branch, wrapped download.file()
in tryCatch() for graceful failure.requireNamespace("htmltools") check in
view_graph().parallel in
Suggests."Trianual" → "Triennial",
"Multianual" → "Multi-year".recipe(),
extract_surveys(), get_implantation(),
get_follow_up().transpile_stata(), transpile_stata_module(),
parse_do_file(), parse_stata_labels(), and
transpile_coverage(). Converts .do files into
Recipe JSON, supporting gen/replace chains, recode, egen, foreach loops,
mvencode, destring, labels, and more.labels field to Recipe class for
storing variable and value labels from STATA transpilation.validate_weight_time_pattern() crash when
weight is NULL (triggered by shallow_clone()
on surveys without weights).steps_to_recipe() producing unparseable step
strings: long step_recode calls were split across multiple
lines by deparse(), breaking JSON round-trip.bake_recipes() not executing
step_rename and step_remove: recipe step
replay now temporarily disables lazy processing so all steps execute
immediately.category_tag() crash when recipe topic is
NULL.@family tags to all exported functions across 16
groups.@keywords tags (removed commas, fixed
multi-word entries).@title tags in
R/set_engine.R.cran-comments.md to reflect 1 NOTE (new
submission).stata-transpiler vignette covering all supported
STATA patterns.ech-demographics-recipe vignette showing
hand-crafted vs transpiled recipe workflows.stop(message(...)) which
silently called stop(NULL) instead of raising a proper
error (set_engine(), load_survey(),
recipe())..Rbuildignore double-escape bug: patterns had
\\\\. instead of \\., causing hidden files and
non-standard files to leak into the tarball.1:n sequences with
seq_len()/seq.int() to avoid
c(1, 0) when n=0 (7 occurrences across survey.R,
workflow.R, steps.R).set_data() definition in
survey.R.crayon::red()/crayon::green() in
cat_design() for cross-platform compatibility.load_survey_example().requireNamespace("parallel") now uses
quietly = TRUE and checks the return value before using the
package.requireNamespace("rio") return value is now checked
before calling rio::convert().ssl_verifypeer = FALSE; now
user-controllable via
options(metasurvey.ssl_verify = FALSE).api_logout() no longer calls
Sys.unsetenv("METASURVEY_TOKEN") — only clears the R
option, per CRAN policy on environment variables.set_engine(), set_use_copy(),
set_lazy_processing(), and configure_api() now
return the previous value invisibly, allowing users to restore global
options.evaluate_cv() returns English labels (“Excellent”,
“Good”, etc.).extract_time_pattern() returns “Invalid
format”/“Unknown format”.inst/CITATION updated: removed deprecated
citHeader(), dynamic year.Recipe$to_list() and
RecipeWorkflow$to_list() now include
metasurvey_version for reproducibility tracking.bake_step() crash when executing
step_recode(): removed invalid record=FALSE
argument passed to internal recode().bake_steps_survey() iterating over the original
survey’s steps instead of the cloned copy, causing inconsistent
state.bake_recipes() environment leakage: recipe step
evaluation now uses an isolated new.env() instead of the
calling environment().copy() in compute() and
recode() — the shallow_clone() already handles
data copying.step_join() now uses merge.data.table()
directly and set_data() instead of assigning to
$data.observe→lapply→observeEvent pattern with delegated event
handlers.invalidateLater(0) from admin panel
refresh.store_token() no longer calls Sys.setenv()
— API tokens are stored in R options only, not leaked to environment
variables.token_expires_soon() returns TRUE for
NULL, malformed, or missing exp claims,
forcing token refresh instead of using potentially invalid tokens.validate_api_id() with alphanumeric whitelist for
recipe/workflow IDs.api_register().api_download_recipe() and
api_download_workflow() now warn on failure instead of
silently swallowing errors.debounce(300) on Shiny recipe search input to
reduce excessive reactivity.sapply() with vapply() or
lapply() across the codebase for type-safe return
values.set_engine() uses requireNamespace()
instead of eval(install.packages(...)).read_recipe() now warns when step parsing falls back to
raw strings.@return
and @examples roxygen2 tags, meeting rOpenSci documentation
requirements.@return to R6 class exports: Recipe,
RecipeWorkflow, RecipeCategory,
RecipeCertification, RecipeUser,
PoolSurvey, RotativePanelSurvey, and
Survey.@examples to set_engine(),
show_engines(), get_engine(), and all
API/backend functions.@return to get_metadata(),
set_lazy_processing(), set_use_copy(),
set_backend(), set_workflow_backend(), and
explore_recipes().covr::codecov() with covr::package_coverage()
+ codecov/codecov-action@v5 for reliable coverage uploads
with token authentication..Rprofile to skip example-data setup
in GitHub Actions, preventing potential download failures during CI
builds.archive, haven, and
openxlsx to Suggests in DESCRIPTION, fixing R
CMD check warnings about undeclared dependencies..claude/ directory with development agents and
commands for code review, testing, documentation auditing, and rOpenSci
preparation.AST engine removed: The Abstract Syntax Tree
evaluation engine (R/ast.R) has been replaced by direct
data.table expression evaluation. This simplifies the step
pipeline and removes ~760 lines of internal code. Users who relied on
internal AST functions (parse_ast,
evaluate_ast_node, recode_with_ast,
optimize_node) will need to update their code. The public
API (step_compute(), step_recode(),
bake_steps()) remains unchanged.
API client migrated to JWT: The API client
(R/api_client.R) now uses JWT authentication with a
self-hosted Plumber backend, replacing the previous Atlas Data API
approach.
Lazy design initialization:
Survey$new() no longer builds svydesign
objects at construction time. Designs are created on-demand when needed
(e.g., during workflow() or bake_steps()),
significantly improving construction performance for large datasets. The
psu parameter is now optional (default
NULL).
Recipe ecosystem: New infrastructure for publishing, discovering, and managing reproducible survey processing recipes:
RecipeRegistry R6 class with pluggable backends (JSON
file, MongoDB).RecipeCategory for hierarchical categorization of
recipes.RecipeCertification for quality certification levels
(official, reviewed, community).RecipeUser for author/maintainer profiles with
institutional affiliations.list_recipes(),
search_recipes(), filter_recipes(),
rank_recipes().Recipe$doc() for auto-generated documentation.Recipe$validate(svy) for dependency checking against a
survey.print.Recipe S3 method for human-readable output.save_recipe() / read_recipe() for JSON
serialization.Workflow system: New estimation workflow infrastructure:
RecipeWorkflow R6 class capturing
svymean/svytotal/svyratio/svyby
calls with recipe references and reproducibility metadata.WorkflowRegistry with pluggable backends (JSON,
MongoDB)..capture_workflow() when surveys contain recipes.workflow_from_list() for reconstructing workflows from
JSON/API responses.list_workflows(),
search_workflows(), filter_workflows(),
rank_workflows(), find_workflows_for_recipe(),
publish_workflow().save_workflow() / read_workflow() for
serialization.REST API (inst/api/plumber.R):
Self-hosted Plumber API with JWT authentication for recipe and workflow
publishing. Endpoints for CRUD operations on recipes, workflows, and
user management. Includes Docker deployment configuration.
ANDA catalog integration
(R/anda.R): Functions for accessing INE Uruguay’s ANDA5
catalog metadata:
anda_catalog_search() for searching the catalog.anda_list_editions() for listing available survey
editions.anda_fetch_ddi() for fetching DDI metadata.anda_parse_variables() for extracting variable
definitions.anda_variable_detail() for detailed variable
information.Shiny explorer app (inst/shiny/):
Interactive recipe and workflow browser launched via
explore_recipes():
bake_recipes() handles string
steps: Steps stored as strings (from JSON/API) are now parsed
to call objects before evaluation, fixing the previous “attempt to apply
non-function” error.
is_blank() now handles NULL and
zero-length inputs without error.extract_time_pattern(): type prefix
detection now correctly matches only at string start."2019_2021") are
now correctly parsed when separated by underscore.validate_time_pattern() gracefully handles
NULL/empty edition strings.public_key() error handling with informative
messages.workflow_from_list() is now exported (was
@keywords internal), fixing Shiny app workflow parsing from
API responses.eval=TRUE examples:
getting-started: Survey creation, steps, recipes, and
workflows.recipes: Recipe ecosystem, publishing, and
discovery.workflows-and-estimation: Estimation workflows and
registry.complex-designs: Replicate weights, stratified
designs.panel-analysis: Rotating panel surveys.ech-case-study: Full ECH processing example.shiny-explorer: Interactive app usage guide.api-database: REST API and database architecture.*-es.Rmd).references.bib for academic citations.pkgdown site configuration with rOpenSci
template.step_join() for merging external reference data
into surveys.step_remove() and step_rename() for
variable management.RotativePanelSurvey and PoolSurvey
R6 classes for complex designs.extract_surveys(),
get_implantation(), get_follow_up().
Need mirroring services?
Contact our team at info@vpspulse.com.
Mirror powered by VPSpulse
Infrastructure sponsored by VPSPulse & Secure Payments by ArionPay.