--- title: "The config_ambre database" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{The config_ambre database} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", message = FALSE, warning = FALSE, fig.width = 7, fig.height = 4.5, out.width = "100%" ) ``` ```{r setup} library(ambre) set.seed(2024) ``` ```{r attach-dplyr} library(dplyr) ``` Every risk figure `ambre` produces is ultimately read out of one bundled object: `config_ambre`. It is the *knowledge base* of the package -- pathogen concentrations, dose-response models, log-reduction credits for treatment and field barriers, exposure paths, crops and costs, all in one nested list. When you run `create_scenario()`, a full copy of `config_ambre` is embedded in each row of the scenario and mutated in place as the pipeline runs (see `vignette("a-get-started", package = "ambre")`). This vignette is the practical reference for that object: what is inside it, how the tables join, and how you would adapt it to your own case. ## Overview: 8 slots, 16 CSVs `config_ambre` has **eight** top-level slots. Three of them (`treatment`, `path`, `economic`) are themselves lists of sub-tables, so the object holds **16 tables** in total. Each table is assembled from one CSV in `data-raw/` by the build script `data-raw/config_ambre.R`. ```{r overview-str} str(config_ambre, max.level = 1) ``` The sub-tables live under the three list slots: ```{r overview-subtables} lapply(config_ambre[c("treatment", "path", "economic")], names) ``` Here is the full file-to-slot mapping, read straight from `data-raw/config_ambre.R`. This is the map you need when you want to edit the source data: ```{r mapping-table} mapping <- tibble::tribble( ~`CSV in data-raw/`, ~`Slot in config_ambre`, "exposure.csv", "exposure", "ambre_pathogene.csv", "inflow", "ambre_barriere_general.csv", "treatment$processes", "treatment_schemes.csv", "treatment$schemes", "ambre_barriere_voie.csv", "treatment$barriere_path", "ambre_barriere_specifique.csv", "treatment$barriere_specific", "ambre_barriere_deperissement.csv", "treatment$barriere_decay", "ambre_doseresponse.csv", "doseresponse", "ambre_voie_description.csv", "path$description", "ambre_voie_frequence.csv", "path$frequency", "ambre_voie_volume.csv", "path$volume", "ambre_culture_hauteur.csv", "crop", "ambre_sante.csv", "health", "ambre_culture_eau.csv", "economic$water_need", "ambre_population.csv", "economic$population", "ambre_barriere_cout.csv", "economic$cost" ) knitr::kable(mapping) ``` In one sentence per slot: `exposure` holds the Monte-Carlo controls (how many runs, how many events, the per-event volume); `inflow` is the raw pathogen concentration in the treated wastewater; `treatment` carries the log-reduction credits; `doseresponse` and `health` turn a dose into an infection, an illness and a DALY; `path` describes the 19 exposure routes; `crop` and `economic` support the agronomic and costing calculations. ## The entity-relation keys The tables are relational: they share a small set of ID columns and are joined on them during the pipeline. Knowing the keys is what lets you cross-reference (and safely extend) the database. | Key | Meaning | Appears in | |-----|---------|------------| | `PathogenID` / `PathogenGroup` | one of 38 pathogens (Bacteria / Viruses / Protozoa) | `inflow`, `doseresponse`, `health`, `treatment$processes` | | `TreatmentID` | one barrier (treatment step or field practice) | `treatment$*`, `economic$cost` | | `PathID` | one of 19 exposure routes | `path$description`, `path$frequency`, `path$volume` | | `PopulationID` | one of 6 exposed populations | `path$description`, `economic$population`, `barriere_*` | | `MatrixID` | the environmental compartment (Water / Air / Plant / Soil / Product) | `path$description`, `barriere_path` | | `CropID` / `CropHeight` | one of 6 crops and its height above the irrigation system | `crop`, `economic$water_need` | `path$description` is the hub that ties a route to *who* is exposed and to *which* matrix: ```{r er-path} config_ambre$path$description |> select(PathID, PopulationName, MatrixName, PathDescription) |> head(4) ``` And `PathogenID` is what links a pathogen's concentration to its health burden: ```{r er-pathogen} config_ambre$health |> select(PathogenID, PathogenName, dalys_per_case) |> inner_join( config_ambre$inflow |> select(PathogenID, PathogenGroup), by = "PathogenID" ) |> head(4) ``` ## The barrier families The multi-barrier philosophy of `ambre` rests on the PhD thesis of Alice-Rose Thomas (INSA Lyon, 2024), carried out in the Multiware research context, which catalogues up to **49 candidate log-reduction barriers** covering both the treatment plant and the field. `config_ambre$treatment$processes` ships a curated subset of that catalogue. Every barrier has a `TreatmentName` prefixed by its type: * **`Q.`** -- *Quality*: the wastewater-treatment process (Activated Sludge, Maturation Pond, UV Reactor, Chlorination, ...). * **`E.`** -- *Equipment*: on-field hardware (drip irrigation, signage, fences, personal protective equipment, ...). * **`P.`** -- *Practices*: cultivation and irrigation practices (non-edible crop, night-time irrigation, cooking, peeling, washing, ...). The shipped `TreatmentGroup` labels and the number of distinct barriers in each family are: ```{r barrier-groups} config_ambre$treatment$processes |> distinct(TreatmentID, TreatmentName, TreatmentGroup) |> count(TreatmentGroup, name = "n_barriers") ``` ```{r barrier-prefix} config_ambre$treatment$processes |> distinct(TreatmentID, TreatmentName) |> mutate(family = sub("[.].*", "", TreatmentName)) |> count(family, name = "n_barriers") ``` So the bundled database currently encodes 28 distinct barriers across three families -- a subset of the PhD's 49, and an intentionally extensible one. Credits from several barriers combine by simple **log-additivity** when a scenario stacks a treatment scheme and a field barrier (`vignette("b-treatment-vs-multibarrier", package = "ambre")`). ## Anatomy of a distribution Eight of the sixteen tables (`exposure`, `inflow`, `treatment$processes`, the three `barriere_*` tables, `path$frequency` and `path$volume`) share the same nine-column **distribution block**: ``` type | value | min | max | mode | mean | sd | meanlog | sdlog ``` This is what makes `ambre` a Monte-Carlo engine: instead of a single number, a barrier or a concentration is a *distribution* that gets resampled on every run. The `type` column decides which of the other columns are actually read: ```{r dist-legend} knitr::kable(tibble::tribble( ~type, ~`columns read`, ~draws, "value", "value", "a constant, repeated", "uniform", "min, max", "runif(min, max)", "triangle", "min, max, mode", "EnvStats::rtri(min, max, mode)", "norm", "mean, sd", "rnorm(mean, sd)", "lognorm", "meanlog, sdlog", "rlnorm(meanlog, sdlog)", "log10_uniform", "min, max", "10^runif(min, max)", "log10_norm", "mean, sd", "10^rnorm(mean, sd)" )) ``` Here is a real barrier row. `Q.1 - Activated Sludge` removes a `uniform` 0.5-to-1 log of protozoa; every other distribution column is `NA` because a uniform draw only needs `min` and `max`: ```{r dist-row-barrier} config_ambre$treatment$processes |> filter(TreatmentName == "Q.1 - Activated Sludge") |> select(TreatmentName, PathogenGroup, type, value, min, max, mode) ``` And the matching inflow row: the raw *Campylobacter jejuni* concentration is a `uniform` draw between 100 and 5000 organisms per litre: ```{r dist-row-inflow} config_ambre$inflow |> filter(PathogenName == "Campylobacter jejuni") |> select(PathogenName, PathogenGroup, type, value, min, max) ``` The `exposure` slot uses the same block for the sampling controls themselves -- 1000 runs, 365 events, and a triangular per-event volume: ```{r dist-exposure} config_ambre$exposure ``` How these rows are turned into 1000 x 365 random values is the subject of `vignette("g-monte-carlo-engine", package = "ambre")`, via `create_random_distribution()` and `generate_random_values()`. ## Customizing the database for your case Because the object is rebuilt from plain CSVs, adapting it is a three-step loop: **edit the right CSV, rebuild, verify**. 1. **Edit** the CSV that maps to the slot you want to change (see the mapping table above). Add a *coherent* row -- a fresh `TreatmentID`, a `type` from the legend above, and the columns that `type` requires. 2. **Rebuild** by re-running `data-raw/config_ambre.R`, which re-reads the 16 CSVs and writes `data/config_ambre.rda`. 3. **Verify** the new entry is reachable with the `query_*` helpers. The chunk below writes to `data-raw/` and rebuilds the dataset, so it is shown for reference only (`eval = FALSE`): ```{r customize, eval = FALSE} # 1. add a new quality barrier to the treatment CSV csv_path <- file.path("data-raw", "ambre_barriere_general.csv") new_barrier <- data.frame( TreatmentID = 99, TreatmentName = "Q.8 - Ozonation", TreatmentGroup = "Quality", PathogenGroup = "Bacteria", type = "uniform", value = NA, min = 2, max = 4, mode = NA, mean = NA, sd = NA, meanlog = NA, sdlog = NA ) readr::write_csv( rbind(readr::read_csv(csv_path), new_barrier), csv_path ) # 2. rebuild data/config_ambre.rda from the 16 CSVs source(file.path("data-raw", "config_ambre.R")) # 3. verify the new barrier resolves to its ID query_barrier(barrierName = "Q.8 - Ozonation") ``` The verification helpers each translate a *name* into the *ID* the engine works with (and error early if the name is unknown), so they are the quickest way to confirm an edit landed. They run against the live `config_ambre`: ```{r query-helpers} query_pathogen(pathogenName = "Campylobacter jejuni") query_barrier(barrierName = "Q.3 - UV Reactor") query_crop(cropName = "Tomato") ``` `query_volume()` and `query_frequency()` take a `PathID` instead and return the per-path `min`/`max` block that gets injected into a scenario's exposure (this is what `update_volume()` / `update_frequency()` do automatically inside the pipeline): ```{r query-path} query_volume(pathID = 1) query_frequency(pathID = 1) ``` ## Known pitfalls A few sharp edges to keep in mind before you rely on or extend the database. **The tables are `spec_tbl_df`.** Because they are read with `readr`, each slot carries a `readr` column specification and an extra class: ```{r pitfall-class} class(config_ambre$crop) ``` This is harmless -- `dplyr` verbs work as usual -- but if you compare tables with `identical()` or write your own I/O you may trip over the `spec` attribute. Filtering or mutating usually drops it. **`economic$cost` is the least polished table.** `mean` / `meanlog` are currently all `NA`, so a normally- or lognormally-distributed cost you add expecting `mean`/`meanlog` will not be picked up, see `vignette("g-monte-carlo-engine", package = "ambre")`. **`CropHeight` is a signed height in centimetres** above the irrigation outlet: `-1` means underground (Onion), `0` means at ground level (Potato, Salad), and positive values mean the edible part sits above the spray (Tomato 25, Corn seed / Apple 50). It is the intended key for height-dependent barrier credits. ```{r pitfall-crop} config_ambre$crop |> select(CropName, CropHeight) ``` ## See also * `vignette("a-get-started", package = "ambre")` -- how `config_ambre` is embedded in a scenario and mutated through the pipeline. * `vignette("g-monte-carlo-engine", package = "ambre")` -- how the distribution block is sampled into 1000 x 365 values. * `vignette("c-input-file", package = "ambre")` -- the Excel catalogues (crops, populations, paths, barriers) whose allowed values come from these tables. * `vignette("e-economic-analysis", package = "ambre")` -- how `economic$cost`, `water_need` and `population` drive the costing.