Processing and analysis

Inputs and outputs

make_data.jl is a delivery-specific template. Supply one DataFrame per wave and a corresponding wave vector, in the same order. The RCT cleaners combine wave tables after harmonizing columns; uncollected variables can remain missing. Missingness alone does not establish whether a question was asked: consult the study-specific codebook.

using HondurasTools
waves = [1, 2, 3]
paths = ["raw/respondents_w1.csv", "raw/respondents_w2.csv", "raw/respondents_w3.csv"]
raw = [CSV.read(path, DataFrame; missingstring="NA") for path in paths]
resp = clean_respondent(raw, waves; nokeymiss=false, onlycomplete=false)
subset!(resp, :data_source => ByRow(==("Census, survey")))

Paths above are illustrative. The template also processes household, village, connection, and microbiome tables and writes one dated CSV per level. It does not automatically join those outputs.

Cleaning defaults

nokeymiss=true in clean_respondent removes rows missing village code, gender, date of birth, or building ID. onlycomplete=true restricts to completed surveys. Neither is enabled in the template.

For connections, same_village=true drops missing same-village indicators and keeps code 1. alter_source=true keeps raw code 1. removemissing=true calls disallowmissing! on ego, alter, question, alter source, and relationship: it rejects remaining missing values rather than dropping all incomplete rows. Required columns must still be present even when filtering is disabled. The cleaner converts source codes to labels after filtering.

Some recoding operations check whether variables exist, but other operations access columns unconditionally. Support for arbitrary subsets of requested variables is not guaranteed.

Identifiers and joins

Cleaned respondent IDs use name; household processing uses building_id. Do not assume a building is a stable household across waves. Include wave and village identifiers where the intended unit requires them, and check key uniqueness before joining. The respondent cleaner removes the legacy household_id column.

Microbiome data are separate from the RCT wave tables. Choose and document how to handle changed villages between administrations before joining on person and village. The package includes codebook/microbiome_villages.csv; study-specific sample restrictions remain analysis decisions.

An edgelist omits people without recorded ties. Construct the vertex population from the appropriate respondent roster when isolates matter, and apply compatible inclusion criteria to both tables.

Further analysis

Longitudinal utilities include respprocess, householdprocess, and villageprocess, with respwave, hhwave, and villwave for wave-specific outputs. Their carry-forward and variable-selection choices require review for the analysis; do not assume every variable is time-invariant.

Network utilities include jointnetwork, addsymmetric!, tiestrength, constraint, and neighbor-data helpers. Inspect their API documentation and required graph/table structure before use. The old src/process_edgelist.jl file is not included by the module and is not a supported exported entry point.

Validation and partial responses

Donation amounts in microbiome data accept numeric integers and integer strings. Missing or unparseable text becomes missing; fractional numeric amounts raise an error. The donation column may be absent.

Outcome treatment flags preserve missing values and require nonmissing codes to be 0 or 1. Known OCR errors in outcome classifications are normalized by the codebook loader; unknown classifications raise an error rather than silently skipping recoding.

For a terminal risk response without its preceding-stage response, the terminal risk score is retained while the unavailable green and purple scores remain missing. This does not reconstruct unobserved choices.

Repeated HMB administrations

The two-table call still means cohort 1 baseline and cohort 2. For follow-up data, specify the codebook administration IDs explicitly:

mb = clean_microbiome([baseline, followup, cohort2], [1, 2, 3])

Outputs retain cohort and administration. Person/cohort/administration is the observation key, including for risk scoring and its join back to survey data. Repeated people across administrations are retained; duplicate observations within an administration raise an error. Inputs to this cleaner are copied. Missing identifiers are dropped with nokeymiss=true; otherwise records remain, but missing person identifiers are not assigned risk scores. Do not join repeated observations on person alone.

Input contracts

Wave-based cleaners require a nonempty table list, a matching vector of distinct integer RCT waves 1–4, and structural columns in each table. Wave suffixes may be present on required columns where the cleaner supports suffix removal. Missing required columns raise an ArgumentError naming the cleaner and columns.

These checks validate column availability, not every value or every conditional dependency. A present optional field can still require a companion field (for example, HMB cognitive score and cognitive status).

Optional perception and norms processing

clean_perception! is a separate analysis transformation and is not invoked by the main cleaning template. See Perception and norms for its input assumptions, measures, and reasons to keep it opt-in pending further validation.