# confval > Span-first configuration parsing, validation, and lowering for Rust This file contains all documentation content in a single document following the llmstxt.org standard. ## Contributing ## Just recipes Various just recipes are available, but these are the most useful: - `just docs`: run the docs site locally - `just validate`: Test everything (lint, unit tests, etc.) ## Design confval is built around five design decisions. These design decisions should be adhered to. #### Spans travel with values. - Every parsed value carries the byte range it came from. - Any later check resolves that range to a line and column in the source file. #### All errors are collected and displayed to an operator. - Parsing and validation append to a shared report instead of returning on the first error. - The caller fixes one batch of problems rather than rerunning to find the next one. #### Validation happens in stages. - Parsing checks shape only, meaning the field exists and has the right type. - Range checks, closed sets, and cross-field rules run after parsing, in plain validation functions. #### The core does not know any file format. - Parsing produces a format-neutral field model. - A frontend converts one syntax into that model. - HCL and TOML ship today, each behind its own feature, and a new format is another frontend over the same model. #### The core has no required dependencies. serde, owo-colors, hcl-edit, toml_edit, and the derive macros are each behind a feature flag. confval aims to stay free of required dependencies. Put any new dependency behind a feature flag. ## Crate layout confval is organized into four layers, each a module, plus a prelude. The dependency direction is strictly downward: `format` builds on `pipeline`, which builds on `diagnostic`, which builds on `source`. | Module | Holds | |-----------------------|---------------------------------------------------------------------------------------------| | `confval::source` | `Located`, `Span`, `SourceId`, `Source`, `SourceMap` (the "where") | | `confval::diagnostic` | `Report`, `Issue`, `IssueBuilder`, `Severity`, the renderers (the "what") | | `confval::pipeline` | `Lower`, `LowerAuto`, `Validate`, `narrow`, `RangeConstraint`, `KeywordSet` (the transform) | | `confval::format` | the neutral field model (`field`) and the frontends (`hcl`, `toml`) | | `confval::prelude` | a glob re-export of the common imports across those layers | `use confval::prelude::*;` pulls the everyday names (`Located`, `Span`, `Report`, `Lower`, `Validate`, `narrow`, `RangeConstraint`, `KeywordSet`, and the derives) in one line. --- ## Getting Started confval is a Rust crate for parsing, validating, and lowering configuration files. It records a source span for every parsed value, so a validation error can report the line and column in the file the value came from. Use it to build the configuration layer of an application. You define the shape of the config as Rust types, parse a file into those types, run validation, and lower the result into the runtime types the rest of the program uses. ## Installation Add confval to your `Cargo.toml`. The crate has no default features. Enable the format frontends and extras you use. For example, for TOML format, derive macros, JSON diagnostics, and console color support: ```shell cargo add confval --features "toml,derive,serde,color" ``` Or, the HCL format, derive macros, and plain output: ```shell cargo add confval --features "hcl,derive" ``` ### Feature flags | Flag | Default | Brings in | Enables | |----------|---------|------------------|------------------------------------------------------------| | `serde` | off | `serde` | `Located` serde impls, `render_json` | | `color` | off | `owo-colors` | `render_pretty` with ANSI color | | `hcl` | off | `hcl-edit` | The `confval::format::hcl` frontend | | `toml` | off | `toml_edit` | The `confval::format::toml` frontend | | `derive` | off | `confval-derive` | `#[derive(Spec)]` and `#[derive(Config)]` (format-neutral) | Frontends (that define the configuration format) are independent opt-ins. Pick `hcl` or `toml` for the format you want. The `derive` feature emits the format-neutral `FromFields`, so it brings in no parser on its own. ## A complete example This example parses an HCL document, validates it, checks the report for errors, and lowers the validated spec into a runtime config. The crate ships the same program as multiple runnable examples. `hcl.rs` and `toml.rs` each supply a source document and one parse call, and both pull everything after parsing from a shared `common/mod.rs`. Read through it once for the overall shape. The walkthrough below steps through it piece by piece, following the [pipeline contract](pipeline.md) in order. ```rust use confval::prelude::*; range_constraint!(PORT, i64, min: 1, max: 65535); range_constraint!(WORKERS, i64, min: 1, max: 512); range_constraint!(MAX_BODY_MB, i64, min: 1, max: 1024); const LIMIT_MODES: [&str; 3] = ["enforce", "log", "off"]; #[derive(confval::Spec)] struct ServerSpec { hostname: Located, port: Located, #[confval(default = 4)] workers: Located, #[confval(nested)] limits: Option>, } #[derive(confval::Spec)] struct LimitsSpec { #[confval(default = 16)] max_body_mb: Located, #[confval(default = "enforce".to_string())] mode: Located, } impl Default for LimitsSpec { fn default() -> Self { Self { max_body_mb: Located::detached(16), mode: Located::detached("enforce".to_string()), } } } impl Validate for LimitsSpec { fn validate(&self, report: &mut Report) { MAX_BODY_MB.check_located(&self.max_body_mb, "max_body_mb", report); KeywordSet::new(&LIMIT_MODES).check_located(&self.mode, "mode", report); } } impl Validate for ServerSpec { fn validate(&self, report: &mut Report) { PORT.check_located(&self.port, "port", report); WORKERS.check_located(&self.workers, "workers", report); if self.hostname.value.is_empty() { report .error("hostname must not be empty") .at(self.hostname.span) .help("Set hostname to a reachable address, e.g. \"127.0.0.1\".") .emit(); } } } #[derive(confval::Config)] #[confval(lower_from = ServerSpec)] struct ServerConfig { hostname: String, #[confval(lower(from = port, with = narrow::i64_to_u16))] port: u16, #[confval(lower(from = workers, with = workers_to_usize))] workers: usize, #[confval(nested, default)] limits: LimitsConfig, } #[derive(confval::Config)] #[confval(lower_from = LimitsSpec)] struct LimitsConfig { #[confval(lower(from = max_body_mb, with = narrow::i64_to_u16))] max_body_mb: u16, mode: String, } fn workers_to_usize(value: &Located, _report: &mut Report) -> Option { // Safe: the range was validated and lowering only runs on a clean report. Some(value.value as usize) } fn main() { let input = r#"hostname = "127.0.0.1" port = 8080 limits { mode = "log" } "#; let mut sources = SourceMap::new(); let mut report = Report::new(); let id = sources.add("server.hcl", input); let spec: Option = confval::format::hcl::parse_hcl(&sources, id, &mut report); if let Some(spec) = &spec { spec.validate_all(&mut report); } if report.has_errors() { let mut out = String::new(); report.render_pretty(&sources, &mut out).unwrap(); eprint!("{out}"); std::process::exit(1); } let spec = spec.expect("parse returned None without reporting an error"); let config = ServerConfig::lower(&spec, &mut report).expect("validated config lowers"); println!( "listening on {}:{} with {} workers", config.hostname, config.port, config.workers ); } ``` ## Walking through the example The code breaks into four parts: the spec types you parse into, the validation rules, the config types you lower to, and the `main` that ties them together. Each part lines up with a phase of the [pipeline contract](pipeline.md). ### The spec types When a file is parsed, it is deserialized into a spec type. Every field is wrapped in a [`Located`](./guide/parsing.md#located-values), which links the value back to where it came from in the configuration file. That location is called a `span`. ```rust #[derive(confval::Spec)] struct ServerSpec { hostname: Located, port: Located, #[confval(default = 4)] workers: Located, #[confval(nested)] limits: Option>, } ``` `#[derive(confval::Spec)]` writes the parser for the struct. It only checks structure: whether each field is present and has the right type. Anything else is reported. - `hostname` and `port` are required. Leave one out and it comes back as an error. - `#[confval(default = 4)]` gives `workers` a value of `4` when the file omits it, so it is not reported as missing. - `#[confval(nested)]` parses `limits` with its own `LimitsSpec` parser, and `Option` makes the whole block optional. Spec fields use the loosest type that still parses, like `i64` for `port`. A `port` of `99999` parses fine here. Validation catches it later, and points at its span. `LimitsSpec` works the same way. ```rust #[derive(confval::Spec)] struct LimitsSpec { #[confval(default = 16)] max_body_mb: Located, #[confval(default = "enforce".to_string())] mode: Located, } ``` The defaults are declared twice. The `#[confval(default = ...)]` attributes fill a field the file left out of a `limits` block that is present. The example input writes `limits { mode = "log" }`, so `max_body_mb` comes from its attribute default. The handwritten `impl Default` supplies the whole struct when the block is absent entirely. That is what `#[confval(nested, default)]` on `LimitsConfig` uses during lowering. Nothing checks that the two agree. Keeping `16` and `"enforce"` in step across both is manual. The flexibility is deliberate, since confval cannot account for every use case. See [Parsing](./guide/parsing.md#optional-fields-and-defaults) for the full set of field rules. ### The validation rules After parsing, validation checks what the values mean: ranges, allowed keywords, and rules that cross more than one field. Each field already carries its span, so every message it reports points back at the file. Each spec type checks its own fields in a `Validate` impl. ```rust range_constraint!(PORT, i64, min: 1, max: 65535); const LIMIT_MODES: [&str; 3] = ["enforce", "log", "off"]; impl Validate for ServerSpec { fn validate(&self, report: &mut Report) { PORT.check_located(&self.port, "port", report); if self.hostname.value.is_empty() { report .error("hostname must not be empty") .at(self.hostname.span) .help("Set hostname to a reachable address, e.g. \"127.0.0.1\".") .emit(); } } } ``` - `range_constraint!` declares a numeric bound once. `PORT.check_located(&self.port, "port", report)` flags `port` at its span when it falls outside the range. - [`KeywordSet`](./guide/validation.md#keywordset) checks a value against a fixed set of allowed strings. `LimitsSpec` uses it for `mode` in its own impl. - For anything else, go through the report builder: `.at(span)` attaches the location, `.help(text)` adds a suggestion, and `.emit()` records the issue. Implementing `Validate` is not optional. Every generated `Lower` impl is bound on it, so a spec with no validator makes its config fail to compile. A spec type with nothing to check writes an empty impl. A `Validate` impl only sees its own fields. Something has to walk into the nested `limits` block. `validate_all` does that walk. The traversal is generated for you, but you still have to call `validate_all`: ```rust spec.validate_all(&mut report); ``` `validate_all` runs `ServerSpec`'s own rules and then descends into every `#[confval(nested)]` field, recursively. The one call above therefore also reaches `LimitsSpec`. `#[derive(Spec)]` writes that descent from the struct definition. A nested block added next month is validated without anyone editing a validator. Implement `validate`. Call `validate_all`. [Validation](./guide/validation.md#validate-impl-contains-the-rules-validate_all-runs-them) covers the difference. [Where a rule lives](./guide/validation.md#where-a-rule-lives) covers which rules belong in an impl rather than in a validator function. Validation never stops at the first problem. It adds every issue it finds to the report, so one run shows you all of them. ### The config types Lowering turns a validated spec into a config type, the form your program runs on. `#[derive(confval::Config)]` writes that step from the spec named in `lower_from`. ```rust #[derive(confval::Config)] #[confval(lower_from = ServerSpec)] struct ServerConfig { hostname: String, #[confval(lower(from = port, with = narrow::i64_to_u16))] port: u16, #[confval(lower(from = workers, with = workers_to_usize))] workers: usize, #[confval(nested, default)] limits: LimitsConfig, } ``` - `hostname` has no attribute, so it maps across on its own. Lowering drops the `Located` wrapper, so `Located` becomes `String`. - `#[confval(lower(from = port, with = narrow::i64_to_u16))]` narrows the `i64` port down to a `u16`. [`narrow`](./guide/lowering.md#narrowing-helpers) refuses a value that does not fit instead of quietly truncating it. - `#[confval(nested, default)]` lowers `LimitsSpec::default()` when the file left the block out, so `limits` is always set at runtime. A `with` function has the signature `fn(&SpecField, &mut Report) -> Option`. ```rust fn workers_to_usize(value: &Located, _report: &mut Report) -> Option { // Safe: the range was validated and lowering only runs on a clean report. Some(value.value as usize) } ``` Lowering only runs after the gate, so a conversion here never sees a bad value. ### Running the pipeline `main` runs the four phases in order. First, add the source text to a [`SourceMap`](./guide/diagnostics.md#spans-and-source) and parse it. `parse_hcl` only returns `None` when the input is broken enough that no tree comes out of it. ```rust let mut sources = SourceMap::new(); let mut report = Report::new(); let id = sources.add("server.hcl", input); let spec: Option = confval::format::hcl::parse_hcl(&sources, id, &mut report); ``` Second, validate the parsed spec. ```rust if let Some(spec) = &spec { spec.validate_all(&mut report); } ``` Third, check the report. If it holds any errors, render them and stop before lowering. ```rust if report.has_errors() { let mut out = String::new(); report.render_pretty( &sources, &mut out).unwrap(); eprint ! ("{out}"); std::process::exit(1); } ``` Fourth, lower the validated spec into the runtime config. ```rust let spec = spec.expect("parse returned None without reporting an error"); let config = ServerConfig::lower(&spec, &mut report).expect("validated config lowers"); ``` To watch the report work, put some bad values in the input: an empty `hostname`, a `port` of `99999`, an unknown `mode`. The `has_errors()` check stops the run before lowering, and all three come back reported, each at its own line and column. ## Running the crate examples The crate ships two runnable examples that define the same types and differ only in the format they read. Run the HCL example: ```shell cargo run -q -p confval --example hcl --features derive,color,hcl ``` Configuration file is intentionally invalid: ```shell error: port must be at most 65535 --> server.hcl:2:8 | 2 | port = 99999 | ^^^^^ = help: Set port to at most 65535 error: hostname must not be empty --> server.hcl:1:12 | 1 | hostname = "" | ^^ = help: Set hostname to a reachable address, e.g. "127.0.0.1". error: unknown mode: yolo --> server.hcl:5:10 | 5 | mode = "yolo" | ^^^^^^ = help: expected one of: enforce, log, off ``` Run the TOML example: ```shell cargo run -q -p confval --example toml --features derive,color,toml ``` Configuration file is validated: ```shell listening on 127.0.0.1:8080 with 8 workers limits: max_body_mb=16 mode=enforce ``` Run the issue_severity example that shows a warning: ```shell cargo run -q -p confval --example issue_severity --features derive,color,toml ``` Configuration file is validated with a warning: ```shell warning: hostname set to listen on every available network device --> server.toml:1:12 | 1 | hostname = "0.0.0.0" | ^^^^^^^^^ = help: This might be undesired. listening on 0.0.0.0:8080 with 8 workers limits: max_body_mb=16 mode=enforce ``` Run the validate_traversal example that shows what `validate_all` reaches: ```shell cargo run -q -p confval --example validate_traversal --features derive,color,toml ``` The same invalid nested block is validated twice, differing only in whether the enclosing block is enabled: ```shell upstream enabled: the nested child is validated error: attempts must be at most 10 --> service.toml:5:12 | 5 | attempts = 99 | ^^ = help: Set attempts to at most 10 upstream disabled: descend breaks, so the child is skipped no issues ``` ## Additional Examples Additional examples are available for reference: - An example PR for [mini-redis](https://github.com/ethanhann/mini-redis/pull/1). - Snakeway reverse proxy's [snakeway-conf crate](https://github.com/snakewayhq/snakeway/tree/main/crates/snakeway-conf/src) (advanced usage) --- ## Diagnostics When parsing or validation finds a problem, it does not throw. It records the problem in a `Report`, and confval renders that report for people to read. Spans are what let each message point at the exact place in the file. ## Report and IssueBuilder `Report` collects issues. Validators receive `&mut Report` and emit through a builder: ```rust report .error("port must be between 1 and 65535") .at(spec.port.span) .help("Choose a port in the range 1..=65535.") .emit(); ``` - `report.error(msg)` and `report.warning(msg)` return an `IssueBuilder`. - `.at(span)` attaches the primary span. Issues without a span render without a source location. - `.help(text)` adds a suggestion line. - `.related(span, label)` attaches secondary spans, used for messages like "first declared here". - `.emit()` finalizes the issue. The builder is `#[must_use]`, so forgetting `.emit()` is a compile-time warning. Query methods: `has_errors()`, `has_warnings()`, `has_issues()`. Severity is the two-variant `Severity` enum (`Error`, `Warning`). ## Rendering Renderers write into any `fmt::Write` sink and take the `SourceMap` to resolve spans: | Method | Feature gate | Format | |-----------------|------------------|-------------------------------------------------| | `render_plain` | always available | One line per issue with `file:line:col`, for CI | | `render_pretty` | `color` | rustc-style caret output with source excerpts | | `render_json` | `serde` | Structured JSON for tooling | ```rust let mut out = String::new(); report.render_pretty(&sources, &mut out)?; eprint!("{out}"); ``` Pretty output underlines the offending value in its source line: ``` error: unknown load_balancing_strategy: failovr --> ingress.d/api.hcl:12:31 | 12 | load_balancing_strategy = "failovr" | ^^^^^^^^^ = help: expected one of: failover, round_robin, request_pressure, sticky_hash, random ``` Line and column lookups are O(log n) via a per-source line index. Columns count characters, not bytes. An issue records only a severity, message, optional span, optional help, and related spans, and never reads source text until render time. ## Spans and source A span is a byte range inside one registered source: ```rust pub struct Span { pub source: SourceId, pub start: u32, pub end: u32, } ``` `SourceId` is a lightweight handle issued by the `SourceMap`. Spans are plain data. Resolving them to line and column numbers happens only at render time. The `SourceMap` interns source text. Each file (or in-memory string) is registered once and identified by its `SourceId`: ```rust let mut sources = SourceMap::new(); let id = sources.add("config.hcl", text); ``` Reports do not own source text. Renderers take `&SourceMap` so the text is stored exactly once no matter how many issues reference it. --- ## Lowering Once a spec is validated, lowering converts it into a config type: the runtime form your program uses. Because lowering runs only after the [gate](../pipeline.md), the narrowing conversions inside it never see a bad value. ## Defining a config `#[derive(confval::Config)]` writes the `Lower` impl that converts a validated spec into a runtime config: ```rust #[derive(confval::Config)] #[confval(lower_from = ServerSpec)] pub struct ServerConfig { #[confval(lower(from = version, with = i64_to_u32))] pub version: u32, #[confval(nested)] pub limits: Option, pub ca_file: Option, } ``` The `Lower` trait is: ```rust pub trait Lower: Sized { fn lower(spec: &S, report: &mut Report) -> Option; } ``` Field rules: - **No attribute**: the field auto-maps via the `LowerAuto` trait, which strips `Located` wrappers without narrowing: `Located -> T`, `Option> -> Option`, `Vec> -> Vec`, `Located>> -> Vec`, and the optional variant of the last. - **`#[confval(nested)]`**: the field type implements `Lower` itself. Works for single, `Option`, and `Vec` shapes. - **`#[confval(nested, default)]`**: a non-optional config field lowered from an `Option>` spec field. When the source omits the block, `S::default()` is lowered in its place, so the runtime field is always populated while the spec stays source-faithful (an absent block stays `None`). This spelling also exists on the spec side, where it fills the omitted block during parsing instead of at lowering. See [Optional fields and defaults](./parsing.md#optional-fields-and-defaults) for the difference. - **`#[confval(lower(from = field, with = fn))]`**: explicit conversion through a function `fn(&SpecField, &mut Report) -> Option`. All narrowing (`i64` to `u16`, string to enum, string to `IpNet`) goes through these functions. `from` also accepts a tuple `(a, b)` when one config field derives from several spec fields. - **`#[confval(spec_only(field, ...))]`** at the struct level names spec fields that intentionally have no runtime counterpart. The generated impl destructures the spec exhaustively with no rest pattern. Adding a field to either struct without accounting for it on the other side is a compile error, which keeps spec and config in lockstep. ## Narrowing helpers `confval::pipeline::narrow` provides ready-made `with` functions. For integer width changes: `i64_to_u16`, `i64_to_u32`, `i64_to_u64`, `i64_to_usize`, and `opt_` variants for optional fields. They narrow with `try_from` rather than `as`. A value that does not fit is reported at its span and lowering fails, so a missing range rule surfaces as a located error instead of a silent truncation. `i64_secs_to_duration` (and `opt_i64_secs_to_duration`) route a seconds count through the same checked narrow into a `Duration`, rejecting a negative value rather than wrapping it. `i64_to_f64` widens to `f64` for the ratio and rate fields where an `as` cast cannot be named in a `with` attribute. ```rust use confval::pipeline::narrow; #[derive(confval::Config)] #[confval(lower_from = ServerSpec)] struct ServerConfig { #[confval(lower(from = port, with = narrow::i64_to_u16))] port: u16, } ``` --- ## Parsing Parsing turns a configuration file into a spec type. A spec type is a plain Rust struct whose fields are the settings you expect. Spec is short for "specification." The collection of spec types is the specification for an application's operator-facing configuration surface. Parsing checks structure only. It determines whether each field is present and has the right type. What the values mean is left to [validation](./validation.md). ## Located values Every field on a spec is wrapped in a `Located`. ```rust pub struct Located { pub value: T, pub span: Span, } ``` A `Located` contains a span. A span is a byte range in the configuration file. It gives each field its provenance. The span records where the value came from, so a later error can point at the exact line and column. `Span` and the `SourceMap` that resolves it are covered under [Diagnostics](./diagnostics.md#spans-and-source). A few behaviors are worth knowing: - **Value-only equality.** `PartialEq`, `Eq`, and `Hash` ignore the span, so two configs with the same values compare equal regardless of formatting. - **`Deref` to `T`.** Method calls pass through to the inner value. - **`Located::detached(value)`** produces a value with a sentinel span. Use it to build a spec in code (tests, builders, generated templates) with no source file behind it. - **`Default`** is `detached(T::default())`. - With the `serde` feature, `Located` serializes transparently as `T` and deserializes detached. ## Defining a spec `#[derive(confval::Spec)]` writes the parser for a struct. Parsing is purely structural, so the macro never embeds semantic rules. ```rust #[derive(confval::Spec)] pub struct ServerSpec { pub version: Located, pub threads: Option>, #[confval(nested)] pub limits: Option>, #[confval(default = 30)] pub refresh_interval_seconds: Located, } ``` ### Field types A field's type tells the parser how to read it. These are the types you can use: - **Scalars**: `Located`, `Located`, `Located`, `Located`, and `Located`. - **Lists of strings**: `Vec>`, or `Option>>>` for an optional list. - **Nested structs**: another `Spec` type marked with `#[confval(nested)]`, described below. ### Optional fields and defaults Every field is required by default. Leave a required field out of the file and the parser reports a `missing field` error against the block it belongs to. Two things make a field optional. - `Option<...>` on the type turns an absent field into `None`. - `#[confval(default)]` or `#[confval(default = expr)]` fills an absent field instead of reporting it. A bare `#[confval(default)]` uses the field type's `Default`. The `default = expr` form uses `expr` instead, so `#[confval(default = 30)]` gives the field the value `30` when the file leaves it out. A filled-in value carries a detached span, because no source text stands behind it. Which form a field accepts depends on its shape. | Field shape | `#[confval(default)]` | `#[confval(default = expr)]` | |------------------------------------------------|-----------------------|------------------------------| | `Located` | `T::default()` | `expr` | | `Option>` | `Some(T::default())` | `Some(expr)` | | `Vec>` | empty list | compile error | | `Located` with `#[confval(nested)]` | `S::default()` | compile error | | `Option>` with `#[confval(nested)]` | compile error | compile error | | `Vec>` with `#[confval(nested)]` | compile error | compile error | | `Option>>>` | compile error | compile error | Three rows are worth calling out. Combining `Option` with a default means the field is never `None` for an absent value. The default fills it in. Leave the default off when you need the `Option` to report what the source omitted. An optional nested block rejects a default because an absent block already yields `None`. A nested list rejects one because a list of blocks is zero-or-more already. A string list accepts only the bare form, where the default is the empty list. There is no `default = expr` for a list. :::caution The spelling `#[confval(nested, default)]` also exists on the config side, where it means something different. On a spec it fills the omitted block during parsing, so the spec itself holds the default. On a config it leaves the spec field `None` and lowers `S::default()` in its place, so the spec stays faithful to the source and only the runtime value is filled in. The two are independent, and one setting can use either, both, or neither. See [Lowering](./lowering.md#defining-a-config). ::: ### Nested structs `#[confval(nested)]` tells the parser to read a field with its own `Spec` type instead of as a scalar. It works three ways: - a single struct, `Located` - an optional struct, `Option>` - a list of structs, `Vec>`, which reads a block that may repeat ### Unknown fields A setting in a configuration file that does not exist in the Rust struct will be interpreted as a parsing error. There is no lenient mode that ignores extra settings/keys. Stricter is better in general with configuration file structure, but this is particularly useful for LLM-edited configuration files as LLMs tend to invent settings that do not exist. ### What the derive does not handle The derive only handles plain structs. It cannot express an enum. That is rarely a problem, because a field with a discrete set of values is a `Located` in a spec by convention rather than an enum. [Writing parsers by hand](#writing-parsers-by-hand) covers that convention, along with the shapes that do need a handwritten parser. ## Parsing a file To parse, call the frontend for the format you enabled with the appropriate feature: | Entry point | Feature | Backed by | |-------------------------------------|---------|-------------| | `confval::format::hcl::parse_hcl` | `hcl` | `hcl-edit` | | `confval::format::toml::parse_toml` | `toml` | `toml_edit` | Each takes a `SourceMap`, a `SourceId`, and a `&mut Report`, and returns your spec as an `Option`. ```rust let spec: Option = confval::format::hcl::parse_hcl(&sources, id, &mut report); ``` The result is the same whichever format you read, so validation and lowering never depend on which frontend ran. Both HCL spellings of a nested block parse into the same thing. A block, `bind { port = 8080 }`, and an attribute set to an object, `bind = { port = 8080 }`, are equivalent. TOML lines up with this: a `[table]` is a block, an inline `{ ... }` is an object, and an array of tables (`[[x]]`) is a repeating block, so a `Vec` of nested structs reads from it the same way it reads from an HCL list of objects. :::note `hcl-edit` rejects duplicate attribute keys while parsing, so a repeated attribute is a syntax error. A repeated block parses, and confval reports it with a related span pointing at the first occurrence. ::: ## Writing parsers by hand Most specs never need this. The `Spec` derive covers plain structs, which is nearly everything given the confval [pipeline contract](../pipeline.md). By convention, confval reduces a spec's fields to primitive types wherever it can. A spec holds the most broadly typed form of a field. Lowering narrows that value to a more specific type once validation has run. A discrete set of values follows the same pattern rather than becoming an enum in the spec. Take a `mode` field that accepts "red", "green", or "blue" as an example. The spec holds a `Located`. Validation checks it against a [KeywordSet](./validation.md#keywordset). Lowering converts the string to an enum. Handwritten parsers cover the shapes that pattern cannot express. The clearest case is a block whose remaining fields depend on a discriminator, where the parser reads the discriminator first and dispatches on it. The same mechanism lets you put an enum directly in a spec. That compiles and parses correctly. It also abandons the convention above, which is why it is not the recommended shape. A parser is an implementation of confval's `FromFields` trait. It is the same trait the derive generates. ```rust pub trait FromFields: Sized { fn from_fields(fields: &Fields, report: &mut Report) -> Option; } ``` An implementation parses every field before deciding what to return. Parsing all of them first keeps one bad field from hiding the problems in the others. Report each problem as you find it. Then return `None` when a field failed and the value cannot be built. The `None` itself carries no reason. Whatever explains the failure must already be in the report. ### The field model A handwritten parser reads `Fields`, confval's format-neutral view of one level of structure. A frontend builds it, and from there nothing knows which format the text was. - **`Fields`** is one level: the named entries of a body, table, or inline object, plus the span a missing-field error points at. - **`Field`** is one entry: its name, the span of the name, the span of the whole entry, and a `FieldKind`. - **`FieldKind`** is either `Value` for an attribute (`name = value`) or `Block` for a block (`name { ... }` in HCL, `[name]` in TOML). The split lets a diagnostic say "found block" rather than "found object". - **`Value`** is a span plus a `ValueKind`: a `Scalar`, a `Seq` (a list), a `Map` (nested fields), or `Other`. - **`Scalar`** is `String`, `Int(i64)`, `Float(f64)`, or `Bool`. Integers and floats stay distinct so a format that separates them, like TOML's `1` and `1.0`, round-trips faithfully. - **`Other(label)`** is a value that exists in the file but falls outside the model, such as an HCL template or a TOML datetime. It always surfaces as a plain type mismatch named by the label, for example `expected string, found datetime`. ### Helpers confval ships helpers so a handwritten parser reports exactly like the derive does. Leaf parsers turn one `Field` into a `Located` value, reporting a typed error on mismatch: - `parse_string_field`, `parse_int_field` (i64), `parse_float_field`, `parse_bool_field` - `parse_string_list_field` for arrays of strings Structural parsers recurse through `FromFields`: - `parse_struct_field`: one nested struct from a block or a map value - `parse_single_struct`: like `parse_struct_field`, but reports duplicates when the field appears more than once - `parse_struct_list_field`: repeated blocks or a sequence of maps, collected into a `Vec` Reporting helpers keep messages uniform: `report_unknown_field`, `report_missing_field`, `report_duplicate_field`. Unknown fields are always errors. There is no lenient mode. --- ## Validation [Parsing](./parsing.md), which precedes validation, ensures the spec is structurally correct. Validation is where you exhaustively check what the values mean: ranges, allowed keywords, and rules that cross more than one field. confval provides a `Validate` trait, described under [Validate](#validate) below. Its main purpose is to be named in a bound on the [lower](./lowering.md) stage. Every spec lowered into a config must implement it, or the config does not compile. The bound guarantees a validator exists. It does not guarantee that every field is checked inside that validator. The checks are (currently) intentionally minimal. confval provides only two domain-agnostic checks: `RangeConstraint` and `KeywordSet`. ## Where a rule lives Validation rules live in one of two places: a `Validate` impl and plain validator functions (when necessary). ### Validate trait implementations A `Validate` impl on a spec type holds rules that the type can check from its own fields. It receives `&self`, so it can read every field of that struct. A rule that spans several of the struct's fields can therefore live here. A nested child block is not this type's own field. A `Validate` impl therefore does not itself validate nested child specs. However, there is also no need to call `validate()` manually if you use `validate_all()` on the root spec. :::info You implement the `Validate` trait, but you call `validate_all` once on the root spec. More information on this can be found [in this section](#validate-impl-contains-the-rules-validate_all-runs-them). ::: ### Validator Functions Validator functions are necessary primarily for cross-file and cross-block validation. Depending on the domain, there may be complex semantic rules between files/blocks. For example, imagine a server with a central configuration file that has global settings, like enabling TLS, and subconfiguration files that may or may not be correct if TLS is enabled. A validator function handles this case. A validator function takes whatever it needs to check and appends to the report: ```rust fn validate_tls_agreement(server: &ServerSpec, upstreams: &[UpstreamSpec], report: &mut Report) { /* ... */ } ``` Nothing generates these and nothing calls them for you. They run alongside `validate_all`, before the gate. ## RangeConstraint Numeric bounds are declared once and checked against located values: ```rust range_constraint!(PORT, i64, min: 1, max: 65535); range_constraint!(DRAIN, i64, min: 0, max: 300, units: "seconds"); range_constraint!(WORKERS, i64, min: 1, max: 512, help: "Match this to your CPU core count."); PORT.check_located( &spec.port, "port", report); ``` `check_located` emits an error at the value's span when out of range. When **help** is provided, it overrides the auto-generated suggestion. Otherwise, confval generates one like "Set port to at least 1". ## KeywordSet Closed sets of allowed keyword strings are checked against located values. This is the string counterpart of `RangeConstraint` for fields like strategies, log levels, and fail policies: ```rust const LOAD_BALANCING_STRATEGIES: [&str; 5] = ["failover", "round_robin", "request_pressure", "sticky_hash", "random"]; KeywordSet::new( &LOAD_BALANCING_STRATEGIES) .check_located( &spec.load_balancing_strategy, "load_balancing_strategy", report); ``` `check_located` reports `unknown {field}: {value}` at the value's span, with a help line of `expected one of: `. Every keyword field reports the same way, so a wrong value in any closed-set field produces the same message shape and lists the allowed set. ## Validate `Validate` holds the semantic checks a spec type can perform on itself: ```rust pub trait Validate { fn validate(&self, report: &mut Report); fn descend(&self) -> ControlFlow<()> { /* ... */ } fn validate_all(&self, report: &mut Report) where Self: ValidateNested { /* ... */ } } ``` `validate` is the only method with no default. It is the one to implement. The other two are covered [below](#validate-impl-contains-the-rules-validate_all-runs-them). A `Validate` impl checks what a spec value can prove from its own fields, reporting at the span each field already carries. Because it receives `&self`, it can read every field of that struct. A rule spanning two fields of the same spec type belongs here. It receives no span and no origin parameter, so two kinds of rule do not fit: - A rule that must report at the span of the block itself rather than at one of its fields, such as a required child that is absent. - A rule that needs something outside the struct, such as a sibling spec type or a value assembled from the whole configuration. Those belong in a validator function. Such a function holds the surrounding `Located` wrappers. It can therefore report at any span it needs. Beyond holding those checks, the trait gives the lowering bound something to name. The `Config` derive puts that bound on every generated `Lower` impl: ```rust #[derive(confval::Config)] #[confval(lower_from = ServerSpec)] struct ServerConfig { /* ... */ } // generates: impl Lower for ServerConfig // where ServerSpec: Validate + ValidateNested { ... } ``` A config whose spec has no `Validate` impl fails to compile. A spec that can be lowered into a runtime config but carries no validator is therefore unrepresentable. An empty impl satisfies the bound. A spec type with nothing worth checking writes one, which states that validation was considered rather than forgotten. Handwritten `Lower` impls add the same `where S: Validate + ValidateNested` clause directly. A flattening lowering, meaning one with no per-entity `Lower` impl, can put the bound on the function that performs it. The bound guarantees the validator exists. It does not guarantee that lowering calls it. Validation is still invoked explicitly before the gate. The trait closes the "forgot to write a validator" gap. Calling the validator remains the caller's responsibility. ## `Validate` impl contains the rules, `validate_all` runs them A `Validate` impl covers one spec type's own fields. It does not reach the nested blocks underneath it, because those are separate types with rules of their own. The specs for a configuration surface form a tree. Something has to walk the tree. That walk is generated rather than written by hand, though you can write it yourself. `validate_all` runs this type's `validate`, then descends into every `#[confval(nested)]` field, recursively. One call at the root therefore covers the whole spec tree: ```rust spec.validate_all(&mut report); ``` An absent `Option>` and an empty `Vec>` contribute nothing to the walk. Fields without `#[confval(nested)]` are skipped, because a scalar is checked by its own type's rules or not checked. The traversal itself is a generated `ValidateNested` impl, which is the second half of the lowering bound shown above. A spec type with a handwritten `FromFields` has no derive to generate it and writes the impl itself. :::warning Calling `spec.validate(&mut report)` at the top of a pipeline checks the root block and leaves every nested block unchecked. Nothing in the type system catches that, because both methods compile and both take the same arguments. Keep `validate` out of your call sites. The examples fold validation into the gate helper. `validate_all` then runs in the one place that decides whether a spec is safe to lower. ::: ### Pruning a subtree with `descend` Sometimes a block turns off the feature it configures while its child blocks remain in the file. If the traversal validates those children, the operator receives errors about settings that will not be used. Those errors then have to be separated from the ones that apply to the running configuration. The `descend` method lets a spec type skip its own children. It runs after `validate` and returns a `ControlFlow` value: - `ControlFlow::Continue(())`, the default, validates every nested child. - `ControlFlow::Break(())` stops, leaving the children unvisited. For example, an `UpstreamSpec` that has been disabled may skip the retry and timeout blocks beneath it: ```rust impl Validate for UpstreamSpec { fn validate(&self, report: &mut Report) { /* ... */ } fn descend(&self) -> ControlFlow<()> { if self.enable.value { ControlFlow::Continue(()) } else { ControlFlow::Break(()) } } } ``` You may also find `descend` useful when a spec has already reported that the file was written for a different schema version. The operator needs to correct the version before the individual field errors are worth reading. Because `descend` runs after `validate`, anything the type reported about itself stays in the report. Only the children are skipped. The `validate_traversal` example runs the same invalid configuration twice, changing only the `enable` field, and prints both reports. --- ## The Pipeline Contract The confval pipeline approach is inspired by the ["Parse, don't validate"](https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/) blog post by Alexis King. However, it does not use newtypes to couple construction with validation. Instead, think of the entire process as a multipass parser that sends a configuration file through this sequence: **parse**, **validate**, **gate**, and **lower**. confval assumes a fixed ordering of these stages, and the derives are designed around them. ## The four stages ### 1. Parse (structural) A frontend (`parse_hcl` or `parse_toml`) builds the neutral `Fields`, runs `FromFields`, and reports shape problems. Unknown fields, wrong types, missing required fields, and duplicate blocks are reported with spans. Parsing continues across inputs. An input whose tree was built keeps flowing into validation even if some of its fields failed, so parse and validation problems appear together in one pass. Only an input that produced no tree (a syntax error) stops the load. ### 2. Validate (semantic) Validation checks ranges, closed sets, and cross-field rules against the spans stored in `Located` fields. Rules live in two places: a `Validate` impl on a spec type, which takes `&self` and `&mut Report`, and validator functions you write and call yourself. [Validation](./guide/validation.md#where-a-rule-lives) covers which rule goes where. One call runs the impls: ```rust spec.validate_all(&mut report); ``` `validate_all` runs a spec type's own rules and then descends into every `#[confval(nested)]` field, recursively. The descent comes from `#[derive(Spec)]`. A nested block added later is therefore validated without editing a validator. Calling `validate` instead checks the root and stops there. [Validation](./guide/validation.md#validate-impl-contains-the-rules-validate_all-runs-them) covers the distinction. Validation never panics. In a system with hot reload, a panic during a reload would crash a long-lived service on a simple misconfiguration. The issues are reported instead. Every validator appends issues to the report. An issue is usually a semantic validation rule that enforces a constraint on a setting, not a handled Rust error. For example, a date might pass the parse stage, confirming it is a date, but violate a setting-specific rule requiring it to be at least 90 days in the future. Spans come from the `Located` fields, so validation works the same whether the spec was parsed from a file or constructed in code. :::info The `Validate` trait exists so the requirement can be written as a bound. Every generated `Lower` impl carries `where SpecType: Validate + ValidateNested`, so a config does not compile unless its spec has a validator and a traversal. That catches the forgotten validator and the unreachable child block. An empty `Validate` impl satisfies its half of the bound, so it does not prove any field is checked. Neither half makes lowering call the validator, which stays an explicit step before the gate. ::: ### 3. Gate Lowering must not run when the report contains errors. Nothing in confval enforces this. The caller performs the check. Call `report.has_errors()` after validation and return before lowering when it is true. [Getting Started](./getting-started.md#running-the-pipeline) shows the check in place. Report also has `has_warnings()` and `has_issues()` (i.e., has warnings or errors). This granularity gives the implementor the flexibility to decide if warnings should also prevent lowering or perhaps take some other implementation-specific action. In practice, a sensible approach would be to gracefully exit a running program when errors are found or gracefully stop a hot reload request. If warnings are found, they may be shown, but the start/reload operation continues. ### 4. Lower `Lower::lower` converts spec types to runtime config types. Because the gate ran, the narrowing conversions inside lowering (string to `IpNet`, `i64` to `u16`) are safe. A failure here indicates a missing validation rule rather than invalid input. Unlike parsing and validation, lowering does not accumulate errors. It reports the one error and short-circuits, since a lowering error is rare and means an earlier stage let something through. Say so in the message. For example, "this is likely a bug that should have been caught during validation". An operator reading that knows the problem is in the software rather than in their configuration file. Finally, the error still carries a span, so it renders with a source location like any other. ## Spec types vs. config types Each setting exists in two parallel structs. | Layer | Derives | Purpose | |------------|-----------------------------------|---------------------------------------------------------------| | **Spec** | `confval::Spec` (and `Serialize`) | Populated from the source file, with every field span-tracked | | **Config** | `confval::Config` (and serde) | Resolved, executable form used at runtime | Spec fields are wrapped in `Located`: ```rust #[derive(Debug, confval::Spec)] pub struct ServerSpec { pub version: Located, pub threads: Option>, #[confval(nested)] pub limits: Option>, #[confval(default = 30)] pub refresh_interval_seconds: Located, } ``` Config structs declare how each field lowers: ```rust #[derive(Debug, Clone, confval::Config)] #[confval(lower_from = ServerSpec)] pub struct ServerConfig { #[confval(lower(from = version, with = i64_to_u32))] pub version: u32, #[confval(nested)] pub limits: Option, pub ca_file: Option, // auto-mapped, Located stripped } ``` The generated lowering destructures the spec exhaustively, so adding a field to one side without accounting for it on the other is a compile error. ## Type selection principle **Spec types use the rawest type that parses infallibly.** Strings, `i64`, bools, paths. The structural parsers never reject a value for semantic reasons, so a port of `99999` or a strategy of `"failovr"` parses fine and is caught by validation with a span, alongside every other problem. **Keyword fields are `Located` in specs.** Closed sets like strategies or log levels are validated against a constant slice with a help line listing the options. The runtime enum implements `TryFrom<&str>` and the conversion happens at lowering. A serde keyword enum in the spec layer would abort parsing with a single error instead of joining the report. **Config types use the fully parsed, typed form.** `IpNet`, `SocketAddr`, runtime enums. Downstream code never re-parses a string it received from config. **Handwritten `FromFields` impls cover the shapes the derive does not.** Tagged unions parse their discriminator first and dispatch. A free-form block can be captured as an arbitrary value rather than a struct by reading the neutral field model directly. ## Both spellings normalize Operators write nested structures either as blocks or as attribute-with-object, and real configs mix the two: ```hcl limits { enable = true } limits = { enable = true } ``` The `Fields` view normalizes both, so every nested spec accepts either spelling with identical spans and identical error messages. ## Runnable examples End-to-end examples ship in `crates/confval/examples/`. `hcl.rs` and `toml.rs` each hold a source document and one parse call. Everything after parsing lives in `common/mod.rs`: the spec types, the validators, the config types, and the lowering functions. Both examples share that file verbatim. The format-neutrality of the later stages is visible in the layout, and explicitly annotated. `issue_severity.rs` reuses the same types to show a warning passing the gate. `validate_traversal.rs` stands alone to show what `validate_all` reaches and what a `descend` override prunes. See [Getting Started](getting-started.md) to run them. --- ## confval v0.4.0 The `confval` and `confval-derive` crates now share a single version and are released together. From this release on, one version number and one set of release notes cover the whole workspace. {/* truncate */} ## Highlights ### Enforced `Validate` bound on specs A compiler error will occur if a spec does not implement the `Validate` trait. The trigger for this is lowering. The compilation error will be similar to this, where the trait is not implemented for the example `LimitsSpec`. It materializes as an `unsatisfied trait bound` on `#[derive(confval::Config)]`: ```shell error[E0277]: the trait bound `spec::LimitsSpec: confval::pipeline::Validate` is not satisfied --> crates/confval/examples/common/config.rs:21:10 | 21 | #[derive(confval::Config)] | ^^^^^^^^^^^^^^^ unsatisfied trait bound | help: the trait `confval::pipeline::Validate` is not implemented for `spec::LimitsSpec` --> crates/confval/examples/common/spec.rs:23:1 ``` To resolve such errors, implement the trait for the spec. ### Unified workspace versioning `confval` and `confval-derive` are now versioned in lockstep through `[workspace.package]`. `confval` pins its derive crate with `confval-derive = "=0.4.0"`, so a given `confval` always resolves the matching derive. ## Changed - `confval-derive` moves from `0.1.1` to `0.4.0` to align with `confval`. This is a version-number change only. There is no change to the derive macro output or its public API. ## Upgrading Depend on `confval` as before. The `derive` feature continues to pull in the matching `confval-derive` automatically. ```toml [dependencies] confval = { version = "0.4", features = ["derive", "hcl", "color"] } ``` --- ## confval v0.5.0 This release adds a way to automatically ensure all nested specs are validated. {/* truncate */} ## Highlights ### One call validates the whole spec tree `Validate` gains a `validate_all` method that runs a spec type's own rules and then descends into every `#[confval(nested)]` field, recursively. A `Validate` impl covers one spec type's own fields, so reaching a nested block used to mean calling into it manually from the parent spec's `Validate` impl. Forgetting that call left the nested block unvalidated with nothing to indicate it. ```rust spec.validate_all(&mut report); ``` The descent comes from `#[derive(Spec)]`, so a nested block added later is validated without editing a validator. See [Validation](/docs/guide/validation) for the full picture. ### Existing `Validate` impls do not change `Validate::validate` keeps its signature and stays the only method you have to write. `validate_all` and `descend` are provided methods, so an impl written against v0.4.0 compiles unchanged. ## Added - `Validate::validate_all`, the entry point that runs a spec's rules and then its nested children. - `Validate::descend`, which decides whether the children of a block are visited. - `ValidateNested`, the traversal trait that `#[derive(Spec)]` implements for you. - `ControlFlow` and `ValidateNested` in the prelude. - A `validate_traversal` example. ## Changed - `#[derive(Spec)]` now emits an `impl ValidateNested` alongside the `impl FromFields`. Nothing about the parser changed. - The bound on every generated `Lower` impl moves from `where S: Validate` to `where S: Validate + ValidateNested`. A spec that derives `Spec` satisfies the second half automatically. - A nested spec without a `Validate` impl is now a compile error at its parent rather than a block that is skipped. ## Upgrading Replace the top level `spec.validate(&mut report)` with `spec.validate_all(&mut report)`. Delete any calls into child specs from your `Validate` impls. A leftover call reports the child's issues twice, so it is worth a grep to find the calls and remove them: ```shell grep -rn ".validate(" crates/ ``` A spec with a handwritten `FromFields` has no derive to generate its traversal and needs its own `impl ValidateNested`. The lowering bound reports the missing impl. ```toml [dependencies] confval = { version = "0.5", features = ["derive", "hcl", "color"] } ```