Rust for Data Scientists: Build a Typed Data-Quality CLI

Rust is useful to a data scientist when a boundary must fail predictably. A model-training notebook may reasonably remain in Python or R, and a relational transformation may belong in SQL. But a small command-line program that accepts an external file, enforces a versioned contract and emits one stable report can benefit from compilation, explicit error paths and a narrow deployable interface.

This extension does that one job. It reads the same immutable hour.csv used by the SQL data-quality workflow, verifies its bytes and analytical meaning, and writes rust_data_quality.json. It neither trains a second model nor claims that Rust is faster because it handled a 17,379-row teaching file. The lesson is how to make an analytical handover difficult to misuse.

Define the boundary before choosing a crate

The source has three different kinds of truth, and the program treats them separately:

Layer	Evidence	Question
Byte provenance	dataset.lock.json and SHA-256	Is this the reviewed repository copy?
Structural contract	ordered CSV header and typed fields	Can every record be interpreted as the promised table?
Analytical contract	contract.json and cross-field checks	Does the table still mean what the workflow assumes?

A matching digest is necessary but insufficient. It proves identity with one reviewed byte sequence; it does not explain why casual and registered are prohibited predictors. Conversely, a table can satisfy a few semantic assertions while having come from an unrecorded source. The quality gate therefore checks both provenance and meaning.

The prediction moment is before rentals for an hour have been observed. cnt is the target, while casual and registered are its post-outcome components because cnt = casual + registered. The gate rejects any contract that permits those three fields as predictors. That is a data-science decision, not a property Rust can infer from types.

Install the exact review toolchain

The companion package owns rust-toolchain.toml and Cargo.lock. From the repository root, install the declared formatting and linting components, then run the package without updating dependencies:

rustup toolchain install 1.97.1 --profile minimal \
  --component rustfmt --component clippy
cd examples/programming-for-data-science/rust
cargo fmt --check
cargo clippy --locked --all-targets -- -D warnings
cargo test --locked

--locked makes dependency resolution part of the reproduction contract. If Cargo.toml and Cargo.lock disagree, Cargo stops instead of silently selecting a new graph. A dependency update is still expected over time, but it should be a visible review: change the pin, regenerate the lock, rerun every negative fixture and record a new verification date.

The project uses Rust 1.97.1 and Polars 0.55.2 as explicit teaching targets. They belong to this standalone package; the website application does not need to adopt the Rust toolchain.

Combine a lazy scan with typed rows

Polars’ lazy API lets the program describe table-level work before executing it. The shortened excerpt below assumes that schema has already fixed all 17 column types; the runnable source also adds error context and extracts each scalar result explicitly:

let path = PlRefPath::try_from_path(input)?;
let profile = LazyCsvReader::new(path)
    .with_has_header(true)
    .with_schema(Some(schema))
    .with_ignore_errors(false)
    .with_missing_is_null(false)
    .finish()?
    .select([
        len().alias("row_count"),
        col("instant").n_unique().alias("distinct_instants"),
        col("cnt").cast(DataType::UInt64).sum().alias("target_total"),
        col("cnt").min().alias("target_minimum"),
        col("cnt").max().alias("target_maximum"),
    ])
    .collect()?;

That is a real tabular execution plan, not a benchmark demonstration. On a larger boundary the same design can benefit from projection and predicate pushdown. Here its purpose is architectural: table-level aggregation stays concise, while a second typed row-by-row pass supplies record-level context for semantic failures.

Serde and the CSV deserialiser establish a typed input boundary by mapping each record to an HourRow. Fields first become concrete integer and floating-point values; explicit checks then enforce category domains, non-negativity and finiteness. Parsing can still succeed on an analytically invalid row, so validation remains explicit:

let component_total = row
    .casual
    .checked_add(row.registered)
    .context("rental component total overflowed")?;
ensure!(
    row.cnt == component_total,
    "CSV row {csv_row} violates cnt = casual + registered"
);

ensure!(
    row.workingday == expected_workingday,
    "CSV row {csv_row} has an inconsistent workingday flag"
);

Ownership is helpful here because the byte buffer, parsed rows and final report have clear ownership boundaries. It does not make the source trustworthy. The useful discipline is that validation returns Result and the CLI does not create or replace the requested artefact after an error; a report from an earlier successful run may remain.

Fail closed on the ordered schema

The program requires these columns in this order:

instant,dteday,season,yr,mnth,hr,holiday,weekday,workingday,weathersit,temp,atemp,hum,windspeed,casual,registered,cnt

This is intentionally stricter than accepting any superset. A renamed field, duplicated column or reordered export may indicate that a producer changed. Silent accommodation would make the checked digest and downstream expectations less useful.

For every row, the gate requires:

• a unique instant and a unique date-hour timestamp;
• instant values from 1 through 17,379 in source order;
• a valid date, an hour from 0 through 23 and a timestamp inside 2011–2012;
• yr equal to calendar year minus 2011 and mnth equal to calendar month;
• weekday encoded with Sunday as zero;
• workingday equal to one only for a non-holiday Monday–Friday;
• category codes inside the documented ranges;
• finite normalised weather values and non-negative component and target counts; and
• exact target identity for all 17,379 observations.

Errors name the failed check and record identifier. That context matters operationally: “deserialisation failed” is less useful than “row 6842 has month 8 for a September date.” Tests mutate one property at a time so a later refactor cannot replace a specific guard with a broad row-count check.

Treat missing hours as evidence, not corruption

The dataset is hourly in meaning but not a complete hourly time series. After ordering timestamps, the gate calculates the difference between adjacent observations. It expects 75 intervals longer than one hour, a maximum gap of 37 hours and 165 missing hourly slots in total.

Those checkpoints serve two purposes. First, a future source refresh cannot quietly change the sample while retaining the same broad date range. Second, the report does not invent observations to make the grid regular. An analyst can decide later whether a model requires imputation, an explicit availability indicator or a different unit of analysis.

The expected temporal boundary is exact:

Check	Expected value
First timestamp	2011-01-01T00:00:00
Last timestamp	2012-12-31T23:00:00
Recorded rows	17,379
Gaps longer than one hour	75
Longest gap	37 hours
Missing hourly slots	165

These are descriptive properties of the archived sample. They are not claims about present-day bicycle availability or the causes of missing observations.

Keep the prediction-time contract machine-readable

The package reads contract.json rather than copying only its row count into Rust. It confirms the primary key, time columns, target, target derivation, allowed predictors, prohibited predictors and chronological split definitions. A test changes a prohibited field into an allowed predictor and expects the CLI to fail even when the CSV bytes remain untouched.

This is the central lesson for a data scientist: a clean schema does not prevent leakage. The same numeric column can be legitimate for retrospective accounting and invalid for a forecast made before the outcome. The report therefore publishes the prediction moment and prohibited predictor list beside the structural checks.

Keeping this boundary as a small CLI also limits integration choices. A scheduler, upload service or feature pipeline can execute one command and inspect its exit status. It does not need to embed a Rust library into the model runtime or expose a web service merely to validate a file.

Emit a deterministic evidence artefact

Run the release build against the checked-in inputs:

cd examples/programming-for-data-science/rust
cargo run --locked --release -- \
  --input ../data/hour.csv \
  --contract ../data/contract.json \
  --dataset-lock ../data/dataset.lock.json \
  --output ../artifacts/rust_data_quality.json

The JSON contains dataset attribution, source and licence links, the observed digest, pinned tool versions, prediction-time exclusions, temporal checkpoints and named check results. Struct field order and pretty-printing are fixed, and the file ends with one newline. The test suite generates the report twice and compares bytes.

A shortened shape looks like this:

{
  "schemaVersion": 1,
  "dataset": {
    "name": "UCI Bike Sharing — hourly data",
    "creator": "Hadi Fanaee-T",
    "doi": "10.24432/C5W894",
    "sha256": "b03a2d02e8c10f435c43c7f0b358b7e34a003afea53dbc37f0183f2763295133"
  },
  "observed": {
    "rows": 17379,
    "distinctInstants": 17379,
    "targetIdentityFailures": 0,
    "gapIntervals": 75,
    "maximumGapHours": 37,
    "missingHourlySlots": 165
  },
  "allChecksPassed": true
}

Console prose is for the person running the command; the JSON is for durable comparison. A downstream gate should parse the named fields and schema version rather than search terminal text for “passed.”

Test failures that could mislead an analysis

The golden dataset test is only the start. Negative fixtures establish which changes must stop the handover:

1. a byte change without an updated dataset lock fails the digest gate;
2. a modified target component with a deliberately updated digest still fails target identity;
3. a duplicate identifier or timestamp fails uniqueness;
4. a date/category mismatch fails the calendar contract;
5. a non-finite measurement fails before aggregation or JSON serialisation;
6. a changed first or last timestamp fails the sample boundary;
7. a contract that permits outcome components fails the prediction-time rule; and
8. a changed header fails before typed row processing.

The distinction in the second test is important. If every semantic test relied on the original digest, one byte mutation would exercise only the provenance error. Updating the fixture’s expected digest lets the test reach the independent analytical guard.

CI runs cargo fmt, Clippy with warnings denied, locked tests and the release reproduction in an isolated Rust job. The output is uploaded for review but is not committed. A green run means the declared snapshot reproduced under the pinned environment; it does not certify an arbitrary future file.

Decide whether Rust earns its place

Rust is a defensible choice when the quality boundary is reused across teams, deployed where a Python environment is undesirable, exposed to untrusted inputs or important enough that explicit error handling and a single-purpose binary reduce operational risk. It can also be useful when the same validated parser will later become part of a higher-throughput ingestion system.

It is not automatically the best choice for exploratory analysis. If a ten-line DuckDB query already runs beside the only consumer, a separate compiled CLI may add review and maintenance cost without reducing risk. If requirements are still changing daily, a Python validation library with strong tests may make iteration easier. And if the data is already governed inside a warehouse, duplicating every rule in Rust can create two competing sources of truth.

The right comparison is not “Rust versus Python” in the abstract. Ask whether this specific boundary needs a stable binary interface, whether the team can review and maintain Rust, and whether the new implementation independently catches a failure that matters. This guide answers yes for a narrowly scoped teaching gate and leaves the modelling workflow in the languages already suited to it.

Interpret the result within its limits

Passing the gate establishes that the repository copy matches its lock and the declared checks hold. It does not establish that the UCI catalogue count is correct, that missing hours are random, that recorded rentals equal unconstrained demand or that the data transfers to another city or period.

The observations cover Washington, DC in 2011–2012. They omit station capacity, real-time bicycle and dock availability, rebalancing, price changes and other operational constraints. The source landing page reports 17,389 instances, while the archived hourly file contains 17,379 data rows; the report preserves that discrepancy instead of manufacturing ten records.

Use the artefact as evidence about this immutable pedagogical snapshot. For a production boundary, add ownership for each rule, a response procedure for failures, current representative data, privacy and security review, and monitoring that distinguishes an upstream contract change from a damaged file.

Continue the learning path

Return to Programming for Data Scientists to compare the core handover and applied extensions. Revisit SQL for Data Scientists for the feature-table implementation that motivates this independent gate.

The durable pattern is language-agnostic: pin the source, validate structure, enforce analytical meaning, test realistic failures and publish evidence that another system can read. Rust contributes a strong implementation boundary; data-science judgement still defines what the boundary must protect.