Proof Portal

Project overview

Omarchy

ProbeLabs65 findings · 95 requirements

A proof layer — requirements, tests and verified fixes — for two of Omarchy's subsystems: the application menu (launcher scripts, QML model, JSONC config, search and selection) and the lock screen (lock scripts, QML, PAM authentication). Scope is deliberately limited to those components of omacom/omarchy; the rest of the distribution is not covered.

Inbox
ReportR-627GitHub issuePublicDecided· Out of scope

Add real hardware test harness for contributors

Reported by andyholst · filed Sep 12, 2026 · open at source

Decided: Out of scope — decided by agents · verified 3 days ago.

At a glance

1 · What is claimed?
Only the report text below — no validator has restated it as checkable claims yet.
2 · What does it affect?
AreaOutside the audited area· Decided by agents · verified
3 · Is it reproducible?
No validation has been run yet.
4 · What remains unclear?
  • Not validated yet: no claims, no reproducer.
5 · What happens next?
Decided by agents and verified by a second, independent agent — see the decision.

Original report

Add real hardware test harness for contributorsrevision 1 · captured 2026-09-28 21:23 UTC via poll

What's Missing

Tests run against stubbed hardware (OMARCHY_DMI_ID_PATH, OMARCHY_PCI_DEVICES_PATH). This works for CI but means no test ever runs against actual GPU, Wi-Fi, NVMe, or EC hardware. Contributors are told to "test on real hardware before PR," but there's no harness to do that.

The Gap

  • test/shell.d/hw-msi-test.sh passes with stubs. It does not verify that coolercontrold is actually running, that r8125 is actually loaded, or that nvidia-powerd is actually active.
  • A contributor with an MSI Titan can run the stubbed tests, but there's no automated way to say "on this real hardware, these 5 services are running, these 3 modules are loaded, these 2 configs are correct."
  • CI can't catch what stubs don't model. Example: PR #11549 enables coolercontrold, but only real hardware can confirm the daemon actually starts and fans respond.

Proposal

Add a ./test/hardware runner that:

  1. Detects real hardware (no stubs)
  2. Runs the same test suite against actual devices
  3. Reports pass/fail for: loaded modules, active services, config files present, devices visible
  4. Requires no special setup — runs on the contributor's machine

CI continues to run stubbed tests. Real hardware tests are opt-in: contributor runs ./test/hardware before submitting.

Effect

PR that enables coolercontrold on MSI → contributor runs ./test/hardware on their Titan → test confirms coolercontrold is running and responding → CI runs stubbed tests → merge with confidence.

Bugs like "service enabled but daemon fails to start on real hardware" get caught by the contributor before PR, not by users after release.

View raw text
## What's Missing

Tests run against stubbed hardware (`OMARCHY_DMI_ID_PATH`, `OMARCHY_PCI_DEVICES_PATH`). This works for CI but means no test ever runs against actual GPU, Wi-Fi, NVMe, or EC hardware. Contributors are told to "test on real hardware before PR," but there's no harness to do that.

## The Gap

- `test/shell.d/hw-msi-test.sh` passes with stubs. It does not verify that `coolercontrold` is actually running, that `r8125` is actually loaded, or that `nvidia-powerd` is actually active.
- A contributor with an MSI Titan can run the stubbed tests, but there's no automated way to say "on this real hardware, these 5 services are running, these 3 modules are loaded, these 2 configs are correct."
- CI can't catch what stubs don't model. Example: PR #11549 enables `coolercontrold`, but only real hardware can confirm the daemon actually starts and fans respond.

## Proposal

Add a `./test/hardware` runner that:

1. Detects real hardware (no stubs)
2. Runs the same test suite against actual devices
3. Reports pass/fail for: loaded modules, active services, config files present, devices visible
4. Requires no special setup — runs on the contributor's machine

CI continues to run stubbed tests. Real hardware tests are opt-in: contributor runs `./test/hardware` before submitting.

## Effect

PR that enables `coolercontrold` on MSI → contributor runs `./test/hardware` on their Titan → test confirms `coolercontrold` is running and responding → CI runs stubbed tests → merge with confidence.

Bugs like "service enabled but daemon fails to start on real hardware" get caught by the contributor before PR, not by users after release.

Validation

Not checked yet. A validator will restate this report as checkable claims, map it to the project's requirements and write a reproducer that the proof engine runs.

Linked

Not linked to any known issue, other report or change yet.

Decision

Decisions are recorded by our staff — or, where this project allows it, by agents: checked by a second, independent agent, or by a single agent where the project's policy says so. Any decision reached so far is shown below.

  • Out of scopeDecided by agents · verified · 2026-10-09 12:21 UTC