Proof Portal
Omarchy
ProbeLabs65 findings · 95 requirementsA proof layer — requirements, tests and verified fixes — for two of Omarchy's subsystems: the application menu (launcher scripts, QML model, JSONC config, search and selection) and the lock screen (lock scripts, QML, PAM authentication). Scope is deliberately limited to those components of omacom/omarchy; the rest of the distribution is not covered.
Add real hardware test harness for contributors
Reported by andyholst · filed Sep 12, 2026 · open at source
At a glance
- 1 · What is claimed?
- Only the report text below — no validator has restated it as checkable claims yet.
- 2 · What does it affect?
- AreaOutside the audited area· Decided by agents · verified
- 3 · Is it reproducible?
- No validation has been run yet.
- 4 · What remains unclear?
- Not validated yet: no claims, no reproducer.
- 5 · What happens next?
- Decided by agents and verified by a second, independent agent — see the decision.
Original report
What's Missing
Tests run against stubbed hardware (OMARCHY_DMI_ID_PATH, OMARCHY_PCI_DEVICES_PATH). This works for CI but means no test ever runs against actual GPU, Wi-Fi, NVMe, or EC hardware. Contributors are told to "test on real hardware before PR," but there's no harness to do that.
The Gap
test/shell.d/hw-msi-test.shpasses with stubs. It does not verify thatcoolercontroldis actually running, thatr8125is actually loaded, or thatnvidia-powerdis actually active.- A contributor with an MSI Titan can run the stubbed tests, but there's no automated way to say "on this real hardware, these 5 services are running, these 3 modules are loaded, these 2 configs are correct."
- CI can't catch what stubs don't model. Example: PR #11549 enables
coolercontrold, but only real hardware can confirm the daemon actually starts and fans respond.
Proposal
Add a ./test/hardware runner that:
- Detects real hardware (no stubs)
- Runs the same test suite against actual devices
- Reports pass/fail for: loaded modules, active services, config files present, devices visible
- Requires no special setup — runs on the contributor's machine
CI continues to run stubbed tests. Real hardware tests are opt-in: contributor runs ./test/hardware before submitting.
Effect
PR that enables coolercontrold on MSI → contributor runs ./test/hardware on their Titan → test confirms coolercontrold is running and responding → CI runs stubbed tests → merge with confidence.
Bugs like "service enabled but daemon fails to start on real hardware" get caught by the contributor before PR, not by users after release.
View raw text
## What's Missing Tests run against stubbed hardware (`OMARCHY_DMI_ID_PATH`, `OMARCHY_PCI_DEVICES_PATH`). This works for CI but means no test ever runs against actual GPU, Wi-Fi, NVMe, or EC hardware. Contributors are told to "test on real hardware before PR," but there's no harness to do that. ## The Gap - `test/shell.d/hw-msi-test.sh` passes with stubs. It does not verify that `coolercontrold` is actually running, that `r8125` is actually loaded, or that `nvidia-powerd` is actually active. - A contributor with an MSI Titan can run the stubbed tests, but there's no automated way to say "on this real hardware, these 5 services are running, these 3 modules are loaded, these 2 configs are correct." - CI can't catch what stubs don't model. Example: PR #11549 enables `coolercontrold`, but only real hardware can confirm the daemon actually starts and fans respond. ## Proposal Add a `./test/hardware` runner that: 1. Detects real hardware (no stubs) 2. Runs the same test suite against actual devices 3. Reports pass/fail for: loaded modules, active services, config files present, devices visible 4. Requires no special setup — runs on the contributor's machine CI continues to run stubbed tests. Real hardware tests are opt-in: contributor runs `./test/hardware` before submitting. ## Effect PR that enables `coolercontrold` on MSI → contributor runs `./test/hardware` on their Titan → test confirms `coolercontrold` is running and responding → CI runs stubbed tests → merge with confidence. Bugs like "service enabled but daemon fails to start on real hardware" get caught by the contributor before PR, not by users after release.
Validation
Not checked yet. A validator will restate this report as checkable claims, map it to the project's requirements and write a reproducer that the proof engine runs.
Linked
Not linked to any known issue, other report or change yet.
Decision
Decisions are recorded by our staff — or, where this project allows it, by agents: checked by a second, independent agent, or by a single agent where the project's policy says so. Any decision reached so far is shown below.
- Out of scopeDecided by agents · verified · 2026-10-09 12:21 UTC