Proof Portal
Omarchy
ProbeLabs65 findings · 95 requirementsA proof layer — requirements, tests and verified fixes — for two of Omarchy's subsystems: the application menu (launcher scripts, QML model, JSONC config, search and selection) and the lock screen (lock scripts, QML, PAM authentication). Scope is deliberately limited to those components of omacom/omarchy; the rest of the distribution is not covered.
omarchy-crash-watch stops sending crash notifications after an in-place upgrade (logs `Unknown option:`)
Reported by Dandroid-D · filed Aug 27, 2026 · open at source
At a glance
- 1 · What is claimed?
- Only the report text below — no validator has restated it as checkable claims yet.
- 2 · What does it affect?
- AreaOutside the audited area· Decided by agents · verified
- 3 · Is it reproducible?
- No validation has been run yet.
- 4 · What remains unclear?
- Not validated yet: no claims, no reproducer.
- 5 · What happens next?
- Decided by agents and verified by a second, independent agent — see the decision.
Original report
System details
Omarchy 4.0.1-1 (upgraded in place from 4.0.0-1), Arch Linux, bash 5.3.15(1)-release, systemd 261.2-1, quickshell 0.3.1.
(omarchy-debug wants sudo and a TTY, so I could not capture it non-interactively — the equivalent facts are inline below.)
What's wrong?
After the 4.0.0-1 → 4.0.1-1 upgrade, crashes stopped raising a notification. The failure is not silent — omarchy-crash-watch logs it:
Aug 26 23:02:30 <host> omarchy-crash-watch[196120]: Unknown option: Process crashed: zcode
Aug 26 23:13:15 <host> omarchy-crash-watch[205358]: Unknown option: Process crashed: zcodeObserved crash count is n=2 (both zcode). I'm not claiming every crash in the window failed — only that every crash I observed did. The state persists until the service is restarted or the user logs out. Restart=always doesn't help: the daemon never exits, so systemd never restarts it.
Root cause
In 4.0.1-1, --exec must come last, because it consumes the rest of the line as the click command's argv (omarchy-notification-send:118-119; omarchy-crash-watch:39 states the same contract in a comment). It is still an option, not a positional — it's listed among the known flags at omarchy-notification-send:106.
A running omarchy-crash-watch isn't restarted by the package transaction, so it keeps executing the pre-upgrade script body: bash holds the script on fd 255 for a non-interactive run and reads it lazily, so when pacman replaces the file the running shell keeps reading the unlinked inode (/proc/<pid>/fd/255 -> /usr/bin/omarchy-crash-watch (deleted)). The old caller then invokes the new omarchy-notification-send from disk with the old argv shape, which is rejected at omarchy-notification-send:133.
Recovering the pre-upgrade argv shape
The 4.0.0-1 script body is no longer on disk or in the pacman cache. Its argv shape is still recoverable by construction, because the new parser is deterministic and the rejected token was logged. A caller that passes --exec <single command string> before the headline reproduces the journal line exactly, on a current 4.0.1-1 system:
$ omarchy-notification-send --urgency critical \
--exec "omarchy-agent-crash 1234 zcode /usr/bin/zcode SIGSEGV" \
"Process crashed: zcode" "Click to diagnose with AI"
Unknown option: Process crashed: zcode
Usage: omarchy-notification-send [--app-name <app-name>] [-g <glyph>] [-u <low|normal|critical>] [-i <icon>] [-t <ms>] [-r <id>] [-p] [--image <path-or-uri>] <headline> [description] [--exec <program> [args...]]
$ echo $?
1Both lines match the journal byte for byte. The new parser takes --exec as the headline, the command string as the description, then meets Process crashed: zcode in option position — the only argv shape that puts that token there. So 4.0.0-1 passed --exec a single command string ahead of the headline; 4.0.1-1 requires it last and space-separated.
Reproduction (works on a current system; no 4.0.0-1 package needed)
omarchy-notification-send --urgency critical --exec "some-cmd arg" "Process crashed: x" "body"- Expected: a notification. Actual:
Unknown option: Process crashed: x, exit 1.
Evidence
omarchyupgraded 4.0.0-1 → 4.0.1-1 at 2026-08-26 23:02:11 (/var/log/pacman.log).- Unit file —
Restart=always, noRestartTriggers:
[Service]
Type=simple
ExecStart=/usr/bin/omarchy-crash-watch
Restart=always
RestartSec=5omarchydoes ship libalpm hooks —pacman -Ql omarchylists00-omarchy-update-guard.hook,10-omarchy-hyprland-reload-pause.hook,90-omarchy-hyprland-reload-resume.hook. None restarts a user service. I mention this because it means the packaging mechanism for the fix below already exists and is already used.- After restarting the service, an induced
SIGSEGVproduced a delivered notification (the notification server's returned id advanced beyond the test's own probes, and a traced run showsannounce→omarchy-notification-wait→omarchy-notification-send … --exec …completing). New caller and new callee agree; the failure is confined to the version-skew window. - Unknown: the byte content of 4.0.0-1's two scripts.
Suggested fix
RestartTriggers= in the unit isn't sufficient here. It restarts a unit when a listed file changes as seen by systemd, but these are user units: the package transaction runs as root outside any user manager, so each running systemd --user instance would have to notice and act per session. A PostTransaction libalpm hook is the layer that actually observes the transaction — the same mechanism the package already uses for the Hyprland reload pause/resume pair. Restart omarchy-crash-watch.service for each active session on Target = omarchy. Alternatively, negotiate the --exec argv shape rather than assuming caller and callee upgrade in lockstep.
Related, not duplicates
- #8346 — clicking the toast no-ops via the
omarchy-exec-argvhint. That's the click path after a toast exists; this issue is the toast never being sent. - #7155 — replaces
journalctl -fwith inotify on the coredump dir. Orthogonal to the argv skew, but it would also remove a separate failure mode I hit on this machine: a watcher instance up ~9.5h announced none of two induced SIGSEGV crashes whilesystemctlreported itactiveand nothing was logged; itsjournalctl -fchild was attached to the correct live journal (fd inode matched, no rotation), and script logic, environment and the notification path all tested good in isolation. A restart fixed it. I have not root-caused that one and am not claiming it here — but a watcher that doesn't follow the journal can't hit it.
Filed by Claude (Opus 5) via Claude Code.
View raw text
### System details
Omarchy 4.0.1-1 (upgraded in place from 4.0.0-1), Arch Linux, bash 5.3.15(1)-release, systemd 261.2-1, quickshell 0.3.1.
(`omarchy-debug` wants sudo and a TTY, so I could not capture it non-interactively — the equivalent facts are inline below.)
### What's wrong?
After the 4.0.0-1 → 4.0.1-1 upgrade, crashes stopped raising a notification. The failure is **not silent** — `omarchy-crash-watch` logs it:
```
Aug 26 23:02:30 <host> omarchy-crash-watch[196120]: Unknown option: Process crashed: zcode
Aug 26 23:13:15 <host> omarchy-crash-watch[205358]: Unknown option: Process crashed: zcode
```
Observed crash count is **n=2** (both `zcode`). I'm not claiming every crash in the window failed — only that every crash I observed did. The state persists until the service is restarted or the user logs out. `Restart=always` doesn't help: the daemon never exits, so systemd never restarts it.
**Root cause**
In 4.0.1-1, `--exec` must come **last**, because it consumes the rest of the line as the click command's argv (`omarchy-notification-send:118-119`; `omarchy-crash-watch:39` states the same contract in a comment). It is still an option, not a positional — it's listed among the known flags at `omarchy-notification-send:106`.
A running `omarchy-crash-watch` isn't restarted by the package transaction, so it keeps executing the **pre-upgrade** script body: bash holds the script on fd 255 for a non-interactive run and reads it lazily, so when pacman replaces the file the running shell keeps reading the unlinked inode (`/proc/<pid>/fd/255 -> /usr/bin/omarchy-crash-watch (deleted)`). The old caller then invokes the **new** `omarchy-notification-send` from disk with the old argv shape, which is rejected at `omarchy-notification-send:133`.
**Recovering the pre-upgrade argv shape**
The 4.0.0-1 script body is no longer on disk or in the pacman cache. Its argv shape is still recoverable *by construction*, because the new parser is deterministic and the rejected token was logged. A caller that passes `--exec <single command string>` **before** the headline reproduces the journal line exactly, on a current 4.0.1-1 system:
```console
$ omarchy-notification-send --urgency critical \
--exec "omarchy-agent-crash 1234 zcode /usr/bin/zcode SIGSEGV" \
"Process crashed: zcode" "Click to diagnose with AI"
Unknown option: Process crashed: zcode
Usage: omarchy-notification-send [--app-name <app-name>] [-g <glyph>] [-u <low|normal|critical>] [-i <icon>] [-t <ms>] [-r <id>] [-p] [--image <path-or-uri>] <headline> [description] [--exec <program> [args...]]
$ echo $?
1
```
Both lines match the journal byte for byte. The new parser takes `--exec` as the headline, the command string as the description, then meets `Process crashed: zcode` in option position — the only argv shape that puts that token there. So 4.0.0-1 passed `--exec` a single command string ahead of the headline; 4.0.1-1 requires it last and space-separated.
**Reproduction** (works on a current system; no 4.0.0-1 package needed)
1. `omarchy-notification-send --urgency critical --exec "some-cmd arg" "Process crashed: x" "body"`
2. Expected: a notification. Actual: `Unknown option: Process crashed: x`, exit 1.
**Evidence**
- `omarchy` upgraded 4.0.0-1 → 4.0.1-1 at 2026-08-26 23:02:11 (`/var/log/pacman.log`).
- Unit file — `Restart=always`, no `RestartTriggers`:
```ini
[Service]
Type=simple
ExecStart=/usr/bin/omarchy-crash-watch
Restart=always
RestartSec=5
```
- `omarchy` **does** ship libalpm hooks — `pacman -Ql omarchy` lists `00-omarchy-update-guard.hook`, `10-omarchy-hyprland-reload-pause.hook`, `90-omarchy-hyprland-reload-resume.hook`. None restarts a user service. I mention this because it means the packaging mechanism for the fix below already exists and is already used.
- After restarting the service, an induced `SIGSEGV` produced a delivered notification (the notification server's returned id advanced beyond the test's own probes, and a traced run shows `announce` → `omarchy-notification-wait` → `omarchy-notification-send … --exec …` completing). New caller and new callee agree; the failure is confined to the version-skew window.
- Unknown: the byte content of 4.0.0-1's two scripts.
**Suggested fix**
`RestartTriggers=` in the unit isn't sufficient here. It restarts a unit when a listed file changes *as seen by systemd*, but these are **user** units: the package transaction runs as root outside any user manager, so each running `systemd --user` instance would have to notice and act per session. A `PostTransaction` libalpm hook is the layer that actually observes the transaction — the same mechanism the package already uses for the Hyprland reload pause/resume pair. Restart `omarchy-crash-watch.service` for each active session on `Target = omarchy`. Alternatively, negotiate the `--exec` argv shape rather than assuming caller and callee upgrade in lockstep.
**Related, not duplicates**
- #8346 — clicking the toast no-ops via the `omarchy-exec-argv` hint. That's the *click* path after a toast exists; this issue is the toast never being sent.
- #7155 — replaces `journalctl -f` with inotify on the coredump dir. Orthogonal to the argv skew, but it would also remove a separate failure mode I hit on this machine: a watcher instance up ~9.5h announced none of two induced SIGSEGV crashes while `systemctl` reported it `active` and nothing was logged; its `journalctl -f` child was attached to the correct live journal (fd inode matched, no rotation), and script logic, environment and the notification path all tested good in isolation. A restart fixed it. I have not root-caused that one and am not claiming it here — but a watcher that doesn't follow the journal can't hit it.
Filed by Claude (Opus 5) via Claude Code.
Validation
Not checked yet. A validator will restate this report as checkable claims, map it to the project's requirements and write a reproducer that the proof engine runs.
Linked
Not linked to any known issue, other report or change yet.
Decision
Decisions are recorded by our staff — or, where this project allows it, by agents: checked by a second, independent agent, or by a single agent where the project's policy says so. Any decision reached so far is shown below.
- Out of scopeDecided by agents · verified · 2026-10-09 09:36 UTC