Proof Portal
Omarchy
ProbeLabs65 findings · 95 requirementsA proof layer — requirements, tests and verified fixes — for two of Omarchy's subsystems: the application menu (launcher scripts, QML model, JSONC config, search and selection) and the lock screen (lock scripts, QML, PAM authentication). Scope is deliberately limited to those components of omacom/omarchy; the rest of the distribution is not covered.
herdr: remote client aborts (SIGABRT) on teardown — panic hook panics on a dead tty
Reported by lucletoffe · filed Sep 3, 2026Closed upstream on 2026-09-21 by bjarneo via an automated message, as a duplicate of #7331 (R-1700). GitHub close reason: completed.
At a glance
- 1 · What is claimed?
- Only the report text below — no validator has restated it as checkable claims yet.
- 2 · What does it affect?
- Not yet mapped to any requirement or component.
- 3 · Is it reproducible?
- No validation has been run yet.
- 4 · What remains unclear?
- Not validated yet: no claims, no reproducer.
- 5 · What happens next?
- Our staff decide whether the report is valid. Where this project allows it, agents decide whether it duplicates another report or a known issue. No decision has been recorded yet.
Original report
Summary
herdr client in remote mode aborts with SIGABRT and dumps core on shutdown, when the
terminal it was attached to is already gone. The failure is a panic-inside-the-panic-hook:
herdr's goodbye message goes to stderr, the write returns EIO, std panics, and the
terminal-restoring panic hook then panics on the same dead tty. Result:thread panicked while processing panic. aborting. → abort().
Nothing is lost — the server and all panes survive — but every remote-client teardown of
this kind leaves a core dump and fires a "Process crashed" desktop notification.
Environment
- Omarchy 4.0.0.alpha, Arch, kernel 7.1.8-arch1-3
herdr 0.8.0.r13-1, kitty 0.48.2-1, Hyprland 0.56.2-1- Client launched as
kitty --class herdr-main --title herdr -- herdr, remote mode
(herdr --remote <host>), SSH bridge socket /tmp/herdr-remote-<pid>-<host>-default.sock
Reproduction
Happened three times in 90 seconds, deterministically, on every remote client that
completed its handshake:
| PID | connected | aborted | lifetime |
|---|---|---|---|
| 4022319 | 23:38:24.96 | 23:38:37.70 | 13 s |
| 4030302 | 23:38:28.53 | 23:38:40.94 | 13 s |
| 4098219 | 23:39:44.71 | 23:40:01.75 | 17 s |
Two weeks of ~/.config/herdr/herdr-client.log show zero aborts for clients on the
local socket (~/.config/herdr/herdr-client.sock) — only remote-mode clients are affected.
Server side logged a clean client disconnected for each one.
Evidence
All three cores carry the same Rust panic payload:
failed printing to stderr: Input/output error (os error 5)The backtrace enters the panic runtime twice, around a pair of herdr frames:
#19–#25 main → app loop
#13–#18 first panic (panic runtime)
#11–#12 herdr + 0xfa30ba / 0xfa2963 <-- panic hook
#5–#10 second panic, raised inside the hook
#3–#4 abort()
#2 abort (libc.so.6 + 0x25685)The package is stripped, so I resolved the frames by disassembling and reading the.rodata they reference:
- frame #19 (
herdr + 0xabe7da) loads the string at0xdc573:
"herdr: " … "Run \{}\ to reattach" — the shutdown message, printed to stderr.
The same region holds "If the remote server survived the SSH or network drop, its panes may still be running."
- frames #11–#12 (herdr + 0xfa30ba) load 0xb3210 = \x1b[?1049l (leave alternate
screen) and 0xd13b9 = "Failed to restore terminal: " — i.e. the panic hook restoring
the terminal and printing the failure. That print is the second panic.
The binary also contains `"failed to install termination handler; terminal restore relies on
TerminalGuard::Drop and the panic hook"`, confirming the hook is the intended restore path.
Chain
- Remote client session ends.
- Client prints its goodbye / reattach hint to stderr.
- The tty is already gone → write returns
EIO→std::io::print_topanics. - The panic hook tries to restore the terminal and print
Failed to restore terminal: …
to the same dead tty → panics again.
5. Double panic → abort() → SIGABRT (si_code: SI_TKILL) + core dump.
Ruled out on evidence: not OOM (46/62 GiB used, no OOM kill in the journal), not a recent
update (herdr installed 11 days before), and not an SSH drop — the sshd sessions for
the bridge were still open at the moment of each abort.
Suggested fix
Both writes on the teardown path should be infallible:
- the goodbye/reattach message: tolerate a failed stderr write (
let _ = writeln!(…)) rather
than going through println!/eprintln!, which panics on error;
- the panic hook: never unwrap/panic on terminal-restore I/O — a hook that can panic turns
any recoverable teardown error into abort() and a core dump.
An EIO on stderr during shutdown is an expected condition when the hosting terminal closes
first; it should be a clean exit, not a crash.
View raw text
### Summary
`herdr client` in remote mode aborts with SIGABRT and dumps core on **shutdown**, when the
terminal it was attached to is already gone. The failure is a panic-inside-the-panic-hook:
herdr's goodbye message goes to stderr, the write returns `EIO`, `std` panics, and the
terminal-restoring panic hook then panics on the same dead tty. Result:
`thread panicked while processing panic. aborting.` → `abort()`.
Nothing is lost — the server and all panes survive — but every remote-client teardown of
this kind leaves a core dump and fires a "Process crashed" desktop notification.
### Environment
- Omarchy 4.0.0.alpha, Arch, kernel 7.1.8-arch1-3
- `herdr 0.8.0.r13-1`, kitty 0.48.2-1, Hyprland 0.56.2-1
- Client launched as `kitty --class herdr-main --title herdr -- herdr`, remote mode
(`herdr --remote <host>`), SSH bridge socket `/tmp/herdr-remote-<pid>-<host>-default.sock`
### Reproduction
Happened three times in 90 seconds, deterministically, on every remote client that
completed its handshake:
| PID | connected | aborted | lifetime |
|---|---|---|---|
| 4022319 | 23:38:24.96 | 23:38:37.70 | 13 s |
| 4030302 | 23:38:28.53 | 23:38:40.94 | 13 s |
| 4098219 | 23:39:44.71 | 23:40:01.75 | 17 s |
Two weeks of `~/.config/herdr/herdr-client.log` show **zero** aborts for clients on the
local socket (`~/.config/herdr/herdr-client.sock`) — only remote-mode clients are affected.
Server side logged a clean `client disconnected` for each one.
### Evidence
All three cores carry the same Rust panic payload:
```
failed printing to stderr: Input/output error (os error 5)
```
The backtrace enters the panic runtime **twice**, around a pair of herdr frames:
```
#19–#25 main → app loop
#13–#18 first panic (panic runtime)
#11–#12 herdr + 0xfa30ba / 0xfa2963 <-- panic hook
#5–#10 second panic, raised inside the hook
#3–#4 abort()
#2 abort (libc.so.6 + 0x25685)
```
The package is stripped, so I resolved the frames by disassembling and reading the
`.rodata` they reference:
- **frame #19** (`herdr + 0xabe7da`) loads the string at `0xdc573`:
`"herdr: "` … `"Run \`{}\` to reattach"` — the shutdown message, printed to stderr.
The same region holds `"If the remote server survived the SSH or network drop, its panes may still be running."`
- **frames #11–#12** (`herdr + 0xfa30ba`) load `0xb3210` = `\x1b[?1049l` (leave alternate
screen) and `0xd13b9` = `"Failed to restore terminal: "` — i.e. the panic hook restoring
the terminal and printing the failure. That print is the second panic.
The binary also contains `"failed to install termination handler; terminal restore relies on
TerminalGuard::Drop and the panic hook"`, confirming the hook is the intended restore path.
### Chain
1. Remote client session ends.
2. Client prints its goodbye / reattach hint to **stderr**.
3. The tty is already gone → write returns `EIO` → `std::io::print_to` panics.
4. The panic hook tries to restore the terminal and print `Failed to restore terminal: …`
to the same dead tty → panics again.
5. Double panic → `abort()` → SIGABRT (`si_code: SI_TKILL`) + core dump.
Ruled out on evidence: not OOM (46/62 GiB used, no OOM kill in the journal), not a recent
update (herdr installed 11 days before), and **not** an SSH drop — the `sshd` sessions for
the bridge were still open at the moment of each abort.
### Suggested fix
Both writes on the teardown path should be infallible:
- the goodbye/reattach message: tolerate a failed stderr write (`let _ = writeln!(…)`) rather
than going through `println!`/`eprintln!`, which panics on error;
- the panic hook: never `unwrap`/panic on terminal-restore I/O — a hook that can panic turns
any recoverable teardown error into `abort()` and a core dump.
An `EIO` on stderr during shutdown is an expected condition when the hosting terminal closes
first; it should be a clean exit, not a crash.
Validation
Not checked yet. A validator will restate this report as checkable claims, map it to the project's requirements and write a reproducer that the proof engine runs.
Linked
Not linked to any known issue, other report or change yet.
Decision
Decisions are recorded by our staff — or, where this project allows it, by agents: checked by a second, independent agent, or by a single agent where the project's policy says so. Any decision reached so far is shown below.
No decision has been recorded yet.