Proof Portal

Project overview

Omarchy

ProbeLabs65 findings · 95 requirements

A proof layer — requirements, tests and verified fixes — for two of Omarchy's subsystems: the application menu (launcher scripts, QML model, JSONC config, search and selection) and the lock screen (lock scripts, QML, PAM authentication). Scope is deliberately limited to those components of omacom/omarchy; the rest of the distribution is not covered.

Inbox
ReportR-4868GitHub issuePublicReceived

herdr: remote client aborts (SIGABRT) on teardown — panic hook panics on a dead tty

Reported by lucletoffe · filed Sep 3, 2026Closed upstream on 2026-09-21 by bjarneo via an automated message, as a duplicate of #7331 (R-1700). GitHub close reason: completed.

Received and recorded verbatim. Nothing has been checked yet.

At a glance

1 · What is claimed?
Only the report text below — no validator has restated it as checkable claims yet.
2 · What does it affect?
Not yet mapped to any requirement or component.
3 · Is it reproducible?
No validation has been run yet.
4 · What remains unclear?
  • Not validated yet: no claims, no reproducer.
5 · What happens next?
Our staff decide whether the report is valid. Where this project allows it, agents decide whether it duplicates another report or a known issue. No decision has been recorded yet.

Original report

herdr: remote client aborts (SIGABRT) on teardown — panic hook panics on a dead ttyrevision 1 · captured 2026-10-04 12:10 UTC via poll

Summary

herdr client in remote mode aborts with SIGABRT and dumps core on shutdown, when the
terminal it was attached to is already gone. The failure is a panic-inside-the-panic-hook:
herdr's goodbye message goes to stderr, the write returns EIO, std panics, and the
terminal-restoring panic hook then panics on the same dead tty. Result:
thread panicked while processing panic. aborting. → abort().

Nothing is lost — the server and all panes survive — but every remote-client teardown of
this kind leaves a core dump and fires a "Process crashed" desktop notification.

Environment

  • Omarchy 4.0.0.alpha, Arch, kernel 7.1.8-arch1-3
  • herdr 0.8.0.r13-1, kitty 0.48.2-1, Hyprland 0.56.2-1
  • Client launched as kitty --class herdr-main --title herdr -- herdr, remote mode

(herdr --remote <host>), SSH bridge socket /tmp/herdr-remote-<pid>-<host>-default.sock

Reproduction

Happened three times in 90 seconds, deterministically, on every remote client that
completed its handshake:

PIDconnectedabortedlifetime
402231923:38:24.9623:38:37.7013 s
403030223:38:28.5323:38:40.9413 s
409821923:39:44.7123:40:01.7517 s

Two weeks of ~/.config/herdr/herdr-client.log show zero aborts for clients on the
local socket (~/.config/herdr/herdr-client.sock) — only remote-mode clients are affected.
Server side logged a clean client disconnected for each one.

Evidence

All three cores carry the same Rust panic payload:

failed printing to stderr: Input/output error (os error 5)

The backtrace enters the panic runtime twice, around a pair of herdr frames:

#19–#25  main → app loop
#13–#18  first panic (panic runtime)
#11–#12  herdr + 0xfa30ba / 0xfa2963   <-- panic hook
#5–#10   second panic, raised inside the hook
#3–#4    abort()
#2       abort (libc.so.6 + 0x25685)

The package is stripped, so I resolved the frames by disassembling and reading the
.rodata they reference:

  • frame #19 (herdr + 0xabe7da) loads the string at 0xdc573:

"herdr: " … "Run \{}\ to reattach" — the shutdown message, printed to stderr.
The same region holds "If the remote server survived the SSH or network drop, its panes may still be running."
- frames #11–#12 (herdr + 0xfa30ba) load 0xb3210 = \x1b[?1049l (leave alternate
screen) and 0xd13b9 = "Failed to restore terminal: " — i.e. the panic hook restoring
the terminal and printing the failure. That print is the second panic.

The binary also contains `"failed to install termination handler; terminal restore relies on
TerminalGuard::Drop and the panic hook"`, confirming the hook is the intended restore path.

Chain

  1. Remote client session ends.
  2. Client prints its goodbye / reattach hint to stderr.
  3. The tty is already gone → write returns EIO → std::io::print_to panics.
  4. The panic hook tries to restore the terminal and print Failed to restore terminal: …

to the same dead tty → panics again.
5. Double panic → abort() → SIGABRT (si_code: SI_TKILL) + core dump.

Ruled out on evidence: not OOM (46/62 GiB used, no OOM kill in the journal), not a recent
update (herdr installed 11 days before), and not an SSH drop — the sshd sessions for
the bridge were still open at the moment of each abort.

Suggested fix

Both writes on the teardown path should be infallible:

  • the goodbye/reattach message: tolerate a failed stderr write (let _ = writeln!(…)) rather

than going through println!/eprintln!, which panics on error;
- the panic hook: never unwrap/panic on terminal-restore I/O — a hook that can panic turns
any recoverable teardown error into abort() and a core dump.

An EIO on stderr during shutdown is an expected condition when the hosting terminal closes
first; it should be a clean exit, not a crash.

View raw text
### Summary

`herdr client` in remote mode aborts with SIGABRT and dumps core on **shutdown**, when the
terminal it was attached to is already gone. The failure is a panic-inside-the-panic-hook:
herdr's goodbye message goes to stderr, the write returns `EIO`, `std` panics, and the
terminal-restoring panic hook then panics on the same dead tty. Result:
`thread panicked while processing panic. aborting.` → `abort()`.

Nothing is lost — the server and all panes survive — but every remote-client teardown of
this kind leaves a core dump and fires a "Process crashed" desktop notification.

### Environment

- Omarchy 4.0.0.alpha, Arch, kernel 7.1.8-arch1-3
- `herdr 0.8.0.r13-1`, kitty 0.48.2-1, Hyprland 0.56.2-1
- Client launched as `kitty --class herdr-main --title herdr -- herdr`, remote mode
  (`herdr --remote <host>`), SSH bridge socket `/tmp/herdr-remote-<pid>-<host>-default.sock`

### Reproduction

Happened three times in 90 seconds, deterministically, on every remote client that
completed its handshake:

| PID | connected | aborted | lifetime |
|---|---|---|---|
| 4022319 | 23:38:24.96 | 23:38:37.70 | 13 s |
| 4030302 | 23:38:28.53 | 23:38:40.94 | 13 s |
| 4098219 | 23:39:44.71 | 23:40:01.75 | 17 s |

Two weeks of `~/.config/herdr/herdr-client.log` show **zero** aborts for clients on the
local socket (`~/.config/herdr/herdr-client.sock`) — only remote-mode clients are affected.
Server side logged a clean `client disconnected` for each one.

### Evidence

All three cores carry the same Rust panic payload:

```
failed printing to stderr: Input/output error (os error 5)
```

The backtrace enters the panic runtime **twice**, around a pair of herdr frames:

```
#19–#25  main → app loop
#13–#18  first panic (panic runtime)
#11–#12  herdr + 0xfa30ba / 0xfa2963   <-- panic hook
#5–#10   second panic, raised inside the hook
#3–#4    abort()
#2       abort (libc.so.6 + 0x25685)
```

The package is stripped, so I resolved the frames by disassembling and reading the
`.rodata` they reference:

- **frame #19** (`herdr + 0xabe7da`) loads the string at `0xdc573`:
  `"herdr: "` … `"Run \`{}\` to reattach"` — the shutdown message, printed to stderr.
  The same region holds `"If the remote server survived the SSH or network drop, its panes may still be running."`
- **frames #11–#12** (`herdr + 0xfa30ba`) load `0xb3210` = `\x1b[?1049l` (leave alternate
  screen) and `0xd13b9` = `"Failed to restore terminal: "` — i.e. the panic hook restoring
  the terminal and printing the failure. That print is the second panic.

The binary also contains `"failed to install termination handler; terminal restore relies on
TerminalGuard::Drop and the panic hook"`, confirming the hook is the intended restore path.

### Chain

1. Remote client session ends.
2. Client prints its goodbye / reattach hint to **stderr**.
3. The tty is already gone → write returns `EIO` → `std::io::print_to` panics.
4. The panic hook tries to restore the terminal and print `Failed to restore terminal: …`
   to the same dead tty → panics again.
5. Double panic → `abort()` → SIGABRT (`si_code: SI_TKILL`) + core dump.

Ruled out on evidence: not OOM (46/62 GiB used, no OOM kill in the journal), not a recent
update (herdr installed 11 days before), and **not** an SSH drop — the `sshd` sessions for
the bridge were still open at the moment of each abort.

### Suggested fix

Both writes on the teardown path should be infallible:

- the goodbye/reattach message: tolerate a failed stderr write (`let _ = writeln!(…)`) rather
  than going through `println!`/`eprintln!`, which panics on error;
- the panic hook: never `unwrap`/panic on terminal-restore I/O — a hook that can panic turns
  any recoverable teardown error into `abort()` and a core dump.

An `EIO` on stderr during shutdown is an expected condition when the hosting terminal closes
first; it should be a clean exit, not a crash.

Validation

Not checked yet. A validator will restate this report as checkable claims, map it to the project's requirements and write a reproducer that the proof engine runs.

Linked

Not linked to any known issue, other report or change yet.

Decision

Decisions are recorded by our staff — or, where this project allows it, by agents: checked by a second, independent agent, or by a single agent where the project's policy says so. Any decision reached so far is shown below.

No decision has been recorded yet.