Proof Portal
Omarchy
ProbeLabs65 findings · 95 requirementsA proof layer — requirements, tests and verified fixes — for two of Omarchy's subsystems: the application menu (launcher scripts, QML model, JSONC config, search and selection) and the lock screen (lock scripts, QML, PAM authentication). Scope is deliberately limited to those components of omacom/omarchy; the rest of the distribution is not covered.
initcpio: btrfs-overlayfs hook unconditionally overlays / for snapper boots, defeating fstab-level fixes
Reported by arief-lab · filed Aug 8, 2026 · open at source
At a glance
- 1 · What is claimed?
- Only the report text below — no validator has restated it as checkable claims yet.
- 2 · What does it affect?
- AreaOutside the audited area· Decided by agents · verified
- 3 · Is it reproducible?
- No validation has been run yet.
- 4 · What remains unclear?
- Not validated yet: no claims, no reproducer.
- 5 · What happens next?
- Decided by agents and verified by a second, independent agent — see the decision.
Original report
System details
Hostname: omarchy | OS: Arch Linux x86_64 (Omarchy 3.8.4) | Kernel (running): 7.1.3-arch1-2 | CPU: AMD Ryzen 5 1600 | GPU: NVIDIA GeForce GT 1030 | Memory: 7.7 GiB total | DE/WM: Hyprland 0.55.4 + waybar, SDDM, Wayland | btrfs-progs: v7.0 | Filesystem: btrfs on LUKS2 (/dev/mapper/root), @ subvolid 287 | Disk: 221.6 GiB root | limine-snapper-sync: 1.30.1-1 | Snapshot tool: snapper (-c root) | Bios/UEFI: UEFI boot via Limine
What's wrong?
Why this matters
The hook is the second layer of the same systemd-remount-fs failure that Issue A #6633 addresses. Maintenance releases that only fix Issue A will leave users in a partial fix whose first pacman install reverts them to the red splash on every snapper-menu boot.
The hook has been a deterministic installer output from day one of the install — no opt-out, no fstab-aware gating, and the mkinitcpio -P rebuild fires it into the new initramfs on every kernel upgrade. Observed rebuilds on this machine re-affirmed the bug on 2026-06-02, 2026-07-14, 2026-07-25.
A user who reads "Issue A #6633 is fixed" and stops there will hold an incomplete mental model: live boot will be clean, but the snapper menu recovery path they trusted will still hit the red banner on first use. Tracking both layers together is required for a clean fix.
Summary
After fixing /etc/fstab to use a numeric subvolid= plus x-systemd.remount=false (companion Issue A #6633), booting a snapper snapshot from Limine still fails with the same surface error. mount | grep ' on / ' shows the reason:
overlay on / type overlay (rw,relatime,
lowerdir=/lowdir_XXX,
upperdir=/ramdir_XXX/upper,
workdir=/ramdir_XXX/work,uuid=on)The overlay is set up by Omarchy's btrfs-overlayfs initcpio runtime hook, baked into every initramfs via /etc/mkinitcpio.conf.d/omarchy_hooks.conf. The hook fires whenever the booted subvol path contains the literal substring /snapshot — i.e. every Limine snapper menu entry. There is no escape hatch; systemd-remount-fs.service then has nothing to apply fstab options to.
Steps to reproduce
Prereq: Omarchy install with both (a) the fstab/cmdline workaround from Issue A #6633 applied, and (b) Omarchy's default /etc/mkinitcpio.conf.d/omarchy_hooks.conf (which appends btrfs-overlayfs to HOOKS).
- Create a fresh snapper snapshot:
sudo snapper -c root create --description "boot-fix-verification"- Reboot. From Limine, pick
Omarchy snapper ▸→ the newest entry. - Observe:
● systemd-remount-fs.service ... Active: failed (Result: exit-code)
mount: /: fsconfig() failed: overlay: No changes allowed in reconfigure.- Confirm with:
findmnt -n -o FSTYPE / # returns: overlay (NOT btrfs)
mount | grep ' on / ' # shows lowerdir=/lowdir_XXX, upperdir=/ramdir_XXX/upper, workdir=/ramdir_XXX/workRoot cause traced
| Where | What's happening | |
|---|---|---|
/etc/mkinitcpio.conf.d/omarchy_hooks.conf | Appends btrfs-overlayfs to HOOKS (line at the end, after filesystems fsck). | |
mkinitcpio -P (install + regen after kernel updates) | bakes hooks/btrfs-overlayfs into initramfs-linux. | |
journalctl -b | Lists mounts for /home, /var/log, /var/cache/pacman/pkg, /boot, /tmp — no Mounting /… line, so systemd didn't place the overlay. | |
| Kernel journal line | overlayfs: "xino" feature enabled using 2 upper inode bits. is logged before any systemd unit, confirming the overlay is created in initramfs. | |
| `lsinitcpio .../initramfs-linux \ | grep btrfs-overlayfs` | Confirms the hook is in the initramfs. |
| Conclusion | The btrfs-overlayfs latehook in the initramfs is what places the overlay. |
/usr/lib/initcpio/hooks/btrfs-overlayfs (the runtime body) wraps the kernel mount at /. with an overlayfs (lower = the btrfs snapper subvol, upper + work = /ramdir_XXX tmpfs), unconditionally on any subvol path that contains /snapshot. There is no gate keyed off the user's fstab. The user's x-systemd.remount=false in /etc/fstab is silently overridden because the hook fires before systemd ever starts.
Suggested fix
Ship a fstab-aware guard in the runtime hook. When /etc/fstab already declares / with both numeric subvolid=<N> and x-systemd.remount=false — the user's explicit signal that they're opting out of systemd-remount-fs for / — the hook should skip the overlay entirely:
# Prepend this guard at the top of run_latehook()
# in /usr/lib/initcpio/hooks/btrfs-overlayfs:
run_latehook() {
if [ -f /etc/fstab ] \
&& grep -qE '^[[:space:]]*UUID=[^[:space:]]+[[:space:]]+/[[:space:]]+[^[:space:]]+.*subvolid=[0-9]+.*x-systemd\.remount=false' /etc/fstab 2>/dev/null; then
msg "btrfs-overlayfs: /etc/fstab opts out of remount; skipping overlay"
return 0
fi
# ... existing body continues unchanged ...
}This preserves Omarchy's design intent (overlay for users on legacy fstab layouts) AND silences the failure for users on the modern fstab layout. No omarchy_hooks.conf change needed; only the runtime hook body.
If you'd rather not edit the Omarchy-shipped file in-place, ship a parallel 90-omarchy-snapshot-fstab-aware hook that runs before btrfs-overlayfs and exits early on the same condition. Either way works.
Workaround applied locally (verified end-to-end)
The exact patch is preserved in /tmp/apply-option-c.sh (on the author's machine). Steps:
- Backup the runtime hook:
sudo cp /usr/lib/initcpio/hooks/btrfs-overlayfs \
/usr/lib/initcpio/hooks/btrfs-overlayfs.bak.$(date +%s)- Append the fstab-aware guard (see "Suggested fix" snippet above).
- Seed the
localbakclone:
sudo cp -a /usr/lib/initcpio/hooks/btrfs-overlayfs \
/usr/lib/initcpio/hooks/btrfs-overlayfs.localbak- Install a NoUpgrade alpm hook so Omarchy package updates can't clobber the guard (
/etc/pacman.d/hooks/90-nosave-btrfs-overlayfs.hook):
[Trigger]
Operation = Install
Operation = Upgrade
Type = Path
Target = usr/lib/initcpio/hooks/btrfs-overlayfs
[Action]
Description = Preserve local btrfs-overlayfs fstab-aware guard
When = PreTransaction
Exec = /bin/cp -a /usr/lib/initcpio/hooks/btrfs-overlayfs.localbak /usr/lib/initcpio/hooks/btrfs-overlayfs
NeedsTargets- Re-bake the initramfs:
sudo mkinitcpio -P (mkinitcpio -P also invokes limine-mkinitcpio to refresh /boot/limine.conf.)
Verification result (apply Issue A + this fix together for these results)
Live boot, 2026-08-08 17:32 WIB (top-level `Omarchy` entry, post-patch):
● systemd-remount-fs.service ... Main PID: 513 (code=exited, status=0/SUCCESS)
0 loaded units listed. # systemctl --failed
btrfs # findmnt -n -o FSTYPE /
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=287,subvol=/@)Snapshot boot, 2026-08-08 17:39 (booted into `#21 boot-fix-verification-c`):
● systemd-remount-fs.service ... Main PID: 512 (code=exited, status=0/SUCCESS)
btrfs # findmnt -n -o FSTYPE / (was "overlay" before this fix)
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=290,subvol=/@/.snapshots/21/snapshot)Round-trip live boot, 2026-08-08 17:46 WIB (after returning from snapshot `#21`):
● systemd-remount-fs.service ... Main PID: 512 (code=exited, status=0/SUCCESS)
0 loaded units listed. # systemctl --failed — empty, proving the 10 unrelated snapshot-boot "failures" aren't regressions
btrfs # findmnt -n -o FSTYPE /
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=287,subvol=/@)Snapshot boots also show 10 unrelated failed units (docker.service,containerd.service,warp-svc.service, gpg-agent*/dirmngr sockets) becausesnapper createonly snapshots the@subvol —/usris frozen while/var//home//bootcome from the live subvols, producing a version skew with nothing to do with this fix. They are absent on the live boot.
Severity
High. Even after Issue A is fixed upstream, snapper-menu boots still fail unless this hook is also fixed or guarded. The bug re-asserts itself on every kernel upgrade — mkinitcpio rebuilds observed at 2026-06-02, 2026-07-14, 2026-07-25 all re-installed this hook into the initramfs on this install. A user who only tracks "Issue A is fixed" will hold an incomplete mental model; partial-fix users will still see the red splash on snapper-menu boots.
Labels
bug btrfs boot limine initcpio snapper
Related
A companion issue (#6633: installer: ship subvolid + x-systemd.remount=false in /etc/fstab (and btrfs opts in cmdline)) covers the first layer of this failure — the fstab/cmdline disagreement that allows subvol= reconfigure attempts at all. This issue and #6633 are filed together; both fixes are required for snapper-menu boots to come up clean on a default install. Apply #6633 first (it's the prerequisite for the symptom to even appear), then this fix on top.
View raw text
### System details
Hostname: omarchy | OS: Arch Linux x86_64 (Omarchy 3.8.4) | Kernel (running): 7.1.3-arch1-2 | CPU: AMD Ryzen 5 1600 | GPU: NVIDIA GeForce GT 1030 | Memory: 7.7 GiB total | DE/WM: Hyprland 0.55.4 + waybar, SDDM, Wayland | btrfs-progs: v7.0 | Filesystem: btrfs on LUKS2 (/dev/mapper/root), @ subvolid 287 | Disk: 221.6 GiB root | limine-snapper-sync: 1.30.1-1 | Snapshot tool: snapper (-c root) | Bios/UEFI: UEFI boot via Limine
### What's wrong?
### Why this matters
The hook is the **second layer** of the same `systemd-remount-fs` failure that Issue A #6633 addresses. Maintenance releases that *only* fix Issue A will leave users in a partial fix whose first pacman install reverts them to the red splash on every snapper-menu boot.
The hook has been a deterministic installer output from day one of the install — no opt-out, no fstab-aware gating, and the `mkinitcpio -P` rebuild fires it into the new initramfs on every kernel upgrade. Observed rebuilds on this machine re-affirmed the bug on **2026-06-02, 2026-07-14, 2026-07-25**.
A user who reads "Issue A #6633 is fixed" and stops there will hold an incomplete mental model: live boot will be clean, but the snapper menu recovery path they trusted will still hit the red banner on first use. Tracking both layers together is required for a clean fix.
### Summary
After fixing `/etc/fstab` to use a numeric `subvolid=` plus `x-systemd.remount=false` (companion **Issue A #6633**), booting a snapper snapshot from Limine **still fails** with the same surface error. `mount | grep ' on / '` shows the reason:
```
overlay on / type overlay (rw,relatime,
lowerdir=/lowdir_XXX,
upperdir=/ramdir_XXX/upper,
workdir=/ramdir_XXX/work,uuid=on)
```
The overlay is set up by Omarchy's `btrfs-overlayfs` initcpio runtime hook, baked into every initramfs via `/etc/mkinitcpio.conf.d/omarchy_hooks.conf`. The hook fires whenever the booted subvol path contains the literal substring `/snapshot` — i.e. *every* Limine snapper menu entry. There is no escape hatch; `systemd-remount-fs.service` then has nothing to apply fstab options to.
### Steps to reproduce
Prereq: Omarchy install with both (a) the fstab/cmdline workaround from Issue A #6633 applied, **and** (b) Omarchy's default `/etc/mkinitcpio.conf.d/omarchy_hooks.conf` (which appends `btrfs-overlayfs` to HOOKS).
1. Create a fresh snapper snapshot:
```bash
sudo snapper -c root create --description "boot-fix-verification"
```
2. Reboot. From Limine, pick `Omarchy snapper ▸` → the newest entry.
3. Observe:
```
● systemd-remount-fs.service ... Active: failed (Result: exit-code)
mount: /: fsconfig() failed: overlay: No changes allowed in reconfigure.
```
4. Confirm with:
```bash
findmnt -n -o FSTYPE / # returns: overlay (NOT btrfs)
mount | grep ' on / ' # shows lowerdir=/lowdir_XXX, upperdir=/ramdir_XXX/upper, workdir=/ramdir_XXX/work
```
### Root cause traced
| Where | What's happening |
|---|---|
| `/etc/mkinitcpio.conf.d/omarchy_hooks.conf` | Appends `btrfs-overlayfs` to HOOKS (line at the end, after `filesystems fsck`). |
| `mkinitcpio -P` (install + regen after kernel updates) | bakes `hooks/btrfs-overlayfs` into `initramfs-linux`. |
| `journalctl -b` | Lists mounts for `/home`, `/var/log`, `/var/cache/pacman/pkg`, `/boot`, `/tmp` — **no** `Mounting /…` line, so systemd didn't place the overlay. |
| Kernel journal line | `overlayfs: "xino" feature enabled using 2 upper inode bits.` is logged *before* any systemd unit, confirming the overlay is created in initramfs. |
| `lsinitcpio .../initramfs-linux \| grep btrfs-overlayfs` | Confirms the hook is in the initramfs. |
| **Conclusion** | The `btrfs-overlayfs` latehook in the initramfs is what places the overlay. |
`/usr/lib/initcpio/hooks/btrfs-overlayfs` (the runtime body) wraps the kernel mount at `/.` with an overlayfs (lower = the btrfs snapper subvol, upper + work = `/ramdir_XXX` tmpfs), unconditionally on any subvol path that contains `/snapshot`. There is no gate keyed off the user's fstab. The user's `x-systemd.remount=false` in `/etc/fstab` is silently overridden because the hook fires before systemd ever starts.
### Suggested fix
Ship a fstab-aware guard in the runtime hook. When `/etc/fstab` already declares `/` with both numeric `subvolid=<N>` **and** `x-systemd.remount=false` — the user's explicit signal that they're opting out of systemd-remount-fs for `/` — the hook should skip the overlay entirely:
```bash
# Prepend this guard at the top of run_latehook()
# in /usr/lib/initcpio/hooks/btrfs-overlayfs:
run_latehook() {
if [ -f /etc/fstab ] \
&& grep -qE '^[[:space:]]*UUID=[^[:space:]]+[[:space:]]+/[[:space:]]+[^[:space:]]+.*subvolid=[0-9]+.*x-systemd\.remount=false' /etc/fstab 2>/dev/null; then
msg "btrfs-overlayfs: /etc/fstab opts out of remount; skipping overlay"
return 0
fi
# ... existing body continues unchanged ...
}
```
This preserves Omarchy's design intent (overlay for users on legacy fstab layouts) AND silences the failure for users on the modern fstab layout. No `omarchy_hooks.conf` change needed; only the runtime hook body.
If you'd rather not edit the Omarchy-shipped file in-place, ship a parallel `90-omarchy-snapshot-fstab-aware` hook that runs *before* `btrfs-overlayfs` and exits early on the same condition. Either way works.
### Workaround applied locally (verified end-to-end)
The exact patch is preserved in `/tmp/apply-option-c.sh` (on the author's machine). Steps:
1. Backup the runtime hook:
```bash
sudo cp /usr/lib/initcpio/hooks/btrfs-overlayfs \
/usr/lib/initcpio/hooks/btrfs-overlayfs.bak.$(date +%s)
```
2. Append the fstab-aware guard (see "Suggested fix" snippet above).
3. Seed the `localbak` clone:
```bash
sudo cp -a /usr/lib/initcpio/hooks/btrfs-overlayfs \
/usr/lib/initcpio/hooks/btrfs-overlayfs.localbak
```
4. Install a NoUpgrade alpm hook so Omarchy package updates can't clobber the guard (`/etc/pacman.d/hooks/90-nosave-btrfs-overlayfs.hook`):
```ini
[Trigger]
Operation = Install
Operation = Upgrade
Type = Path
Target = usr/lib/initcpio/hooks/btrfs-overlayfs
[Action]
Description = Preserve local btrfs-overlayfs fstab-aware guard
When = PreTransaction
Exec = /bin/cp -a /usr/lib/initcpio/hooks/btrfs-overlayfs.localbak /usr/lib/initcpio/hooks/btrfs-overlayfs
NeedsTargets
```
5. Re-bake the initramfs:
```bash
sudo mkinitcpio -P
```
(`mkinitcpio -P` also invokes `limine-mkinitcpio` to refresh `/boot/limine.conf`.)
### Verification result (apply Issue A + this fix together for these results)
**Live boot, 2026-08-08 17:32 WIB (top-level `Omarchy` entry, post-patch):**
```
● systemd-remount-fs.service ... Main PID: 513 (code=exited, status=0/SUCCESS)
0 loaded units listed. # systemctl --failed
btrfs # findmnt -n -o FSTYPE /
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=287,subvol=/@)
```
**Snapshot boot, 2026-08-08 17:39 (booted into `#21 boot-fix-verification-c`):**
```
● systemd-remount-fs.service ... Main PID: 512 (code=exited, status=0/SUCCESS)
btrfs # findmnt -n -o FSTYPE / (was "overlay" before this fix)
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=290,subvol=/@/.snapshots/21/snapshot)
```
**Round-trip live boot, 2026-08-08 17:46 WIB (after returning from snapshot `#21`):**
```
● systemd-remount-fs.service ... Main PID: 512 (code=exited, status=0/SUCCESS)
0 loaded units listed. # systemctl --failed — empty, proving the 10 unrelated snapshot-boot "failures" aren't regressions
btrfs # findmnt -n -o FSTYPE /
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=287,subvol=/@)
```
> Snapshot boots also show 10 unrelated failed units (`docker.service`, `containerd.service`, `warp-svc.service`, gpg-agent*/dirmngr sockets) because `snapper create` only snapshots the `@` subvol — `/usr` is frozen while `/var`/`/home`/`/boot` come from the live subvols, producing a version skew with nothing to do with this fix. They are absent on the live boot.
### Severity
**High.** Even after Issue A is fixed upstream, snapper-menu boots still fail unless this hook is also fixed or guarded. The bug re-asserts itself on every kernel upgrade — mkinitcpio rebuilds observed at 2026-06-02, 2026-07-14, 2026-07-25 all re-installed this hook into the initramfs on this install. A user who only tracks "Issue A is fixed" will hold an incomplete mental model; partial-fix users will still see the red splash on snapper-menu boots.
### Labels
`bug` `btrfs` `boot` `limine` `initcpio` `snapper`
### Related
A companion issue (#6633: `installer: ship subvolid + x-systemd.remount=false in /etc/fstab (and btrfs opts in cmdline)`) covers the **first layer** of this failure — the fstab/cmdline disagreement that allows `subvol=` reconfigure attempts at all. This issue and #6633 are filed together; **both fixes are required** for snapper-menu boots to come up clean on a default install. Apply #6633 first (it's the prerequisite for the symptom to even *appear*), then this fix on top.
Validation
Not checked yet. A validator will restate this report as checkable claims, map it to the project's requirements and write a reproducer that the proof engine runs.
Linked
Not linked to any known issue, other report or change yet.
Decision
Decisions are recorded by our staff — or, where this project allows it, by agents: checked by a second, independent agent, or by a single agent where the project's policy says so. Any decision reached so far is shown below.
- Out of scopeDecided by agents · verified · 2026-10-09 04:00 UTC