Proof Portal

Project overview

Omarchy

ProbeLabs65 findings · 95 requirements

A proof layer — requirements, tests and verified fixes — for two of Omarchy's subsystems: the application menu (launcher scripts, QML model, JSONC config, search and selection) and the lock screen (lock scripts, QML, PAM authentication). Scope is deliberately limited to those components of omacom/omarchy; the rest of the distribution is not covered.

Inbox
ReportR-1879GitHub issuePublicDecided· Out of scope

initcpio: btrfs-overlayfs hook unconditionally overlays / for snapper boots, defeating fstab-level fixes

Reported by arief-lab · filed Aug 8, 2026 · open at source

Decided: Out of scope — decided by agents · verified 4 days ago.

At a glance

1 · What is claimed?
Only the report text below — no validator has restated it as checkable claims yet.
2 · What does it affect?
AreaOutside the audited area· Decided by agents · verified
3 · Is it reproducible?
No validation has been run yet.
4 · What remains unclear?
  • Not validated yet: no claims, no reproducer.
5 · What happens next?
Decided by agents and verified by a second, independent agent — see the decision.

Original report

initcpio: btrfs-overlayfs hook unconditionally overlays / for snapper boots, defeating fstab-level fixesrevision 1 · captured 2026-09-28 21:29 UTC via poll

System details

Hostname: omarchy | OS: Arch Linux x86_64 (Omarchy 3.8.4) | Kernel (running): 7.1.3-arch1-2 | CPU: AMD Ryzen 5 1600 | GPU: NVIDIA GeForce GT 1030 | Memory: 7.7 GiB total | DE/WM: Hyprland 0.55.4 + waybar, SDDM, Wayland | btrfs-progs: v7.0 | Filesystem: btrfs on LUKS2 (/dev/mapper/root), @ subvolid 287 | Disk: 221.6 GiB root | limine-snapper-sync: 1.30.1-1 | Snapshot tool: snapper (-c root) | Bios/UEFI: UEFI boot via Limine

What's wrong?

Why this matters

The hook is the second layer of the same systemd-remount-fs failure that Issue A #6633 addresses. Maintenance releases that only fix Issue A will leave users in a partial fix whose first pacman install reverts them to the red splash on every snapper-menu boot.

The hook has been a deterministic installer output from day one of the install — no opt-out, no fstab-aware gating, and the mkinitcpio -P rebuild fires it into the new initramfs on every kernel upgrade. Observed rebuilds on this machine re-affirmed the bug on 2026-06-02, 2026-07-14, 2026-07-25.

A user who reads "Issue A #6633 is fixed" and stops there will hold an incomplete mental model: live boot will be clean, but the snapper menu recovery path they trusted will still hit the red banner on first use. Tracking both layers together is required for a clean fix.

Summary

After fixing /etc/fstab to use a numeric subvolid= plus x-systemd.remount=false (companion Issue A #6633), booting a snapper snapshot from Limine still fails with the same surface error. mount | grep ' on / ' shows the reason:

overlay on / type overlay (rw,relatime,
  lowerdir=/lowdir_XXX,
  upperdir=/ramdir_XXX/upper,
  workdir=/ramdir_XXX/work,uuid=on)

The overlay is set up by Omarchy's btrfs-overlayfs initcpio runtime hook, baked into every initramfs via /etc/mkinitcpio.conf.d/omarchy_hooks.conf. The hook fires whenever the booted subvol path contains the literal substring /snapshot — i.e. every Limine snapper menu entry. There is no escape hatch; systemd-remount-fs.service then has nothing to apply fstab options to.

Steps to reproduce

Prereq: Omarchy install with both (a) the fstab/cmdline workaround from Issue A #6633 applied, and (b) Omarchy's default /etc/mkinitcpio.conf.d/omarchy_hooks.conf (which appends btrfs-overlayfs to HOOKS).

  1. Create a fresh snapper snapshot:
   sudo snapper -c root create --description "boot-fix-verification"
  1. Reboot. From Limine, pick Omarchy snapper ▸ → the newest entry.
  2. Observe:
   ● systemd-remount-fs.service ... Active: failed (Result: exit-code)
   mount: /: fsconfig() failed: overlay: No changes allowed in reconfigure.
  1. Confirm with:
   findmnt -n -o FSTYPE /       # returns: overlay  (NOT btrfs)
   mount | grep ' on / '         # shows lowerdir=/lowdir_XXX, upperdir=/ramdir_XXX/upper, workdir=/ramdir_XXX/work

Root cause traced

WhereWhat's happening
/etc/mkinitcpio.conf.d/omarchy_hooks.confAppends btrfs-overlayfs to HOOKS (line at the end, after filesystems fsck).
mkinitcpio -P (install + regen after kernel updates)bakes hooks/btrfs-overlayfs into initramfs-linux.
journalctl -bLists mounts for /home, /var/log, /var/cache/pacman/pkg, /boot, /tmp — no Mounting /… line, so systemd didn't place the overlay.
Kernel journal lineoverlayfs: "xino" feature enabled using 2 upper inode bits. is logged before any systemd unit, confirming the overlay is created in initramfs.
`lsinitcpio .../initramfs-linux \grep btrfs-overlayfs`Confirms the hook is in the initramfs.
ConclusionThe btrfs-overlayfs latehook in the initramfs is what places the overlay.

/usr/lib/initcpio/hooks/btrfs-overlayfs (the runtime body) wraps the kernel mount at /. with an overlayfs (lower = the btrfs snapper subvol, upper + work = /ramdir_XXX tmpfs), unconditionally on any subvol path that contains /snapshot. There is no gate keyed off the user's fstab. The user's x-systemd.remount=false in /etc/fstab is silently overridden because the hook fires before systemd ever starts.

Suggested fix

Ship a fstab-aware guard in the runtime hook. When /etc/fstab already declares / with both numeric subvolid=<N> and x-systemd.remount=false — the user's explicit signal that they're opting out of systemd-remount-fs for / — the hook should skip the overlay entirely:

# Prepend this guard at the top of run_latehook()
# in /usr/lib/initcpio/hooks/btrfs-overlayfs:
run_latehook() {
    if [ -f /etc/fstab ] \
       && grep -qE '^[[:space:]]*UUID=[^[:space:]]+[[:space:]]+/[[:space:]]+[^[:space:]]+.*subvolid=[0-9]+.*x-systemd\.remount=false' /etc/fstab 2>/dev/null; then
        msg "btrfs-overlayfs: /etc/fstab opts out of remount; skipping overlay"
        return 0
    fi
    # ... existing body continues unchanged ...
}

This preserves Omarchy's design intent (overlay for users on legacy fstab layouts) AND silences the failure for users on the modern fstab layout. No omarchy_hooks.conf change needed; only the runtime hook body.

If you'd rather not edit the Omarchy-shipped file in-place, ship a parallel 90-omarchy-snapshot-fstab-aware hook that runs before btrfs-overlayfs and exits early on the same condition. Either way works.

Workaround applied locally (verified end-to-end)

The exact patch is preserved in /tmp/apply-option-c.sh (on the author's machine). Steps:

  1. Backup the runtime hook:
   sudo cp /usr/lib/initcpio/hooks/btrfs-overlayfs \
           /usr/lib/initcpio/hooks/btrfs-overlayfs.bak.$(date +%s)
  1. Append the fstab-aware guard (see "Suggested fix" snippet above).
  2. Seed the localbak clone:
   sudo cp -a /usr/lib/initcpio/hooks/btrfs-overlayfs \
               /usr/lib/initcpio/hooks/btrfs-overlayfs.localbak
  1. Install a NoUpgrade alpm hook so Omarchy package updates can't clobber the guard (/etc/pacman.d/hooks/90-nosave-btrfs-overlayfs.hook):
   [Trigger]
   Operation = Install
   Operation = Upgrade
   Type = Path
   Target = usr/lib/initcpio/hooks/btrfs-overlayfs
   [Action]
   Description = Preserve local btrfs-overlayfs fstab-aware guard
   When = PreTransaction
   Exec = /bin/cp -a /usr/lib/initcpio/hooks/btrfs-overlayfs.localbak /usr/lib/initcpio/hooks/btrfs-overlayfs
   NeedsTargets
  1. Re-bake the initramfs:
   sudo mkinitcpio -P

(mkinitcpio -P also invokes limine-mkinitcpio to refresh /boot/limine.conf.)

Verification result (apply Issue A + this fix together for these results)

Live boot, 2026-08-08 17:32 WIB (top-level `Omarchy` entry, post-patch):

● systemd-remount-fs.service ... Main PID: 513 (code=exited, status=0/SUCCESS)
0 loaded units listed.    # systemctl --failed
btrfs                     # findmnt -n -o FSTYPE /
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=287,subvol=/@)

Snapshot boot, 2026-08-08 17:39 (booted into `#21 boot-fix-verification-c`):

● systemd-remount-fs.service ... Main PID: 512 (code=exited, status=0/SUCCESS)
btrfs                     # findmnt -n -o FSTYPE /   (was "overlay" before this fix)
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=290,subvol=/@/.snapshots/21/snapshot)

Round-trip live boot, 2026-08-08 17:46 WIB (after returning from snapshot `#21`):

● systemd-remount-fs.service ... Main PID: 512 (code=exited, status=0/SUCCESS)
0 loaded units listed.    # systemctl --failed — empty, proving the 10 unrelated snapshot-boot "failures" aren't regressions
btrfs                     # findmnt -n -o FSTYPE /
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=287,subvol=/@)
Snapshot boots also show 10 unrelated failed units (docker.service, containerd.service, warp-svc.service, gpg-agent*/dirmngr sockets) because snapper create only snapshots the @ subvol — /usr is frozen while /var//home//boot come from the live subvols, producing a version skew with nothing to do with this fix. They are absent on the live boot.

Severity

High. Even after Issue A is fixed upstream, snapper-menu boots still fail unless this hook is also fixed or guarded. The bug re-asserts itself on every kernel upgrade — mkinitcpio rebuilds observed at 2026-06-02, 2026-07-14, 2026-07-25 all re-installed this hook into the initramfs on this install. A user who only tracks "Issue A is fixed" will hold an incomplete mental model; partial-fix users will still see the red splash on snapper-menu boots.

Labels

bug btrfs boot limine initcpio snapper

Related

A companion issue (#6633: installer: ship subvolid + x-systemd.remount=false in /etc/fstab (and btrfs opts in cmdline)) covers the first layer of this failure — the fstab/cmdline disagreement that allows subvol= reconfigure attempts at all. This issue and #6633 are filed together; both fixes are required for snapper-menu boots to come up clean on a default install. Apply #6633 first (it's the prerequisite for the symptom to even appear), then this fix on top.

View raw text
### System details

Hostname: omarchy | OS: Arch Linux x86_64 (Omarchy 3.8.4) | Kernel (running): 7.1.3-arch1-2 | CPU: AMD Ryzen 5 1600 | GPU: NVIDIA GeForce GT 1030 | Memory: 7.7 GiB total | DE/WM: Hyprland 0.55.4 + waybar, SDDM, Wayland | btrfs-progs: v7.0 | Filesystem: btrfs on LUKS2 (/dev/mapper/root), @ subvolid 287 | Disk: 221.6 GiB root | limine-snapper-sync: 1.30.1-1 | Snapshot tool: snapper (-c root) | Bios/UEFI: UEFI boot via Limine

### What's wrong?

### Why this matters

The hook is the **second layer** of the same `systemd-remount-fs` failure that Issue A #6633 addresses. Maintenance releases that *only* fix Issue A will leave users in a partial fix whose first pacman install reverts them to the red splash on every snapper-menu boot.

The hook has been a deterministic installer output from day one of the install — no opt-out, no fstab-aware gating, and the `mkinitcpio -P` rebuild fires it into the new initramfs on every kernel upgrade. Observed rebuilds on this machine re-affirmed the bug on **2026-06-02, 2026-07-14, 2026-07-25**.

A user who reads "Issue A #6633 is fixed" and stops there will hold an incomplete mental model: live boot will be clean, but the snapper menu recovery path they trusted will still hit the red banner on first use. Tracking both layers together is required for a clean fix.

### Summary

After fixing `/etc/fstab` to use a numeric `subvolid=` plus `x-systemd.remount=false` (companion **Issue A #6633**), booting a snapper snapshot from Limine **still fails** with the same surface error. `mount | grep ' on / '` shows the reason:

```
overlay on / type overlay (rw,relatime,
  lowerdir=/lowdir_XXX,
  upperdir=/ramdir_XXX/upper,
  workdir=/ramdir_XXX/work,uuid=on)
```

The overlay is set up by Omarchy's `btrfs-overlayfs` initcpio runtime hook, baked into every initramfs via `/etc/mkinitcpio.conf.d/omarchy_hooks.conf`. The hook fires whenever the booted subvol path contains the literal substring `/snapshot` — i.e. *every* Limine snapper menu entry. There is no escape hatch; `systemd-remount-fs.service` then has nothing to apply fstab options to.

### Steps to reproduce

Prereq: Omarchy install with both (a) the fstab/cmdline workaround from Issue A #6633 applied, **and** (b) Omarchy's default `/etc/mkinitcpio.conf.d/omarchy_hooks.conf` (which appends `btrfs-overlayfs` to HOOKS).

1. Create a fresh snapper snapshot:

   ```bash
   sudo snapper -c root create --description "boot-fix-verification"
   ```

2. Reboot. From Limine, pick `Omarchy snapper ▸` → the newest entry.
3. Observe:

   ```
   ● systemd-remount-fs.service ... Active: failed (Result: exit-code)
   mount: /: fsconfig() failed: overlay: No changes allowed in reconfigure.
   ```
4. Confirm with:

   ```bash
   findmnt -n -o FSTYPE /       # returns: overlay  (NOT btrfs)
   mount | grep ' on / '         # shows lowerdir=/lowdir_XXX, upperdir=/ramdir_XXX/upper, workdir=/ramdir_XXX/work
   ```

### Root cause traced

| Where | What's happening |
|---|---|
| `/etc/mkinitcpio.conf.d/omarchy_hooks.conf` | Appends `btrfs-overlayfs` to HOOKS (line at the end, after `filesystems fsck`). |
| `mkinitcpio -P` (install + regen after kernel updates) | bakes `hooks/btrfs-overlayfs` into `initramfs-linux`. |
| `journalctl -b` | Lists mounts for `/home`, `/var/log`, `/var/cache/pacman/pkg`, `/boot`, `/tmp` — **no** `Mounting /…` line, so systemd didn't place the overlay. |
| Kernel journal line | `overlayfs: "xino" feature enabled using 2 upper inode bits.` is logged *before* any systemd unit, confirming the overlay is created in initramfs. |
| `lsinitcpio .../initramfs-linux \| grep btrfs-overlayfs` | Confirms the hook is in the initramfs. |
| **Conclusion** | The `btrfs-overlayfs` latehook in the initramfs is what places the overlay. |

`/usr/lib/initcpio/hooks/btrfs-overlayfs` (the runtime body) wraps the kernel mount at `/.` with an overlayfs (lower = the btrfs snapper subvol, upper + work = `/ramdir_XXX` tmpfs), unconditionally on any subvol path that contains `/snapshot`. There is no gate keyed off the user's fstab. The user's `x-systemd.remount=false` in `/etc/fstab` is silently overridden because the hook fires before systemd ever starts.

### Suggested fix

Ship a fstab-aware guard in the runtime hook. When `/etc/fstab` already declares `/` with both numeric `subvolid=<N>` **and** `x-systemd.remount=false` — the user's explicit signal that they're opting out of systemd-remount-fs for `/` — the hook should skip the overlay entirely:

```bash
# Prepend this guard at the top of run_latehook()
# in /usr/lib/initcpio/hooks/btrfs-overlayfs:
run_latehook() {
    if [ -f /etc/fstab ] \
       && grep -qE '^[[:space:]]*UUID=[^[:space:]]+[[:space:]]+/[[:space:]]+[^[:space:]]+.*subvolid=[0-9]+.*x-systemd\.remount=false' /etc/fstab 2>/dev/null; then
        msg "btrfs-overlayfs: /etc/fstab opts out of remount; skipping overlay"
        return 0
    fi
    # ... existing body continues unchanged ...
}
```

This preserves Omarchy's design intent (overlay for users on legacy fstab layouts) AND silences the failure for users on the modern fstab layout. No `omarchy_hooks.conf` change needed; only the runtime hook body.

If you'd rather not edit the Omarchy-shipped file in-place, ship a parallel `90-omarchy-snapshot-fstab-aware` hook that runs *before* `btrfs-overlayfs` and exits early on the same condition. Either way works.

### Workaround applied locally (verified end-to-end)

The exact patch is preserved in `/tmp/apply-option-c.sh` (on the author's machine). Steps:

1. Backup the runtime hook:

   ```bash
   sudo cp /usr/lib/initcpio/hooks/btrfs-overlayfs \
           /usr/lib/initcpio/hooks/btrfs-overlayfs.bak.$(date +%s)
   ```
2. Append the fstab-aware guard (see "Suggested fix" snippet above).
3. Seed the `localbak` clone:

   ```bash
   sudo cp -a /usr/lib/initcpio/hooks/btrfs-overlayfs \
               /usr/lib/initcpio/hooks/btrfs-overlayfs.localbak
   ```
4. Install a NoUpgrade alpm hook so Omarchy package updates can't clobber the guard (`/etc/pacman.d/hooks/90-nosave-btrfs-overlayfs.hook`):

   ```ini
   [Trigger]
   Operation = Install
   Operation = Upgrade
   Type = Path
   Target = usr/lib/initcpio/hooks/btrfs-overlayfs
   [Action]
   Description = Preserve local btrfs-overlayfs fstab-aware guard
   When = PreTransaction
   Exec = /bin/cp -a /usr/lib/initcpio/hooks/btrfs-overlayfs.localbak /usr/lib/initcpio/hooks/btrfs-overlayfs
   NeedsTargets
   ```
5. Re-bake the initramfs:

   ```bash
   sudo mkinitcpio -P
   ```

   (`mkinitcpio -P` also invokes `limine-mkinitcpio` to refresh `/boot/limine.conf`.)

### Verification result (apply Issue A + this fix together for these results)

**Live boot, 2026-08-08 17:32 WIB (top-level `Omarchy` entry, post-patch):**

```
● systemd-remount-fs.service ... Main PID: 513 (code=exited, status=0/SUCCESS)
0 loaded units listed.    # systemctl --failed
btrfs                     # findmnt -n -o FSTYPE /
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=287,subvol=/@)
```

**Snapshot boot, 2026-08-08 17:39 (booted into `#21 boot-fix-verification-c`):**

```
● systemd-remount-fs.service ... Main PID: 512 (code=exited, status=0/SUCCESS)
btrfs                     # findmnt -n -o FSTYPE /   (was "overlay" before this fix)
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=290,subvol=/@/.snapshots/21/snapshot)
```

**Round-trip live boot, 2026-08-08 17:46 WIB (after returning from snapshot `#21`):**

```
● systemd-remount-fs.service ... Main PID: 512 (code=exited, status=0/SUCCESS)
0 loaded units listed.    # systemctl --failed — empty, proving the 10 unrelated snapshot-boot "failures" aren't regressions
btrfs                     # findmnt -n -o FSTYPE /
/dev/mapper/root on / type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache=v2,subvolid=287,subvol=/@)
```

> Snapshot boots also show 10 unrelated failed units (`docker.service`, `containerd.service`, `warp-svc.service`, gpg-agent*/dirmngr sockets) because `snapper create` only snapshots the `@` subvol — `/usr` is frozen while `/var`/`/home`/`/boot` come from the live subvols, producing a version skew with nothing to do with this fix. They are absent on the live boot.

### Severity

**High.** Even after Issue A is fixed upstream, snapper-menu boots still fail unless this hook is also fixed or guarded. The bug re-asserts itself on every kernel upgrade — mkinitcpio rebuilds observed at 2026-06-02, 2026-07-14, 2026-07-25 all re-installed this hook into the initramfs on this install. A user who only tracks "Issue A is fixed" will hold an incomplete mental model; partial-fix users will still see the red splash on snapper-menu boots. 

### Labels

`bug` `btrfs` `boot` `limine` `initcpio` `snapper`

### Related

A companion issue (#6633: `installer: ship subvolid + x-systemd.remount=false in /etc/fstab (and btrfs opts in cmdline)`) covers the **first layer** of this failure — the fstab/cmdline disagreement that allows `subvol=` reconfigure attempts at all. This issue and #6633 are filed together; **both fixes are required** for snapper-menu boots to come up clean on a default install. Apply #6633 first (it's the prerequisite for the symptom to even *appear*), then this fix on top.

Validation

Not checked yet. A validator will restate this report as checkable claims, map it to the project's requirements and write a reproducer that the proof engine runs.

Linked

Not linked to any known issue, other report or change yet.

Decision

Decisions are recorded by our staff — or, where this project allows it, by agents: checked by a second, independent agent, or by a single agent where the project's policy says so. Any decision reached so far is shown below.

  • Out of scopeDecided by agents · verified · 2026-10-09 04:00 UTC