pfSense Boot Troubleshooting

fw.merox.dev runs on a 6-port Intel I211 box with a Kston 120GB SATA SSD. It doesn’t get rebooted often. When it does, it’s supposed to come back up on its own. This time it didn’t — and it turned out to be three separate failures stacked on top of each other.

Hardware: Intel Kaby Lake (Skylake PCH), Kston 120GB SBFC71.2 SATA SSD (~600 MB/s), 6x Intel I211 Copper NICs (igb0–igb5), Realtek RTL8812AU USB adapter (rtun0). Networks: 10.57.57.0/24, 10.57.97.0/24.

Problem 1: CAM Disk Detection Delay

Symptom

Boot stalled, repeating the same line 6–7 times:

1
Root mount waiting for: CAM
2
Root mount waiting for: CAM
3
Root mount waiting for: CAM

After about 30 seconds it resolved itself and the kernel continued.

Root Cause

The AHCI controller on this board is slow to initialize. FreeBSD’s kernel enumerates storage via CAM (Common Access Method) and expects devices to be ready within a tight window. The Kston SSD on SATA takes longer than that to become visible.

This isn’t a hardware fault — the SSD is fine. The kernel just polls and retries until the controller responds. The delay is deterministic and repeatable.

Fix

No fix needed. The system boots successfully once CAM finds the disk. If this is unacceptable, you can increase the timeout in /boot/loader.conf:

1
kern.cam.boot_delay="10000"

Value is in milliseconds. 10000 = 10 seconds. Default is around 2000ms. Set it to something longer than the observed delay and the stall disappears.

Problem 2: Serial Console Locking Out the Keyboard

Symptom

Boot continued after the CAM delay, loaded the kernel, and then stopped at:

1
pflog0: promiscuous mode enabled

The pfSense menu never appeared. Keyboard input did nothing. The system was running — network interfaces were up based on the LEDs — but the screen was frozen and unresponsive.

Root Cause

The console was configured as Dual Console: Serial Primary. In this mode, pfSense sends its menu to the serial port first and video second. The keyboard works on the serial console, not on the local keyboard attached to the video output.

This is set in /boot/loader.conf:

1
console="comconsole,vidconsole"
2
boot_multicons="YES"
3
boot_serial="YES"
4
-S115200
5
-D

The -D flag enables dual console mode. With serial as primary, anything waiting for input — including the pfSense menu — expects it from the serial port. Pressing keys on the physical keyboard has no effect.

Fix: Bootloader Menu

At power-on, pfSense shows a boot countdown. Press Space to pause it and enter the bootloader menu.

In the bootloader menu, press 6 to cycle the console setting:

1
Cons: Dual (Serial primary)

Press 6 again until it reads:

1
Cons: Video

Then press 1 to boot normally. The pfSense menu will now appear on the monitor and respond to the local keyboard.

Fix: Make It Permanent

From the pfSense menu, select 2 — Boot Single User. This drops you into a FreeBSD single-user shell.

Mount the filesystem read-write and apply the fix:

mount -u /
mount -a
sysrc -f /boot/loader.conf console="vidconsole"
sysrc -f /boot/loader.conf boot_multicons="NO"
sysrc -f /boot/loader.conf boot_serial="NO"
reboot

sysrc edits the target rc/loader file in place without corrupting the rest of it. After reboot, the video console is primary and the serial lines are gone.

Problem 3: Network Interface Mismatch

Symptom

After the clean reboot, pfSense detected 6 physical interfaces (igb0–igb5) but the stored config.xml referenced a different interface assignment. pfSense dropped into the interface assignment wizard automatically.

Root Cause

The USB Realtek RTL8812AU (rtun0) was connected. USB network devices are enumerated late in the boot sequence, and their presence can shift the order in which the kernel assigns interface names. A config.xml saved with one USB enumeration state doesn’t necessarily match the next boot if the USB device connects at a different point.

In this case, pfSense saw all 6 Intel I211 ports plus the USB adapter. The stored config was from a different enumeration — the wizard was the right response.

Fix

Re-assign interfaces in the wizard:

• WAN → igb0 (uplink port, identified by MAC)
• LAN → igb1
• OPT1 → igb2
• Remaining igb ports and VLANs as configured

Confirm and reboot. pfSense came up fully: LAN, WAN, OPT1, VLANs, OpenVPN, DNS Resolver, and firewall rules all loaded from the existing config.

Preventing This

Serial console: after any fresh pfSense install on hardware that has a monitor, immediately set console="vidconsole" in /boot/loader.conf and remove the serial flags. The UI option is under System > Advanced > Admin Access > Console menu — set it to VGA. Both the UI setting and the loader.conf lines need to match.

USB NIC enumeration: if you have a USB NIC plugged in, expect interface names to be unstable across reboots. Either remove it or assign interfaces by MAC in the wizard and verify after every reboot that involved hardware changes.

CAM delay: if the stall bothers you or you’re worried about a watchdog rebooting the box mid-boot, set kern.cam.boot_delay in loader.conf to something above your observed delay. It’s a one-liner and it eliminates the uncertainty.

Total time from accidental reboot to fw.merox.dev fully online: about 25 minutes, most of which was figuring out the serial console issue from a machine with no serial adapter attached.

Epilogue: The 11-Day Blast Radius

After fw.merox.dev came back up, everything looked fine. Tailscale showed online, pfSense passed traffic, the cluster was healthy. What wasn’t obvious: the Tailscale interface had not been re-added to pfSense’s interface list after the wizard re-assignment.

The static route for 100.64.0.0/10 (Tailscale’s CGNAT range) still existed, but its gateway was now WAN_DHCP instead of the Tailscale interface. pfSense was routing all Tailscale peer traffic out the WAN — silently dropped.

The failure surfaced 11 days later when a routine check showed Longhorn had no backup since April 23. The error was a timeout deep in the Longhorn engine binary trying to list objects in s3://longhorn@us-east-1/:

1
timeout executing: longhorn [longhorn system-backup list s3://longhorn@us-east-1/]

The backup target was the Garage S3 instance running on the cloud node at 100.72.22.38:3900, reachable only via Tailscale. Every night at 02:00 the recurring job fired, connected to the gateway, and silently timed out. No alert, no crash — just no backups.

Fix: Edit the gateway entry in pfSense, change Interface from WAN to Tailscale. One field, one click. The static route for 100.64.0.0/10 was correct all along.