Watchdog and Power-Loss Protection: Reliability Design for 24/7 Unattended Devices
Devices live in distribution cabinets, on poles or in unattended equipment rooms. When one hangs, the only recovery is sending someone to power-cycle it on site — and the cost of that trip can exceed the profit on the device itself. Watchdogs and power-loss protection exist to let the device recover on its own.
Why watchdogs fail to do their job
The mechanism is simple: a timer that software must “feed” periodically, resetting the system if it is not fed in time. It sounds foolproof, but the most common mistake in practice is feeding the dog from the same thread that has already hung.
The typical case is a main loop that both runs business logic and feeds the watchdog. If a business function spins or blocks in a system call, the loop never reaches the feed and the watchdog resets — that case actually works. The genuinely dangerous case is the opposite: business logic hangs, but feeding is handled by a separate timer thread that is still healthy. The dog gets fed forever and the system never restarts. The device looks alive while doing nothing.
The correct approach is to tie feeding to health checking: feed only when all critical services pass their self-checks. If any critical service stops responding, stop feeding and let the watchdog reset. The watchdog then verifies that the system is genuinely working, not merely that a process still exists.
Window watchdogs are stricter
An ordinary watchdog only checks that feeding happens on time. A window watchdog also checks that it does not happen too early. That catches a subtle failure mode: after going off the rails, code enters an abnormal loop that feeds the dog at very high frequency — invisible to an ordinary watchdog, but a window watchdog resets on the too-early feed.
Internal or external watchdog
The SoC’s built-in watchdog is convenient and free, but it depends on the SoC’s internal clock and logic. If the SoC itself locks up, or its clock or supply misbehaves, the internal watchdog may fail alongside it. An external watchdog chip has independent power and independent timing, so it is more reliable — at the cost of one chip and a few traces. For devices whose field maintenance is expensive, the external part usually pays for itself.
Platforms aimed at unattended scenarios, such as the 03R EDGE-RT28 (RK3528) industrial router board, the 05A IPA-RK3588 industrial control board and the 02A EDGE-RK3576 V1.1, deserve more attention for reliability design than for performance figures. At selection time you can ask the supplier directly whether an external watchdog is supported and whether the feeding policy is customer-definable.
Power loss: the real risk is a corrupted filesystem
A sudden power cut does not damage hardware by itself; the danger is a write in progress at that moment. Journaling filesystems such as ext4 preserve metadata consistency but cannot prevent a half-written file from being corrupt. Worse, if the storage device’s internal mapping table loses power mid-write, an entire region can become unreadable.
Industry has three mature layers of defence. The first is a read-only root filesystem, concentrating writable data into a separate partition to shrink the write surface. The second is overlayfs with a read-only root and a RAM-backed writable layer, so the system disk is barely written at all and a reboot restores factory state — ideal for field devices. The third is brownout warning plus clean shutdown: hold-up energy from a large capacitor or supercapacitor; on detecting input voltage collapse, stop all writes immediately, flush critical data and shut down deliberately.
Sizing the hold-up energy requires arithmetic: energy equals one half times capacitance times the difference of the squares of the start and end voltages, and it must cover the flush and shutdown time. The answer is usually a capacitor of non-trivial size, which is why this must be decided during hardware design rather than retrofitted.
Do not design power-loss protection separately from OTA
If the device updates remotely, power-loss risk and update risk compound: losing power halfway through an update can brick a non-A/B design outright. This is why field devices are better served by A/B dual partitions — the update writes to the standby slot, a power cut simply means retrying, and the primary slot stays usable. The two concerns should be designed together.
Conclusion
The order is: first bind feeding to health checking (zero cost, largest benefit), then assess whether an external watchdog is needed, then choose a read-only root or brownout-warning approach to match site conditions, and finally confirm the update scheme cannot brick the device on power loss. Settling these four before the hardware is frozen costs the least.
