Checking my Linux log files📝🙈

I had some free time so I decided to check on my Home Lab. The devices are running 24/7 for long enough time already. Everything was running smoothly, I did not have any problems. These are the two commands I ran:

1. dmesg

2. journalctl -p 3 -b

I saved the output into a text file from all my devices and just uploaded to Gemini Pro to have a quick look at it. I was sure my log files would not contain anything unexpected, since all my devices are running smoothly - some longer than a year now.

---

The result (Gemini summary):

1. Raspberry Pi 4 (Docker / Container Host)

• Detected Issues:

  • Log Spam: The NetworkManager daemon was flooding the system logs with (2) No such file or directory errors.

  • Root Cause: NetworkManager was attempting to manage ephemeral virtual network interfaces (veth*) created and destroyed by Docker containers, but the interfaces disappeared before the IPv6 configuration could be applied.

• Implemented Fixes:

  • Created a configuration file (/etc/NetworkManager/conf.d/99-unmanaged-devices.conf) with the rule unmanaged-devices=interface-name:veth* to explicitly tell NetworkManager to ignore virtual container interfaces.

2. Raspberry Pi 3

• Detected Issues:

  • Wi-Fi SDIO Bus Errors: The kernel reported failed: -5 and Controller never released inhibit bit(s) errors, indicating the onboard Wi-Fi chip was dropping connections due to aggressive power management.

  • Hardware Conflicts: The standalone watchdog daemon failed to start because systemd had already claimed the hardware watchdog device.

  • Service Failures: The bluetoothd service was failing to initialize on this headless node.

  • ZRAM Swap Crash: The zram-tools package was crashing on boot (/dev/zram0 is mounted) because another system service had already initialized the 905MB ZRAM block device.

• Implemented Fixes:

  • Disabled Wi-Fi power management permanently by adding @reboot /sbin/iwconfig wlan0 power off to the root crontab.

  • Permanently disabled the conflicting and unnecessary services using sudo systemctl disable --now watchdog and sudo systemctl disable --now bluetooth.

  • Purged the redundant zram-tools package (sudo apt remove --purge zram-tools) to stop the boot crashes, leaving the existing, functional ZRAM swap intact.

3. Raspberry Pi 1 (Flightradar24 Feeder)

• Detected Issues:

  • No issues

4. OpenWrt Router (Raspberry Pi 4)

• Detected Issues:

  • WAN Uplink Dropouts: The Lenovo USB Gigabit Ethernet adapter (eth1) was constantly dropping and regaining its physical link (carrier off / carrier on), causing massive AdGuardHome DNS resolution failures.

  • Memory Exhaustion: The nlbwmon (bandwidth monitor) service was crashing the netlink socket buffer (Out of memory / Object busy) due to extreme state churn from the WAN drops and Smart Queue Management (SQM) toggling promiscuous mode.

  • Bridge Loop (MAC Reflection): The internal gigabit port (eth0) was receiving its own packets back. The TP-Link TL-SG105E managed switch was intercepting IEEE 1905.1 discovery packets via IGMP Snooping, causing the Deco M4 nodes to briefly spin up a Wi-Fi backhaul loop while still wired.

• Implemented Fixes:

  • Completely uninstalled the bandwidth monitor (apk del nlbwmon luci-app-nlbwmon) to stop the memory leaks —> not used anyways

  • Disabled Energy Efficient Ethernet (EEE) on the Lenovo WAN adapter using ethtool --set-eee eth1 eee off. Made this permanent by adding it to /etc/rc.local and backing up the script in /etc/sysupgrade.conf.

  • Logged into the TP-Link switch management interface and disabled IGMP Snooping (and Loop Prevention) to allow the Deco mesh units to maintain a transparent, stable wired backhaul.

---

Verdict

1. Even though something is working it doesn’t mean it is working great. Regularly check the logs!

2. AI is excellent for reading and processing thousands of lines of log files. Use it!

To be honest I did not notice anything after these changes, but I know now I have a much more robust and failure tolerant stable system.