FortiGate-100D: EXT2-fs(sdb1):previous I/O error to superblock detection

When you are facing the issues that the kernel errors something like firewall is rebooting auto or outage of power supply accidently. So all the LAN interfaces shut down in the firewall.

Problem:

The errors will be like this:

EXT2-fs(sdb1):previous I/O error to superblock detected.

EXT2-fs(sdb1):previous I/O error to superblock detected.

EXT2-fs(sdb1):previous I/O error to superblock detected.

Troubleshooting Commands:

1) Try to get the debug logs

https://community.fortinet.com/t5/FortiGate/Technical-Tip-How-to-download-debug-log-file-at-different/ta-p/193211

2) Get the result details

#get hardware status

#get system status

#get system performance status

# diagnose sys flash list

# diagnose autoupdate status

# diagnose autoupdate versions

# diag debug crashlog read

# diagnose hardware sysinfo memory

# diagnose hardware sysinfo shm

# diagnose hardware sysinfo slab

# diagnose debug crashlog read

# diagnose hardware deviceinfo disk

# diagnose sys session stat

Resolution: The issue is resolved by reformatting the boot device and then reinstalling the firmware via TFTP. Here are the example for cluster environment.

Step1: Break the cluster

#show system ha, if override is disabled if it is enabled then disable it.

Disable the monitor ports on the master (temporarily)

Remove Heartbeat ports from the slave

Remove any WAN connections from the slave

Remove any LAN connections from the slave

Step2: Format the flash boot drive- Upload the firmware into the FortiGate firewall

- Reboot the device and press any key to enter the Boot Menu.

- Select the option to format the boot device

- Select the option to adjust the IP parameters to be on the same subnet as with your TFTP server

- Select the option to upload or transfer the firmware from a TFTP server

- Refer to the following KB to format and loading firmware using TFTP

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Formatting-and-loading-FortiGate-firmware-image/ta-p/197617?cmd=displayKC&docType=kc&externalId=10338

Step3: Disk Scan

- You may need to perform a disk scan if necessary.

#execute disk list

- check the reference numbers in the line - ref: - and run the following command

#execute disk scan *ref number*

- Refer to the following KB to do disk scanning

https://community.fortinet.com/t5/FortiGate/Technical-Tip-File-System-check-recommended-message/ta-p/192784

Step4: Perform an HQIP test

- Refer to the following KB to test HQIP

https://community.fortinet.com/t5/FortiGate/RMA-Note-Hardware-troubleshooting-with-built-in-FortiOS-hardware/ta-p/192803?externalID=FD39581

Step5: Bring up the cluster

#get sys ha status

- Connect the Heartbeat ports

#diagnose sys ha checksum cluster

- Check if it has synchronized

- Connect back any LAN and WAN ports that you had disconnected.

- Enable monitoring ports that you had disabled.

- Enable override if it was enabled before.