Four weeks ago my Ubiquiti EdgeMax EdgeRouter Lite decided it was a good idea to not start PHP anymore. Some people would say ‘Good choice and good riddance’ — on a router appliance a web interface comes in handy once in a while though. A Ubnt community member was able to help me very quickly, though something was very fishy. I just recently upgraded the firmware on the router, so how could the file system got corrupted? Did something strange happen during the upgrade? — Not that I could remember.
One evening two weeks ago then I was debugging a VPN problem I encountered a massive packet loss. First I thought it was the remote server, or my Internet provider. Soon though I had to realize that the first hop, the little EdgeRouter, was losing 70% of all packages sent to it. Also 60% packet loss on the second hop. No web interface available, not SSH connection possible. I decided to reboot the device… it did not come back! My first thought was a memory issue, but I needed more information what was happening on the device. The lights were no help, everything looked normal. I connected a Cisco Serial Console Cable and found some kernel “Oops” during boot:
This really looked weird. I contacted Ubnt support. I was told to press the famous ‘any key’ during boot, before the kernel was loaded, and then run some tests. First of all a memory test:
Octeon ubnt_e100# mtest Testing 80100000 ... 80ffffff: Iteration: 6040
It ran for a couple of hours, but showed no issue at all. Support then had the idea to use the ‘EdgeMax Rescue kit’ to re-install the router from scratch.
This unfortunately did not bring a solution either. But it helped to identify the underlying problem. It looks like the filesystem corruption a few weeks back was just a herald of the problems that would I have to face soon:
It looks like the flash memory is totally fucked up. Since it was a little over a year old, and therefore still covered by the EU wide 2 year warranty, I contacted my reseller and filed for RMA and sent the router back to them. I really hope that I will get a new device soon. It is a quite powerful little device, so I really miss it.