Saturday, February 13, 2016

Troubleshooting Verizon Fios Router WiFi Instability

My folks have a router model MI424WR-GEN3I which they rent from Verizon Fios.

Beginning in Fall 2015, after a year of good functioning, they began experiencing instability with the wireless portion of the router. Specifically, after a period of a few hours, days or weeks:
  • New wireless clients would fail to connect to the wireless network, even when the WPA2 password was correct.
  • As their leases expired, existing clients could not renew their leases and thus could no longer access the network either.


What is most bizarre about the situation is that the wired clients continued functioning normally. Ethernet works. Wireless stops working. This is not an issue related to the WAN connection or Verizon's servers. Quite a few issues have now been ruled out:
  • Radio interference, the issue verizon's help desk keep returning to cannot be the explanation:
    • They live in the exurbs and have little radio interference. The router is in the basement, shielded by concrete and soil.
    • Standing directly next to the router, no matter what device is used, does not affect one's ability to connect. When the router is exhibiting the issue, none can.
    • There is no notable change in the electromagnetic environment of the house, like a new microwave or other appliance, nor any pattern of outages happening when any electrical device is being used.
    • During outages, some devices continue to work for some time (eventually they just can't re-authenticate).
    • Both the router and other devices show almost no other networks in the area, all of which are weak. The signal strength shown by clients for the network in question is good, even when the problem is ocurring. It's not that the network can't be seen or is weak, it's that you can't (re-)connect/authenticate.
    • Changing the wireless channel being used has no effect.
  • Issues with the wireless clients:
    • This happens to portable devices (phones, iPads), laptops and different operating systems (android, iOS, windows). Eventually all are affected.
    • It is remotely possible that some incompatibility with some client device is causing the router software to bonk after some time, but the devices being used are all pretty standard, and it doesn't seem to matter which devices are on a network when outages occur.
  • Some kind of virus or something on a client
    • Verizon technicians seem to think this is a possibility, but even if some client did have one (and they almost certainly don't) I can see no way in which this should affect the wireless portion of the router (only) and go away after resetting it.
  • An issue with the specific Verizon router. This was my first thought, that probably something had caused the wireless radio to begin failing or some other hardware problem
    • Verizon has now replaced the router not once but twice, and both replacement routers began experiencing the exact same problem (and only that problem), after a matter of days. The only special configurations they had in common were:
      • Opening a single (SSH) port for forwarding, and of course some UPnP ports.
      • The existence of a wired client which regularly transfers data.
      • A custom WPA key and network name.
    • I should say that both the replacement routers were "refurbished" units, which may or may not have had any real refurbishment performed after customers returned them. I can only speculate about this, but given that the routers all pass a self-test even while exhibiting the problem, and that when you first turn them on they appear to work perfectly fine (the problem takes a minimum of a few hours to begin happening), it is quite possible that Verizon thinks nothing is wrong with them and is simply returning other peoples' broken routers to the next guy expecting that the real problem is incompetent customers returning perfectly good routers (I get the sense that's what the agents think anyway). 
    • All of these routers were, at the time of the issue, running firmware version 40.21.18.
  • Like I said, the internet connection continues working fine as can be verified via wired connection, so that seems to rule out issues with the ONT or other Verizon network infrastructure.
The issue can be "remedied" (temporarily, until the next outage randomly happens) by either turning off wireless radio from the router configuration software and turning it back on, or by resetting the whole router. ADDENDUM: When the issue happens, under System Monitoring -> Advanced Status -> "Full Status / System wide Monitoring of Connections", Wireless Access Point "Status" shows "Disconnected", even though under Wireless Settings, the wireless network is "On". It remains unclear what causes it to become "disconnected" or why it fails to re-connect.

This problem is very puzzling and remains unresolved. I'll most likely have to "resolve" it by buying another wireless router and plugging it into the Verizon router via ethernet, then disabling the Fios internal wireless capability. It's pretty frustrating that you have no choice to replace the Verizon router altogether.

Given the lack of great options, I'm asking you, dear internet, if you are seeing similar issues. I don't trust that Verizon will find them themselves, and Verizon agents appear to be well off the scent.

The only realistic possibilities I see are that:
  1. This is a common flaw that is either tolerated by customers or not correctly diagnosed, possibly since so few people use ethernet.
  2. I've really got bad luck and managed to get three routers which all exhibit the same problem with the wireless radio (possibly because they don't fix it when refurbishing units). A fourth time might be the charm.
  3. There was some automatic update to the firmware last fall and the new firmware (40.21.18) contains this bug. As I noted, it's possible the firmware might have some incompatibility with other implementations of the 802.11a/b/g/n protocol(s). But the most likely device that would be would be iPhones (the only new one on the network since the problems started happening), so again, you'd think lots of customers would see it.
Any comments or suggestions are most welcome.