According to DNS Stuff, all the root servers have the correct name servers stored for whofic (They should now be ns1.whofic.com and ns2.whofic.com). Observe:
So why is it sometimes the old name servers are coming up when people try to load the site? It's not even doing it constantly - it's been flip-flopping back and forth...
*tears hair out*