GEEKNOTE: Another Saturday Bites the Dust

GEEKNOTE:  I REALLY need to dust off my old Geeknotes regularly to remind myself of the mantra: "Check the easy stuff first."

I popped in Saturday morning to kick a couple of weekend diagnostics to the next steps on a customer machine on one of our tech benches and to do some preliminary testing on a planned hardware upgrade for one of our servers. 

As I'm finishing those projects, I noted that one of our mail servers was not responding. The drive on the this particular mail server was beyond ancient, so I figured I'd image the files onto a new drive and call it a day.  Nope. The server continued to become sluggish and within a few seconds of startup and then would crash and do a memory dump. Coming close to guessing the true cause of the problem, I replaced the network card. Still no joy, so I emailed a copy of one of the crash dumps to an uber-geek friend in California and went to get the mail. 

When I got back from the post office, I noticed that my email hadn't gone out. I checked and, sure enough, a SECOND server was now going stupid and rebooting itself every few minutes. Hmmm. Multiple calls to California and hours later, both my geek friend and I were convinced that we either had one or more severely corrupted mailbox files and/or something was totally trashed in the DNS system on both machines.

I got the critical stuff from my backups of the two sick servers moved to a still functioning server, set that server to respond to the IP addresses used by the two sick machines and went home for supper, my brain totally fried from a full day of testing and emergency recovery work. After supper, I tried to do a little more rescue work on the functioning server and noticed that I couldn't touch the IP addresses I'd moved from the sick machines from home, even though I'd been able to do so from the office. Hmmm. At this point Carolyn suggested that things would be clearer in the morning.

Early Sunday morning, before dawn, I headed back in to the office to play a hunch related to why I could touch the IP's from inside the office, but not from home and I planned to spend the whole morning migrating the rest of the non-essential stuff off the two sick servers if my hunch didn't work out. I removed the extra IP addresses from the running server and then power cycled our Roadrunner cable modem. I then powered up one of the two sick servers.  It ran fine for several minutes, so I decided to try my luck and power up the second one.  Sure enough, it came up happy as well. 

I spent the balance of Sunday morning undoing the emergency changes I'd made Saturday.

Actual time to solve the REAL problem?  Five minutes Time I spent between Saturday and Sunday chasing the wrong problems? More like ten hours.

Given that we recently had the cable modem replaced because the old one went stupid from time to time, I'm not real impressed with the cable modems Brighthouse uses. The old modem would just go stupid and NOTHING would work. For the new one to kill routing to two of our IP's and leave all the other IP addresses, including the IP address between them functioning is just plain strange.

In any event, I'm going to print out that saying "Check the easy stuff first" in big letters and frame it on the wall over my monitors. Even Geeks need to be reminded once in a while.

Hopefully you had a more enjoyable weekend.

Feel free to drop me a note or give me a call if you have any questions about your computer.

Rob Marlowe, Senior Geek, Gulfcoast Networking, Inc.
http://www.gulfcoastnetworking.com

(Rob also serves as deputy mayor of the City of New Port Richey. Opinions expressed here are his own and do not necessarily represent the position of the city.)