Why BMod is / was down
For those that are curious what happened… here’s the play-by-play:
(info: This is not Joe I’m dealing with, but the people in Texas that control the boxes)
- Early this morning (around 8am) I see that the web server is throwing a weird component error that I’ve never seen and is waaaaay to generic to troubleshoot. What do I try? Bounce the web server! Work? Nope… web server won’t stop. Next up, bouncing the box!
- I reboot the server, then fire up a local console and ping away! I watch and watch, waiting for the ping to die (this means the server finally initiated the reboot cycle and is just minutes until it’s back up). (5 minutes)Hmmm… still pinging. (10 minutes) still pinging… crap.
- Now I call up the server guys, “Yo, I need you to hit the power button and recycle the server” Them “k”
- 15 minutes later I finally see the ping die. Yay! Now to wait for it to come back up. 5min…10min…..30min… I call back.
- Me “Umm, it’s not coming up.” Them “Oh, yeah. the hard drive failed and is running checkdisk” me “…” Them “hello?” me “…” Them “hello?” me “…FUCK! Fix it and call me back!”
- They call back “yeah, it’s dead. we’ll have to reinstall the OS, and maybe if you’re lucky you can get the data back. Want us to install a RAID card so this can’t happen again? oh… btw it’s another $40/mo.” me… “FUCK! fine. Ass. Fix it and call me back. How long?” Them “3 hours” Me “fuck… whatever. do it.”
- 3 hours later I call them “done yet?” Them…”not yet…20 min. We’ll call ya” Me “fine.”
- 45 minutes later, no call. so i call them, “HEY! WTF AZZHOLEZ?” Them “oh, it’s been up” Me “thanks, bi”
- I try to login, but of course all the login passwords are different. I can’t reach anyone. Another hour down the drain…goddammit. Oh, and the default IIS7 screen is showing… beautiful… asshats.
-I finally get the login credentials and copy all the data over. Woo hoo! all recovered =)
- About an hour in I try to change some permissions on all the mod images so that you peeps can see them, and I see this: “System I/O Error. I Hate You.” (maybe i imagined that last part).
- I tweet: “FUCK FUCK FUCK FUCK FUCK FUCK….+140″
- I try to reboot the server…. oh no.
- Server hangs indefinitely again. Awesome.
- I call them, “d00d! you lame0rz suck! My other HDD is dying now?! AND the server won’t come back up…AGAIN! Fuck you.” Them: “Maybe it’s cuz the old bad HDD is still plugged in.” Me: “I got all the data off, disconnect it” Them: “ok, will do. bye” Me: “Woah! how long?” Them: “10 min” Me: “…fuck you”
- 30 minutes later I call them “Hello???????????? WTF?” Them: “Oh, it’s ready now. Just came up.” Me: “Wonderful…” *click*
- 5 Minutes later I call them back… “um…cocksmokes, YOU DISCONNECTED THE WRONG DRIVE!!!!!! GODDAMMIT!” Them “oops…we’ll fix it.”
(at this point I’m laughing hysterically and eating ice cream…it helps stop the voices and pain in my head)
- It’s now 6pm… the server is DOWN….Down…down… and I’m waiting for them to put the HDD back in. Cocksuckers.
I will update as I find out more and this shit is resolved… Xo/
.: Adam