In my 11 years in IT, I had the dubious honor of going through a few systems disasters. (One actually earned me the nickname, “Waterfall Girl,” when I presented on it at a conference.). Here are the lessons I’ve learned:
- People want someone to blame. “Whose fault is this?” Was an oft-repeated refrain during the disasters. Unfortunately, in IT, “No one’s,” is usually the answer.
- “I told you so” is incredibly dissatisfying to say after numerous all-nighters.
- There’s a big difference between disaster recovery and business continuity. The first is relatively easy; the second will bite you if it’s not properly scaled.
- Geeks surprise you. They will do anything and everything to get things back up and running. If you see your geeks in a disaster, prepare to have your preconceptions challenged.
In a true disaster, my first go-to person would be my brother-in-law, the firefighter. However, in any systems disaster–involving waterfalls or not–my first choice would be to find a good geek!