In my 11 years in IT, I had the dubious honor of going through a few systems disasters. (One actually earned me the nickname, “Waterfall Girl,” when I presented on it at a conference.). Here are the lessons I’ve learned:

1. People want someone to blame. “Whose fault is this?” Was an oft-repeated refrain during the disasters. Unfortunately, in IT, “No one’s,” is usually the answer.
2. “I told you so” is incredibly dissatisfying to say after numerous all-nighters.
3. There’s a big difference between disaster recovery and business continuity. The first is relatively easy; the second will bite you if it’s not properly scaled.
4. Geeks surprise you. They will do anything and everything to get things back up and running. If you see your geeks in a disaster, prepare to have your preconceptions challenged.

In a true disaster, my first go-to person would be my brother-in-law, the firefighter. However, in any systems disaster–involving waterfalls or not–my first choice would be to find a good geek!