Leading Geeks through Disasters

Some of you have read this blog for a while might already know that I have some experience with technology disasters. Specifically, two back-to-back disasters involving many gallons of water and a server room, thereby earning me the nickname “Waterfall Girl” a few years back. (Which didn’t really stick, luckily.)

Here are some lessons I’ve learned:

  • Geeks surprise you. You never know what they’ll do in high-stress situations.
  • Communication is key. No change? Tell people that. During stressful situations, people just want INFORMATION (dammit!), and sometimes telling them that there’s been no change and you’re still working on it still actually helps them.
  • Apologies help. Folks know the disaster isn’t your fault, but apologizing anyhow somehow helps them. I’m going to guess that it’s because it addresses how they feel and demonstrates that you realize the crisis has caused them no small inconvenience.
  • You can’t please everyone. Did you make an announcement via the PA system? Well, some people would really rather have email. Did you send email? Well, prepare for responses vilifying you for not walking the floors or making an announcement Did you and your team walk the floors? Well, they’re not doing it fast enough. All you can do is your best.
  • You can’t do everything right. Maybe you didn’t communicate fast enough. Maybe you didn’t figure out the problem in time to prevent a cascade event (or maybe the cascading events were inevitable). Maybe you estimated that things would be back in two hours but it took two days. You’re not infallible, and you will probably make even more mistakes in crisis situations. Forgive yourself, pick up the pieces, apologize, and move on.
  • Acknowledge emotion. If you’ve already worked 70 hours by Thursday, you will be a bit, uh, grumpier than usual. Once when I took a post-disaster phone call, I said something to the effect of, “I realize that I haven’t slept and that you’re very stressed as well because of the disaster. My goal is to get through this conversation without either of us getting too testy or angry.” The caller laughed (as people will when you do or say something unexpected), and we got through a 12-minute conversation without excess grumpiness. Realize that your geeks will feel stress and get upset easily as well.

I’m sure there are more things to add. What have your experiences been?
Photo courtesy of Maciej Szczepaniak