Recently in Writing Category

[This article first appeared in the SAGE-AU newsletter.]

Have you heard about the New York City broadway show Spider-Man Turn Off the Dark? It should have been a big success. The music was written by Bono and the Edge from U2. It was directed by Julie Taymor, who had previously created many successful shows including The Lion King. Sadly, before it opened, the show was already making headlines due to six actors getting seriously injured and other issues.

The show opened late, but it did finally open. It ran from June 2011 to January 2014.

When the show closed Taymor said that one of the biggest problems with bringing the show to production was that they were using new technology that was difficult to work with. Many of the scenes involved choreography that was highly dependent on pre-programmed robotics. Any changes that involved the robotics required a 5 hour wait.

A 5 hour wait?

Normally directors and choreographers can attempt dozens of changes in a day of rehearsal to get a scene or dance number "just right." Imagine finding yourself in a situation where you can only attempt a new change once or twice a day.

The ability to confidently make changes at will is key to being able to innovate. Innovation means trying new things and keeping what works, throwing away what doesn't. If you can't make changes, then you can't innovate.

Consider the opposite of innovation. We've all been at a company that resists change or has calcified to the point where they are unable to make change. Nothing can get better if you can't change policies, procedures, or technology. Since entropy means things slowly get worse over time, an organization that is unable to change, by definition, is an organization that is spiraling towards doom.

I'm reminded of this recently due to the Heartbleed security issue. Like most system administrators, the Heartbleed bug meant I had to spend a lot of time upgrading the software and firmware of nearly every system in their organization. For many of us it meant discovering systems that hadn't been upgraded in so long that the implications were unknown. As sysadmins we wanted to protect ourselves against this security flaw, but we also had to face our own fear of change.

We need to create a world where we are able to change, or "change-able".

There are many factors that enable us to be "change-able". One factor is frequency: we can make change, one after the next, in rapid succession.

Software upgrades: Every 1-3 years there is a new Microsoft Windows operating system and upgrading requires much care and planning. Systems are wiped and reloaded because we are often re-inventing the world from scratch with each upgrade. On the other hand, software that is upgraded frequently requires less testing each time because the change is less of a "quantum leap". In addition, we get better at the process because we do it often. We automate the testing, the upgrade process itself, we design systems that are field-upgradable or hot-upgradable because we have to... otherwise these frequent upgrades would be impossible.

Procedures: Someone recently told me he doesn't document his process for doing something because it only happens once a year and by then the process has changed. Since he has to reinvent the procedure each time the best he can do is keep notes about how the process worked last time. Contrast this to a procedure that is done weekly or daily. You can probably document it well enough that, barring major changes, you can delegate the process to a more junior person.

Software releases: If you collaborate with developers who put out releases infrequently, each release contains thousands of changes. A bug could be in any of those changes. Continuous Delivery systems compile and test the software after every source code change. Any new bugs discovered are likely to be found in the very small change that was recently checked in.

Another factor in being "change-able" is the how difficult it is to make a change.

I've been at companies where making a DNS change required editing 5 different files on two different systems, manually running a series of tests and then saying a prayer. I've been at others where one typed a command to insert to delete the record, and the rest just happened for me.

When it is difficult to make a change, we make them less often. We are tempted to avoid any action that requires that kind of change. This has a domino effect that slows and delays other projects. Or, it means we make the decision to live with a bad situation rather than fix it. You settle for less.

When we make changes less frequently, we get worse at doing them. Therefore they become more risky to do. Because they are more risky, we do them even less. It becomes a downward spiral.

DevOps is, if anything, about making operations more "change-able". Everyone has their own definition of DevOps, but what they all have in common is that DevOps makes operations better able to change: change more frequently and change more easily. The result is confidence in our ability to make changes. In that way, confidence is a precondition to being able to innovate.

Which brings us back to Spider-Man Turn Off the Dark. How much innovation could really happen if each change took 5 hours? Imagine paying a hundred dancers, actors, and technicians to do nothing for 5 hours waiting for the next iteration. You can't send them home. You can't tell them "come in for a few minutes every 5 hours". You would, instead, avoid change and settle for less. You would settle for what you have instead of fixing the problems.

Would DevOps have saved Spiderman? Would a more change-able world make me less fearful of the next Heartbleed


Posted by Tom Limoncelli in DevOpsWriting

[This was originally published on The Usenix Update Blog]

We want YOU to submit a paper this year to the LISA conference  Really.  Yes, you!  Whether you are in academia developing new algorithms that improve system administration, leader of an open source project that sysadmins find valuable, or a practitioner in industry that has written new software to improve productivity, we believe there's a paper inside all of you that wants to get out!  (Usenix LISA is December 4-9, 2011 in Boston).  LISA is also a great venue for student papers: it is a friendly audience and we have a "Best Student Paper" award that pays cash.

Usenix LISA is doing three big things this year to make it easier to submit a paper:

1.  We provide mentoring.

Submitting a paper to a conference can be intimidating, a lot of work and stressful. To make the process easier, the members of the LISA Program Committee (PC) are available to provide mentoring.  You can bounce ideas off of us by email or phone, we'll proofread your drafts, and we'll try to answer any questions about the conference or submission process.  Just write, "assign me a mentor" in email to the conference co-chairs at [email protected].

Mentors can help turn your accepted abstract into a "print ready" final draft.  We'll also work with you over video chat to rehearse and strengthen your presentation for the conference.

2.  You don't have to submit a full paper.

It can be heartbreaking to write a complete paper only to learn it wasn't accepted for this year's conference.   Papers are 8 to 18 pages; that's a lot of writing. In recent years about 20 of the approximately 80 submitted papers were accepted.

While you may submit a complete paper, we also accept an "extended abstract" of 4-8 pages.  You only write the full paper by the publication deadline if your abstract is accepted.

In an extended abstract, you document the meat of your paper. You want to make sure you don't leave out important points such as what you have achieved along with how you achieved it.  Phrases like "the full paper will reveal the new algorithm" don't allow the PC to evaluate your efforts. Working with a mentor can help you through this process to ensure you submit the best abstract possible.

3. You don't have to be a scientist.

"But I haven't invented anything!"   Refereed Papers describe work that advances the art or practice of system administration and are held to high research standards.  However, LISA has an additional category called "Practice and Experience Reports" (PER) that describe a substantial system administration project whose story reveals lessons worth sharing.  In other words, you did something awesome and want to tell the world about it so they can learn from your mistakes (Did I say mistakes?  I meant "learn from your awesomeness".)  Actually failures are often worth documenting as we learn the most!

A PER is judged on the basis of whether it addresses a pressing or rising need in the industry and the usefulness of the lessons learned. If accepted, a final draft of the full report (4-10 pages) is due by the publication deadline, just like refereed papers.

The first paper I presented at a LISA conference would have been a PER, if the category had existed then.  That was 1997!  My paper wasn't rocket science (or even computer science), but we were able to explain some valuable insights into what to do (but mostly what not to do).

We're also looking for proposals for general "Talks", special Q&A talks called "The Guru Is In", and "Poster Session".


Every PC member is currently reaching out to friends, calling universities, and visiting user groups to encourage people to submit papers. We'd love for you to announce the Call For Participation at your local user group meetings (and we'll give you a little gift if you do). Let us know if you're interested in getting more involved by participating on a future PC.

LISA11 is making an extra big effort to seek out new papers and new authors.  We're doing outreach, we're making the submission process easier, and we're providing mentoring. So, if you have never submitted an abstract to LISA, maybe this is your year.  Contact us if you are on the fence.  Maybe we can answer your questions and concerns to put you on the path to successful author.

The submission deadline is June 9, 2011.  That may seem far in the future but it creeps up on us very fast.  Start brainstorming your paper now and we look forward to receiving your submission soon!

Tom Limoncelli
LISA11 Co-Chair

Key dates:

  • Submission deadline: June 9, 2011, 11:59 p.m. PDT: Extended abstracts, papers, experience reports, and proposals for invited talks, workshops, and tutorials.
  • Notification to all submitters: July 11, 2011
  • Publication deadline: September 15, 2011: Final papers and reports due
  • Poster proposals due: November 11, 2011

  • LISA16