Cisco: top of my list of "difficult to upgrade" things

Heartbleed has reminded me what equipment and products I deal with that are difficult to upgrade. While most people think of DevOps as "rapidly deploying software that your coworkers wrote", it is really about creating a world where we are able to make changes... because change is required to experiment, and innovation requires experimentation... and that means being able to make changes. This includes not just in-house software releases, but all operational changes we do. This includes software and firmware releases we get from vendors.

My new(-ish) job at StackExchange has me actually touching hardware instead of living in the virtualized, everything-is-done-for-you, world of Google. All our networking gear is Cisco. Most of it is upgraded only when we have to.

I used to upgrade Cisco firmware in the 1990s (well, up until 2003 I guess). I figured, "How much different could it be?" (Wow, do I feel old for saying that).

Anyway...

The process hasn't changed much. It is still TFTP firmware from a server. The version numbers are all different, and much more complex selection process, but I can deal with that.

However I'm shocked that there isn't a Windows app that just does it all for me. Something where I enter the IP address of the router, my username and password, and it says:

"Hi Tom! It looks like you have a Cisco Wizbang 7600 running version 6.4.1.2.3.5(2)! The recommended release for this device is:

  • If you are conservative: 7.4.1.2.3.5(2)
  • If you live on the edge: 7.6.3.1.4(123)
  • If you are insane: 8.0.0.0(0)

Then I click on one of those 3 and it just freaking does the right thing: Gets it from Cisco, uploads it to the device, asks me to confirm and reboots it.

Hasn't anyone thought of this before? It seems so obvious.

I talked with my friends at Cisco and they told me that the "Prime Infrastructure" product does this but it is one feature out of a huge, expensive, product.

Why hasn't there been an open source project to do this? It seems so obvious.

It's two parts: Helping choose the version and upgrading the firmware. The first might be difficult unless Cisco provides an API to their vast sea of IOS versions. I can forego that part for now. The second half seems to be pretty straight forward. Half of the code is already in RANCID.

I'm not a network engineer so maybe this already exists.

Post a comment if you know of one.

Posted by Tom Limoncelli in Rants

No TrackBacks

TrackBack URL: http://everythingsysadmin.com/cgi-bin/mt-tb.cgi/1750

6 Comments | Leave a comment

Most good Cisco admins (or network admins in general) are pretty conservative about doing things. Which is a good thing (IMHO).

Yes making software updates is painful. So is using any Cisco management applications (At least for me. I fail to see the promised simplicity and always run into bugs).

When I'm working on Cisco projects I do critical system (Core routers / switches) manually and not so critical systems (user access, TOR switches) via script (using RANCIDs clogin).

I still have to take a look at netconf (http://en.wikipedia.org/wiki/NETCONF).

And BTW: If you are brave you can also scp your software image to a router. On smaller devices: turn your phone off. Start the copy, go out for a good long lunch. It will take time and render the device almost useless.

I've never seen usability or convenience on any of the Cisco selling points.

Basically, the products aren't designed for people who haven't already been doing the same thing forever. There are new bells and whistles in the next generation of products (specifically the NX-OS line) but overall, ease of use and convenience still aren't on the list.

This is probably a moot point due to many reasons...

I recently was exposed to the Juniper upgrade process and it is pretty amazing. Just point it to a URL and it will download it to the device and install it for you. It will even reboot in the same command if you do. When it upgrades it also only upgrades one of the partitions so you can fall back to the previous OS image if you need to.

copy http (https / ftp) also works on (newer) IOS versions. So you can use more then tftp for copying IOS images.

The JunOS CLI has some other nice features like commit and rollback.

And to be honest: All vendors have strange bugs (which they sometime call features) but at least you can get all documentation from Cisco and Juniper. I try to avoid vendors where I have to have a service contract just to look at the documentation.

You're right, this sucks on most network device platforms. Some are effectively embedded systems with little API exposed. SNMP is often the best we have, and if the Vendor has custom MIBs a series of SNMP set/gets can be used to automate software retrieval, installation, reload.

Time for the plug... you might want to take a look at the Netdisco app which is designed to make the life of "accidental" network sysadmins just a little easier:

https://metacpan.org/pod/App::Netdisco

Just so you know, Gerty is a so-called replacement for RANCID which is supposed to be network vendor agnostic: https://github.com/ssinyagin/gerty

Leave a comment