Awesome Conferences

How to provide infinite disk storage

A user recently asked for a lot of disk space. Not just a lot of disk space, but growing at an astounding rate per month. (Not big for some places, but bigger than my current employer was used to providing). It was an archive that would start large and grow in leaps and bounds. It had to be actual disk (not tape or other off-line technology) because the data would be accessed constantly.

He joked that what he really wanted was infinite disk space. I replied, "I can give you infinite storage." and I wasn't joking.

He told me to prove it so I explained:

Your data will start large and grow quickly. If I can install storage capacity faster than you need it, it is indistinguishable from if we actually had infinite storage. I can afford it because the equipment will come from your budget, so it isn't my problem (directly) and if this project is as valuable to the company as you say it is, your management will gladly fund it. (and if they don't fund it, then we know the value of the project is not what you think it is).

To stay ahead of your capacity demands I need to know your predicted growth rate. You'll have to provide estimates based on engineering estimates but I'll have monitoring that will give you "ground truth" as confirmation and will help you make better predictions.

The key is communication and cooperation. The biggest barriers to a project are often funding and capacity planning. If you can solve those two issues everything else falls into place and we can do anything.

That was true when sysadmins were providing megabytes of storage in the 90s and it is true when providing petabytes of storage today.

Posted by Tom Limoncelli in Technical Management

No TrackBacks

TrackBack URL:

Leave a comment