Awesome Conferences

Wouldn't it be nice if you could write a program that could reach into an Apache config file (or an AptConf file, or an /etc/aliases file, Postfix master.cf, sshd/ssh config, sudoers, Xen conf, yum or other) make a change, and not ruin the comments and other formatting that exists?

That's what Augeas permits you to do. If a config file's format as been defined in the Augeas "lens" language, you can then use Augeas to parse the file, pull out the data you want, plus you can add, change or delete elements too. When Augeas saves the file it retains all comments and formatting. Any changes you made retain the formatting you'd expect.

Augeas can be driven from a command-line tool (augtool) or via the library. You can use the library from Ruby, Puppet, and other systems. There is a project to rewrite Puppet modules so that they use Augeas (augeasproviders.com/providers)

Version 1.5 of Augeas was released this week. The number of formats it understands (lenses) has increased (see the complete list here). You can also define your own lens, but that requires an understanding of parsing concepts (get our your old CS textbook, you'll need a refresher). That said, I've found friendly support via their mailing list when I've gotten stuck writing my own lens.

The project homepage is http://augeas.net/ and the new release was just announced on the mailing list (link).

Posted by Tom Limoncelli in Technical Tips

I maintain an open source project called Blackbox which makes it easy to store GPG-encrypted secrets in Git, Mercurial, Subversion, and others.

I've written up my ideas for where the project should go in the future, including renaming the commands, change where the keys are stored, add a "repo-less" mode, and possibly rewrite it in a different language:

https://github.com/StackExchange/blackbox/blob/master/Version2-Ideas.md

Feedback welcome!

Tom

Posted by Tom Limoncelli in Blackbox

The new issue of acmqueue magazine contains a preview of a chapter from our next book, the 3rd edition of TPOSANA. This issue contains a preview of a chapter from our next book, the 3rd edition of TPOSANA. The chapter is called "The Small Batches Principle". We are very excited to be able to bring you this preview and hope you find the chapter fun and educational. The book won't be out until Oct 7, 2016, so don't miss this opportunity to read it early!

ACM members can access it online for free, or a small fee gets you access to it online or via an app. Get it now!

Posted by Tom Limoncelli in ACM Queue Column

It makes me sad to see people type more than they have to. With these aliases, you reduce the 4 most common commands to 2 letter abbreviations:

git config --global alias.co checkout
git config --global alias.br branch
git config --global alias.ci commit
git config --global alias.st status

NOTE: This updates your ~/.gitconfig file and adds aliasses "co", "br", "ci", and "st".

If you collaborate with others, git pull makes a messy log. Instead, always type git pull --rebase --ff-only. This will make the merge history a lot more linear when possible, otherwise it falls back to the normal pull behavior. Of course, if you set this alias git p is all you need to remember:

git config --global alias.p "pull --rebase --ff-only"

These last aliases pretty-print the output of git log five different ways. They make the logs colorful, beautiful, and much more useful. To be honest, I haven't spent the time to review the git manual to figure out how they work. I don't care. I copied them from someone else, who copied them from someone else. They work great. Thanks to the anonymous person that gave them to me. These aliases will help you love git logs:

git config --global alias.lg "log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --date=relative"
git config --global alias.lgg "log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --date=relative --name-only"
git config --global alias.ll 'log --pretty=format:"%C(yellow)%h%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate --numstat'
git config --global alias.ld 'log --pretty=format:"%C(yellow)%h\\ %C(green)%ad%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate --date=short --graph'
git config --global alias.ls 'log --pretty=format:"%C(green)%h\\ %C(yellow)[%ad]%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate --date=relative'

To install all these aliases:

Option A: Run the above commands. A copy has been placed at this link so you can download the script and run it.

Option B: Copy them out of my .gitconfig which you can access via this link

Option C: If you trust me, and you shouldn't, you can run:

curl https://raw.githubusercontent.com/TomOnTime/tomutils/master/gitstuff/install-fav-git-aliases.sh.txt | sh -x

Posted by Tom Limoncelli in Technical Tips

There is a long and fraught history in Bitcoin of claims and counterclaims about who Satoshi is. I might as well confess that he is me.

I come forward at this time because Craig Wright claims to be Satoshi and I can't stand such intentional scammery.

If you read any of my pre-Bitcoin books, you'll see there are many pages where the first letter of each line reads "I am Satoshi Nakamoto" and "Someday I will invent Bitcoin". If you can't find the page that contains this, buy more copies of the books. You just haven't found the right one. Please use this link, since it includes my Amazon Associates code. Buy additional copies since each one might be slightly different.

Further proof: The first letter of the chapter titles of Time Management for System Administrators, it spells "tfrttttpseeda". I mean, what hacker doesn't know what that means?

Oh, I guess you aren't elite. My bad.

Now let me address my critics: Some say that this is just Tom trying to promote his books. Well, if that's what I was doing, do you think I'd write this on my book-promotion blog?

By the way... you can get a sneak preview of my next book: First, the new issue of ACM Queue magazine has the complete text of chapter 2 (free to ACM members, everyone else pays a small fee). Alternatively you can also see the latest complete draft on Safari Books Online which you probably already have a subscription to.

That's all I have to say on this matter.

Posted by Tom Limoncelli in Funny

The March/April issue of acmqueue - the magazine written for and by software engineers that leaves no corner of the development world unturned - is now available for download.

This issue contains a preview of a chapter from our next book, the 3rd edition of TPOSANA. This issue contains a preview of a chapter from our next book, the 3rd edition of TPOSANA. The chapter is called "The Small Batches Principle". We are very excited to be able to bring you this preview and hope you find the chapter fun and educational. The book won't be out until Oct 7, 2016, so don't miss this opportunity to read it early!

The bimonthly issues of acmqueue are free to ACM Professional members. (One-year subscription cost is $19.99 for non-ACM members.) You can also buy a single issue. For more information.

Posted by Tom Limoncelli in ACM Queue Column

[This is a rant. Take it with a grain of salt.]

You know what's great about "the cloud"? I don't have you deal with [insert server vendor's name] support process that is so complex and broken that it makes me want to die. If a machine in AWS/GCP/Azure dies I don't have to load a f***ing flash-based web page that breaks on .... oh my god... every browser except one that is 10 years old and runs on an OS that I don't use... and .... god damn it what do you mean my account isn't cleared for that product and... F***!!! what do you mean I'm required to lie to get the service I need??? and... no.. don't ship it to "me" ship it to the datacenter and.... AAAAAHHHHRRRRGHHH!!!

Here's a clue: if your support process requires your customers to lie, it is broken.

Oh, and every vendor has a different process that takes months to learn. If we have many vendors, it is an entirely different set of frustrating and illogical processes that must be learned for each one. If we only have incidents occasionally, we'll never actually learn the process.

[Insert vendor name here]... you're competition isn't [other hardware company]. It is switching to AWS/GCP/Azure so that I don't have to f'ing deal with you and your broken processes any more.

The same goes for [other hardware company]. ..and [that other one too]. You're all terrible and deserve to go out of business.

You will, of course, as everyone moves to the cloud. The cloud providers make their own hardware. Everyone that "moves to the cloud" is a customer you'e lost. A knife in your back. The more popular cloud providers become, the less need there is for Dell/HP/etc. to exist.

Eventually a time will come where the only people that aren't using AWS/GCP/Azure/DigitalOcean/Rackspace are people that can't for regulatory reasons. The market for on-prem hardware will be so small that the industry will have to consolidate. You'll have the cloud providers that make their own hardware plus "Bob's house of server hardware that I sell to the sorry lot that can't use the cloud". BHOSH will be like dealing with Roz from Monster's Inc. You don't want to deal with her if it can be avoided, but she can't be avoided.

It will be so terrible that industries lobby to change the regulations to permit use of cloud providers.

If they succeed then the market for on-prem hardware will shrink more and the only people that will actually need server hardware won't have any vendors to buy from. That sorry lot will have to buy desktop hardware and iPads and retrofit them with Linux to run their local services. Maybe ARM and IoT devices will become powerful enough that they can run [insert ironic service for comic effect] for on-prem computation.

Posted by Tom Limoncelli in Rants

CU-Boulder will be hosting an event on April 26, 2016, to celebrate the life of Evi Nemeth who passed away three years ago. You may remember Evi from her many books on system administration, her tutorials at Usenix LISA, or many of her other projects that influenced system administration as it exists today.

... we will celebrate the life of retired professor Evi Nemeth, an accomplished sailor who was lost at sea in June 2013. Evi joined the department in 1980 and was one its foundational figures for 20 years. She is best remembered for her rigorous data structures class and for providing a safe haven and confidence-building experiences for a generation of students who didn't fit the typical academic mold.

All parents, alumni and friends of the Department of Computer Science and of Evi Nemeth are invited to attend the expo. Alumni, parents and friends are asked to RSVP at www.colorado.edu/cs/expo2016.

(from this article at CU-Boulder's website)

If anyone that reads this blog is able to attend, please send a report about the event. I'd be glad to post it here on this blog.

Posted by Tom Limoncelli in Evi Nemeth

A program I wrote that worked for quite some time started failing. It turns out someone tried to use it to process a file with text encoded as UTF16. The file came from a Windows system and, considering things like UoW, this situation is just going to start happening more frequently.

Golang has a great package for dealing with various UTF encodings. That said, it still took me a few hours to figure out how to make an equivalent of ioutil.ReadFile(). I wrapped up what I learned and made it into a module. Everything should just work like magic.

  • Instead of using os.Open(), use utfutil.OpenFile().
  • Instead of ioutil.ReadFile(), use utfutil.ReadFile().

The module is available on Github: https://github.com/TomOnTime/utfutil

Posted by Tom Limoncelli in Technical Tips

Our next guest will be kc claffy. We'll be discussing her talk from LISA '15 titled Named Data Networking.

Watch live! We'll be recording the episode on Tuesday, April 26, 2016 at 3:30-4:30 p.m. Pacific Time. Particpate in the live Q&A by submitting your questions during the broadcast. Pre-registration is recommended but not required. Register and/or watch via this link.

The recorded episode will be available shortly afterwards on YouTube.

You won't want to miss this!

Posted by Tom Limoncelli in LISA Conversations

 
  • Register Today