I hope to teach a "how to interview" class at an upcoming conference. Here's one of the points I'll be making.
How can one interview question help me understand what the candiate does and doesn't know Unix?
Here's the question:
What happens with I type this at a shell prompt:
telnet www.wikipedia.org 80 RETURN
Usually the candiate will explain just the command: "It opens a connection to port 80 on wikipedia". That's a good answer.
The follow-up question is, "Please give me more detail".
At this point they might explain how DNS works or how TCP network connections work. Whichever they explain, you can ask about the other one.
That is, of course, skipping the fact that the command was started: You can ask them to explain how a process is created in Unix. You can ask them to explain how command lines are parsed.
Basically you can drill down and down and down and down. Any time the candiate says, "I don't know" ask them to guess. After the guess, move to some other aspect of the system.
You can do this in 15 minutes or 15 hours depending on how detailed you ask the candidate to get.
The point is that you will learn a lot about what the candidate does and doesn't know. Don't get hung up if they don't know half the issues you bring up, just make sure the parts that are known are the parts the job opening requires knowledge. Watch the "guesses" to see if they are logical or do they require what I will politely call "magical thinking". Not knowing how DNS works is forgivable for some positions. Thinking that TCP works with hostnames instead of IP addresses is not forgivable. Thinking that "the computer just knows" is a magical thinking and is not forgivable.
I call this "Zeno's Interview Question" because you never get to the end. There is always more to discuss if you break the situation down into smaller and smaller parts to be described. Generally the conversation doesn't go linearly. You often ask "what happens before that?" When all is said and done you probably have traversed the timeline back and forth and back and forth; moving up and down layers of abstractions again and again.
You can use different command lines depending on if the position is for a sysadmin or software developer. The above example is good for a sysadmin because it involves DNS and networking. A systems programmer might be asked "cat *" instead, especially if you want to introduce that there is a file called "-f" in the current directory. You'll learn a lot about whether or not the candidate understands how command line parsing works.
Here is a list of topics this can generate:
- How USB keyboards transmit keystrokes.
- How keystrokes get processed through the kernel.
- How the unix concept of pty "raw vs. cooked" mode.
- How the shell parses lines (quotes then variables then {} then globs).
- How the shell executes commands: aliases then relative path or search $PATH.
- How commands are executed: all about fork() and exec()
- How a process starts: the Unix disk I/O system, virtual memory, paging executables into memory, bss vs data. Jumping to the first instruction.
- How argv is passed to the process.
- How a process parses flags.
- How DNS lookups work (in the simple case)
- How DNS lookups work (in the recursive case)
- To send the DNS packet we need to discuss:
- How L3 routing works
- How L2 packet passing works (Ethernet Switching, "cam" tables)
- VLANs
- Finally we have an IP address. Now we discuss...
- How a TCP connection is created.
- How sequence numbers work.
- How a TCP connection sends data.
- Nagle's Algorithm
- My fingers are getting tired.
(Note: the above list is incomplete, by definition. It probably also includes a few errors.)
We discussed this at the monthly meeting of the New Jersey Chapter of LOPSA a few years ago. It was 60 minutes before we got to the point that a packet hit the network. At the 2-hour mark we stopped the meeting because, to be honest, we were all pretty exhausted.
If you use this as an interview question I highly recommend you first tell the candidate how much time you've allocated for this question. If they know you want this to last 20 minutes, for example, it helps them give an indication of how much detail is being expected.
The best part of this question is that there's no way to "cheat". If the candidate has heard this question before it doesn't help them. The only way to "cheat" is to read all the books and other resources required to learn all of this. That's called education.
Last week I started my new job at StackExchange.com. You may know SE as the company that runs great Q&A websites like ServerFault.com and StackOverflow.com plus their amazing job site Careers 2.0. I know the company as Joel Spolsky's startup that has been hiring a lot of great people that I know through conferences such as like LOPSA-East, the Cascadia IT Conference, and USENIX LISA. I just joined yet I already feel like everyone is an old friend.
