The protocol is called "SS7" (Signaling System 7). Like most teleco protocols it is difficult to parse and ill-defined. This is how telcos keep new competition from starting. They hype SS7 as something so complicated that only rocket scientists could ever understand it. Of course, it is an ITU standard, so it isn't a secret how it works. You just have to pay a lot of money to get a copy of the standard. In fact, once Cisco had a working SS7 software stack the downfall of Lucent/AT&T/others was only years away. Heck, Cisco published a book demystifying SS7. It turns out the emperor had no clothes and Cisco wanted everyone to know. SS7 is big and scary, but only as bad as most protocols. I guess SMTP or SNMP would be scary too if you had never seen a protocol before. (Remember that non-audio networks are still "new" to the telecom world, or at least their executives.)
SS7 is all about setting up "connections". When I dial a number, SS7 packets are sent out that query databases to translate the phone number I want to dial to a physical address to connect to, then an SS7 query goes out to request that all the phone switches from point A to point B allocate bandwidth and start letting audio through. The nomenclature dates back to what was used when phone calls were set up by ladies sitting in front of switchboards.
What makes international dialing work is that there are SS7 gateways between all the carriers. They don't charge each other for this bandwidth because it is just the cost of doing business. The logs of what calls are actually made is used to create billing records, and the carrier do charge each other for the actual calls. Thus, there is no charge for the SS7 packets between AT&T and O2 (O2 is a big cell provider in Europe), but O2 does back-bill AT&T for the phone call that was made. (This is called "Settlement" and my previous employer processed 80% of the world's settlement records on behalf of the phone companies.)
Setting up a connection for an SMS would be silly. An entire connection for just a 160-byte message? No way. That's more trouble than it is worth. Therefore, SMS is the only service where the actual service is provided over SS7. The 160-byte limit comes from a limit in SS7 packet size.
However, the phone companies don't really do anything for free. The SMS records are used to construct billing data and the companies certainly do back-bill each other for SMS carried by each other's networks. If you SMS from AT&T to O2, there is settlement going on after the fact. However, SMS between two AT&T customers has no real cost.
"Multimedia SMS" (photos) are not sent over SS7, though SS7 is used to setup/teardown the connection just like a phone call. If they were smart they'd use SS7 to just transmit an email address and then send the photo over the internet. It would probably be cheaper. (Though, when has a telco has a well-run email system? Sigh.)
So, SMS is "free" because it rides on the back of pre-existing infrastructure. The "cost" is due to the false economics created to "extract value" out of the system (i.e. "charge money").
If they were doing it all from scratch, they could probably run it all over the internet for "free" too. Heck, it wouldn't be much bandwidth even if people learned to type 100x faster.
Why was SMS permitted to use SS7 unlike any other service? The real reason, I'm told, wasn't entirely technical. It was due to the fact that the telecos thought that nobody would actually use the service. Little did they know that it would catch on among teens and then spread!