The Internet’s Secret is No Sauce

The Internet is very good at what it does, though rarely is it given the praise it deserves. More often than not, when people discuss the Internet’s support of an application, they do so to complain about the Internet’s limitations. But if someone is asked to find another example of anything that has grown so big so fast, and affected our society so deeply, they are at a loss for words.

The Internet doesn’t do much more than get information from one place to another anywhere in the world (and eventually beyond). But that is exactly what it was intended to do, and it does this very well and very cheaply. It is a basic communication medium that transports data packets (called IP datagrams. think of a mail envelope) virtually anywhere anyone cares to communicate. See image below for a representative interaction on the Internet.

Click to enlarge

Less Really is More

Sometimes less really is more when it comes to the Internet. I have watched the Internet grow from its humble beginnings to the juggernaut it is today, and at many points along the way, it grew at truly stupendous rates. This is the kind of growth rate that kills, but the Internet not only survived, but thrived. There were times when its infrastructure strained to handle the load, but by and large, it did an admirable job serving demands it was never remotely anticipated to serve.

What has enabled the Internet to grow so fast and well is its limited nature. The key factor allowing this kind of unbridled growth is not what the Internet attempts to accomplish, but what it does not do. The Internet provides a bare minimum set of functions that allow universal communications. Its transport of IP packets is best effort, latency is highly variable, and there is absolutely no guarantee of delivery. If data can get through, it will. And if it cannot, well then it won’t. No guarantees. But, virtually all of the time, the data does get through.

With the Internet, it is Bring Your Own Resources. The devices connected to the Internet provide most of the power needed to handle their applications. As they proliferate, the resources available for handling their applications increase at almost the same rate. The Internet does the absolute minimum necessary to allow devices all over the world to communicate, and everything else is handled by devices connecting to it. When it comes to building a worldwide public networking infrastructure like the Internet, less really is more.

Stateless Networking

What was the one factor most responsible for allowing the Internet’s rapid growth over the last 15 years?  This is the one thing that, if not true, would have fundamentally changed the success of the Internet. I say it is the fact that the Internet itself maintains no state information for user connections; they are stateless, meaning that every IP packet transported between a client and a server is handled individually and has nothing to do with any that preceded it and no effect on those that follow it.

Click to enlarge

The Internet knows only that a particular IP datagram has to go from here to there and not much else (see diagram above). Connections exist only as associations in the client and server to each other. Interestingly, the simplest, cheapest router on the Internet can route millions of connections because it is totally, completely, and utterly unaware of these. It is occupied only with forwarding each data packet individually.

Best Effort Delivery

As in life in general, there are no guarantees on the Internet. Your data may take 20ms to reach its destination (that is very fast), or it may take 200ms or 2000ms, or it may never arrive at all, but that is all par for the course. The protocols and applications on the Internet must deal with this variance and non-delivery to provide a useful service. In some cases, the Internet is not perfectly suited to an application (say interactive voice telephony), but since it costs so little, people will still use it for this purpose.

The Internet is often criticized for not providing Quality of Service (QoS). Since the Internet only provides best effort delivery, it does not maintain Quality of Service (QoS) information for connections or really any information about connections at all. QoS is one form of state information, and for a router to guarantee QoS, it must track connections and refuse connections after a certain point. This dramatically decreases the scalability of a router. Although not suited to a public network like the Internet, QoS can be useful in enterprise networks where the entire network is under one organization’s complete control.

Lack of QoS allows the Internet to gracefully degrade, slowing down but not completely stopping when a portion of it is overloaded. If connections required reservation of resources, when those resources ran out, new connections would be refused. This rather undemocratic method, of course, has tremendous potential for abuse.

The Stateless Web

Web browsing is largely a stateless activity. When a web browser requests a web page, the server delivers all the files required to render the page, largely forgets what just happened, and then moves on to serve other users. Because servers can forget clients once served, they can easily serve many thousands of users every day and many users at once.

Cookies are the mechanism whereby some limited state information is maintained in web clients. The web client is responsible for storing all state information about interactions with a particular web server. As web clients proliferate, so does the storage capacity of this state information.

In Closing

The brilliance of the Internet’s design lies in its simplicity. If it had incorporated even a small fraction of all the features people have said it needed, it would almost certainly have been a failure. It would never have become the universal medium it is today. The Internet has succeeded largely because of what it has not attempted to do.

© 2009, The Product Group LLC. All rights reserved.

Related posts:

1. Broadband Internet Can be Deadly When people think about Broadband Internet access, they tend to think only good things.  I have made the analogy that Broadband Internet access  is now...

1. cyberdoyle says:

   October 9, 2009 at 9:55 am

   ‘The Internet has succeeded largely because of what it has not attempted to do.’
   Excellent and so true. Now we just need to get rid of obsolete copper bottlenecks and we can see what it will accomplish. The first country to do this will reap untold rewards…
   chris
2. Richard Bennett says:

   October 21, 2009 at 2:00 am

   Incredibly silly post. The cell phone network has grown larger and faster than the Internet, and managed to provide Quality of Service while doing so. It is a statetful network that supports the killer app for networking: live interpersonal communication.

   It proves that the Internet design dogma is wrong.
3. admin says:

   October 21, 2009 at 5:41 am

   In terms of traffic carried, cell phone networks have, until now, managed far, far less than the Internet. The Internet has effectively dealt with both high growth in number of users and in bandwidth consumed per user. Though it really has nothing to do with stateful networking, data traffic from iPhones has more than challenged the AT&T network. As stated in my article “The Death of Telephony”, I suspect AT&T thought they were selling a phone and not an Internet terminal. And, realize that with 4G technology, “cell phone” networks become just a wireless part of the Internet and carry only IP traffic, both voice and data.