Is Latency the Bane of Broadband?

Defined as the time from a stimulus to the completion of its response, latency is an often under-appreciated aspect of the broadband experience.  When considering broadband, most people ignore network latency and focus almost exclusively on download performance, repeatedly measuring their download rate with Internet speed tests like Speedtest.net. Although download rate is very important, it is certainly not the only thing that contributes to a good broadband experience. The Internet is becoming a very social medium, and high network latency’s major adverse affect on interactively limits the social value of the Internet.

The Internet was not designed for low latency transmission, so reducing latency to better serve interactive communications is quite difficult. With the current architecture of the Internet, some factors cannot be improved as they require changes to the laws of physics. No one will increase the speed of light, and the speed of light is rather slow when packets are routed around the world to serve a request. A single trip from the east coast to the west coast of the US requires about 25 ms just owing to the speed of propagation (about 2/3 the speed of light in a vacuum for transmission of electrical signals in copper, faster in fiber), and there is nothing that can be done to significantly reduce this. Other factors can be improved, but there is certainly a limit.

Latency’s effects depend on the application. These are discussed below by application.

Broadcast Media

Network latency has very little effect on broadcast communications. Examples of broadcast communications include watching video on YouTube and Hulu, listening to a podcast, or watching a recorded slide presentation. There is no interaction between the originator of the content and the user of the content, so if the content delivery is delayed by even a second or so, there will be little effect on the value of the communications.

Where latency can be a problem for broadcasting is in its variance, a value known as jitter . To compensate for changing latencies, a client receiving the broadcast stream will buffer the arriving packets and then steadily feed them to the user. However, if the jitter buffer is too small, and the latency variation is larger than the buffer’s capacity, then the user experience will suffer from jerky video and/or stuttering audio. The jitter buffer must be as least as large as the largest variance in delay experienced by a connection. Ultimately, the size of the jitter buffer is just an educated guess. Latencies can increase suddenly, and occasional problems are possible.

Interactive Media

Latency adversely affects interactive communications, both audio and video communications. Quality interactive Voice over IP (VoIP) communications requires low latency, and for the latency to be undetectable, it needs to be less than 50 ms. With higher latencies, interactive conversation with VoIP is difficult because a user has a hard time determining when to start talking. Arguing or having a heated discussion with someone is almost impossible. One party begins talking without realizing that the other party is also talking, so the two talk over one another and have to restart. Skype is a common interactive voice application used on the Internet.

Interactive video has even higher requirements for low latency. For someone to appear as though they are truly listening, their visual and audio feedback (nodding, uh huh, etc.) needs to be immediate and certainly less than 50ms. Longer latencies make it impossible for someone to properly convey that they are listening, and video becomes a distraction and a hindrance rather than an enabler. Interactive video with audio has the same audio problems that voice-only communication does.

Surfing the web is another interactive broadband Internet experience that is adversely affected by latency. A single web page is typically composed of tens of different files that must all be downloaded to display a single page. Serially downloading these files, even across very fast broadband, can tax the patience of almost anyone. For example, NYTimes.com requires 87 files to fully display its home page. If roundtrip latency is 200ms, serially downloading the home page requires at least 17 seconds just for the roundtrip latency, and other factors will make an actual download of NYTimes.com take even longer. Many viewers are lost once a page load exceeds 10 seconds, so parallel downloads are used by web browsers to optimize the user experience.

File Downloads

File downloads are largely unaffected by network latency owing to the windowing capability of TCP (the end-to-end layer four protocol that runs on top of IP). The most important factor is the TCP receive window size, which determines the amount of received data can be buffered in the client (typically a PC) before the sender (server) requires an acknowledgement to continue sending. By adjusting its receive window size, TCP allows the server to send packets to completely “fill the pipe” so that the it does not have to wait for acknowledgements from the client to continue sending.

For a server to use the entire bandwidth available, the client must have a receive window size equal to the bandwidth delay product. The bandwidth delay product is calculated as follows: (lowest link bandwidth on a connection) x (roundtrip delay). For a connection with the slowest link being a T1 (1.536 Mbps), and roundtrip delay of 200 ms, the bandwidth delay product is 307200 bits. The TCP window size on the connection must be equal to at least (307200 bits) / (8 bits/byte) = 38400 bytes for the full 1.536 Mbps to be used for a single download, and this is well within TCP’s maximum receive window size of 64k bytes.

To get beyond the 64k byte limit, Vista supports a feature called TCP Window Scaling to efficiently handle higher latency and higher bandwidth connections (both affect the bandwidth delay product). With scaling, an option is sent in synchronize (SYN) segments during the TCP connection establishment process. The option scales the TCP receive window to support up to a quite large value of about one gigabyte.

XP’s handling of TCP’s receive window is rather crude, but Vista includes a feature called Receive Window Auto-Tuning that automatically optimizes TCP throughput, and it is especially effective on paths with a high bandwidth delay product. Receive Window Auto-Tuning measures the bandwidth delay product and the rate at which data is cleared from the receive buffer and then periodically adjusts the receive window size for optimum throughput. Windows XP is much less sophisticated and requires manual configuration of the receive window size, which applies identically to all connections. Vista’s Receive Window Auto-Tuning works on each connection separately, individually tuning the receive window for maximum throughput.

Supporting a maximum scaled receive window size of 16 megabytes, which is 256 times larger than TCP’s unscaled receive window, Vista will accept around 600 Mbps on a single TCP connection having a roundtrip delay of 200ms. This is well beyond the capabilities of most PCs to digest.

Testing Network Latency

The easiest way to measure latency is to use the ping command, which is a tool built into many operating systems like Windows and Linux to assist in network performance management. On Windows, bring up a command prompt window and just type “ping google.com” to try it out (Note that not all websites will respond. Google.com and Yahoo.com will both respond.). Windows will send out several pings and report on the latency for each (mostly network latency usually). The diagram below (click to enlarge) shows a representative path of a test with the ping utility from a residence to google.com and back. Note that every element in the connection contributes to latency.

On links, the contributors to latency are the serialization delay and transmission delay. Serialization delay is the time required for a link to assimilate a packet and depends on the speed of the link. Transmission delay has to do with the time required for bits to travel a link, which could be as much as 25 ms should the link cross the US or the Atlantic ocean. Transmission delay has nothing to do with the speed of the link. Additional delay is introduced in the various routers, in the server processing the request, and even some in the local PC.

Other options for latency testing include tools available from Speedguide.net and Pingdom.com. Speedguide.net offers a ping utility as well as a traceroute tool that allow you to monitor latency on the connection from their site to your PC or any IP address. Their tools allow more net latency testing options beyond just the latency from your PC to a server on the Internet.

Pingdom.com provides a good test of website latency, and its tool also shows all the files downloaded to display a page. In web page response time, the number of files is more important than the size of the files unless they are unreasonably large; browsers are limited in the number of files they will download at one time. A web page requiring 40 files, and accessed with a browser supporting only 4 file downloads at a time, may require 10 separate sets of downloads to render a page. Note that many web pages are viewable well before all the data describing it is delivered to the client PC, and this reduces the problem.

To Sum Up

No, network latency is not the bane of broadband, but it is the adverse factor of broadband performance that is by far the most difficult to improve. In the coming years, we will have hundreds of megabits per second of broadband delivered to our homes, but the latency of our connections likely will not improve much at all. The Internet was originally designed to carry non-real time data traffic, and it has done a fine job of that along with many other things it was never designed to do. Without radical changes to the Internet’s architecture, interactive communications will forever suffer from the latency inherent in the current design of the broadband Internet.

For more, see this ACM article, Latency Lags Bandwidth (registration required), and this GigaOM article, Pain at the Pipe: Latency Matters.

© 2009, The Product Group LLC. All rights reserved.

Related posts:

1. “Enough” Broadband is Never Enough I read lots of discussions about broadband, and the “enough” word always seems to pop up.  As in “10 Mbps to the home is enough...
2. Wireless Broadband the Stimulus Winner? I am hearing that wireless may be the big technology winner for the first round of Broadband Stimulus funding.  Wireless does provide a lot of...
3. The Death of Telephony The era of voice driving network investment is over. Voice is here, it is still useful, but it is almost invisible in the design of...
4. The Internet’s Secret is No Sauce The Internet is very good at what it does, though rarely is it given the praise it deserves. More often than not, when people discuss the...
5. Broadband Wireless Overview Broadband wireless Internet access is simply the delivery of broadband Internet rates over a wireless connection.  Especially popular for rural areas, broadband wireless competes with...