What is Latency in Networking?
What is latency? You will get a variety of answers depending on who you ask. As network engineers, when we say latency, we’re generally referring to the amount of time it takes to send a packet of data from one location to another. Users generally aren’t affected by the latency itself but by the effect of the network latency on applications. A 100ms latency to download a webpage would be wonderful, but 100ms of latency causes the underlying protocols to limit the transmission speed, so what the end-user sees is seconds of lag, not 100ms of physical layer latency. Whether you’re playing a game online, sending an email, or browsing the web, the way applications respond to latency is critical in successfully accomplishing any task.
On an enterprise level, milliseconds can make a world of difference, and application responsiveness can make or break a company, so it’s vital to properly understand the concept of latency.
Jump To Section
What Causes Latency?
No matter the network, three primary factors significantly contribute to latency; propagation delay, routing and switching, and queuing and buffering.
Latency vs Delay
The term delay is often used interchangeably with latency, but there is a subtle difference between the two. Propagation delay refers to the amount of time it takes for the first bit to travel over a link between sender and receiver, whereas network latency refers to the total amount of time it takes to send an entire message.
Propagation delay is computed as a function of distance over wave propagation speed ( d / s ). In this equation, speed is expressed in relation to the speed of light, but that doesn’t mean you’ll be able to share cat memes with your friend on the other side of the world in the blink of an eye. The speed of light through fiber optic cable is approximately 1ms per 200km, which is around 30% lower than the speed of light in a vacuum (299,792,458 meters per second).
If you’re sending a .JPG from Los Angeles to your friend in New York, you may expect a latency of ~22.45ms to transmit that data over a distance of 4,488.9km on a fiber link, but end-to-end latency is usually measured as a round trip time instead of 1-way. Since the path is usually not direct between any two points, cross-country latency can usually sit around 100-120 ms as opposed to 22 ms.
Propagation speed also varies based on the physical medium of the link in use. When transmitting over copper cable, propagation speed can be as low as 60% less than the speed of light. This degradation in speed is referred to as the velocity factor (VF), but whether you’re using fiber, copper, or coax, the primary contributing factor to latency is propagation delay. Since the primary contributing factor to propagation delay is distance, whenever dealing with network latency, one should mostly be concerned with distance.
The difference between the transmission of the first and last byte in a packet is the transmission or serialization delay. This is infinitesimal with a small packet on a high bandwidth backbone link (10G or 100G) but could add hundreds of milliseconds for a large packet on a low bandwidth link.
Routing & Switching
During its journey, data passes through various controllers, routers, and switches that help it reach its destination. Each one of these gateway nodes is responsible for a different task in figuring out what to do with the data. With the advent of software-defined wide-area networking (SD-WAN), the routing of data can take a minimal amount of time. For example, an SD-WAN controller can constantly monitor each available path and dynamically choose the least congested available to route data most efficiently. A network latency simulator can help you with real-world WAN conditions prior to deployment. Routing and switching delay is infinitesimal. The main delay through routers and switches is the queuing delay.
Queuing & Buffering
No two networks are exactly alike. One network may not experience a high volume of traffic while another may serve a multitude of users. If one link is heavily saturated with traffic, TCP/IP attempts to avoid packet loss and preserve data integrity by placing packets in a queue for later processing. This ensures packet loss is kept at a minimum within the network.
The amount of time a packet sits in a queue is referred to as queuing delay and the number of packets waiting to be processed is referred to as a buffer. As the number of packets increase, so too does the length of the buffer, which consecutively contributes to network latency. There are different types of queues and buffers which handle packets using various algorithms, but it’s important to understand what affects latency and the transmission of data.
Of course, this may be an oversimplification of complex networking events and there is a multitude of other factors that can cause latency. Either way, the idea is that network latency significantly affects application responsiveness and the primary contributing factor to latency is distance. Applications that appear highly responsive on a local network may perform terribly when deployed on a wide area network once the distance is introduced. Though it can never be reduced to zero, well-developed applications and latency simulators can help you avoid performance issues.
How to Reduce Latency and Improve Performance
As a developer of network latency and bandwidth simulators, many people ask us, “How can I reduce network latency?” As previously mentioned, network latency is mostly a function of distance, so it can be difficult to reduce substantially. Of course, switching from a satellite network or even a wireless network to a terrestrial network can make a big difference. And especially for shorter distances, finding a carrier that has the most direct connection between your two sites may help. Getting rid of any congestion on the network can reduce queueing delay. But in most situations, the latency is a simple function of the distance between the two sites and that doesn’t change.
What the end-user cares about, whether playing a networked game, surfing the web, or opening a PowerPoint presentation from the company file server, is application responsiveness. And that often can be improved.
Taking the simple example of downloading a large file, we usually use FTP. If the network latency is high or there is significant packet loss, changing the TCP window settings and other parameters can improve FTP performance dramatically. Or instead of using FTP, we could use a different data transfer application entirely. Or if neither of those options is convenient, we can insert a WAN optimization device onto the network.
Similarly, other applications can be fast or slow to respond depending on many factors: the size of each of the elements to be downloaded, the number of separate round trips required to load all of the elements, and the protocols used and their settings. Each of these is affected by the network latency as well as the amount of bandwidth available and any packet loss. Applications that may be extremely responsive on the local network with gigabit bandwidth and no network latency or packet loss may be painfully sluggish in the real world.
Applications, therefore, need to be tested with real-world WAN conditions prior to deployment, and ideally during the development phase while it is still easy to optimize the design for a wide range of user conditions. So our answer is this: you probably can’t change the network latency, but by simulating the network latency and other network conditions, you can optimize the application design to change what matters – the application responsiveness.
It’s the Speed of Light, Dude! Testing with Our Network Latency Simulators
At the recent Sharkfest conference where Apposite was exhibiting our PCAP replay feature, a number of people walked up to our booth and said, “You’re the dudes who inject latency to simulate all the router hops along the network?” Well, kind of. We certainly make products that inject latency to simulate WAN links, but the latency is primarily caused by the speed of light. Most people refused to believe us.
“The speed of light is, like, close to infinite, dude. It’s all the routers holding up the packets,” was the common reply. And our answer: “Nope, it’s the speed of light.” Router latency is on the order of tens of microseconds, not milliseconds (though to give them partial credit, they probably meant queuing delay which can be significant if there is any congestion).
Indeed, the speed of light is fast, so fast it’s awfully hard to measure. On human scales, it’s essentially infinite. Of course, everyone knows it takes years for light to reach other stars, but they’re zillions of miles away. But at computer scales over continental distances, the speed of light is significant and is the immediate cause of WAN delay.
Let’s take the example of San Francisco to New York City. With a straight shot, that’s a distance of approximately 3000 miles or 5000 km. And the speed of light through fiber is approximately 2000 km/sec, or 5 ms per 1000 km (note that this is a good deal less than the 3000 km/sec speed of light through a vacuum cleaner.) So SF to NY takes 25 ms, and NY back to SF is another 25 ms, for a round-trip latency of 50 ms.
Of course, backbone networks are not a straight shot, and the actual path, even a relatively straight one, may be closer to 6000 or 7000 km in each direction for a round trip latency of around 70 ms. Add in some transmission delay and maybe some queuing delay along the way for a typical value of 100 ms RTT. But the majority of that is the speed of light.
Test the performance of your application against potential network impairments by emulating network latency and much more at speeds up to 100Gbps. Netropy enables users to import live traffic, vary impairments, and measure their impact in real-time. If you have questions about how to reduce latency and improve performance with our emulators, please Contact Us or call (310) 477-9955.