Sometimes it seems strange that a problem can develop and go undetected, even under watchful eyes. That’s exactly what has happened with the phenomenon called bufferbloat.
While its technical detail is very complex, the basic ideas behind bufferbloat are follows:
• HTTP and TCP flows (such as streamed video) are dependent on the sending TCP software correctly estimating the available bandwidth between the ends of the circuit.
• To reduce packet loss, vendors, carriers, and even OS developers have increased buffer sizes across the network path.
• Big buffers are a problem, generally. End-to-end latency is sometimes ten to 100 times what is should be because a retransmitted packet or an acknowledgement packet is sitting somewhere in a large buffer.
• There are basic methods to detect whether bufferbloat is affecting your sending application.
• Strong evidence indicates that it may be increasing the latencies of our web sessions by a factor of three to ten times. How could this happen? What can be done about it?
This has happened because only recently have we have started to understand the TCP protocol, even though 80-90% of our web traffic runs over it. This is where the core of the bufferbloat problem starts. TCP sends data and waits for an acknowledgement. If the acknowledgement is delayed or lost, TCP slows its transmission rate. The user sees this slow down. Bufferbloat, or described another way as bloated buffers, means that the network has large storage buffers in both the sending and receiving ends; some of them are filled to or near capacity. So, acknowledgements sit in buffers behind other traffic. Bufferbloat can affect all traffic.
What can we do about it? We need to shape (restrict) the traffic at the sending station, so that buffers don’t overfill. That isn’t a trivial exercise. However, router manufacturers are slowly modifying their software to make it possible.
This problem is not widely understood, and it is difficult to fix. However, end users should be aware of it and not assign blame to another source such as a vendor or competing traffic.
We’ll do our best to keep you up to date on developments about this issue.
