Earlier this week the research consortium M-Lab released a big report on the way data moves through the guts of our internet infrastructure. It focused on interconnection points, the shared equipment nodes where different networks exchange data so that it can move around the world and into your home. What it showed was that business disputes — between transit ISPs like Cogent and Level, which carry data around the world, and access ISPs like Verizon and Comcast, which carry it the last mile to your home — were having a dramatic effect on the ability of many American consumers and businesses to access the internet, reducing the flow of data to the point where even basic tasks like email would be slow or impossible.
This was important new research that highlighted how strained the infrastructure underpinning our internet is. But depending on which side of the business dispute you sit on, the data showed two very different things. For ISP advocates, the data was clear evidence that Netflix was to blame for the degradation in service. For ISP opponents, the data was clear evidence that Comcast and Verizon were trying to punish Netflix, and harming others in the process. But as the author of the M-Lab study argues, that heated argument is causing them to miss the forest for the trees. "What’s more important, and more troubling, is the extensive and continuing issues of capacity at many of the interconnections points which had nothing to do with Netflix," said Collin Anderson, a researcher at M-Lab.
If the congestion had been isolated to a single interconnection between one transit and one access ISP, it might have been possible to lay the blame on Netflix or Comcast for allowing things to degrade while they bargained over the price of a direct interconnection. But the M-Lab study found that "these issues cannot be laid at the feet of any one Access ISP, or any one Transit ISP: no Access ISP performs badly to all Transit ISPs, and no Transit ISP performs badly for all Access ISPs. Therefore, if the problem is not at one end, and not at the other, it must be in the middle around the interconnection between the two." As the companies that move our data haggle over who should foot the bill for the exploding amount of internet traffic, our most vital modern utility can become nearly inaccessible to consumers.
If the congestion had been isolated to a single interconnection between one transit and one access ISP, it might have been possible to lay the blame on Netflix or Comcast for allowing things to degrade while they bargained over the price of a direct interconnection. But the M-Lab study found that "these issues cannot be laid at the feet of any one Access ISP, or any one Transit ISP: no Access ISP performs badly to all Transit ISPs, and no Transit ISP performs badly for all Access ISPs. Therefore, if the problem is not at one end, and not at the other, it must be in the middle around the interconnection between the two." As the companies that move our data haggle over who should foot the bill for the exploding amount of internet traffic, our most vital modern utility can become nearly inaccessible to consumers.
OUR MOST VITAL COMMUNICATIONS UTILITY CAN BECOME NEARLY INACCESSIBLE TO CONSUMERS
The report found the problems follow a predictable pattern, one that not surprisingly is also at the center of Netflix's dispute with ISPs. "Observed performance degradation was nearly always diurnal, such that performance for access ISP customers was significantly worse during peak use hours, defined by the Federal Communications Commission (FCC) as the hours between 7pm and 11pm local time. This allows us to conclude that congestion and under-provisioning were causal factors in the observed degradation symptoms." We’re increasingly relying on the internet for our evening entertainment, and that is where the real problems are cropping up. That’s Netflix, but also Hulu, YouTube, Twitch, and millions of other services. With cable channels and network broadcaster both embracing a streaming only model, this problem is going to get a lot worse, and soon.
How can this problem be solved? Netflix would argue that the FCC should expand the definition of net neutrality to cover interconnection, and force the access ISPs to pay for interconnection upgrades to handle consumer’s ever larger demands for internet entertainment. The FCC has signaled it will look into interconnections, but is not likely to place them in the same bucket as net neutrality. The ISPs have argued that Netflix is just looking for a free ride, and that content companies should pay them for a direct interconnect. Netflix has done this, and the performance of its traffic has improved drastically. HBO recently signaled that it believed content companies and ISPs should share the financial burden of upgrading our infrastructure to handle the increased demand of streaming entertainment.
How can this problem be solved? Netflix would argue that the FCC should expand the definition of net neutrality to cover interconnection, and force the access ISPs to pay for interconnection upgrades to handle consumer’s ever larger demands for internet entertainment. The FCC has signaled it will look into interconnections, but is not likely to place them in the same bucket as net neutrality. The ISPs have argued that Netflix is just looking for a free ride, and that content companies should pay them for a direct interconnect. Netflix has done this, and the performance of its traffic has improved drastically. HBO recently signaled that it believed content companies and ISPs should share the financial burden of upgrading our infrastructure to handle the increased demand of streaming entertainment.
One could look at the Netflix situation and conclude that the marketplace works. After they paid up, things got better, and the cost for Netflix was less than what they paid to third party transit ISPs. The problem for Netflix is that they don’t want to be beholden to the big access ISPs, many of which also own content companies that compete with Netflix. If the cost for a direct connect were to change in the future, Netflix is worried Comcast and Verizon would have all the leverage.
That brings us back to the nuclear option: the FCC using Title II to declare our broadband internet infrastructure a common carrier. That would turn Comcast and Verizon’s infrastructure into dumb pipes shared by all, allowing any company to offer itself up as an access ISP, something which worked well in England and France. And as we’ve reported, this is exactly how Verizon itself described the fiber when it was getting tax breaks and right of way to put it into the ground.
Given the relative timidity of the FCC and the ISPs winning record in court, however, this seems unlikely to happen. Which means for the time being we’re left with a troubling state of affairs, one in which companies haggling over costs is threatening the performance of our most important communication and information utility. As M-Lab concluded in its report, "ISP interconnection has a substantial impact on consumer internet performance --sometimes a severely negative impact -- and that business relationships between ISPs, and not major technical problems, are at the root of the problems we observed."
That brings us back to the nuclear option: the FCC using Title II to declare our broadband internet infrastructure a common carrier. That would turn Comcast and Verizon’s infrastructure into dumb pipes shared by all, allowing any company to offer itself up as an access ISP, something which worked well in England and France. And as we’ve reported, this is exactly how Verizon itself described the fiber when it was getting tax breaks and right of way to put it into the ground.
Given the relative timidity of the FCC and the ISPs winning record in court, however, this seems unlikely to happen. Which means for the time being we’re left with a troubling state of affairs, one in which companies haggling over costs is threatening the performance of our most important communication and information utility. As M-Lab concluded in its report, "ISP interconnection has a substantial impact on consumer internet performance --sometimes a severely negative impact -- and that business relationships between ISPs, and not major technical problems, are at the root of the problems we observed."
© 2014 http://www.theverge.com/2014/10/31/7138449/m-lab-netflix-comcast-verizon-isp-business-dispute-congestion-traffic-interconnection
