Welcome Comcast USA users to the club of Internet blocking. You now share a bond with millions of Internet users in China! It may seem annoying, but with the right tools and some perseverance, you too can keep downloading without any hiccups.
The story
It seems the US Internet service provider has been using Great Firewall-style tactics to prevent customers from running P2P protocols like BitTorrent. Some sleuthing by the EFF found that TCP reset packets (RST) are sent to kill connections related to P2P file transfers by Comcast customers. This clandestine connection sniping is pretty hard to diagnose without geeky tools like Wireshark or ethereal, but the shutdown technique is used by more and more ISPs. It’s what the Great Firewall here in China depends on for blocks triggered by keywords.
This revelation comes at a particularly bad time for ISPs in the US, when the network neutrality debate had died down. But this will re-energize the Internet purists, as it directly hurts the credibility of ISPs who say the US does not need regulation of “neutrality.” If Comcast had given fair notice to customers via service agreements about proper and improper use of their connections, that would be one thing. But users had their IP connections shut down mysteriously for unstated reasons. That’s something that usually happens in other places. Like China.
There is a solution
While there is widespread piracy over P2P networks, there are absolutely legitimate uses for them. Comcast seems to have classified any BT P2P file transfers as something that should be shuttered for copyright infringement. That would be a bad assumption.
The other day I downloaded NeoOffice (open source) for the Mac at 140 Mbytes using BitTorrent because it’s much faster than FTP. I was able to get 120 kilobytes/second on P2P versus 15 kilobytes/second via straight download. Many folks download Linux distributions and operating system patches via BT for exactly this reason.
Is there a solution for customers? Well EFF is considering a legal challenge, as this seems ripe for a class action lawsuit. In the meantime, there are ways to circumvent RST-based tactics of firms like Comcast.
Here, China Netcom also frowns on P2P by slamming shut transfers and tracker communication. A combination of techniques, like using BT clients supporting encrypted connections, can solve the problem. The following is what works for me, and it should work for most nearly anyone that has to deal with a firewall/fitlering system with BT.
BACKGROUND: There are two different “channels” that BitTorrent uses — tracker communication and peer communication. Tracker communication is basically what the BT client needs to connect to a tracker server, which has the particulars of the transfer: what file is being transferred, which peers have it, and the progress of the client. It’s basically the coordination center for the entire session and is the only real vulnerable hub of a P2P system, becoming a single point of failure/blocking. The other part is peer communication. This is what takes place between your computer and the multitude of other computers on the Internet. This makes up the big bulk of traffic on P2P, when your computer is perhaps chatting with 100+ other clients to transfer little chunks of the file you want.
So the tactic of ISPs is to block either or both of these types of communication. In days of old, when BitTorrent was new (or ISPs didn’t care or notice) all peer communication happened on port 6881 and tracker communication happened on 6969. For a long time this worked fine. But since these port numbers are well known, to block BT the ISP could simply block all packets to those ports. Game over for the client.
So people started changing port numbers to high numbered random values (37412 for example) used for peer communication and to less known port number for tracker communication. That worked for a while. But in this escalating game of cat and mouse, ISPs started putting in systems to actually inspect packets across all ports to see if they had telltale BitTorrent “headers,” and shutting down those connections. Thus high numbered, randomly selected ports were not good enough. The power swung back to the ISPs.
SOLUTION. What’s fascinating is the furious software arms race the P2P open source community engaged in to solve this problem. Programmers have upped the ante by using encryption and de-centralizing the tracker function to the point where BT is now nearly unblockable. But it’s not for the average user, since you do need some special configuration with the right clients.
The basic solution is to use encrypted peer communication, and a proxy server for the tracker communication.
Newer clients like uTorrent (Windows) and Azureus (nearly every platform), now support encrypting all traffic between peers using RC4 encryption, and setting an arbitrary port number. The only thing ISPs see then are IP packets with encrypted gibberish going from one random port number to another computer’s random port number. They cannot tell whether it is VoIP traffic, a file transfer, VPN, MMORPG data, or anything else. It is completely opaque to them, and filtering cannot work on the packets. Because the two peers do a handshake to establish a unique session key that no one else knows, the ISP is out of luck.
The RC4 encryption used by clients, while not the state of the art, is hard enough to crack that it isn’t practical to inspect those packets without major horsepower (like supercomputer horsepower). Comcast, China Netcom, or anyone else as intermediary ISP have no real options but to pass it along as an ordinary IP packet.
Tracker communication needs a different treatment. It’s much easier for ISPs to block this, because there are only a few dozen popular BitTorrent trackers in the world. By simply blocking all traffic to them, or watching each packet, they can just shut down those connections. The simplest way to circumvent this is to use a proxy. Azureus supports the use of a SOCKS proxy server. As a China Internet user, I always have an SSH tunnel open and in use for my proxy communication. It’s just a normal part of my day to get to blocked sites like BBC, Blogger, YouTube, etc.
However, SSH is not something everyone has. I happen to have it as part of a hosting plan, but it’s fairly easy to get one as part of a $5.95/month plan like on BlueHost. There are also sites that give free SSH accounts, like silenceisdefeat.org. In the Azureus options, you can simply instruct the client to use a proxy for tracker communication. That way, the ISP you are using cannot even tell any P2P is happening since your proxy server is doing all the tracker communication on your behalf, and it’s encrypted in the SSH tunnel. (There is a full tutorial about this technique for Windows here and here).
With this whole thing setup — high numbered random ports, encrypted peer communication and proxy tracker communication — your local ISP is none the wiser to what’s going on, even when employing basic surveillance techniques like packet inspection. I’ve been able to max-out my connection speed using this arrangement for torrents that have lots of peers. There are some small caveats — not all clients support RC4 encryption, so not all the seeds/leechers listed will be available to you. Also, if your SSH connection breaks off for some reason, it will likely stall your transfer. (I use a command line tool like “autossh” to keep a persistent SSH connection.)
As I warned though, this is not for the average person. The most exotic part of the solution is an SSH tunnel, which only real hard-core Internet users would have.
The final tally
What this arms race means in the long run is more interesting. If the US government will not regulate the maintenance of “neutrality” into the operation of ISPs, users can demand it in part by encrypting everything and preventing operators from discriminating against (or currying favor towards) certain types of traffic.
This has always been the problem with the perennial hope of ISP-supported Quality of Service (QoS) because it depends on the operator being a relatively fair or accommodating intermediary. This assumes there is a telco/ISP being purely a “common carrier” whose job is to expeditiously relay traffic efficiently and for the benefit of the customer. The problem is one side of the connection is the ISP’s customer, the other end is usually not. Also, more and more ISPs have a vested interested in pushing their own VoD, VoIP or walled garden product over the exact same lines that Google, Facebook and Joost are using for their multibillion dollar ambitions.
It is natural, though problematic, for a “common carrier” to mix its own product’s fortunes into its relaying policy. And that’s where the heart of the debate is.
Congratulations to the Chinese Wikipedia, which just hit 150,000 articles, despite being blocked in China.
