Well, you’ll answer yes (of course) and if you live in South Africa that’s a justifiable response. But here the reason for the sky-high prices, and third-world bandwidth is Telkom. And ICASA, the toothless regulator.

But that’s not what I’m talking about here. I’m talking about a practice that to me seems a complete rip-off. I have to stop myself from using the word “fraudulent” but …. (grits teeth) ok, I’ll refrain from that.

What is the practice, I hear you ask? It’s simple – and it’s technical. ISPs regularly operate caching servers (of which the most popular is the Squid proxy server). Caching servers are often a Good Thing.

In your house, or small office, one bored individual watches the Death Star Canteen youtube video. She likes it. She sends it on to her colleague. He likes it. He sends it on to his. If you are serving your web traffic through Squid, or some such proxy, the 1st user downloads the video, the proxy caches it, and users 2 and 3 get served the cached content. You’ve eaten up only as much of your bandwidth as it took to serve the 1st video (and users 2 and 3 are impressed with the lightning fast download!).

Sounds like Utopia, right? Well, yes – if you run the proxy. You manage it. You chose to install it.

But what if your ISP runs a caching server for you? Without telling you? I suppose you think, well, that’s even better – I benefit from even more pioneer surfers discovering content before me. But think of this – the ISP is storing this stuff locally. You’re being charged at an international rate. And it’s costing them (almost) nothing to serve it to you. Does that sound right to you?

Another point – do you WANT cached content all the time? Sometimes you want the page to be fresh.

We recently found that we couldn’t hit search.twitter.com. As one of our products is feedalizr, with a strong tie-in to twitter and twitter’s search API, this was problematic for us. We found ways to circumvent that problem, but the backbone provider took weeks (effectively) to explain to us what the problem was. It turns out that twitter’s search page disallows requests from web caches. So we were prevented from being as productive as we could have been, because the provider wanted to cut their costs by providing us a reduced service. Without informing us that that’s what they were doing.

And that sucks, frankly.