Goodbye Privacy. Goodbye Open Internet?

The famous story of the pregnant teen outed to her father by Target epitomizes the power of big data and advanced psychometrics to wring potent conclusions from seemingly innocuous snippets of data. Andreas Weigend, the former chief data scientist at Amazon has written a must-read book that dives deep into this topic. And an article in The Observer shows how these learnings are being applied to influence voters:

“With this, a computer can actually do psychology, it can predict and potentially control human behaviour. It’s what the scientologists try to do but much more powerful. It’s how you brainwash someone. It’s incredibly dangerous.

“It’s no exaggeration to say that minds can be changed. Behaviour can be predicted and controlled. I find it incredibly scary. I really do. Because nobody has really followed through on the possible consequences of all this. People don’t know it’s happening to them. Their attitudes are being changed behind their backs.”

– Jonathan Rust, Director, Cambridge University Psychometric Centre.

So metadata about your internet behavior is valuable, and can be used against your interests. This data is abundant, and Target only has a drop in the ocean compared to Facebook, Google and a few others. Thanks to its multi-pronged approach (Search, AdSense, Analytics, DNS, Chrome and Android), Google has detailed numbers on everything you do on the internet, and because it reads all your (sent and received) Gmail it knows most of your private thoughts. Facebook has a similar scope of insight into your mind and motivations.

Ajit Pai, the new FCC chairman, was a commissioner when the privacy regulations that were repealed last week were instituted last fall. He wrote a dissent arguing that applying internet privacy rules only to ISPs (Internet Service Providers, companies like AT&T or Comcast) was not only unfair, but ineffective in protecting your online privacy, since the “edge providers” (companies like Google, Facebook, Amazon and Netflix) are not subject to FCC regulations (but the FTC instead), and they are currently far more zealous and successful at destroying your privacy than the ISPs:

“The era of Big Data is here. The volume and extent of personal data that edge providers collect on a daily basis is staggering… Nothing in these rules will stop edge providers from harvesting and monetizing your data, whether it’s the websites you visit or the YouTube videos you watch or the emails you send or the search terms you enter on any of your devices.”

True as this is, it would be naive to expect now-chairman Pai to replace the repealed privacy regulations with something consistent with his concluding sentiment in that dissent:

“After all, as everyone acknowledges, consumers have a uniform expectation of privacy. They shouldn’t have to be network engineers to understand who is collecting their data. And they shouldn’t need law degrees to determine whether their information is protected.”

So it’s not as though your online privacy was formerly protected and now it’s not. It just means the ISPs can now compete with Google and Facebook to sell details of your activity on the internet. There are still regulations in place to aggregate and anonymize the data, but experiments have shown anonymization to be surprisingly difficult.

If you don’t like playing the patsy, it’s possible to fight a rearguard action by using cookie-blockers, VPNs and encryption, but such measures look ever more Canute-like. Maybe those who tell us to abandon the illusion that there is such a thing as privacy are right.

So, after privacy, what’s the next thing you could lose?

Goodbye Open Internet?

Last week’s legislation was a baby-step towards what the big ISPs would like, which is to ‘own’ all the data that they pipe to you, charging content providers differentially for bandwidth, and filtering and modifying content (for example by inserting or substituting ads in web pages and emails). They are currently forbidden to do this by the FCC’s 2015 net neutrality regulations.

So the Net Neutrality controversy is back on the front burner. Net neutrality is a free-market issue, but not in the way that those opposed to it believe; the romantic notion of a past golden age of internet without government intrusion is hogwash. The consumer internet would never have happened without the common carrier regulations that allowed consumers to attach modems to their phone lines. AT&T fought tooth and nail against those regulations, wanting instead to control the data services themselves, along the lines of the British Prestel service. If AT&T had won that battle with the regulators, the internet would have remained an academic backwater. Not only was the internet founded under government sponsorship, but it owes its current vibrant and innovative character to strongly enforced government regulation:

“Without Part 68, users of the public switched network would not have been able to connect their computers and modems to the network, and it is likely that the Internet would have been unable to develop.”

For almost all US consumers internet access service choice is limited to a duopoly (telco and cableco). On the other hand internet content services participate in an open market teeming with competition (albeit with near-monopolies in their domains for Google and Facebook). This is thanks to the net neutrality regulations that bind the ISPs:

  • No Blocking: broadband providers may not block access to legal content, applications, services, or non-harmful devices.
  • No Throttling: broadband providers may not impair or degrade lawful Internet traffic on the basis of content, applications, services, or non-harmful devices.
  • No Paid Prioritization: broadband providers may not favor some lawful Internet traffic over other lawful traffic in exchange for consideration of any kind — in other words, no “fast lanes.” This rule also bans ISPs from prioritizing content and services of their affiliates.

If unregulated, ISPs will be compelled by structural incentives to do all these things and more, as explained by the FCC:

“Broadband providers function as gatekeepers for both their end user customers who access the Internet, and for edge providers attempting to reach the broadband provider’s end-user subscribers. Broadband providers (including mobile broadband providers) have the economic incentives and technical ability to engage in practices that pose a threat to Internet openness by harming other network providers, edge providers, and end users.”

It’s not a simple issue. ISPs must have robust revenues so they can afford to upgrade their networks; but freedom to prioritize, throttle and block isn’t the right solution. Without regulation, internet innovation suffers. Instead of an open market for internet startups, gatekeepers like AT&T and Comcast pick winners and losers.

Net neutrality simply means an open market for internet content. Let’s keep it that way!

FCC Title II Ruling

I got an email from the Heartland Institute today, purporting to give an expert opinion about today’s Net Neutrality ruling. The money quote reads: “The Internet is not broken, it is a vibrant, continually growing market that has thrived due to the lack of regulations that Title II will now infest upon it.”

This is wrong both on Internet history, and on the current state of broadband in the US.

It was the common carriage regulatory requirement on voice lines that first enabled the Internet to explode into the consumer world, by obliging the phone companies to allow consumers to hook up modems to their voice lines. It is the current unregulated environment in the US that has caused our Internet to become, if not broken, at least considerably worse than it is in many other countries:

America currently ranks thirty-first in the world in terms of average download speeds and forty-second in average uploads speeds, according to a recent study by Ookla Speedtest. Consumers pay much more for Internet access in the U.S. than in many other countries.

Net Neutrality, Congestion, DRM

Videos burn up a lot more bandwidth than written words, per hour of entertainment. The Encyclopedia Britannica is 0.3 GB in size, uncompressed. The movie Despicable Me is 1.2 GB, compressed. Consequently we should not be surprised that most Internet traffic is video traffic:

The main source of the video traffic is Netflix, followed by YouTube:

Internet Service Providers would like to double-dip, charging you for your Internet connection, and also charging Netflix (which already pays a different ISP for its connection) for delivering its content to you. And they do.

To motivate content providers like Netflix to pay extra, ISPs that don’t care about their subscribers could hold them to ransom, using network congestion to make Neflix movies look choppy, blocky and freezy until Neflix coughs up. And they do:


This example illustrates the motivation structure of the industry. Bandwidth demand is continuously growing. The two basic strategies an ISP can use to cope with the growth are either to increase capacity or to ration the existing bandwidth. The Internet core is sufficiently competitive that its capacity grows by leaps and bounds. The last mile to the consumer is far less competitive, so the ISP has little motivation to upgrade its equipment. It can simply prioritize packets from Netflix and whoever else is prepared to pay the toll, and let the rest drop undelivered.

One might expect customers to complain if this was happening in a widespread way. And they do:

Free market competition might be a better answer to this particular issue than regulation, except that this problem isn’t really amenable to competition; you need a physical connection (fiber ideally) for the next generation of awesome immersive Internet. Running a network pipe to the home is expensive, like running a gas pipe, or a water pipe, or a sewer, or an electricity supply cable, or a road; so like all of those instances, it is a natural monopoly. Natural monopolies work best when strongly regulated, and the proposed FCC Title II action on Net Neutrality is a good start.

Digital Rights Management

Unrelated but easily confused with Net Neutrality is the issue of copyright protection. The Stop Online Piracy Act, or SOPA, was defeated by popular outcry for being too expansive. The remedies proposed by SOPA were to take down websites hosting illegal content, and to oblige ISPs to block illegal content from their networks.

You might have noticed in the first graphic above, about 3% of what consumers consume (“Downstream”) online is “filesharing,” a.k.a music and video piracy. It is pretty much incontrovertible that the Internet has devastated the music business. One might debate whether it was piracy or iTunes that did it in, but either way the fact of Internet piracy gave Steve Jobs a lot of leverage in his negotiations with the music industry. What’s to prevent a similar disembowelment of the movie industry, when a consumer in Dallas can watch a movie like “Annie” for free in his home theater before it has even been released?

The studio that distributes the movie would like to make sure you pay for seeing it, and don’t get a pirated copy. I think so too. This is a perfectly reasonable position to take, and if the studio was also your ISP, it might feel justified in blocking suspicious content. In the US it is not unusual for the studio to be your ISP (for example if your ISP is Comcast and the movie is Despicable Me). In a non-net-neutral world an ISP could block content unilaterally. But Net Neutrality says that an ISP can’t discriminate between packets based on content or origin. So in a net-neutral world, an ISP would be obliged to deliver pirated content, even when one of its own corporate divisions was getting ripped off.

This dilemma is analogous to free speech. The civilized world recognizes that in order to be free ourselves, we have to put up with some repulsive speech from other people. The alternative is censorship: empowering some bureaucrat to silence people who say unacceptable things. Enlightened states don’t like to go there, because they don’t trust anybody to define what’s acceptable. Similarly, it would be tough to empower ISPs to suppress content in a non-arbitrary but still timely way, especially when the content is encrypted and the source is obfuscated. Opposing Net Neutrality on the grounds of copyright protection is using the wrong tool for the job. It would be much better to find an alternative solution to piracy.

Actually, maybe we have. The retail world has “shrinkage” of about 1.5%. The credit card industry remains massively profitable even while factoring in a provision for fraud at about 3% of customers compromised.

Total Existing Card Fraud Losses and Incidence Rate by Year. Source: Lexis/Nexis.

“Filesharing” at 3% of download volume seems manageable in that context, especially since it has trended down from 10% in 2011.

ALA Troubled by Court’s Net Neutrality Decision

My thoughts on network neutrality can be found here and some predictions contingent on its loss here, so obviously I am disheartened by this latest ruling. The top Google hit on this news is currently a good story at GigaOm, and further down Google’s hit list is a thoughtful article in Forbes, predicting this result, but coming to the wrong conclusion.

I am habitually skeptical of “slippery slope” arguments, where we are supposed to fear something that might happen, but hasn’t yet. So I sympathize with pro-ISP sentiments like that Forbes article in this regard. On the other hand, I view businesses as tending to be rational actors, maximizing their profits under the rules of the game. If the rules of the game incent the ISPs to move in a particular direction, they will tend to move in that direction. Because competition is so limited among broadband ISPs (for any home in America there are rarely more than two options, regardless of the actual number of ISPs in the nation), they are currently incented to ration their bandwidth rather than to invest in increasing it. This decision is a push in that same direction.

Arguably the Internet was born of Federal action that forced a corporation to do something it didn’t want to do: without the Carterfone decision, there would have been no modems in the US. Without modems, the Internet would never have gotten off the ground.

Arguments that government regulation could stifle the Internet miss the point that all business activity in the US is done under government rules of various kinds: without those rules competitive market capitalism could not work. So the debate is not over whether the government should ‘interfere,’ but over what kinds of interference the government should do, and with what motivations. I take the liberal view that a primary role of government is to protect citizens from exploitation by predators. I am an enthusiastic advocate of competitive-market capitalism too, where it can exist. The structure of capitalism pushes corporations to charge as much as possible and provide as little as possible for the money (‘maximize profit’). In a competitive market, the counter-force to this is competition: customers can get better, cheaper service elsewhere, or forgo service without harm. But because of the local lack of competition, broadband in the US is not a competitive market, so there is no counter-force. And since few would argue that you can live effectively in today’s US without access to the Internet, you can’t forgo service without harm.

The current rules of the broadband game in the US have moved us to a pathetically lagging position internationally so it seems reasonable to change them. Unfortunately this latest court decision changes them in the wrong direction, freeing ISPs to ration and charge more for connectivity rather than encouraging them to invest in bandwidth. If you agree that this is a bad thing, you can do some token venting here: http://act.freepress.net/sign/internet_FCC_court_decision2/

Here is a press release from an organization that few people could find fault with.

Net neutrality – Holland leads the way

Service providers can offer any product they wish. But consumers have certain expectations when a product is described as ‘Internet Service.’ So net neutrality regulations are similar to truth in advertising rules. The primary expectation that users have of an Internet Service Provider (ISP) is that it will deliver IP datagrams (packets) without snooping inside them and slowing them down, dropping them, or charging more for them based on what they contain.

The analogy with the postal service is obvious, and the expectation is similar. When Holland passed a net neutrality law last week, one of the bill’s co-authors, Labor MP Martijn van Dam, compared Dutch ISP KPN to “a postal worker who delivers a letter, looks to see what’s in it, and then claims he hasn’t read it.” This snooping was apparently what set off the furor that led to the legislation:

“At a presentation to investors in London on May 10, analysts questioned where KPN had obtained the rapid adoption figures for WhatsApp. A midlevel KPN executive explained that the operator had deployed analytical software which uses a technology called deep packet inspection to scrutinize the communication habits of individual users. The disclosure, widely reported in the Dutch news media, set off an uproar that fueled the legislative drive, which in less than two months culminated in lawmakers adopting the Continent’s first net neutrality measures with real teeth. New York Times

Taking the analogy with the postal service a little further: the postal service charges by volume. The ISP industry behaves similarly, with tiered rates depending on bandwidth. Net neutrality advocates don’t object to this.

The postal service also charges by quality of service, like delivery within a certain time, and guaranteed delivery. ISPs don’t offer this service to consumers, though it is one that subscribers would probably pay for if applied voluntarily and transparently. For example, suppose I wish to subscribe to 10 megabits per second of Internet connectivity, I might be willing to pay a premium for a guaranteed minimum delay on UDP packets. The ISP could then add value for me by prioritizing UDP packets over TCP when my bandwidth demand exceeded 10 megabits per second. Is looking at the protocol header snooping inside the packets? Kind of, because the TCP or UDP header is inside the IP packet, but on the other hand, it might be like looking at a piece of mail to see if it is marked Priority or bulk rate.

A subscriber may even be interested in paying an ISP for services based on deep packet inspection. In a recent conversation, an executive at a major wireless carrier likened net neutrality to pollution. I am not sure what he meant by this, but he may have been thinking of spam-like traffic that nobody wants, but that neutrality regulations might force a service provider to carry. I use Gmail as my email service, and I am grateful for the Gmail spam filter, which works quite well. If a service provider were to use deep packet inspection to implement malicious-site blocking (like phishing site blocking or unintentional download blocking) or parental controls, I would consider this a service worth paying for, since the PC-based capabilities in this category are too easily circumvented by inexperienced users.

Notice that all these suggestions are for voluntary services. When a company opts to impose a product on a customer when the customer prefers an alternative one, the customer is justifiably irked.

What provoked KPN to start blocking WhatsApp, was that KPN subscribers were abandoning KPN’s SMS service in favor of WhatsApp. This caused a revenue drop. Similarly, as VoIP services like Skype grow, voice revenues for service providers will drop, and service providers will be motivated to block or impair the performance of those competing services.

The dumb-pipe nature of IP has enabled the explosion of innovation in services and products that we see on the Internet. Unfortunately for the big telcos and cable companies, many of these innovations disrupt their other service offerings. Internet technology enables third parties to compete with legacy cash cows like voice, SMS and TV. The ISP’s rational response is to do whatever is in its power to protect those cash cows. Without network neutrality regulations, the ISPs are duty-bound to their investors to protect the profitability of their other product lines by blocking the competitors on their Internet service, just as KPN did. Net neutrality regulation is designed to prevent such anti-competitive behavior. A neutral net obliges ISPs to allow competition on their access links.

So which is the free-market approach? Allowing network owners to do whatever they want on their networks and block any traffic they don’t like, or ensuring that the Internet is a level playing field where entities with the power to block third parties are prevented from doing so? The former is the free market of commerce, the latter is the free market of ideas. In this case they are in opposition to each other.

Net Neutrality Fallout

Stacey Higginbotham posted an analysis of the FCC Net Neutrality report and order on GigaOM. She concludes:

As a consumer, it’s depressing, …it leaves the mobile field open for the creation of walled gardens and incentivizes the creation of application-specific devices.

Sure enough, just two weeks after the publication of the R&O, Ryan Kim reports on GigaOM that MetroPCS announced on January 3rd plans to charge extra based on what you access, rather than on the quantity or quality of the bandwidth you consume.

Net Neutrality and consumer benefit

A story in Wired dated December 17th reports on a webinar presented by Allot Communications and Openet.

A slide from the webinar shows how network operators could charge by the type of content being transported rather than by bandwidth:

DPI integrated into Policy Control & Charging

In an earlier post I said that strict net neutrality is appropriate for wired broadband connections, but that for wireless connections the bandwidth is so constrained that the network operators must be able to ration bandwidth in some way. The suggestion of differential charging for bandwidth by content goes way beyond mere rationing. The reason this is egregious is that the bandwidth costs the same to the wireless service provider regardless of what is carried on it. Consumers don’t want to buy content from Internet service providers, they want to buy connectivity – access to the Internet.

In cases where a carrier can legitimately claim to add value it would make sense to let them charge more. For example, real-time communications demands traffic prioritization and tighter timing constraints than other content. Consumers may be willing to pay a little bit more for the better sounding calls resulting from this.

But this should be the consumer’s choice. Allowing mandatory charging for what is currently available free on the Internet would mean the death of the mobile Internet, and its replacement with something like interactive IP-based cable TV service. The Internet is currently a free market where the best and best marketed products win. Per-content charging would close this down, replacing it with an environment where product managers at carriers would decide who is going to be the next Facebook or Google, kind of like AOL or Compuserve before the Internet. The lesson of the Internet is that a dumb network connecting content creators with content consumers leads to massive innovation and value creation. The lesson of the PSTN is that an “intelligent network,” where network operators control the content, leads to decades of stagnation.

In a really free market, producers get paid for adding value. Since charging per content by carriers doesn’t add value, but merely diverts revenue from content producers to the carriers, it would be impossible in a free market. If a wireless carrier successfully attempted this, it would indicate that wireless Internet access is not a free market, but something more like a monopoly or cartel which should be regulated for the public good.

Dumb mobile pipes

An interesting story from Bloomberg says that Ericsson is contemplating owning a wireless network infrastructure. Ericsson is already one of the top 5 mobile network operators worldwide, but it doesn’t own any of the networks it manages – it is simply a supplier of outsourced network management services.

The idea here is that Ericsson will own and manage its own network, and wholesale the services on it to MVNOs. If this plan goes through, and if Ericsson is able to stick to the wholesale model and not try to deliver services direct to consumers, it will be huge for wireless network neutrality. It is a truly disruptive development, in that it could lower barriers to entry for mobile service providers, and open up the wireless market to innovation at the service level.

[update] Upon reflection, I think this interpretation of Ericsson’s intent is over-enthusiastic. The problem is spectrum. Ericsson can’t market this to MVNOs without spectrum. So a more likely interpretation of Ericsson’s proposal is that it will pay for infrastructure, then sell capacity and network management services to spectrum-owning mobile network operators. Not a dumb pipes play at all. It is extremely unlikely that Ericsson will buy spectrum for this, though there are precedents for equipment manufacturers buying spectrum – Qualcomm and Intel have both done so.

[update 2] With the advent of white spaces, Ericsson would not need to own spectrum to offer a wholesale service from its wireless infrastructure. The incremental cost of provisioning white spaces on a cellular base station would be relatively modest.