Technology, Progress, and the Inevitability of the Surveillance State, Part 1
The recent kerfuffle over the Canadian government’s decision to wire for sound the Ottawa Airport, and begin recording travellers’ private conversations, has touched a nerve — the same nerve which was tweaked earlier this year by the government’s proposal to commence en masse surveillance of people’s Internet browsers, email, and cell phones. Canadians, it seems, are uncomfortable with the idea of the government engaging in massive surveillance, police state-style.
And I agree with them on this. But we also need to get serious about something else. On one point I agree with the Mark Zuckerbergs of the world: traditional ideas about privacy are obsolete. Knee-jerk reactionary attempts to preserve them are unlikely to succeed beyond the short term. We need to begin thinking very seriously about what privacy will mean in the 21st century. The longer we put off having this conversation, the less options we will have to do anything about it.
The problem, in essence, is that very shortly now we will become victims of our own technological progress: the surveillance state will become so easy and inexpensive to set up that resisting it will be fruitless. We have a limited and closing window in which we can set the rules in such a way that future surveillance programs are as palatable and acceptable as possible. We’re not there yet, but we are encountering plenty of warning signs. I want to use a couple of examples of those warning signs to show what I mean.
First, let’s begin with the Cold War-era Stasi of East Germany — widely regarded as one of the most effective secret policing organizations in modern history. The Stasi maintained files on an astonishing 6 million of the country’s subjects. It was a mammoth task. Even after allowing for pilfering by the Russians, the Americans, and anyone else who could get their hands on its precious paperwork, the Stasi’s archives still contain 110 kilometres of paper files and 1.4 million photographs. 20 years after the end of the Cold War, the archives still employ hundreds of personnel, at a cost of tens of millions of euros per year, simply to sift through all the old materials and preserve them for posterity.
Here’s the thing, though. 1.4 million photos, as JPEG files, would take up a few terabytes of hard disk space. A couple billion pages of text would take up another a few terabytes. Right now, you can buy hard disk storage at the rate of around $100 per terabyte — assuming you don’t get your hard disk on sale, in which case it might be half as much. Your PC wouldn’t be able to hold the entire Stasi archives, but I suspect most of my readers have enough disposable income that they could purchase the computer apparatus necessary to do so without going hungry or losing their house.
There are disturbing implications to this. Essentially, if the Canadian government wanted to build a database consisting of about 10 megabyets per citizen on average (meaning a couple pictures, a couple audio clips, and a few hundred pages worth of emails, Facebook posts, tweets, vital statistics, etc.), this task would require total hard disk storage space of around 300 terabytes. The cost of setting up this storage would be about $30,000.
Sound impressive? Not exactly. Facebook’s database is a couple of dozen petabytes — something over 20,000 terabytes. And IBM has secured an unidentified customer to buy a 120 petabyte hard drive — meaning, about 400 times the size of my hypothetical “Stasi Canada” database. At $100 per terabyte, that’s still only $12 million. The pricetag for the IBM system is presumably far larger, but then again, theirs will also come with a search system capable of sifting through all the data once collected.
This illustrates how far the surveillance state has already moved. There are only two likely possibilities with respect to IBM’s 120 petabyte drive. Either it is being ordered by the National Security Agency, suggesting that the American intelligence community a database proportionately equivalent to 40 times the size of my “Stasi Canada” example, or the NSA already has larger and more capable databases available for their use, which the public doesn’t know about. Right now the NSA is building a special computer data storage facility in Utah which encompasses 1 million square feet of floor space. I’m not exactly sure how many hard drives you can fit in a million-square-foot building, but I’ll bet it’s a lot.
Tweet
Edstock
So, you find the locations, and it’s Firesign Theater time . . . be creative, Mr. Tirebiter; if you said anything untrue to any airport employee, you’d be in jail shortly, but you could walk up to Mrs. Tirebiter after parking the car and complain about some jerks that were blocking the parking space unloading their van . . . “Besides, do they really think Air Canada will ship boxes of Kalashnikovs? — and that ammonia smell!”
ktron
It’s fun to ponder how much the Stasi operating budget contributed to the failure of the East German state.
You also have to wonder how effectively one can find anything of use in the ever growing quantities of data collected. Care to consider analogies/parallels to OCR effectiveness? A two percent failure rate in a search can be a pretty significant amount of information when you’re searching petabytes of data. Throw in a few lines of poor code, a lesser used heavy regional dialect, or a bad cold, and how effective can it all really be?
The costs will continually spiral upwards and they’ll either have to pare it back, shut it down, or invent successes to justify the expense (Gilliam’s Brazil anyone?)
Meanwhile the bad guys go back to chalk, pencils and paper – or will Big Bro outlaw those as a public security measure? And that’s not a rhetorical question: do we honestly think they really consider what constitutes a hazard – for fun let’s contrast nail clippers onboard planes against burner cell phones all over the place – which can seriously cause more of a risk to public safety? Doh – I guess it is a rhetorical question after all: these are authoritarian die-hards, their goal isn’t public safety, it’s public intimidation.
Sixth Estate
ktron — Given the scanty public details of what we know about, say, the NSA’s and CSE’s setup, I’d say at this point the intelligence community has been quite willing to collect far more information than they know what to do with in the hopes that they’ll come up with some sort of system for using it in the future.
Talking about “effectiveness” has to begin with the question of what it would be “effective” AT DOING. My point here was simply that the technical barriers to mass collection, storage, and processing of information are falling at an exponential rate. And given that, we’d be fairly foolish to assume that there won’t be attempts to take advantage of that.
P. D. Carswell
One comment, one question.
First, assume the worst, and you will be correct. We are brilliant at technology, and abject failures at morality and ethics. Therefore technological advances will always be used for ugly purposes as well as more benign ones.
I kept hearing about Canadians being recorded at airports, border crossings, etc. Not “travellers,” but “Canadians.” That signifies to me that the conversations to be recorded were of people leaving Canada, not entering. How does monitoring of Canadians exiting the country qualify as “protecting” Canada?
Sixth Estate
I assume this is a rhetorical question.
If pressed, I imagine the government may offer the explanation that the airport microphones are intended in part to pick up trafficking and smuggling activity.
P. D. Carswell
In that case, they should be focused (instead or equally) on inbound passengers – those trafficking and/or smuggling drugs, arms, etc. INTO Canada.
Also, terrorism was thrown out as an excuse.
Sixth Estate
Well of course it was. It’s related to transportation.
For all we know, they ARE focusing on both inbound and outbound passengers. Canadians enter Canada as well as leave it, and non-Canadians leave Canada as well as enter it.
e.a.foster
Toews saw he might not get his “listening in bill” passed so he is trying to do it in stages. People might not object so much if it was just those in airports, because they don’t go to airports, etc.
Yes, it might be time to have a conversation about privacy but facebook & such things are all voluntarily used & people know the risks to their privacy. However, many do not use facebook & other things because they wish to protect their privacy.
Constant surveillance does nothing much for security but does provide companies with a new way to make money & governments to watch the citizens of the country. Neither is a good think. The surveilance systems are just more electronic junk which harms the enviornment in its making & disposal.
Sixth Estate
e.a. — The government has learned that bills that create massive public uproar can usually be slipped through later. Our copyright bill, for instance, which just passed with a minimum of debate after years of controversy because it was being pushed through in parallel to the budget. I expect some token changes will be made and then we’ll see the e-snooping legislation back in Parliament later this year or maybe next year. And so on.
I don’t necessarily mean privacy with respect to Facebook, but more with respect to what we expect from the government in terms of surveillance. But Facebook offers an abject lesson for people who worry about privacy: it is possible to assemble an unprecedentedly large database of personal information relatively cheaply and, as long as you provide some social benefit in exchange, with the active assistance of the people you’re monitoring.
P. D. Carswell
“For all we know, they ARE focusing on both inbound and outbound passengers.”
I read (can’t find it, sorry!) that the US objected to directional microphones being pointed toward them, and refused recording devices within X distance of their border stations.
The whole thing is, as with many of the Con botch-ups, ridiculous, either in execution or explanation.
kootcoot
“The cost of setting up this storage would be about $30,000.” Cost of storage media for simple DB containing all Canadians.
There is a lot more involved in actually operating the surveillance state than the increasingly cheap hardware. A recent example being the ludicrous Long Gun Registry that supposedly cost $1 Billion, yet still law enforcement had no idea where thousands if not hundreds of thousand of guns might be ready to shoot at them.
I also agree with ktron, and have always been not very paranoid about surveillance, being confident that even years ago there was so much data available that the authorities couldn’t deal with it in a effective manner. I also imagine that as surveillance increases people will naturally devise ways to mislead or confound it, if they have any reason to do so.
Managing and exploiting data is a whole ‘nuther ball game than just having the ability to acquire it.
Fundies won’t mind surveillance since they know that God sees all anyway……………
Sixth Estate
Yes, there is more than mere storage. I have more to say on that in a future post, but I am aware of it, yes.
My point isn’t that we’ve already reached the point where effective total surveillance is possible. My point is that the technical barriers to that surveillance have fallen vastly in recent years, and, presumably, that this trend will continue at a similar pace into the future. Which is why it’s important to talk about it now, rather than later down the road. The cheaper and simpler it becomes to solve what used to be pressing problems, the more tempting it will be for the government to set aside some resources for those solutions.
I’m not terribly paranoid about it either, which is why surveillance and the security services haven’t really been a major focus of this blog over the past year. But I also spend a good deal of time speculating about the future based on current trends, and it is a fact that the sort of surveillance dreamed of by the Stasis and the Toewses of the world is becoming increasingly inexpensive.