[Cross-posted at Digital Fourth]
Drowning in data, starved for wisdom
The NSA has just vigorously denied that their new Utah Data Center, intended for storing and processing intelligence data, will be used to spy on US citizens. The center will have a capacity of at least one yottabyte, and will provide employment for 100-200 people. With the most generous assumptions [200 employees, all employed only on reviewing the data 24/7, only one yottabyte of data, ten years to collect the yottabyte, 5GB per two-hour movie], each employee would be responsible on average for reviewing 4500 billion terabytes, or approximately 23 million years’ worth of Blu-ray quality movies, every year.
Must…keep…watching…my…country…needs…me
This astounding and continually increasing mismatch shows that we are well beyond the point where law enforcement is able to have a human review a manageable amount of the data in its possession potentially relating to terrorist threats. Computer processing power doubles every two years, but law enforcement employment is rising at a rate of about 7% every ten years, and nobody’s going to pay for it to double every two years instead. Purely machine-based review inevitably carries with it a far higher probability that important things will be missed, even if we were to suppose that the data was entirely accurate to begin with – which it certainly is not.
So none of us should be surprised that the FBI and the other agencies who have claimed the ability to prevent terrorist attacks, failed to prevent the Boston Marathon bombings. With this ocean of data, no matter how much artificial intelligence you use, your chances are very remote of being able to catch a threat ahead of time. Of course, after an attack, and especially once you have a suspect, you can go back and do a search on all the data points in the ocean that you almost inevitably missed. Why was Tamerlan Tsarnaev, the elder of the Boston Marathon bombing suspects and one of around 750,000 people in the TIDE database, was not stopped at the border? Why was facial recognition software not able to flag him as a match for a suspect? Why do the fusion centers, intended to synthesize data into actionable “suspicious activity reports”, flag things too late for them to be of any use? Why is the Air Force panicking a little at not having enough people to process the data provided by our drone fleet? Because there’s not only too much data, but much too much too much.
It’s in this context, then, that we should understand the calls for more surveillance after the Boston Marathon attacks for what they are. More cameras, more surveillance drones and more wiretapping, without many more humans to process the data, will make this problem worse, not better. These calls are being driven not by a realistic assessment that surveillance will help prevent the next attack, but by the internal incentives of the players in this market. Neither the drone manufacturers, nor law enforcement, nor elected officials, have an interest in being the ones to call a halt. So instead they’re promoting automation – automated drones, automated surveillance, and email scanning software techniques – which inevitably has a much higher error rate than review by humans.
It’s much easier to claim you need more data, than to solve the mismatch problem. The real solution is much harder politically. In truth, we don’t need a terrorism database with 750,000 names on it. There are not 750,000 people out there who pose any sort of realistic threat to America. If the “terrorism watch list” were limited by law to a thousand records, then law enforcement would have to focus only on the thousand most serious threats. Given the real and likely manpower of the federal government, and the rarity of actual terrorism, that’s more than enough. If law enforcement used the power of the Fourth Amendment, instead of trying to find ways round it, it could focus more on the highest-probability threats.
If law enforcement leveled with the public about their increasingly limited ability to thwart attacks ahead of time, instead of selling us a bill of goods to make us feel safe, they would catch less heat when they fail. They’re going to miss stuff. That’s inevitable under both a tight and a loose system. But a tight system has the added advantages that it protects more people’s liberties, and costs a lot less.
Before we invest in even more surveillance cameras in public places, I suggest that we demand that the live video feeds be available to everyone via the net. I suggest that the problem we face is less with the video (we can’t stop it anyway) and more with the privileged access we grant police and military authorities.
Because we are a republic, we explicitly delegate to government power that by construction originates in ourselves. The military and police receive their power and authority because we delegate it to them. The quite defensible posture is that we delegate to government things we cannot do by ourselves.
I suggest that public surveillance, in the context of the net, is an area where the public can both improve on the efforts of police and the military, and can provide valuable oversight while doing so.
Of all the thing Orwell got right about “1984”, he missed the net altogether. Public access to the same information received by government authorities changes everything.
It seems that Orwell missed the part that it won’t be the government controlling the Big Brother cameras, it will be we the citizens. The idea of the US moving towards the Big Brother State, where the government posts surveillance cameras on every street corner like London, is quickly being made irrelevant by the advent of cheap, decent quality personal video recording devices, especially ones in mobile devices in cell phones.
It’s really only a matter of time until everyone is equipped with Google Glass-type technology and cheap, drug store sunglasses will have HD video recording capability that is beamed directly to the cloud where that video data will be accessible from anywhere and impossible to destroy through the confiscation of the recording device.
It’s not the Government that will be the Big Brother State, it will be you and I and our fellow citizens. The Government will have no need to invest heavily in surveillance cameras when it is inevitable that almost every citizen will be walking around video recording everything. (Look at the proliferation of Russian dash cameras.)
I fully expect “whole life” recording to become a thing, as soon as the data capacity expands to the point of making it possible, which is no more than five years or so away.
But somervilletom is right, that the key is not whether the information exists – it’s going to exist whatever – but who gets to control it. Governments naturally want perfect secrecy for themselves and perfect transparency for the people. We may well have no need any more for government cameras, but the government should be sharply restricted in how it can use the data it collects to investigate, imprison or kill people.
If the citizens are recording everything and immediately uploading it all to the cloud or YouTube/Facebook-type platforms, will anyone be able to control the video data given the shear volume of data we may potentially see when recording technology becomes cheaper and more ubiquitous?
Is asking “who will control it” even the right question?
We see now that things that go on the web can’t be controlled, there’s no reason to believe that video and audio will be any different.
While it’s true that genuinely draconian steps might be attempted by totalitarian regimes (if that ever happens here), it is also true that various techniques exist to circumvent those efforts. Short of denying access to computing and communication technology altogether, I think that the genie is already out of the bottle.
In my view, this is a good thing. We are already discovering that we have to rethink the conceptual foundations of things like “privacy”, however.
I think that when it comes to government surveillance, the “who will guard the guards” question is ultimately the most important.
I would be concerned about amateurs not correctly interpreting what they see, or they start to gossip and blog about what they think they see. I’d much rather leave this to trained professionals.
Crowdsourced photography and crowdsourced detection are going to very rapidly become common features of our society. What we’re seeing now, with Reddit’s attempt to identify suspects and Anonymous’s investigations of suspected rapists, is just the beginning.
There is no need to exclude the “trained professionals”.
My point is that I have at least as much faith in the collective perception of a few thousand net viewers as I do in the interpretation of a handful of “trained professionals” — especially in situations where the facts of the video may not necessarily reflect well on the authorities involved.
As a micro-example, I suspect a stoplight cam at, say, Harvard Street in Brookline Village or the big intersection at Coolidge Corner will show more cops in official vehicles parking in crosswalks and in front of hydrants while they drop into DDs for their Joe-And-Donut then red-light violators. I’m only half-joking.
When the amateurs are incorrect, I’m sure the professionals will make their case.