While I agree with the advocates of the “transparent society”, I don’t think they appreciate for the most part how radical that transparency is going to have to be, or how fundamentally our society is going to have to change because of it.

The problem here is not just that you can’t put the surveillance genie back in the bottle, but that both the things that enable surveillance only get cheaper and more widely available with time. and that intentionally weak cryptographic systems can be exploited by everyone. If the surveillance state grows — even if we are able to extract equivalent levels of transparency and accountability — then eventually we’re going to run into a situation where it’s trivially east to spy on everyone, both because the technology is cheap and ubiquitous, and also because the systems that would act as a counterbalance will have been effectively neutered.

I’m skeptical of the law’s ability to constrain behavior at that point (law moves slowly enough that it already plays the role of court jester to increasingly dislocating technological innovations). This doesn’t just mean that everyone’s going to get 15 minutes of amateur porn-star fame. It means that it’s going to be essentially impossible to guarantee the integrity of our communications — the emails you write, the phone calls you make, and the bank accounts you put your money into. All are secured by the same technologies, which means all are gong to become (practically) wide-open.

You might try to prevent that with massive, totalitarian-style surveillance, but that just creates a bigger, juicier target. I can envision people adapting to this by adopting radically open and accepting cultural norms (though I worry about the homogenizing nature of such global “total information awareness,” both due to outright coercion and the inescapable cacophony of the dominant culture), but I can’t envision any economic system built around tallying up one’s “contributions” (i.e., anything that involves any form of currency) surviving in a world where the records of such transactions are so totally vulnerable to being rewritten.

That’s a far weirder, more disturbing, and more perilous world than Dr. Brin & co. seem willing to consider.