Data Blizzard

Over the past few weeks or so I've enjoyed reading David Brin's essay-in-parts on modernism but I've read enough a Brin to know where he was headed most of the time. For me, nothing Earth shattering. However, there was a anonymous comment that has had me thinking for a while. I'm not quite sure what to make of it, but I think it's an important idea. Here's the post in whole:

In part 10 you imply that what you call modernism should be judged on the "pragmatic success-failure ratio" of various endeavors. Presumably you believe that ratio is greater than 1, and that any honest observer will come to the same conclusion.

The problem is that many honest observers won't, and here's why, via an analogy. There's a paradox in mathematics where it can be shown that the number of even numbers is "the same" as the number of numbers in general. The problem is that both sets are infinite. For every number you can pair up a corresponding even number, and vice versa, so there's justification in saying there are as many of each kind, even though one is "obviously" larger than the other.

A similar principle applies when it comes to evaluating data points in support of political positions today. For any situation where there is some disagreement, both sides will have an effectively infinite number of facts to support their position, due to the fact that a vast amount of raw data is collected about everything these days. Anything one side comes up with to support its position can be match on the other side with something to support its position.

So now it's not simply a matter of "my side has more facts to support its position than yours". Both sides have "the same" nearly-infinite number of facts supporting their positions. Your view of the world then comes down not to some kind of objective comparison, but by how things *feel* to you personally.

So when you say "obviously, the modernist program has had more successes than failures", someone who feels the opposite can say "that's not so obvious" and counter every success you cite with a failure.

(The environment created by this blizzard of data is what allows modern spin-based politics to exist, and figuring out how to deal with it will be necessary if we think the future of the country shouldn't necessarily be directed by whoever has the best public-relations apparatus.)

This is certainly something I've thought of before, but never at quite this level. It's an awfully nihilistic view of today's information overload. I'm not positive this is relevant in the Real World, but in the blogosphere and perhaps the media and public arena at large I this may be dead-on correct. This certainly is an idea to mull over.

Almost random, quasi-related thought: I once drew a stair with four steps, Data, Information, Knowledge and Wisdom. Each step required additional intelligence and, ironically, fewer datums. Sometimes we confuse the first step for the last.