Bogosity Alert! Bogosity Alert!
Wait till I tell you about who’s coming to sleepy good ol’ U.
N. of H.: the wares this man is peddling
are inherently suspect– and that’s the reason for the alert. On the other hand,
there’s a good chance he’ll blow your mind.
Bruce Bueno de Mesquita will lecture here on campus on
Wednesday, October 24. Bruce is a social scientist – a Professor at NYU and fellow at the
Stanford’s Hoover Institution - that has had the temerity to venture onto the
world of forecasting. The tool of his
trade is game theory – the mathematical modeling of interactions between
parties to a transaction. He has built a
game theory model that enables him to analyze and predict social outcomes – and a successful consulting practice
based on it.
In game theory the focus is (at the outset) on two key
factors. First, is the assumption that
players are rational, that is to say recognizing that what drives our decisions
is nothing more complicated than doing what is good for Number One. The second component is to recognize that
very few outcomes in our social world are entirely our own doing. By far when it comes to the things that
matter, results and outcomes depend on our actions and the actions of others.
The elementary game theory model used to explain how the elements
of this framework work together is the prisoners’ dilemma. And even if you are not familiar with the game
or its formal structure – it’s more than likely that you are familiar with the
logic. The prisoners’ dilemma and related game theoretic structures underscore the
plot in numerous sitcoms/movies. We find
it in the now classic War Games (the
one with Matthew Broderick) in which the computer ultimately discovers that the
only winning move is “not to play” and in Denzel and Gene Hackman’s Crimson Tide, in which a US nuclear
submarine is seemingly instructed to launch a pre-emptive strike on Russia but
is unable to obtain confirmation when the radio is destroyed. The strategic nature of the interaction emerges
starkly: Denzel: “In my humble opinion, in the nuclear world, the true enemy is
war itself.” But it’s not only in popular culture. We find instances of game theory applications
everywhere. Game theory models
underscore the ranking algorithms in Google searches, voting models such as the
popular Intrade, matching models like those in
Netflix, as well as the bidding models used in E-Bay and much more.
Why is being able to predict successfully such a big-deal in
the social sciences? Well, ‘cause successful
prediction is what separates the chaff from the wheat (and there’s very little
wheat, frankly), the ultimate, the triple-crown, the holy grail of it all.
Cause till now we do it badly, very badly.
Does anyone remember the Berlin Wall?; the Arab Spring; and dare I bring
it up, - the financial meltdown? Did
anyone see it coming? Big fat no: here’s
the latest (in what is now a veritable torrent) chest- thumping from economists and the great financial quant Emanuel Derman.
Models are abstractions – they circumscribe time and space,
limit the number of actors and actions and establish cause and effect. Events in the social sciences are
visualizations of an interesting slice (to us) of a complex adaptive system.
And when we intervene – be it a prediction or a policy prescription, the system
will change – or not: the point being that we don’t know what will happen to
it. Put differently, often the best we
can do – when it comes to model based predictions - is to make broad pattern
predictions or provide explanations of the system’s underlying principles. As a result, anyone venturing into the
forecasting business – especially trying to predict anything with any
meaningful detail – is more likely than not going to be wrong.
So – here’s the mind-blowing thing, the reason for the too-good-to-be-true
alerts. De Mesquita has been wildly successful at the prediction thing. In his words, “According to a declassified
CIA assessment, the predictions for which I’ve been responsible have a 90
percent accuracy rate.” His feats are documented in great detail in his book The Predictioneer’s Game (2009) where he
describes in rich, intricate detail the subtle combinations of all of his model’s
constituent elements and predictions in a simple, elegant manner. In fact, probably because he has been right so
many times he has become a bona fide academic superstar. Predictioneering success has brought him his own
TED TALK, appearances in many popular sites:
several instances in Big Think and a guest many times in my
favorite show: EconTalk with Russ Roberts.
The cool thing about De Mesquita’s game-theoretic forecasting
model is that it is portable. Its framework
is built with elements that can be replaced and substituted with the particular
narrative data necessary to illustrate cause and effect, drawn from the
different contexts in which the model is deployed – like a car: on the inside an
automobile is internal combustion; outside, it can become a Lexus or a Hummer,
red, automatic, two-seater, or blue. De
Mesquita ranges widely: he analyzes and predicts outcomes of wars, voting
outcomes, regime change, corporate lawsuits and even the Israeli-Palestinian
conflict.
Generally, when someone is hyping a prediction based on some
fancy model we probably only hear about the model’s successes. To appraise the
performance of any model one must also know how many times the model has been
wrong. De Mesquita acknowledges –and
rebuts this pitfall up front: “Of
course, it will be much more convincing if you go back over the book’s
predictions, as well as forecasts I have made online in speeches, podcasts and
so forth, and judge for yourself. There is always the danger that I will
unwittingly focus on the best of my predictions and give less credence to those
that were wrong, but I will certainly try not to do that; “and there you have it: he is so brazen, and so good at
what he does – that he is daring you to call him on it.
And we will. And I’m sure he flubs and will flub forecasts on
occasion. But you should expect that
error creeps in everywhere – and Bruce will not always get it right. Establishing how many times the model’s
prediction went awry is not the only criteria we must evaluate. And getting it wrong does not necessarily
mean the model is wrong. And herein is
the task for us, the audience, when you show up at Dodds Auditorium at 12:30 pm
next Wednesday, 24th.
To fully determine the quality of a model we need to know not
only when the model gets it right but most importantly, whether it was right either
by coincidence or because of the model’s capability. And, similarly, when it gets it wrong, we
should know whether it was either the luck of the draw or that the model
flawed. Alas, in many social science
events – the counterfactual – what would have happened but for what did happen-
is unknowable. So how are we to figure
this out?
The essence of modeling – as we have noted in previous blogs
– is to select salient elements of a situation and set forth their interplay in
a manner that conforms to understood principles of natural and social
behavior. Your task is to deploy
your knowledge of understood principles of natural and – especially – social
behavior to determine whether they impart sufficient discipline on the model’s
inner workings for us to ascertain the power of game theory modeling.
Be prepared to be wowed – especially when he gets it
right. And even if he gets it
wrong.
arod
arodriguez@newhaven.edu