One of the best illustrations of the arguments for and against basic research funding comes from a fascinating series of hierarchical maps of arguments about whether computers can think. The second map discusses the validity of the (in)famous Turing test. One page of this map concerns the Loebner Prize, a rather contentious modern-day variant of the Turing test woorht $100,000 to the first program (or perhaps only its programmers) that passes.
In the lower right of the page, several panels describe an argument between Hugh Loebner and Stuart Shieber over the validity and utility of the Lobnner Prize. Shieber argues (among other things) that the test is too far beyond the reach of current technology, and as an anology describes a fictional 15th century contest for human-powered flight. (All emphasis is in the original.)
The da Vinci prize, we shall imagine, is constituted in 1492 and is to be awarded to the highest human-powered flight. Like the Loebner prize, a competition is held every year and a prize must be awarded each time it is held. The first da Vinci competition is won by a clever fellow with big springs on his shoes. Since the next competition is only one year away (no time to invent the airfoil), the optimal strategy is universally observed by potential contestants to involve building a bigger pair of springs. Twenty-five years later, the head of the prize committee announces that little progress has been made in human-powered flight since the first round of the prize as everyone is still manufacturing springs.Of course, a lot of progress had been made in human-powered flight in those twenty-five years. Da Vinci himself was studying human physiology and anatomy and the flight of birds, and — although his own work directly on the topic of human-powered flight, ornithopter design, was essentially meritless beyond its decorative qualities — the apparently tangential work was, in the long run, pertinent to the technologies that would eventually enable the Gossamer Condor to be constructed. [...] However, over that period, and indeed at every point during the following four centuries, the kind of progress that needed to get made to solve the problem was not directly observable at that time in incremental improvement in solutions to the problem, the kind of improvement that might be observable in an annual contest. Nonetheless, tremendous scientific progress was made between the fifteenth and twentieth centuries. The Gossamer Condor and the digital computer are two outgrowths of this progress.
Loebner responds that even if such a contest did not achieve its stated goal, it might still have unforseen and useful consequences.
When Mozart rode to Vienna in 1781 he wrote complaining of the pain the mail coach inflicted on his backside. [...] This was a result, I must suppose, in part from poor suspension of the coach. The study of elasticity, stress, and strain did not result in a swift and straight arrival at understanding. Suppose a concerted effort had been made, early on, to fly using springs. Perhaps the concepts of stress and strain would have been invented sooner, along with advances in spring technology that would have been a boon to humanity, and Mozart's buttocks. There is probably still room for improvements in springs. [...] Research would be a boring, indeed, if every effort resulted in answers only to the question or problem intended. Perhaps my prize will not lead down the straightest path to AI. It will prove useful, nonetheless, perhaps in very unexpected ways.
Predictably, and correctly, Shieber shoots back [pdf]:
[Loebner] believes that the competition might have valuable unexpected spinoffs. [Footnote:] For the record, by the way, Dr. Loebner could have chosen a better example in this vein than the story about Mozart's fundament. The great composer's discomfort did not owe so much to the world's inadequate understanding of stress and strain, but his decision to travel by hard-seat mail coach, rather than by a more expensive class of transportation. Within a few miles of setting out for Vienna, he “upgraded.” [Footnote ends.] Whether this use of funds would be more apt to generate useful spinoffs than some other use (which is the crucial issue) seems highly dubious. Again, the history of the competition itself tells the story.
Given the scorn that large sections of the AI community pour on the Loebner Prize, one has to wonder why Shieber (or Minsky, or any of the prize's other detractors) felt it necessary to argue against it. Why not just let Loebner do what he wants with his own money? The beginning of Shieber's rebuttal reveals his motivations.
How Dr. Loebner spends his money in his private life is his own business. But when he spends it in a public way, when he leverages it with the scarce resources of public funding agencies like the National Science Foundation and private foundations like the Sloan Foundation and the Computer Museum, when he solicits industrial co-sponsors like IBM, Digital Equipment Corporation, and GDE Systems, when he calls in members of he national press as onlookers (and in the most recent third Loebner competition held on Decemberr 8, 1993, as the actual judges), it becomes everyone's business. With our special expertise, computer scientists have both the right and the responsibliity to make our opinions known on matters that affect the research environment.
This exchange displays several hallmarks of the debate over basic research funding.
- The debate is not primarily about intrinsic worth but about resource allocation. The question is not whether basic research is worth funding in principle, but rather, whether basic research should be funded at the expense of something else. Bad science (and bad science policy) attracts the most criticism when it pulls scarce resources—research funding, students, public interest—away from good science (and good science policy). As Kissinger put it, the fight is fiercest when the stakes are small.
- As Loebner suggests, idealistic research programs can have significant, far-reaching, and totally unexpected consequences, even when they have no hope whatsoever of reaching their stated goals. Of course, most fundamental research ideas do not have far-reaching consequences. In this respect, basic research is a gamble, but one with the potential for enormous payoffs. Witness, for example, the important results on interactive proofs, hardness of approximation, and cryptography that have emerged from work on the P=NP question (for which we not only have no solution, but no idea how to begin looking for a solution [pdf]), or the many fields of mathematics (Diophantine analysis, algebraic geometry, number theory) that emerged from trying to prove Fermat's utterly useless Last Theorem.
- On the other hand, cushioning Mozart's ass is not the most compelling motivation for a research program. To sell high-risk research, one needs a story that speaks to the intended audience, whether that audience is a granting agency, a Congressional committee, a university administration, or the general public. If we basic scientists want people to gamble on us, we must convince them that their expected long-term payoff, however they measure payoff—knowledge, profit, beauty, prestige, security, power, quality of life, whatever—is higher if they support us than if they ignore us.
- Failure to live up to past promises undermines even the most cogent argument about future results. Over-enthusiastic claims about impact may be convincing the the short run, but if the impact doesn't materialize, those claims will eventually backfire.
- Finally, as Shieber says, scientists have both the right and the responsibility to make our opinions known on matters that affect our research environment.
Recent Comments