Whom should the CIA hire? With the United States engaged in a war in which intelligence is the critical front, finding the best and brightest and putting them in charge of counterterrorism and related black arts is an essential task.

The good news is that the CIA is being flooded with applicants at the staggering rate of 10,000 a month. The bad news is that the process of sifting and screening these aspiring spies remains distorted.

One problem is an affirmative-action program that seeks to replicate the ethnic balance in the United States rather than focus singlemindedly on hiring men and women steeped in knowledge of our adversaries. Another is a security-screening program that remains ferociously suspicious of applicants with foreign roots. A third is an organizational culture that in some measure remains, despite radical changes introduced after September 11, risk averse.

How can we do better? One place to begin is by looking at what has worked, and/or failed to work, in the past.

Five months before Pearl Harbor, Franklin D. Roosevelt established an office called the Coordinator of Information, a precursor to the Office of Strategic Services (OSS). When Japanese bombs fell on American territory on December 7, 1941, the U.S. had a fledgling intelligence service to ramp up.

One of the more fascinating documents in the history of American intelligence has been published under the bland title of How Assessment Centers Were Started in the United States: The OSS Assessment Program. Written by Donald W. MacKinnon, the man in charge of screening personnel at the time, it tells the story of our first efforts to screen candidates for an espionage agency in the midst of wartime.

At the very beginning, some disastrous mistakes were made. It seemed logical to some that “it takes dirty men to do dirty works.” A number of initial recruits were thus drawn from the ranks of organized crime, including Murder Inc., and the Philadelphia Purple Gang. But several clandestine operations in operations employing such underworld types ended in catastrophe. By 1943, a professional screening program at a secret installation known as Station S was put in place.

Applicants to the OSS were soon being subjected to an extended set of tests. These were not designed to measure specific skills but to assess “the man as a whole” across eight dimensions: motivation, practical intelligence, emotional stability, social relations, leadership, physical ability, observation and reporting, propaganda skills, and maintaining cover.

Many of the tests were exceptionally grueling, including especially the one designed to measure one component of emotional stability, “resistance to stress and frustration tolerance,” and which came to be known as Behind the Barn.

Candidates were required to direct two helpers in the task of building a five-foot cube structure with seven-foot diagonals on its four sides, using an immense “tinker-toy” set of materials:

The candidate had 10 minutes in which to accomplish the task. All the physical work was to be done by the helpers, junior staff members who played the role of Kippy (passive, sluggish, and something of a stumblebum) and Buster (aggressive, critical, constantly making impractical suggestions). Both were insulting, faultfinding characters.

In the history of the assessment center, the construction task was never completed in the allotted time, but that was not the point. Rather, the point was to evaluate how the applicants behaved:

Some candidates gained insight into the problem, but more often they became so involved and so frustrated that they had difficulty in handling their frustration and controlling their anger. A few physically attacked their helpers, and some asked to be relieved from the program after this exercise.

Did the men who made it through this and the many other rigorous challenges at Station S make for good spies? The answer to that questions is by no means uncomplicated, as Mackinnon shows. All told, it remains doubtful that the initial OSS approach is the right one for the challenges before us today.

But there is no question that in the years running up to September 11, too many mediocrities and dangerous misfits came to occupy pivotal positions in the CIA. Preventing a recurrence of such organizational decay is a critical task.