Episode 1: What is Judgment?

In the first episode, we establish a working definition of Judgment and explain the three types of judgment. 

We use the process of analytical discovery of how to match up an understanding of facts, assessment of their relative importance, and creative thought about what to do. 


What is Judgment?

Judgment is an expression of a mental activity that may be exercised with greater or less skill.


What are the three types of judgment?

  • Action Judgment "What is to be done?"
  • Value Judgment: "What difference does it make?"
  • Reality Judgment: "What is going on?"

Key Considerations:

Perfect knowledge of reality is not of much use without criteria for separating the important from the unimportant.

Knowing clearly what is important is not much use in practical affairs without capacity for matching knowledge to action.

Initial reality judgments can be sharpened by taking care to distinguish clearly what is known from what is presumed to be true.


Dealing with Dictators: Dilemmas of U.S. Diplomacy and Intelligence Analysis, 1945-1990, by Enerst May and Philip Zelikow

The Art of Judgment: A Study of Policy Making, by Sir Geoffrey Vickers

Col. John Boyd's Observe-Orient-Decide-Act "OODA" Loop

Boyd's OODA Loop Presentation Slide Deck


Thinking in Time: The Art & Science of Judgment origins.

From 1986 to 2002, Harvard University's Kennedy School of Government had an executive program for senior managers in the U.S. intelligence community, known as the Intelligence and Policy Program. It ran once or twice a year for one to three week. Participants typically had twelve to twenty years of experience. Most came from the Directorate of Intelligence (DI) in the Central Intelligence Agency (CIA). Some came from CIA's Directorate of Operations (DO), the home of the U.S. clandestine service; end-of-course certificates for the latter had blanked-out names because the names under which they enrolled were not their own. Some other participants came from the National Security Agency, the center for U.S. signal interception and code-breaking; the National Reconnaissance Organization, which oversees collection of intelligence by satellites; the Defense Intelligence Agency, the military establishment's not-so-mini-CIA; and the intelligence branches of the armed services, the Department of State, and the Department of Energy. A handful came from federal law enforcement agencies.

This executive program was the brainchild of Robert Gates, who in 1986 had just become Deputy Director of Central Intelligence. He would later be Deputy National Security Advisor for President George H.W. Bush, as well as Bush's Director of Central Intelligence (and later Secretary of Defense under President Barack Obama). In 2002, he became president of Texas A&M University. As he outlines in his unique, revealing, and quite wonderful memoir, Gates was a career analyst in the CIA who chanced early on to have tours of duty in the White House, first under Henry Kissinger, then under Zbigniew Brzezinski. It struck him that his work in the White House on policy issues did not connect often with the work of his former colleagues in the intelligence community. When back in that community, he puzzled over the fact that work there connected so little with what was being done by his whilom colleagues in the National Security Council staff and other parts of the government where policy decisions were framed, taken, and executed.

Gates devised the Kennedy Schools new executive program after consulting with, among others in the School, Richard Neustadt and Ernest May, Dean Graham Allison and Academic Dean Albert Carnesale, and Peter Zimmerman and Nancy Huntington, who managed the other executive programs. The Intelligence and Policy Program aimed to teach managers in the intelligence community how to think about the needs in the policy community and about ways in which they and their associates might better serve those needs. This would be done in part by exposing them to elements of decision, bargaining, and organization theory, but primarily through Socratic discourse centered on case studies. 

Neustadt and May had, for several years, taught a case-based course, on the basis of which they had then just published Thinking in Time: The Use of History for Decisionmakers. The first cases used in the new program were cases that they had developed, with assistance from the National Endowment for the Humanities, all focused on presidential choice-making but tangentially involving use of intelligence. Of these cases, those that seemed best-suited for the new program concerned the Pearl Harbor attack, the onset of the Korean War, the Bay of Pigs affair, the 1962 missile crisis, and the Americanization of the Vietnam War.

A contract between the Kennedy School and CIA's Center for the Study of Intelligence provided not only for the new executive program but also for development of one to three new case studies annually, designed specifically to foster learning about the intelligence-policy nexus. The other cases were to be wholly unclassified and subject to no constraints other than those normal in any academic research project. The CIA undertook, however, to facilitate the effort, particularly by declassifying documents and identifying potential interviewees.

A steering committee helped guide the selection of topics for case studies. Chaired jointly by the Dean of the Kennedy School and the Director of Central Intelligence, the committee consisted of senior faculty from Harvard University, current and former high-level officials from Washington's intelligence and policy communities, and majority and minority members of the two Congressional intelligence oversight committees.

Between 1986 and 2002, the Program developed more than forty cases, some of which had multiple parts and two of which became books; one on the 1962 missile crisis, the other on Germany's defeat of France in 1940. 

The CIA-Harvard contract was itself necessarily unclassified, for Harvard has since World War II been unbending in its refusal to do classified research. The CIA had as little experience of writing unclassified contracts as Harvard had of concluding contracts with an intelligence agency. On the Harvard side, with memories of Vietnam-era riots still all too fresh, deans and professors worried lest news of the contract lead to protest demonstrations. Although Harvard sponsors had agrees that the objective was important enough to justify running such a risk, they tried to minimize it by issuing a low-key press release lat on a Friday afternoon, when it was least likely to catch the attention of newspaper editors. The ploy failed; the story was featured in both Harvard student newspapers and in the Boston press. However, to the wonderment of the Harvard contingent, the headlines lauded Harvard for "opening up" the CIA.

The CIA had already placed a certain number of senior officials in universities as "officers-in-residence" to teach, do research, or just provide information about the intelligence community. The program at the Kennedy School benefited from having a succession of such officials, whom the School designated as research associates, who stayed for at least a semester, and who helped develop teaching cases for the program. Ocupants of this post included William Kline and James Worthen from the Directorate of Intelligence and two others who, after their retirement from the CIA, spent some additional years at Harvard: Charles Cogan, who had headed the Near East Division of the Directorate of Operations and been station chief in Paris, and Douglas MacEachin, an analyst of strategic forces, who had been head of the Directorate of Intelligence.

Neustadt and May managed the program from the Harvard side--May from beginning to end, and Neustadt keeping a hand in long after his formal retirement in 1989. They were joined from time to time by, among others, Gregory Treverton, who went on to become a Vice Chairman of the National Intelligence Council and then a Senior Policy Analyst for the RAND Corporation; Richard Haass, later Director of Foreign Policy Studies at the Brookings Institution, head of policy planning for Secretary of State Colin Powell, and president of the Council on Foreign Relations; Robert Blackwill, later Ambassador to India and then a deputy to National Security Advisor Condoleezza Rice; Kurt Campbell, later senior vice president of the Center for Strategic and International Studies; and Jessica Stern, a member of the Kennedy School faculty and an expert on terrorism.

From his arrival at the Kennedy School in 1991, Philip Zelikow became, along with May and Neustadt, a central figure in the program. After Neustadt retired, May and Zelikow became the principal architects, editors, and sometimes authors of its case studies. The executive program in its early years depended, like most programs in Harvard's professional schools, on cameo appearances by a variety of faculty and visitors. Later, the program was taught almost exclusively by May and Zelikow, who were sometimes in the classroom six to eight hours a day.

May and Zelikow--coached, of course, by fellow faculty members and by friends outside--gradually developed a set of precepts which participants in the executive program reported in feedback to have proved of real use in their work. The reader is warned that they are less than fully meaningful absent the opportunity to discuss them along with the case studies, which serve to explain them and amplify them.

We hope that this will be widely used in the study and teaching of intelligence analysis.