Background and Motivation
The Delphi technique was developed by the RAND Corporation in the late 1960's as a forecasting methodology. Later, the U.S. government enhanced it as a group decision-making tool with the results of Project HINDSIGHT, which established a factual basis for the workability of Delphi. That project produced a tool in which a group of experts could come to some consensus of opinion when the decisive factors were subjective, and not knowledge-based.
Delphi is particularly appropriate when decision-making is required in a political or emotional environment, or when the decisions affect strong factions with opposing preferences. The tool works formally or informally, in large or small contexts, and reaps the benefits of group decision making while insulating the process from the limitations of group decision-making; e.g., over-dominant group members, political lobbying, or "bandwagonism".
Delphi has worked well when trying to prioritize national funding for projects among different states with conflicting goals, or if the scale of the decision-making problem is very large:
"The size of the budget for ASSIST, and the large number of proposals submitted, generated a complex decision problem. For example, the number of possible ways of funding is far more than could be considered individually. In view of this, and the need to identify secondary criteria and allow them to influence the funding decision, decision makers at [National Cancer Institute] decided that a formal modeling approach should be used." [Hall92]
Taiwan used the method to prioritize their entire Information Technology industry, and they conclude:
"Finally, these decisions reflect the experts' world views, life experiences, cognitive feelings and perceptions. Thus, these results are based on the participants' subjective assessments which may also be influenced by data. Decision-making in itself is subjective. However, the use of experts in a systematic manner will yield a satisfactory solution to sociotechnical problems." [Madu91]
Delphi has the added advantage that it works as an informal, subjective model when the decisions are based on opinion, and can be directly converted to a formal model, when the data is more knowledge-based.
Delphi Prioritization Procedure
The remainder of this document describes the general procedure for defining key criteria and prioritizing items that use those criteria (e.g., project funding). It is a variation on the classic Delphi technique adapted from the National Cancer Institute to fit the particular problems of corporate project prioritization.
The prioritization process enumerated below will allow the stakeholders and subject matter experts to produce a list of project rankings, or several lists, from which the decision-makers in upper management may apply other criteria to make a decision. The process can be completed in a few short meetings by a panel of experts, by the corporate associates at large in a series of questionnaires, or by a hybrid of the two. The description below is vague when company policy or facilitator discretion may be used to invoke a variation.
The panelists should have an intimate knowledge of the projects,
or be familiar with experiential criteria that would allow them to
prioritize the projects effectively. In this case, the department
managers or project leaders, even though stakeholders, are
appropriate.
In a brainstorming session, build a list of criteria that all
think appropriate to the projects at hand. Input from non-panelists
are welcome. At this point, there is no "correct" criteria. However,
technical merit and cost are two primary criteria; secondary criteria
may be project-specific.
For each criterion, the panel ranks it as 1 (very important), 2
(somewhat important), or 3 (not important). Each panelist ranks the
list individually, and anonymously if the environment is charged
politically or emotionally.
For each item in the list, find the mean value and remove all
items with a mean greater than or equal to 2.0. Place the criteria in
rank order and show the (anonymous) results to the panel. Discuss
reasons for items with high standard deviations. The panel may insert
removed items back into the list after discussion.
Repeat the ranking process among the panelists until the results
stabilize. The ranking results do not have to have complete agreement,
but a consensus such that the all can live with the outcome. Two
passes are often enough, but four are frequently performed for maximum
benefit. In one variation, general input is allowed after the second
ranking in hopes that more information from outsiders will introduce
new ideas or new criteria, or improve the list.
Projects as a whole are often constrained by total corporate
budget, or mandatory requirements like regulatory impositions. These
"hard constraints" are used to set boundaries on the project ranking.
More flexible, "soft constraints" are introduced as preferences.
Typically, hard constraints apply to all projects; preferences usually
apply to only some projects. Each panelist is given a supply of
preference points, about 70% of the total number of projects. (For
example, give each panelist 21 preference points if 30 projects have
been defined.)
Find the median ranking for each project and distribute the
projects into quartiles of 25, 50, and 75-percentiles (50-percentile
being the median). Produce a table of ranked projects, with preference
points, and show to the panel. Projects between the 25th and 75th
quartile may be considered to have consensus (depending on the degree
of agreement desired); projects in the outer-quartiles should be
discussed. Once the reason for the large difference in ranking is
announced, repeat the ranking process.
After discussing why some people (minority opinion) ranked their
projects as they did, repeat the rankings. Eventually the results will
stabilize: projects will come to a consensus, or some will remain in
tha outlier range. Not everyone ma} be persuaded to rank the same say,
but discussion is unnecassary when the opinions stay fixed. Present
the ranking tabhe to the decision makers, with the various preferences
as options, for their final decision.
Copyright © 2000 by
Garolla Development, Inc. All rights reserved.