Optimizer's Curse: Removing the Effect of this Bias in Portfolio Planning
- John R. Schuyler (OGCI-PetroSkills) | Timothy Nieman (Geomatrix Consultants)
- Document ID
- Society of Petroleum Engineers
- SPE Projects, Facilities & Construction
- Publication Date
- March 2008
- Document Type
- Journal Paper
- 1 - 9
- 2008. Society of Petroleum Engineers
- 7.1.5 Portfolio Analysis, Management and Optimization, 4.1.2 Separation and Treating, 4.1.1 Process Simulation, 4.1.5 Processing Equipment, 7.3.3 Project Management, 7.2.3 Decision-making Processes, 5.7.5 Economic Evaluations, 4.3.4 Scale, 7.1.9 Project Economic Analysis
- 2 in the last 30 days
- 520 since 2007
- Show more detail
- View rights & permissions
|SPE Member Price:||USD 5.00|
|SPE Non-Member Price:||USD 35.00|
Even with unbiased judgments and project calculations, an insidious bias will still be present in an approved projects portfolio. The screening and ranking process used to select projects causes an optimizer's curse (OC) effect (Smith and Winkler 2006). A portfolio's true outcome value will tend to fall below the forecast distribution mean. Stated another way, we should expect to be disappointed in the value of the eventual outcomes. This optimizer's curse effect can be huge: the true portfolio value might be half its forecast value when forecast in the usual way.
This paper describes a correction process based upon Bayesian inversion obtained with Monte Carlo simulation. Estimate/Actual distributions characterize component evaluation errors. The mean and standard deviation of the E/A distribution measures the quality of judgments and estimates. Examples demonstrate the value of better information in project or asset evaluation, portfolio optimization, and competitive bidding.
The premise of our approach to correct for optimizer's curse is straightforward enough, though implementing this method requires judging the population of evaluated projects—or at least assessing the shape of this population's distribution. We offer a process and guidelines for correcting for the optimizer's curse in both project and portfolio value calculations. Correcting estimates for systematic biases restores luster to the recommended expected value-maximizing decision policy. We also tie back to the winner's curse phenomenon (Capen et al. 1971) experienced in competitive bidding and how it affects bid optimization.
Preface—Why Project Managers Should Be Interested
Most of this paper deals with the optimizer's curse afflicting project appraisal, selection, and portfolio management processes. Project management begins well before a project is commissioned. Project requirements and scope define the project, and project modeling is usually the basis for forecasting costs, schedule, and, perhaps, performance.
Depending upon the quality of a candidate project, substantial project planning may be required for the economic evaluation. Very poor projects are easy to reject, and very good projects are easy to approve. Marginal projects require the most evaluation effort, and this typically involves detailed project design and planning in order to obtain confident estimates of costs, schedule, and performance under uncertainty.
Following are some additional thoughts about what we feel are the most important aspects for project management.
Consolidating Decision Criteria. Making tradeoffs is inevitable, and decision making is more complicated with multiple criteria. We recommend translating all criteria into monetary-equivalents. Project cost is in the money units already, though best if translated into after-tax, present value. Translating schedule and performance into monetary terms is straightforward, as these directly affect value to the asset owner: The project appraisal model represents how schedule and performance drive project value. Health, safety, environment and other considerations also influence project value, and the most straightforward way to reflect these dimensions is to convert their metrics into monetary-equivalents (e.g., $25/ton of CO2 abatement). (Please note that all costs in this paper are presented in U.S. dollars.)
|File Size||1 MB||Number of Pages||9|
Brown, K.C. 1974. A note on the apparent bias of net revenue estimates forcapital investment projects. J. Finance 29: 1215.
Capen, E.C., Clapp, R.V., and Campbell, W.M. 1971. Competitive Bidding in High-RiskSituations. JPT 23 (6): 641-653. SPE-2993-PA. DOI:10.2118/2993-PA.
Harrison, J.R. and March, J.G. 1984. Decision making and postdecisionsurprises. Admin. Sci. Quart. 29: 26.
Kahneman, D., Slovic, P., and Tversky, A. 1982. Judgment UnderUncertainty: Heuristics and Biases. New York: Cambridge UniversityPress.
Newendorp, P. and Schuyler, J. 2000. "Utility Theory Concepts," Ch. 5 inDecision Analysis for Petroleum Exploration, second edition. Aurora,Colorado: Planning Press.
Schuyler, J. 1995. Rational isPractical: Better Evaluations Through the Logic of Shareholder Value. PaperSPE 30066 presented at the SPE Hydrocarbon Economics and Evaluation Symposium,Dallas, 26-28 March. DOI: 10.2118/30066-MS.
Schuyler, J. 2001. "Stochastic Variance," Ch. 13 in Risk and DecisionAnalysis in Projects, second edition. Newtown Square, Pennsylvania: ProjectManagement Institute.
Smith, J.E. and Winkler, R.L. 2006. The Optimizer's Curse: Skepticism andPostdecision Surprise in Decision Analysis. Management Science 52(3): 311-322.