Use of Approximate Dynamic Programming for Production Optimization
- Zheng Wen | Louis J. Durlofsky (Stanford University) | Benjamin Van Roy (Stanford University) | Khalid Aziz (Stanford University)
- Document ID
- Society of Petroleum Engineers
- SPE Reservoir Simulation Symposium, 21-23 February, The Woodlands, Texas, USA
- Publication Date
- Document Type
- Conference Paper
- 2011. Society of Petroleum Engineers
- 5.3.2 Multiphase Flow, 7.1.5 Portfolio Analysis, Management and Optimization, 5.1.5 Geologic Modeling, 4.3.4 Scale, 6.5.2 Water use, produced water discharge and disposal, 5.4.1 Waterflooding, 5.5 Reservoir Simulation
- 1 in the last 30 days
- 362 since 2007
- Show more detail
- View rights & permissions
|SPE Member Price:||USD 8.50|
|SPE Non-Member Price:||USD 25.00|
In production optimization, we seek to determine the well settings (bottomhole pressures, flow rates) that maximize an objective function such as net present value. In this paper we introduce and apply a new approximate dynamic programming (ADP) algorithm for this optimization problem. ADP aims to approximate the global optimum using limited computational resources via a systematic set of procedures that approximate exact dynamic programming algorithms. The method is able to satisfy general constraints such as maximum watercut and maximum liquid production rate in addition to bound constraints. ADP has been used in many application areas, but it does not appear to have been implemented previously for production optimization. The ADP algorithm is applied to twodimensional
problems involving primary production and water injection. We demonstrate that the algorithm is able to provide clear improvement in the objective function compared to baseline strategies. It is also observed that, in cases where the global optimum is known (or surmised), ADP provides a result within 1-2% of the global optimum. Thus the ADP procedure may be appropriate for practical production optimization problems.
Many optimization algorithms have been applied to maximize reservoir performance. Most of these optimization algorithms can be classified into two categories: gradient-based/direct-search [20, 15] and global stochastic search [13, 1, 7]. Both classes of optimization algorithms face limitations: gradient-based and direct-search algorithms settle for local optima, while global stochastic search algorithms such as genetic algorithms typically require many function evaluations for convergence, and even then there is no assurance that the global optimum has been found.
In principle, one can formulate production optimization as a nonlinear optimal control problem and find a global optimum using dynamic programming (DP) . The key idea in DP is to decompose the optimization problem into a sequence of sub-problems, each representing optimization of a control action at one point in time. These sub-problems are related through the value function, which maps the system state to the net present value of future revenues. Once the value function is computed, the optimal control action at each time can be found by solving a sub-problem at that time. In general, these sub-problems are much simpler than the original optimization problem; in many cases, a global optimum of each sub-problem can be efficiently computed.
For problems of practical scale, the computational requirements of DP become prohibitive due to the curse of dimensionality. In particular, time and memory requirements typically grow exponentially with the number of state variables. Approximate dynamic programming (ADP) aims to address this computational burden by efficiently approximating the value function (see [3, 25, 19] for more on ADP). The result is an approximate value function, which is typically represented by a linear combination of a set of predefined basis functions. ADP algorithms provide methods for computing the coefficients associated with these basis functions. ADP has been successfully applied across a broad range of domains such as asset pricing [23, 18, 24], transportation logistics , revenue management [26, 10, 27], portfolio management [14, 12], and even to games such as backgammon .
|File Size||957 KB||Number of Pages||16|