document

Abstract

We present the extension of a serial execution commercial streamline simulator to mulit-core architectures based on the OpenMP programming model and its performance on various field examples. This work is an extension of recent work by Gerritsen et al. (2008) in which a research streamline simulator was extended for parallel execution due to its intrinsically parallel algorithm.

We identified that the streamline-transport step represents the bulk of the run time (65%-85%). It is exactly this step that is straightforward to parallelize owing to the independent solution of each streamline that is at the heart of streamline simulation. Because we are working with an existing large serial code base, we used specialty software to quickly and easily identify variables that required special handling for implementing the parallel extension. Minimal rewrite to existing code was required to extend the streamline-transport step to OpenMP. As part of this work we also parallelized an additional 2% of the runtime code which included the gravity-line solver and some simple routines required for constructing the pressure matrix.

We tested our parallel simulator on a variety of models including SPE10, a Forties UK oil/water model, a Judy Creek waterflood/WAG model, and a Middle East dual-porosity model. We noted speedup factors of between 2.5x to 3.5x for 8-threads. In terms of real time, this implies that large-scale streamline simulation models as tested here can be simulated in less than 4 hrs. We noted speedup scaling results that were reasonable when compared with Amdahl’s ideal scaling law. Beyond 8-threads we did observe reduced speedups and attribute this to memory bandwidth limits on our test machine (AMD 8x Opteron 8218 2.6Ghz dual core).

Introduction

Reservoir simulation models are routinely limited in size due to runtime and memory constraints. Yet modern reservoir engineering workflows for history matching, uncertainty assessment, and optimizing forecast scenarios rely on the ability to run large, finely gridded models multiple times.

Recent advances in CPU hardware have made multi-core-CPU shared-memory workstations the normal computing environment for today’s reservoir engineers, with workstations usually having 4 to 16 cores. This means that simulation codes that are parallelized via OpenMP directives can take advantage of these shared-memory systems. All major commercial finite-difference (FD) simulators are available with a parallel compute option, either OpenMP and/or MPI, (Collins et al. 2003, DeBaun et al. 2005, Shiralkar et al. 2005).

Commercial streamline (SL) based reservoir simulators have traditionally been less mature than FD methods due to their introduction only within the last 10 years. SL simulators are not readily available with a parallel computing option meaning existing workflows can not take advantage of the latest multi-core hardware developments. However, because SL simulation is ideally suited for simulating large, geologically heterogeneous models, quantifying upscaling processes, history matching, and/or ranking/screening, a parallel extension would be highly desirable. Fortunately the SL method is inherently parallelizable. Recent work on a research streamline simulator (Gerritsen et al. 2008) presented promising results. Gerritsen (2008) focused on parallelization of the transport step only and measured speedups of approximately 6x for 8-threads (AMD chip) and 13.5x for 16-threads (SUN chip) for a large, synthetic reservoir model.