Distributed Parallel Hybrid CPU-GPGPU Implementation of the Phase-Field Method for Accelerated High-Accuracy Simulations of Pore-Scale Two-Phase Flow
- Authors
- Christopher Thiele (Rice University) | Mauricio Araya-Polo (Shell International Exploration & Production, Inc.) | Faruk Omer Alpak (Shell International Exploration & Production, Inc.) | Beatrice Riviere (Rice University)
- DOI
- https://doi.org/10.2118/193922-MS
- Document ID
- SPE-193922-MS
- Publisher
- Society of Petroleum Engineers
- Source
- SPE Reservoir Simulation Conference, 10-11 April, Galveston, Texas, USA
- Publication Date
- 2019
- Document Type
- Conference Paper
- Language
- English
- ISBN
- 978-1-61399-634-8
- Copyright
- 2019. Society of Petroleum Engineers
- Disciplines
- Keywords
- high performance computing, GPUs, linear solvers, digital rock physics, parallel computing
- Downloads
- 2 in the last 30 days
- 84 since 2007
- Show more detail
- View rights & permissions
| SPE Member Price: | USD 9.50 |
| SPE Non-Member Price: | USD 28.00 |
Direct numerical simulation of multi-phase pore-scale flow is a computationally demanding task with strong requirements on time-to-solution for the prediction of relative permeabilities. In this paper, we describe the hybrid-parallel implementation of a two-phase two-component incompressible flow simulator using MPI, OpenMP, and general-purpose graphics processing units (GPUs), and we analyze its computational performance. In particular, we evaluate the parallel performance of GPU-based iterative linear solvers for this application, and we compare them to CPU-based implementations of the same solver algorithms. Simulations on real-life Berea sandstone micro-CT images are used to assess the strong scalability and computational performance of the different solver implementations and their effect on time-to-solution. Additionally, we use a Poisson problem to further characterize achievable strong and weak scalability of the GPU-based solvers in reproducible experiments. Our experiments show that GPU-based iterative solvers can greatly reduce time-to-solution in complex pore-scale simulations. On the other hand, strong scalability is currently limited by the unbalanced computing capacities of the host and the GPUs. The experiments with the Poisson problem indicate that GPU-based iterative solvers are efficient when weak scalability is desired. Our findings show that proper utilization of GPUs can help to make our two-phase pore-scale flow simulation computationally feasible in existing workflows.
| File Size | 865 KB | Number of Pages | 14 |
Supporting information
- SUPPLEMENTARY/SPE-193922-SUP.pdf
Agarwal, U.,Alpak, F. O.,Koelman, J. M. V. A. 2018. Permeability From 3D Porous Media Images: A Fast Two-Step Approach. Transport in Porous Media 124 (3): 1017-1033. https://doi.org/10.1007/s11242-018-1108-0.
Alpak, F. O.,Gray, F.,Saxena, N.. 2018a. A Distributed Parallel Multiple-Relaxation-Time Lattice Boltzmann Method on General-Purpose Graphics Processing Units for the Rapid and Scalable Computation of Absolute Permeability From High-Resolution 3D Micro-CT Images. Computational Geosciences 22 (3): 815-832. https://doi.org/10.1007/s10596-018-9727-7.
Alpak, F. O.,Samardžic, A.,Frank, F. 2018b. A Distributed Parallel Direct Simulator for Pore-Scale Two-Phase Flow on Digital Rock Images Using a Finite Difference Implementation of the Phase-Field Method. Journal of Petroleum Science and Engineering 166: 806-824. https://doi.org/10.1016/j.petrol.2017.11.022.
Anciaux-Sedrakian, A.,Eaton, J.,Gratien, J.. 2015. Will GPGPUs be Finally a Credible Solution for Industrial Reservoir Simulators?. SPE Reservoir Simulation Symposium, Houston, Texas, USA, 23-25 February 2015. SPE-173223-MS. https://doi.org/10.2118/173223-MS.
Andrew, M.,Menke, H.,Blunt, M. J.. 2015. The Imaging of Dynamic Multiphase Fluid Flow Using Synchrotron-Based X-ray Microtomography at Reservoir Conditions. Transport in Porous Media 110 (1): 1-24. https://doi.org/10.1007/s11242-015-0553-2.
Armstrong, R. T.,Berg, S. 2013. Interfacial Velocities and Capillary Pressure Gradients During Haines Jumps. Physical Review E 88 (4): 043010. https://doi.org/10.1103/PhysRevE.88.043010.
Armstrong, R. T.,Berg, S.,Dinariev, O.. 2016. Modeling of Pore-Scale Two-Phase Phenomena Using Density Functional Hydrodynamics. Transport in Porous Media 112 (3): 577-607. https://doi.org/10.1007/s11242-016-0660-8.
Armstrong, R. T.,Evseev, N.,Koroteev, D.. 2015. Modeling the Velocity Field During Haines Jumps in Porous Media. Advances in Water Resources 77: 57-68. https://doi.org/10.1016/j.advwatres.2015.01.008.
Berg, S.,Ott, H.,Klapp, S. A.. 2013. Real-Time 3D Imaging of Haines Jumps in Porous Media Flow. Proceedings of the National Academy of Sciences 110 (10): 3755-3759. https://doi.org/10.1073/pnas.1221373110.
Berg, S.,Rücker, M.,Ott, H.. 2016. Connected Pathway Relative Permeability From Pore-Scale Imaging of Imbibition. Advances in Water Resources 90: 24-35. https://doi.org/10.1016/j.advwatres.2016.01.010.
Bernaschi, M.,Fatica, M.,Melchionna, S.. 2010. A Flexible High-Performance Lattice Boltzmann GPU Code for the Simulations of Fluid Flows in Complex Geometries. Concurrency and Computation 22 (1): 1-14. https://doi.org/10.1002/cpe.1466.
Demianov, A.,Dinariev, O.,Evseev, N. 2011. Density Functional Modelling in Multiphase Compositional Hydrodynamics. The Canadian Journal of Chemical Engineering 89 (2): 206-226. https://doi.org/10.1002/cjce.20457.
Dinariev, O.,Evseev, N. 2016. Multiphase Flow Modeling With Density Functional Method. Computational Geosciences 20 (4): 835-856. https://doi.org/10.1007/s10596-015-9527-2.
Frank, F.,Liu, C.,Alpak, F. O.. 2017. A Discontinuous Galerkin Finite Element Framework for the Direct Numerical Simulation of Flow on High-Resolution Pore-Scale Images. Proc., SPE Reservoir Simulation Conference, Montgomery, TX, USA, 20-22 February 2017. SPE-182607-MS. https://doi.org/10.2118/182607-MS.
Frank, F.,Liu, C.,Alpak, F. O.. 2018a. A Finite Volume/Discontinuous Galerkin Method for the Advective Cahn-Hilliard Equation With Degenerate Mobility on Porous Domains Stemming From Micro-CT Imaging. Computational Geosciences22 (2): 543-563. https://doi.org/10.1007/s10596-017-9709-1.
Frank, F.,Liu, C.,Alpak, F. O.. 2018b. Direct Numerical Simulation of Flow on Pore-Scale Images Using the Phase-Field Method. SPE Journal 23 (5): 1833-1850. SPE-182607-MS. https://doi.org/10.2118/182607-MS.
Heroux, M. A.,Bartlett, R. A.,Howle, V. E.. 2005. An overview of the Trilinos project. ACM Trans. Math. Softw. 31 (3): 397-423. http://doi.acm.org/10.1145/1089014.1089021.
Joekar-Niasar, V.,Hassanizadeh, S. M.,Dahle, H. K. 2010. Non-Equilibrium Effects in Capillarity and Interfacial Area in Two-Phase Flow: Dynamic Pore-Network Modelling. Journal of Fluid Mechanics 655: 38-71. https://doi.org/10.1017/S0022112010000704.
Joekar-Niasar, V.,Hassanizadeh, S. M. 2011. Effect of Fluids Properties on Non-Equilibrium Capillarity Effects: Dynamic Pore-Network Modeling. International Journal of Multiphase Flow 37 (2): 198-214. https://doi.org/10.1016/j.ijmultiphaseflow.2010.09.007.
Karypis, G.,Kumar, V. 1998. A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs. SIAM Journal on Scientific Computing 20 (1): 359-392. https://doi.org/10.1137/S1064827595287997.
McClure, J. E.,Prins, J. F.,Miller, C. T. 2014. A Novel Heterogeneous Algorithm to Simulate Multiphase Flow in Porous Media on Multicore CPU-GPU Systems. Computer Physics Communications 185 (7): 1865-1874. https://doi.org/10.1016/j.cpc.2014.03.012.
Meakin, P.,Tartakovsky, A. M. 2009. Modeling and Simulation of Pore-Scale Multiphase Fluid Flow and Reactive Transport in Fractured and Porous Media. Reviews of Geophysics 47 (3): 1-47. https://doi.org/10.1029/2008RG000263.
Naumov, M.,Arsaev, M.,Castonguay, P.. 2015. AmgX: A Library for GPU Accelerated Algebraic Multigrid and Preconditioned Iterative Methods. SIAM Journal on Scientific Computing 37 (5): S602-S626. https://doi.org/10.1137/140980260.
Ryazanov, A. V.,van Dijke, M. I. J.,Sorbie, K. S. 2009. Two-Phase Pore-Network Modelling: Existence of Oil Layers During Water Invasion. Transport in Porous Media 80 (1): 79-99. https://doi.org/10.1007/s11242-009-9345-x.
Saxena, N.,Hofmann, R.,Alpak, F. O.. 2017. References and Benchmarks for Pore-Scale Flow Simulated Using Micro-CT Images of Porous Nedia and Digital Rocks. Advances in Water Resources 109: 211-235. https://doi.org/10.1016/j.advwatres.2017.09.007.
Thiele, C.,Araya-Polo, M.,Alpak, F. O.. 2017. Inexact Hierarchical Scale Separation: A Two-Scale Approach for Linear Systems From Discontinuous Galerkin Discretizations. Computers and Mathematics With Applications 74 (8): 1769-1778. https://doi.org/10.1016/j.camwa.2017.06.025.
Tierra, G.,Guillén-González, F. 2015. Numerical Methods for Solving the Cahn-Hilliard Equation and Its Applicability to Related Energy-Based Models. Archives of Computational Methods in Engineering 22 (2): 269-289. https://doi.org/10.1007/s11831-014-9112-1.
Tölke, J. 2010. Implementation of a Lattice Boltzmann Kernel Using the Compute Unified Device Architecture Developed by NVIDIA. Computing and Visualization in Science 13: 29-39. https://doi.org/10.1007/s00791-008-0120-2.
Yu, S.,Liu, H.,Chen, Z. J.. 2012. GPU-Based Parallel Reservoir Simulation for Large-Scale Simulation Problems. SPE Europec/EAGE Annual Conference, Copenhagen, Denmark, 4-7 June 2012. SPE-152271-MS. https://doi.org/10.2118/152271-MS.
