Optimization of Lattice Boltzmann Simulation With Graphics-Processing-Unit Parallel Computing and the Application in Reservoir Characterization
- Cheng Chen (Virginia Tech) | Zheng Wang (Rice University) | Deepak Majeti (Rice University) | Nick Vrvilo (Rice University) | Timothy Warburton (Virginia Tech) | Vivek Sarkar (Rice University) | Gang Li (Anadarko Petroleum)
- Document ID
- Society of Petroleum Engineers
- SPE Journal
- Publication Date
- August 2016
- Document Type
- Journal Paper
- 1,425 - 1,435
- 2016.Society of Petroleum Engineers
- Lattice Boltzmann, GPU, OCCA, Reservior Characterization, Parallel Computing
- 2 in the last 30 days
- 226 since 2007
- Show more detail
- View rights & permissions
|SPE Member Price:||USD 10.00|
|SPE Non-Member Price:||USD 30.00|
Shale permeability is sufficiently low to require an unconventional scale of stimulation treatments, such as very-large-volume, high-rate, multistage hydraulic-fracturing applications. Upscaling of hydrocarbon transport processes in shales is challenging because of the low permeability and strong heterogeneity. Rock characterization with high-resolution imaging [X-ray tomography and scanning electron microscope (SEM)] is usually highly localized and contains significant uncertainties because of the small field of view. Therefore, an effective high-performance computing method is required to collect information over a larger scale to meet the ergodicity requirement in upscaling. The lattice Boltzmann (LB) method has received significant attention in computational fluid dynamics because of its capability in coping with complicated boundary conditions. A combination of high-resolution imaging and LB simulation is a powerful approach for evaluating the transport properties of a porous medium in a timely manner, on the basis of the numerical solution of the Navier-Stokes equations and Darcy’s law. In this work, a graphics-processing-unit (GPU) -enhanced lattice Boltzmann simulator (GELBS) was developed, which was optimized by GPU parallel computing on the basis of the inherent parallelism of the LB method. Specifically, the LB method was used to implement the computational kernel; a sparse data structure was applied to optimize memory allocation; the OCCA (Medina et al. 2014) portability library was used, which enables the GELBS codes to use different application-programming interfaces (APIs) including open computing language (OpenCL), compute unified device architecture (CUDA), and open multiprocessing (OpenMP). OpenCL is an open standard for cross-platform parallel computing, CUDA is supported only by NVIDIA devices, and OpenMP is primarily used on central processing units (CPUs). It was found that the GPU-accelerated code was approximately 1,000 times faster than the unoptimized serial code and 10 times faster than the parallel code run on a standalone CPU. The CUDA code was slightly faster than OpenCL code on the NVIDA GPU because of the extra cost of OpenCL used to adapt to a heterogeneous platform. The GELBS was validated by comparing it with analytical solutions, laboratory measurements, and other independent numerical simulators in previous studies, and it was proved to have a second-order global accuracy. The GELBS was then used to analyze thin cuttings extracted from a sandstone reservoir and a shale-gas reservoir. The sandstone permeabilities were found relatively isotropic, whereas the shale permeabilities were strongly anisotropic because of the horizontal lamination structure. In shale cuttings, the average permeability in the horizontal direction was higher than that in the vertical direction by approximately two orders of magnitude. Correlations between porosity and permeability were observed in both rocks. The combination of GELBS and high-resolution imaging methods makes for a powerful tool for permeability evaluation when conventional laboratory measurement is impossible because of small cuttings sizes. The constitutive correlations between geometry and transport properties can be used for upscaling in different rock types. The GPU-optimized code significantly accelerates the computing speed; thus, many more samples can be analyzed given the same processing time. Consequently, the ergodicity requirement is met, which leads to a better reservoir characterization.
|File Size||1 MB||Number of Pages||11|
Bailey, P., Myre, J., Walsh, S. et al. 2009. Accelerating Lattice Boltzmann Fluid Flow Simulations Using Graphics Processors. Presented at the International Conference on Parallel Processing, Vienna, Austria, 22–25 September. http://dx.doi.org/10.1109/ICPP.2009.38.
Benzi, R., Succi, S., and Vergassola, M. 1992. The Lattice Boltzmann Equation: Theory and Applications. Phys. Rep. 222 (3): 145–197. http://dx.doi.org/10.1016/0370-1573(92)90090-M.
Bernaschi, M., Fatica, M., Melchionna, S. et al. 2010. A Flexible High-Performance Lattice Boltzmann GPU Code for the Simulations of Fluid Flows in Complex Geometries. Concurrency Computat.: Pract. Exper. 22: 1–14. http://dx.doi.org/10.1002/cpe.1466.
Bisson, M., Bernaschi, M., Melchionna, S. et al. 2012. Multiscale Hemodynamics Using GPU Clusters. Communications in Computer Physics 11 (1): 48–64. http://dx.doi.org/10.4208/cicp.210910.250311a.
Chen, S. and Doolen, G. D. 1998. Lattice Boltzmann Method for Fluid Flows. Annu. Rev. Fluid Mech. 30: 329–364. http://dx.doi.org/10.1146/annurev.fluid.30.1.329.
Chen, C., Packman, A. I., and Gaillard, J. F. 2008. Pore-Scale Analysis of Permeability Reduction Resulting From Colloid Deposition. Geophysical Research Letters 35 (7): L07404. http://dx.doi.org/10.1029/2007GL033077.
Chen, C., Lau, B. L. T., Gaillard, J. F. et al. 2009. Temporal Evolution of Pore Geometry, Fluid Flow, and Solute Transport Resulting From Colloid Deposition. Water Resources Research 45: W06416. http://dx.doi.org/10.1029/2008WR007252.
Chen, C. and Zhang, D. 2009. Lattice Boltzmann Simulation of the Rise and Dissolution of Two-Dimensional Immiscible Droplets. Physics of Fluids 21 (103301). http://dx.doi.org/10.1063/1.3253385.
Chen, C., Packman, A. I., Zhang, D. et al. 2010. A Multi-Scale Investigation of Interfacial Transport, Pore Fluid Flow, and Fine Particle Deposition in a Sediment Bed. Water Resources Research 46 (11). http://dx.doi.org/10.1029/2009WR009018.
Chen, C., Hu, D., Westacott, D. et al. 2013. Nanometer-Scale Characterization of Microscopic Pores in Shale Kerogen by Image Analysis and Pore-Scale Modeling. Geochemistry, Geophysics, Geosystems 14 (10): 4066–4075. http://dx.doi.org/10.1002/ggge.20254.
Chen, C., Balhoff, M. T., and Mohanty, K. K. 2014. Effect of Reservoir Heterogeneity on Primary Recovery and CO2 Huff ‘n’ Puff Recovery in Shale-Oil Reservoirs. SPE Res Eval & Eng 17 (3): 404–413. SPE-164553-PA. http://dx.doi.org/10.2118/164553-PA.
Chen, C., Martysevich, V., O’Connell, P. et al. 2015. Temporal Evolution of the Geometrical and Transport Properties of a Fracture/Proppant System Under Increasing Effective Stress. SPE J. 20 (3): 527–535. SPE-171572-PA. http://dx.doi.org/10.2119/171572-PA.
Curtis, M. E., Ambrose, R. J., Sondergeld, C. H. et al. 2010. Structural Characterization of Gas Shales on the Micro- and Nano-Scales. Presented at SPE Canadian Unconventional Resources and International Petroleum Conference, Calgary, 19–21 October. SPE-137693-MS. http://dx.doi.org/10.2118/137693-MS.
d’Humières, D. and Ginzburg, I. 2009. Viscosity Independent Numerical Errors for Lattice Boltzmann Models: From Recurrence Equations to “Magic” Collision Numbers. Computers & Mathematics With Applications 58: 823–840. http://dx.doi.org/10.1016/j.camwa.2009.02.008.
De Prisco, G., Toelke, J., and Dernaika, M. R. 2011. Computation of Relative Permeability Functions in 3d Digital Rocks by a Fractional Flow Approach Using the Lattice Boltzmann Method. Presented at the International Symposium of the Society of Core Analysts, Aberdeen, 27–30 August. SCA-2012-A36.
Fan, Z., Qiu, F., Kaufman, A. et al. 2004. GPU Cluster for High-Performance Computing. In Proc., the ACM/IEEE Conference on Supercomputing. IEEE Computer Society.
Gaster, B., Howes, L., Kaeli, D. R. et al. 2012. Heterogeneous Computing With OpenCL, Second Edition: Revised OpenCL 1.2 Edition, second edition, Burlington, Massachusetts: Morgan Kaufmann.
Gaurav, A., Dao, E. K., and Mohanty, K. K. 2012. Evaluation of Ultralight-Weight Proppants for Shale Fracturing. J. Petrol. Sci. & Eng. 92–93: 82–88. http://dx.doi.org/10.1016/j.petrol.2012.06.010.
Ginzbourg, I. and Adler, P. 1994. Boundary Flow Condition Analysis for the Three-Dimensional Lattice Boltzmann Model. J. de Physique II 4: 191–214. http://dx.doi.org/10.1051/jp2:1994123>.<jpa-00247955>
Hu, D., Yang, V., Li, G. et al. 2013. Understanding True Unconventional Reservoir Properties Using Nanoscale Technology. Presented at the SPE Kuwait Oil and Gas Show and Conference, Kuwait City, Kuwait, 8–10 October. SPE-167326-MS. http://dx.doi.org/10.2118/167326-MS.
Inamuro, T., Yoshino, M., and Ogino, F. 1999. Lattice Boltzmann Simulation of Flows in a Three-Dimensional Porous Structure. Int. J. Numer. Methods Fluids 29: 737–748. http://dx.doi.org/10.1002/(SICI)1097-0363(19990415)29:7<737::AID-FLD813>3.0.CO;2-H.
Jarvie, D. M., Hill, R. J., Ruble, T. E. et al. 2007. Unconventional Shale-Gas Systems: The Mississippian Barnett Shale of North Central Texas as One Model for Thermogenic Shale-Gas Assessment. Special Issues: Barnett Shale, ed. R. J. Hill and D. M. Jarvie. American Association of Petroleum Geologists Bull. 91 (4): 475–499. http://dx.doi.org/10.1306/12190606068.
Ji, L., Zhang, T., Milliken, K. et al. 2012. Experimental Investigation of Main Controls to Methane Adsorption in Clay-Rich Rocks. Applied Geochemistry 27 (12): 2533–2545. http://dx.doi.org/10.1016/j.apgeochem.2012.08.027.
Ketcham, R. A. 2005. Computational Methods for Quantitative Analysis of Three-Dimensional Features in Geological Specimens. Geosphere 1 (1): 32–41. http://dx.doi.org/10.1130/GES00001.1.
Li, Wei, Fan, Z., Wei, X. et al. 2005. Chapter 47. Flow Simulation With Complex Boundaries. In GPU Gems, Vol. 2, 747–764.
Loucks, R. G., Reed, R. M., Ruppel, S. C. et al. 2009. Morphology, Genesis and Distribution of Nanometer-Scale Pores in Siliceous Mudstones of the Mississippian Barnett Shale. J. Sedimentary Research 79: 848–861. http://dx.doi.org/10.2110/jsr.2009.092.
Manwart, C., Aaltosalmi, U., Koponen, A., et al. 2002. Lattice-Boltzmann and Finite-Difference Simulations for the Permeability for Three- Dimensional Porous Media. Phys. Rev. E 66: 016702. http://dx.doi.org/10.1103/PhysRevE.66.016702.
Medina, D. S., St-Cyr, A., and Warburton, T. 2014. OCCA: A Unified Approach to Multi-Threading Languages. arXiv preprint arXiv:1403.0968.
Melchionna, S., Bernaschi, M., Succi, S. et al. 2010. Hydrokinetic Approach to Large-Scale Cardiovascular Blood Flow. Computer Physics Communications 181: 462–472. http://dx.doi.org/10.1016/j.cpc.2009.10.017.
Pan, C., Luo, L.-S., and Miller, C. T. 2006. An Evaluation of Lattice Boltzmann Schemes for Porous Medium Flow Simulation. Computers & Fluids 35: 898–909. http://dx.doi.org/10.1016/j.compfluid.2005.03.008.
Passey, Q. R., Bohacs, K. M., Esch, W. L. et al. 2010. From Oil-Prone Source Rock to Gas-Producing Shale Reservoir—Geologic and Petrophysical Characterization of Unconventional Shale-Gas Reservoirs. Presented at the CPS/SPE International Oil and Gas Conference and Exhibition, Beijing, 8–10 June. SPE-131350-MS. http://dx.doi.org/10.2118/131350-MS.
Pohl, T., Kowarschik, M., Wilke, J. et al. 2003. Optimization and Profiling of the Cache Performance of Parallel Lattice Boltzmann Codes. Parallel Processing Letters 13 (4): 549–560. http://dx.doi.org/10.1142/S0129626403001501.
Succi, S. 2001. The Lattice Boltzmann Equation for Fluid Dynamics and Beyond. Oxford: Oxford Science Publishers.
Talon, L., Bauer, D., Gland, N., et al. 2012. Assessment of the Two Relaxation Time Lattice-Boltzmann Scheme to Simulate Stokes Flow in Porous Media. Water Resources Research 48: W04526. http://dx.doi.org/10.1029/2011WR011385.
Tölke, J. 2010. Implementation of a Lattice Boltzmann Kernel Using the Compute Unified Device Architecture Developed by nVIDIA. Computing and Visualization in Science 13 (1): 29–39. http://dx.doi.org/10.1007/s00791-008-0120-2.
Wang, F. P. and Reed, R. M. 2009. Pore Networks and Fluid Flow in Gas Shales. Presented at SPE Annual Technical Conference and Exhibition, New Orleans, 4–7 October. SPE-124253-MS. http://dx.doi.org/10.2118/124253-MS.