Build your own low-level game engine in Metal!
Metal is a unified application programming interface (API) for the graphics processing unit, or GPU. It’s unified because it applies to both 3D graphics and data-parallel computation paradigms. Metal is a low-level API because it provides programmers near-direct access to the GPU. Finally, Metal is a low-overhead API because it reduces the central processing unit (CPU) cost by multi-threading and pre-compiling resources.
But beyond the technical definition, Metal is the most appropriate way to use the GPU’s parallel processing power to visualize data or solve numerical challenges. It’s also tailored to be used for machine learning, image / video processing or, as this book describes, graphics rendering.
This book will introduce you to low-level graphics programming in Metal – Apple’s framework for programming on the graphics processing unit (GPU). As you progress through this book, you’ll learn many of the fundamentals that go into making a game engine and gradually putting together your own engine. Once your game engine is complete, you’ll be able to put together 3D scenes and program your own simple 3D games. Because you’ll have built your 3D game engine from scratch, you’ll be able to customize every aspect of what you see on your screen.
This book is for intermediate Swift developers interested in learning 3D graphics or gaining a deeper understanding of how game engines work.
Before You Begin
This section tells you a few things you need to know before you get started, such as what you’ll need for hardware and software, where to find the project files for this book and more.
Section I: Beginning Metal
It takes a wealth of knowledge to render a simple triangle on the screen or animated game characters. This section will guide you through the necessary basics of vertex wrangling, lighting, textures and creating a game scene. If you’re worried about the math, do not be! Although computer graphics are highly math-intensive, each chapter explains everything you need, and you’ll get experience creating and rendering models.
In this chapter, you’ll get an introduction to Metal as you render your first primitive object. You’ll also learn why you would use Metal over other graphics frameworks.
In this chapter, you’ll dive into the world of 3D models, learning how to work with them in Blender, and how to render them on screen with Metal.
In this chapter, you’ll go further with Metal by creating a macOS app from scratch, where you’ll learn how the rendering pipeline works to create the beautiful graphics you see onscreen.
In this chapter, you’ll get a look at the first of two programmable stages in Metal, the vertex stage and the vertex function.
In this chapter, you’ll become a matrix master as you learn what matrices can do for you and how to manipulate them painlessly.
In this chapter, you’ll get an introduction to some of the mathematical concepts used in Metal, and how to find your way around the scene using Metal’s coordinate system.
In this chapter, you’ll learn about the fragment function, which is responsible for returning a color for each fragment that successfully passes through the rasterizer.
In this chapter, you’ll learn about UV coordinates, texturing a model, samplers, mipmaps and the asset catalog.
In this chapter, you’ll learn how to manage scenes as you create three different cameras and add the ability to capture user input.
In this chapter, you’ll learn how to add professional lighting to your scenes.
Section II: Intermediate Metal
With the basics under your belt, you can move on to multi-pass rendering. You’ll add shadows and learn several new rendering techniques. Programming the GPU using compute shaders can be intimidating, so you’ll create particle systems to learn how fast multi-threaded solutions can be.
In this chapter, you’ll find out how to use material groups to describe a surface, and how to design textures for micro detail.
In this chapter, you’ll work with renderer passes as you add more complexity to your scene.
In this chapter, you’ll learn about shadows and how you can use them to bring a sense of depth to your scenes.
In this chapter, you’ll learn about deferred rendering, an approach that takes the quadratic runtime down to linear runtime when processing models.
In this chapter, you’ll learn how to handle scenes that contain many lights by using a technique known as tile-based deferred rendering.
In this chapter, you’ll play with GPU programming as you explore how to use the GPU in ways other than vertex rendering.
In this chapter, you’ll learn about generators, emitters, particle life, the compute kernel and more.
In this chapter, you’ll work with particles as you build and control a swarm that mimics natural behaviors.
Section III: Advanced Metal
In this section, you’ll learn many advanced features of Metal and explore realistic rendering techniques. You’ll animate characters, and also manage rendering your scenes on the GPU.
In this chapter, you’ll learn about tessellations as you create a detailed terrain using a small number of points.
In this chapter, you’ll learn how to use post-processing effects to take your scenes to the next level.
In this chapter, you’ll learn about skyboxes, cube maps, image-based lighting and more as you add some surrounding features to your virtual environment.
In this chapter, you’ll work with reflection and refraction to add a sense of realism to your scenes.
In this chapter, you’ll learn how to bring your scenes to life as you get started with animation. You’ll do so by bouncing a beachball around your scene.
In this chapter, you’ll continue your journey through animation by rendering a friendly skeleton.
In this chapter, you’ll learn how to properly manage your resources using argument buffers and heaps.
In this chapter, you’ll learn how to move the bulk of the rendering work to the GPU.
Section IV: Ray Tracing
In this section, you’ll trace rays to render objects with more realism than the rasterization techniques you’ve used up to now. As a bonus, you’ll also do some post-processing with Metal Performance Shaders. To wrap up, you’ll consider how best to profile and optimize your game.
In this chapter, you’ll learn how to do detailed ray tracing, ray casting, path tracing and ray marching to render your graphics.
In this chapter, you’ll learn about hard and soft shadows, ambient occlusion, screen space rendering and more.
In this chapter, you’ll learn about the rendering equation, reflection and refraction, the Fresnel effect, lens flare, subsurface scattering and more lighting details.
In this chapter, you’ll dive a bit deeper into the world of the Metal Performance Shaders (MPS) framework. The MPS kernels make use of data-parallel primitives that are written in such a way that they can take advantage of each GPU family’s characteristics.
In this chapter, you’ll learn more ways to optimize the performance of your game. You’ll understand how to categorize GPUs, how to manage memory, how to synchronize resources between the CPU and GPU, how to move your game on multiple threads and what are the best practices for a smooth rendering.
In this chapter, you’ll review the best practices to follow when creating your Metal apps – from choosing the right resolution to compressing textures, no best practice stone is unturned in this chapter.