Render Pipeline System

This document outlines Trial's render pipeline system and details how rendering behaves internally and externally.

1. Parts

The system deals with objects of the following types:

• container
  Containers that are not themselves renderable, but may contain other renderables and provide positioning information.

• renderable
  An object to be drawn in the scene in some way. It is not specified how this object is drawn.

• shader-pass
  A rendering pass. Note that one pass may draw many objects, or none, such as for a post-processing pass.

• shader-pipeline
  A bundle of shader passes into a consistent pipeline that produces a set of textures or, more typically, renders to the screen.

2. Lifecycle

In order to understand rendering, we have to look at two distinct cases where the renderer needs to change its state:

1. Setup
   After the scene is set up all of the renderable objects contained within need to be registered with the shader pipelines, so that effective shader programs can be computed and allocated. This typically occurs as a bulk change within a loading context.

2. Runtime

   While the game is running, dynamic changes may add or remove renderable entities. Renderables may also come in or out of view, or for other reasons start or stop rendering. In this case the shader pipelines need to adapt appropriately to cull their draw calls.

To actually perform the rendering, each shader pipeline simply iterates through each shader pass in topological order. Each pass then conceptually proceeds as follows:

1. Allocate the objects that should be rendered for the current frame in a frame sequence:
   1. If the frame is considered not to have changed, skip
   2. For every renderable that is tied to the pass:

   1. If the renderable is considered visible within the pass' context (may be culled for occlusion, frustum, or other reasons):

   1. Store the object and its shader-program for the pass in the frame

   3. Sort the frame to ensure objects are drawn in the correct order

2. Perform the render operations needed to render the frame:

   1. Bind the pass' textures

   2. Iterate over the renderables in order of the frame:

      1. Bind the object's textures and transforms

      3. Call render with the associated shader-program

Important to note here are that construct-frame is called for every frame, meaning the potential culling and ordering is called automatically once per frame to ensure consistency. Passes may elide changing the frame, but by default without optimisations they will not.

Additionally, only renderables that are tied to the pass will be considered for rendering within the pass' context at all. Meaning that any object first needs to be tied to the pass by entering it. An object may also be dynamically removed by using leave.

A pass only distinguishes between 1. Setup and 2. Runtime above by whether a container is passed to the pass or a single renderable. Entering and leaving renderables potentially causes allocations to happen, as new shader programs may need to be compiled. More details on this are illustrated in §3.

The specifics of frame construction and rendering are elaborated upon in §4.

3. Allocation

Broadly passes are distinguished in the following manner:

1. Post-processing passes
   These passes contain all the rendering logic they need in themselves. The scene graph does not interact with them at all.

2. Per-object passes

   These passes share some rendering logic with objects from the scene graph, and render control usually lies with the actual objects, rather than with the pass itself.

For passes of type 1 there isn't much to discuss. The pass manages its own shader and geometry logic, and allocates that at the beginning when the pass data in general is allocated.

For type two, things are more interesting, as the pass needs to potentially allocate a shader program for the object, and needs to appropriately manage the lifetime of this program. The pass also needs to figure out the "effective shader sources" for each object tied to it.

When an object is entered into a pass, whether on its own or through a container, it proceeds as follows:

1. If the object is already in the renderable table, skip

2. Check if the object's class has an allocated shader program. If not:

   Note that this test always fails if the object is a dynamic-renderable

   1. Construct a shader program calling make-pass-shader-program, which:
      1. For each shader type:

      1. Call compute-shader with the shader type, the pass, and the object

      2. If the list of shaders returned is not empty:

      3. Merge the list of shaders with GLSL-toolkit and construct the shader object

      2. Merge the list of effective buffers for both pass and object

      3. Construct the shader program using the list of shaders and buffers

   2. Check if an identical shader program exists. If not:

      1. Generate the shaders and shader program

      2. Associate a load trigger with the program to run update-uniforms once loaded

   3. Tie the object's class to the generated shader program

3. Increase the refcount of the shader program

4. Associate the object with its shader program in the renderable table

If the object was entered on its own, the staging area is now committed. Otherwise the commit is deferred until all resource construction is complete. Note that this process is very cheap for instances of classes that have already been entered once before.

When an object is entered from a collection, it also first calls object-renderable-p on the object and pass, to preemptively cull objects that should not be rendered on the given pass.

When an object leaves a pass, it proceeds as follows:

1. If an associated shader program is found in the renderable table:
   1. Reduce the refcount of the associated shader program
   2. Remove the object from the renderable table

Deallocation of shader programs is not done even if the refcount reaches zero, as this could lead to bad dynamic allocations in the future when the same object type is entered again, such as can often happen for effects or other dynamic object types.

Instead, deallocation is deferred to when the pass is next staged, at which point the programs with a zero refcount are culled again, and not staged, leading to a deallocation on commit.

4. Rendering

The rendering for post-processing passes is fairly straight-forward, simply calling render on the pass itself with the singular shader-program it has allocated for its operations.

Things only become interesting for per-object passes, as they first need to construct a sequence according to which the objects are rendered, and which also excludes objects that should be culled for the current frame. As described briefly in §2, this proceeds in two steps, in the first building the frame sequence, and in the second actually rendering that sequence.

In the first step, construct-frame is called, which proceeds as follows:

1. If entities were added or removed, or visibility state of entities changed:
   The visibility check especially is dependent on the specific rendering pass used and the camera model used. In many cases, recomputing the frame can be avoided as visibility of entities does not change frequently
   1. Call map-visible on the pass' camera and scene, and iterate over every visible object:

   1. If the object has an entry in the renderable table:

   1. Store the object and its shader-program for the pass in the frame

3. Sort the frame by the sort-frame

   The specifics of how this sorting occurs is up to the pass internals, but suffice to say that in most cases the cost can be held down a lot, and in cases where it's clear that order has not changed ahead of time, elided completely

Most of the magic here is in the function map-visible. However, the precise operation of this is up to the individual camera models, which know how to properly perform frustum culling. The pass may also specialise camera to return NIL in order to iterate over every object in the scene, and/or specialise scene to return the pass itself in order to iterate over every object that was entered into the pass. Doing so can be useful to handle rendering of objects not part of the scene graph, or objects that would by default be culled from visibility.

In the second step, render-frame is called, which proceeds as follows:

1. Call bind-textures on the pass
   This causes necessary texture units that the pass needs in its shader fragments to get bound appropriately. We only have to do this once per pass, so best to do it here

2. Iterate over the renderables in order of the frame:

   1. Call apply-transforms
      This sets up the transform matrices as required for the specific object in play. The transform matrices are reset to their prior values after each object completes its render step

   2. Call bind-textures

      Same as before, but this time the object's textures are bound

   3. Call render with the associated shader-program

      This performs the actual draw call, as well as any potentially needed setting of uniforms and so forth. Note that the setting of many uniforms should not be done here, as the uniforms are set once per frame per object, and many of the uniform values do not change that frequently

For per-object-passes, render-frame is called as part of the pass' render method, after which the pass may perform extra steps, such as blitting the framebuffer to the screen.