LLMs and shaders #
In the last post I described how chat interfaces to LLMs are giving a limited view of what the LLMs could do. In this post I wanted to compare this to graphics pipelines.
Graphics pipelines used to be "fixed function", where the graphics system (such as Silicon Graphics hardware) would offer you the graphics features:
- you always got a matrix multiply (translate, scale, rotate, perspective, etc.)
- you always got gouraud shading
- you always got their standard lighting model (ambient, diffuse, specular, emission, shininess)
- you got to choose up to 8 lights with glLight() with a handful of styles
- you always got a color per object, but later they offered a texture lookup
- you got to control the fog level with glFog() (density, color, etc.)
But what if you wanted a different model like phong shading? Too bad! What if you wanted any effect other than fog? Too bad! What if you wanted more than 8 lights? Too bad! What if you wanted 2 textures? Too bad! At least until they added a feature to enable a second texture lookup. If you wanted anything else, you needed to implement your own graphics pipeline and give up on hardware acceleration.
Over time, these fixed features were replaced by "shaders". The vertex shader would handle the overall structure with matrix multiply, clipping, fog, colors. Once we could control it, people came up with cloth, grass, water, terrain, animation, procedural geometry, and lots of other things. The fragment shader would handle fine details with texture lookup and lighting. Once we could control it, people came up with custom lighting, organic textures, animation, procedural art, and lots of other things.
Labels: speculation, structure