Sfx: a generic effect compiler for shaders

So Microsoft, Nvidia and the rest used to support effect files: a text source file that contained multiple shaders, but also "techniques" and "passes", where each pass can have state specified: blending, rasterization etc.

Some time ago, for reasons I guess of supply and demand, fx fell out of fashion. Microsoft still provides the D3D11 version of its Effects library as open source here, and this reads binary output that the fxc tool can create. But Fxc is being replaced by by this, which doesn't support effects. Nvidia's Tristan Lorach proposed a new framework, nvFX (pdf), but I don't think it's under active development.

D3D12 has no effect support. So we need to add it.

My approach at Simul is called Sfx. The idea is to take an initial file that's compatible with Microsoft's HLSL effect format, with the code written in HLSL. Sfx will extract the passes and compile all the relevant shaders by building smaller individual shader source files and calling an external compiler. A small json file will specify which compiler to use, and various other parameters to allow translation from HLSL to whatever language the compiler expects.

For example, HLSL.json looks like:

{
  "compiler": "C:/Program Files (x86)/Windows Kits/10/bin/x64/fxc.exe",
  "defaultOptions": "/T {shader_model} /nologo",
  "sourceExtension": "hlsl",
  "outputExtension": "cso",
 "outputOption": "/Fo",
 "entryPointOption":  "/E{name}",
  "multiplePixelOutputFormats": false
}

So Sfx should be compiler-independent. It outputs two things: a text file with the .sfxo extension, and a number of platform-specific shader binaries. The sfxo looks like this:

SFX
texture fontTexture 2d read_only 0 single
SamplerState clampSamplerState 9,LINEAR,CLAMP,CLAMP,CLAMP,
SamplerState cmcNearestSamplerState 13,POINT,CLAMP,MIRROR,CLAMP,
RasterizerState RenderNoCull (false,CULL_NONE,0,0,false,FILL_SOLID,true,false,false,0)
RasterizerState wireframeRasterizer (true,CULL_NONE,0,0,false,FILL_WIREFRAME,false,false,false,0)
BlendState AlphaBlendRGB false,(true),1,1,4,5,0,0,(7)
DepthStencilState DisableDepth false,0,4
group 
{
 technique backg
 {
  pass p0
  {
   rasterizer: RenderNoCull
   depthstencil: DisableDepth 0
   blend: AlphaBlendRGB (0,0,0,0) 4294967295
   vertex: font_FontVertexShader_vv.cso,(),(),()
   pixel: font_FontPixelShader.cso,(),(),()
  }
 }
}

This stands to improve over time. In this case, we've taken an sfx file containing this definition:

VertexShader vs = CompileShader(vs_4_0, FontVertexShader());

technique text
{
    pass p0
    {
  SetRasterizerState( RenderNoCull );
  SetDepthStencilState( DisableDepth, 0 );
  SetBlendState(AddBlendRGB,vec4( 0.0, 0.0, 0.0, 0.0), 0xFFFFFFFF );
         SetGeometryShader(NULL);
  SetVertexShader(vs);
  SetPixelShader(CompileShader(ps_4_0,FontPixelShader()));
    }
}

so we've compiled FontPixelShader() and FontVertexShader() into the cso files: this example is for HLSL. As we extend Sfx to other languages, the json definition in particular will become more complex - possibly using regexes to specify how HLSL is translated into other C-style shader languages.