June 17th, 2023
Ok, let's tackle the Draw method. It's going to be considerably longer than previously, because it's now handling a lot of state changes and different draw calls.
First, we need a couple of helpers. Direct3D11 doesn't allow you to use one texture as both the input and output at the same time. So, if say a layer's texture is set as the shader resource view, it can't simultaneously be the render target view. To work around this, we can create a couple of intermediate textures and make RTVs and SRVs for those. You could put this initialization anywhere, here it's in the CreateSwapChain method.
private ID3D11Texture2D intermediateTexture1;
private ID3D11Texture2D intermediateTexture2;
private ID3D11RenderTargetView intermediateRTV1;
private ID3D11RenderTargetView intermediateRTV2;
private ID3D11ShaderResourceView intermediateSRV1;
private ID3D11ShaderResourceView intermediateSRV2;
public void CreateSwapChain()
{
...
Texture2DDescription intermediateTextureDescription = new Texture2DDescription()
{
Width = (int)SwapChainCanvas.Width,
Height = (int)SwapChainCanvas.Height,
MipLevels = 1,
ArraySize = 1,
Format = Format.B8G8R8A8_UNorm,
SampleDescription = new SampleDescription(1, 0),
Usage = ResourceUsage.Default,
BindFlags = BindFlags.RenderTarget | BindFlags.ShaderResource,
CPUAccessFlags = CpuAccessFlags.None,
MiscFlags = ResourceOptionFlags.None
};
intermediateTexture1 = device.CreateTexture2D(intermediateTextureDescription);
intermediateTexture2 = device.CreateTexture2D(intermediateTextureDescription);
intermediateRTV1 = device.CreateRenderTargetView(intermediateTexture1);
intermediateRTV2 = device.CreateRenderTargetView(intermediateTexture2);
intermediateSRV1 = device.CreateShaderResourceView(intermediateTexture1);
intermediateSRV2 = device.CreateShaderResourceView(intermediateTexture2);
}In this part of the draw method, we're using two shaders to draw instances of brush strokes.
private Color4 colorTransparent = new Color4(0.0f, 0.0f, 0.0f, 0.0f);
private void Draw()
{
// 1. Draw Instances to intermediateRTV1
deviceContext.VSSetShader(vertexInstances, null, 0);
deviceContext.PSSetShader(pixelInstances, null, 0);
deviceContext.IASetInputLayout(instancesInputLayout);
deviceContext.ClearRenderTargetView(intermediateRTV1, colorTransparent);
deviceContext.OMSetRenderTargets(intermediateRTV1, depthStencilView);
...
}The variables vertexInstances and pixelInstances point to our old shaders, now renamed to VertexInstances.hlsl and PixelInstances.hlsl. Nothing needs to change there, because they're still just handling the instance drawing. The strokes are now drawn onto an intermediate render target, which is a temporary canvas that we'll use to combine everything later.
int vertexStride = Marshal.SizeOf<Vertex>();
int instanceStride = Marshal.SizeOf<InstanceData>();
int offset = 0;
deviceContext.IASetVertexBuffer(0, instanceVertexBuffer, vertexStride, offset);
deviceContext.IASetVertexBuffer(1, instanceBuffer, instanceStride, offset);
deviceContext.IASetIndexBuffer(instanceIndexBuffer, Format.R32_UInt, 0);
Let's move the buffer setters from update to right here for clarity. Then, set shader resource to point to the brush texture and draw the instances for this frame.
deviceContext.PSSetShaderResource(0, brushSRV);
deviceContext.PSSetSampler(0, samplerState);
deviceContext.DrawIndexedInstanced(quadIndices.Length, brushStamps.Count, 0, 0, 0);It's worth noting, that the instance buffer does not get cleared at any point. The maxInstances variable at CreateBuffers determines the size of the buffer and the new instances keep getting written over the old ones. Depending on multiple variables, like the mouse polling rate, framerate of the software, speed of the motion etc. you might be able to draw more than 20 instances (current maxInstances value) on the canvas per frame. The maximum I could do with these settings was four per frame, but that's something you'd want to keep an eye on and make adjustments if needed.
Now that we have our brush strokes drawn, we want to merge them with the existing content of the current layer.
// 2. Merge new instances with old layer texture
deviceContext.VSSetShader(vertexMerger, null, 0);
deviceContext.PSSetShader(pixelMerger, null, 0);
deviceContext.IASetInputLayout(mergerInputLayout);
deviceContext.IASetVertexBuffer(0, fullscreenVertexBuffer, vertexStride, 0);
deviceContext.IASetIndexBuffer(fullscreenIndexBuffer, Format.R32_UInt, 0);
For this, we use another pair of shaders. Let's take a look at the VertexMerger.hlsl
struct VertexInput
{
float3 position : POSITION0;
float2 uv : TEXCOORD0;
};
struct VertexOutput
{
float4 position : SV_POSITION;
float2 uv : TEXCOORD0;
};
VertexOutput VS(VertexInput input)
{
VertexOutput output;
output.position = float4(input.position, 1.0f);
output.uv = input.uv;
return output;
}We take the fullscreen quad position and UVs and pass them along to the pixel shader. The only change we do is type conversion on the position from float3 to float4, because it's easy to do here. The PixelMerger.hlsl does the combination.
Texture2D tex1 : register(t0);
Texture2D tex2 : register(t1);
SamplerState sam : register(s0);
struct PixelInput
{
float4 position : SV_POSITION;
float2 uv : TEXCOORD0;
};
float4 PS(PixelInput input) : SV_TARGET
{
float2 uv = input.uv;
float4 color1 = tex1.Sample(sam, uv);
float4 color2 = tex2.Sample(sam, uv);
float4 finalColor = color2 + color1 * (1 - color2.a);
return finalColor;
}Here, color2 is the color of the top layer, color1 is the color of the bottom layer, and color2.a is the alpha value of the top layer. The final formula calculates the color as a weighted average of the source and destination colors, with the weights determined by the top layer alpha.
So, in the Draw method, after we've set the shaders, input layout, and the buffers, we set the shader resource views to point to null. This is good practice, before setting new active render target views, so that there is no circular dependency even if somebody goes and changes the targets or resources to something else in the code.
deviceContext.PSSetShaderResource(0, null);
deviceContext.PSSetShaderResource(1, null);This combined image is then saved into a second intermediate render target.
deviceContext.OMSetRenderTargets(intermediateRTV2, depthStencilView);
if (brushStamps.Count > 0)
{
deviceContext.PSSetShaderResource(0, layers[activeLayer].ShaderResourceView);
deviceContext.PSSetShaderResource(1, intermediateSRV1);
deviceContext.DrawIndexed(6, 0, 0);
using (ID3D11Resource resource = layers[activeLayer].RenderTargetView.Resource)
{
using (ID3D11Resource resource2 = intermediateRTV2.Resource)
{
deviceContext.CopyResource(resource, resource2);
}
}
deviceContext.ClearRenderTargetView(intermediateRTV2, colorTransparent);
}We check to see if there are new brush stamps to add, befor doing another draw call and copy.
With our updated layer ready, the next step is to merge all layers of our painting together. Note that we're not changing the shaders. We're still using the same merger as before.
// 3. Merge Layers together
for (int i = 0; i < layers.Count; i++)
{
deviceContext.PSSetShaderResource(0, null);
deviceContext.PSSetShaderResource(1, null);
if (i % 2 == 0)
{
deviceContext.ClearRenderTargetView(intermediateRTV1, colorTransparent);
deviceContext.OMSetRenderTargets(intermediateRTV1, depthStencilView);
deviceContext.PSSetShaderResource(0, intermediateSRV2);
deviceContext.PSSetShaderResource(1, layers[i].ShaderResourceView);
deviceContext.DrawIndexed(6, 0, 0);
}
else
{
deviceContext.ClearRenderTargetView(intermediateRTV2, colorTransparent);
deviceContext.OMSetRenderTargets(intermediateRTV2, depthStencilView);
deviceContext.PSSetShaderResource(0, intermediateSRV1);
deviceContext.PSSetShaderResource(1, layers[i].ShaderResourceView);
deviceContext.DrawIndexed(6, 0, 0);
}
}We're not adding any new data in this step. Instead, we are combining existing layers into a single image. We alternate the render targets we're drawing to, to avoid trying to read and write from the same resource simultaneously, which isn't allowed in Direct3D11.
The result of all this work is a final image that represents all layers of our painting merged together. The nested using blocks are utilized to prevent the CopyResource call creating new references to the textures on every frame.
// 4. Copy merged layers to backbuffer
if ((layers.Count - 1) % 2 == 0)
{
using (ID3D11Resource resource = renderTargetView.Resource)
{
using (ID3D11Resource resource2 = intermediateRTV1.Resource)
{
deviceContext.CopyResource(resource, resource2);
}
}
}
else
{
using (ID3D11Resource resource = renderTargetView.Resource)
{
using (ID3D11Resource resource2 = intermediateRTV2.Resource)
{
deviceContext.CopyResource(resource, resource2);
}
}
}This final image is what we want to show on the screen. To do that, we copy the result into the backbuffer of the swap chain.
Lastly, we have a bit of cleanup to do.
brushStamps.Clear();
swapChain.Present(1, PresentFlags.None);We clear the brush strokes for the next frame, so that they're not being drawn multiple times. Then, we call Present on the swap chain, which pushes our backbuffer to the front, updating the screen with our newly drawn image. This completes a single frame of our application, and the whole process begins anew for the next frame.
Visual Studio project:
d3dpaint-pt3.zip (32 KB)
D3DPaint part 3 in GitHub