Efficient Alpha-Blended Windows: DirectComposition

You can find sample source code for this article over on my GitHub.
DirectComposition, introduced in Windows 8, interacts directly with the DWM to composite your window using the GPU: you no longer have to read your image back to system memory like you have to do for Layered Windows, and the composition is no longer done in software. As you might expect, this provides a huge performance win, especially when using per-pixel alpha values. Even better, DirectComposition gives you far more choice of output formats than UpdateLayeredWindow, no longer shackles you to DIBs with their non-existent alpha channel support, and it slots in seamlessly next to code using Direct3D 11/12 or Direct2D. (OpenGL or Vulkan users, I am not aware of a way to make this work with those APIs.) The steps to enable DirectComposition are fairly trivial:

Step 0: Link with DirectComposition

Set up your project to link against dcomp.lib. Then, #include <dcomp.h>.

Step 1: Disable the Window's Redirection Bitmap

By default, Windows will always render everything in the client area to an off-screen bitmap and use that image in the composition engine. However, we don't want that any more: we want to blast our pixels into a DXGI surface and have the compositor use that. As such, the first thing we need to do is eliminate that redundant buffer using the Windows Extended Style WS_EX_NOREDIRECTIONBITMAP:
HWND hwnd = CreateWindowEx(
    WS_EX_NOREDIRECTIONBITMAP, // note: no WS_EX_LAYERED!
    L"MyWindowClass", 
    L"My Transparent Window",
    WS_OVERLAPPEDWINDOW,
    CW_USEDEFAULT, CW_USEDEFAULT,
    width, height,
    nullptr, nullptr,
    hInstance,
    nullptr);
Note that I am not using WS_EX_LAYERED here. The result of this code is a normal window but with a crucial difference: the client area is completely gone!

Step 2: Configure your Swap Chain

To prepare your swap chain for DirectComposition, you only need two small changes to your regular swap chain set-up. The first is you need to set the AlphaMode property of your DXGI_SWAP_CHAIN_DESC1. This describes to DXGI how you want the swap chain to be blended. We're going to use DXGI_ALPHA_MODE_PREMULTIPLIED.
DXGI_SWAP_CHAIN_DESC1 swapChainDesc = {};
swapChainDesc.Width = backBufferWidth;
swapChainDesc.Height = backBufferHeight;
swapChainDesc.Format = backBufferFormat;
swapChainDesc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
swapChainDesc.BufferCount = m_backBufferCount;
swapChainDesc.SampleDesc.Count = 1;
swapChainDesc.SampleDesc.Quality = 0;
swapChainDesc.Scaling = DXGI_SCALING_STRETCH;
swapChainDesc.SwapEffect = DXGI_SWAP_EFFECT_FLIP_DISCARD;
swapChainDesc.AlphaMode = DXGI_ALPHA_MODE_PREMULTIPLIED; // <--
For a normal Direct3D/Direct2D application you would create a swap chain against a traditional HWND or Windows Store's CoreWindow. However, we're going to use the little-known third sibling of the IDXGIFactory2::Create* APIs: CreateSwapChainForComposition.
ComPtr<IDXGISwapChain1> swapChain;
ThrowIfFailed(m_dxgiFactory->CreateSwapChainForComposition(
    m_commandQueue.Get(), // <- DirectX 12 specific; see below
    &swapChainDesc,
    nullptr,
    swapChain.GetAddressOf()));
The code above is specific to DirectX 12, so we pass an instance of a ID3D12CommandQueue object to this routine (just like you do with all CreateSwapChain variants.) However, if you are using Direct3D 11 or Direct2D, you should use a ID3D11Device pointer here.

Step 3: Set up DirectComposition

Now all we need to do is connect our swap chain output to Windows' composition engine, which is where DirectComposition comes into play. Next to your swap chain and device, store pointers to these objects somewhere:
// Swap chain objects.
Microsoft::WRL::ComPtr<IDXGIFactory4>       m_dxgiFactory;
Microsoft::WRL::ComPtr<IDXGISwapChain3>     m_swapChain;
// DirectComposition objects.
Microsoft::WRL::ComPtr<IDCompositionDevice> m_dcompDevice;
Microsoft::WRL::ComPtr<IDCompositionTarget> m_dcompTarget;
Microsoft::WRL::ComPtr<IDCompositionVisual> m_dcompVisual;
And here's the code. I'll break it down afterwards:
// (1) Create the DirectComposition device
ThrowIfFailed(DCompositionCreateDevice(
    nullptr, // <- DirectX 12 specific; see below
    IID_PPV_ARGS(m_dcompDevice.ReleaseAndGetAddressOf())));

// (2) Create a DirectComposition target associated with the window (pass in hWnd here)
ThrowIfFailed(m_dcompDevice->CreateTargetForHwnd(hWnd,
     true,
     m_dcompTarget.ReleaseAndGetAddressOf()));

// (3) Create a DirectComposition "visual"
ThrowIfFailed(m_dcompDevice->CreateVisual(m_dcompVisual.ReleaseAndGetAddressOf()));

// (4) Associate the visual with the swap chain
ThrowIfFailed(m_dcompVisual->SetContent(swapChain.Get()));

// (5) Set the visual as the root of the DirectComposition target's composition tree
ThrowIfFailed(m_dcompTarget->SetRoot(m_dcompVisual.Get()));
ThrowIfFailed(m_dcompDevice->Commit());
In step (1) we initialize DirectComposition and instantiate a IDCompositionDevice object. Once again we have to make a slight change for non-DirectX 12 implementations. Where I pass nullptr to DCompositionCreateDevice you should pass a pointer to an IDXGIDevice object. You can get this from your D3D11 device:
ComPtr<IDXGIDevice> dxgiDevice;
HR(m_d3dDevice.As(&dxgiDevice));
ThrowIfFailed(DCompositionCreateDevice(
    dxgiDevice.Get(),
    IID_PPV_ARGS(m_dcompDevice.ReleaseAndGetAddressOf())));
Step (2) creates a composition target that we associate with our window via an HWND. We pass true as the first parameter ('topmost') to make sure that this target is displayed on top of any other composition elements associated with the window. Step (3) creates a composition visual element, which represents a node in the composition tree. In step (4) we set our swap chain to be the renderable content of the visual using the SetContent API. Finally, in step (5) we set the root of the composition tree to our visual and submit all our DirectComposition commands using Commit. The end result? Let's take a look: The top row of triangles have been blended with the desktop with varying alpha values. (The bottom row of triangles are deliberately fully opaque.) You can find sample source code for this over on my GitHub.