Data Transfer in Vulkan

In this post, we will discuss how data is transferred between hosts and devices.

In Vulkan, there are two main types of resources: VkBuffer and VkImage. Each resource has a VkDeviceMemory associated with it, which is bound using either vkBindBufferMemory or vkBindImageMemory.`

To upload or transfer data to GPU memory for compute or rendering, we typically use an intermediate data layer called a staging buffer. A staging buffer is not a special hardware type; it’s simply a VkBuffer created with memory properties that make it Host (CPU) Visible, allowing us to upload or read data from the host. Vulkan provides a way to write to a VkBuffer through vkMapMemory, which lets us access a specified range of VkDeviceMemory via a pointer. Once mapped, we can use memcpy() to write data directly.

As illustrated above, staging buffers usually reside in system RAM for discrete GPUs, whereas on integrated GPUs, the same physical RAM is shared between the CPU and GPU. For a staging buffer to be host-visible, its memory must be allocated with VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT. The second flag is optional but ensures that the host and device see updated data without explicitly calling vkFlushMappedMemoryRanges or vkInvalidateMappedMemoryRanges.

After writing data to a staging buffer, we can transfer it to GPU memory. The most common Vulkan commands for this are vkCmdCopyBuffer and vkCmdCopyBufferToImage. For VkDeviceMemory allocated on the device side, the recommended property is VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT.

To retrieve compute or rendering results from the GPU, we can use vkCmdCopyImageToBuffer to copy data from a GPU VkImage into a staging buffer that is CPU-visible, making it easy to read the results in C++. Likewise, vkCmdCopyBuffer can be used to copy data between a staging buffer and a GPU buffer, as long as both are of type VkBuffer.

There is one special case. Data doesn’t have to be in GPU memory for shader programs to fetch the resources. For example, as the figure shown above, staging buffer can also be used directly as descriptor sets for shaders to use. In other words, we can bind a host-visible “staging” buffer directly to shaders as a uniform/storage/vertex/index buffer.

Vulkan lets shaders read from any buffer as long as: the buffer was created with the right usage flags (UNIFORM_BUFFER_BIT, STORAGE_BUFFER_BIT, VERTEX_BUFFER_BIT, etc.).

GPU can access host-visible memory via PCIe. Ideally, it is not a good idea to directly bind staging buffer for shaders to use, though it should be fine for small amount of data(like UBO).

---

More Vulkan visualizations: https://chuzcjoe.github.io/CGV/cgv-vulkan-visualization/