backend/drm: introduce wlr_drm_bo_handle_table

Using GBM to import DRM dumb buffers tends to not work well. By
using GBM we're calling some driver-specific functions in Mesa.
These functions check whether Mesa can work with the buffer.
Sometimes Mesa has requirements which differ from DRM dumb buffers
and the GBM import will fail (e.g. on amdgpu).

Instead, drop GBM and use drmPrimeFDToHandle directly. But there's
a twist: BO handles are not ref'counted by the kernel and need to
be ref'counted in user-space [1]. libdrm usually performs this
bookkeeping and is used under-the-hood by Mesa.

We can't re-use libdrm for this task without using driver-specific
APIs. So let's just re-implement the ref'counting logic in wlroots.
The wlroots implementation is inspired from amdgpu's in libdrm [2].

Closes: https://github.com/swaywm/wlroots/issues/2916

[1]: https://gitlab.freedesktop.org/mesa/drm/-/merge_requests/110
[2]: https://gitlab.freedesktop.org/mesa/drm/-/blob/1a4c0ec9aea13211997f982715fe5ffcf19dd067/amdgpu/handle_table.c

1 files changed, 24 insertions, 0 deletions

diff --git a/include/backend/drm/bo_handle_table.h b/include/backend/drm/bo_handle_table.h
new file mode 100644
index 00000000..d086df45
--- /dev/null
+++ b/include/backend/drm/bo_handle_table.h
@@ -0,0 +1,24 @@
+#ifndef BACKEND_DRM_BO_HANDLE_TABLE_H
+#define BACKEND_DRM_BO_HANDLE_TABLE_H
+
+/**
+ * Table performing reference counting for buffer object handles.
+ *
+ * The BO handles are allocated incrementally and are recycled by the kernel,
+ * so a simple array is used.
+ *
+ * This design is inspired from amdgpu's code in libdrm:
+ * https://gitlab.freedesktop.org/mesa/drm/-/blob/1a4c0ec9aea13211997f982715fe5ffcf19dd067/amdgpu/handle_table.c
+ */
+struct wlr_drm_bo_handle_table {
+	size_t *nrefs;
+	size_t len;
+};
+
+void drm_bo_handle_table_finish(struct wlr_drm_bo_handle_table *table);
+bool drm_bo_handle_table_ref(struct wlr_drm_bo_handle_table *table,
+	uint32_t handle);
+size_t drm_bo_handle_table_unref(struct wlr_drm_bo_handle_table *table,
+	uint32_t handle);
+
+#endif