backend/drm: get rid of BO handle table

The BO handle table exists to avoid double-closing a BO handle,
which aren't reference-counted by the kernel. But if we can
guarantee that there is only ever a single ref for each BO handle,
then we don't need the BO handle table anymore.

This is possible if we create the handle right before the ADDFB2
IOCTL, and close the handle right after. The handles are very
short-lived and we don't need to track their lifetime.

Because of multi-planar FBs, we need to be a bit careful: some
FB planes might share the same handle. But with a small check, it's
easy to avoid double-closing the same handle (which wouldn't be a
big deal anyways).

There's one gotcha though: drmModeSetCursor2 takes a BO handle as
input. Saving the handles until drmModeSetCursor2 time would require
us to track BO handle lifetimes, so we wouldn't be able to get rid
of the BO handle table. As a workaround, use drmModeGetFB to turn the
FB ID back to a BO handle, call drmModeSetCursor2 and then immediately
close the BO handle. The overhead should be minimal since these IOCTLs
are pretty cheap.

Closes: https://github.com/swaywm/wlroots/issues/3164

1 files changed, 0 insertions, 2 deletions

diff --git a/backend/drm/backend.c b/backend/drm/backend.c
index 9ae4dbf0..5e01eddf 100644
--- a/backend/drm/backend.c
+++ b/backend/drm/backend.c
@@ -52,8 +52,6 @@ static void backend_destroy(struct wlr_backend *backend) {
 	wl_list_remove(&drm->dev_change.link);
 	wl_list_remove(&drm->dev_remove.link);
 
-	drm_bo_handle_table_finish(&drm->bo_handles);
-
 	if (drm->parent) {
 		finish_drm_renderer(&drm->mgpu_renderer);
 	}