wp_list_categories walker without links on categories that have subcategories (to make a nested dropdown menu)

I’d like to make a custom walker that works with wp_list_categories to generate a menu of nested uls using my category, sub-category, and sub-sub-category(?) structure. Any category that has sub-categories can’t be a link so it can be clicked in order to reveal the sub-categories (or sub-sub-categories).

Here’s what I want (only items with a 🔗 should be links):

See also this CodePen demo with the hard-coded HTML I’d like to generate, ideally with a custom walker for wp_list_categories.

My menu system is at most three levels deep, and categories without descendants could be anywhere. The reason I’d really like to use wp_list_categories is so my editors have the flexibility to reorder categories. If I hard-coded in each top-level category they can’t reorder them freely using WP’s built-in category editor.

Goal: Right now, wp_list_categories does output my categories properly. I just need to figure out how to override the code that is making category titles with subcategories be links. I need them to just be plain text.

I’m trying to make a custom walker myself but can’t seem to figure out where it decides if a category is a sub (or a “sub of a sub”) in order to omit links where they’re not wanted.