In the olden times, I made static web pages. They sat quietly on tubular displays that squatted like toads on the desk of every office worker. Early humans grunted at these pages with mice and keyboards. That was it.
To write the simplest of web apps, the classic to-do list. I would design and build a dozen or so user interface elements, including the text input fields, buttons, labels, and checkboxes. To design and build a static page to-do list, I would create a dozen elements and write one-off code to handle keyboard and mouse input.
12 elements multiplied by 2 input types equals 24 considerations.
Then came smart-phones and tablets, and I would create elements that worked in three unique layouts and accepted touchscreen input along with the keyboards and mice.
12 elements in 3 layouts handling 3 input types means 108 considerations.
Not terribly easy, but possible.
Here come the headsets! And not just one kind of headset, no, there are two divergent screen technologies, see-through and pass-through, with different design and engineering constraints. Even worse, these headsets ship with a radical variety of input types. A person using a to-do list today could hold in their hands something as simple as a two button pointer or something as complex as a pair of tracked wands that offer eight buttons, two thumb-sticks, and two touch-pads. Oh, and the headsets also track individual eye direction, finger movements, and spoken commands.
12 elements in 3 layouts and in 2 headset types, handling 10 input types means that for this measly to-do app there are 600 considerations.
I require another path.
I have to encapsulate in opinionated tools with intelligent defaults the design complexity and engineering overhead of this combinatorial explosion of displays and inputs. To design, I need mental frames, templates, and testing rigs so that I'm no longer designing from scratch. To build, I need patterns, frameworks, and skeletons so that on day one of a build I start with a reactive and progressive experience.
Here are the mental frames and patterns that I'm working with today:
There are three display modes:
There are three control types: (in green)
Display modes and control types can be mixed and matched per design:
Finally, there are action maps. These declare routes from low level inputs like keyboard presses or voice commands to high level actions like "activate", "move", or "delete". These actions are fed into the controls so that the experience can react to the user's intent.
As I design and engineer wider* web applications, I'll think about and work within display modes, control types, and action maps. These three ideas represent the essential considerations of this paradigm.
This is freshly formed. I might be wrong. I will definitely be partially wrong.
So, I'm experimenting with an app framework that handles the common work of tracking these display modes, control types, and action maps. It communicates the transition between display modes so that application specific logic can be progressive and reactive.
No app framework can eliminate the inherent complexity of the wider web, but perhaps this one will exchange an impossible situation for one that allows me to design and code at human scale.
Using Three.js to build a reusable border geometry for spatial UIs!
🌸 Updated potassium style system (KSS), now with margins!
🌸 The path to the vNext
🌸 A couple of new spatial controls
🌸 A brief intro to the existing samples
🌸 Building UI components that work in flat, portal, and immersive display modes