Designing The Transreality Web

In the olden times, I made static web pages. They sat quietly on tubular displays that squatted like toads on the desk of every office worker. Early humans grunted at these pages with mice and keyboards. That was it.

To write the simplest of web apps, the classic to-do list. I would design and build a dozen or so user interface elements, including the text input fields, buttons, labels, and checkboxes. To design and build a static page to-do list, I would create a dozen elements and write one-off code to handle keyboard and mouse input.

12 elements multiplied by 2 input types equals 24 considerations.

Easy!

Then came smart-phones and tablets, and I would create elements that worked in three unique layouts and accepted touchscreen input along with the keyboards and mice.

12 elements in 3 layouts handling 3 input types means 108 considerations.

Not terribly easy, but possible.

Here come the headsets! And not just one kind of headset, no, there are two divergent screen technologies, see-through and pass-through, with different design and engineering constraints. Even worse, these headsets ship with a radical variety of input types. A person using a to-do list today could hold in their hands something as simple as a two button pointer or something as complex as a pair of tracked wands that offer eight buttons, two thumb-sticks, and two touch-pads. Oh, and the headsets also track individual eye direction, finger movements, and spoken commands.

12 elements in 3 layouts and in 2 headset types, handling 10 input types means that for this measly to-do app there are 600 considerations.

Impossible!

I require another path.

I have to encapsulate in opinionated tools with intelligent defaults the design complexity and engineering overhead of this combinatorial explosion of displays and inputs. To design, I need mental frames, templates, and testing rigs so that I'm no longer designing from scratch. To build, I need patterns, frameworks, and skeletons so that on day one of a build I start with a reactive and progressive experience.

Here are the mental frames and patterns that I'm working with today:

There are three display modes:

There are three control types: (in green)

Display modes and control types can be mixed and matched per design:


click for full size

Finally, there are action maps. These declare routes from low level inputs like keyboard presses or voice commands to high level actions like "activate", "move", or "delete". These actions are fed into the controls so that the experience can react to the user's intent.

As I design and engineer transreality web applications, I'll think about and work within display modes, control types, and action maps. These three ideas represent the essential considerations of this paradigm.

This is freshly formed. I might be wrong. I will definitely be partially wrong.

So, I'm experimenting with an app framework that handles the common work of tracking these display modes, control types, and action maps. It communicates the transition between display modes so that application specific logic can be progressive and reactive.

No app framework can eliminate the inherent complexity of the transreality web, but perhaps this one will exchange an impossible situation for one that allows me to design and code at human scale.

More posts

Recent posts

  • Augmented Reality is for Seeing Better

    A product designer friend recently asked me, "What is the fundamental capability that augmented reality provides?"

    The fundamental capability that current eye glasses provide is to see better. They help near sighted people see far away things. They help older eyes see things that are ...

  • Neighborhood XR

    Scenario: What is this bug?

    Stephen sits at the bus stop and sees a little bug with too many legs. He wonders what it is, so he pulls out his handset and browses over to the Tree of Life site, then clicks on the link ...

  • Link (Story 4 of 4)

    Hitchcock was awake. Starlight filtered through the forest canopy and dappled the walls of the cabin he shared with Lester. A trio of Clytemnestra’s beetle bots was resting in Lester's sleep-crumpled dreads, occasionally flicking their wings in response to dreams. Sometimes he forgot ...

  • Sphere (Story 3 of 4)

    Clytemnestra was in fragments. Her programs were spread across her stolen space ship, the station she just stole it from, and the small bots that she used to steal it. As each program relayed its experiences to the other, an avalanche of memory effects like ...

  • Ship (Story 2 of 4)

    Lester was obsessed. It had been a year since the massive beam of light printed the white sphere that invited humanity to a distant star, and he still spent most of his time inspecting the sensor logs in his office. Pictures of the beam and ...

  • Beam (Story 1 of 4)

    Elizabeth Stinton was frustrated. Her simulations for turbulence in her theoretical air sinter were a mess and if she didn't have something to show at the next board meeting she was pretty certain that they'd sell her startup for parts.

    Standing up from ...

  • One Hour Fiction: The Panelist

    The pounding in my head is in sync with the ticking of the escalator steps as they rise from the netherworld of the convention center's floor. I pull a smile from memory and ignore the sweat in my eyebrows. So many happy attendees, clipping ...

  • Agreements: Fuel for the Engine of Business

    Note: This is an old post. I no longer offer this service.

    The first person I watched run a company was Bly, the owner of a computer sales and service shop in Athens, Georgia. She gave me, a painfully young and skinny townie kid, a ...

  • What I hope Amazon is doing in secret

    Last weekend I had coffee at the wonderful Uptown Espresso with a friend from a nearby Amazon office. I write "a" nearby office because he's working on a multi-year project so secret that he can't even tell people which building he's in ...

  • Top 5 opportunities for growth in software

    I am a software engineering newbie. I thought that I knew a lot about it after I coded my first BASIC program on a TRS-100 way back in the before time, but now I see that the creation of software is a vast landscape of ...