Update: This is still a handy reference but you might be interested in the new PotassiumES site.
This is a post about PotassiumES, an ECMAScript library that enables browser-side development for the wider web. If you're not sure about the wider web, click that link because otherwise the rest of this post won't make a lick of sense.
I wrote this after a few cups of questionable airplane coffee, but I like how it turned out, warts and opinions and all. I'm OK. You're OK.
All of the code is super rough, incomplete, and in flux at the time I'm writing this but it should give you the general idea.
Still here? Toot toot! Let's go!
If you're used to the old-school models and views of Backbone.js or the slightly less old-school reactive components of React and Vue then PotassiumES will feel somewhat familiar but also more modern and (I think) delightful.
Unlike older (some might say "narrow") app frameworks, PotassiumES only supports browsers that support the core tech of the wider web. In general, browsers with native ES'15+ syntax, WebAudio, and the WebXR Device API are supported.
Once version 0.1.0 is ready to go out I'll create a browser compatibility page, but in the meantime look for the ones with the WebXR Device API behind a flag. WebVR won't work.
Did I mention that all software is political?
No? Dang it.
Long story short, the PotassiumES code of conduct for contributors explicitly excludes fascists, so if you're against that then I suggest that you find another framework. Something something punch fascists.
Moving right along...
App: the conductor of the wider web
Wider web applications meet the user where they are, regardless of their hardware or their mood.
Instead of writing a separate code base for each type of experience, or worse yet tacking half-baked portal or immersive experiences onto the side of mostly flat applications,
Apps share logic and control elements across all three display modes. They also can be updated for the ever changing sea of input hardware without much fuss.
If you've used AR or VR apps on the web that only work on one type of hardware or in one display mode then you know why you need PotassiumES.
App contains the root data structures for each display mode:
For flat mode it holds a DOM element.
For portal mode it holds a DOM element for overlay controls and a 3D scene for spatial controls and virtual environments.
For immersive mode it holds a 3D scene for spatial controls as well as virtual environments.
App manages WebXR sessions for portal and immersive modes. It also toggles the visibility of the flat and portal DOM fragments as display modes change.
One super nice thing: When using PotassiumES you'll probably never have to directly use WebXR or WebGL. (thank goodness!)
I can hear you asking, "But what does an
App put into those DOM and 3D scene roots?"
I'm glad you asked.
Wait, are you a shill? Did Blair send you?
Anyway, let's talk about
Component: one class for all control types and all display modes
A single class, Component, manages the visual, auditory, and interactive logic for a given user interface element. Each
Component works across all three control types (page, overlay, and spatial) and all three display modes (flat, portal, and immersive).
The crowd roars, "That's pretty vague. How about an example?"
The ButtonComponent holds the DOM elements, 3D assets, audio sources, and action logic for a general purpose button that works across the entire range of hardware on the wider web.
You can click that button in flat mode, poke it in portal mode, and laser point it in immersive mode!
Spatial controls are created with 3D assets, not by drawing DOM elements into space.
Flat DOM elements like radio buttons and text input fields are not drawn to WebGL textures and then used as spatial controls. Flat controls feel wrong in 3D scenes and they don't take advantage of the unique and delightful new design opportunities of portal and immersive modes.
You're better than that!
Component holds data structures for each display mode. Instead of holding root DOM elements or root 3D scenes like an
Component holds DOM or 3D scene fragments:
Now put it all together!
Component fragments are assembled into hierarchies by an
App and then the
App controls which roots (remember the
App roots for each display mode?) are hidden and shown as the display mode changes.
I think of the
Components as an orchestra and the
App as the conductor, but that might just be me. You pick your own metaphor, because diversity.
Oh, yeah. One neat thing is that audio playback for a
Component is spatially mixed in immersive mode. When you throw a lever in immersive mode the user will hear the sound actually coming from the lever and you didn't need to write a spatial audio mixer!
(well, that will be true once I port over the spatial mixer from Spaciblō)
This next bit is something missed by pretty much every other app framework, even the ones that currently support WebVR. Without this next bit, your web app will work for about 10 minutes before new XR hardware arrives and all of your nice code breaks. Oh, also current VR app frameworks screw anyone except the perfectly healthy, average, and un-opinionated.
Curious? Read on!
Because the wider web has accessibility built into it at the core and because there are so many types of input hardware,
Components simply can not directly react to the flood of lower level input events like mouse clicks, touch drags, gaze hovers, voice commands, wand movements, and keyboard taps.
It's just too much!
Instead, PotassiumES uses the snazzy action-input library to map low level input events to high level actions and
Components react to those.
For example, instead of a button DOM element reacting directly to a mouse
click event, a
ButtonComponent reacts to an action of the type
activate. The action-input library maps the lower level mouse click event to a higher level action so that the
Component never needs to know about mice, or touchpads, or wands, or tongue sensors, or whatever weird things we hook up next.
At a later date, when wider web users start to use new input types (maybe gestures or voice controls) those new low level inputs can be mapped to high level actions and you don't need to change the
Components react to actions, not input events.
(I know that "accessibility" doesn't have "action" in it. Forgive me?)
People with differing abilities use the PotassiumES map editor (TBD -ed) to modify the action-input maps to support their specific needs. A person without fine motor control can dampen wand movements by adding a filter. A person who uses a breath sensor system can map those input events to the actions that makes sense to them.
And yes, hard-core gamers who want to reverse the vertical look direction because they have twenty years of muscle memory (I'm looking at you, Blair) can also edit the maps to find relief.
Making an entirely new input map for every control type and display mode is a lot of work. Designing and implementing the large set of
Components for a wider web application is also a lot of work.
I hear that!
Luckily, the PotassiumES project comes with a standard library of
Components that work nicely with a standard set of action-input maps.
(or rather, they will once eRacoon makes the 3D assets and I hook them up)
These resources are built for extensibility so that you can start from a good place that is good for your users and then you can make it your own.
PotassiumES has a lot of the other pieces and parts of an app framework that you've come to expect, like a Router, snazzy chaining APIs for quickly creating and data binding DOM and 3D fragments, as well as data models and collections.
It's deep and it's fun and I love it so much that at this point it's kind of my kink.
Is that too weird?
No. No, it is not. I mean, look at the Mastodon federated timeline some time! Be who you are is clearly in the gestalt.
This post is already HUGE and it covers enough of the basics that the next step should be just giving it a try. You know so much!
Things are still really rough and sometimes broken. WebXR is still half baked and not ready for prime time. But it's all really quite fun, and I look forward to seeing you on the wider web.
I'll be the tall one with flowers.
Using Three.js to build a reusable border geometry for spatial UIs!
🌸 Updated potassium style system (KSS), now with margins!
🌸 The path to the vNext
🌸 A couple of new spatial controls
🌸 A brief intro to the existing samples
🌸 Building UI components that work in flat, portal, and immersive display modes