What is PotassiumES?

This is a post about PotassiumES, an ECMAScript library that enables browser-side development for the wider web. If you're not sure about the wider web, click that link because otherwise the rest of this post won't make a lick of sense.

I wrote this after a few cups of questionable airplane coffee, but I like how it turned out, warts and opinions and all. I'm OK. You're OK.

OK!

For the twitchy: This post describes the major aspects of PotassiumES, but if you'd prefer to hop right into code then you should click over to the samples repo or the docs.

All of the code is super rough, incomplete, and in flux at the time I'm writing this but it should give you the general idea.

Still here? Toot toot! Let's go!

Technical design

If you're used to the old-school models and views of Backbone.js or the slightly less old-school reactive components of React and Vue then PotassiumES will feel somewhat familiar but also more modern and (I think) delightful.

Unlike older (some might say "narrow") app frameworks, PotassiumES only supports browsers that support the core tech of the wider web. In general, browsers with native ES'15+ syntax, WebAudio, and the WebXR Device API are supported.

Once version 0.1.0 is ready to go out I'll create a browser compatibility page, but in the meantime look for the ones with the WebXR Device API behind a flag. WebVR won't work.

Politics

Did I mention that all software is political?

No? Dang it.

Long story short, the PotassiumES code of conduct for contributors explicitly excludes fascists, so if you're against that then I suggest that you find another framework. Something something punch fascists.

Moving right along...

App: the conductor of the wider web

Wider web applications meet the user where they are, regardless of their hardware or their mood.

Each PotassiumES App orchestrates a web application across all three display modes: flat, portal, and immersive.

Instead of writing a separate code base for each type of experience, or worse yet tacking half-baked portal or immersive experiences onto the side of mostly flat applications, Apps share logic and control elements across all three display modes. They also can be updated for the ever changing sea of input hardware without much fuss.

If you've used AR or VR apps on the web that only work on one type of hardware or in one display mode then you know why you need PotassiumES.

An App contains the root data structures for each display mode:

  • For flat mode it holds a DOM element.

  • For portal mode it holds a DOM element for overlay controls and a 3D scene for spatial controls and virtual environments.

  • For immersive mode it holds a 3D scene for spatial controls as well as virtual environments.

An App manages WebXR sessions for portal and immersive modes. It also toggles the visibility of the flat and portal DOM fragments as display modes change.

One super nice thing: When using PotassiumES you'll probably never have to directly use WebXR or WebGL. (thank goodness!)

I can hear you asking, "But what does an App put into those DOM and 3D scene roots?"

I'm glad you asked.

Wait, are you a shill? Did Blair send you?

Anyway, let's talk about Components.

Component: one class for all control types and all display modes

A single class, Component, manages the visual, auditory, and interactive logic for a given user interface element. Each Component works across all three control types (page, overlay, and spatial) and all three display modes (flat, portal, and immersive).

The crowd roars, "That's pretty vague. How about an example?"

OK!

The ButtonComponent holds the DOM elements, 3D assets, audio sources, and action logic for a general purpose button that works across the entire range of hardware on the wider web.

You can click that button in flat mode, poke it in portal mode, and laser point it in immersive mode!

Spatial controls are created with 3D assets, not by drawing DOM elements into space.

Flat DOM elements like radio buttons and text input fields are not drawn to WebGL textures and then used as spatial controls. Flat controls feel wrong in 3D scenes and they don't take advantage of the unique and delightful new design opportunities of portal and immersive modes.

You're better than that!

Like an App, each Component holds data structures for each display mode. Instead of holding root DOM elements or root 3D scenes like an App, each Component holds DOM or 3D scene fragments:

  • For flat mode they hold a DOM fragment
  • For portal mode they hold a DOM fragment for overlay controls and a separate 3D scene graph fragment for spatial controls
  • For immersive mode they hold a separate 3D scene graph fragment for spatial controls

Now put it all together!

Component fragments are assembled into hierarchies by an App and then the App controls which roots (remember the App roots for each display mode?) are hidden and shown as the display mode changes.

I think of the Components as an orchestra and the App as the conductor, but that might just be me. You pick your own metaphor, because diversity.

Oh, yeah. One neat thing is that audio playback for a Component is spatially mixed in immersive mode. When you throw a lever in immersive mode the user will hear the sound actually coming from the lever and you didn't need to write a spatial audio mixer!

(well, that will be true once I port over the spatial mixer from Spaciblō)

Go, team!

This next bit is something missed by pretty much every other app framework, even the ones that currently support WebVR. Without this next bit, your web app will work for about 10 minutes before new XR hardware arrives and all of your nice code breaks. Oh, also current VR app frameworks screw anyone except the perfectly healthy, average, and un-opinionated.

Curious? Read on!

The ever-changing flood of low level inputs

Because the wider web has accessibility built into it at the core and because there are so many types of input hardware, Components simply can not directly react to the flood of lower level input events like mouse clicks, touch drags, gaze hovers, voice commands, wand movements, and keyboard taps.

It's just too much!

Instead, PotassiumES uses the snazzy action-input library to map low level input events to high level actions and Components react to those.

For example, instead of a button DOM element reacting directly to a mouse click event, a ButtonComponent reacts to an action of the type activate. The action-input library maps the lower level mouse click event to a higher level action so that the Component never needs to know about mice, or touchpads, or wands, or tongue sensors, or whatever weird things we hook up next.

At a later date, when wider web users start to use new input types (maybe gestures or voice controls) those new low level inputs can be mapped to high level actions and you don't need to change the ButtonComponent.

Remember: Components react to actions, not input events.

Put the "action" into "accessibility"

(I know that "accessibility" doesn't have "action" in it. Forgive me?)

People with differing abilities use the PotassiumES map editor (TBD -ed) to modify the action-input maps to support their specific needs. A person without fine motor control can dampen wand movements by adding a filter. A person who uses a breath sensor system can map those input events to the actions that makes sense to them.

And yes, hard-core gamers who want to reverse the vertical look direction because they have twenty years of muscle memory (I'm looking at you, Blair) can also edit the maps to find relief.

Component libraries save the day

Making an entirely new input map for every control type and display mode is a lot of work. Designing and implementing the large set of Components for a wider web application is also a lot of work.

I hear that!

Luckily, the PotassiumES project comes with a standard library of Components that work nicely with a standard set of action-input maps.

(or rather, they will once eRacoon makes the 3D assets and I hook them up)

These resources are built for extensibility so that you can start from a good place that is good for your users and then you can make it your own.

And all the rest

PotassiumES has a lot of the other pieces and parts of an app framework that you've come to expect, like a Router, snazzy chaining APIs for quickly creating and data binding DOM and 3D fragments, as well as data models and collections.

It's deep and it's fun and I love it so much that at this point it's kind of my kink.

Is that too weird?

No. No, it is not. I mean, look at the Mastodon federated timeline some time! Be who you are is clearly in the gestalt.

This post is already HUGE and it covers enough of the basics that the next step should be just giving it a try. You know so much!

You'll eventually want to read the docs to get the fine details, but head over to the samples repo and start tweaking.

Things are still really rough and sometimes broken. WebXR is still half baked and not ready for prime time. But it's all really quite fun, and I look forward to seeing you on the wider web.

I'll be the tall one with flowers.

More posts

Recent posts

  • Wider Web Lingo

    People sling around a lot of lingo when talking about the wider web, and even the term "wider web" is lingo!

    So, I wrote a series of short posts defining words and phrases ...

  • Wider Web Lingo: Voice

    There's a lot of lingo around the wider web so this is one of a series of short definition posts.

    Voice: Phrases or other vocal noises that can be recognized and used as input

    Computers are getting pretty good at understanding ...

  • Wider Web Lingo: Gesture

    There's a lot of lingo around the wider web so this is one of a series of short definition posts.

    Gesture: A body motion that can be recognized and used as input

    Computers are getting better at watching how we position ...

  • Wider Web Lingo: More Hugs, Fewer Thugs

    There's a lot of lingo around the wider web so this is one of a series of short definition posts.

    More hugs, fewer thugs

    The wider web is inherently more intimate than the flat web that you hold in your hand ...

  • Wider Web Lingo: Spatial Controls

    There's a lot of lingo around the wider web so this is one of a series of short definition posts.

    Spatial controls: Interactive visual elements that are placed in the real or virtual environment

    Unlike page controls or overlay controls (the ...

  • Wider Web Lingo: Overlay Controls

    There's a lot of lingo around the wider web so this is one of a series of short definition posts.

    Overlay controls: Visual elements that seem to sit on top of a portal display

    Overlay controls on the wider web float ...

  • Wider Web Lingo: Page Controls

    There's a lot of lingo around the wider web so this is one of a series of short definition posts.

    Page controls: Visual elements that you're used to on the web like buttons, images, text fields, and blocks of text ...

  • Wider Web Lingo: Wand

    There's a lot of lingo around the wider web (which is itself lingo) so this is one of a series of short definition posts.

    Wand: A handheld device with tracked orientation and position, often with touchpads, thumbsticks, buttons, or other controls ...

  • Wider Web Lingo: Immersive Display

    There's a lot of lingo around the wider web (which is itself lingo) so this is one of a series of short definition posts.

    Immersive display: A display into a real or virtual environment that covers most of your field of ...

  • Wider Web Lingo: Flat Display

    There's a lot of lingo around the wider web (which is itself lingo) so this is one of a series of short definition posts.

    Flat display: A display that does not change its view as your position changes.

    This is the ...