In Speakeasy (my main project at Xerox PARC) I took a description for a theoretical approach to heterogenous device interconnection and built a web-scale toolkit that was subsequently licensed to Samsung Electronics.
Within the Computer Science Lab we developed several research prototypes to experiment with the set of ideas that we called recombinant networking.
Note: The basic tenets of user interface design were often ignored in the interest of time. These applications were developed by two or three people in a week or two and then used by small groups for short experiments.
You might notice that the project has had several names, including RNC (recombinant network components), Speakeasy, and finally Obje.
Many people were involved with this research and you should reference the authors on related papers in the publications section for their names. You may safely attribute all of the good ideas to them and all of the mistakes to me.
The Nexus was a debug application that provided a simple hierarchical view of the component network and allowed connections between components.
The browse tab provided a hierarchical view of the components on the network, with the local components at the top level. Connections were created by dragging a source icon onto a sink icon.
During a connection, components added controllers to the transfer session. The Nexus opened a tabbed window for the controllers, as shown by this simple speaker connections controller.
The search tab allowed users to create boolean combinations of constraints. Components were filtered in and out of reset sets that were then shown on the lower panel. Constraints included the components' display names, contextual information, and data type.
The manage tab provided a table of local components with controls for starting or stopping them, and the option to publish them over Jini so that they can be discovered by remote hosts.
The connections tab provided a history of data transfers initiated from the Nexus. It detailed which components provided and consumed the data during the connection and allowed the user to easily initiate another connection between those components.
The program tab displayed the list of templates known to the local template manager. While it ded not provide any mechanism for editing or creating templates it did allow users to start and stop templates that were created using other means like a programmer's editor or a command line template publishing tool.
The user context tab displayed the key/value pairs stored in the context for the local component host.
The Music application originated from the urge to have a simple application demonstrating that the platform could be used for simple daily tasks. Because we had components for speakers and file systems, we created a simple music playing application.
In addition to shuttle controls, the UI listed all sources of music that could be played on the local speaker component.
Set Top Demo
We spent a bit more time on this application because we intended to use it in demonstrations and because we wanted to litter the lab with media installations driving big plasma screen TVs and manipulated by IR remote controls.
The top level menu showed the various features of the demo.
The Image Library displayed a list of all still image sources that could be displayed on the local screen component.
The Video Library displayed a list of all video sources that could be displayed on the local screen component.
Expert mode allowed users to select any data source...
...and then any local or remote destination for the data...
...and then initiate and monitor a connection between them.
During the grand winter rewhack of 2002, the first application built on top of the new architecture was Wander.
Wander provided a hierarchy of components and allowed drag and drop connections. The right hand panel displayed the contextual information for the selected component and the lower left hand panel displayed connection status.
Cowpoke provided a debug view of the various programmatic entities within an installation.
Here you can see the various hosted component IDs on a server running on the laptop called Dr. Wolfgang Morgenstern.
Bigmap was a quickly assembled public display showing a map of where components are located within the lab.
Unfortunately, the location framework was not working when this screenshot was taken, so it shows all of the components running on the patio outside of the Weiser Commons.
Casca provided a simple simple UI for hosting and sharing applications with other users, with a focus on creating lightweight, secure workgroups with dynamic resources.
The Casca window (middle) showed resources on the local machine, a search/browse panel, and all of the workgroups that the user joined. It also showed integration of instant messaging and sharing of status among online members of the group.
Office Node and Display Manager
We installed dedicated machines (3 eMacs, an iMac, and an iBook driving a screen in a picture frame) in our offices and ran an application on them that took over the screen and speakers. There was no direct input to these office nodes via keyboard and mouse. Instead they were controlled remotely by a "Display Manager" application that provided component management controls as well as controls for making connections to other people's office nodes.
The Display Manger provided two ways to locate components: searching by keyword and browsing through a tree view. In this screen shot you can see the tree view. Also shown is the "News" panel that displays a web page on that I posted news about the Office Node experiment.
Once you located a component that you would like to connect to someone's Office Node, you selected that Office Node's name from the tabs and click the "Connect" button. All of the transfer session's controls would appear in the bottom right panel, in addition to the component UIs, that were always displayed in the upper right hand panel.
We also ran a "Notes Server" component that allowed you to dynamically send text to Office Nodes, via a controller.
On the Office node side of things, multiple connections were arranged to share the screen without overlapping, with a username (or context value for "Name") displayed on the bottom to indicate who was making the connection.
This was a single purpose application consisting of a simple UI (shown) that discovered "Screen" sinks and connected a local VNC source to them, mirroring the local screen. This was deployed over Java WebStart and screens were installed on projectors and plasma screens in conference rooms.
Here the display mirror has discovered four displays, and upon pressing the "Connect" button the local screen would be mirrored on the remote creen.
The Orbital Browser was a simplified UI for finding and connecting components. We designed it to be both pleasing in a demo, and interesting in terms of limited input scenarios. The input device was a Griffen Powermate knob that provided only rotation and selection. All interaction was through those two actions.
Each cluster represented a host, with components in orbit around it.
Traversal through the aggregates was possible, with the results of the search saved as stacks in orbit around the host.
Connections were represented by lines between endpoint widgets in the shape of biological receptors and senders. These allowed users to choose between connection types.
Some project scenarios required that a cell phone act as an service composer and image component host. I ported the platform to the J2ME (CDC, Personal, + Swing) platform on the i-mate JAM devices running the CrEme JVM 4.0, and on top of that I built a wizard-style UI similar to the "Expert Mode" of the set top box demo UI.
The first screen showed a list of all DataSources on the network.
After choosing a source and using the stylus to select the right arrow button, the user was shown all of the DataSinks that could connect to that DataSource.
After choosing a DataSource and the right arrow button, the user was shown the connection panel with the status of the connection and any controllers. Shown here is a Screen controller which was obviously not designed for the small display of the handheld.
The whiteboard application was similar to the Display Mirror, as it was a single purpose application. In this case, the application connected whiteboard cameras to printers and displays around the lab.
On the first screen, the user chose from a list of available whiteboards.
The user then chose the destination for the whiteboard image from drop downs menus of printers or displays.
Once a user chose to connect, the various components displayed status widgets. Shown here is the progress bar from the slow progress of the whiteboard camera.
At some point we had a critical mass of components deployed on the network and we wanted to experiment with a social application that took advantage of our ability to combine networked media streams. With the excellent Nicolas Ducheneaut, I built a test application that monitored various contextual events (browsing and music listening are shown below) and shared them with the group.
The main panel on the left was a constantly scrolling list of the group's media events and their available actions. Here "Dr. Wolfgang Morgenstern" noticed that "Trevorolio" was reading BBC news so he joined the browsing session.