Semantic Connections Demonstrator
From SOFIA
Contents |
Introduction
The demonstrator consists of a set of devices; surround sound-set, mobile mp3 players, an ambient lighting system and interaction device(s). The interaction device is a tile-like interactive object that allows for both exploration of the Smart Space in terms of connections and manipulation of these connections and information/data streams. Coloured LED lighting and light dynamics visualize the connections and connection possibilities between the various devices. By means of putting devices close to one of the four sides of the tile, a user can check if there is a connection and if not if a connection is possible. By simply picking up the tile, and shaking it a user can make or break the connection between the devices present at the interaction tile.
Background
The use-case demonstrator is based on and supports the following scenario:
Mark is a 12-year-old boy and he is at home receiving his friend Dries from school. Dries arrives with a portable music player loaded with his favourite songs. He wants to play some recent collections for Mark. Mark‘s home is equipped with a sophisticated surround sound system, and they have recently installed an ambient lighting system that is connected to the sound system and renders the mood of the music by dynamic colour lighting in the room. They decide to use both to enjoy the music. Dries starts streaming his music to the environment.
An object (or several objects) shows possible input and output ports for streaming music in the environment. By interaction with the object/objects, Mark connects the output from Dries' music stream to the input of the sound system. Now the room is full with Dries‘ music and the colourful lighting effects. Mark‘s mom, Sofia, now comes back from work. She starts preparing dinner for the family. Mark and Dries don‘t want to bother her with their loud music. They again use the object(s) to re-arrange the music stream. Now the music is streamed to Mark‘s portable music player while playing back at Dries‘. It is also connected to the ambient lighting system directly, bypassing the sound system. They both are enjoying the same music using their own favourite earphones, and the colourful lighting effects, but without loud music in the environment.
The object(s) shows the connection possibilities with a high level semantic abstraction, hiding the complexity of wired or wireless network. By interacting with the object(s), semantic connections can be built, redirected, cut or bypassed.
Requirements
Supported interactions by the interaction device(s)
- Viewing/exploring existing connections
- Viewing/exploring connection possibilities
- Making connections
- Breaking connections
Functionality of the demonstrator
- Playing music from mp3 player on sound-set
- Playing music from mp3 player on sound-set and rendering music by ambient lighting system
- Playing music from one mp3 player on a second one and rendering music by ambient lighting system
Design
Interaction-tile
A tile-like object inspired by Jeevan Kalanithi’s and David Merrill’s Siftables was designed to explore connections and interaction possibilities and manipulation by direct manipulation, making simple spatial arrangements. The interaction device visualises the various connections by allowing a user to explore which objects are connected to which objects and what can be connected to what.
We want to allow users to explore and manipulate the connections within the smart space without having to bother with the lower-level complexity of the architecture. Therefore, knowledge about the devices and appliances connected to the smart space and their capabilities is needed. We envision this to be a simplified view (model) of the actual architecture of the smart space. An example of an instance of the ontology used to store (and reason about) the necessary data is shown in figure 2. The ontology with "is-a" relationships indicated is shown in figure 3.
Description of the various knowledge processors and the information that is exchanged:
1.1.1 Main device that interact with user/SS
Portable music player(s) (Nokia Series 60 mobile phones)
1.1.2 Purpose
Plays music on user's headphones, or streams music to other portable/stationary devices
1.2. How does this device interact with user/environment/SS?
GUI - Dialogs/Menus/Selection lists
Keypad input
1.3. Info and related events affecting interaction model that are known by SS
| Interaction that triggers SIB exchange | Triples exchanged with SIB | Required entities/properties to enable expected device functionality |
|---|---|---|
| User plays a song from music collection on device | sofia:PlayEvent; sofia:generatedBy; sofia:SmartObject; sofia:inXSDDateTime; xsd:dateTime | sofia:MediaPlayerEvent; sofia:CueEvent; sofia:PlayEvent; sofia:StopEvent |
2.1.1 Main device that interact with user/SS
Surround sound system (Speakerset connected to a PC)
2.1.2 Purpose
Plays music on speakers in the environment
2.2. How does this device interact with user/environment/SS?
GUI
Key input
2.3. Info and related events affecting interaction model that are known by SS
| Interaction that triggers SIB exchange | Triples exchanged with SIB | Required entities/properties to enable expected device functionality |
|---|---|---|
| User streams
music from portable music player to surround sound system | sofia:PlayEvent; sofia:generatedBy; sofia:SmartObject; sofia:inXSDDateTime; xsd:dateTime | sofia:MediaPlayerEvent; sofia:CueEvent; sofia:PlayEvent; sofia:StopEvent; sofia:SmartObject |
3.1.1 Main device that interact with user/SS
Ambient lighting system (Arduino/microprocessor controlled light)
3.1.2 Purpose
Renders mood of music in the environment
3.2. How does this device interact with user/environment/SS?
LED or other indicators
Light switch
3.3. Info and related events affecting interaction model that are known by SS
| Interaction that triggers SIB exchange | Triples exchanged with SIB | Required entities/properties to enable expected device functionality |
|---|---|---|
| User plays music on surround sound system | sofia:PlayEvent; sofia:generatedBy; sofia:SmartObject; sofia:inXSDDateTime; xsd:dateTime | sofia:SmartObject; sofia:PlayEvent |
| User bypasses surround sound system and plays music on headphones connected to portable music player mood still rendered by ambient lighting system | Sofia:NFCEnterEvent; sofia:hasRFIDTag; xsd:string; sofia:inXSDDateTime; xsd:dateTime; hasPosition; xsd:integer | Sofia:NFCEvent; Sofia:NFCEnterEvent; Sofia:NFCExitEvent |
4.1.1 Main device that interact with user/SS
Smart/interactive object (Arduino-based)
4.1.2 Purpose
User manipulates interactive object to create connections between devices in the environment
4.2. How does this device interact with user/environment/SS?
Direct manipulation of object
4.3. Info and related events affecting interaction model that are known by SS
| Interaction that triggers SIB exchange | Triples exchanged with SIB | Required entities/properties to enable expected device functionality |
|---|---|---|
| User creates a (mental) connection between two devices using interactive object | sofia:SmartObject; sofia:connectedTo; sofia:SmartObject; sofia:hasRFIDTag; xsd:string | sofia:SmartObject |
| User re-arranges music stream from from surround sound system to two portable music players | Sofia:NFCEnterEvent; sofia:hasRFIDTag; xsd:string; Sofia:inXSDDateTime; xsd:dateTime; hasPosition; xsd:integer | Sofia:NFCEvent; Sofia:NFCEnterEvent; Sofia:NFCExitEvent |
Realization
The interaction tile consists of the following components:
- Arduino board (Duemilanove)
- 13.56MHz RFID reader (ACS/MiFare)
- Multi-colour LED's
- Accelerometer
- Vibration motor
- Magnetic switches
The demonstrator consists of the following devices:
- Mobile MP3 players (Nokia N95 and 5800 XpressMusic Smartphones)
- Ambient lighting system (homebrew lamp with RGB LED’s)
- Sound system (speaker-set connected to notebook PC)
- Notebook PC (acting as SIB)
- Interaction tile
