Workshop: Integration of Asterics, MyUI and URC for adaptive User Interfaces in Smart Environments

From wiki.gpii
Jump to: navigation, search

Place and time

Location: Hochschule der Medien (Stuttgart - Media University)

Nobelstraße 5, d-70569 Stuttgart https://goo.gl/maps/LbjnR

Date: Feb. 10th 2015; 10:00-17:00 CET
Hosts: Gottfried Zimmermann, Lukas Smirek
related to: Prosperity4All, T. 203.3

Action Items

UCY:

  • Invite to regular telecons on every other Tue at 3pm CET.
  • Finish the REST interface for ARE (UCY)
  • JavaScript library for communication from web browser with ARE (UCY)
  • Direct communication from ARE to UCH based on OSGi component (UCY)
  • Event emission from ARE to any web application (HTML interface)

KII:

  • Extend ACS to define mappings of user actions to thermostat socket elements - define an OSGi bundle for the thermostat socket (KII)
  • Future (and more complex): Implement an OSGi bundle generator for any given user interface socket (KII)

FHG:

  • Implement MyUI runtime based on web components and glue code for UCH (FHG)
  • Converter for converting a Socket Description to Interaction Situations (FHG+HDM)

HDM:

  • JavaScript framework (glue code) for communication from web browser to UCH (HDM)
  • JavaScript framework for communication from web browser to resource server (HDM)
  • Find thermostat that is widely available on the European market
  • Release device template for thermostat (HDM)
  • Define resource properties to mark up alternative user interface components on the resource server (HDM)
  • Let students implement alternative user interfaces for thermostat, some of them with ARE (HDM)

KIT:

  • Validation of the development process involved.

Other:

  • Video tutorial for a thermostat UI?

Future meetings

Illustration

2015-02-10 Sketch.png Figure: Relation of Asterics, MyUI and URC components at development and runtime

Description of figure

At runtime, the following components are used:

  • A wall-mounted thermostat with a small display and buttons. This is inaccessible for many persons.
  • The UCH communicating with the thermostat through a proprietary interface. Inside the UCH a UI socket represents the state of the thermostat.
  • An HTML interface (GUI) running in a web browser on a mobile device. In the HTML interface, glue code communicates with a UI socket running on the UCH.
  • The Asterics Runtime Environment (ARE). It communicates with the HTML interface (GUI) via Events, but can also directly communicate with the socket in the UCH via REST.

At development time, several components support the runtime components:

  • The URC socket builder generates a Socket Description to be used by the UCH.
  • The MyUI Interaction Modeller, generating a self-adapting HTML Interface (GUI) that will run in a web browser.
  • A converter could convert a Socket Description (generated by the URC Socket Builder) into Interaction Situations to be consumed by the MyUI Interaction Modeller.
  • The Asterics Configuration Suite (ACS) defining an Asterics model (& mapping) to be used by the ARE.

Agenda/Minutes

Time Topic
10:00-10:30 Welcome:
  • What is written in the DOW?
  • Expectations

Discussion:

  • Simplified view: WP202 is the layer of building blocks.  WP203 integrates them in the form of frameworks.  And SP3 applies them in applications.
  • If possible, all user interfaces should use ISO/IEC 24751 (personal preferences).
  • !! Create Wiki pages on the standards, and link to them.
  • !! Use schema.org for marking up our Wiki.
10:30-11:00 Introduction to Asterics (UCY):
  • General purpose/ concept,
  • Main components/ interfaces,
  • Short demo

Discussion:

  • Communication between ACS and ARE via RESTful protocol: Upload model, change model, start model, stop model, etc. These functions should also be available to other components at runtime.
  • There should be a stable subset of RESTful functions available for the ARE.
  • The Asterics model could be changed when an application is launched.  This would be driven by the match-maker and flow manager.
  • We need mechanisms for security and resource-based access control. This should be considered in the design: The REST interface should be resource-based rather than function-based. There should be a consistent security mechanism for these kind of things within GPII. Single sign-on will be an important component. See T201.3 Security Architecture and Secure Payment
  • The Asterics model is an executable component for the runtime environment. How can we harvest personal preferences from it so we can adapt the user interface at runtime? E.g. provide a minimal-click user interface for a person who uses switch-based access?
  • We need some metadata on the Asterics model. But where would we store these metadata? In the preference set, or attached to the Asterics model?
  • !! Fraunhofer: Find out what properties are suitable for being exposed in a personal preference set.
  • The input aspect of Asterics can be captured as independent events (IndieUI).
11:00-11:30 Introduction to MyUI (Fraunhofer):
  • User profile: functional approach based on WHO ICF
  • Automatic adaptation of user interface based on user profile, even during runtime
  • Simulation of user profile modifications by profile editor
  • In Prosperity4all, everything would run inside the browser. The HTML code would be deployed to the Resource Server, and the UCH would get it from there.
11:30-12:00 Introduction to URC (HDM):


12:00-13:00 Discussion:
  • Where makes integration sense and where not?
  • Identification of integration points


13:00-14:00 Lunch break
14:00-17:00 Discussion:

Connection between ARE and HTML Interface:

  • HTML Event Source (emitting from a Web server running outside the browser)
  • WebRTC. Supports time-to-live.

Alternative communications (see sketch):

  • ARE connected directly to UCH via REST
  • ARE drives the HTML interface via Events, and HTML interface connected to UCH via REST

Still required (based on Linz meeting):

  • Auto-personalization of HTML interface
  • Personal preferences shared

Adaptation mechanisms:

  • Web components for UI elements that adapt themselves inside (Micro adaptation)
  • Choose between alternative HTML interfaces from the resource server (Macro adaptation)

Direct link between ACS and UCH:

  • UCH running as a processing unit of the ACS (on top of OSGi). Or a special actuator inside ACS connects to the UCH
  • UCH configuring the ARE by models (via REST) downloaded from the resource server

ARE:

  • Can have multiple REST clients. Some publisher-subscribe mechanism for updates necessary.
  • Reduce the REST functionality for manipulating the loaded model. No model store.

Steps of interaction at runtime:

  • ARE always running.
  • UCH running on a remote computer in the local network.
  • User starts the portal app/page of the UCH, and picks a user interface.
  • UCH can discover the ARE on the client's port 8081.
  • UCH can load models to the ARE which are downloaded from the Resource Server.

Use case

  • Thermostat
16:00-17:00 Further Planning:
  • Definition of tasks/ steppingstones
  • Timeline
  • Further communication
18:00 Dinner:

T203.3 Runtime Environment (Leader: UCY, Duration: M6-M30)

The Prosperity4All runtime environment will include three major modules to accessible user interfaces:
  • Support of diverse AT (represented by AsTeRICS).
  • Universal Remote Console (URC) to access a variety of products and services via a personal interaction device (represented by URC).
  • Run-time adaptations to cover temporal changes of user needs (e.g. during one day, or in the course of ageing or rehabilitation) and to overcome barriers of use directly when they occur (represented by MyUI).

For this purpose, adaptations of the three existing runtime environments will be required together with enhancements of multi-platform support:

  • (a) Adaptation and integration of the AsTeRICS Runtime Environment (UCY) This activity will adapt the AsTeRICS Runtime Environment (ARE) specification and design in order to support major platforms (e.g. Windows, Mac, Linux) and adhere to the notion of accessibility for all. It will be implemented and integrated into the overall architecture and will aim principally to complemend the functionality of the model-driven programming environment. The runtime environment is responsible for hosting the models designed using the model-driven environment, which can be automatically deployed and executed through the dynamic composition of software modules on the runtime toolkit. This will allow serving the dynamic deployment of diverse Prosperity4All modules (e.g. sensors) on different platforms, forming part of the overall infrastructure that intends to support modular development of deployable AT applications. The Java OSGi framework serves as the technology for implementing the runtime toolkit on major platforms, since it defines a set of principles for designing and developing software in the form of interoperable software modules/plug-ins. The runtime environment design and implementation process will be hugely assisted by building on the architecture design knowledge of the ARE and exploiting the implementation for porting to other platforms. The development activities include: · Adapting the AsTeRICS Runtime Environment to major platforms · Integration of Prosperity4All modules for data and event exchange with other components and user interfaces
  • (b) Adaptation and integration of the URC runtime environment (HDM) This activity will integrate the URC runtime environment with the context-driven approach of MyUI and the AT environment provided by Asterics. We will adapt the current URC runtime libraries (based on the URC socket approach) to interface and collaborate with the MyUI and Asterics interfaces at runtime. (
  • c) Adaptation and integration of the MyUI Runtime Environment (FHG) This activity will adapt the MyUI Runtime environment to work with AT as plugged and configured by AsTerRICS and with the Prospoerity4All Intelligence modules which detect the current context. For this purpose specific interfaces will be developed to capture information about current AT I/O devices, user needs and environmental factors. This information will be interpreted and processed by the MyUI runtime environment to generate and adapt accessible user interfaces. Finally, the MyUI runtime environment will be adapted to benefit from new HTML 5 capabilities.