Difference between revisions of "GPII Architecture Overview"

From wiki.gpii
Jump to: navigation, search
m (Ontologies and the Ontology Server)
m (Value Mapping)
Line 441: Line 441:
 
An example can be seen below. Here the application setting "mouse-tracking" can take two values: "mouse" and "caret". In the user preferences ("display.screenEnhancement.tracking"), the valid values are "centered" and "none". When this transformation is applied, a user preference of "centered" would be translated in "mouse" and be assigned to the application setting.
 
An example can be seen below. Here the application setting "mouse-tracking" can take two values: "mouse" and "caret". In the user preferences ("display.screenEnhancement.tracking"), the valid values are "centered" and "none". When this transformation is applied, a user preference of "centered" would be translated in "mouse" and be assigned to the application setting.
  
There are two optional and additional options to this tranformation:
+
There are two optional and additional options to this transformation:
 
* ''''"defaultInputValue"''': if no branch matches, it is attempted again using this value - WARNING, this seriously threatens invertibility
 
* ''''"defaultInputValue"''': if no branch matches, it is attempted again using this value - WARNING, this seriously threatens invertibility
 
* ''''"defaultOutputValue"''': if no branch matches, this value will be used as the output value (ie. in the below example, if the input value was neither "mouse" nor "caret", "mouse-tracking" would be assigned the value of the "defaultOutputValue" key
 
* ''''"defaultOutputValue"''': if no branch matches, this value will be used as the output value (ie. in the below example, if the input value was neither "mouse" nor "caret", "mouse-tracking" would be assigned the value of the "defaultOutputValue" key

Revision as of 02:37, 10 June 2013

Contents

Executive Summary

The Cloud4all architecture encompasses an overall technical approach, a development framework, and a working reference implementation that serves as the basis for developing an ecosystem of tools to support user interface personalization. This architecture is designed to work across platforms and devices, including desktop computers, mobile devices, and web applications. It includes the specification and implementation of the data interchange formats, component APIs, and extension points that comprise Cloud4ll's personalization workflow.

This architecture is designed to promote the long term growth, sustainability, and viability of the Cloud4all technology. The primary concerns of the architecture are to ensure that all critical components within the system are built in a performant, modular, reusable, secure, and well-tested manner. These goals are primarily accomplished by providing a core framework that manages the personalization workflow and all relationships between components within the system, ensuring that it can be extended and reconfigured to support new functionality and new implementation strategies over time.

The Cloud4all architecture reference implementation (often referred to as "the realtime framework") builds on the familiar cross-platform and standards-driven approach of the web. In particular, it makes extensive use of both client and server-side JavaScript (using Node.js as its primary cross-platform runtime) as well as HTML and CSS for user interfaces. This enables the core of Cloud4all to be easily ported to a wide variety of devices and operating systems without requiring costly code rewrites and duplication. JSON, a lightweight interchange format, is used for all data in the system. The Fluid Project's accessible Infusion framework is used extensively to provide Cloud4all's context-aware, model-driven, and highly decoupled development infrastructure.

This document provides a detailed overview of the first iteration of the Cloud4all architecture, including the design approaches employed, the components that comprise the system, and the technical details of the reference implementation.

Overview of the GPII Architecture

Design Goals

< fill in >

  • Supporting Multiple Platforms
  • Cloud-Level Scalability
  • Extensibility and long-term growth

Approach

Web-Oriented Architecture

Web-based technologies offer a nearly ubiquitous development platform for building cross-platform applications. On both desktop and mobile development, the trio of JavaScript, HTML, and CSS provide a highly flexible and increasingly performance-oriented technology stack that is available everywhere. Notably, the web has succeeded where other attempts at creating a cross-platform runtime failed—Java and Flash, despite their "write once, run anywhere" promise, both struggled to deliver in the consumer application space due their single-vendor control and inflexibility when faced with new device capabilities and form factors.

Over the past decade, significant effort has been invested to ensure that web technologies are accessible. The declarative approach of the web, particularly the built-in separation of structure from presentation inherent in the HTML and CSS model, makes web technologies particularly amenable to realizing the goals of the Cloud4all architecture.

Web technologies, JavaScript in particular, are no longer confined to the browser. JavaScript has emerged as a popular general-purpose programming language that can run on both the client and server (using tools such as Node.js). JavaScript-based code can also on mobile devices. The web technology stack has also become an integral part of many commercial and open source desktop and mobile development toolkits, including those for Windows 8 and Windows Phone, Blackberry, Adobe PhoneGap, Google Chrome OS, and more. JavaScript performance has, over the past several years, made significant performance improvements, leapfrogging other familiar dynamically-interpreted languages such as PHP, Python, and Ruby. In many cases, it is now comparable in performance to traditional statically typed enterprise programming languages such as Java and C++.

There are also millions of skilled web developers and designers around the world; this is a significant factor when considering the long-term sustainability of Cloud4all in terms of being able to continue to attracted skilled and enthusiastic developers to our community.

REST and Repartitioning

The Cloud4all architecture has been designed using a RESTful approach. REST, which stands for [http://en.wikipedia.org/wiki/Representational_state_transfer

Components of the Architecture

Overview of Components

GPII Program Flow-small.png

< caption >

The Auto-Personalization Flow

The purpose of Cloud4all/GPII architecture is to enable the personalization of desktop, mobile, and web applications and content based on the user's needs and preferences. Here's how it works:

  • The user starts the process by describing their needs and preferences to the system using an editor, wizard, or discovery aid (sometimes referred to as a "Preferences Discovery Aid & Editor").
  • From there, the system is able to automatically configure a mobile device, computer, or web site with the user's preferred applications (for example, a screen reader) and settings (e.g. their preferred voice speed). Web content will be automatically adapted to suit the user's preferences as well.
  • When the user approaches a GPII/Cloud4all-enabled device, such as a computer at a library or their new smartphone:
  1. The user offers a key (a unique string) to the the device using a range of techniques such as:
    1. a USB device (flash drive or special device)
    2. an NFC ring or phone or other NFC Tag
    3. typing in a special string on the keyboard
    4. a continually changing key code
    5. etc.
  2. The User Listener component, typically running on the local device, will detect this key and send it to the Flow Manager.
  3. The Flow Manager will orchestrate the process of configuring the device by bring together:
    1. The user's preferences (from the Preferences Server)
    2. The range of applications and access features (Solutions) available to the user either on the local device, on the web, or in the cloud (from the Solutions Registry)
    3. Information about the device's characteristics and the user's working environment (with their permission) (from the Device Characteristics Reporter)
  4. The Matchmaker is the part of the system that makes decisions about how to meet the needs of the user. It is responsible for:
    1. Matching the user's preferences with the available solutions
    2. Resolving device, context, and environment-specific conditions for those settings
    3. Assembling a list of the the features and applications that need to be configured on the device
  5. The list of solutions and their settings is then passed to the Lifecyle Manager, which manages the process of configuring settings
  6. The Lifecycle Manager, in turn, invokes a set of Lifecycle Handlers and Settings Handlers that do the actual launching and setting of applications and access features.

Flow Manager

The Flow Manager is the event-driven, asynchronous orchestrator of the personalization workflow. It is implemented as a very small server that runs on the local device and is responsible for bringing together the various components in the system and invoking them at the appropriate points in the process. In the next release of the Cloud4all realtime framework (version 0.2), an instance of the Flow Manager will also be available in the cloud (running as a web service) for use by web applications and resource-constrained devices such as Java Mobile Edition (JME)-based feature phones.

The key responsibility of the Flow Manager is to manage the workflow of the personalization process, providing a means to register components with the system and bind them to specific events and lifecycle points within this workflow. The Flow Manager uses an Inversion of Control approach (provided by Fluid Infusion) to instantiate and wire up dependencies between components. This means that components rarely have a direct coupling with other components in the Cloud4all system. Instead, they are implemented as small, self-contained objects that are responsible for a single concern. Components are registered and bound to the Flow Manager via JSON-based configuration. This means that the system is trivially unit testable (since default implementations can be replaced easily with testing mocks) and, more importantly, that it can be easily extended and evolved over time. New features and components can be added to the Flow Manager without requiring code changes to the other components or the Flow Manager itself.

Flow Manager Event Diagram

Gpii-architecture-event-flow.png

Figure x: A simplified view of the GPII Flow Manager and Matchmaker's event-driven workflow. Solid lines are events fired by a component, while dashed lines denote listeners to an event. It is important to note that the Flow Manager's declarative, inversion of control design ensures that each of these events can be fully reconfigured without requiring code-level changes to the Flow Manager; new listeners can be registered, old ones can be removed, and new events can be synthesized from a series of existing ones, simply by modifying the Flow Manager's JSON-based configuration file.

User Listeners

The User Listener is the first step in the process of customizing a device or application to the user's needs and preferences. The user shares their key with the listener, which is then responsible for starting the process personalization. Since users may want to stay as anonymous as possible, this key may be a random, unique, opaque string—not necessarily a personal ID.

The architecture is designed to support many different types of interchangable User Listeners, and the rest of the system is decoupled from the particular listener used. Listener implementations may include RFID tag readers, USB key readers, 2D barcode readers, username/password-based login, face recognition, voice recognition, etc. Currently, there are two implemented User Listeners:

  • RFID Reader, which reads the user token from a low-cost Near Field Communication (NFC) tag that can be embedded in almost any object (cards, pens, rings, and even many Android mobile devices are available with NFC built-in). The RFID Reader works on all currently supported Cloud4all platforms including Windows, Linux, and Android.
  • USB Drive Reader, which reads a user token from a file on any USB storage media such as a thumb drive or SD card reader. The USB Reader is available on all supported desktop platforms including Windows and Linux.

Matchmakers

The Matchmakers are responsible for deciding which solutions and settings will best meet the user's needs. To do this, they take into account:

  1. the user's preferences
  2. the applications, features, and assistive technologies available on their current device and out on the cloud
  3. information about the user's device, environment, and context

Invoked by the Flow Manager, the Matchmaker will return a comprehensive list of Solutions and settings for the user, which is in turn passed on to the Launch Manager. This information will then be used to invoke the relevant Settings Handlers and Launch Handlers.

The Cloud4all/GPII architecture currently ships with several default Matchmakers, and several others are in active development:

  • A statistical Matchmaker, which directly infers application-specific settings using a complex statistical algorithm
  • A rule-based Matchmaker, which evaluates a set of expert-defined rules for resolving conflict scenarios (such as when a preferred solution is unavailable on the current device)
  • The Flat Matchmaker, which makes a direct match between user preferences and the declared capabilities of available solutions

The Matchmaker Flow

The Cloud4all architecture defines a common, high-level workflow for all Matchmakers. This process is managed by the Cloud4all framework. Like the rest of the architecture, it is event based and configuration-driven, enabling each Matchmaker to easily customize the default workflow and inject custom behaviour at any phase in the process without requiring and code to be cut and pasted or duplicated. Currently, the matchmaking workflow is roughly as follows:

  1. Fetch solutions from the Solutions Registry
  2. Filter the solutions based on:
    1. Platform
    2. Platform version
    3. Device
  3. Match
    1. Modify, synthesize, or infer user preferences
    2. Modify solutions list
    3. Run custom matchmaking strategy (using preferences set and solutions list)
    4. Produce and return the appropriate list of solutions and settings for the Lifecycle Manager

Lifecycle Manager and Lifecycle Actions

Located on the user's device, the Lifecycle Manager receives the list of appropriate solutions and settings from the Matchmaker and manages the process of setting them up. This configurable process generally includes:

  1. Taking a snapshot of the device's current settings so they can be restored later
  2. Invoking the appropriate Settings Handler for each solution
  3. Launching each solution
  4. Stopping each solution when the user is done
  5. Restoring the previous settings (if required) once the user is finished

Each of these activities is accomplished by Lifecycle Actions, which are functions that can be registered with the architecture and are responsible for performing a single, specific task of the configuration workflow.

The Cloud4All realtime framework has a set of built-in Lifecycle Actions, including:

Special Lifecycle Handlers:

  • "setSettings" special lifecycle handler that tells the settings handlers to set the settings
  • "restoreSettings" special lifecycle handler that tells the settings handlers to set the settings

Lifecycle Handlers:

  • GSettings handler: can set gsettings values used in Gnome (linux) and other
  • System call handler: execs system commands.

Settings Handlers

Settings Handlers are a type of Lifecycle Action that are responsible for configuring the settings of an application, access feature, or assistive technology. They are often, but not always, specific to a particular platform. Platform-specific Settings Handlers typically interface with the operating system's built in APIs for storing settings (such as the Registry on Windows and GSettings on Linux). Others are capable of configuring standard formats such as XML and JSON files.

Settings Handlers are configured declaratively (within a JSON-based Solutions Registry entry), and assistive technology developers will typically use the ones that are built in to the system. In cases where the default Settings Handlers aren't sufficient, developers can choose to write their own Settings Handlers and hook them into the GPII Personalization Framework.

Out of the box, the Cloud4all/GPII architecture ships with the following Settings Handlers:

  • XML files
  • JSON Files
  • INI files
  • Linux GSettings
  • Linux DBus
  • Windows Registry
  • Windows SystemParametersInfo

Preferences Server

The Preferences Server stores user preferences. These preferences include general preferences ("I prefer high contrast"), context-specific preferences ("In the evening, I want the text to be 1.5x larger"), and application-specific settings ("I want X screenreader to read at Y speed").

The Preferences Server is a web-based service that provides a REST API for accessing preferences in a JSON format. Preferences are be represented according to a new version of the AccessForAll ISO 24751 standard. More information about this evolving standard is available in the GPII wiki on the AccessForAll working group's discussion page, and a summary is also provided below.

In the case of the GPII Preferences Server, the primary resource (in the RESTful sense) is the user and their preferences. These are identified by URLs such as:

http://preferences.gpii.net/users/<anonymous user token>/preferences

The Preferences Server supports multiple representations of a user's preferences, corresponding to particular formats and structures that are optimized for distinct use cases and implementations. While the server does indeed offer a standardized JSON representation, consisting of the preferences mapped as flat key-value pairs that correspond to the AccessForAll preferences registry, other formats can also be provided and used as needed via the Transformer. For example, a more terse and hierarchical JSON structure suitable for the high-performance processing and interchange requirements of the GPII Personalization Framework. One can even imagine HTML representations of the user's preferences, enabling them to review the preferences they've set and edit them accordingly.

Example Preference Sets

Flat, URL-keyed Preferences Profile in JSON Format (this is the default format)

{
    "http://registry.gpii.net/common/language": [
        {
            "value": "fr-ca"
        }
     ],
    "http://registry.gpii.net/common/speechRate": [
        {
            "value": 100
        }
     ],
    "http://registry.gpii.net/common/repeatKeysRepeatRate": [
        {
            "value": 0.5
        }
     ],
    "http://registry.gpii.net/comon/repeatKeysAutomaticDelay": [
        {
            "value": true
        }
     ]
}
 

Hierarchical Preferences Profile in JSON Format Conforming to the Current ISO 24751 Standard

{
    user: {
        language: "fr-ca"
    },

    display: {
        screenReader: {
            speechRate: 100
        }
    },

    control: {
        alternativeKeyboard: {
            repeatKeys: {
                repeatRate: 0.5
                automaticDelay: true
            }
        }
    }
}

In each case where the the Preferences Server exposes a particular data representation (in JSON format), there will also be an accompanying "schema" or map, describing how a data structure corresponds to the underlying AccessForAll preferences registry. This will ensure that multiple formats can coexist harmoniously; each representation will be self-describing, ensuring that implementations can easily understand the format and its meaning. Here's an example of how this schema might look:


{
    "user.language": "http://registry.gpii.net/common/language",
    "display.screenReader.speechRate": "http://registry.gpii.net/common/speechRate",
    "control.alternativeKeyboard.repeatKeys.repeatRate": "http://registry.gpii.net/common/repeatKeysRepeatRate"
    "control.alternativeKeyboard.repeatKeys.automaticDelay": "http://registry.gpii.net/common/repeatKeysAutomaticDelay"
}

In this example, keys within the hierarchical structure (separated by dots) are mapped to their corresponding AccessForAll Registry entries via their unique URI. Any application built in a Registry-aware way would be able to understand the meaning of this preferences profile, regardless of its particular structure.

In this way, the Preferences Server takes advantage of RESTful principles and self-describing Resources to ensure that we can widely interoperate, supporting the diverse needs and requirements of implementers.

Solutions Registry

The Solutions Registry is a comprehensive source for information about applications, access features, and assistive technologies (referred to as Solutions). It can be queried to determine which solutions are available to the user on their particular platform. The Solutions Reporter consists of declarative information about a solution, how it is configured, and what features it provides. This information can then be used to make a successful match with the user's needs and preferences. The Solutions Registry is where new implementers of GPII/C4A will add entries that describe their solution.

The solution registry is held in a document-oriented database. Like all other documents in the Cloud4All reference implementation, the format is JSON.

This list of solutions will include both those locally available (AT installed on computers, features built into the OS) and available via the cloud (services that the user is subscribed to or free services listed as preferable).

In order to register their software with the Solutions Reporter, developers need simply to create a standard JSON file called a Solutions Description, which will contain:

  • a name, ID, and other basic information about the assistive technology
  • the name of a built-in Settings Handler suitable for how it stores its settings
  • a Settings Transformation Map, which describes how to map standard AccessForAll user preferences into the assistive technology's native settings format

In the future, graphical tools can be provided so that solutions developers (or even the broader AT user community) can create new Solutions Registry entries without having to hand-author JSON files directly.

This solution metadata is used by several components of the GPII:

  • The Match Maker to know what needs and preferences are matched by solution.
  • The Lifecycle Handler to know how to start and stop the application and detect whether it's running, and similar information used by other lifecycle actions.
  • The Settings Handler for information about how to find and set the preferences
  • The Device Reporter via the [Local Solution Registry] to find out what solutions are installed on the system.
  • The Transformer for information about transformation between common terms and application specific settings

The Format of a Solution Entry

Each application will have at least one solution entry. If it behaves and/or is handled differently on different operating systems, versions, etc., it might have more than one solution entry.

The solution entry is a JSON object with at least the following mandatory fields:

  • name (string): The name of the application/solution in a human readable format.
  • id (string): Should be a unique ID for the application. The general format used is eg. "com.microsoft.win7.magnifier"
  • contexts (object): Describes the context required for this solution to be applicable
  • lifecycleManager (object): Instructions for the Lifecycle Manager describing what should happen when a user logs in and logs out
  • settingsHandlers (array): An array of settings handlers (and their settings) that should used to configure this solution

Device Characteristics Reporter

The Device Characteristics Reporter provides information about the user's device. This includes operating system, versions, and the list of applications currently installed. It will also provide information about the device's capabilities, such as screen resolution and size, support for features such as HTML5, and the availability of devices such as microphones and cameras, etc.

Over time, the Device Characteristics Reporter will be harmonized with evolving standards such as W3C Media Queries and the HTML5 device APIs.

Preferences Editors

Given Cloud4all's broad and diverse audience, it is clear that no single preferences editor user interface design will meet the needs of all users. There is an incredible diversity of experience levels, comfort with technology, ability, age, and other factors that influence the design of user interfaces for preference editing. We need to speak the language of learners, giving them an environment in which they feel comfortable discovering what they need and experimenting with new settings. The Cloud4all architecture needs to take into account this diversity, making it easier for designers and developer to create different user interfaces for different users.

To this end, the project's architectural approach emphasizes modularity and adaptability. Rather than creating the infrastructure for a single, hard-coded preferences editor, the goal is to provide the technical infrastructure that will allow for maximum diversity of user experiences while avoiding the proliferation of duplicated and mostly unshared code typical of ad-hoc UI development. The Cloud4all architecture follows Fluid Infusion's declarative Inversion of Control (IoC) philosophy, making it easier for third-party developers and designers to refine and extend Cloud4all's preferences editing environment over time. With Infusion IoC, instead of writing code with tightly-coupled dependencies, developers express their application structure as a declarative "component tree" that is instantiated and managed by a context-aware framework (Basman 1). This makes it possible to change the application later, either statically by third-party developers or at run time by evaluating contextual information such as device capabilities and physical sensors.

Given the diversity of platforms, tools, and devices that pepole are increasingly using in community centres, classrooms, and at home, the Cloud4all architecture needs to be broadly cross-platform. As mentioned above, web technologies such as HTML, CSS, and JavaScript are ideally suited to delivering cross-platform, cross-device user interfaces without requiring extensive code to be written for each new platform. The Cloud4all architecture embraces these standard tools on both the client and server (with Node.js), making it easier to integrate our preferences environment into the applications learners use the most.

In keeping with the overall AccessForAll approach, in which we understand disability and accessibility to be usability issues which we all experience, preferences are always represented functionally. They never require the user to identify medical conditions or limitations; instead, they express functional preferences that may be useful to a wide variety of users.

The PMT and PCP Editors

The two main preferences editors that are part of the Cloud4all deliverables are the Preferences Management Tool (PMT) and the Personal Control Panel (PCP). Both will be developed with a common core framework (the UI Options Preference Framework) and will share a consistent set of user interface controls and interaction idioms. They will, however, provide distinct user experiences that are optimized for different use cases. The PMT provides a more in-depth means to edit an entire preference set (including contexts and conditions where appropriate). The PCP, on the other hand, is optimized for quick, on-the-fly adjustment of frequently-used settings.

From an architectural perspective, both the PMT and PCP will be built using web technologies (HTML, CSS, and JavaScript) to ensure that they:

  1. can themselves be automatically personalized and transformed to suit a users preferences
  2. will be available across the myriad of platforms, operating systems, and devices we anticipate supporting in Cloud4all without costly rewrites
  3. will share a common design approach and technical foundation to many other aspects of the Cloud4all architecture, including the Flow Manager and the Preferences Server

The PMT will be accessed as a familiar in-browser web application, optimized for larger screens and longer time spans of usage. The PCP, on the other hand, will be always available on the particular device or system the user is interacting with. As a result, the PCP will be wrapped in a "native app" runtime so that, despite being cross-platform and built with web technologies, it will still work in offline contexts and idiomatically like any other app on the platform. This native wrapping will be accomplished on desktop using a Node.js-based runtime such as the Node Webkit project and on mobile with Apache Cordova (also known as "PhoneGap").


The UI Options Preference Framework

User Interface Options (UI Options) provides a programming framework with which new user interfaces for editing preferences can be more easily built. UI Options will provide all the lifecycle events, configuration hooks, and persistence infrastructure required to support a variety of different editing experiences, ensuring that there is a robust and well-tested framework in which to incrementally move the Cloud4all designs from prototype to production.

The UI Options framework is intended to give developers the ability to supply custom appearances, behaviours, and schemas for any preference, including new preferences that aren't part of the default UI Options package. This is achieved architecturally by modeling each preference as three discrete objects: a View (presentation) component, an Enactor (action) component, and a Persistence component. This three-part model ensures that developers and designers who either are using UI Options as a general framework for building a preferences editor (such as the Preferences Management Tool and Personal Control Panel) have complete control over how a preference 1) appears to the user, 2) is enacted, and 3) is stored.

Preference Components

Preferences-Architecture-Diagram.png

Diagram showing the primary architectural components of Preferences for Global Access, including the UI Options framework, the GPII Preferences Server, and server side transformation components.

  1. The View component is responsible for specifying the markup, styling, and template required to render the panel.
  2. The Enactor does the actual work of accommodating the user's preference. In the case of web application, Enactors are typically registered with the UI Options framework's UI Enhancer component, which is responsible aggregating multiple Enactors together and managing the interaction between these Enactors and the page's Document Object Model (DOM).
  3. The Persistence component provides schematic information about the preference---its value types, ranges and default values. Developers can also plug in custom Data Store components, which enable the preferences editor to connect to different data stores. The primary DataStore in Cloud4all saves preferences directly into the GPII cloud-based Preferences Server via its REST API.

The UI Options framework provides a declarative JSON data structure for registering custom components, enabling easy customization of the PMT, PCP, and other editors by third-party developers and integrators without having to modify any built-in code. As a result, the job of the developer is to compose preferences together into a tree of JSON configuration that is passed to the UI Options framework at instantiation time.

By separating the responsibilities of view and action, this architectural approach will enable developers to more rapidly create diverse preference editing experiences. For example, a designer may want to provide both a highly simplified user interface that is more fun and approachable for children as well as an in-depth experience for more technical users. In this case, the same Enactor code can be reused while different Views can be written and swapped in without having to change any core application code.

The UI Options framework also provides support for pluggable previews, enabling a user to see the effect that a preference change will have before committing to it. Developers can provide their own custom preview strategies. The default "live preview" strategy shows the changes applied to a live view of the application, allowing users to see their preferences in context. Another built-in strategy in include a "sample window" preview showing a small selection of typical content. We also envision specialized previews where the user can try out the preferences in practice in a safe environment before committing to them globally.

The Transformer and Data Lensing

As mentioned above, the Transformer is an interoperability service provided by the Cloud4All architecture to enable transformations between different formats, data structures, and ontologies. Currently, the Transformer is primarily used by Matchmakers to translate preferences between common terms and application specific settings. It is stateless, relatively easy to extend and has an easy-to-use API. Generally, transformations are defined by a developer (or user) in the description of a solution in the Solution Registry. They describe how an application specific setting should be understood and transformed into a common term and vice versa.


Examples of Transformation

As mentioned above, transformation rules for solution settings are declared along with the solutions in the settingsHandler section of the Solution Registry.

In its simplest form (without any explicit transformation), it might look something like this:

<syntaxhighlight lang="javascript">

              "capabilitiesTransformations": {
                   "Magnification": "display.screenEnhancement.magnification"
               }

</syntaxhighlight>

which would mean that the setting Magnification (which the application needs and understands) should take the value set by the user for the common term "display.screenEnhancement.magnification". Implied in the above declaration is that the unit and range for both Magnification and "display.screenEnhancement.magnification" is the same (eg. magnification in percent).

Another example would be:

<syntaxhighlight lang="javascript">

                   "Magnification": {
                       "transform": {
                           "type": "gpii.transformer.scaleValue",
                           "inputPath": "display.screenEnhancement.magnification",
                           "outputPath": "value",
                           "factor": 100
                       }

</syntaxhighlight>

Here, a proper transformation will take place. The user preference value for the common term "display.screenEnhancement.magnification" will be scaled by 100 before being assigned to the "Magnification" application setting. This would be relevant (as is in fact the case using the Screen Magnifier tool supplied by Microsoft Windows) if the common term was specified as a factor and the application only understands Magnification as a percentage magnification.

Before diving into the available transformations, a few words about the general structure of transformations.

The general structure of transformations

Transformations will always have the keyword "transform", which tells the system that it needs to do a transformation. The "transform" block will be replaced by the value of the transformation result, except for the special case where outputPath is declared. Each transform will have a type, describing what type of transformation is being declared. The "type" field is mandatory. inputPath refers to the preference of the users needs and preferences set that should be transformed. This can be implicit and defaulted for some transform situations, but is usually present. Depending on the type of transformation, several other parameters may be required - see the individual transformation type details below. Transformations can also take an optional outputPath key. When using this, the value of the transformation will be assigned to a key (perhaps composed of hierarchical segments) by that name, rather than the key of the object containing the transformation.

<syntaxhighlight lang="javascript">

                   "applicationSettingFoo": {
                       "transform": {
                           "type": "type.of.transformation",
                           "inputPath": "preference.from.np.set",
                           "outputPath": "optional-key",
                           "transformationParam1": "foo",
                           "transformationParam2": "bar",
                           (...)
                       }

</syntaxhighlight>

In the above example the key applicationSettingFoo would be a setting the application understands. After the transformation, the value of this key will become an object containing the key "optional-key" (from outputPath), with value given in inputPath ("preference.from.np.set") transformed as declared in the type using the parameters transformationParam1 and transformationParam2. Resulting in:

<syntaxhighlight lang="javascript">

                   "applicationSettingFoo": {
                       "optional-key": "transformed value of the preference preference.from.np.set"
                    }

</syntaxhighlight>

If no outputPath key had been present, the applicationSettingFoo would get the output of the transformation assigned directly:

<syntaxhighlight lang="javascript">

                   "applicationSettingFoo": "transformed value of the preference preference.from.np.set"

</syntaxhighlight>

Available Transformations

No Transformation

If the application setting should be set directly from the users Common Terms preference without changing the value, no explicit transformation is required. The syntax can be seen below:

<syntaxhighlight lang="javascript">

              "capabilitiesTransformations": {
                   "ApplicationSetting": "http://reg.gpii.net/ns/Some-Common-Term"
               }

</syntaxhighlight>

So <syntaxhighlight lang="javascript">

              "capabilitiesTransformations": {
                   "Magnification": "display.screenEnhancement.magnification"
               }

</syntaxhighlight>

which would mean that the setting Magnification (which the application needs and understands) should take the value set by the user for the common term "display.screenEnhancement.magnification". Implied in the above declaration is that the unit and range for both Magnification and "display.screenEnhancement.magnification" is the same (e.g. magnification in percent).

Scaling

If the units of the term and the application setting are related by some factor, the "scaleValue" transformation can be used. It requires the following key-value pairs:

  • "type": "gpii.transformer.scaleValue".
  • "factor"': the scaling factor. A factor of 2 would mean that the application setting will get the value of the user preferences multiplied by two.

An example can be seen below, where the application setting "Magnification" will be assigned the value of the user's setting multiplied by a 100. So if the user has a value of 2 in the "display.screenEnhancement.magnification" setting, the Magnification setting set for the application would be set to 200.

<syntaxhighlight lang="javascript">

                   "Magnification": {
                       "transform": {
                           "type": "gpii.transformer.scaleValue",
                           "inputPath": "display.screenEnhancement.magnification",
                           "factor": 100
                       }

</syntaxhighlight>

Value Mapping

If you have a finite set of mappings from user preferences to application settings, you can use the valueMapper transformation. As parameters it takes a set of options describing the mapping.

  • "type": "fluid.model.transform.valueMapper"
  • "options": an object with with keys being the input value looked up. The value matching the key will be output if the key matches the value of the inputPath

An example can be seen below. Here the application setting "mouse-tracking" can take two values: "mouse" and "caret". In the user preferences ("display.screenEnhancement.tracking"), the valid values are "centered" and "none". When this transformation is applied, a user preference of "centered" would be translated in "mouse" and be assigned to the application setting.

There are two optional and additional options to this transformation:

  • '"defaultInputValue": if no branch matches, it is attempted again using this value - WARNING, this seriously threatens invertibility
  • '"defaultOutputValue": if no branch matches, this value will be used as the output value (ie. in the below example, if the input value was neither "mouse" nor "caret", "mouse-tracking" would be assigned the value of the "defaultOutputValue" key

<syntaxhighlight lang="javascript">

               "capabilitiesTransformations": {
                   "mouse-tracking": {
                       "transform": {
                           "type": "fluid.model.transform.valueMapper",
                           "inputPath": "display.screenEnhancement.tracking",
                           "options": {
                               "mouse": {
                                   "outputValue": "centered"
                               },
                               "caret”: {
                                   “outputValue”: “none”
                               }
                           }
                       }
                   }
               }

</syntaxhighlight>

Ontologies and the Ontology Server

The ease of producing JSON schematic structures represented in JSON itself makes it economic and desirable to produce a dedicated piece of architecture for managing this process. Each of the JSON formats in which user preferences documents can be served can be thought of as being derived from an ontology - a schema describing the layout and meaning of the terms appearing in the format. Use of the model transformation and data lensing features referred to above makes it easy to provide an open infrastructure for ontologies - users can define and register new ontologies without writing programming language code.

The two formats we have described above in the Preferences Server section—the standard "Flat, URL-keyed Preferences Profile" as well as the "Format Conforming to the Current ISO 24751 Standard"—both constitute ontologies in our terminology, even though the first format conveys no ontological information beyond the names of keys. Further ontologies may easily be added to the system by integrators, AT vendors and other stakeholders simply by defining a transformation document in the format accepted by the model transformation system, which translates their ontology into one of the two standard ones. As long as there exists a path linked by transformations between any one ontology and another, the system can broker between preferences stored in the two formats automatically, ultimately for writing as well as for reading. Here is a sample document defining a transformation between the standard registry format and a part of the ISO 24751-2 schema (the hierarchy under "display.screenEnhancement") described above:

{
    "transformations": {
        "display.screenEnhancement.fontSize": "{http://registry.gpii.org/common/fontSize}.0.value",
        "display.screenEnhancement.foregroundColor": "{http://registry.gpii.org/common/foregroundColor}.0.value",
        "display.screenEnhancement.backgroundColor": "{http://registry.gpii.org/common/backgroundColor}.0.value",
        "display.screenEnhancement.fontFaceFontName": "{http://registry.gpii.org/common/fontFaceFontName}.0.value",
        "display.screenEnhancement.fontFaceGenericFontFace": "{http://registry.gpii.org/common/fontFaceGenericFontFace}.0.value",
        "display.screenEnhancement.magnification": "{http://registry.gpii.org/common/magnification}.0.value",
        "display.screenEnhancement.tracking": "{http://registry.gpii.org/common/tracking}.0.value",
        "display.screenEnhancement.invertImages": "{http://registry.gpii.org/common/invertImages}.0.value",
        "": {
            "transform": {
                "type": "gpii.ontologyServer.transform.application",
                "inputPath": "",
                "outputPath": "applications"
            }
        }
    }
}

(note that this transformation format is not final and will evolve over the next few releases of the Cloud4all realtime framework, in particular to eliminate the indirection ".0.value" appearing on the right hand side of rules)

The Cloud4all Architecture Reference Implementation

Components

Each component in the GPII Personalization Framework reference implementation is developed using a standard cross-platform technology stack. Components are written in JavaScript and hosted within the Node.js runtime environment. Node.js is a highly scalable, event-driven system that can run on all major desktop platforms (Mac, Windows, and Linux) as well as mobile platforms such as Android. It is commonly used as a server-side web development toolkit, but can be used to create any kind of local or connected application. It features prominently within the development tools offered on desktop and mobile platforms such as Windows 8.

Due to this design, most components within the realtime framework have the ability to run across platforms without recoding.

Messaging and Payloads

Components in the GPII Personalization Framework take input data and produce output in a simple JSON (JavaScript Object Notation) format. JSON is the lowest common denominator of interchange formats: it's widely supported and can be trivially parsed by any programming language and toolkit available today. It has a simple and lightweight syntax and maps directly to standard data structures. It is ideal for representing user preferences and application settings, and enables them to be easily transformed into other formats for interoperability.

Components in the system communicate with each other using simple REST (REpresentational State Transfer) conventions over the HTTP protocol. Each component is represented as a "resource" available at a unique URL. This approach specifically avoids a dependence on language or toolkit-specific messaging APIs that would prevent cross-platform compatibility. The system can easily be extended with new components and features by adding defining new URLs, and the dependency between components is configured, not hard-coded. Most importantly, it is trivial to move components between the local system and the web, ensuring that the system works as well in highly connected, cloud-hosted environments as it does without an Internet connection.

{
    "preferences": {
        "url": "http://gpii.net/preferences/%db/user/%token"
    },
    "deviceReporter": {
        "url": "http://localhost:8081/device"
    },
    "solutionsReporter": {
        "url": "http://localhost:8081/solutions"
    },
    "matchMaker": {
        "url": "http://localhost:8081/match?%query"
    },
    "launchManager": {
        "url": "http://localhost:8081/launch?%query"
    }
}

Local components are hosted within a single Node.js instance that responds only to internal HTTP requests. Implementers of the GPII Desktop system need only redeploy a local component to a remote web server and update a plain text configuration file in order to re-partition the system according to their needs.


Integration with Native Technologies

Although the Cloud4all realtime framework extensively uses JavaScript as the primary development language, it is also possible to integrate with native technologies such as operating system settings and device APIs. For example, Settings Handlers (described above), which are responsible for configuring the settings of assistive technologies, are often written in C, C++, or Java in order to bind to platform APIs such as the Windows Registry or Linux GNOME's GSettings. In cases where native bindings are required, the architecture provides several means to integrate them:

  • On Windows, Linux, and Android, native code can be called by creating Node.js modules in either C++ or Java that bridge between platform code and JavaScript
  • The Cloud4all real-time framework also ships with node-ffi, a Node.js module for calling "foreign" (i.e. C-based) functions based on the widely-used libffi library. The use of node-ffi enables Cloud4all developers to bind JavaScript code to C-based APIs without having to write any actual C code.
  • Non-JavaScript code can, in many cases, pass data to and from the Flow Manager by invoking its RESTful service endpoints and exchanging payloads in JSON. The USB User Listener, for example, which is responsible for invoking the Flow Manager whenever a user plugs in their GPII-enabled USB drive, communicates with the Flow Manager via REST.

The end goal is to ensure as much code as possible is rewritten in a cross-platform and highly reusable manner. However, whenever platform-specific native code is required, the architecture enables this while ensuring that it is carefully localized to avoid difficulties when unit testing or porting the code to new platforms.

Deployment

With the exception of the components that are designed to bind with platform-specific APIs, any component in the system can be located either locally on the device or out on the web. For example, some Settings Handlers and User Listeners are bound to the local device, though others are cross-platform. The Flow Manager, Preferences Server, Matchmaker, and Solutions Registry can easily move between locations. It's simply a matter of updating a configuration file for the Flow Manager.

Default Deployment Layout

Default-Architecture-Deployment-Profile.png

This is the default deployment layout of the Cloud4all/GPII architecture, but does not represent all of the possible places that any component could reside. For example:

  • Preferences can be on the server, or a USB drive, or in a phone, or in an NFC ring, or other
  • Matchmaking can be done locally, on the server, or distributed across both locations
  • Content transformation and transcoding can be done on the server, on the local device, or distributed

Web

  • Preferences Server
  • Solutions Registry
  • Matchmaker-specific server-side components (e.g. the Statistical Matchmaker's stats processing engine)

Cross-platform capable, but installed locally

  • Flow Manager
  • Device Reporter
  • Matchmaker
  • Transformer
  • Lifecycle Manager
  • XML and JSON File Settings Handlers

Platform-specific and installed locally

  • USB Drive User Listener (Windows and Linux implementations)
  • Registry SettingsHandler (Windows)
  • SystemParametersInfo SettingsHandler (Windows)
  • GSettings Settings Handler (Linux GNOME)
  • GSettings Lifecycle Handler (Linux GNOME)
  • DBus Settings Handler (Linux)

It's important to recognize that each component in the system doesn't be its own server. By default, we have two web servers:

  1. one that runs on the local device and is fully firewalled and secure
  2. a second server that contains both the the cloud-based Preferences Server the Solutions Registry

When components are co-located in the same application, they are able to communicate via direct function calls (without the overhead of REST). This, too, is a configurable aspect of the Flow Manager.

We expect mobile devices can be supported on most platforms using the same approach. iOS is always a more complex issue due to Apple's very restrictive terms, but a comparable implementation using Apple's own JavaScript runtimes should be possible to implement.

Platform-Specific Support

Mobile

Web Applications

Web-App-Deployment-Profile.png

Architectural Considerations & Non-Functional Requirements

Security and Privacy

A user's preferences may reveal personal information about their lives and needs, either directly or indirectly; privacy and security is thus of critical importance to the design of the Cloud4all architecture. While we are still in the process of designing the Cloud4all security and privacy infrastructure as part of Work Package 104.2, the intention is to use tools and standards that are broadly used on the web and in industry, such as the OAuth 2.0 framework. OAuth will help ensure that the Security Gateway and core architecture will protect access to a user's preferences by third party applications such as the web sites, content delivery tools, and other appilcations (Hardt 2012). The use of OAuth will let users approve or deny access to their information on a per-site basis, ensuring they control who gets to see and act on their needs and preferences.

After the basic OAuth 2.0-based authentication infrastructure is established, an attribute release layer such as the Kantara Initiative's User Managed Access will be layered on top (Machulak 2010). This will give users the further ability to specify that a site or application is able to see only portions of their preferences set, ensuring that the risk of "privacy leakage" is reduced by only sharing the minimum information required by a service to meet the user's needs (Krishnamurthy 2009).

The architecture's security and privacy infrastructure will be developed as a set of low-level libraries that can be integrated directly into the realtime framework's Node.js-based web services, ensuring that all components within the system can be protected appropriately and in a performance-optimized manner.

A key goal of this effort is to ensure that this architecture can support diverse privacy and security requirements internationally.

For other information about Cloud4all's security and privacy infrastructure goals, see available in Cloud4all Deliverable 104.2, Security, privacy and ethical policy assurance system requirements and design.

Performance, Scalability, and Cloud Technology

Performance and scalability are critical concerns of the Cloud4all architecture. The Cloud4all architecture and realtime framework have been designed specifically to be deployed on cloud-based infrastructure such as OpenStack, Amazon EC2, and more.

In particular, key components of the Cloud4all architecture have been designed to avoid heavy server-side session state, which is a common barrier to cloud-scale horizontal load balancing. As a result of this statelessness, many instances of the Cloud4all services can be "spun up" and run cooperatively with a load balancer such as HAProxy in order to spread high load more evenly across multiple servers and cores. Traditional state-heavy models such as Enterprise Java Beans-based architectures have struggled to scale in this manner, whereas Cloud4all's realtime framework makes use of non-blocking, event-based JavaScript tools such as Node.js that are specifically designed for this kind of scaling.

The use of document-oriented databases and map/reduce querying techniques, originally popularized by Google for their large-scale search infrastructure, also contributes to Cloud4all's potential scalability (Dean and Ghemawat). The realtime framework extensively uses Apache's CouchDB database, which enables master-to-master replication over a large cluster of database instances and provides several options for cloud-based sharding of the database. CouchDB has been used in a number of cloud-scale production deployments in industry.

In order to ensure that the realtime framework is "cloud native," we have continually deployed and tested it on a large OpenStack-based cloud cluster (currently comprising 35 compute nodes powered by three Nebula cloud controllers running at the IDRC in Toronto). Work is underway to create automated benchmarking and load testing scripts to ensure that scalability is an integral metric to our architectural and development process. Automated deployment recipes and virtual machine profiles are also being created in order to streamline the process of spinning up new Cloud4all virtual machine instances. These recipes and tools will ultimately ship with the Cloud4all realtime framework as a toolkit for administrators and production engineers who will deploy Cloud4all internationally.

Modularity and long-term growth

Community-driven

As mentioned above in the section on #Open Source Development, open source practices are an integral part of the design and development of Cloud4all's architecture and realtime framework. Fostering a community is a significant strategy to support the long-term growth, viability, and sustainability of Cloud4all's technology throughout and beyond the initial grant funding period. By promoting open governance practices and an inclusive meritocracy, contributors and users will be able to influence and support the direction and ongoing progress of Cloud4all's technical infrastructure.

Summary, How the Cloud4all Architecture Pushes the State of The Art, Next Steps