Bridging the Browser-OS Personalization Gap

From wiki.gpii
Jump to: navigation, search

Overview

In previous versions of the GPII architecture, the personalization workflow within the browser and on the local operating system were completely separate. On one hand, there is the locally-installed Flow Manager and Settings Handlers, which are capable of configuring, launching, and restoring the settings of native applications and operating systems features. On the other, there is the Cloud-Based Flow Manager and web content personalization enactors such as those that are part of the Preferences Framework and Fluid Infusion.

The Cloud-based Flow Manager’s personalization workflow is "pull based," while the local workflow is based on a push model. As a result, web applications have been expected until now to be explicitly GPII aware; such apps independently manage a linkage between their account system and the GPII's user tokens (via OAuth 2.0), querying the Cloud-Based Flow Manager to retrieve preferences and settings. In contrast, on the local device the Flow Manager does the work of pushing settings down to the Settings Handler components without the applications having any knowledge of the personalization process.

While these two separate scenarios helped to address a number of basic personalization situations across both the web and local devices, the Cloud4All architecture is being extended to support a hybrid approach where a web-based application can be notified—using a browser-based JavaScript API—when there are user needs that need to be fulfilled. This improved workflow allows a personalization-aware web application to automatically adapt itself as soon as a user browses to the site, rather than relying on the user first authenticating with the web application.

In practice, this involves the locally-installed Flow Manager pushing user preferences into the browser's sandbox, where they can be accessed by web content. This, of course, depends on the custom GPII browser extension to mediate the relationship, since Web content is not allowed to make persistent connections to domains other than the one that serves the site. This is called the Same Origin Policy, and is a crucial basis for Web security . Notably, any implementation that intends to provide a communication mechanism across origins needs to meet a high standard of security and privacy; in the case of GPII, it should:

  • always ask for user approval prior to sharing a preference set with a third-party application
  • only reveal preferences that are appropriate for the requesting application and its autopersonalization capabilities
  • never reveal indirect personal data such as tokens or account information

Another challenge of a hybrid approach is to avoid the problem of "doubly enacting" the user's needs and preferences. For example, imagine that a user needs a larger text size to be able to read the screen. Depending on the type of preference editor, they may have expressed this need in relative terms, for example "I need text that is 2x larger than the system default." If a locally installed access feature (such as the built-in OS font size settings) is able to satisfy this particular need, a web application should not also enact it. Otherwise, the risk is that the user will actually get a font size that is 4x larger than the default. A bridge between the local system and browser will, along with the Matchmakers, need to take into account preferences that have already been satisfied and not pass them on to web content (or vice versa).


Technical Background

Support for the IETF and W3C Web Socket standards was introduced in the GPII real-time framework in early 2013. It was initially provided in order to support the Preferences Management Tool (PMT) and Personal Control Panel (PCP) preference editors, which required a bi-directional means to share preference set changes and status information with the Flow Manager component.

More recently, Web Socket support was improved in order to support integration with other distributed Cloud4all components including:

  • The Context Aware Server, which is a remote server that requires a persistent means by which it can stream data about changes in the environment such as ambient lighting, temperature, and sound levels to a locally-installed (i.e. on a personal Windows, Linux, or Android device) Flow Manager instance.
  • The Chrome and Firefox extensions, which provide automatic adaptation of web-based content and need to be informed whenever preferences change, but can only make limited calls outside of the browser sandbox.

The Web Socket Settings Handler

The Flow Manager includes a Web Socket Settings Handler, which provides a socket server to which web-based clients (such as the Chrome and Firefox extensions) can connect on startup and receive push-style preference notifications whenever a user logs in, changes their settings in an application such as the PCP, or logs out. Since browser extensions cannot typically make persistent connections using traditional protocols such as TCP or UDP, the Web Socket Settings Handler provides a web-friendly means by which data can be streamed from the Flow Manager to the browser extensions.

  1. The Flow Manager exposes a new bidirectional Web Socket-based service API called browserChannel.
  2. After connecting to the Flow Manager’s browserChannel service, the client sends a message to the channel identifying itself to the Flow Manager. In this case, the Cloud4All Chrome extension would pass along a string containing its unique application ID, org.chrome.cloud4chrome.
  3. The Flow Manager will emit either the connectionSucceeded event if the connection was successful.
  4. Whenever a user keys in or out of the system, or if a setting is changed in an editor such as the PCP, or the onBrowserSettingsChanged event is fired, passing along the fully filtered and contextually-appropriate preference set (in application-unique format).


The following is an example of the payload provided to clients of the Web Sockets Settings Handler:

{
   "screenReaderTTSEnabled":false,
   "highContrastEnabled":true,
   "invertColours":false,
   "magnifierEnabled":true,
   "magnification":2,
   "fontSize":"medium",
   "simplifier":false,
   "highContrastTheme":"white-black"
}

Figure 2: An example payload of settings sent from the Web Sockets Settings Handler to the Cloud4All Chrome Extension. These settings have already been matched, filtered, and transformed by the rest of the autopersonalization components in the system.

The Proposed Mechanism

Flow Manager Support

As mentioned above, the Flow Manager provides Web Socket-based infrastructure for exposing a bi-directional persistent connection that represents a stream of "preference changes." A subscriber to this stream will receive a privacy-filtered set of preferences and settings whenever a user keys in, updates their preference set, or keys out. If a subscriber connects after a user has keyed in, they will be provided with the current preferences and settings immediately. As with all other components of Cloud4All, the user’s NP set will be filtered by the user's security policies and the stated capabilities of the requesting application.

Browser Extensions

On the browser side, an extension (we currently support Firefox and Chrome) will mediate between sandboxed web content and the local Flow Manager. Upon startup, the extension will open the Web Socket connection with the local Flow Manager. It will expose new methods to content scripts for requesting access to the user's preferences and registering/unregistering event listeners for the stream of preferences changes. The extension will also provide the user with a user interface for confirming/denying a site's access to their preferences.

This work relates directly to the W3C’s Indie UI: User Context specification, which among other things attempts to provide an API for browser-based applications to access key user settings related to accessibility. However, Indie UI limits itself to a fixed collection of queryable settings, where as Cloud4All supports a potentially unbounded taxonomy of user needs, preferences, and settings. The Indie UI User Context specification is still in early development, and efforts will be made to contribute to and harmonize with it based on our practical and research experiences. Several Cloud4All contributors participate in the Indie UI working group at the W3C.

Global gpii

requestPreferencesAccess() returns Promise resolving to a PreferencesAccess instance

PreferencesAccess

Function addEventListener(String path, Function callback)
EventHandler onPreferencesChanged(Change change)

Change

valueany valueThe new value held at the model path
oldValueany valueThe previous value, before it was overwritten by the change
pathStringThe path at which this change occurred.