Personal Control Panel API

From wiki.gpii
Jump to: navigation, search

Overview

The goal of this page is to describe the communication protocol that allows the PCP to communicate with the core architecture and vice versa. Within this document, we will describe the requirements and the proposed approach to cover this functionality, that includes:

  • The communication channel between the PCP and the core architecture
  • The events and metadata that we want to handle

Introduction

During the Cloud4all project, a PCP API was created and implemented and although it did cover the requirements so far, it wasn’t incorporated into our master branch because it was mostly an initial proof of concept made up for demoing the concept. As part of the APCP project, the PCP API is one priority since we are going to deliver a PCP to real users for the very first time.

First, let’s describe the different terminology used along this page:

  • The Personal Control Panel (PCP) is the user interface to the GPII.
  • The communication channel is the messaging transport system between the PCP and the core architecture. In other words, the nexus between the PCP and the core architecture.
  • The Application Programming Interface (API) is the contract that both the PCP and the core architecture follow in order to communicate to each other.
  • An adjuster is the visual control that allows the user to change the value of a setting. We also call Widget set to the full set of adjusters that the PCP needs to support.
  • The metadata is the data that every supported setting must contain in order to render its adjuster in the PCP.
  • The PCP connector is the chosen name to describe the component that will be the entry point for the PCP, in other words, the endpoint in the core architecture that the PCP will connect to.

Requirements

The requirements for the PCP API come from the wireframes provided by the UX team, which at the same time has been collaborating with the core team to create such designs.

The wireframes of the PCP can be found here (https://issues.gpii.net/browse/GPII-2137). Along with this wireframes, we also have designs for the different adjusters (https://issues.gpii.net/browse/GPII-2138) that the PCP must show to deal with the different kind of settings that the system supports.

The API and the communication channel

General assumptions

  • The connection is being started by the client (the PCP)
  • The PCP must be able to receive information about changes in login/logout status
  • The PCP must be able to receive information about settings changed in the system
  • The PCP connector must emit an event whenever a user has logged in (or out)
  • The PCP connector must emit an event whenever changes in the settings or context happen
  • The PCP must receive the active settings and its metadata (see Settings’ metadata below)
  • The PCP must be able to adapt according to changes in the settings
  • Ideally, and according to the wireframes, The PCP must be able to start/stop applications/ATs
  • The PCP must be able to receive messages from the system
    • These messages can be either info, warning or error messages
    • The error messages will be described by the UX team

Also, and in relation to the communication channel, since the PCP runs as part of the electron process, this will allow us to use IPC as the communication channel. But for the first implementation, the communication with the rest of GPII happens via sockets (and potentially, will be moved to use IPC in the future). Since the browser channel already implements a similar concept we will use the same approach for the PCP connector, and potentially, they will share some logic in the architecture.

Adjusters and metadata

Settings' Metadata

The metadata of the settings will provide all the relevant information about a concrete setting so the PCP can render the adjuster appropriately.

Next, here is the list of some of the information that we would like to include as part of the metadata.

  • Application: Information about the application, including human readable name and application id. Should be undefined if the setting is common term.
  • Dynamicity of the setting (dynamic/non-dynamic)
  • The schema that provides all the information to the PCP in order for this to render the widget of a particular setting, this will include:
    • Setting name (in a human readable format)
    • Description - Additional human readable information about the setting. This could be a short descriptive paragraph, or perhaps just a link to further documentation on the applications website.
    • Type of the value
    • Default value
    • Value range
    • Increment value

Also, we would like apply some validation to this data, specially, to the schema that represents the value.

Adjusters (or widget set)

An adjuster is the control that allows the user to “tweak” a setting and we have identified the following controls that need to be handled by the PCP.

  • Text entry
  • Button
  • Binary on/off slider - on/off switch
  • Binary x/y "radio-style" button (we can use the "discrete" type to represent the value range)
  • Number slider, +/- and manual number input
  • Multiple list checkbox-style
  • List drop down-style

All the widgets will be rendered appropriately in the PCP based on the information provided by the core architecture, more precisely, from the information provided in the settings metadata.

Examples of settings metadata

Here are a few examples of the settings' metadata that can be used to render the different adjusters described above.

String from a list

The value of this setting is a string and must be either "word", "line", "sentence" or "paragraph". Also, it defines the default one, although we will receive its current value as part of the payload.

"http://registry.gpii.net/common/readingUnit": {
    schema: {
        title: "Reading Unit",
        description: "Reading mode when producing text to speech",
        type: "string",
        default: "sentence",
        enum: ["word", "line", "sentence", "paragraph"]
    }
}

Binary setting

"http://registry.gpii.net/common/wordEcho": {
    dynamic: false,
    schema: {
        title: "Word echo",
        description: "Whether to speak each word as it is typed",
        type: "boolean",
        default: false
    }
}

Number setting

The value of this setting is a number, has a minimum value (1), the incremented unit is 1 and its default value is 150.

"http://registry.gpii.net/common/speechRate": {
    dynamic: false,
    schema: {
        title: "Speech rate",
        description: "Text to speech output speech rate",
        type: "number",
        default: 150,
        min: 1,
        divisibleBy: 1
    }
}

Multiple selection

This setting can accept multiple values from a given list.

"http://registry.gpii.net/common/trackingTTS": {
    dynamic: false,
    schema: {
        title: "TTS tracking mode",
        description: "Tracking mode for TTS",
        type: "array",
        default: "focus",
        enum: ["mouse", "caret", "focus"]
    }
}

The PCP API (or messaging protocol)

The PCP connector will keep a model that will include the relevant information that helps the PCP to render the UI. For this, the API will consist of an endpoint in the flowManager that will allow the PCP to subscribe to the changes in the bind model and that can also issue changes on this model.

This concept has been successfully implemented already and it is known as The Nexus, we are just reusing the same approach.

The flowManager will keep a model for the PCP and the information included in this model will consist in filtered content from payloads coming from different components and the solutions registry.

This is a draft example of this model:

{
    userToken: "430500ee-a614-4032-ba7d-fda5bfbd8636",
    solutions: {
        "com.freedomscientific.jaws": {
            name: "JAWS",
            settings: {
                "options.SayAllIndicateCaps": {
                    dynamic: false,
                    schema: {
                        title: "Announce capitals",
                        description: "During SayAll, indicate initial cap or capitalized word",
                        type: "boolean",
                        default: 0
                    }
                }
            }
        }
    },
    preferencesSet: {
        gpii-default: {
            applications: {
                "com.freedomscientific.jaws": {
                    active: true,
                    settings: {
                        "options.SayAllIndicateCaps": 1
                    }
                }
            }
        },
        subway: {
            applications: {
                "com.freedomscientific.jaws": {
                    active: true,
                    settings: {
                        "options.SayAllIndicateCaps": 0
                    }
                }
            },
            conditions: [
                {
                    type: "http://registry.gpii.net/conditions/inRange",
                    min: 400,
                    inputPath: "http://registry\\.gpii\\.net/common/environment/visual\\.luminance"
                }
            ]
        }
    },
    activeContextName: "gpii-default",
    activeConfiguration: {
        applications: {
            "com.freedomscientific.jaws": {
                isRunning: true, // with the help of the process reporter
                settings: {
                    "options.SayAllIndicateCaps": 1
                }
            }
        }
    }
}


These are example of payloads that should cover most of the functionalities described in the General assumptions section. Most of the messaging that affects the model will be propagated bidirectionally between the PCP and the PCP Channel. Invoking actions such as logout or sending error messages between the two endpoints will use custom messages.

The PCP connects and there’s no user logged in into the system

Given the draft payload above, the PCP will connect to the PCP connector and this will return the empty model like the following.

{
    solutions: null,
    preferencesSet: null,
    activeContextName: null,
    activeConfiguration: {
        applications: null
    }
}

The PCP connects and there’s a user already logged in

In case there's an already logged in user, the payload received in the PCP side will be the same as the draft payload.

Key-in/Key-out events

Whenever there is a key-in event, the PCP connector will update the model and this change will be propagated to the PCP through WebSockets. The message will consist in the change of the model, expressed in the following way:

{
    type: "modelChanged",
    payload: {
        change: {
            path: "", // the change affects all the model
            value: { ... } // new value
        }
    }
}

Messaging

{
    type: "message",
    payload: {
        messageType: "error|info|warning",
        message: "Hey there!!"
    }
}

i.e.: an error event looks like the following

PCP connector message (event)

{
    type: "message",
    payload: {
        messageType: "error",
        message: "Aw, Snap!"
    }
}

The PCP performs logout

In order for the PCP to trigger logout actions, the PCP will be bound to a model in the core architecture that holds the logic for triggering login/logout actions. This way, the PCP will propagate a change on the model in order to perform a logout action.

PCP Request:

{
    type: "modelChanged",
    payload: {
        change: {
            path: "logonChange",
            value: {
                type: "logout",
                inProgress: true, // boolean
                userToken: "430500ee-a614-4032-ba7d-fda5bfbd8636", // string with user token
                timeStamp: Date.now()
            }
        }
    }
}

The PCP updates a setting

PCP Request:

{
    type: "modelChanged",
    payload: {
        change: {
            path: "activeConfiguration.applications["com.freedomscientific.com"].settings["options.SayAllIndicateCaps"]"
            value: 1
        }
    }
}

The PCP changes the context

{
    type: "modelChanged", // this event can be triggered at any time without any interaction from the PCP, ie: when context changes
    payload: {
        change: {
            path: "activeContextName",
            value: "subway"
        }
    }
}

The PCP connector must tell the PCP whenever the context changes

{
    type: "modelChanged", // this event can be triggered at any time without any interaction from the PCP, ie: when context changes
    payload: {
        change: {
            path: "", // it affects activeContextName and activeConfiguration
            value: {}
        }
    }
}

Doubts/Open Questions

  • What happens when the PCP gets settings for an application that is not currently installed on the system? (i.e.: JAWS) - Should we filter this out from the PCP connector side? Of course, It shouldn’t be a headache for the PCP but for the flowManager. UPDATE: UX will take a look at this and provide with wireframes.
  • Are we going to support feedback between the user and the core architecture? Should we implement the PCP equivalent of confirm or alert … so we can get feedback from users straight into the gpii architecture without making changes to PCP? We should check whether this needs to be supported or not. UPDATE: This is going to be supported but it is still unclear to what extent. This still needs to pass through UX.