Browser Adaptation Design Research

From wiki.gpii
Jump to: navigation, search


Browser adaptions refer to settings that are applied/enacted on a web page, mediated by the GPII browser extension. The exact means of enacting the settings may be through other browser extensions that are enabled by the GPII browser extension or through content scripts loaded into a web page by the GPII browser extension.


Colour & Contrast

The colour and contrast preference is a combination of two settings, highContrastEnabled and highContrastTheme. The first determines if it should be applied and the latter which contrast theme to use when enabled. Two potential solutions for applying contrasts themes to a webpage are 1) setting the colours and background colours of the elements on the page 2) applying a CSS filter to the entire page. A combination of the two options could also be used, for example changing the colours of text and backgrounds, and applying the CSS filter to <img> elements. Applying a page wide CSS filter may be the easier of the two options, although requires further investigation, as we could apply the change to a single location. However, it's unlikely that we'll be able to get true binary contrast e.g. white on black and etc because we are just applying a filter over the actual colours used on the page. Changing the foreground and background colours directly would allow for more specific control, but will be more difficult to accomplish in a general fashion across sites, as CSS specificity and priority rules may interfere with our ability to apply our setting to each element.

Questions / Answers
  • Should the contrast theme apply to images as well?
    • Answer: Do not change pictures
Example: Contrast applied to images
Default styling

 Exploration Tool website in the default style
Exploration Tool website in the default style
High Contrast: Images pre-processed by site creator
In the following example the Exploration Tool demo site itself provides images in an alternate contrast. The image used on the site has been pre-processed ahead of time by the site creator and made to work with the high contrast mode.
 Exploration tool website in high contrast
Exploration tool website in high contrast
High Contrast: Filter applied to entire page

In the following example, the CMHR website is shown with Google Accessibility's high contrast browser extension applying a grey scale theme onto the page. This is similar to how applying a CSS filter would work.

 CMHR website with grey scale applied by Google Accessibility High Contrast extension
CMHR website with grey scale applied by Google Accessibility High Contrast extension
Example: Contrast now applied to images
Default styling

 CMHR website in the default styling
CMHR website in the default styling
High Contrast: Applied to text, backgrounds and borders
In cases where foreground and background colours are adjusted, the images will remain in their original colours. Note that in cases where a background image is used on an element, these will either be removed or show up as normal. Unfortunately these are used for many different cases. For example an image on a page may be implemented as a background image to make changing or sizing easier or an image may be used to create a gradient or other specific colour/pattern for the background of a button.
 CMHR website in high contrast
CMHR website in high contrast


The Text-to-Speech setting, selfVoicingEnabled, refers to an implementation where the text of a webpage is auto-read to the user and highlights the text as it plays. There should also be a control to play/pause the speech. The Preferences Framework Demo and the First Discovery Tool both support self voicing, although neither have a pause/resume feature nor support text highlighting. Initial designs for a more full featured text-to-speech widget were mocked up by the Fluid team, but have yet to be implemented. The full set of designs are probably not in scope for this stage of development. Furthermore, in browser speech synthesis requires that the text be passed to a JavaScript API for processing. Unfortunately we only know when the text passed in has finished being uttered, but not each word unless we pass in text one word at a time. The benefit of passing in text word-by-word, is that we'll be able to highlight individual words as they are being uttered. However, there is a higher chance that words will be mispronounced, as there will be no context for the speech synthesis. An additional benefit to passing in text as a group (based on existing HTML tags on the page like <p>), is that it will be easier to implement the highlighting and will provide more context for the speech synthesizer.

The Cast site provides an example of play/pause and stop features for Text-to-Speech. Also Browsealoud provides a "toolbar" like widget with additional features.

Questions / Answers
  • What is the user interface for the controls?
    • Answer:
      • RIGHT click to highlight a paragraph and auto read it
      • RIGHT select will only read the selected block
      • ESCAPE exits the reading mode
      • X that appears at top of paragraph being highlighted and read will also EXIT the read mode
      • OPTIONs (Preference settings)
        • Highlight and read block (no word by word highlighting)
        • Highlight word by word as Read
        • Text block stays highlighted when words within it are individually highlighted as they are read.
      • Read OPTIONS (Preference settings)
        • Read till end of selected paragraph.\/block and then stop
        • Read from this point until user stops
      • COLOR OPTION (Preference settings)
        • Can choose highlight color for block
        • Can choose highlight color for word by word
      • ICON appears on end of the UI OPTIONS+ TAB that allows you to turn  this on and off with a click.
        • This allows user to get their right click back when they need it
        •  ??  A speaker icon also shows up that turns off the speech which clicked
          • So user can use highlight without speech at any time    (how useful?)
  • What should happen to the highlight when the speech is paused? Should it remain on the last word, should it disappear?
    • Answer: Highlight should stay where pauses, and restart from that location when unpaused.

Table of Contents

The table of contents setting, tableOfContentsEnabled, will display a table of contents for the current page based on the headers in its content. With UI Options, the table of contents is injected into the page at a location specified by the integrator. For browser extensions, it tends to be handled in a UI developed by the extension author and made visible by the user clicking on a button in the browser's toolbar.

Questions / Answers
  • Should we use a browser extension? ( Likely will require the user to click a button to show the table of contents )
    • Answer: Okay for short term
  • If injecting the table of contents into the page, where should it be placed?
    • Answer: As on current UI Options. Right before <H1> in <MAIN>. Top of the page with skip link to jump over content. Maybe on left side of wide pages.
Example: UIO Injected Table of Contents

UI Options injects a table of contents into a location on the web page that is predefined by the integrator.

 UI Options demo showing the Table of Contents
UI Options demo showing the Table of Contents
Example: Browser Extension Table of Contents
HTML5 Outliner

HTML5 Outliner adds a button to the browser's chrome that a user can click to bring up a Table of Contents in a dialog.

 HTML5 Outliner's table of contents dialog
HTML5 Outliner's table of contents dialog

Table-of-contents-crx injects adds a button to the browser's chrome that a user can click to bring up a Table of Contents as sidebar on the page.

 Table-of-contents-crx's table of contents sidebar
Table-of-contents-crx's table of contents sidebar


The syllabification setting, syllabificationEnabled, will break down a word into its syllables. For example "inclusion" will appear as in·clu·sion. Modifying the text in place will require locating, transforming, and replacing the text in place. Will also need to find a library or service for determining how a word should be broken down into syllables.

  • What is the user interface for Syllabification?
    • Should all text be syllabified or only selected text?
      • Answer: All content text, but not navigation text or interface elements
    • Should syllabified text be changed inline or be displayed in a dialog?
      • Answer: inline like in Learning Tools
  • What is the use case for Syllabification? There aren't a lot of tools available for splitting words into their syllables. I've implemented a test page to compare two such tools against results from some dictionaries.
    • Answers:
      • Syllabification can be helpful for some people when reading and trying to sound out words. Microsoft has found utility in including syllabification in some of their tools.
      • From the test page, it looks like Hypher would be the best tool for now. The other option (nlp-syllables) seems to make more mistakes that would affect pronunciation. In the future it would be good to improve the performance of the hyphenation engine (perhaps with an exception table, which Hypher supports).
Instant Dictionary

The Instant Dictionary browser extension allows a user to double click a word to open a dialog containing the definition and syllabification.

 Instant Dictionary's displaying a dialog with the syllabification and definition of the word "welcome"
Instant Dictionary's displaying a dialog with the syllabification and definition of the word "welcome"
Microsoft Learning Tools

Microsoft's Learning Tools in OneNote provides an option to enable Syllabification of all of the words in a page, but requires the user to place OneNote in the "Immersive Reader" mode first.

 Microsoft Learning Tools in OneNote with the Syllables feature enabled.
Microsoft Learning Tools in OneNote with the Syllables feature enabled.

Character Spacing

Character Spacing, also referred to as letter-spacing or tracking, will adjust the space between all of the characters uniformly, as opposed to kerning which adjusts the space between individual characters. (See: ). In terms of character spacing, I haven't located research on dynamically changing it, or what exact values are helpful. Some suggestions include wide spading or monospaced fonts ( ) for those with vision impairments and that wider spacing may help for those with dyslexia distinguish the characters ( ). Also the font weight, size, and capitalization may all affect the need and amount of spacing required ( e.g. content text vs headers ). And in practice, the readability of the spaced characters may be affected by word spacing, line spacing and potentially other aspects of the layout. Also of note, letter-spacing using CSS in the browser is additive. That is, it adds some amount, which can be in a relative value, to the fonts own default letter spacing ( ). This means that, unlike with font size, we cannot do a true multiplier for the value change. Another thing to note is that if a letter-spacing value in CSS is already applied on the text we either won't be able to affect it, or we will override it.

Questions / Answers 
  • What is the user interface for Character Spacing?
    • Should it be some sort of range like font size and line spacing ( Note: these are defined as multipliers ) or should it be more like fixed options (e.g. small, medium, large, x-large).
      • Answer: Should be a range adjuster list font size and line spacing. The range should be wider than anyone has called for and we can figure out the optimum range over time.
    • What should the amount of letter-spacing be?
      • Answer:
        • 0.2 or 0.5 CSS pixel increments times fontsize/12
          • We should be able to make use of em values for this
    • Should letter-spacing apply uniformly across all text? That is same for headers and paragraph text. (Note: if we need to know more detailed characteristics of the content like current font-weight, capitalization and etc. we won't be able to implement in pure CSS. It will be more difficult to implement and require more resources to run).
      • Answer: Yes across all, and relative to font size

Testing / Adaptation Coverage

The goal of the browser extension is to provide a reasonably generalizable set of strategies for applying adaptations to web pages. Due to the many variations in website creation from things like styling strategies, adherence (or not) to specifications and best practices, legacy markup/styling, specificity/precedence, and etc, it is not possible to perfectly adapt every site on the Web. Ideally we'll have a set of sites to test as a baseline, perhaps sites important to and/or likely to be used at the American Job Center (AJC). However, it may still not be possible to make all of these suggested sites fully adapted, without site specific strategies employed.

Questions / Answers

  • Which websites should we test the browser extension adaptations with?
  • Do these sites need to be 100% adapted?
    • Answer: Most important parts are the content.

Example: High Contrast

Default Styling

 A screenshot of a google sheet with the default styling
A screenshot of a google sheet with the default styling
High Contrast

Notice in the case with the high contrast applied to the google sheet, the grid lines are no longer visible, some of the backgrounds don't change colour, and some of the icons are hard to see (top left).

 A screen shot of a google sheet with a high contrast theme applied
A screen shot of a google sheet with a high contrast theme applied

Example: Line Space

Default Styling

 A screen shot of the CalJobs job search page
A screen shot of the CalJobs job search page
Line Space Doubled

Notice that Line Space setting expands the line height in the main paragraph at the top of the page. The height of the top nav bar also expands, which is actually causing some overlap, but in this cases, doesn't present any negative outcomes as the overlap doesn't reach any text. Also note, that the left navigation is not adapted at all. In some cases this may be intentional to prevent overlapping issues ( e.g. text expanding out of the container, containers expanding over each other, and etc. )

 Screen shot of the CalJobs job search page with the line height doubled.
Screen shot of the CalJobs job search page with the line height doubled.

UIO+ Tab

For some settings there has been mention of a UI Options+ Tab in the UX designs. I'm not entirely sure what this refers to. Does it mean the pull down tab and preferences editor provided by UI Options? If this is the case, I wasn't aware of it being part, at least not at this stage. It was my understanding that it would be the enactor portions of UI Options that wold be used in the UIO+. Supporting a preferences editor as well will involve added complexity for sending data back to GPII.

Questions / Answers

  • What is meant by UI Options+ Tab?
    • Answer:
      • Preference is for horizontal ( Similar to current UIO in Infusion ) --  with horizontal scrolling
      • Scroll bar is not good enough by itself though - not visible enough.  Need to have an arrow at the end to show there is more and how to get to them.
      • Best if it will push down content so it doesn’t cover any (but this is not critical  - the person can pop it up and down to see what is underneath)
      • It SHOULD be persistent (stay on screen) as you scroll down the page so you can scroll down to where you want and still make adjustments
  • For the settings modified via user controls from the UIO+ Tab how should the persistence work. Should they be forgotten after the browser is closed? What should happen when a user/snapset is keyed in/out? What if the browser is open when a user keys in/out and there are changes made in UIO+?

Managed Extensions

Some user settings are enacted via managed extensions (e.g Google Dictionary, click2speech). To make use of these extensions they need to be installed in the browser. Currently if at the managed extensions are not installed we prompt the user to install them, using a notification, when UIO+ is installed, when the browser is launched, and when any of the managed extensions are removed.

Questions / Answers

  • Is it too intrusive to prompt for installation at all of the above mentioned points?
    • Answer: The notifications on first install seems fine. If we can remember whether a person declined an installation, would prefer to not keep notifying them to install when the browser is launched or when the other extensions are removed/disabled.
  • Should we remember that a user didn't install and not ask for it to be installed any more?
    • Answer:
      • Yes - we should remember that they refused - but have some mechanism for them to reverse their decision.
        • Once they say no - it doesn't ask again - move it to the HIDDEN MENU
          • If they UNHIDE then it reasks for install
          • If they don't install - it REHIDES it
        • But if they REINSTALL UIO+ it RE ASKS
        • And if they install the item later -- UIO+ would recongnize it and show it
  • Should we only prompt to install if the related setting is set?
    • Answer:
      • Yes. If they don’t have any preference to use the feature, then there is no need to prompt them to install it.
      • If a person had not installed the other extension (or the extension was uninstalled or disabled)--warn the person that the functionality they are trying to enable needs a helper extension installed. “In order to use <feature>, you will also need to install <extension>. To download and install <extension>, click OK.”


Below are issues that have been identified with UIO+ extension implementation, but for which there are still questions about how to best address.

GPII-2560: Enabling "Simplify" hides google search engine text box

The current implementation works by making all of a page invisible and then making the main content visible. The main content is identified by searching for an "article" like element ( article tag, role article, article class, article id ). If that isn't found it looks for a "main" like element (main tag, role main, main class, main id). If that isn't found it looks for some generic content identifier (content class, content id, body class that isn't on the body tag, body id that isn't on the body tag). If none of those are found, no simplification is applied.

Google has content marked up with a "main" id; however, the search field and buttons are not located within this element. Which is why it is removed from the interface. In general we'd consider cases like this to be the result of "bad" markup. I do not know why Google has chosen to markup their page like this, and feel it may be a recent change. I vaguely remember testing things out when the simplification was first implemented.

Questions / Answers
  • Should we try to make Google work with simplification and if so how?
    • Never simplify on any Google page?
      • Answer: CORRECT
    • Never remove search fields on any site?
      • Answer: CORRECT
  • Should we expect that a user will be toggling simplify on/off as they use a site?
    • Answer: YES !
    • If this is the case, maybe the Google search field being removed is a lesser issue?
  • Can we rename Simplify to something like "Reader Mode"?
    • This seems more inline with the current state of just removing extraneous content.
      • Answer: YES --  lets call it READING MODE
    • Like the first question, can we "whitelist" Google and never apply simplification to it?