Cloud4all Integration Tests

From wiki.gpii
Jump to: navigation, search

Note: This page describes work done for the first pilot phase in Cloud4all (2013).

1. Introduction

Integration tests are our second tier of testing, and supplement the unit tests by testing the full Cloud4All/GPII autopersonalization stack. These tests should be run before the Cloud4all Pilot Scenario Tests.

In the near future the integration tests will be automated, but for the first rounds of pilots they will be done manually.

2. Windows 7

Section 2.1 describes the general procedure that has to be carried out with all user profiles. Next, section 2.2 will describes the specific procedure and the expected results for each user profile. Any deviation from the expected results should be reported in Cloud4all/GPII's issue tracker

2.1. General procedure

This section presents an overview of the testing procedure that will be done with each set of needs and preferences. For detailed, step-by-step information about each test with each user profile check secton 2.2. 

2.1.1. Pre-requisites

2.1.2. APfP using the Rule-based matchmaker

  • [FOR PROFILES TESTING NVDA] Take a snapshot of current NVDA setting
    • Open a command line and go to the folder that contains nvda_snapshot.cmd
    • Enter the command: nvda_snapshot.cmd > before_<USER_TOKEN>.txt
  • Start the Matchmaker
    • Open a new command line
    • Go to the windows folder under gpii (eg. cd c:\gpii_pilots\windows\)
    • Launch the Rule Based matchmaker: start_RB
  • Log in with the corresponding test user token
    • Go to the open browser, and in LogInLogOut.html, write down <USER_TOKEN> in the textbox
    • Click on the 'Log in' button
  • Check that the login worked as expected
    • In the iFrame below the heading 'Log In / Log Out Result http://localhost:8081/user/USER_TOKEN/login, the text should display: "User with token <USER_TOKEN> has successfully logged in"
    • In the iFrame below the heading 'Preference set http://preferences.gpii.net/user/<USER_TOKEN>', it should appear the JSON file displaying the preferences of user <USER_TOKEN>
    • Follow the instructions in section 'User has logged in' described in each profile.
  • Logout
    • from the integration_test_logins.html page in your browser click the 'Log out' button
  • Check that the logout worked as expected
    • n the iFrame below the heading 'Log In / Log Out Result http://localhost:8081/user/USER_TOKEN/login, the text should display: "User with token <USER_TOKEN> has successfully logged in"
    • Check that all configuration parameters are exactly the same as before logging in.
      • [FOR PROFILES TESTING NVDA]
        • Check whether the settings are reset by going to the folder with nvda_snapshot.cmd and running: nvda_snapshot.cmd > after_<USER_TOKEN>.txt
        • Compare the content of the "before_<USER_TOKEN>.txt" file to the content of after_<USER_TOKEN>.txt. If both files differ, it's a bug
  • Quit the rules based matchmaker
    • Close the window with the USB Listener
    • Close the window where GPII is running

2.1.3. APfP using the Statistical MatchMaker

Follow the same steps as in 2.1.2., except in step 'Start the matchmaker', where:

  • Start the Matchmaker
    • Open a new command line
      • Go to the windows folder under gpii (eg. cd c:\gpii_pilots\windows\)
      • Launch the Rule Based matchmaker: start_ST.cmd

2.2. TESTING SCRIPT

This section presents the instructions for performing the tests, as well as the expected results.

[1] screenreader_nvda.json

Before logging in

  • Make sure the pre-requisites (2.1.1) are met. 
  • Take a snapshot of the current NVDA settings:
    • Open a CMD and type, go to the folder where nvda_snapshot.cmd is and type: nvda_snapshot.cmd > before_screenreader_nvda.txt
  • Start GPII with the rule-based matchmaker or with the statistical matchmaker (but be sure to try both)
  • Log in
    • In LogInLogOut.html, go to the text are and type 'screenreader_nvda', then click the 'Login' button

User has logged in

  • Check that the user has successfully logged in in LogInLogOut.html
  • NVDA should start - if it doesn't do so, this is a bug
  • In the first command line you opened, type: 
    • If you are using the Rule Based MM, type 'nvda_snapshot.cmd > during_screenreader_nvda_RB.txt'. 
    • If you are using the Statistical MM, type 'nvda_snapshot.cmd > during_screenreader_nvda_ST.txt'
  • Both files should display the following configuration. If not, it is a bug.
reportHelpBalloons = False
speakTypedCharacters = False
speakTypedWords = True
sayCapForCapitals = False
autoSayAllOnPageLoad = False
rate = 17.20430107526882
rateBoost = true
symbolLevel = 300
voice = en\en-wi
  • Go to LogInLogOut.html, click on the 'Log out' button

After logging out

  • NVDA should stop - if it doesn't do so, this is a bug
  • Close GPII (USBUserListener and node)
  • Go to the first command line you opened and type:
    • If you used the RB MM, type: 'nvda_snapshot.cmd > after_screenreader_nvda_RB.txt'
    • If you used the ST MM, type: 'nvda_snapshot.cmd > after_screenreader_nvda_ST.txt'
  • Check that both files are EXACTLY the same as before_screenreader_nvda.txt. If not, report the bug to the issues tracker. 


[2] TECH9.json

Before logging in

  • Make sure the pre-requisites (2.1.1) are met. 
  • Take a snapshot of the current NVDA settings
    • Open a command line, go to the folder where nvda_snapshot.cmd is stored and type: nvda_snapshot.cmd > before_TECH9.txt
  • Start GPII with the rule-based matchmaker or with the statistical matchmaker (be sure to try both)
  • Log in
    • In LogInLogOut.html, go to the text area and type 'TECH9', then click the 'Log in' button

User has logged in

  • Check that the user has successfully logged in in LogInLogOut.html
  • NVDA should start up - if it doesn't do so, this is a bug
  • In the first command line you opened, type:
    • If you are using the Rule Based MM, type 'nvda_snapshot.cmd > during_TECH9_RB.txt'. 
    • If you are using the Statistical MM, type 'nvda_snapshot.cmd > during_TECH9_ST.txt'
  • Both files should display the following configuration. If not, it is a bug.
reportHelpBalloons = True
speakTypedCharacters = False
speakTypedWords = True
sayCapForCapitals = True
autoSayAllOnPageLoad = False
rate = 80
rateBoost = false
symbolLevel = 300
voice = en\en_US
  • Go to LogInLogOut.html and click the 'Log out button'

After logging out

  • NVDA should stop - if it doesn't, this is a bug
  • Close GPII (USBUserListener and node)
  • In the first command line you opened, type:
    • If you used the RB MM, type: 'nvda_snapshot > after_TECH9_RB.txt'
    • If you used the ST MM, type: 'nvda_snapshot > after_TECH9_ST.txt'
  • Check that both files are EXACTLY the same as 'before_TECH9.txt'

[3] os_win7.json

Before logging in

  • Make sure the pre-requisites (2.1.1) are met. 
  • Take a snapshot of the current win7 settings:
    • Open a CMD and type, go to the folder where win7_snapshot.cmd is and type: win7_snapshot.cmd > before_os_win7.txt
  • Start GPII with the rule-based matchmaker or with the statistical matchmaker (be sure to try both)
  • Log in
    • In LogInLogOut.html, go to the text area and type 'ow_win7', then click the 'Log in' button


User has logged in

  • In the CMD, to to the folder where win7_snapshot is stored and type:
    • If you are using the RB MM: win7_snapshot.cmd > during_os_win7_RB.txt
    • If you are using the ST MM: win7_snapshot.cmd > during_os_win7_ST.txt
  • Make sure the configuration presented in both files is exactly as follows:

[TBD]

  • High contrast should activate. If not, it is a bug.
  • Go to LogInLogOut.html and click on 'Log out'

After logging out

  • Magnification should deactivate. If not, it is a bug.
  • Check which theme is currently active. It should be the same as the one noted in step 1.
  • Check that the text size of the legends in the windows is the same as before logging in.  

[4] TECH11.json

Before logging in

  • Make sure the pre-requisites (2.1.1) are met. 
  • Take a snapshot of the current win7 settings:
    • Open a CMD and type, go to the folder where win7_snapshot.cmd is and type: win7_snapshot.cmd > before_TECH11.txt
  • Start GPII with the rule-based matchmaker or with the statistical matchmaker (be sure to try both)
  • Log in
    • In LogInLogOut.html, go to the text area and type 'ow_win7', then click the 'Log in' button


User has logged in

After logging out

3. Linux/Fedora 18

3.1. General procedure

3.1.1. Pre-requisites:

3.1.2. APfP using the Matchmakers

[FOR PROFILES TESTING LINUX SETTINGS] 'Take a snapshot of current Linux settings

    • Open a terminal and go to the folder where linux_snapshot is stored
    • execute: python linux_snapshot <USER_TOKEN> before_<USER_TOKEN>.txt
  • Start GPII and the Matchmaker
    • Press the System key, then write down 'GPII...' 
    • Click on 'GPII Rule Based' (for the first test) or 'GPII Statistical' (for the second test)
  • Log in 
    • Write down <USER_TOKEN> in the text area
    • Click on 'Log in'
  • Check that the login worked as expected
    • In LogInLogOut.html, in the iFrame below the heading 'Log In / Log Out Result http://localhost:8081/user/USER_TOKEN/login, the text should display: "User with token <USER_TOKEN> has successfully logged in"
    • In the iFrame below the heading 'Preference set http://preferences.gpii.net/user/<USER_TOKEN>', it should appear the JSON file displaying the preferences of user <USER_TOKEN>
    • Check that the success criteria defined in section 'User has logged in' are met for each profile.
  • Logout
    • from the LogInLogOut.html page in your browser click the 'Log out' button
  • Check that the logout worked as expected
    • Check that all configuration parameters are exactly the same as before logging in.
      • [FOR PROFILES TESTING LINUX SETTINGS]
        • Check whether the settings are reset by going to the folder with nvda_snapshot.cmd and running: python linux_snapshot <USER_TOKEN> > after_<USER_TOKEN>.txt
        • Compare the content of the "before_<USER_TOKEN>.txt" file to the content of after_<USER_TOKEN>.txt. If both files differ, it's a bug
  • Close GPII
    • Close the terminal where GPII is running.

TESTING SCRIPTS

TECH8

Before logging in

  • Make sure ORCA is closed.
  • Start GPII and the corresponding MatchMaker (GPII Rule Based, GPII statistical)
  • In LogInLogOut.html, type 'TECH8' in the textbox and click on 'Log in'

User has logged in

  • Check LogInLogOut.html to make sure that the upper iFrame says 'User with token TECH8 has successfully logged in' and that the lower iFrame displays the json file with the TECH8's needs and preferenes.
  • ORCA launches. If not, this is a bug.
  • Open a terminal and go to the folder where you have stored orca_snapshot. Then type:
    • If you are using the Rule Based MM: python orca_snapshot TECH8 > TECH8_RB_orca_snapshot.txt
    • If you are using the Statistical MM: python orca_snapshot TECH8 > TECH8_ST_orca_snapshot.txt
  • Check that TECH8_orca_snapshot.txt contains the following information. If not, it is a bug.
sayAllStyle:  1
enableEchoByWord:  True
enableEchoByCharacter: False
enableTutorialMessages:  True
enableBraille:  False
verbalizePunctuationStyle: 3
enableSpeech:  True
voices.default.family: es / spanish
voices.default.rate:  75
Check if the GNOME Orca Screen Reader is running: YES

  • On LogInLogOut.html, click on the 'Log out' button

After logging out

  • ORCA closes. If not, this is a bug.

TECH10

Before logging in

  • Start GPII and the corresponding MatchMaker (GPII Rule Based, GPII statistical)
  • In LogInLogOut.html, type 'TECH8' in the textbox and click on 'Log in'

User has logged in

  • Check LogInLogOut.html to make sure that the upper iFrame says 'User with token TECH10 has successfully logged in' and that the lower iFrame displays the json file with the TECH10's needs and preferenes.
  • The high contrast activates. If not, this is a bug.
  • The magnification activates. If not, this is a bug.
  • The icons in the menu are in high contrast. If not, this is a bug.
  • Open a terminal and go to the folder where you have stored linux_settings. Then, type:
    • If you are using the Rule Based MM: 'python linux_snapshot TECH10 > TECH10_RB_linux_settings.txt'
    • If you are using the Statistical MM: 'python linux_snapshot TECH10 > TECH10_ST_linux_settings.txt'
  • Check that TECH10_linux_settings.txt contains EXACTLY the following information:
## Magnifier settings
mag-factor: 2.0
screen-position: full-screen
show-cross-hairs: False
## Magnifier settings
gtk-theme: High contrast
icon-theme: High contrast
text-scaling-factor: 0.9
cursor-size: 24

After logging out

os_gnome.json

Before logging in

  • Start GPII and the corresponding MatchMaker (GPII Rule Based, GPII statistical)
  • In LogInLogOut.html, type 'TECH8' in the textbox and click on 'Log in'

User has logged in

  • Check LogInLogOut.html to make sure that the upper iFrame says 'User with token os_gnome has successfully logged in' and that the lower iFrame displays the json file with os_gnome's needs and preferenes.
  • The magnification activates. If not, this is a bug.
  • The size of the cursor size increases. If not, it is a bug.
  • Open a terminal and go to the folder where you have stored linux_settings. Then, type: 
    • If you are using the Rule Based MM: 'python linux_snapshot TECH10 > TECH10_RB_linux_settings.txt'
    • If you are using the Statistical MM: 'python linux_snapshot TECH10 > TECH10_ST_linux_settings.txt'
  • Check that TECH10_linux_settings.txt contains EXACTLY the following information:


## Magnifier settings
mag-factor: 1.5
screen-position: full-screen
show-cross-hairs: False
## Magnifier settings
gtk-theme: Adwaita
icon-theme: gnome
text-scaling-factor: 0.75
cursor-size: 90

After logging out

  • Make sure that all settings are back to normal.

screenreader_orca.json

Before logging in

  • Make sure ORCA is closed.
  • Start GPII and the corresponding MatchMaker (GPII Rule Based, GPII statistical)
  • In LogInLogOut.html, type 'screenreader_orca' in the textbox and click on 'Log in'

User has logged in

  • Check LogInLogOut.html to make sure that the upper iFrame says 'User with token screenreader_orca has successfully logged in' and that the lower iFrame displays the json file with screenreader_orca's needs and preferenes.
  • ORCA launches. If not, this is a bug.
  • Open a terminal and go to the folder where you have stored orca_snapshot. Then type:
    • If you are using the Rule Based MM: python orca_snapshot screenreader_orca > screenreader_orca_RB_orca_snapshot.txt
    • If you are using the Statistical MM: python orca_snapshot screenreader_orca > screenreader_orca_ST_orca_snapshot.txt
  • Check that TECH8_orca_snapshot.txt contains the following information. If not, it is a bug.
sayAllStyle:  1
enableEchoByWord:  True
enableEchoByCharacter:  False
enableTutorialMessages:  False
enableBraille:  False
verbalizePunctuationStyle: 0
enableSpeech:  True
voices.default.family:  en / en-westindies
voices.default.rate:  102.27272727272727
Check if the GNOME Orca Screen Reader is running: YES