Pilot Phase 2 Snapshotter Testing

From wiki.gpii
Jump to: navigation, search

This page describes a number of test processes related to snapshotting and the outcomes of the statistical matchmaker. Tests A and B represent Ignacio's two tests from 21 May. Tests C, D and E represent Ignacio's tests from 26 May. Each section has a table where testers can add their findings when trying to reproduce issues.

See also Pilot phase 2 problems for a list of other issues. See Cloud4all Second Pilot Phase for all other links related to this evaluation phase.

Related JIRA tickets:

Test A

Summary: From Windows machine to Windows machine, setting NVDA and using snapshotting and the statistical MM.

Process:

  1. Start GPII using gpii_ctrl.bat and start Firefox (with the PMT etc)
  2. Fill in PMT and fill in a few screen reader settings (e.g. "Speak text", Spanish TTS, speak most punctuation, magnifier docked, screen reader follows mouse) and click on "Save and apply"
  3. Copy the token to a USB stick.
  4. In the pilotsConfig tool, select "Snapshot".
  5. In the pilotsConfig tool, set the matchmaker strategy to "Statistical Matchmaker". (After this, you can also grab the content of the logging window.)
  6. In the pilotsConfig tool, select "Logging".
  7. Log out using the PMT. (This step was not in Ignacio's original description.)
  8. Go to another Windows machine. (Rebooting the same Windows machine should also work.)
  9. Start GPII using gpii_ctrl.bat and start Firefox (with the pilotsConfig tool etc.)
  10. Open pilotsConfig and select 'Windows (NVDA and Windows magnifier) or Linux', and click on 'Set device configuration'. Check the feedback message.
  11. Insert the USB with the token.
  12. Check whether the USB listener shows an error or says that the token was logged in successfully.
    • Check especially the error {"isError":true, "message":"arrayToMembership didn't find array at inputPath nor passed as value.[object Object][object Object]"}.
  13. If the above worked, continue: In the pilotsConfig tool, select "Logging".
  14. In Firefox, open the PMT and log out. Check whether everything is set back to normal again.
Test A: Results
Tester and date Platform A and B (32/64 bits, language) Outcome(s)
Ignacio, 21 May 2014 A: Windows 7. B: Windows 7. The USB listener displays an error ({"isError":true, "message":"arrayToMembership ...) when inserting the USB token. (See mail from 21 May.)

Token: 318887ca-bbb3-404e-9cdb-0a7277ac0540 (login/logout)

Christophe, 21 May 2014 A & B: Windows 7, 64 bits, English Could not reproduce the issue (see message from 21 May).

Token: 65640120-08a2-435c-92ca-14376c17df0e (login/logout).

... ... ...
... ... ...

Test B

Summary: From Windows machine to Windows machine, setting magnifier and using snapshotting and the statistical MM.

Process:

  1. Start GPII using gpii_ctrl.bat and start Firefox (with the PMT etc)
  2. Open the pilotsConfig tool.
  3. In the pilotsConfig tool, select "Windows (NVDA and Windows Magnifier) or Linux". Check whether the feedback is "Device information successfully set!"
  4. Open the PMT and press "Save and Apply"
    • (Ignacio's original test process does not say that any settings need to be adapted, so at this point I expected screen reader TTS will remain turned off).
  5. In the pilotsConfig tool, set the matchmakers strategy to "statistical matchmaker".
  6. In the pilotsConfig tool, trigger a snapshot and check the feedback.
  7. In the pilotsConfig tool, trigger a local log.
  8. Log out in the PMT; check whether this works and whether the system is set back to its original state.
  9. Go to another Windows 7 PC (or reboot the same PC).
  10. Start GPII.
  11. Start Firefox and open the pilotsConfig tool.
  12. In the pilotsConfig tool, select "Windows (NVDA and Windows Magnifier) or Linux". Check whether the feedback is something like "Device information successfully set!"
  13. Insert the USB token. Check the message in the USB listener windows (an error or a success message).
    • Check what solutions are launched by GPII.
  14. In the pilotsConfig tool, trigger a local log.
  15. Open the PMT. Check whether it notices that a user is keyed in.
  16. Log out using the PMT. Check whether NVDA is turned off again.
Test B: Results
Tester and date Platform A and B (32/64 bits, language) Outcome(s)
Ignacio, 21 May 2014 A: Windows 7. B: Windows 7. (See mail from 21 May).

The USB listener displays an error message when keying in: {"isError":true, "message":"arrayToMembership didn't find array at inputPath nor passed as value.[object Object][object Object].

Christophe, 21 May 2014 ... (See message from 21 May.)

Gets the same error message as Ignacio using Ignacio's token: 7353e9ed-6484-4bea-8cbf-f5f8fa5e34e9. (login/logout)

Christophe, 23 May 2014 A & B: Windows 7, 64 bits (German?). Token: 63818ae0-c681-46ba-ac62-bdd11bb1c811. (login/logout)

After keying in, the UsbUserListener window said: "User with token 63818ae0-c681-46ba-ac62-bdd11bb1c811 was successfully logged in." The GPII started NVDA in German, even though no language settings had been set in the PMT. (This may be a leftover from a previous test, where we set NVDA to German after a crash had caused a reset to the factory settings.)

... ... ...
... ... ...

Test C

(Test 1 from Ignacio's mail, 26 May 2014.)

Summary: From Windows machine to Fedora 20 machine, setting screen reader settings and using snapshotting and the statistical MM.

Process:

  1. Start GPII using gpii_ctrl.bat and start Firefox (with the PMT etc).
  2. Open the pilotsConfig tool.
  3. In the pilotsConfig tool, select "Windows (NVDA and Windows Magnifier) or Linux" as device configuration. Check whether the feedback is "Device information successfully set!"
  4. Open the PMT and select the following settings:
    • ScreenReaderTTSEnabled: true
    • Punctuation: Some
    • Say "Cap" before "Capitals"
  5. Press "Save and Apply" and check whether the chosen settings get applied.
  6. Copy the generated token to a USB stick.
    • At this point, you can also download the corresponding preference set for later comparison.
  7. In the pilotsConfig tool, select the "Statistical Matchmaker" and check whether the feedback is something like "matchmaker successfully selected".
  8. In the pilotsConfig tool, select "Snapshot" and check whether the feedback is something like "Snapshot saved to preferences server".
    • At this point, you can also download the corresponding preference set to compare it with the version before snapshotting.
  9. Sign out from the PMT.
  10. Move to a Fedora 20 machine and start GPII and the GPII USB listener.
  11. Insert the USB stick with the token.
    • Check whether the USB listener window displays an error message.
    • You can optionally log the current settings for comparison with the preference sets.
  12. Sign out from GPII on Linux (i.e. tell Fedora to eject the USB key or log out from the PMT).
  13. Go to a Windows 7 machine and start GPII using gpii_ctrl.bat.
  14. Start Firefox and open the pilotsConfig tool. In the pilotsConfig tool, select the device configuration "Windows (NVDA and Windows Magnifier) or Linux".
    • Check the USB listener window for error messages.
    • You can optionally log the current settings (in the pilotsConfig tool or using the .bat file).


Test C: Results
Tester and date Platform A and B (32/64 bits, language) Outcome(s)
Ignacio, 26 May 2014 A: Windows 7, 32 bit. B: Fedora 20. C: Windows 7, 32 bit. (see mail from 26 May)

Token 0cc28f04-5a59-4dc9-9222-15eb6824bd71. (login/logout)

... ... ...
... ... ...

Test D

Summary: From Windows machine to Fedora 20 machine, setting magnifier settings and using snapshotting and the statistical MM.

Process:

  1. Start GPII using gpii_ctrl.bat and start Firefox (with the PMT etc).
  2. Open the pilotsConfig tool.
  3. In the pilotsConfig tool, select "Windows (NVDA and Windows Magnifier) or Linux" as device configuration. Check whether the feedback is "Device information successfully set!"
  4. Open the PMT and select the following settings:
    • Magnifier: ON
    • Magnification: 150%
    • Magnification to full screen
  5. Press "Save and Apply" and check whether the chosen settings get applied.
  6. Copy the generated token to a USB stick.
    • At this point, you can also download the corresponding preference set for later comparison.
  7. In the pilotsConfig tool, select the "Statistical Matchmaker" and check whether the feedback is something like "matchmaker successfully selected".
  8. In the pilotsConfig tool, select "Snapshot" and check whether the feedback is something like "Snapshot saved to preferences server".
    • At this point, you can also download the corresponding preference set to compare it with the version before snapshotting.
  9. Sign out from the PMT.
  10. Move to a Fedora 20 machine and start GPII and the GPII USB listener.
  11. Insert the USB stick with the token.
    • Check whether the USB listener window displays an error message.
    • You can optionally log the current settings for comparison with the preference sets.
  12. @@continue


Test D: Results
Tester and date Platform A and B (32/64 bits, language) Outcome(s)
Ignacio, 21 May 2014 A: Windows 7, 32 bit. B: Fedora 20. C: Windows 7. (see mail from 26 May)
... ... ...
... ... ...

Test E

Summary: From Windows machine to Fedora 20 machine, setting cursor size and using snapshotting and the statistical MM.

Process:

Test E: Results
Tester and date Platform A and B (32/64 bits, language) Outcome(s)
Ignacio, 21 May 2014 A: Windows 7, 32 bit. B: Fedora 20. (see mail from 26 May)

Token: ab3eb88b-7f3d-496a-a83d-7d50b53556a3 (login/logout)

... ... ...
... ... ...

Test F

Test F: Results
Tester and date Platform A and B (32/64 bits, language) Outcome(s)
xx, ... 2014 A: ... . B: ...
... ... ...
... ... ...


Wiki Categories