QA Testing

Quality Assurance (QA) A Testing short

This page describes the QA testing process of the core components of the architecture.

The QA Testing will take place in one of the last phases of the Release Process of the core architecture. The purpose is to test that the framework works, probe it for bugs and ensure that the code is ready for release. The tests to be carried out during this phase is described below.

(**) denotes we should consider writing unit tests for them

A: Basic System Tests

A1: Run all unit tests / integration tests

Steps:

1. Run all the unit tests and integration tests implemented

Expected:

• All tests passes, unless there is a non-blocking JIRA for it

A2: Basic System functioning

Steps:

1. Make sure you have an NP set
2. Start the system
3. log in as the user using any means of login (eg. http://localhost:8081/user/USERNAME/login)

Expected:

• The system logs the user in, the system is configured accordingly and the expected applications start

Steps:

1. Log out

Expected:

• The system logs the user out, the system is reset to it's previous state accordingly and any launched applications killed

A2.1: Multiple login-logout successions

Steps:

1. Do two repeats of A2 with the same user
2. Do two repeats of A2 with a different user the second time

A3: User launching several programs

Steps:

1. You need an NP set that you know will modify/start several applications on the platform
2. Start the system
3. log in as that user

Expected:

• The system logs the user in
• All the required applications are launched and configured

Steps:

1. Log out using the user Listener

Expected:

• The system logs the user out, the system is reset to it's previous state accordingly and any launched applications killed

A5: Launch in production and development mode (ie. use remote prefs server)

Production (deployment for which all components use external (CouchDB, etc.) persistence layer)

Steps:

1. Navigate into a linux/windows repository
2. Run build.sh/build.cmd command to setup all dependencies.
3. Start gpii server with node environment variable set to "fm.ps.sr.dr.mm.os.production":

NODE_ENV=fm.ps.sr.dr.mm.os.production ./start.sh(or ./start.cmd)

Expected:

• All dependencies should be configured
• The server is running (log is disabled)
• You can now login/logout to GPII as any user present in the persistence layer.

Development (deployment for which all components use local file system as persistence layer)

Steps:

1. Navigate into a linux/windows repository
2. Run build.sh/build.cmd command to setup all dependencies.
3. Start gpii server with node environment variable set to "fm.ps.sr.dr.mm.os.development":

NODE_ENV=fm.ps.sr.dr.mm.os.development ./start.sh(or ./start.cmd)

Expected:

• All dependencies should be configured
• The server is running (log is enabled)
• You can now login/logout to GPII as any user present in the persistence layer.

A6: starting with different configs works from start scripts

To start an application with any other configuration you need to:

Steps:

1. Navigate into a linux/windows repository
2. Make sure that the config file with configuration you are going to be using is present in ../node_modules/universal/gpii/configs directory.
3. Start gpii server with node environment variable set to the name of the json configuration file:

NODE_ENV={your configuration file name} ./start.sh(or ./start.cmd)

Expected:

• The server is running
• You can now login/logout to GPII as any user present in the persistence layer.

B: Test each settings handler (done for each settings handler)

The below tests should be done for all settings handlers:

• XML Settings Handler
• JSON Settings Handler
• Windows Registry Settings Handler
• SPI Settings Handler
• GSettings Settings Handler

B1: add non-existent setting (**)

Steps:

1. Make sure the current settings file/location does not contain some setting (called Setting A from here on)
2. Create an NP set that sets that setting (eg. in a settings file, this would be setting a value that is not present, in a registry type handler it would be writing to a non-existing location)
3. Start up the system and log in as the user
4. Check the Setting A

Expected:

• If the system supports it, Setting A should be set. Otherwise an appropriate error message should be given.
• The rest of the system should configure as usual

Steps:

1. Log out

Expected:

• Setting A should no longer be present
• Rest of logout process should be as usual.

B2: remove/reset setting that previously existed (ie. reset to default/non-specified) - NOT SUPPORTED IN 0.1

Steps:

1. Make sure the current settings file contain/location some setting (called Setting A from here on)
2. Create an NP set that unsets that setting - or resets to default (depending on settings handler how this is done / whether possible)
3. Start up the system and log in as the user
4. Check the Setting A

Expected:

• Depending on the settings handler, this setting should be reset to default/unspecified, depending on the handler
• The system should be configured accordingly

B3: read from/write to non-existing file (**)

Steps:

1. Make sure the current settings file does not exist
2. Create an NP set that has preferences referring to one or more settings writting by that handler
3. Start up the system and log in as the user

Expected:

• The file should be written with the users settings

Steps:

1. Log out

Expected:

• The settings file should no longer be present

B4: Writing unsupported types (**)

This is relevant for registry-type settings handlers

Steps:

1. Decide on settings which value range/types you know
2. Create an NP set that has preferences referring to one or more settings writting by that, which values are outside the handlers range for that setting
3. Start up the system and log in as the user

Expected:

• An error message should be given, informing the user that the action failed (due to value being out of range is supported by settings handler)
• Otherwise the configuration should continue as normal

B5: Test that settings are changed and restored

Steps/expectations:

1. Note the values of the preferences that some User A will modify on login
2. Login as User A
3. Check that User A's preferences are set as expected and programs are launched as expected
4. Logout User A
5. Check that the settings are restored to their original state

C: Login:

C1: RFID, USB, HTTP - Windows

This should be repeated for each of the user listeners

Steps:

1. Prepare the device/method to hold the token and ensure the related NP set exist
2. Start the system
3. log in as the user using the Listener.

Expected:

• The system logs the user in, the system is configured accordingly and the expected applications start

Steps:

1. Log out using the user Listener

Expected:

• The system logs the user out, the system is reset to it's previous state accordingly and any launched applications killed

C2: RFID, USB, HTTP - Linux

This should be repeated for each of the user listeners

Steps:

1. Prepare the device/method to hold the token and ensure the related NP set exist
2. Start the system
3. log in as the user using the Listener.

Expected:

• The system logs the user in

Steps:

1. Log out using the user Listener

Expected:

• The system logs the user out

C3: login several times in a row with the same user

Steps:

1. Login with some user A
2. System should be configured accordingly
3. Login as user A

Expected:

1. System should remain as before (logged in state)
2. Message should indicate the user is already logged in

Steps:

1. Logout user A

Expected:

1. System should reset to the state it was in before User A logged in

C4: logout several times in a row

Steps:

1. Login with some user A
2. System should be configured accordingly
3. Logout user A
4. System should reset
5. Logout user A
6. Nothing should happen (perhaps an error message indicating no user A is logged in

C5: Login with different user

Steps:

1. Login with some user A
2. System should be configured accordingly
3. Then login again as user B

Expected:

1. System should configure to user B's preferences

Steps:

1. Logout user B

Expected:

1. System should reset to the state it was in before User A logged in

C6: Login with non-existing user

Steps:

1. Login with some user A that does not exist on the preferences server

Expected:

1. System should report that User A doesn't exist

D: Installation

D1: Manual Installation

For 0.1, this involves splitting the instructions up into Linux/Windows specific

Steps:

1. Make sure a "GPII Installation Instructions - Linux, version 0.X" page exist on the wiki (with Windows if thats the OS you're testing, and version 0.X replaced with the relevant version)
2. Update the instructions to include any new requirements
3. Set up a clean install of windows or Linux
4. Go through the installation process following the instructions on the page

Expected:

1. Installation works.
2. It is possible to login and out of the system
3. System configures according to user settings

D2: Windows automatic installation (32/64 bit) (GEORGE?):

Steps:

1. Download/get the single installer file
2. Start the installation process

Expected:

1. Installation goes as expected
2. If not handled by installer, start GPII
3. It is possible to login and out of the system
4. System configures according to user settings
5. The expected User Listener works (RFID, USB, URL)

D3: Linux automatic installations (fedora/ubuntu)

Steps:

1. Download/get the single installer file
2. Start the installation process

Expected:

1. Installation goes as expected
2. If not handled by installer, start GPII
3. It is possible to login and out of the system
4. System configures according to user settings
5. The expected User Listener works (RFID, USB, URL)