QA Testing
Contents
- 1 Quality Assurance (QA) A Testing short
- 2 A: Basic System Tests
- 2.1 A1: Run all unit tests / integration tests
- 2.2 A2: Basic System functioning
- 2.3 A2.1: Multiple login-logout successions
- 2.4 A3: User launching several programs
- 2.5 A4: Relaunching applications NOT IMPLEMENTED YET
- 2.6 A5: Launch in production and development mode (ie. use remote prefs server)
- 2.7 A6: starting with different configs works from start scripts
- 3 B: Test each settings handler (done for each settings handler)
- 4 C: Login:
- 5 D: Installation
- 6 E: Servers: (schemas via VASILLIS?)
Quality Assurance (QA) A Testing short
This page describes the QA testing process of the core components of the architecture.
The QA Testing will take place in one of the last phases of the Release Process of the core architecture. The purpose is to test that the framework works, probe it for bugs and ensure that the code is ready for release. The tests to be carried out during this phase is described below.
(**) denotes we should consider writing unit tests for them
A: Basic System Tests
A1: Run all unit tests / integration tests
Steps:
- Run all the unit tests and integration tests implemented
Expected:
- All tests passes, unless there is a non-blocking JIRA for it
A2: Basic System functioning
Steps:
- Make sure you have an NP set
- Start the system
- log in as the user using any means of login (eg. http://localhost:8081/user/USERNAME/login)
Expected:
- The system logs the user in, the system is configured accordingly and the expected applications start
Steps:
- Log out
Expected:
- The system logs the user out, the system is reset to it's previous state accordingly and any launched applications killed
A2.1: Multiple login-logout successions
Steps:
- Do two repeats of A2 with the same user
- Do two repeats of A2 with a different user the second time
A3: User launching several programs
Steps:
- You need an NP set that you know will modify/start several applications on the platform
- Start the system
- log in as that user
Expected:
- The system logs the user in
- All the required applications are launched and configured
Steps:
- Log out using the user Listener
Expected:
- The system logs the user out, the system is reset to it's previous state accordingly and any launched applications killed
A4: Relaunching applications NOT IMPLEMENTED YET
A5: Launch in production and development mode (ie. use remote prefs server)
Production (deployment for which all components use external (CouchDB, etc.) persistence layer)
Steps:
- Navigate into a linux/windows repository
- Run build.sh/build.cmd command to setup all dependencies.
- Start gpii server with node environment variable set to "fm.ps.sr.dr.mm.os.production":
NODE_ENV=fm.ps.sr.dr.mm.os.production ./start.sh(or ./start.cmd)
Expected:
- All dependencies should be configured
- The server is running (log is disabled)
- You can now login/logout to GPII as any user present in the persistence layer.
Development (deployment for which all components use local file system as persistence layer)
Steps:
- Navigate into a linux/windows repository
- Run build.sh/build.cmd command to setup all dependencies.
- Start gpii server with node environment variable set to "fm.ps.sr.dr.mm.os.development":
NODE_ENV=fm.ps.sr.dr.mm.os.development ./start.sh(or ./start.cmd)
Expected:
- All dependencies should be configured
- The server is running (log is enabled)
- You can now login/logout to GPII as any user present in the persistence layer.
A6: starting with different configs works from start scripts
To start an application with any other configuration you need to:
Steps:
- Navigate into a linux/windows repository
- Make sure that the config file with configuration you are going to be using is present in ../node_modules/universal/gpii/configs directory.
- Start gpii server with node environment variable set to the name of the json configuration file:
NODE_ENV={your configuration file name} ./start.sh(or ./start.cmd)
Expected:
- The server is running
- You can now login/logout to GPII as any user present in the persistence layer.
B: Test each settings handler (done for each settings handler)
The below tests should be done for all settings handlers:
- XML Settings Handler
- JSON Settings Handler
- Windows Registry Settings Handler
- SPI Settings Handler
- GSettings Settings Handler
B1: add non-existent setting (**)
Steps:
- Make sure the current settings file/location does not contain some setting (called Setting A from here on)
- Create an NP set that sets that setting (eg. in a settings file, this would be setting a value that is not present, in a registry type handler it would be writing to a non-existing location)
- Start up the system and log in as the user
- Check the Setting A
Expected:
- If the system supports it, Setting A should be set. Otherwise an appropriate error message should be given.
- The rest of the system should configure as usual
Steps:
- Log out
Expected:
- Setting A should no longer be present
- Rest of logout process should be as usual.
B2: remove/reset setting that previously existed (ie. reset to default/non-specified) - NOT SUPPORTED IN 0.1
Steps:
- Make sure the current settings file contain/location some setting (called Setting A from here on)
- Create an NP set that unsets that setting - or resets to default (depending on settings handler how this is done / whether possible)
- Start up the system and log in as the user
- Check the Setting A
Expected:
- Depending on the settings handler, this setting should be reset to default/unspecified, depending on the handler
- The system should be configured accordingly
B3: read from/write to non-existing file (**)
Steps:
- Make sure the current settings file does not exist
- Create an NP set that has preferences referring to one or more settings writting by that handler
- Start up the system and log in as the user
Expected:
- The file should be written with the users settings
Steps:
- Log out
Expected:
- The settings file should no longer be present
B4: Writing unsupported types (**)
This is relevant for registry-type settings handlers
Steps:
- Decide on settings which value range/types you know
- Create an NP set that has preferences referring to one or more settings writting by that, which values are outside the handlers range for that setting
- Start up the system and log in as the user
Expected:
- An error message should be given, informing the user that the action failed (due to value being out of range is supported by settings handler)
- Otherwise the configuration should continue as normal
B5: Test that settings are changed and restored
Steps/expectations:
- Note the values of the preferences that some User A will modify on login
- Login as User A
- Check that User A's preferences are set as expected and programs are launched as expected
- Logout User A
- Check that the settings are restored to their original state
C: Login:
C1: RFID, USB, HTTP - Windows
This should be repeated for each of the user listeners
Steps:
- Prepare the device/method to hold the token and ensure the related NP set exist
- Start the system
- log in as the user using the Listener.
Expected:
- The system logs the user in, the system is configured accordingly and the expected applications start
Steps:
- Log out using the user Listener
Expected:
- The system logs the user out, the system is reset to it's previous state accordingly and any launched applications killed
C2: RFID, USB, HTTP - Linux
This should be repeated for each of the user listeners
Steps:
- Prepare the device/method to hold the token and ensure the related NP set exist
- Start the system
- log in as the user using the Listener.
Expected:
- The system logs the user in
Steps:
- Log out using the user Listener
Expected:
- The system logs the user out
C3: login several times in a row with the same user
Steps:
- Login with some user A
- System should be configured accordingly
- Login as user A
Expected:
- System should remain as before (logged in state)
- Message should indicate the user is already logged in
Steps:
- Logout user A
Expected:
- System should reset to the state it was in before User A logged in
C4: logout several times in a row
Steps:
- Login with some user A
- System should be configured accordingly
- Logout user A
- System should reset
- Logout user A
- Nothing should happen (perhaps an error message indicating no user A is logged in
C5: Login with different user
Steps:
- Login with some user A
- System should be configured accordingly
- Then login again as user B
Expected:
- System should configure to user B's preferences
Steps:
- Logout user B
Expected:
- System should reset to the state it was in before User A logged in
C6: Login with non-existing user
Steps:
- Login with some user A that does not exist on the preferences server
Expected:
- System should report that User A doesn't exist
D: Installation
D1: Manual Installation
For 0.1, this involves splitting the instructions up into Linux/Windows specific
Steps:
- Make sure a "GPII Installation Instructions - Linux, version 0.X" page exist on the wiki (with Windows if thats the OS you're testing, and version 0.X replaced with the relevant version)
- Update the instructions to include any new requirements
- Set up a clean install of windows or Linux
- Go through the installation process following the instructions on the page
Expected:
- Installation works.
- It is possible to login and out of the system
- System configures according to user settings
D2: Windows automatic installation (32/64 bit) (GEORGE?):
Steps:
- Download/get the single installer file
- Start the installation process
Expected:
- Installation goes as expected
- If not handled by installer, start GPII
- It is possible to login and out of the system
- System configures according to user settings
- The expected User Listener works (RFID, USB, URL)
D3: Linux automatic installations (fedora/ubuntu)
Steps:
- Download/get the single installer file
- Start the installation process
Expected:
- Installation goes as expected
- If not handled by installer, start GPII
- It is possible to login and out of the system
- System configures according to user settings
- The expected User Listener works (RFID, USB, URL)