The testing is required on the installation software of an operating system. The target hardware does not contain any existing software or an existing operating system. The installation is aimed for raw hardware.
The operating system can be installed by using the installation wizard or by following the instructions detailed in the installation manual.
The overall testing approach taken in this scenario is a two-stage approach. Firstly an installation test stage is performed. This is followed by a functionality test stage. For the purposes of this standard only the first stage is documented. The overall installation testing objective is to measure the installation against defined acceptance criteria and requirements, including to measure the successful installation of the operating system. This includes all aspects of system and software configuration.
Valid and invalid hardware platforms are required, with no software or existing operating systems present. It must be possible to refresh the environment to its initial state when required. The capability for performing a post-installation audit check must be available. All necessary and required devices and storage mediums must be present in the valid hardware platform.
AC101 - The operating system can be successfully installed by following the instructions in the installation manual
AC102 - The operating system can be successfully installed using the installation wizard
AC103 - The installation wizard contains clear and understandable processes and information instructions according to specified requirement
AC104 - All system configuration settings and file additions will be present according to the specified requirement
AC105 - The user can select pre-determined applications and use them to perform basic tasks successfully
AC106 - The installation wizard can successfully identify an invalid hardware platform and notify the user accordingly
AC107 - When the system boots up it will recognise the device required by the operating storage medium system
The installation tester will be an experienced technician with skills similar to the target Installation Engineer.
Measure the validity of the installation manual against the actual installation procedure.
The scope of the test case is the execution of the installation scripts and following the installation procedures from the manual.
AC101 - The operating system can be successfully installed by following the instructions in the installation manual
The installation manual must be present. An appropriate hardware platform containing no existing operating software must be available. All necessary and required access devices and storage mediums must also be available. Software and hardware must be ready and available for installation.
1. Boot the machine and establish what devices are detected from its boot profile. The information will be displayed on the screen as the boot profile is executed.
2. Depending on the result of recognising drive devices and on what medium the operating system is stored on, CD, disk or tape, create a boot file that will aid the boot profile.
3. The boot file will have to identify the necessary device on the hardware. Place the necessary storage medium (containing the operating system software), disk or tape, in the device and proceed with accessing the drive by using the basic DOS or boot language.
4. Identify the installation scripts on the drive and execute the necessary set-up scripts to start the installation of the operating system according to the installation manual.
5. If hard drive disk partitioning is necessary, plan accordingly on sizing and installation of necessary file systems. This will be specified in the installation manual.
6. Perform system information checks to determine all areas of the operating system have been installed. This will be specified in the installation manual.
7. Check configuration settings according to the installation manual with regard to security, permission, caching memory, etc.
8. Set-up procedures for preparing recovery disk when prompted by the installation procedure.
The installation procedures required during the actual installation correspond to those in the manual. (AC101)
Measure the usability of the installation wizard.
The scope of the test is the execution of the installation scripts and following the installation wizard.
AC102 - The operating system can be successfully installed using the installation wizard
AC103 - The installation wizard contains clear and understandable processes and information instructions according to specified requirement
The installation wizard must be available. An appropriate hardware platform containing no existing operating software must be available. All necessary and required access devices and storage mediums must also be available. Software and hardware must be ready and available for installation.
1. Boot the machine and establish what devices are detected from its boot profile. The information will be displayed on the screen as the boot profile is executed.
2. Depending on the result of recognising drive devices and on what medium the operating system is stored on, CD, disk or tape, create a boot file that will aid the boot profile.
3. The boot file will have to identify the necessary device on the hardware. Place the necessary storage medium (containing the operating system software), disk or tape, in the device and proceed with accessing the drive by using the basic DOS or boot language.
4. Identify the installation wizard scripts on the drive and execute the necessary set-up scripts to start the installation of the operating system, using the installation wizard.
5. If hard drive disk partitioning is necessary, plan accordingly on sizing and installation of necessary file systems. The user will be guided by the installation wizard to achieve this.
6. Perform system information checks to determine all areas of the operating system have been installed. The user will be guided by the installation wizard to achieve this.
7. Check configuration settings with regard to security, permission, caching memory, etc. The user will be guided by the installation wizard to achieve this
8. Set-up procedures for preparing recovery disk when prompted by the installation wizard.
The installation procedures required during the actual installation correspond to those achieved using the installation wizard. (AC102)
The installation wizard must contain clear and understandable processes and information instructions according to specified requirement. (AC103)
Note: Usability is measured according to specified requirements. This standard does not aim to duplicate usability testing, which is covered in a separate section of this standard.
Measure the system configuration according to the operating system set-up requirements
The scope of this test case is the system configuration itself.
AC104 - All system configuration settings and file additions will be present according to the specified requirement
The installation must have completed successfully according to the installation wizard.
1. Identify all necessary configuration files and settings according to the specification
2. Locate all files and settings and verify the values accordingly
All system configuration settings and file additions must be present and correct according to the specified requirement. (AC104)
Measure the successful installation and execution of applicable application software.
The scope of this test case is to install additional software compliant with the operating system and execute the selected software to perform basic tasks, though not going into functionality testing.
AC105 - The user can select pre-determined applications and use them to perform basic tasks successfully
The installation wizard must have completed successfully
Install additional application software compliant to the operating system and execute that software to determine if there are any failures on the operating systems' caching or memory functionality (Not going to far, that would be functionality testing).
The selected application must execute successfully. (AC105)
Perform negative testing to measure the error notification functionality of the installation wizard.
The scope of this test case is to:
· Identify installation scripts
· Execute scripts
· Monitor and note any information displayed on screen
· Validate error messages and information according to necessary specifications
AC106 - The installation wizard can successfully identify an invalid hardware platform and notify the user accordingly
An invalid hardware platform must be prepared for installation
1. Boot the machine and establish what devices are detected from its boot profile. The information will be displayed on the screen as the boot profile is executed.
2. Depending on the result of recognising drive devices and on what medium the operating system is stored on, CD, disk or tape, create a boot file that will aid the boot profile.
3. The boot file will have to identify the necessary device on the hardware. Place the necessary storage medium (containing the operating system software), disk or tape, in the device and proceed with accessing the drive by using the basic DOS or boot language.
4. Identify the installation scripts on the drive and attempt to execute the necessary set-up scripts to start the installation of the operating system using the installation wizard.
The installation wizard must identify the error and notify the users with the necessary information. (AC106)
Measure the device recognition required for storage medium access
The scope of the test case is device recognition.
AC107 - Device recognition is performed automatically by the operating system
All necessary devices and storage mediums must be present in hardware
1. Boot the machine
2. Identify from the boot information on the screen what devices are recognised
3. If required, prepare a boot file that points towards the required device for the operating system storage medium
4. Place the storage medium in the device
5. Access the device
6. Locate and execute the installation scripts
When the system boots up, it will recognise the device required by the operating system storage medium. (AC107)
Valid and invalid hardware is to be used during testing. The target hardware will not contain any existing software or an existing operating system. It must be switched on with all necessary cables and be ready to function up to its general boot capability.
Note: It is not necessary to carry out a pre-installation audit check in this test scenario as new hardware is involved and no software or existing operating system will be present.
Performed by the Installation tester.
The above test cases will be executed and the actual results matched against the expected results.
A post-installation audit check will be conducted against a predetermined list to ensure that all software and files present are as expected.
The completion criteria of the installation testing in this scenario will be measured in terms of the following:
· Were the acceptance criteria defined earlier measured satisfactorily?
· Were the results of the post-installation audit check satisfactorily?
After installation was the operating system:
· Resident and running?
· Resident and not running?
· Not resident and run-able?
· Not resident and not run-able until installed?
The installation was for an upgrade to an existing personnel system within a large UK organisation.
The software development of the personnel system had recently been out-sourced to a third party supplier. The unit and systems testing had also been out-sourced to the same third party supplier.
Prior to the upgrade, individual stand alone personnel systems existed at each of the 120 sites across the country and at the organisations headquarters in London.
The upgrade involved the conversion of the existing databases to provide site identification details. It also involved the creation of a new corporate database that would contain all personnel details across the organisation. This corporate database would be maintained at the organisation headquarters in London and would be updated via an overnight batch run.
The conversion of the personnel databases to their new format would be conducted out as part of the installation process itself with instructions for this process detailed in the installation manual.
There were three different types of site:
- Headquarters site
- Non-headquarters - 120 of these sites
- A new corporate database site - to be created
The personnel system consisted of a number of ‘bolt-on modules’, which may or may not be loaded at a particular site dependant on the type of site. For example, both the corporate database and headquarters site produced a variety of different reports and statistics for senior management, which were not required at the non-headquarters sites.
As the personnel system contained sensitive personal information such as pay roll and disciplinary offences details, different levels of security existed at different sites.
The confidence test stage was to take place on delivery of the new personnel system. This test stage would be performed prior to the start of the user acceptance testing. Once the personnel system had been seen to pass the confidence test stage, it would then be accepted from the third party suppliers for the purposes of conducting the user acceptance test.
The installation test would be conducted as the first part of the confidence test stage.
The overall installation testing objective is to measure the successful installation of the new personnel system on the three different types of site. To complete the cycle an additional test will be conducted to ensure that the test corporate database can be populated with personnel details from the other two test sites via the overnight batch run.
Three test environments would be required, one for the test headquarters site, one for a test non-headquarters site and one for the test corporate database site.
Mostly live data would be used during the confidence test. To ensure the confidentially of the sensitive personnel details, this data would be scrambled making the individuals details unrecognisable.
The capability for performing both pre and post-installation audit checks on all test environments must be available.
Acceptance of the application from the third party supplier for the purpose of conducting the user acceptance test, would be dependent on all the following tests being completed successfully:
AC201 - The test headquarters site personnel database can be successfully converted to the new format
AC202 - The test non-headquarters site personnel database can be successfully converted to the new format
AC203 - The existing test headquarters site personnel system can be successfully upgraded to the new version by following the instructions in the installation manual
AC204 - The existing test non-headquarters site personnel system can be successfully upgraded to the new version by following the instructions in the installation manual
AC205 - A copy of the personnel system can be successfully installed on the new test corporate database site by following the instructions in the installation manual
AC206 - The test corporate database site personnel database can be successfully populated with the personnel details from both the headquarters and non-headquarters sites via the overnight batch update
The installation tester would be a typical non-technical user who would normally be expected to conduct the actual installations.
This test case has two objectives:
· To measure that the test headquarters site personnel database can be successfully converted to the new format.
· To measure that by following the instructions in the installation manual, the existing test headquarters site personnel system can be successfully upgraded to the new version.
The scope of this test case is the test headquarters site personnel system conversion and upgrade.
AC201 - The test headquarters site personnel database can be successfully converted to the new format
AC203 - The existing test headquarters site personnel system can be successfully upgraded to the new version by following the instructions in the installation manual
The installation manual must have been successfully reviewed.
Test headquarters site environment preparation must be complete.
The upgraded personnel system on the test headquarters site has the correct site location displayed. (AC201)
The test headquarters site environment has the correct personnel system modules installed and that they can be accessed using correct user profiles. (AC203)
This test case has two objectives:
· To measure that the test non-headquarters site personnel database can be successfully converted to the new format.
· To measure that by following the instructions in the installation manual, the existing test non-headquarters site personnel system can be successfully upgraded to the new version.
The scope of this test case is the test non-headquarters site personnel system conversion and upgrade.
AC202 - The test non-headquarters site personnel database can be successfully converted to the new format
AC204 - The existing test non-headquarters site personnel system can be successfully upgraded to the new version by following the instructions in the installation manual
The installation manual must have been successfully reviewed.
Test non-headquarters headquarters site environment preparation complete.
1. Perform upgrade of existing non-headquarters site personnel system following instructions in installation manual.
2. Check the non-headquarters database has been converted correctly - i.e. does the new non-headquarters personnel system have the correct site location displayed on the main menu.
3. Check the non-headquarters site test environment has the correct modules installed and that they can be accessed using correct user profiles.
The upgraded personnel system on the test non-headquarters site has the correct site location displayed. (AC202)
The test non-headquarters site environment has the correct personnel system modules installed and that they can be accessed using correct user profiles. (AC204)
The test case has two objectives:
 | To measure that by following the instructions in the installation manual, the personnel system can be successfully installed on the test corporate database site. |
| To measure that the test corporate database site personnel database can be successfully populated with the personnel details from both the headquarters and non-headquarters sites via the overnight batch update |
The scope of this test case is initially the installation of the personnel system on the test corporate database site. This is followed by the running of the overnight batch update and subsequent population of the test corporate database site personnel database with the personnel details from both the headquarters and non-headquarters sites.
AC205 - The personnel system can be successfully installed on the test corporate database site by following the instructions in the installation manual
AC206 - The corporate database site personnel database can be successfully populated with the personnel details from both the headquarters and non-headquarters sites via the overnight batch update
The installation manual must have been successfully reviewed.
Test corporate database site environment preparation complete with no personnel system present.
Prior to the overnight batch run test cases TC201 and TC202 must have been completed successfully.
The test corporate database site environment has the correct site location displayed. The correct personnel system modules have been installed and they can be accessed using correct user profiles. (AC205)
Selected personnel details from test headquarters and test non-headquarters sites can be viewed on corporate database. (AC206)
Three test environments would be required, one for the headquarters site, one for a non-headquarters site and one for the corporate database site.
Headquarters environment
The test headquarters environment would be as close to the live headquarters environment as possible. To achieve this the following steps would be taken:
· Install existing version of personnel system on to the test headquarters environment.
· Copy live headquarters site database to this test environment.
· Scramble data on test environment to ensure confidentially of sensitive personnel details.
· Update test headquarters site database with additional test scenario data to be used during user acceptance testing.
Non-headquarters site environment
The test non-headquarters environment would be as close to a typical live non-headquarters environment as possible. To achieve this the following steps would be taken:
· Install existing version of personnel system on to the test non-headquarters environment.
· Copy one of the live non-headquarters site databases to this test environment.
· Scramble data on test environment to ensure confidentially of sensitive personnel details.
· Update test headquarters site database with additional test scenario data to be used during user acceptance testing.
Corporate database site environment
The test corporate database environment would be as close as possible to what will be the live environment. No personnel system would be present on this environment at the start of the confidence test stage.
A pre-installation audit check will be conducted on all test environments to establish a base list to be checked against after the post-installation audit check has been performed.
The installation tester performs the installation manual review.
The above test cases will be executed and the actual results matched against the expected results.
A post-installation audit check will be conducted on all test environments. This will be matched to the results of the pre-installation audit check to ensure that all software and files present are as expected.
The success of the installation testing in this scenario will be measured in terms of the following:
· Were the acceptance criteria defined earlier measured satisfactorily?
· Were the results of the pre and post-installation audit check satisfactorily?
· After installation is the personnel system in all three environments resident and run-able?