This example is for review for review at the meeting on 8th April, with comments required by 3rd April.  All comments should be submitted using the feedback form.

Example Test Scenario 1 – Database Compatibility Oracle Designer 9i

Background

Oracle Designer is a software tool for analyzing business requirements and for designing and generating client/server systems that meet those requirements. Oracle Designer incorporates support for business process modeling, systems analysis, software design and system generation.

Oracle Designer provides a multi-user repository and is closely integrated with Oracle Developer, Oracle's client/server development toolset. In this way, Oracle Designer allows organizations to design and rapidly deliver scalable, client/server systems that can adapt to changing business needs.

Oracle9i Designer models business processes, data entities and relationships. Models are transformed into designs from which complete applications and databases are generated.

Designer provides extensive capabilities for modeling and generating database-oriented, web-based applications deployed using Java, XML, Oracle9i Forms and Reports, and dynamic HTML. Core to this capability is Designer's powerful database design, generation, and capture functionality, globally recognized as the only complete solution for Oracle9i database design.

Designer 9i is shipped as part of a pack with the 9i database and other development tools, there are a number of areas which require compatibility testing including for any new Database release:

·         Install in Oracle9i Home (9i RSF stack)

·         All components run against a 9i database

·         Generate and capture Oracle9i database 2

·         Generate and capture Oracle9i Forms & Reports

·         Run with other supported non-oracle databases

Note:

This compatibility testing is being performed using a production release version of Designer, i.e. Designer 9.0.2.1 against a pre-release and Production release of the 9i Database.  Each new database release is received in a number of builds which contain fixes for errors detected in previous builds.  Testing will need to be repeated, although depending on the bugs fixed it may not be necessary to repeat all tests for each new build, the Test Manager will analyse defects corrected between builds and assess any possible impact on the Designer Product and schedule testing appropriately.

Test preparation and strategy

Test approach:

There are a number of key measurements used to assess database compatibility for Designer:

Installation testing: Designer can be successfully installed in the same Oracle home as the 9i database

Transaction Timings: Data can be submitted to and retrieved from the database within the same time or quicker times than the current release available externally

Generation: Data stored within the repository can be generated to either an Oracle or Non-Oracle database

Design Capture: Data can be captured from either an Oracle or Non-Oracle database; Oracle 9i Designer will capture the design of any ODBC compliant database

Migration: Data can be migrated from previous releases of Designer

Upgrade: Data can be upgraded from previous releases of Designer

Import/Export: Data can be imported/exported into the Repository

Component Compatibility:: Designer Product sanity test which exercises all components to be run against a 9i database to ensure that there are no issues

Test Planning

Based on testing against pervious database releases the following priorities have been given to the test areas identified above:

Establishing Compatibility Goals

All Designer Components run against the Oracle 9i database

As part of testing it is imperative that the entire Designer toolset is verified for compatibility with the Oracle database.  Running the Designer sanity test, which can be run either manually, or through automation and utilises the all components within the product, does this.  As the emphasis of this test is on compatibility testers should run the test against the database version currently in production then rerun against the database version under tests, results should be the same or better. 

Migration

Migration is a key area for issues with database compatibility; typically this is as a result of new database types introduced with subsequent database releases or more often the result of more stringent data integrity rules.  Testers must ensure that all supported migration routes are verified against the database under test.

In addition, testers must use a selection of different customer sets of data, it is important that data which has been upgraded and/or migrated from earlier releases from 1.3.2 is used for testing as typically customer sites have data sets which have gone through a number of different iterations.  This is the most likely way to identify the type of data integrity issues, which have proved problematic during past compatibility testing.

Upgrade

Upgrade is another key area for issues with database compatibility; typically this is as a result of new database types introduced with subsequent database releases or more often the result of more stringent data integrity rules.  Testers must ensure that all supported Upgrade routes are verified against the database under test.

In addition, testers must use a selection of different customer sets of data, it is important that data which has been upgraded and/or migrated from earlier releases from 1.3.2 is used for testing as typically customer sites have data sets which have gone through a number of different iterations.  This is the most likely way to identify the type of data integrity issues, which have proved problematic during past compatibility testing.

Installation testing  

Testers must ensure that the database can be installed into the same Oracle Home as the database.  Ideally Testers should install in this way prior to running the Sanity Test which will then provide a method of verification of the installation option under test, whilst maximising testing resource.  Testers must ensure that both Standard and Enterprise editions of the database are tested and incorporate both local and remote installation.

Transaction Timings

Performance is a high priority for any database release.  Testers should run the set of defined user actions for each component.  The average time should then be calculated for all tests by component and this should be compared to results stored from previous performance testing, this will enable the measurement of and degradation in performance.

Generation

Testers should focus on generating data created using earlier releases the Oracle Database, data captured from earlier releases of the Oracle database but also data created in and captured from non-Oracle databases.  Testers should pay particular attention to timings of transactions.

Design Capture:

This testing will focus primarily on capturing data from the Oracle 9i database, as this testing is not aimed at certifying against any other Database.  However, testing will include capture from the following non-oracle databases:

To facilitate the testing of generation of this captured information into the Oracle 9i database and will form part of the testing for Generation.

Import/Export:

Testers must ensure that they can import Application systems created using either the release of Designer under test or previous releases, from Oracle 8.1.7 upwards.

Metrics

The following Metrics are used to assist in determining release readiness and test execution status.

Pass Rate:

This testing can pass if errors are detected, providing those errors are present when the same test is run against a different database version and therefore, cannot be as a result of running Designer against a new database version. 

Pass rates for tests should be compared with historical data from previous compatibility testing when running the same tests against the database version currently in production.

Test Execution:

All tests scheduled to be run should be executed, in the case of the 9i Database, testing started prior to the Database code being frozen and at production quality.  It is important to start testing at this early phase in order that any errors detected in the Database code which affect Designer can be corrected prior to the database going production as apposed to releasing patches post release and forcing customers to take this route.