AllPeers Manual Testing

At the time of this writing, AllPeers is only performing manual testing. Yet when writing manual test cases, automation is considered. Specifically, more test cases of fewer steps are written as opposed to less test cases of many steps, yet with thorough expected results detailed.

For test case writing, test planning, and the recording of test results, Testopia is used. Testopia is an extension of Bugzilla and its integration with Bugzilla is the primary reason it is used. For Testopia instructions, see its manual. The purpose of this document is to explain AllPeer’s usage of Testopia.

One test plan will exist for each version of AllPeers and the name of the plan will have the same name as the version. For example, the first Testopia plan is called Version 0.55. The Product tested by the plan shall be “Extension,” the Product Version shall be the AllPeers version to be tested, and the Type shall be “Function.” At the present time, “functional” tests are the only type performed. Later, separate test plans for “performance” as well as other test types may also be written. Every new “functional” test plan will be created by...

1. cloning the last plan.
2. changing the status of obsolete test cases to “disabled” and unlinking them.
3. editing the test case IDs of the system tests of the previous release to regression test IDs. (For more information see the section below on the test case naming convention.)
4. removing the “System” Tag from the system tests of the previous release.
5. modifying existing test cases as need be due to upcoming design changes.
6. adding the system test cases for the release’s enhancements.

First a test case is given a test case ID which contains five segments separated by a hyphen. The first segment is two characters and indicates the test case type. Current types are system tests (SY) and regression tests (RG). The second segment is three characters and indicates the operating system — Windows (WIN), Macintosh (MAC) or Linux (LNX). The third segment is 3-8 characters and indicates the functional area. The functional area corresponds to Testopia’s Category field. Current functional areas are Chat (CHAT), Import Contacts (IMPCON), Invite (INVITE), Properties (PROP), Registration (REGIST), Remove (REMOVE), Share (SHARE), and Transfers (TRANS). The fourth segment denotes a subfunction. When the subfunction isn’t clear MISC is used. And the fifth segment is three numeric characters and indicates the scenario number within the functional area.

New test cases written shall contain the following attributes...

1. its test case ID in the Alias field.
2. a scenario description in the Summary field.
3. one tag called “System” in the Tag field (unless it is for a gap regression scenario found).
4. another tag in the Tag field indicating the OS it tests on — Windows, MacOS or Linux (32-bit).
5. a category indicating the general function tested (new categories may need to be created depending on the scale of the enhancement tested).
6. the enhancement name in its Requirement field.
7. a “Confirmed” Status.
8. a “Normal” Priority.
9. and a “Manual” Automatic value.

The Component field, which is a Bugzilla field, relates more to the code and are not always applicable. Still, if the tester finds a relevant component, it may be used. But it is optional at the present time.

The Set Up section shall contain the scenario prerequisites. In the Action section shall be listed each step the scenario entails. In the Expected Results section shall be listed detailed expected results corresponding to each step in the Action section. The Breakdown section is currently not used.

Once the plan is complete, test runs can be built from the plan in which test results can be recorded. The initial status of every test case is idle. The passed status is self-explanatory. The blocked status is used for failures due to bugs already reported which haven’t been worked yet. And the failed status is used specifically for new bugs found. Additional Testopia test case statuses running and paused are not used. Whether or not a bug is worked in the release found depends upon the bug severity, priority and level of effort. Typically only low priority bugs are postponed. So an AllPeers release may be deployed with test cases in blocked status due to bugs postponed, but not with any test cases in failed or idle status.