Types of Tests

Last modified by chrisby on 2024/03/30 11:54

Introduction

Incorporating multiple test types into a software test architecture has numerous advantages:

Readability: easy to understand test architecture and purpose of specific tests through separation of concerns/problem isolation
Speed Optimization: Different types of tests have different execution speeds, which can be exploited to reduce test development time, see this article.
Comprehensive Coverage: Each test type focuses on a specific area of testing, all of which must be covered. A good test suite therefore contains a wide range of tests, both in terms of the number of tests and the types of tests.

List of Test Types

Unit tests are the lowest level and most isolated tests. As you go down the table, the tests become higher level and more integrated.

Type of Test	Purpose	Scope of Test	Comments
Unit Test	Checks all low-level details of a single, isolated unit.	Single unit, often a single class	Class dependencies are mocked for isolation. Rarely will there be testing of a couple of very tightly coupled classes that can't reasonably be tested separately.
Core Test	A hybrid between unit and component tests which combines the best of both.	Business logic of a component	Units with I/O operations such as networking and file interactions are mocked away for fast test execution. The business logic runs integrated. Tests are executed against the business logic interfaces.
Component Test	Checks a component running in isolation from other components.	Single component	All units of the component run integrated. Self-contained: Mock clients for external communication with other components (databases, other services, etc.) allow the component under test to run in isolation. An example of a component test would be to build a backend web server and mock the database client so that no real database is actually involved in the test. The backend is the only running component. Then run the backend and throw HTTP requests at it, checking for correct behavior in terms of functionality, security and performance.
Integration Tests	Checks communication between two or a few components.	Small subset of components	Often realized via REST calls between the components. Dependencies to components not included in the subset being tested are mocked.
Acceptance Tests	Verifies that the software requirements defined in the contract are met.	All components	Complete deployment of all components in a production-ready state. Automated tools are often used to simulate user interaction with the GUI to test its functionality.
Exploratory Test	Human testing of the final product to find problems that may have been overlooked.	All components	Even friends, colleagues, and family can sometimes be involved in the process, since GUIs are often designed for non-technical people.

Miscellaneous

Distinct Responsibilities: Each type of test covers a unique scope and provides value, even if two different types of tests cover the same line of code. This is neither duplication nor redundancy. Therefore, multiple test types are needed to provide optimal value for the test strategy. Another aspect is test execution speed - while some tests are very fast and can be run frequently on the developer's PC to get quick feedback (Test-Driven Development), other tests take too much time and are better executed via the CI pipeline but provide more precise feedback.
Cost and Coverage: Human work is expensive and fallible. Automated testing, as opposed to manual testing, minimizes costs and ensures comprehensive coverage without the need for human involvement.
Test Quantities: The higher levels of the testing pyramid require only a fraction of the functionality tested at the previous lower level. For example, unit tests check the low-level use cases of each unit, so they are more numerous. In contrast, the test use cases for a component are often executed against a comparatively small REST API. This means that there are fewer use cases for the component than there are for the aggregate use cases of its individual units, so fewer of the higher-level component tests are required.
- Test Pyramid: Test types are usually represented in the form of a pyramid, with each type representing a layer. The base is populated by a large number of unit tests, while the top contains numerically fewer high-level tests such as acceptance tests, reflecting the narrower structure of the pyramid.
Trends: Low-level tests are more self-contained/isolated, verify fine-granular low-level requirements, detailed/numerous, but fast to execute, while higher-level tests are more integrated, verify high-level project requirements, fewer in number, but slow to execute.