Skip to Content

Changes for page Testing Principles

Last modified by chrisby on 2024/04/01 12:52
From version 5.9
edited by chrisby
on 2023/10/27 09:23
Change comment: There is no comment for this version
To version 5.10
edited by chrisby
on 2023/10/27 09:31
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -7,10 +7,10 @@
7 7  * **Falsifiability** = The ability to fail.
8 8  ** Wrongfully failing tests are easy to detect, because a good developer will pay attention to all failing tests and fix the problems before moving on.
9 9  ** Wrongfully passing tests lack the ability to fail properly and are hard to detect because everything seems fine, which is why it is so important to focus on preventing them. They do damage by appearing to validate functionality that they do not. A properly executed Test Driven Development workflow is one of the best ways to avoid these kinds of problems, which are described in [[this article>>doc:Software Engineering.Agile.Extreme Programming.Test-Driven Development.TDD Workflow Example.WebHome]].
10 -* **Determinism and Reproducibility**: Repeated tests should always produce the same result. The success of tests should never depend on anything other than the production code. Avoid sources of randomness that could cause the test to fail even though the production code is fine. These include random number generators, threads, host-specific infrastructure (operating system, absolute paths, hardware: I/O speed or CPU load), networking over the local network or Internet (since the network or other hosts may sometimes be down), time/timestamps, pre-existing values in the database or files on the host system that are not in the source code.
11 -* **Single Action Execution**: The entire setup and testing process should require a single click or shell command. Otherwise, they will not be used regularly, which reduces the quality assurance of the production code.
10 +* **Determinism and Reproducibility**: Repeated tests should always produce the same result. The success of tests should never depend on anything other than the production code. Avoid sources of randomness that could cause the test to fail even though the production code is the same as it was when the tests passed in a previous test run. These include random number generators, threads, host-specific infrastructure (operating system, absolute paths, hardware: I/O speed or CPU load), networking over the local network or Internet (since the network or other hosts may sometimes be down), time/timestamps, pre-existing values in the database or files on the host system that are not in the source code.
11 +* **Single Action Execution**: The entire setup and testing process execution should require a single click or shell command. Otherwise, they will not be used regularly, which reduces the quality assurance of the production code.
12 12  * **Independence**: Tests should be independent and not influenced by other tests; the order in which tests are executed should not matter.
13 -** Promote test independence by **avoiding mutable states**. Use constants for shared data between tests where feasible. For mutable shared data that may be modified by tests, setup methods typically initialize class variables before each test to ensure correct values. The class or application under test can be restored to its original state by calling cleanup methods after each test.
13 +** Promote test independence by **avoiding mutable states**. Use constants for shared data between tests where feasible. For mutable shared data that may be modified by tests, setup methods typically initialize instance variables before each test to ensure correct values. The class or application under test can be restored to its original state by calling cleanup methods after each test.
14 14  * **Self-Containment**: The result of tests should depend only on the source code and not on external factors to support reproducibility.
15 15  * **Clean Test Code**: Maintain the same high standards for test code as for production code. The only exception is execution speed - tests need to be fast, but not highly optimized, as they typically deal with only small amounts of test data.
16 16  * **Easy Bug Tracing**: Failed tests should clearly indicate the location and reason for the failure. Measures to achieve this __could__ include using only **one assertion per test**, using **error logs**, and using **unique exceptions and error messages** where appropriate.