What should be tested? ➡ Every functionality you expect the software to provide at any moment. You should test:
- User Cases
- Defined in project requirements
- Use cases derived from the high-level project use cases. For example, expected behavior of functions, classes, modules and components that the user does not see.
- Border cases that could theoretically always occur, such as maximum/minimum values, nulls, invalid input, nulls, negative numbers, empty lists, values with special meaning, exceptions, etc.
Principles
- High Coverage:
- Code Coverage Definition: The percentage of classes/methods/lines of code executed during testing. There is class coverage, method coverage and line coverage, the latter being the most important.
- 100% code coverage is the goal. Untested code is insecure code and prone to bugs.
- Leverage code coverage tools that determine code coverage during test execution.
- Meaningful Tests: Don't add tests that don't add value or test something that has already been tested.
- Strictness: If at least one test fails, it requires the developer's attention. He is not allowed to proceed with other work until that problem is fixed, and all tests must pass without exception.
- Falsifiability = The ability to fail. False-negative tests are easy to catch, but false-positive tests aren't, which is what makes them so tricky. Use TDD, and get into the habit of mutating the critical section of production code to trigger the test failure, see TDD article. A test should fail before production code is written. Tests without assertions always pass, which is why the assertion should be the first thing written in a test to prevent forgetting it.
- Determinism: Repeated tests should always produce the same result. The success of tests should never depend on anything other than the production code. Avoid sources of randomness that could cause the test to fail only some of the time. These include random number generators, threads, host-specific infrastructure (operating system, absolute paths, hardware: I/O speed or CPU load), networking over the Internet, time/timestamps, pre-existing values in the database or files on the host system that are not in the source code.
- Single Action Execution: The entire setup and testing process should require a single click or shell command. Otherwise, they will not be used regularly, which reduces the quality assurance of the production code.
- Independence: Tests should be independent and not influenced by other tests; the order in which tests are executed should not matter.
- Clean Test Code: Maintain the same high standards for test code as for production code. The only exception is execution speed - tests need to be fast, but not highly optimized.
- Easy Bug Tracing: Failed tests should clearly indicate the location and reason for the failure. Measures to achieve this include using only one assertion per test, using logs, and using unique exceptions and error messages where appropriate.