Skip to Content

Changes for page Test Speedup

Last modified by chrisby on 2025/03/08 11:39
From version 1.5
edited by chrisby
on 2023/07/10 20:33
Change comment: There is no comment for this version
To version 1.60
edited by chrisby
on 2024/08/27 08:45
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -Enhance Test Execution Speed
1 +Test Speedup
Syntax
... ... @@ -1,1 +1,1 @@
1 -XWiki 2.1
1 +Markdown 1.2
Content
... ... @@ -1,18 +1,24 @@
1 -Fast-executing tests require less time and are executed more frequently, contributing to improved code quality. Therefore a high test execution speed is beneficial.
1 +Fast testing not only saves time, but also enables more frequent execution, leading to improved code quality. Optimizing the speed of test execution is therefore critical. While extensive and frequent testing is ideal, it shouldn't excessively slow the pace of development.
2 2  
3 +### General Measures
3 3  
4 -=== Measures ===
5 -
6 -* **Test type segregation**: Unit tests typically run much faster than other types of tests. For large test suites, consider running unit tests regularly on the developer's local machine, while scheduling more resource-intensive tests to run at a fixed rate in a continuous integration environment, for example.
7 -* **Partial testing**: You don't have to run all tests every time. Often, it is sufficient to run only the tests related to recently changed code.
5 +* **Selective Testing**: You don't need to run all tests every time. It can be sufficient to run only the tests related to recently changed code, or only the fast tests, and then run all the tests when you finished a major implementation step.
8 8  * **Mock slow dependencies** to minimize code execution time, especially operations such as I/O, transaction management, and networking.
9 9  * **Prefer in-memory databases during testing** for cleaner and faster operations compared to standard databases.
10 10  * **Identify performance bottlenecks** by increasing the number of threads:
11 -** If execution time remains constant, CPU is the bottleneck. Mitigate with faster CPUs, more cores, or additional machines.
12 -** If execution time decreases, I/O is the bottleneck. Use more threads, faster storage (such as SSDs), or additional storage for concurrent persistence operations.
9 + * If execution time remains constant, CPU is the bottleneck. Mitigate with faster CPUs, more cores, or additional machines.
10 + * If execution time decreases, I/O is the bottleneck. Use more threads, faster storage (such as SSDs), or additional storage for concurrent filesystem operations.
13 13  * **Improve I/O speed by using RAM disks**, such as Linux's tmpfs tool. Configure your tests to direct all file interactions to the RAM disk.
14 -* **Parallelize test execution.** Multiple threads can improve execution speed even on single-core processors by keeping the CPU busy while other threads wait for disk I/O.
15 -* **Offload CPU-intensive tasks** to cloud-based computing resources:
16 -** Upload project files to the cloud.
17 -** The cloud service builds the project, runs tests, and generates a test report.
18 -** Upon completion, download the test report from the cloud.
12 +* **Parallelize test execution.** Multiple threads can improve execution speed even on single-core processors by keeping the CPU busy while other threads wait for disk
13 +
14 +### Use DevOps Infrastructure
15 +
16 +A good developer should learn modern DevOps concepts, especially CI/CD pipelines and jobs. Beginners may want to start with a third-party DevOps infrastructure provider, such as GitLab or GitHub.
17 +
18 +* **Test type segregation**: Regularly run fast tests, such as unit tests, on the developer's local machine, while scheduling more resource-intensive tests in a CI environment.
19 +* **Asynchronous Testing**: Running the entire test suite locally may result in long wait times that block development. Instead, push the code to the source repository and let the CI environment run the test suite while you continue development. If a CI job fails, you will be notified so that you can prioritize fixing the problem.
20 + * **CI Concurrency**: Drastically enhance test execution speed by enabling concurrent CI pipelines and jobs.
21 + * **Concurrent CI pipelines**: Allow the developer to push code and spawn new CI pipelines immediately on different machines, even if previous pipelines are still running.
22 + * **Concurrent CI jobs**: Split the test suite into multiple independently executable CI jobs to allow concurrent execution.
23 + * Both of these measures require multiple machines to run the jobs, which can be demanding on the DevOps infrastructure but is often worth the cost.
24 + * **Scheduled Testing**: A less resource-intensive alternative is to run large test suites at a fixed interval, typically once a day at midnight, and run only the fast or important tests locally.