Testing Concurreny

Problems

• Hard for Developers: Testing for concurrency is very difficult, which is why concurrency should be avoided in the first place.
• Resource Intensive: In general, this requires a lot of iteration, which makes it resource-intensive.
• Undeterministic: The result is architecture dependent (OS, hardware), which introduces randomness and makes error detection unreliable.

Solution Approaches

• Monte Carlo Testing: Write flexible, adaptive tests that can be executed with a variety of test settings, such as the number of threads used, types of operations, order of operations, and so on. Run these tests repeatedly on a test server, randomly varying the test settings. If something fails, then there is a problem with the code, and the settings used should be logged.
• CI Integration: Do Monte Carlo Testing early to gain tests ASAP for your test repertoire or CI server. Run tests on each platform over time to either confirm the correctness of the production code or to uncover bugs in the test code.
• Frequent Execution: Run the tests on a single machine, simulating application loads whenever possible.
• Tools: There are tools for testing concurrent code, such as ConTest.