Concurrency
- Objects are abstractions of processing, threads are abstractions of timing.
Why concurrency?
- Concurrency is a decoupling strategy. The what is decoupled from the when.
- Concurrency is can improve the throughput and structure of an application.
Why not concurrency?
- Unclean: It is hard to write clean concurrent code, and it is harder to test and debug.
- Design Changes: Concurrency doesn't always improve performance behavior and but it always requires fundamental design changes.
- Extra Management: Concurrency demands a certain amount of management effort, which degrades performance behavior and requires additional code.
- Complexity: Proper concurrency is complex, even for simple problems.
- Unreproducible: Concurrency bugs are usually not reproducible; therefore, they are often written off as one-time occurrences (cosmic rays, glitches, etc.) rather than treated as true defects, as they should be.
- Side-Effects: When threads access out-of-sync data, incorrect results may be returned.
Principles of Defensive Concurrency Programming
- Single-Responsibility Principle
- Separation of code: Changes to concurrent code should not be mixed with changes to the rest of the code. So you should separate the two cleanly.
- Separation of change: Concurrent code has special problems that are different, and often more serious, than sequential code. This means that concurrent and sequential code should be changed separately, not within the same commit, or even within the same branch.
- Principle of Least Privilege: Limit concurrent code to the resources it actually needs to avoid side effects. Minimize the amount of shared resources. Divide code blocks and resources into smaller blocks to apply more granular, and therefore more restrictive, resource access.
- Data Copies: You can sometimes avoid shared resources by either working with copies of data and treating them as read-only objects, or by making multiple copies of data, having multiple threads compute results on them, and merging those results into a single thread. It is often worth creating multiple objects to avoid concurrency problems.
- Independence: Threads should be as independent as possible. Threads should not share their data or know anything about each other. Instead, they should prefer to work with their own local variables. Try to break data into independent subsets that can be processed by independent threads, possibly in different processes.
Things to learn before working with concurrency
- Get to know your library
- Use the thread-safe collections provided.
- Use the executor framework to execute disjointed tasks.
- Use non-blocking solutions if possible.
- Multiple library classes are not thread-safe.
- Thread-safe collections
- So you should use ConcurrentHashMap instead of HashMap.
- Author's recommendations: java.util.concurrent, java.util.concurrent.atomic, java.util.concurrent.locks.
- Get to know execution models
- Basic definitions
- Bound Resources
- Mutual Exclusion
- Starvation
- Deadlock
- Livelock
- Thread Pools
- Future
- Also these??
- Synchronization: General term for techniques that control the access of multiple threads to shared resources.
- Race Condition: A situation where the system's behavior depends on the relative timing of events, often leading to bugs.
- Semaphore: An abstract data type used to control access to a common resource by multiple threads.
- Locks: Mechanisms to ensure that only one thread can access a resource at a time.
- Atomic Operations: Operations that are completed in a single step relative to other threads.
- Thread Safety
- Race Conditions
- Statelessness, Statefulness
- Functional Programming
- Cloning Data to avoid side effects
- Side effects
- Producer-consumer
- Reader Writer
- Philosopher problem → Study algorithms and their application in solutions.
- Basic definitions
A few more suggestions from ChatGPT:
Learning about concurrency algorithms and common concurrency problems is a great way to deepen your understanding of concurrent programming. Here's a list of key algorithms and problems, along with their typical solutions:
Concurrency Algorithms:
Producer-Consumer:
Problem: How to handle scenarios where one or more threads (producers) are producing data and one or more threads (consumers) are consuming it.
Solution: Use buffers, queues, semaphores, or condition variables to synchronize producers and consumers.
Readers-Writers:
Problem: How to manage access to a shared resource where some threads (readers) only read data, and others (writers) write data.
Solution: Implement mechanisms to ensure that multiple readers can access the resource simultaneously, but writers have exclusive access.
Dining Philosophers:
Problem: A classic synchronization problem dealing with resource allocation and avoiding deadlocks.
Solution: Strategies include resource hierarchy, arbitrator, or limiting the number of philosophers.
Barriers:
Problem: Synchronizing a group of threads to wait until they have all reached a certain point in their execution.
Solution: Use barrier constructs that block threads until all have reached the barrier.
Concurrency Problems and Solutions:
Deadlocks:
Problem: Occurs when multiple threads or processes are waiting on each other to release resources, and none of them can proceed.
Solution: Deadlock prevention techniques (like resource ordering), deadlock avoidance (like Banker’s algorithm), and deadlock detection and recovery.
Race Conditions:
Problem: Occurs when the outcome of a program depends on the relative timing of threads or processes.
Solution: Use mutual exclusion (mutexes), atomic operations, or transactional memory to ensure that only one thread can access the shared resource at a time.
Livelocks:
Problem: Threads or processes are actively performing concurrent operations, but these operations do not progress the state of the program.
Solution: Careful algorithm design to ensure progress and avoid situations where processes continuously yield to each other.
Starvation:
Problem: A thread or process does not get the necessary resources to proceed, while others continue to be serviced.
Solution: Implement fair locking mechanisms, priority scheduling, or resource allocation strategies that ensure all processes get a chance to proceed.
Priority Inversion:
Problem: A lower-priority thread holds a resource needed by a higher-priority thread, leading to the higher-priority thread waiting unexpectedly.
Solution: Priority inheritance protocols where the lower-priority thread temporarily inherits the higher priority.
Thread Interference:
Problem: When multiple threads are accessing and modifying shared data, causing unexpected results.
Solution: Ensure that critical sections of code that access shared resources are protected using synchronization mechanisms like locks.
Concurrency Algorithms:
Producer-Consumer:
Problem: How to handle scenarios where one or more threads (producers) are producing data and one or more threads (consumers) are consuming it.
Solution: Use buffers, queues, semaphores, or condition variables to synchronize producers and consumers.
Readers-Writers:
Problem: How to manage access to a shared resource where some threads (readers) only read data, and others (writers) write data.
Solution: Implement mechanisms to ensure that multiple readers can access the resource simultaneously, but writers have exclusive access.
Dining Philosophers:
Problem: A classic synchronization problem dealing with resource allocation and avoiding deadlocks.
Solution: Strategies include resource hierarchy, arbitrator, or limiting the number of philosophers.
Barriers:
Problem: Synchronizing a group of threads to wait until they have all reached a certain point in their execution.
Solution: Use barrier constructs that block threads until all have reached the barrier.
Concurrency Problems and Solutions:
Deadlocks:
Problem: Occurs when multiple threads or processes are waiting on each other to release resources, and none of them can proceed.
Solution: Deadlock prevention techniques (like resource ordering), deadlock avoidance (like Banker’s algorithm), and deadlock detection and recovery.
Race Conditions:
Problem: Occurs when the outcome of a program depends on the relative timing of threads or processes.
Solution: Use mutual exclusion (mutexes), atomic operations, or transactional memory to ensure that only one thread can access the shared resource at a time.
Livelocks:
Problem: Threads or processes are actively performing concurrent operations, but these operations do not progress the state of the program.
Solution: Careful algorithm design to ensure progress and avoid situations where processes continuously yield to each other.
Starvation:
Problem: A thread or process does not get the necessary resources to proceed, while others continue to be serviced.
Solution: Implement fair locking mechanisms, priority scheduling, or resource allocation strategies that ensure all processes get a chance to proceed.
Priority Inversion:
Problem: A lower-priority thread holds a resource needed by a higher-priority thread, leading to the higher-priority thread waiting unexpectedly.
Solution: Priority inheritance protocols where the lower-priority thread temporarily inherits the higher priority.
Thread Interference:
Problem: When multiple threads are accessing and modifying shared data, causing unexpected results.
Solution: Ensure that critical sections of code that access shared resources are protected using synchronization mechanisms like locks.
Watch out for dependencies between synchronized methods
- Dependencies between synchronized methods in concurrent code cause subtle bugs.
- Avoid applying more than one method to a shared object. If this is not possible, you have three options:
- Client-based locking: the client should lock the server before the first method is called and ensure that the lock includes the code that calls the last method.
- Server-based locking: Create a method in the server that locks the server, calls all methods, and then unlocks the server. Have the client call the new method.
- Adapted Server: Create an intermediate component that performs the lock. This is a variant of server-based locking if the original server cannot be changed.
- Keep synchronized sections small.
- Locks are expensive because they add administrative overhead to delays. On the other hand, critical sections must be protected.
- Critical sections, are parts of the code that are only executed correctly if several threads do not access it at the same time.
- Keep synchronized sections as small as possible.
Writing correct shutdown code is difficult
- You should think about a shutdown as early as possible and get it running as soon as possible. If you wait, it will always take longer. Study the available algorithms because this task is probably harder than you think.
- TODO: Why do I need shutdown code?
Some Notes:
- When I/O is the bottleneck of your application, more threads will increase the performance in opposite to when CPU is the bottleneck.
- Stress Testing: Checking the throughput of an application by sending a huge amount of requests and examining the response times.
- Isolate concurrent code by putting it in a few separate classes.
- Always consider the concept of execution paths: The amount of possible interleaving of instructions that are processed by at least two threads. For example, objects with mutable states could unintentionally cause different results doing the same operation twice.
- Atomic operation = operation which can not be interrupted by other threads. But for unsynchronized processes threads can put instructions between two atomic operations.
- synchronized prevents unintended side-effects.
- Server-based locking is preferred over client-based locking.
- Server-based locking: The class used takes care of internal locking, so the user has nothing else to worry about.
- Client-based locking: User has to manually implement locking. This approach error prone and hard to maintain.
- If there is no access to the server an adapter class can be used instead. Even better would be thread-save collections using extended interfaces.
- As little synchronized code (synchronized) as possible should be used. And if, then only for small, critical code sections.