Threading and Multiprocessing
Python offers two main modules for concurrency: `threading` (for I/O‑bound tasks) and `multiprocessing` (for CPU‑bound tasks). Threads share memory but have the GIL (Global Interpreter Lock), limiting true parallelism for CPU work. Processes bypass the GIL but have higher overhead.
Threading Example
Create threads that run concurrently.
import threading
import time
def worker(name, delay):
print(f"{name} starting")
time.sleep(delay)
print(f"{name} finished")
t1 = threading.Thread(target=worker, args=("A", 2))
t2 = threading.Thread(target=worker, args=("B", 1))
t1.start()
t2.start()
t1.join()
t2.join()
Multiprocessing Example
Processes run in parallel, useful for CPU‑intensive tasks.
import multiprocessing
def cpu_bound(n):
return sum(i*i for i in range(n))
if __name__ == "__main__":
with multiprocessing.Pool(processes=4) as pool:
results = pool.map(cpu_bound, [10_000_000, 10_000_000])
print(results)
Choosing the Right Tool
- Threads: I/O‑bound tasks (network calls, file I/O).
- Processes: CPU‑intensive tasks.
- `asyncio` (covered elsewhere) for high‑concurrency I/O.
Two Minute Drill
- `threading` for I/O‑bound tasks (GIL limits parallelism).
- `multiprocessing` for CPU‑intensive tasks (true parallelism).
- Both provide similar APIs (start, join).
- Use `Pool` for parallel mapping.
- Be careful with shared state; use queues or locks.
Need more clarification?
Drop us an email at career@quipoinfotech.com