- Every Python file gets compiled into VM instructions
- In cpython, it is unsafe to execute instruction concurrently
- Hence: Locking
duh missed this
- Threads using multiple CPUs
- The GIL is unlikely to go away anytime soon
- Can it be improved? Yes!
- How can it be done?
- How about Python 3!
- request/reply server for size-prefixed messages
- each method has payload/header
- Comes up in a lot of contexts
- Involves IO
- Used as a foundation for a lot of other things
1000 iterations of some simple code
Done on EC2 with nothing else running
- C + 0mq
- Python + 0mq
- Python + multiprocessing
- Python + blocking sockets
- Python + nonblocking sockets
- All finish in about 13 seconds
What does it do to the performance?
- C + 0mq (samish seed)
- Python + 0mq (7x slower)
- Python + multiprocessing (8.9x slower)
- Python + blocking sockets (approx 10.x slower)
- Python + nonblocking sockets (approx 10.x slower)
import sys import threading def spin(value): sys.set_priority(-1)
Huge boost in Python with only minor changes to a few files
Is this the only GIL improvement?
- There are other ways to do it
- GIL released on non-blocking I/O operations