Python Internal Series - Global Interpreter Lock (GIL) and Memory Management

python-internal
Despite the bad reputation of GIL, it was arguably designed on purpose. The GIL actually comes with a lot of benefit.
Author

noklam

Published

May 29, 2021

Is GIL a bad design?

Most people first learn about GIL because of how it slows down Python program and prevent multi-threading running efficiently, however, the GIL is one of the reason why Python survive 30 years and still growing healthyly.

GIL is nothing like the stereotype people think, legacy, slow. There are multiple benefits GIL provide:

  • It speed ups single thread program.
  • It is compatible with many C Program thanks to the C API of CPysthon.

Global Interpreter Lock a.k.a Mutex Lock

To start with, GIL is a mutex lock.

Why GIL is needed in the first place?

Memory management. Python use something called “reference counting”, which make it different from many modern programming lanaguage. It is what allow Python programmer to lay back and let Python take care when to release memory. Precisely, it is actually the C program controlling the memory life cycle for Python (Cpython). Cpython is known as the default Python interpreter. It first compiles Python to intermediate bytecode (.pyc files). These bytecode then being interpreted by a virtual machine ane executed. It is worth to mention that other variants of Python exist, i.e. IronPython(C#), Jython(Java), Pypy(Python) and they have different memory management mechanisms.

Python Memory Management - Reference Count & Garbage Collection (gc)

import sys
sys.getrefcount(a)
3

Reference counting is a simple idea. The intuition is that if a particular object is not referenced by anything, it can be recycled since it will not be used anymore.

For example, the list [1] is now referenced by the variable a, so the reference count is incremented by 1.

import sys
a = [1]
sys.getrefcount(a)
2

Note that the reference count is 2 instead of 1. 1. The first reference is a = [1] 2. When the variable a is passed to sys.getrefcount(a) as an argument, it also increases the reference count.

del a

When del a is called, the list [1] have 0 reference count, and it is collected by Python automatically behind the scene.

Lock & Deadlock

Memory Management

Reference

  • https://www.youtube.com/watch?v=KVKufdTphKs&t=731s
  • https://realpython.com/python-gil/
  • https://devguide.python.org/garbage_collector/