Sharing a `contextvars.Context` iterator across threads crashes (HAMT iterator cursor corruption) under free-threading

[devdanzin]

Crash report

What happened?

On the free-threaded build, advancing a single shared iterator over a contextvars.Context from multiple threads corrupts the iterator's traversal cursor and dereferences a garbage node pointer → SIGSEGV.

The HAMT iterator keeps its whole cursor — i_nodes[] (borrowed node pointers), i_pos[], i_level — inside the iterator object and advances it with plain reads/writes and no critical section (Python/hamt.c, main@a1d580430c8):

static hamt_iter_t
hamt_iterator_next(PyHamtIteratorState *iter, PyObject **key, PyObject **val) {
    if (iter->i_level < 0)                       /* :2188  plain read of the cursor level */
        return I_END;
    PyHamtNode *current = iter->i_nodes[iter->i_level];   /* :2194  read node at current level */
    if (IS_BITMAP_NODE(current)) {               /* :2196  Py_TYPE(current) -> SEGV on a wild node */
        return hamt_iterator_bitmap_next(iter, key, val);
    }
    ...
}

static hamt_iter_t
hamt_iterator_bitmap_next(PyHamtIteratorState *iter, ...) {
    int8_t level = iter->i_level;
    PyHamtNode_Bitmap *node = (PyHamtNode_Bitmap *)(iter->i_nodes[level]);
    if (pos + 1 >= Py_SIZE(node)) {              /* :2092  Py_SIZE(node) -> SEGV on a wild node */
        iter->i_level--;                         /* :2097  plain write of the cursor level */
        return hamt_iterator_next(iter, key, val);
    }
    ...
}

Two threads calling next() on the same iterator desync i_level against i_nodes[], so a thread reads current = iter->i_nodes[iter->i_level] as a stale / NULL / wild pointer and immediately evaluates IS_BITMAP_NODE(current) = Py_TYPE(current) (or Py_SIZE(node) in hamt_iterator_bitmap_next) — dereferencing garbage. Because the nodes in i_nodes are held borrowed (hamt_iterator_init: "we don't incref/decref nodes in i_nodes"), a concurrent structural change is not even required — cursor desync alone produces the wild dereference. This is safe under the GIL (one thread runs the advance at a time).

The HAMT iterator backs contextvars.Context iteration — iter(ctx), ctx.keys(), ctx.values(), ctx.items() — via hamt_baseiter_tp_iternext (hamt.c:2479).

Reproducer

Free-threaded build, PYTHON_GIL=0:

import contextvars, threading
NT = 8; ITERS = 40000
vs = [contextvars.ContextVar(f"v{i}") for i in range(16)]
ctx = contextvars.copy_context()
def populate():
    for i, v in enumerate(vs):
        v.set(i)
ctx.run(populate)
cell = [iter(ctx)]
def worker():
    for _ in range(ITERS):
        it = cell[0]
        try: next(it)
        except StopIteration: cell[0] = iter(ctx)
        except Exception: pass
ts = [threading.Thread(target=worker) for _ in range(NT)]
for t in ts: t.start()
for t in ts: t.join()

• 6/6 SIGSEGV on a plain free-threaded debug build (no sanitizer).
• On a free-threaded ASan build:

AddressSanitizer: SEGV in _Py_TYPE_impl  Include/object.h:234
  hamt_iterator_bitmap_next  Python/hamt.c:2092
  hamt_iterator_next         Python/hamt.c:2196
  hamt_baseiter_tp_iternext  Python/hamt.c:2479
  context_run                Python/context.c:731

• Under TSan: WARNING: ThreadSanitizer: data race … Python/hamt.c:2188 in hamt_iterator_next.

The crash is not Py_DEBUG-only — it reproduces as a raw SIGSEGV from the wild-pointer dereference on the plain free-threaded build.

Relationship to gh-124397

Per gh-124397 ("Strategy for Iterators in Free Threading") point 3, C iterators should get "the minimal changes necessary to cause them to not crash … Concurrent access is allowed to return duplicate values, skip values, or raise an exception." The HAMT / Context iterator was never brought under that hardening, so it crashes rather than returning dup/skip. It is the contextvars.Context sibling of the already-tracked shared-iterator crashes: dict (#154130), set (#144357), and memoryview.

Suggested fix

Bring the HAMT iterator advance under a per-iterator critical section (or make the i_level/i_pos/i_nodes cursor accesses atomic with a NULL/bounds check on the borrowed i_nodes[i_level] before dereferencing), matching the dictiter/setiter hardening. The guard must span the whole hamt_iterator_next → *_bitmap_next / *_array_next descent because i_nodes holds borrowed pointers.

Found with fusil's --tsan mode; reproducer reduced and this report drafted with AI assistance (Claude Code), then verified by hand on free-threaded debug / ASan / TSan builds of main.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.16.0a0 free-threading build (heads/main:a1d580430c8, Jul 18 2026, 20:23:36) [Clang 21.1.8 (6ubuntu1)]