Skip to content

bind: make _gopy_clear_go_tls opt-in, off by default#398

Open
satarsa wants to merge 1 commit into
go-python:masterfrom
satarsa:fix-395-clear-go-tls-opt-in
Open

bind: make _gopy_clear_go_tls opt-in, off by default#398
satarsa wants to merge 1 commit into
go-python:masterfrom
satarsa:fix-395-clear-go-tls-opt-in

Conversation

@satarsa

@satarsa satarsa commented Jul 23, 2026

Copy link
Copy Markdown

Summary

The _gopy_clear_go_tls() call added for issue #370 runs unconditionally before every CGo entry and performs a hardcoded TLS store (movq $0, %fs:-8 on linux/amd64, movq $0, %gs:0x30 on darwin/amd64). On glibc + CPython 3.12+ that offset overlaps the TLS slot CPython uses for the current thread state (_Py_tss_tstate), so the store nulls it and the interpreter segfaults on the first call: in the common single-extension case, which structurally cannot have the cross-runtime collision the clear was meant to guard against.

This PR makes the clear opt-in via a new -clear-go-tls build flag (default false): the default build is safe, and the previous behavior stays one flag away.

Fixes #395.

The two problems with the unconditional clear

  1. It runs for a single extension too. That is the common case and the one that cannot have a cross-runtime TLS collision, so there the clear is pure downside.
  2. The address is wrong for c-shared under glibc. In a normal Go binary the runtime lays out its TLS so the g pointer sits at %fs:-8. In a c-shared library loaded by glibc the loader owns the static TLS block and the g pointer is reached through a linker-assigned TLS variable (runtime.tls_g), not a fixed -8. So movq $0, %fs:-8 does not clear Go's g at all: it zeroes whatever thread-local happens to sit there, which on CPython 3.12+ is _Py_tss_tstate.

PR #393 already shipped the mechanism that actually isolates the runtimes: each extension is loaded without RTLD_GLOBAL, and gopy links each .so with a version script exporting only PyInit_*. Both keep every Go runtime symbol (including the g-pointer TLS) local to its own .so, so there is nothing shared to collide over. The raw-TLS clear is therefore redundant, and with the wrong offset actively harmful.

What this PR changes

  • Adds a bind.ClearGoTLS option and a -clear-go-tls build flag (default false), mirroring the existing -no-make flag across cmd_build.go, cmd_gen.go, cmd_pkg.go, cmd_exe.go.
  • Gates the single _gopy_clear_go_tls() emission site in bind/gen_func.go on that flag.
  • Leaves the C helper and its pybindgen registration in place, so -clear-go-tls restores today's behavior exactly.
  • Leaves the RTLD_GLOBAL handling untouched and unconditional.

6 files, +33/-8. gofmt, go vet ./..., and go test ./bind/ are clean.

Verification

Downstream (the crashing project). Gentoo, glibc, CPython 3.14.6, go 1.26.4,
gcc 16. Rebuilding slidge-whatsapp's extension:

  • Default (clear off): the generated wrapper contains zero _gopy_clear_go_tls() sites and the first call succeeds (exit 0).
  • gopy build -clear-go-tls: the call reappears (one per CGo entry) and the crash returns (SIGSEGV, exit 139).

So the flag is a faithful toggle, and off-by-default removes the crash with no post-build patching.

Multi-extension case, clear off. Two independent gopy extensions (two Go runtimes) loaded in one interpreter, 200000 interleaved cross-calls on the same OS thread with periodic gc.collect() (the condition #370 describes): 0 wrong results, no crash, no corruption. And each .so exports exactly one global dynamic symbol (PyInit_*) and zero runtime.* symbols, so the g pointer is per-runtime-private regardless of RTLD_GLOBAL.

I do not have the original #370 reproducer, so I cannot claim to have re-run that exact case; the opt-in keeps the old behavior available for anyone who needs it.

Follow-up (optional)

Happy to add regression tests, here or in a follow-up:

  1. A codegen assertion (layout-independent): default flags produce a wrapper without _gopy_clear_go_tls(), -clear-go-tls produces one with it.
  2. A two-extension _examples coexistence example, cross-called under gc.collect(), asserting a clean run with the clear off.

The unconditional _gopy_clear_go_tls() call (issue go-python#370) performs a
hardcoded TLS store (movq $0, %fs:-8 on linux/amd64). On glibc + CPython
3.12+ that offset overlaps CPython's current-thread-state TLS slot, so the
store nulls it and the interpreter segfaults on the first CGo entry in the
common single-extension case (issue go-python#395).

Add a -clear-go-tls build flag (default false) and gate the single call
site on it. The C helper and its pybindgen registration are left in place
so opt-in restores the previous behavior exactly. RTLD_GLOBAL-local
loading, which is what actually isolates each runtime's goroutine-pointer
TLS, is untouched and remains unconditional.

Fixes go-python#395
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#370's _gopy_clear_go_tls (movq $0, %fs:-8) hard-crashes CPython 3.12+ in a shipping extension

1 participant