Repository navigation
Conversation
…ibutes Setting __name__, __qualname__, __doc__ or __module__ on a function stopped the world in the free-threaded build since pythongh-154826 and pythongh-154851, making functools.wraps() orders of magnitude slower with multiple threads. Use the per-object critical section and _PyObject_XSetRefDelayed() instead, as gen_set_name() already does. Use acquire loads in the readers and hold strong references across error reporting, tracing, and eval-frame recording that may execute Python or detach the thread. The __code__, __defaults__, __kwdefaults__ and __closure__ setters still stop the world to clear func_version. Document synchronization requirements for direct field access and the lifetime of borrowed function attributes.
Documentation build overview
6 files changed ·
|
Windows uses uppercase letters for hex addresses.
| #ifdef Py_GIL_DISABLED | ||
| return (PyObject *)_Py_atomic_load_ptr_acquire( | ||
| &_PyFunction_CAST(func)->func_module); | ||
| #else | ||
| return _PyFunction_CAST(func)->func_module; | ||
| #endif |
There was a problem hiding this comment.
Would something like this work? It's a little shorter and should have the same effect. (Guessing there's a macro-expansion related reason you didn't use FT_ATOMIC_LOAD_PTR_ACQUIRE here because it's used liberally elsewhere in the PR.)
| #ifdef Py_GIL_DISABLED | |
| return (PyObject *)_Py_atomic_load_ptr_acquire( | |
| &_PyFunction_CAST(func)->func_module); | |
| #else | |
| return _PyFunction_CAST(func)->func_module; | |
| #endif | |
| PyFunctionObject *fn = _PyFunction_CAST(func); | |
| return FT_ATOMIC_LOAD_PTR_ACQUIRE(fn->func_module); |
| Py_XSETREF(func->func_annotations, annotations); | ||
| Py_CLEAR(func->func_annotate); |
There was a problem hiding this comment.
I think we need to use the same pattern as before, otherwise I think it's possible to end up with a race like:
- Thread A stores the new annotations dictionary.
- Decrefing the old dictionary runs a value’s
__del__, which blocks and suspends the critical section. - Thread B sets
f.__annotate__ = callback, installing the callback and clearingfunc_annotations. - Thread A resumes, reacquires the function lock, and executes
Py_CLEAR(func->func_annotate), deleting B’s callback.
Something like this would work:
| Py_XSETREF(func->func_annotations, annotations); | |
| Py_CLEAR(func->func_annotate); | |
| Py_BEGIN_CRITICAL_SECTION(func); | |
| PyObject *old_annotations = func->func_annotations; | |
| PyObject *old_annotate = func->func_annotate; | |
| func->func_annotations = annotations; | |
| func->func_annotate = NULL; | |
| Py_END_CRITICAL_SECTION(); | |
| Py_XDECREF(old_annotations); | |
| Py_XDECREF(old_annotate); |
There was a problem hiding this comment.
Separately, codex tells me that we need to update PyFunction_GetAnnotations in light of these changes. There are a lot of non-atomic reads that were previously fine b/c of the STW pause in PyFunction_SetAnnotations.
|
When you're done making the requested changes, leave the comment: |
Setting
__name__,__qualname__,__doc__or__module__on a function stopped the world in the free-threaded build since gh-154826 and gh-154851, making functools.wraps() orders of magnitude slower with multiple threads.Use the per-object critical section and
_PyObject_XSetRefDelayed()instead, asgen_set_name()already does. Use acquire loads in the readers and hold strong references across error reporting, tracing, and eval-frame recording that may execute Python or detach the thread. The__code__,__defaults__,__kwdefaults__and__closure__setters still stop the world to clear func_version.Document synchronization requirements for direct field access and the lifetime of borrowed function attributes.
Benchmark results (release free-threaded build, each thread creates its own function, 5000 iterations, time in seconds):
__name____name____name__+__doc____name__+__doc__Note: this PR was created with assistance from LLM agents (Claude Fable and GPT 6.1 Sol).
__name__/__doc__stops the world;functools.wrapsfrom multiple threads is orders of magnitude slower since 3.14.7 #159136