tarfile produces invalid pax archives when a gname is exactly 32 bytes long #159322

Bug report

This is a pretty minor bug, but it's also technically a form of pax non-conformance.

See astral-sh/uv#22510 for context.

Summary

TL;DR: the original tar format allocates a fixed 32 byte block for the gname field. pax says that this field should always be NUL terminated, meaning that any group name above 31 bytes in length should be encoded via a gname extension instead:

Ref: pax spec

However, what tarfile does is use all of the 32 bytes for content, with no NUL terminator. Many tar parsers recover from this, since the pax spec confusingly does allow this kind of non-termination in other fields (like name, linkname, and prefix). However, it's always incorrect for gname as well as magic and uname.

The fix here is behavior preserving: pax also allowing supplying the gname as an arbitrary-length pax extension, and pax emitters should emit that extension when a gname is longer than 31 bytes. pax extensions are always given precedence over their header counterparts, so this is fully compatible.

(A conforming pax emitter can also choose to emit pax extensions unconditionally, even when the values would fit in the header.)

Expected behavior

tarfile uses the gname pax extension to store overlong group names (and similar), rather than removing the required NUL terminator from the tar header representation.

Actual behavior

tarfile produces a pax-style tar that strict pax parsers reject.

In particular, tarfile should honor the NUL termination rule for the uname, gname, and magic fields in a pax archive. The latter is trivial since magic is fixed.

MRE

Copying the MRE from the issue linked above:

# make_sdist.py
import io, sys, tarfile

n = int(sys.argv[1])  # group name length
files = {
    "demo-0.1.0/pyproject.toml": b'[project]\nname = "demo"\nversion = "0.1.0"\n[build-system]\nrequires = ["setuptools>=77"]\nbuild-backend = "setuptools.build_meta"\n',
    "demo-0.1.0/demo/__init__.py": b"",
    "demo-0.1.0/PKG-INFO": b"Metadata-Version: 2.1\nName: demo\nVersion: 0.1.0\n",
}
with tarfile.open("demo-0.1.0.tar.gz", "w:gz", format=tarfile.PAX_FORMAT) as t:
    for name, data in files.items():
        info = tarfile.TarInfo(name)
        info.size = len(data)
        info.gname = "g" * n
        t.addfile(info, io.BytesIO(data))

Try with n=32 and n=40 to demonstrate slightly different encoding states (both result in an invalid gname header encoding, but the latter also produces the pax extension block).

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-159322: NUL-terminate owner names in PAX base headers #159329