gh-137574: Add Support For Special Names in `help` by adqm · Pull Request #137966 · python/cpython

adqm · 2025-08-19T17:25:40Z

Following on from #137574, this is an attempt to provide better help for special names from the built-in help function.

The original question was about help('__name__') and help('__dict__'), which suggested help for the str and dict built-in types, respectively, instead of reporting an error that no help was available for those topics. I also took this a step further, adding help for the special method names described on the 'Data Model' page of the language reference, as well as '__name__' and '__main__'.

Here's a quick summary of the changes I made:

When requesting help on a name '__x__', only report on the result of getattr(builtins, '__x__') if it is a type or a built-in function, preventing the confusion described above.
Added help for __name__, __main__, and the special method names as help topics.
Added "specialnames" as a new top-level category, and included it as a suggestion if we were looking up a name starting and ending with __.
Added a reference to the online Python docs in the message when a help topic isn't found, and slightly changed the wording of that message, hoping not to imply that things that don't have associated help topics aren't documented at all.

If any of these changes is a bridge too far, I can roll them back. Two other concerns, as well:

The code style of pydoc.py doesn't match what ruff would output, in a number of places. I opted to mimic the existing code's style rather than using input that ruff would be happy with; I can reformat, though, if that would be better.
One of the tests I added (which tests that each dunder method shows up in its help output) is quite slow. I could rework that so that it only tests a few names if that test being slow is a problem.

Issue: Add more useful help for special name in help() #137574

📚 Documentation preview 📚: https://cpython-previews--137966.org.readthedocs.build/

…iltins

…l.rst

… goes

AA-Turner · 2025-08-20T01:51:43Z

This PR is way too big. Also, don't regenerate pydoc_data, it's done during the release process.

ZeroIntensity · 2025-08-20T02:07:11Z

I think the majority of the diff is from pydoc_data.

AA-Turner · 2025-08-20T02:31:44Z

I'm on mobile so it's hard to tell (the pydoc data diff doesn't even render). I'll look at the PR substantively over the week though.

adqm · 2025-10-09T04:15:20Z

@AA-Turner, sorry to pester, and no rush at all, but just wondering if you have a sense of when you might be able to take a look at this PR.

github-actions · 2026-05-15T18:32:07Z

This PR is stale because it has been open for 30 days with no activity.

adqm · 2026-10-11T18:55:13Z

@AA-Turner I know it's been a while, but any chance you could take a look at this PR (and #138095) at some point? I'm happy to make adjustments if necessary.

ZeroIntensity

I think he's been on leave recently, so I can commit to reviewing this instead.

The overall approach here looks good, with just a few small oversights that I noticed. Two side notes:

Let's add an entry to "What's New in Python 3.16".
There are merge conflicts that need to be addressed.

StanFromIreland · 2026-10-12T20:11:41Z

Please revert this change, it will be done as part of the release.

The issue is that those changes are necessary for this PR to work, since some of the new topics added to help require topics that aren't currently in there and so would be broken / hard to test. I can revert if the idea is that reviewers will re-generate that file on their own for testing, but just wanted to flag that the PR won't work as expected without it before actually rolling it back.

We do have some precedent for regenerating between releases (#153909, #144934, #125074). Is there any harm in doing it here, especially if the feature can't be used without it?

Regeneration at release is the standard process (it's automatically done by release-tools), it makes this easier to review IMO.

Right, but not regenerating makes this very difficult to test locally. How does this make it more difficult to review? Just ignore the regenerated file.

Since we don't verify it in the CI, it's another file that needs manual verification.

Sure, but manually verifying a generated file is much easier (and less harmful if we mess up) than letting a feature hit main and not work until release. Keep in mind that removing the generated file will also mean we have to remove a bunch of test coverage from this PR. We would, in essence, be trading the guarantee that the generated file is correct for the guarantee that the feature works correctly.

I'm inclined to cite "practicality beats purity" here. Yes, regenerating is a very minor breach of process, but I think reviewing this PR is much more difficult without the regenerated file, not vice versa.

My concern is more around security than correctness here (and not specific to this PR or its author). Since these generated files aren't verified in CI, changes to them in PRs introduce another potential attack surface that reviewers need to manually inspect.

Another downside of regenerating these files outside of releases is the potential for merge conflicts, though I don't think that's much of an issue in the case of pydoc.

That said, I understand the testing concerns, so if you'd like to keep the generated file in and verify the changes, I won't block on it.

Yeah, I'm not saying we should make this a common policy; I'm just arguing that we should make an exception for this PR. I'll make sure it's correct before landing this.

Or, we could verify pydoc_data as part of the "Check if generated files are up to date" CI check, assuming the generation is pretty fast.

read-the-docs-community · 2026-10-12T20:13:20Z

Documentation build overview

📚 cpython-previews | 🛠️ Build #35151281 | 📁 Comparing cac3a02 against main (e29cce5)

🔍 Preview build

77 files changed · ± 77 modified

± Modified

adqm · 2026-10-12T20:19:25Z

Thanks for the quick and thorough review, @ZeroIntensity! I added a what's new entry and I believe I fixed the merge conflicts. I responded to some of your specific comments/requests inline above, and of course, happy for more input if you have it.

adqm · 2026-10-12T20:25:05Z

Oops, sorry, some of these changes broke some tests. Will ping again when that's resolved. Sorry for the noise.

adqm · 2026-10-12T20:53:10Z

The failing test was caused by having two customization labels in the docs (one for basic class customization and the other for run-time config in Doc/using/windows.rst). Having that extra label also broke some internal links in the docs on the tip of main, so for now I just removed that second reference (which didn't seem to be linked from anywhere else), though I suppose we could also consider changing the name of the "Basic Customization" label instead. I'm open to input there.

ZeroIntensity · 2026-10-12T21:35:24Z

We do our best to avoid removing Sphinx labels when possible (notice the failing CI job).

adqm · 2026-10-12T21:47:10Z

We do our best to avoid removing Sphinx labels when possible (notice the failing CI job).

I understand that, and I agree with the rationale of not breaking links, but I would argue that the addition of that new duplicate reference was a bug, since it is the thing breaking existing links (and it isn't referenced anywhere else in the docs as far as I can see), whereas the old customization label has been there for a while and is linked from elsewhere in the docs.

The new duplicate label looks like it was added in bd31583, per git blame, but that label breaks other links on main and this PR (for example, the link for "Configuration" in the expressions section of the language reference, which was working in the 3.15 docs and which goes back to at least version 3.4).

ZeroIntensity · 2026-10-12T22:55:10Z

Would you mind opening a separate issue and PR for that? Feel free to tag me for a review.

ZeroIntensity · 2026-10-13T01:29:31Z

+        """Index of symbols available."""
+
+        def bltinlink(name):
+            return '<a href="topic?key=%s">%s</a>' % (urllib.parse.quote(name), html.escape(name))


urllib.parse.quote does extra things that seemingly don't work with the .html URL handler (such as converting ! to %21). I think we can switch to html.escape(name) if all we're after is the quote escaping?

I was actually originally worried about +, since I know that + is often interpreted as a space in a URL. So I leaned toward urllib.parse.quote just to be safe since I'm not sure what else might be special in the URL bar, and because urllib.parse.quote just felt more right to me in principle, based on the context. I also expect the difference to be largely invisible to the end user, other than slightly-grosser URLs if copy/pasting.

I hadn't noticed any broken pages in my original testing, though I did just now test searching for ! and indeed it shows a bad error message (FWIW, the error is not because of the escaping but because ! doesn't have an entry).

The change I just pushed tries to remedy that by unquoting the given URL on the error page (I was already unquoting when actually doing the lookup, so, e.g., searching for != should have worked before this most recent change).

If you'd prefer, though, I can switch it to use html.escape instead.

I think it would show up if you had a module with some weird Unicode characters in the name. I suppose another fix would be to call urllib.parse.unquote in get_html_page, if you prefer that.

I think I already have the unquote in the only path in get_html_page where these links could possibly lead (behind the check for if '=' in url, this line in the diff).

I could be missing something, though; is there a case that line doesn't cover that I'm not seeing?

adqm added 19 commits August 19, 2025 10:25

first steps: don't show help output for dunders we've grabbed from bu…

b36972d

…iltins

better help for built-in help function

93ac0fa

add __main__ and DUNDERMETHODS to help

0c0f04c

try to associate all dunder methods with relevant pages from datamode…

255b603

…l.rst

show help for, e.g., __import__

da7f55c

bugfix for sequence-related dunders

3294f92

list 'specialnames' as a special topic

2aee131

add help for __name__

bff3522

change title of added reference

d1959d2

faster check for dunder name

1bc0227

suggest online help as well if we can't find something

4ed0e72

fix bad reference (SPECIALNAMES doesn't exist)

5800535

change help(help) output

587740b

clarify message when an entry is not found

69265aa

update test cases for new message

101fff3

revert changes to help(help); those should probably be a separate PR

35b4470

new test cases for help() about dunder methods

e196818

remove extra whitespace

d11fb77

__matmul__ shouldn't reference @ because of where help('@') currently…

2c34fba

… goes

adqm requested review from AA-Turner, hugovk and willingc as code owners August 19, 2025 17:25

bedevere-app Bot added the awaiting review label Aug 19, 2025

bedevere-app Bot mentioned this pull request Aug 19, 2025

Add more useful help for special name in help() #137574

Open

adqm added 3 commits August 19, 2025 13:52

fixes for missing dunder doctest

595b95c

attempt to fix failing test case on windows

0a49cfb

add news blurb

394e401

hugovk removed their request for review April 6, 2026 12:20

github-actions Bot added the stale Stale PR or inactive for long period of time. label May 15, 2026

ZeroIntensity reviewed Oct 11, 2026

View reviewed changes

Comment thread Lib/pydoc.py Outdated

Comment thread Lib/pydoc.py

Comment thread Lib/pydoc.py Outdated

Comment thread Misc/NEWS.d/next/Core_and_Builtins/2025-08-19-16-10-26.gh-issue-137966.nEgFAt.rst Outdated

Comment thread Doc/tools/extensions/pydoc_topics.py

Comment thread Lib/pydoc.py

adqm added 7 commits October 12, 2026 14:56

merge

376e3fb

fixes for checking whether a string should look up special dunder help

8c16306

add sphinx reference to news entry

f6c4f8c

fix typo in __set_name__ dunder

8991bb2

add class-customization to pydoc topics

7f92fbe

what's new entry

d419cad

add symbols to html help from pydoc

f3f2f4d

adqm requested a review from StanFromIreland as a code owner October 12, 2026 20:10

bedevere-app Bot added the type-feature A feature request or enhancement label Oct 12, 2026

StanFromIreland reviewed Oct 12, 2026

View reviewed changes

remove duplicate customization label from Doc/using/windows.rst

b826b60

Merge branch 'main' into pydoc_dunders

e1c7fab

ZeroIntensity reviewed Oct 13, 2026

View reviewed changes

adqm added 2 commits October 12, 2026 21:57

add more built-in modules to SKIP_DUNDER_HELP

58de00a

also unquote url in error page

f4b6f5c

ZeroIntensity removed the stale Stale PR or inactive for long period of time. label Oct 13, 2026

merge

cac3a02

Conversation

adqm commented Aug 19, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AA-Turner commented Aug 20, 2025

Uh oh!

ZeroIntensity commented Aug 20, 2025

Uh oh!

AA-Turner commented Aug 20, 2025

Uh oh!

adqm commented Oct 9, 2025

Uh oh!

github-actions Bot commented May 15, 2026

Uh oh!

adqm commented Oct 11, 2026

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

read-the-docs-community Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

adqm commented Oct 12, 2026

Uh oh!

adqm commented Oct 12, 2026

Uh oh!

adqm commented Oct 12, 2026

Uh oh!

ZeroIntensity commented Oct 12, 2026

Uh oh!

adqm commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZeroIntensity commented Oct 12, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

adqm commented Aug 19, 2025 •

edited by github-actions Bot

Loading

read-the-docs-community Bot commented Oct 12, 2026 •

edited

Loading

adqm commented Oct 12, 2026 •

edited

Loading